Paper deep dive
HugSelect: An Explainable Multi-Criteria Decision-Support Framework for foundation-model selection
Alireza Joonbakhsh, Arda Canser Adalı, Slinger Jansen, Farshad Khunjush, Siamak Farshidi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 8/11/2026, 5:07:28 AM
Summary
The paper introduces HugSelect, an explainable multi-criteria decision-support framework for selecting foundation models from repositories like Hugging Face. It addresses the limitations of current selection methods (popularity-based, keyword search, or opaque LLM recommendations) by building a knowledge base of 71,274 models using metadata, extracted functional capabilities, and community-perceived quality. HugSelect employs a Weighted Sum Model (WSM) to rank models based on weighted criteria, providing transparent, criterion-level score decompositions. Evaluation shows HugSelect achieves recommendation quality comparable to commercial LLM baselines while offering superior traceability and explainability.
Entities (6)
Relation Signals (5)
Hugging Face → hosts → foundation models
confidence 95% · Platforms such as Hugging Face host a large and continuously changing collection of models
HugSelect → uses → Weighted Sum Model
confidence 95% · It ranks candidate models using a weighted additive model that exposes criterion-level score decompositions.
HugSelect → provides → Explainability
confidence 92% · HugSelect ... providing stable, traceable, and inspectable reasoning.
HugSelect → builds → Knowledge Base
confidence 90% · HugSelect builds a knowledge base of 71,274 models by combining repository metadata, extracted functional capabilities, and perceived quality attributes
HugSelect → comparesagainst → commercial LLM-based recommendation systems
confidence 88% · We evaluated HugSelect through ... comparative case studies against four commercial LLM-based recommendation systems
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Foundation models are increasingly reused as software components, making model selection a critical software-engineering decision. Current model hubs primarily support discovery through popularity metrics, often neglecting functional capabilities, operational constraints, and community-perceived quality. We argue that foundation-model selection should be treated as an explicit, auditable software-component selection task rather than as keyword search, popularity ranking, or opaque conversational advice. This paper proposes HugSelect, an explainable decision-support framework for foundation-model selection. HugSelect builds a knowledge base of 71,274 models by combining repository metadata, extracted functional capabilities, and perceived quality attributes derived from community discussions into a unified pipeline. It ranks candidate models using a weighted additive model that exposes criterion-level score decompositions. We evaluated HugSelect through pipeline validation, comparative case studies against four commercial LLM-based recommendation systems (44 scenarios), fine-grained ablation, and an exploratory user study (n = 10). Extraction pipelines achieved an F1 score of 0.801 for functional features and an accuracy of 0.84 for quality-attribute mapping. HugSelect achieved a model-level Coverage@10 of 0.61 and family-level Coverage@10 of 0.91, showing recommendation quality comparable to that of the evaluated commercial systems, with no significant overall differences in ranking quality, while providing stable, traceable, and inspectable reasoning. Ablation confirmed that functional features were the main driver of retrieval accuracy, and preliminary user feedback suggests that the framework is useful and intuitive.
Tags
Links
- Source: https://arxiv.org/abs/2608.08069v1
- Canonical: https://arxiv.org/abs/2608.08069v1
Trouble viewing inline? Open PDF directly →
Full Text
113,299 characters extracted from source content.
Expand or collapse full text
HugSelect: An Explainable Multi-Criteria Decision-Support Framework for foundation-model selection Alireza Joonbakhsh a,∗ , Arda Canser Adalı b , Slinger Jansen b , Farshad Khunjush a and Siamak Farshidi c,∗ a Shiraz University, Shiraz, Iran b Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands c Information Technology Group, Wageningen University & Research, Wageningen, The Netherlands A R T I C L E I N F O Keywords: foundation-models decision model multi-criteria decision-making HuggingFace ecosystem AI models A B S T R A C T Foundation models are increasingly reused as software components, making model selection a critical software-engineering decision. Current model hubs primarily support discovery through popularity metrics, often neglecting functional capabilities, operational constraints, and community-perceived quality. We argue that foundation-model selection should be treated as an explicit, auditable software-component selection task rather than as keyword search, popularity ranking, or opaque conversational advice. This paper proposes HugSelect, an explainable decision-support framework for foundation- model selection. HugSelect builds a knowledge base of 71,274 models by combining repository metadata, extracted functional capabilities, and perceived quality attributes derived from com- munity discussions into a unified pipeline. It ranks candidate models using a weighted additive model that exposes criterion-level score decompositions. We evaluated HugSelect through pipeline validation, comparative case studies against four commercial LLM-based recommendation systems (44 scenarios), fine-grained ablation, and an exploratory user study (푛 = 10). Extraction pipelines achieved an F1-score of 0.801 for functional features and 0.84 accuracy for quality-attribute mapping. HugSelect achieved a model- level Coverage@10 of 0.61 and family-level Coverage@10 of 0.91, showing recommendation quality comparable to the evaluated commercial systems without significant overall differences in ranking quality, while providing stable, traceable, and inspectable reasoning. Ablation confirmed functional features as the main driver of retrieval accuracy, and preliminary user feedback suggests the framework is useful and intuitive. 1. Introduction Motivation and problem context foundation-models have become reusable building blocks for AI-enabled software systems. They are no longer only research artifacts: practitioners increasingly integrate them into applications that perform text generation, question answering, visual understanding, code assistance, recommendation, and decision support Bommasani (2021); Cazzaniga, Jaumotte, Li, Melina, Panton, Pizzinelli, Rockall and Tavares (2024). In this setting, choosing a foundation- model resembles selecting a complex third-party software component. The decision influences whether the resulting system satisfies functional requirements, but it also affects non-functional properties such as performance efficiency, reliability, maintainability, security, deployment effort, licensing constraints, and operational cost. The selection problem has become more difficult because model repositories have grown rapidly. Platforms such as Hugging Face host a large and continuously changing collection of models contributed by researchers, companies, and community members. This openness accelerates innovation and reuse, but it also creates a practical decision problem for software engineers who must identify models that fit a concrete application context Faulconbridge, Sarwar and Spring (2023). A practitioner may need to compare different model families, fine-tuned variants, quantized versions, licenses, hardware requirements, and quality-related trade-offs before making an informed choice. ∗ Corresponding authors: Alireza Joonbakhsh and Siamak Farshidi. alireza.joonbakhsh@hafez.shirazu.ac.ir (A. Joonbakhsh); a.c.adali@students.u.nl (A.C. Adalı); slinger.jansen@u.nl (S. Jansen); khunjush@shirazu.ac.ir (F. Khunjush); siamak.farshidi@wur.nl (S. Farshidi) ORCID(s): 0000-0001-5044-5196 (A. Joonbakhsh); 0000-0003-3752-2868 (S. Jansen); 0000-0002-3339-6051 (F. Khunjush); 0000-0001-6139-921X (S. Farshidi) Joonbakhsh et al.: Preprint submitted to ElsevierPage 1 of 29 arXiv:2608.08069v1 [cs.SE] 8 Aug 2026 HugSelect: Explainable foundation-model selection Relevant evidence is distributed across heterogeneous and unevenly maintained sources. Repository metadata may describe task labels, licenses, model size, and download counts; model cards and README files may describe intended use, datasets, limitations, and capabilities; community discussions may report practical experiences with speed, stability, integration difficulty, or unexpected behavior. These sources differ in completeness, terminology, and reliability. Consequently, foundation-model selection is a cognitively demanding software-engineering decision that requires more than simple keyword search or popularity-based filtering. We argue that foundation-model selection must be treated as an explicit and auditable software-component selection decision, supported by systematic evidence synthesis and transparent multi-criteria decision models. Limitations of existing approaches Existing approaches to foundation-model discovery and selection can be broadly categorized into three groups: popularity and metadata-based filtering, keyword-based retrieval methods, and large language model (LLM)-based recommendation systems. Popularity and metadata-based approaches rely on indicators such as download counts, likes, and manually defined tags to recommend models. While these signals facilitate basic discovery, they provide only surface-level representations of model characteristics and rarely capture task-specific requirements. For example, knowledge graph- based approaches that leverage Hugging Face metadata improve the structural relationships between models and tasks, yet remain constrained by incomplete and inconsistently defined metadata Chen, Huang, Zhou, Luo, Cui and Cheng (2025). Keyword-based retrieval methods attempt to match user queries with textual descriptions of models. These approaches partially address the challenge of heterogeneous information sources by incorporating unstructured text such as documentation or scientific literature. However, they rely primarily on lexical matching and therefore lack semantic understanding of relationships between features and task requirements. As a result, they struggle to differentiate between models with similar descriptions but different functional capabilities Di Sipio, Rubei, Di Rocco, Di Ruscio and Nguyen (2024). More recently, LLM-based recommendation systems have been explored for model selection tasks. These approaches can interpret complex user queries and reason about multiple criteria simultaneously, enabling more context-aware recommendations. Nevertheless, their reasoning processes remain largely opaque due to the black- box nature of LLM architectures. Additionally, they are susceptible to hallucinations and may produce inaccurate or nonexistent recommendations, thereby limiting their reliability in decision-support scenarios Zhao, Fan, Li, Liu, Mei, Wang, Wen, Wang, Zhao, Tang et al. (2024). Research gap Despite the growing need for effective foundation-model selection mechanisms, existing approaches address only isolated aspects of the decision problem. Popularity and metadata-based filters are scalable but shallow; keyword- based retrieval can surface relevant model-card text but does not explicitly model trade-offs; and conversational LLM recommenders can produce fluent advice but do not provide stable, auditable, criterion-level ranking logic. Current approaches therefore provide limited support when practitioners must compare many alternatives under context- specific functional and quality constraints. Data-driven decision frameworks have been successfully applied to other technology-selection domains, including software packages, AI models, blockchain platforms, and blockchain oracle selection Farshidi, Jansen, de Jong and Brinkkemper (2018); Farshidi, Jansen, España and Verkleij (2020); Farshidi, Jansen and Deldar (2021); Ahmadjee, Mera-Gómez, Farshidi, Bahsoon and Kazman (2025); Farshidi, Saberhabibi, Eskafi, Nikfarjam, Eskandari, Jansen, Chaudron and Tekinerdogan (2025); Joonbakhsh, Rostami, Kamalinia, Nazeri, Khunjush, Tekinerdogan and Farshidi (2025). Recent MCDM studies also confirm the value of structured decision support for AI-related choices, including ML reproducibility assessment Leščinskait ̇ e, Paulavičius and Filatovas (2026), healthcare deep learning model evaluation Drissi, El-Kassabi and Serhani (2024), trustworthy AI application assessment Alsalem, Alamoodi, Albahri, Albahri, Martínez, Yera, Duhaim and Sharaf (2024), MLaaS cloud selection Bhol, Mohanty and Pattnaik (2024), AI decision support Olabanjo and Honenberger (2026), and fuzzy software selection Seker and Kahraman (2021). However, these studies usually evaluate small, fixed sets of alternatives using predefined expert criteria. They do not address repository-scale foundation-model ecosystems where criteria and evidence must be derived automatically from incomplete metadata, model descriptions, and community feedback. Joonbakhsh et al.: Preprint submitted to ElsevierPage 2 of 29 HugSelect: Explainable foundation-model selection This creates a gap for explainable, repository-scale decision support that treats foundation-model selection as an auditable software-component selection decision involving functional requirements, non-functional quality concerns, and operational constraints. Such support should not only recommend models, but also expose why a model is recommended, which criteria contribute to its score, and which trade-offs remain for the practitioner to inspect. Research Questions This work addresses the following research questions: • RQ1: How accurately can functional features and perceived quality attributes be extracted from unstructured repository content and community discussions at scale? • RQ2: How does an explainable multi-criteria decision framework compare with practical baseline selection methods in terms of model-level and family-level recommendation quality? • RQ3: How do practitioners perceive the usability, transparency, and usefulness of HugSelect’s explainable decision-support framework? Solution overview (HugSelect) To address the identified research gap, this study proposes HugSelect, a data-driven decision-support framework for foundation-model selection based on Multi-Criteria Decision Making (MCDM) Triantaphyllou (2000). HugSelect operationalizes foundation-model selection as a single coherent decision-support method that integrates automated evidence extraction with explicit multi-criteria ranking. Each candidate model is assessed against functional require- ments, operational constraints, and community-perceived quality attributes using a weighted decision model, making the selection rationale inspectable and auditable. HugSelect integrates heterogeneous sources of model-related information into a unified knowledge base. Auto- mated data pipelines process repository metadata, model-card descriptions, and community feedback using feature extraction, sentiment analysis, and quality-attribute mapping techniques. The resulting knowledge base represents model alternatives together with metadata features, functional capabilities, perceived quality signals, and traceability links to supporting evidence. Users interact with the system through a natural-language interface. An LLM-based intent extractor translates a user query into structured requirements and candidate criteria. These criteria are mapped to the knowledge base and evaluated using a Weighted Sum Model (WSM / SAW). Rather than relying on a mathematical innovation in decision theory, the framework adapts WSM/SAW specifically to maximize ranking transparency and scalability. Instead of returning only a natural-language answer, HugSelect produces a ranked list of candidate models and decomposes each recommendation into criterion-level score contributions, enabling users to inspect trade-offs and adjust priorities. Compared with metadata-based filtering, HugSelect provides richer representations of model capabilities. Com- pared with keyword-based retrieval, it explicitly models trade-offs among criteria. Compared with LLM-only recom- mendation, it provides more transparent and controllable ranking logic. To support open science and reproducibility, the complete source code and accompanying datasets are publicly available Adalı, Joonbakhsh and Farshidi (2026). Contributions This study makes the following contributions: • A repository-scale decision-support method that formulates foundation-model selection as an explicit and auditable software-component selection decision. • An integrated evidence-synthesis pipeline that automatically harvests repository metadata, model-card de- scriptions, functional capabilities, and community-derived perceived quality signals into a unified knowledge base of 71,274 Hugging Face models. • An explainable WSM/SAW-based decision engine that maps user requirements to structured criteria and decomposes recommendation scores into inspectable criterion-level contributions. • An empirical evaluation comprising extraction pipeline validation, comparative retrieval analysis against commercial LLM baselines over 44 literature-derived scenarios, fine-grained ablation experiments, and an exploratory TAM-based practitioner user study. Joonbakhsh et al.: Preprint submitted to ElsevierPage 3 of 29 HugSelect: Explainable foundation-model selection Evaluation Caveats and Scope To support proper interpretation of our findings, three methodological boundaries should be noted early: (i) the 44 selection scenarios utilize peer-reviewed literature model choices as proxy ground truth rather than single absolute benchmark targets; (i) commercial LLM baselines reflect zero-shot, time-bound conversational system outputs; and (i) the 10-participant user study provides exploratory usability and acceptance evidence based on the Technology Acceptance Model (TAM) rather than a comparative trial of decision quality. Paper organization Section 2 reviews related work. Section 3 presents the HugSelect framework and methodology. Section 4 describes the system implementation. Section 5 reports the evaluation. Section 6 discusses implications and limitations. Section 7 concludes. 2. Related Work foundation-model selection sits at the intersection of software engineering, machine learning, repository mining, recommender systems, and decision science. To make the positioning easier to follow, we group the literature around three questions: (i) what kinds of model repositories and alternatives have been studied, (i) what evidence is used to characterize models, and (i) how candidate models are compared and evaluated. Appendix A provides the full gap-analysis table used to derive this positioning. 2.1. Repository Scale and Scope foundation-model repositories have grown rapidly in recent years Bommasani (2021); Awais, Naseer, Khan, Anwer, Cholakkal, Shah, Yang and Khan (2025). Hugging Face has become a central platform for open model sharing and discovery, but its scale also makes manual selection difficult. Existing studies vary substantially in scope. Some surveys focus on general foundation-models or specific technical areas such as vision-language models, large language models, multimodal models, agents, or medical imaging Chen, Xu, Zheng, Chen, Tolba, Zhao, Yu and Feng (2024); Azad, Azad, Eskandari, Bozorgpour, Kazerouni, Rekik and Merhof (2023); Zhang, Ding, Shui, Ma, Zou, Deoras and Wang (2021); Ding, Jiang, Zheng, Xu and Liang (2024); Minaee, Mikolov, Nikzad, Chenaghlu, Socher, Amatriain and Gao (2024); Mienye, Jere, Obaido, Ogunruku, Esenogho and Modisane (2025); Zhou, Lu, Chen, Zhu, Xu, Xing and Harrer (2024). Other work focuses on software engineering use cases or on the role of pre-trained models in software development Faulconbridge et al. (2023); González, Franch, Lo and Martínez-Fernández (2025). Most selection-oriented studies evaluate relatively small sets of alternatives. Exceptions include large-scale repository-mining studies such as those by Cao, Shi, Yu, Wang, Wang, Yan and Chen (2021), Liu, Zhao, Peng, Yu, Wang and Sha (2023), and González et al. (2025). The most closely related large-scale Hugging Face work is the knowledge-graph-based study by Chen et al. (2025), which supports recommendation, classification, and tracing over a large Hugging Face graph. However, that work does not focus on multi-criteria trade-off reasoning or on quality evidence derived from community discussions. Di Sipio et al. (2024) and Suryani, Karmakar and Mathiak (2024) also study Hugging Face models, but their focus is categorization and cross-repository exploration rather than explainable model selection. 2.2. Evidence Sources and Model Characterization Model selection depends on more than task labels or popularity. Practitioners often need to understand functional capabilities, supported modalities, architecture families, license constraints, deployment requirements, and quality concerns. Prior work uses several evidence sources, including academic literature, GitHub repositories, model hubs, public datasets, and benchmarks Jain (2022); González et al. (2025); Di Sipio et al. (2024); Suryani et al. (2024). SearchSECO is particularly relevant as an example of ecosystem-scale indexing, where heterogeneous software artifacts are collected, represented, and searched across repository boundaries Jansen, Farshidi, Gousios, van der Storm, Visser and Bruntink (2020). Repository mining, app-store mining, and software-ecosystem indexing research shows that large-scale software artifacts and unstructured text from documentation, issue trackers, Q&A sites, and user reviews can be transformed into structured indicators such as provenance links, topics, quality attributes, and defect-related signals Martin, Sarro, Jia, Zhang and Harman (2017); Jansen et al. (2020). Existing model-selection approaches usually use only a subset of these signals. Metadata extraction is scalable but shallow; benchmark evidence is useful but expensive and incomplete across large repositories; and manual expert criteria provide structure but do not scale well to thousands of alternatives. HugSelect therefore integrates Joonbakhsh et al.: Preprint submitted to ElsevierPage 4 of 29 HugSelect: Explainable foundation-model selection Table 1 Summary of related-work gaps addressed by HugSelect. ThemeMain focus in prior workRemaining gap addressed by HugSelect Repository scale and scope Prior studies often analyze small sets of alternatives, specific model families, or repository metadata snapshots. Large- scale Hugging Face studies exist, but usually focus on classification, tracing, or metadata-based discovery. Large-scale selection support is still limited when alter- natives are numerous, heterogeneous, and continuously changing. HugSelect analyzes a curated set of 71,274 Hugging Face models with sufficient metadata and textual evidence. Evidence sourcesExisting approaches typically rely on popularity signals, tags, model cards, benchmark results, or manually defined criteria. Functional capabilities and perceived quality signals from community discussions are rarely integrated into a unified decision model. HugSelect combines metadata, model de- scriptions, and community feedback. Decision supportLLM-based and keyword-based systems can produce fluent recommendations, while MCDM studies provide transparent comparison logic in smaller decision settings. There is limited support for transparent multi-criteria trade- off reasoning over large foundation-model repositories. HugSelect uses WSM/SAW to produce auditable criterion- level rankings. EvaluationPrior work commonly evaluates classification, retrieval, benchmark performance, or user perception in isolation. Few studies jointly validate extraction quality, recommenda- tion quality, and perceived usefulness. HugSelect combines pipeline validation, comparative case studies, and a practi- tioner user study. three complementary evidence families: structured repository metadata, functional features extracted from model descriptions, and perceived perceived quality attributes derived from community feedback. This design follows earlier data-driven decision frameworks for software packages, AI models, and technology selection Farshidi et al. (2018, 2020, 2021, 2025); Joonbakhsh et al. (2025), but adapts them to the larger and more text-heavy setting of foundation- model repositories. 2.3. Decision-Making and Evaluation Strategies Several decision-making strategies have been used for model and software-artifact recommendation. AI/ML-based methods infer suitability from usage data or model characteristics Chen et al. (2025); Ding et al. (2024). Knowledge- based and expert-driven systems rely on manually encoded relationships. Data-driven recommender systems use interaction data, collaborative filtering, or ranking models Ricci, Rokach and Shapira (2010); Lü, Medo, Yeung, Zhang, Zhang and Zhou (2012); Adomavicius, Manouselis and Kwon (2010). MCDM methods, such as AHP, TOPSIS, PROMETHEE, WSM, and fuzzy variants, support explicit comparison across multiple criteria Triantaphyllou (2000); Vaidya and Kumar (2006); Behzadian, Otaghsara, Yazdani and Ignatius (2012); Chakrabortty, Abdel-Basset and Ali (2023); Radulescu and Radulescu (2025). Recent MCDM studies confirm the value of structured decision support for AI- and software-related choices, including ML reproducibility readiness, healthcare model evaluation, trustworthy AI assessment, MLaaS selection, blockchain oracle selection, AI decision support, and fuzzy software selection Leščinskait ̇ e et al. (2026); Drissi et al. (2024); Alsalem et al. (2024); Bhol et al. (2024); Ahmadjee et al. (2025); Olabanjo and Honenberger (2026); Seker and Kahraman (2021). However, these studies typically evaluate small, fixed sets of alternatives using predefined expert criteria. They do not address large-scale foundation-model repositories, where criteria and evidence must be derived automatically from incomplete metadata, model descriptions, and community discussions. 2.4. Positioning of HugSelect HugSelect differs from prior work in three ways. First, it targets repository-scale foundation-model selection rather than small fixed alternative sets. Second, it combines structured metadata with textual functional features and community-derived quality indicators. Third, it uses an explicit WSM/SAW-based MCDM model so that recommendations can be decomposed into criterion-level contributions. This choice prioritizes transparency, low elicitation burden, and efficient ranking over methodological complexity. More advanced MCDM methods can be integrated in future versions, but WSM/SAW is appropriate for the first scalable implementation because users can adjust priorities without pairwise comparisons, fuzzy membership functions, or complex preference calibration. The full comparison matrix in Appendix A shows that existing studies rarely combine large-scale automated data collection, multi-source feature extraction, community-feedback-based quality mapping, MCDM-based ranking, and practitioner evaluation in a single framework. HugSelect is designed to fill this gap. Joonbakhsh et al.: Preprint submitted to ElsevierPage 5 of 29 HugSelect: Explainable foundation-model selection 3. The HugSelect Framework (Approach) This section presents the conceptual and methodological foundations of HugSelect, grounded in Design Science Research (DSR) methodology Hevner, March, Park and Ram (2004). The goal is to clarify the problem formulation, architectural structure, data integration strategy, decision-making mechanism, and explainability design, independently of any particular implementation. 3.1. Research Methodology The development of HugSelect follows the DSR methodology, which focuses on solving real-world problems through the systematic design, development, and evaluation of artifacts Hevner et al. (2004). DSR is particularly well- suited to this work because it enables a practical, rigorously evaluated solution to a complex, real-world decision- making problem. The research process comprises the following phases: 1. Problem identification: Identifying gaps and limitations in existing foundation-model selection approaches, particularly the reliance on simple metadata signals and the lack of explainability. 2. Literature review: Conducting a systematic literature study using snowballing Wohlin (2014) to inform design decisions and establish novelty. The review covers related domains including LLM selection, machine learning model selection, and intelligent agent selection. 3. Framework design: Developing a theoretical framework encompassing data collection and integration pipelines, knowledge base construction, and decision model formulation. 4. Artifact construction: Implementing the HugSelect decision support system as a concrete instantiation of the framework. 5. Evaluation: Assessing the validity of feature extraction pipelines and the overall effectiveness of HugSelect through empirical validation, comparative case studies, and user studies. The literature study provides a stable basis for the design decisions of the adapted framework, supported by existing academic work, informing choices regarding information sources, feature extraction methods, knowledge base formation, and decision-making criteria and strategies. 3.2. Problem Formulation and Unit of Analysis Let 푀 = 푚 1 , ...,푚 푛 be the set of available model alternatives. Unit of Analysis. In HugSelect, the unit of analysis 푚 푖 ∈ 푀 is an individual repository artifact hosted on Hugging Face. The indexed collection of 71,274 models includes base foundation-models (e.g., meta-llama/Meta-Llama-3-8B), fine-tuned checkpoints (e.g., meta-llama/Meta-Llama-3-8B-Instruct), quantized variants (e.g., TheBloke/Llama-2-7B-GGUF), and adapter weights. Individual model entries retain their specific metadata and operational characteristics (such as parameter size, license, and format) while being mapped to their broader architectural model family in the knowledge base. Feature Families vs. Decision Criteria. Each model 푚 푖 is characterized by a feature vector 퐟 푖 = (푓 푖1 , ...,푓 푖푘 ) across 푘 individual decision criteria 푐 푗 . These criteria belong to three feature families: • Metadata features (퐹 푀 ): Structured repository attributes (e.g., task tag, license, parameter size, download count). • Functional features (퐹 퐹 ): Capability descriptors extracted from model cards and README documentation (e.g., reasoning capabilities, domain specificity, supported modalities). • Perceived quality attributes (퐹 푄 ): Non-functional signals derived from community feedback, structured according to ISO/IEC 25010 (e.g., perceived reliability, perceived performance efficiency). A user provides a selection context 푞 consisting of a natural-language description of application requirements and constraints. An intent extraction component transforms 푞 into structured criteria weights 퐰 = (푤 1 , ...,푤 푘 ) where 푤 푗 ≥ 0 and ∑ 푘 푗=1 푤 푗 = 1. Joonbakhsh et al.: Preprint submitted to ElsevierPage 6 of 29 HugSelect: Explainable foundation-model selection K B S o u r c e s Quality Standards Quality Mappings Contextual Information Code Repositories Developer Communities Gray & Academic Literature ... Domain Knowledge Software Documentation Domain Ontologies Empirical Evidence ... Feature Mappings D e c i s i o n M o d e l Software Quality ModelDomain AlternativesDomain Features Industry Standards Best Practices & Process Models Community-Driven Quality Models ... ISO/IEC 25010 IEEE Quality Standard ... Foundation Models AI Models Software Packages ... Language Support Functional Features ... I n f e r e n c e E n g i n e Case Definition Weighting Method Solution Figure 1: Adapted multi-layer framework architecture for foundation-model selection (adapted from Farshidi (2020)). The framework integrates heterogeneous data sources into a unified knowledge base and connects it to an MCDM-based inference engine. The framework computes a ranking score 푅(푚 푖 ,푞) for each candidate model using a Weighted Sum Model (WSM / SAW) Triantaphyllou (2000): 푅(푚 푖 ,푞) = 푘 ∑ 푗=1 푤 푗 ⋅ 푠 푖푗 (1) where 푠 푖푗 ∈ [0, 1] is the normalized score of model 푚 푖 on criterion 푐 푗 ∈ 퐹 푀 ∪퐹 퐹 ∪퐹 푄 . The decision goal is to generate a prioritized ordering of 푀 and expose the criterion-level contributions 푤 푗 ⋅ 푠 푖푗 for auditability. Running Example. To illustrate the framework operations throughout Sections 3 and 4, we trace a single running scenario: "A software engineering team requires an open-weights foundation-model to power a clinical Q&A assistant. Contextual constraints require: task = Text-Generation, modality = Text, license = Permissive (Apache-2.0 or MIT), deployment format = Quantized GGUF (< 10GB), with strong community-perceived performance efficiency and reliability." 3.3. Conceptual Architecture and Workflow The architectural design of HugSelect is adapted from the multi-criteria decision-making framework proposed by Farshidi Farshidi (2020). As illustrated in Figure 1, the adapted framework enables automated data collection, handling of heterogeneous information sources, systematic mapping of extracted features, and linking the knowledge base to an MCDM structure for decision-making. The framework comprises four interconnected layers: 1. Source Layer: Ingests heterogeneous evidence from model hubs (Hugging Face metadata and model cards) and developer discussion platforms (Reddit, Stack Overflow). 2. Knowledge Base Layer: Integrates extracted metadata, functional capability phrases, and community-perceived quality signals into a graph-structured schema, preserving traceability to original source snippets. 3. Decision Model Layer: Processes natural-language queries to map user intent into candidate criteria across the three feature families and assign initial criteria weights. Joonbakhsh et al.: Preprint submitted to ElsevierPage 7 of 29 HugSelect: Explainable foundation-model selection 4. Inference Engine Layer: Executes criterion score normalization, applies WSM ranking (Equation 1), and decomposes scores into explainable criterion-level contributions. The architecture is intentionally modular, allowing different extraction techniques, decision models, or data sources to be substituted without structural changes. This modularity supports extensibility and adaptation to evolving model ecosystems. 3.4. Conceptual Model Figure 2 presents the conceptual model of HugSelect, illustrating how the framework integrates metadata, functional features, and quality attributes into a unified knowledge base to enable model selection. The conceptual workflow proceeds as follows: 1. Data Collection: Three pipelines harvest repository metadata, model descriptions (from model cards and README files), and community feedback (user comments, reviews, and Q&A posts). 2. Feature Extraction: Unstructured text from model descriptions and community feedback is transformed into structured functional features and quality attributes through dedicated extraction pipelines. 3. Knowledge Base Integration: All extracted features are mapped to a unified schema that links models to their metadata, capabilities, quality dimensions, and supporting evidence, enabling traceability across heterogeneous sources. 4. Query Processing: User queries are analyzed to determine relevant criteria and weights, which are then used to instantiate the decision model. 5. Ranking and Explanation: The MCDM engine computes model rankings and generates explanations that decompose scores into criterion-level contributions. This conceptual model bridges the abstract architectural layers with the practical application of HugSelect, clarify- ing how heterogeneous information sources are systematically transformed into actionable selection recommendations. 3.5. Data Integration Methodology HugSelect synthesizes three complementary evidence sources to construct a holistic representation of each foundation-model: Repository Metadata (퐹 푀 ). Structured fields (task tags, framework, license, downloads, creation date) provide baseline operational boundaries. Missing or inconsistent tags are flagged and handled via schema mapping. In our running example, metadata filters immediately identify models offering GGUF files and permissive licensing. Model Card Descriptions (퐹 퐹 ). Unstructured README text and model cards provide narrative details on capabilities, training data, and intended use. Dependency parsing extracts technical capability phrases, mapping them to standardized functional descriptors. Community Feedback (퐹 푄 ). User comments, reviews, and Q&A posts related to specific models are collected from the hub and external platforms. After cleaning and model-level linking, they are processed by the quality attribute extraction pipeline to yield scores along selected quality dimensions. The integration methodology defines a unified schema that stores all three data types as attributes of model entities, mapping rules from raw text to standardized functional and quality dimensions, and consistency checks to ensure that extracted features are plausible given the underlying metadata. This multi-source integration strategy addresses the heterogeneity challenge inherent in foundation-model ecosystems, where relevant information is distributed across structured metadata, unstructured documentation, and community-generated content. 3.6. Multi-Criteria Decision Making The MCDM component operationalizes the problem formulation by implementing a systematic approach to criterion definition, normalization, weighting, and ranking Triantaphyllou (2000). Its responsibilities are: Joonbakhsh et al.: Preprint submitted to ElsevierPage 8 of 29 HugSelect: Explainable foundation-model selection Foundation Model Metadata Extraction Pipeline Quality Assessment Data Extraction Pipeline Review Collection 1 Sentiment Analysis 2 Quality Attribute Mapping 3 Fuzzy Aggregation 4 ISO/IEC 25010 Model/Feature Identification Pipeline Linguistic Filtering 2 Noun Phrase Extraction 1 Model Filtering 3 Metadata Extraction 2 Data Acquisition 1 Fuzzy Name Classification 3 Developer Reviews Foundation Models Quality Attributes Functional Features Foundation Models Foundation Models Model Descriptions Figure 2: Conceptual model of the HugSelect framework showing the data collection pipelines, feature extraction, and knowledge base construction. 1. Criterion Definition & Intent Mapping. For query 푞, the framework identifies relevant criteria 푐 푗 across 퐹 푀 ,퐹 퐹 ,퐹 푄 . In our running example, intent extraction identifies 5 criteria: 푐 1 = Task: Text-Generation (퐹 퐹 ), 푐 2 = License: Permissive (퐹 푀 ), 푐 3 = Format: GGUF (퐹 푀 ), 푐 4 = Perceived Efficiency (퐹 푄 ), and 푐 5 = Perceived Reliability (퐹 푄 ). 2. Normalization. Heterogeneous criterion values 푓 푖푗 are normalized to scores 푠 푖푗 ∈ [0, 1]. Categorical matches (e.g., license, task) assign binary scores푠 푖푗 ∈ 0, 1, continuous metrics (e.g., parameter size, downloads) use min-max scaling, and textual/quality attributes assign similarity or aggregated fuzzy quality values. 3. Weighting & WSM Ranking. Weights 푤 푗 are assigned based on user constraints (e.g., 푤 1 = 0.25,푤 2 = 0.25,푤 3 = 0.20,푤 4 = 0.15,푤 5 = 0.15). The engine evaluates Equation 1 for all candidate models. HugSelect uses weighted sum as the default because of its simplicity and interpretability, but the architecture permits alternative methods such as TOPSIS or PROMETHEE when needed. Joonbakhsh et al.: Preprint submitted to ElsevierPage 9 of 29 HugSelect: Explainable foundation-model selection 4. Sensitivity Analysis. The engine computes ranking variations under weight perturbations (±20%), informing users whether top recommendations are sensitive to specific preference trade-offs. 3.7. Explainability Design To ensure complete transparency, HugSelect generates explanations along three dimensions: • Score Decomposition: Exposes the exact breakdown of 푅(푚 푖 ,푞) into criterion-level contributions 푤 푗 ⋅ 푠 푖푗 , allowing users to see why a candidate model ranks highest. • Feature-Level Traceability: Links scores back to source evidence (e.g., highlighting exact model card sentences or community review snippets). • Comparative Trade-off Views: Renders side-by-side criteria comparisons across top candidates, illustrating trade-offs between model variants. 4. System Implementation This section describes the technical implementation of the HugSelect framework, detailing its architecture, data processing pipelines, multi-criteria decision-making engine, and user interface components. 4.1. Implementation Architecture The HugSelect framework is implemented as a four-layer system designed to automate foundation-model selection from the Hugging Face Hub. The architecture comprises: 1. Automated Data Collection Layer: Interfaces with the Hugging Face, Reddit, and Stack Exchange APIs to gather model metadata, community discussions, and developer feedback. 2. Knowledge Base Layer: Stores structured representations of models, their functional features, quality attributes, and relationships in a graph-based format. 3. LLM-based Intent Detection Layer: Processes natural language queries to extract user requirements and map them to structured decision criteria. 4. MCDM Engine Layer: Implements multi-criteria ranking algorithms to generate ranked model recommenda- tions with transparent rationale. 4.2. Data Collection and Processing Data Sources and Collection Pipeline The data collection pipeline interfaces with three primary APIs. The Hugging Face API Hugging Face (2023) retrieves model metadata, including name, task type, architecture family, download counts, license information, and publication dates, resulting in an initial collection of approximately 71,274 models. The Reddit API Reddit Inc. (2023) collects community discussions, user experiences, and informal reviews from machine learning subreddits. The StackExchange API Stack Exchange Inc. (2023) collects technical questions, answers, and developer feedback on specific models. Data quality filtering removes duplicates, deprecated models, and entries with incomplete metadata to ensure knowledge base integrity. The complete open-source implementation of these data collection and processing pipelines along with the initial raw data and the outputs of our functional and quality-attribute processing pipelines are systematically archived in our Mendeley repository Adalı et al. (2026). Functional Feature Extraction Pipeline Functional features describe the technical capabilities and intended use cases of foundation-models. The extraction pipeline employs noun-phrase filtering with spaCy’s dependency parser Honnibal and Montani (2017) to identify technical noun phrases in model descriptions and documentation. Feature clustering then groups extracted features into three primary dimensions: modality (categorizing models by input/output types such as text, image, audio, multimodal, with 11 distinct labels in the knowledge base), task (classifying models by problem type such as text generation, classification, translation, with 52 task categories in the dataset), and family (grouping models sharing architectural lineage or design principles such as LLaMA, BERT, Stable Diffusion, with 87 model families tracked). Because developer documentation is often incomplete, the system employs simple information-extraction methods to classify model features based on available descriptions. Models Joonbakhsh et al.: Preprint submitted to ElsevierPage 10 of 29 HugSelect: Explainable foundation-model selection Category#Items ItemCount Modality11 Text46,489 Multimodal14,470 Image4,356 Audio3,438 Reinforcement Learning473 Other / Unclear1,735 Task52 Text-Generation35,556 Text-to-Image11,579 Text-Classification3,788 Automatic-Speech-Recognition2,241 Image-Classification1,862 Other / Unclear1,735 Family Root 87 LLaMA14,789 BERT8,237 Stable Diffusion7,819 Mistral7,194 Qwen4,697 Other / Unclear8,259 Total Models71,274 Table 2 Top 5 entries per category. In total, the dataset contains 11 modality labels, 52 task categories, and 87 model families. Only the most frequent entries are shown for readability, with remaining entries grouped as Other / Unclear where applicable. that cannot be definitively assigned to clusters are retained in the knowledge base but receive lower recommendation priority. Table 2 summarizes the distribution of models across the modality, task, and family dimensions, showing the top 5 entries in each category. 4.2.1. Quality Attribute Extraction Pipeline Quality attributes in HugSelect represent community-perceived non-functional characteristics of foundation- models, based on feedback collected from Hugging Face discussions, Reddit, and Stack Overflow. Because queries based only on model names can introduce entity ambiguity and unrelated discussions, the pipeline applies several filtering stages before quality mapping. First, only review snippets that explicitly mention the model are retained. Second, an LLM-based relevance check removes cases in which the matched name refers to a different entity rather than to a foundation-model. Third, code-heavy or structurally noisy snippets are removed during preprocessing. The remaining reviews are analyzed with three multilingual transformer-based sentiment models, and neutral reviews are discarded because they do not provide clear evidence about perceived model quality. The filtered review snippets are then mapped to ISO/IEC 25010-inspired quality characteristics, including functional suitability, performance efficiency, reliability, interaction capability, maintainability, security, flexibility, and safety. Mapping is performed through LLM prompting that interprets semantic cues in the review text and assigns the most relevant quality dimension. For example, statements about slow inference or high resource consumption correspond to performance efficiency, comments about crashes or inconsistent outputs indicate reliability, and remarks about documentation or integration difficulty relate to maintainability or interaction capability. Examples of such mappings are shown in Table 3. Quality characteristics are defined in accordance with ISO/IEC 25010, while community reviews provide ob- servable indicators of perceived quality rather than controlled measurements of runtime behavior. In this study, the mapping from review statements to quality attributes is treated as a coding task that operationalizes these constructs. Review snippets serve as qualitative evidence of user-perceived system behavior and can indicate properties such as performance efficiency, reliability, or maintainability. The LLM is therefore used as a coding instrument that assigns review statements to predefined quality categories based on their semantic content. To reduce subjectivity when translating informal feedback into formal quality attributes, the pipeline requires that each review snippet be categorized along with its sentiment and a brief rationale for the classification. Rather than relying on a single observation, quality signals are aggregated across multiple independent reviews. To ensure robustness, a quality characteristic is evaluated for a model only when supported by at least three evidence-bearing (positive or negative) review instances (퐿 + 퐻≥ 3). Neutral reviews are discarded during preprocessing as they do not provide explicit directional evidence regarding perceived quality. This sufficiency threshold reduces the influence of isolated or ambiguous comments. Joonbakhsh et al.: Preprint submitted to ElsevierPage 11 of 29 HugSelect: Explainable foundation-model selection Table 3 Examples of mapping review snippets to ISO/IEC 25010-inspired quality characteristics. Review SnippetQuality AttributeSentimentRationale “DeepSeek-R1’s innovative text generation and reasoning features would be a perfect fit for our project.” Functional SuitabilityPositiveIndicates that the model effectively fulfills the intended task requirements. “The storytelling capabilities of GPT-2 are in- credible.” Functional SuitabilityPositiveHighlights strong task performance in text generation. “I tried 10 to 15 different settings, and it still didn’t work.” ReliabilityNegativeDescribes repeated failure of the model to perform correctly. “The model cannot be found even though it appears in the Hugging Face model list.” ReliabilityNegativeIndicates issues with availability or stabil- ity during use. “I loaded my SentenceTransformer model locally and successfully embedded sample sentences.” Functional SuitabilityPositiveConfirms correct operation of the model’s primary embedding function. Fuzzy Aggregation Scoring. For eligible models (퐿 +퐻≥ 3), the review evidence is aggregated into a normalized perceived quality score 푆 푓푢푧푦 (푚 푖 ,푄 푘 ) ∈ [0, 1] computed as the ratio of positive reviews over all directional evidence: 푆 푓푢푧푦 (푚 푖 ,푄 푘 ) = 퐻 퐿 + 퐻 (2) where 퐻 represents the count of positive review snippets and 퐿 represents the count of negative review snippets mapped to quality characteristic 푄 푘 . If a model has insufficient community evidence (퐿+퐻 < 3), no score is assigned for that attribute due to data sparsity, ensuring newly published or sparsely reviewed models are not unfairly evaluated on unrepresentative feedback. 4.3. Knowledge Base Structure The knowledge base represents foundation-models and their attributes as a graph-structured integration layer, enabling flexible querying, unified access to heterogeneous evidence, and traceability between models, features, quality attributes, and their source data. In this work, the knowledge graph is used primarily as an integration and traceability layer rather than as an autonomous graph-reasoning engine. Its role is to unify heterogeneous evidence from metadata, model descriptions, and community feedback, while the ranking logic is implemented in the MCDM decision engine. Figure 3 illustrates an example knowledge graph comparing three variations of the DeepSeek-R1 model DeepSeek AI (2025): the base model deepseek-ai/DeepSeek-R1, a quantized variant unsloth/DeepSeek-R1-GGUF, and an activation-aware quantized variant cognitivecomputations/DeepSeek-R1-AWQ. The graph captures three types of relationships. Metadata relations link models to their technical specifications, although these are not depicted in the simplified visualization. Functional features establish many-to-many relationships between models and extracted capabilities derived from NLP analysis of descriptions. Quality attributes connect models to community-derived indicators inferred from reviews and discussions. This graph-based representation supports semantic matching by organizing model variants, functional features, and quality attributes in a common structure. The matching and ranking are performed by the decision engine, while the knowledge base provides the integrated evidence needed to compare model variations and near-substitutes when they satisfy the same user intent. This distinction improves the validity of evaluation by separating semantic correctness from exact model-ID matching. The comprehensive set of model clusters, functional features, and quality attributes mapped within this knowledge base is available for review in our data repository Adalı et al. (2026). 4.4. MCDM Engine Implementation The MCDM engine is implemented as a modular Python library that orchestrates the decision-making process from query processing to ranked output generation. 4.4.1. User Intent Extraction Figure 4 illustrates the inference architecture. When users submit natural-language queries describing their intended task, application context, and operational constraints, the system processes them using two parallel extractors. The keyword feature extractor identifies explicit technical terms and requirements using pattern matching and domain- specific lexicons. The LLM-based targeted feature extractor uses LLaMA Touvron, Lavril, Izacard, Martinet, Lachaux, Joonbakhsh et al.: Preprint submitted to ElsevierPage 12 of 29 HugSelect: Explainable foundation-model selection DeepSeek-R1 base model DeepSeek-R1-GGUF quantized variant DeepSeek-R1-AWQ AWQ variant reasoningtext generation API compatibility quantization resource efficiency functional suitability performance efficiency reliability variant of variant of Figure 3: Simplified knowledge-graph example for three DeepSeek-R1 variants. Rectangles denote model entities and functional features; diamonds denote perceived quality attributes derived from community feedback. The full knowledge base contains many more models, features, and evidence links, but this simplified view highlights the relation types used for matching and explanation. Knowledge Base Inference Engine Keyword Features Foundation Models Feature Matching User Input Knowledge Graph Foundation Model Quality Attributes sources 0..* 0..* 1 1 sources 1..* 1..* describes Metadata Features Functional Features Targeted Features Decision Making 1..*0..* evaluates 1..* 1..* Figure 4: Inference engine of the HugSelect framework. User queries are converted into structured features and matched against a knowledge graph of foundation-models to generate ranked model recommendations. Lacroix, Rozière, Goyal, Hambro, Azhar et al. (2023) and Gemini Google DeepMind (2023) variants to interpret implicit requirements and contextual constraints. This component addresses potential biases by using multiple models and aggregating their outputs. Joonbakhsh et al.: Preprint submitted to ElsevierPage 13 of 29 HugSelect: Explainable foundation-model selection The extracted features are mapped to the structured feature space of the knowledge base, creating a user requirement vector aligned with model attributes. This design is related to earlier context-aware search systems in computational environments, such as CANSF, which derives search queries from a user’s active Jupyter notebook context and retrieves semantically related notebooks Li, Farshidi, Bianchi, Koulouzis and Zhao (2022). 4.4.2. Weighted Criteria Vector and MoSCoW Prioritization User requirements are transformed into a weighted criteria vector following the MoSCoW prioritization method Clegg and Barker (1994): • Must-have (M): Critical requirements treated as hard constraints (restriction properties). • Should-have (S): Important preferences with high weights. • Could-have (C): Desirable features with moderate weights. • Won’t-have (W): Explicitly excluded features. The LLM-based extractor assigns importance weights to each identified feature based on linguistic cues in the user query (e.g., "must support," "preferably," "ideally"). 4.4.3. Candidate Retrieval and Matching The inference engine compares the user requirement vector against model feature vectors in the knowledge base. The matching process incorporates: 1. Degree of Match: Cosine similarity between user requirements and model features for functional attributes; threshold-based matching for categorical metadata. 2. Feature Importance: Weights from the MoSCoW prioritization amplify the contribution of critical features to the overall score. 3. Popularity Metrics: Download counts and community engagement serve as tie-breakers when multiple models have similar functional matches. 4.4.4. Normalization and Ranking Retrieved candidates undergo normalization to ensure comparability across heterogeneous criteria: • Min-Max Normalization: Applied to quantitative metrics (downloads, model size) to scale values to [0, 1]. • Binary Normalization: Applied to categorical features (license type, modality) using exact- or partial-match scoring. The final suitability score for each model 푚 is computed as: Score(푚) = 푛 ∑ 푖=1 푤 푖 ⋅ match 푖 (푚,푢) where 푤 푖 is the importance weight of criterion 푖, match 푖 (푚,푢) is the normalized match score between model 푚 and user requirement 푢 on criterion 푖, and 푛 is the total number of criteria. Models are ranked in descending order of their suitability scores, producing a prioritized candidate list. 4.4.5. Explainable Output Generation The engine generates transparent rationale for each recommendation through three mechanisms. Score decomposi- tion breaks down the total score into contributions from metadata, functional features, and quality attributes, enabling users to understand the relative importance of each dimension. Feature highlighting identifies which user requirements were satisfied by each recommended model and which were not, providing clear justification for ranking decisions. Knowledge graph visualization presents contextual information about model relationships and feature overlaps in an interactive graph format, allowing users to explore alternative models and understand trade-offs. This explainability layer supports informed decision-making and builds user trust in the recommendations. Joonbakhsh et al.: Preprint submitted to ElsevierPage 14 of 29 HugSelect: Explainable foundation-model selection Table 4 Illustrative end-to-end workflow in HugSelect. StepExample operation User requestA practitioner asks for a foundation-model for reasoning-heavy question answering, with a preference for text generation, efficient deployment, permissive licensing, and reliable community feedback. Requirement extractionThe intent extractor identifies task-related criteria (reasoning, question answering, text generation), operational preferences (efficient deployment, model size or quantization), and restriction properties (license compatibility). Criteria weightingCritical requirements are treated as must-have constraints, while preferred capabilities and perceived quality attributes receive lower but explicit weights using the MoSCoW-inspired weighting scheme. Candidate retrievalThe knowledge base retrieves models whose metadata and functional features match the task and modality requirements. Related variants, such as base, quantized, or fine-tuned versions, remain comparable through family- level mappings. RankingThe MCDM engine normalizes heterogeneous criteria and computes a weighted suitability score for each candidate model. ExplanationThe output decomposes the score into functional, metadata, and perceived quality contributions and highlights which requirements are satisfied, partially satisfied, or unsupported by available evidence. Decision supportThe practitioner can inspect trade-offs, adjust weights, compare related variants, and use the ranking as decision support rather than as an automatic final decision. 4.5. End-to-End Selection Workflow To clarify how the implementation operationalizes the framework, Table 4 presents an illustrative end-to-end workflow. The example shows how a natural-language request is transformed into structured criteria, how candidate models are retrieved from the knowledge base, and how the final ranking is explained. The example is not intended as a new benchmark case; rather, it demonstrates the decision-support logic implemented by HugSelect. This workflow illustrates the intended use of HugSelect as an auditable selection aid. The system does not claim that the highest-ranked model is universally optimal; instead, it makes the selection rationale explicit so that practitioners can examine whether the ranking aligns with their project constraints and risk tolerance. 5. Evaluation This section presents an empirical evaluation of the HugSelect framework through three complementary ap- proaches: pipeline validation, comparison with commercial LLM-based baselines, and a user study. The evaluation addresses the three research questions outlined in Section 1 and follows Design Science Research (DSR) principles for artifact validation. To ensure full transparency and enable independent verification, all data used and generated throughout these evaluation phases are publicly available in our replication package Adalı et al. (2026). The repository is organized to reflect our methodology, containing dedicated records for the pipeline validation (functional features and quality attributes), the detailed case study execution (including curation papers, recommendation results, and ablation studies), and the complete empirical results from the user study. 5.1. Evaluation Overview The evaluation strategy comprises three phases: 1. Pipeline Validation (RQ1): Assesses the accuracy and reliability of the functional feature extraction pipeline (P2) and the quality attribute extraction pipeline (P3) through ground-truth comparison and expert validation. 2. Framework Validation via Case Studies (RQ2): Evaluates HugSelect’s recommendation performance against four commercial LLM-based baselines (ChatGPT, Claude, Gemini, Perplexity) across 44 real-world model- selection scenarios, using both model-level and family-level metrics. 3. User Study (RQ3): Examines user perceptions of recommendation quality, transparency, and usability through subjective assessments and task performance measurements. 5.2. Pipeline Validation (RQ1) To assess the reliability of the automated extraction pipelines, we validated both functional feature extraction (P2) and quality attribute extraction (P3) against reference datasets constructed through multi-LLM consensus and human annotation. Validation used a stratified random sample of 500 models from the 71,274-model knowledge base (250 from the top 10,000 by popularity, 250 from the remainder) and 500 community reviews. Metrics included precision, recall, Joonbakhsh et al.: Preprint submitted to ElsevierPage 15 of 29 HugSelect: Explainable foundation-model selection SourcePrecision RecallF1IoU Gemini0.8020.778 0.790 0.653 LLaMA0.8340.582 0.686 0.522 ChatGPT0.6920.677 0.684 0.520 Ground truth0.7460.865 0.801 0.668 Table 5 Functional feature extraction evaluation results. Category (Count)Type Example Training metadata (11) FPfine tuned model FPbase language model FPlarge language model Infrastructure details (13) FPCUDA FPgoogle colab FPnormalization Dataset references (6) FPconll2003 dataset FPrussian text FPitalian Deployment formats (14) FNGGUF FNQuantized Model FNWeight-only quantization Optimization techniques (18) FNQLoRA FNlr_scheduler_warmup_ratio FNpositional encoding method Task descriptions (16) FNlatent text-to-image diffusion model FNSQL statement FNfuture n-gram prediction Table 6 Representative false-positive (FP) and false-negative (FN) extraction errors. F1-score, and Intersection over Union (IoU) for feature extraction, and accuracy with macro- and weighted-precision, recall, and F1 for sentiment and quality-attribute classification. 5.2.1. Functional Feature Extraction (P2) To construct a reference dataset, three LLMs (Gemini, GPT, LLaMA) independently extracted functional features from model descriptions. Features that appeared in at least two LLM outputs were retained and validated by a human expert, thereby reducing single-model bias. To reduce circularity in evaluating the functional feature extraction pipeline, we constructed the reference ground truth through a human-in-the-loop validation process rather than relying solely on LLM-generated outputs. For the sampled models, three LLMs independently extracted candidate functional features from the model descriptions, while a domain expert manually annotated functional features using the same criteria. A feature was included in the ground- truth set only when it was identified by the human annotator and confirmed by at least one LLM, ensuring that retained features were both human-validated and reproducible from the source text. This protocol uses the domain expert as the primary filter for factual and contextual relevance, while the LLMs provide an additional consistency check across independent semantic interpretations. Therefore, the resulting ground truth supports evaluation of HugSelect’s extraction quality without depending exclusively on automated model agreement. Table 5 presents validation results against each individual LLM reference set and the unified ground truth. The pipeline achieved strong recall (0.865) and F1-score (0.801) against the unified reference, with IoU of 0.668, indicating reliable capture of model capabilities. Gemini showed the highest agreement with the ground truth. Manual error analysis of 100 mismatched features revealed that false positives primarily stemmed from training metadata (11 cases), infrastructure details (13 cases), and dataset references (6 cases), while false negatives arose from specialized deployment terminology (14 cases), optimization techniques (18 cases), and task-specific architectural descriptions (16 cases). Table 6 provides representative examples. These errors reflect the heterogeneity of developer-written model cards, which range from marketing-focused narratives to benchmark-heavy technical reports. Filtering contextual information that does not directly describe functional capabilities remains a challenge for automated extraction. Joonbakhsh et al.: Preprint submitted to ElsevierPage 16 of 29 HugSelect: Explainable foundation-model selection TaskSourceAccuracyMacroWeighted PrecisionRecallF1PrecisionRecallF1 Sentiment ChatGPT0.650.640.610.590.730.650.66 Gemini0.630.630.600.570.720.630.65 Ground Truth0.700.660.700.630.790.700.72 Quality Mapping ChatGPT0.770.590.520.530.790.770.76 Gemini0.720.560.490.490.800.720.73 Ground Truth0.840.680.630.650.840.840.83 Table 7 Sentiment analysis and quality-attribute mapping performance. Table 8 Sentiment and quality attribute label distribution. TaskCategorySampleFull Dataset Count (%) Reviews(%) Sentiment Analysis Positive135 27.01786734.8 Neutral255 51.02823355.0 Negative110 22.0525210.2 Total500 10051352100 Quality Mapping Functional Suitability119 23.88806 29.15 Unclear97 19.47173 23.74 Reliability66 13.24821 15.96 Compatibility54 10.83983 13.18 Performance Efficiency448.83248 10.75 Security306.04721.56 Interaction Capability306.03821.26 Maintainability306.012454.12 Flexibility306.0460.15 Safety00.0100.03 Total500 10030214100 5.2.2. Quality Attribute Extraction (P3) Quality attribute extraction operates on community reviews filtered for relevance and sentiment polarity. In the available evaluation dataset, the sentiment-analysis stage covers 51,352 filtered review snippets. Of these, 17,867 were labelled positive, 28,233 neutral, and 5,252 negative. Because neutral snippets do not provide clear evidence about perceived quality, the quality-mapping stage focuses on the remaining evidence-bearing snippets and assigns 30,214 review instances to ISO/IEC 25010-inspired categories, including unclear cases. We validated both sentiment classification and ISO/IEC 25010 quality-attribute mapping using 500 annotated reviews. Three LLMs independently labeled each review; labels appearing in at least two outputs formed the reference set, which was then refined by a human annotator. Table 7 reports classification performance, and Table 8 summarizes both the annotated sample and the full filtered dataset. The pipeline achieved accuracy of 0.70 for sentiment classification and 0.84 for quality attribute mapping. Weighted metrics exceeded macro metrics due to class imbalance; neutral-sentiment reviews (55% of the full dataset) and unclear quality attributes (23.74%) were the most frequent. The confusion matrix (Figure 5) shows that unclear attributes were most often misclassified, reflecting the difficulty of distinguishing vague opinions from specific quality concerns in code-heavy, technical discussions. Filtering neutral reviews before quality mapping improved precision by removing low-information content, though separating subjective opinions from technical discourse remains challenging when reviews mix debugging notes, code snippets, and evaluative statements. Table 8 summarizes the distribution of sentiment and quality attribute labels in both the annotated sample and the full filtered dataset, illustrating the prevalence of functional suitability concerns and the relative scarcity of safety- related feedback. Summary for RQ1 (Pipeline Validation): Automated extraction pipelines achieve high accuracy at repository scale, yielding an F1-score of 0.80 (IoU=0.67) for functional capability features and 0.84 accuracy for mapping community review snippets to ISO/IEC 25010 perceived quality attributes. Joonbakhsh et al.: Preprint submitted to ElsevierPage 17 of 29 HugSelect: Explainable foundation-model selection Figure 5: Quality attribute mapping confusion matrix. 5.3. Framework Validation via Case Studies (RQ2) 5.3.1. Experimental Design We conducted a comparative evaluation of HugSelect against four commercial LLM-based recommendation systems across 44 literature-derived model selection scenarios. Baselines and Reproducibility We compared HugSelect against four practical LLM-based recommendation base- lines: ChatGPT, Claude, Gemini, and Perplexity. The baselines were evaluated as zero-shot conversational recom- menders using the same case descriptions. Each system was asked to return a ranked list of candidate foundation- models for the same selection scenario. The goal was not to claim a definitive benchmark of the underlying proprietary models, but to compare HugSelect with the type of recommendation support practitioners can obtain from widely used conversational systems. Because provider-managed interfaces can change over time and do not always expose decoding parameters, the comparison should be interpreted as a time-bound practical baseline comparison rather than a fully controlled model-to-model benchmark. Table 9 summarizes the configuration information recorded for the evaluation. Case Selection The 44 cases were systematically sampled to cover different modalities, task types, and application contexts. For each case, we recorded contextual attributes such as domain, modality, task, and selected model. Table 10 summarizes this distribution. Because some cases belong to more than one domain or task category, category counts should be interpreted as multi-label descriptors rather than mutually exclusive totals. This coverage is important because foundation-model selection depends on interactions among domain characteristics, task requirements, and modality constraints. At the model level, the dataset includes 32 unique models, and the most frequent model appears in only four cases, reducing the risk that the evaluation is dominated by a small set of popular models. Joonbakhsh et al.: Preprint submitted to ElsevierPage 18 of 29 HugSelect: Explainable foundation-model selection Table 9 Baseline configuration information used in the comparative evaluation. SystemInterface / model familyModeRecorded evaluation setting HugSelectLocal implementation with the curated knowledge base Deterministic ranking Same 44 case descriptions; top-10 ranked recommendations; WSM/SAW scoring with criterion-level explanation. ChatGPTGPT-4-based conversational recommender Zero-shotSame 44 case descriptions; top-10 candidate models requested; no fine-tuning or prompt optimization. Provider decoding settings were not exposed. ClaudeClaude 3 Opus conversational recommender Zero-shotSame 44 case descriptions; top-10 candidate models requested; no fine-tuning or prompt optimization. Provider decoding settings were not exposed. GeminiGemini Pro conversational recommender Zero-shotSame 44 case descriptions; top-10 candidate models requested; no fine-tuning or prompt optimization. Provider decoding settings were not exposed. PerplexityPerplexity AI with web-search augmentation Zero-shotSame 44 case descriptions; top-10 candidate models requested; no fine-tuning or prompt optimization. Provider decoding settings were not exposed. TypeCategoryCount Domain Language & Communication14 Education & Knowledge6 Healthcare & Medicine5 Media & Creative10 Security & Ethics3 Human-Computer Interaction5 Science & Methodology6 Modality Text / NLP22 Computer Vision8 Multimodal14 Task Text Generation7 Summarization6 Image Generation5 Image Captioning & Grounding3 Text Classification5 Visual Understanding6 Question Answering10 Translation2 Total Cases44 Table 10 Distribution of domains, modalities, and tasks across the 44 evaluation cases. Counts are multi-label descriptors and therefore may not sum to the total number of cases within each group. Ground Truth Construction The evaluation uses literature-derived proxy ground truth based on expert-informed model selections reported in peer-reviewed scientific studies. For each case, ground truth was established through a three-step process. First, candidate scientific papers were collected from the Semantic Scholar and OpenAlex databases using systematic keyword searches targeting studies that reported the use of Hugging Face models. Second, associated GitHub repositories mentioned in the abstracts were examined to verify explicit references to the reported models, and only papers from established conferences or journals were retained. Third, the remaining papers were manually reviewed to include only application-oriented cases in which a foundation-model was explicitly selected for a defined task, resulting in a curated set of scientific cases whose reported model choices were used as proxy ground truth for evaluation. Recommendations of each system were evaluated according to whether they recovered the reference model or its corresponding model family; other suitable alternatives may exist. Therefore, the case-study labels should be interpreted as proxy ground truth for comparative retrieval analysis, not as proof that the literature-reported model is the only correct or optimal choice. 5.3.2. Evaluation Metrics We employed two evaluation levels to capture complementary aspects of recommendation quality. Joonbakhsh et al.: Preprint submitted to ElsevierPage 19 of 29 HugSelect: Explainable foundation-model selection Table 11 Performance comparison of HugSelect against baseline systems (n=44 cases) at cutoff 10 across Model-Level (ML) and Family-Level (FL). Overlap@10 reports agreement with HugSelect’s top-10 list. Statistical significance relative to HugSelect is assessed via McNemar test; * 푝 < 0.05, *** 푝 < 0.001. System Model-LevelFamily-Level Cov@10 Ovlp@10 Cov@10 Ovlp@10 Prec@10 NDCG@10 HugSelect0.61-0.91-0.370.74 ChatGPT0.520.120.860.360.410.72 Claude0.640.120.930.450.430.79 Gemini0.39*0.060.890.370.290.69 Perplexity0.520.100.840.380.290.67 Model-Level (ML) Metrics Model-level metrics evaluate exact matches of model identifiers. Coverage@10 measures whether at least one reference model appears among the top-10 recommendations for a case. We also report Overlap@10 with HugSelect (Overlap@10) for the baseline systems as a diagnostic agreement measure, indicating the number of baseline recommendations that overlap with HugSelect’s top-10 list. This overlap measure is not a relevance metric; therefore, HugSelect itself is marked as the reference system in Table 11. Family-Level (FL) Metrics Family-level metrics evaluate model family matches (e.g., BERT variants or GPT variants), treating family members as potential substitutes when they satisfy the same selection intent. Coverage@10 measures whether the relevant model family is represented in the top-10 recommendations. Precision@10 is computed as the mean proportion of top-10 recommended families that belong to the expanded relevant family set for each case. This expanded set can contain multiple acceptable families when the source study, associated repository, or expert review indicates that more than one family is a valid substitute. NDCG@10 measures ranking quality with position- weighted relevance: NDCG@퐾 = DCG@퐾 IDCG@퐾 where higher-ranked relevant items contribute more to the score. Family-level metrics are particularly important for practical model selection because users often consider multiple variants within a model family as viable alternatives. The distinction between exact model identifiers and family- level substitutes also reduces the risk of penalizing semantically valid recommendations that differ only in version, quantization, or fine-tuning variant. 5.3.3. Statistical Significance and Effect Sizes An omnibus Friedman test on family-level rank distributions across all systems showed no significant global difference (휒 2 = 8.786, 푝 = 0.0667). Pairwise Wilcoxon signed-rank tests between HugSelect and commercial LLMs confirmed non-significant variations (푝 > 0.18 across all pairs). McNemar tests at the model level showed HugSelect significantly outperformed Gemini (푝 < 0.05, 푂푅 = 6.00) and metadata/filtering baselines (푝 < 0.001), while performing comparably to ChatGPT and Claude. 5.3.4. Results Table 11 presents the full empirical comparison. HugSelect achieved coverage competitive with the strongest LLM baselines at both the model level (0.61, second only to Claude’s 0.64) and family level (0.91, close to Claude’s 0.93 and Gemini’s 0.89). At the model level, HugSelect achieved higher coverage than Gemini and Perplexity, and the McNemar test showed a statistically significant difference only against Gemini (푝 < 0.05). Against Claude, the observed difference was small and not statistically significant. These findings indicate that HugSelect is competitive with commercial conversational systems on retrieval effectiveness while providing structured, criterion-level transparency. The NDCG@10 score of 0.74 places HugSelect among the stronger systems in terms of ranking quality, close to ChatGPT (0.72) and below Claude (0.79). The gap between model-level (0.61) and family-level (0.91) coverage suggests that HugSelect often surfaces relevant model Joonbakhsh et al.: Preprint submitted to ElsevierPage 20 of 29 HugSelect: Explainable foundation-model selection ConstructResponsesStatistics SA (1)A (2)SLA (3)N (4)SLD (5)D (6)SD (7)MeanSDCronbach’s 훼 PU (4)1525000001.620.490.95 PEOU (4)2315200001.480.600.92 ATT (4)112180000 1.930.640.88 ITU (4)1320700001.850.690.87 SN (2)03782003.450.690.76 IMG (2)00695003.950.800.84 JR (2)711200001.750.620.89 OQ (2)91100000 1.550.510.91 RD (2)38540002.500.830.72 Table 12 TAM-based user study results showing response distributions and reliability measures (lower values = stronger agreement) families even when exact model identifiers differ, which is useful in settings where multiple variants may satisfy the same intent. The modest Precision@10 (0.37), in line with all baselines (0.29–0.43), reflects the exploratory nature of model selection, where users may benefit from diverse recommendations spanning multiple families rather than a narrow list of near-duplicates. Qualitative Analysis Manual inspection of recommendation outputs revealed systematic differences between HugSelect and the LLM baselines. HugSelect provided structured recommendations with explicit feature matching, quality scores, and knowledge graph context, enabling users to trace why each model was recommended. In contrast, LLM baselines generated natural language explanations but often recommended popular models (e.g., BERT, GPT-2) regardless of specific requirements, exhibiting recency and popularity bias. While their explanations were fluent, they were sometimes factually inconsistent with the model’s actual capabilities. Figure 3 illustrates how HugSelect’s knowledge graph representation enables transparent comparison of model variations, supporting informed decision-making. 5.4. User Study Ten participants with domain knowledge in AI, computer science, and data science evaluated HugSelect. After a brief orientation, participants explored the system for ten minutes and completed a structured survey using a 7-point Likert scale (1 = Strongly Agree, 7 = Strongly Disagree). Responses were analyzed using descriptive statistics and Cronbach’s alpha for internal consistency. Given the small sample size, reliability values are interpreted as descriptive indicators rather than definitive psychometric evidence. The survey was based on the Technology Acceptance Model (TAM) Davis (1989) and TAM 2 Venkatesh and Davis (2000). TAM posits that system usage is determined by perceived usefulness (PU) and perceived ease of use (PEOU), which influence attitude toward using (ATT) and behavioral intention to use (ITU). TAM 2 introduced five external variables: subjective norm (SN), image (IMG), job relevance (JR), output quality (OQ), and result demonstrability (RD). The survey included four statements for each core construct and two for each external variable. 5.4.1. Results Table 12 presents response distributions, mean scores, standard deviations, and Cronbach’s alpha for each construct. Perceived Usefulness (훼 = 0.95, 푀 = 1.62) received strong agreement, indicating participants recognized the system’s value for improving model selection. Perceived Ease of Use (훼 = 0.92, 푀 = 1.48) showed the lowest mean, reflecting consensus that the system is intuitive and requires minimal effort. Attitude Toward Using (훼 = 0.88, 푀 = 1.93) and Intention to Use (훼 = 0.87, 푀 = 1.85) both received positive evaluations, suggesting users view the system favorably and would adopt it in practice. External constructs showed more diverse responses. Subjective Norm (훼 = 0.76, 푀 = 3.45) and Image (훼 = 0.84, 푀 = 3.95) yielded neutral scores, indicating that adoption is driven more by functionality than by social influence. Job Relevance (훼 = 0.89, 푀 = 1.75) and Output Quality (훼 = 0.91, 푀 = 1.55) reinforced the system’s practical applicability and recommendation quality. Result Demonstrability (훼 = 0.72, 푀 = 2.50) showed moderate agreement with higher variance. Joonbakhsh et al.: Preprint submitted to ElsevierPage 21 of 29 HugSelect: Explainable foundation-model selection Model-Level (ML)Family-Level (FL) System VariantCoverage@10 p-value Coverage@10 p-value HugSelect (complete)0.61–0.91– w/o Functional Features0.23 ∗ <0.0010.800.13 w/o Quality Attributes0.41 ∗ 0.010.770.07 w/o Functional & Quality0.09 ∗ <0.0010.73 ∗ 0.04 Table 13 Ablation study of HugSelect at cutoff 10, evaluating the impact of removing functional features and quality attributes. Coverage@10 is reported at Model-Level (ML) and Family-Level (FL). Statistical significance is computed using the McNemar test relative to the full system; ∗ 푝 < 0.05, ∗ 푝 < 0.01, ∗ 푝 < 0.001. Overall, participants responded positively to HugSelect in terms of usefulness and usability. The reliability coefficients indicate consistent responses across the survey constructs, but the small sample size limits the strength of the statistical conclusions. The findings should therefore be interpreted as exploratory evidence evaluating technology acceptance and user perception. Summary for RQ3 (User Perception Study): In an exploratory evaluation (푛 = 10) based on TAM constructs, practitioners rated HugSelect favorably for Perceived Usefulness (푀 = 1.62) and Perceived Ease of Use (푀 = 1.48), confirming that structured score decomposition and explicit criterion trade-offs provide an intuitive and transparent selection aid. 5.5. Ablation Study One advantage of the MCDM framework is its explainability, which enables analysis of how different knowledge base components influence recommendation results. During the development of HugSelect, several design variations were examined to study the effect of different feature groups and their weights. Because the framework integrates heterogeneous information sources, including metadata, functional features, and quality attributes, evaluating the contribution of each component is necessary. An ablation study was conducted by selectively removing components of the HugSelect pipeline and re-evaluating system performance. This approach isolates the impact of each feature type and reveals the system’s dependence on different information sources. The results in Table 13 show that the complete HugSelect configuration achieves the best performance across both evaluation settings, confirming the benefit of combining all feature types. Removing functional features causes the largest single-component drop at the model level, with Coverage@10 falling from 0.61 to 0.23 (p<0.001), indicating that functional features are the primary driver of exact-model retrieval. Removing quality attributes produces a smaller but still significant decline at the model level (0.61 to 0.41, p<0.05) and the largest relative drop at the family level (0.91 to 0.77), reflecting their complementary role in distinguishing among related model variants. When both functional features and quality attributes are removed, the system relies solely on metadata and yields the lowest performance (0.09 at ML and 0.73 at FL), underscoring the limitations of metadata-only approaches. Overall, the results show that functional features and quality attributes provide complementary signals (functional features dominate exact-model retrieval, while quality attributes are especially valuable for discriminating among model families), and that combining both, together with metadata, yields the most accurate and explainable foundation- model recommendations. 5.6. Threats to Validity Several threats to validity should be considered when interpreting the results. Construct validity. The 44 case studies use literature-reported model choices as proxy ground truth. This is a defensible way to obtain realistic selection scenarios, but it does not imply that the reported model is the only suitable or optimal model for the task. To reduce this threat, we report both model-level and family-level metrics, because semantically appropriate alternatives may differ in version, quantization, or fine-tuning variant. Similarly, community feedback is used as evidence of perceived quality rather than as an objective measurement of runtime behavior. A Joonbakhsh et al.: Preprint submitted to ElsevierPage 22 of 29 HugSelect: Explainable foundation-model selection review about slow inference, for example, is treated as a signal of perceived performance efficiency, not as a controlled benchmark result. Internal validity. The functional-feature and quality-attribute validation datasets were constructed through human- in-the-loop procedures supported by multiple LLMs. This design reduces single-model bias, but it does not fully remove subjectivity because feature identification and quality-attribute coding from unstructured text can be ambiguous. The ground-truth references should therefore be seen as carefully curated evaluation references rather than absolute labels. The MCDM ranking also depends on criterion weights. This is intentional because the system is designed for configurable decision support, but different weights can produce different rankings. The MoSCoW-inspired weighting scheme provides structure, yet future work should study alternative weighting strategies and compare additional MCDM methods such as AHP, TOPSIS, VIKOR, ELECTRE, PROMETHEE, and fuzzy MCDM variants. Baseline validity and reproducibility. The commercial LLM baselines were evaluated as practical zero-shot systems. This reflects common practitioner use, but it may underestimate what could be achieved with specialized prompt engineering, retrieval augmentation, repeated sampling, or fine-tuning. In addition, provider-managed systems evolve over time and do not always expose model versions, decoding settings, or retrieval behavior. For this reason, we interpret the comparison as a time-bound practical baseline rather than a fully controlled benchmark of proprietary LLMs. External validity. The knowledge base and evaluation focus on Hugging Face. Although the framework is repository-agnostic in principle, generalization to TensorFlow Hub, PyTorch Hub, ModelScope, GitHub-hosted model releases, or private enterprise model registries requires further validation. The 44 cases cover multiple domains, modalities, and tasks, but they cannot represent all software-engineering contexts in which foundation-models are selected. Domain-specific settings such as safety-critical healthcare, legal NLP, embedded deployment, and regulated industrial systems require additional evidence. User-study validity. The user study involved ten participants and should be interpreted as exploratory. The TAM- based results provide initial evidence about perceived usefulness, ease of use, output quality, and result demonstrability, but the sample size is too small for strong psychometric or population-level conclusions. Larger studies with professional developers, ML engineers, and domain stakeholders are needed to evaluate adoption, decision quality, and long-term usefulness in realistic development projects. Data quality and ecosystem dynamics. HugSelect depends on the availability and quality of model cards, README files, repository metadata, and community discussions. Sparsely documented or less popular models may receive weaker feature and quality profiles, and popular models may receive more feedback simply because they are widely discussed. The model ecosystem also changes rapidly through new releases, deprecations, fine-tuned variants, and changes in licensing. The current implementation supports periodic updates, but continuous monitoring and longitudinal validation are needed to keep the knowledge base current. Despite these limitations, the evaluation provides preliminary evidence that HugSelect can support transparent foundation-model selection at repository scale. The results should be read as evidence of feasibility and practical potential, not as a final claim that the system identifies the globally optimal model for every task. 6. Discussion This section interprets the findings from Section 5 and discusses their implications for software-engineering research and practice. The central argument of this paper is that foundation-model selection should be treated as an explicit software-engineering decision rather than as an ad hoc search activity. HugSelect operationalizes this view by combining repository mining, knowledge-base construction, and multi-criteria decision support. 6.1. Interpretation of Results The evaluation suggests that HugSelect can provide recommendation quality comparable to widely used conversa- tional systems while offering a different and complementary form of support. At the family level, HugSelect achieved Coverage@10 of 0.91 and NDCG@10 of 0.74 (Table 11), indicating that the knowledge base and MCDM engine often identify model families that are consistent with literature-derived selection scenarios. At the model level, HugSelect achieved Coverage@10 of 0.61, placing it between the strongest and weaker commercial baselines in the evaluated cases. Joonbakhsh et al.: Preprint submitted to ElsevierPage 23 of 29 HugSelect: Explainable foundation-model selection The main benefit of HugSelect is not that it always outperforms conversational recommenders. Rather, the results show that explicit decision logic can reach competitive retrieval performance while providing traceability that general- purpose LLM interfaces do not provide by default. The score decomposition, feature matching, quality-attribute links, and family-level mappings make it possible to inspect why a model is recommended and which trade-offs drive the ranking. This is particularly important when model selection must be justified to project teams, clients, reviewers, auditors, or other stakeholders. The ablation study reinforces this interpretation. Removing functional features produced the largest drop in model-level Coverage@10, while removing quality attributes also reduced performance and weakened the ability to discriminate among related variants. This indicates that model selection benefits from combining multiple evidence sources rather than relying only on metadata or popularity. At the same time, the modest Precision@10 scores across all systems show that foundation-model selection remains exploratory: a useful system should help users inspect a set of plausible candidates, not merely output a single definitive answer. The user study provides preliminary evidence that practitioners value this structured form of decision support. Participants rated perceived usefulness, ease of use, job relevance, and output quality positively (Table 12). Because the study is small, these findings should be interpreted as early usability evidence rather than conclusive proof of adoption. Still, the results support the claim that explicit criteria and explanations can make foundation-model selection more understandable. 6.2. Implications for Software Engineering Practice Model selection as component selection. foundation-models increasingly function as reusable AI components in software systems. Selecting such a component affects architecture, integration effort, deployment cost, license compliance, quality risks, and maintainability. HugSelect makes these concerns explicit by representing models as alternatives with metadata, functional capabilities, and perceived quality attributes. Traceable recommendations. In practical software projects, a recommendation is useful only if stakeholders can understand and challenge it. The MCDM-based ranking mechanism supports this need by exposing criterion-level score contributions and the evidence behind them. This makes the recommendation process more auditable than a purely conversational answer, while still allowing users to adjust priorities. Repository-scale knowledge integration. The results show that repository metadata alone is insufficient for effective selection. Model cards, README files, and community feedback contain complementary information that can improve matching and explanation. HugSelect therefore demonstrates how repository mining can be connected to decision-support models for AI-enabled system development. 6.3. Implications for Research For software-engineering research, HugSelect contributes evidence that MCDM-based technology-selection frameworks can be extended from traditional software and cloud-service selection to foundation-model ecosystems. The work also highlights new research challenges: maintaining current knowledge bases for rapidly changing model hubs, validating perceived quality signals against runtime benchmarks, evaluating recommendations when multiple alternatives are acceptable, and designing interactive tools that help users negotiate trade-offs. 6.4. Future Directions Richer evidence sources. Future versions should incorporate benchmark results, fine-tuning datasets, hardware requirements, energy consumption, license-change histories, vulnerability reports, and deployment documentation. These sources would help connect perceived quality signals to more objective evidence. Interactive refinement. HugSelect can be extended with conversational refinement, relevance feedback, and what-if analysis so users can iteratively adjust criteria and inspect how rankings change under different priorities. Cross-repository integration. Although this study focuses on Hugging Face, the architecture can be adapted to other repositories and private model registries. Cross-repository integration would support organizations that combine public open-source models with internal fine-tuned variants. Joonbakhsh et al.: Preprint submitted to ElsevierPage 24 of 29 HugSelect: Explainable foundation-model selection Longitudinal and industrial evaluation. Future work should evaluate HugSelect in longitudinal industrial settings, where teams use the system during real development projects and later assess whether the selected models satisfy operational requirements over time. 7. Conclusion This paper addressed the problem of selecting foundation-models from large and heterogeneous repositories. We argued that this problem should be treated as a software-engineering decision because foundation-models increasingly act as reusable AI components whose selection affects system functionality, quality attributes, deployment constraints, licensing, cost, and maintainability. HugSelect was proposed as an explainable multi-criteria decision-support framework for this setting. The framework integrates structured repository metadata, functional features extracted from model-card descriptions, and perceived quality signals derived from community feedback into a unified knowledge base. It then applies a weighted WSM/SAW decision model to rank candidate models and expose criterion-level contributions to the final score. The evaluation provides preliminary but encouraging evidence. The extraction-pipeline validation shows that useful functional and perceived quality signals can be obtained from heterogeneous repository and community sources, although imperfectly. The 44-case comparative evaluation shows that HugSelect achieves recommendation quality competitive with practical commercial LLM-based baselines, especially at the family level, while providing more explicit and auditable reasoning. The ablation study demonstrates that functional features and perceived quality attributes contribute complementary value beyond metadata alone. The user study suggests that participants perceive the system as useful, understandable, and relevant, although the small sample size limits the strength of this conclusion. The main contributions of this paper are: (i) a repository-scale MCDM formulation of foundation-model selection for AI-enabled software engineering; (i) a knowledge-base construction pipeline integrating metadata, model descrip- tions, and community-derived perceived quality evidence; (i) an explainable ranking engine with criterion-level score decomposition; and (iv) an empirical evaluation combining extraction validation, comparative case studies, ablation analysis, and an exploratory user study. Several limitations remain. Literature-reported model choices provide realistic but imperfect proxy ground truth, community feedback reflects perceived rather than objective quality, commercial LLM baselines are time-bound, and the user study is exploratory. Future work should expand the evaluation to additional repositories and industrial contexts, incorporate objective benchmark and deployment evidence, support interactive refinement, and study how recommendations evolve as model ecosystems change. Overall, HugSelect demonstrates that repository mining and multi-criteria decision making can be combined to support more transparent foundation-model selection. By exposing evidence and trade-offs rather than hiding them inside a single opaque recommendation, the framework offers a practical basis for informed model selection in software engineering and AI-enabled system development. Declaration of Competing Interest and Data Availability The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Data Availability Data associated with this study have been deposited at Mendeley Data under the accession number 10.17632/9kx- tkvyv5m.2 (https://doi.org/10.17632/9kxtkvyv5m.2). The dataset includes the raw model information, vali- dation data, case study results, user study records, and data collection and processing code. A. Literature Study Gap-Analysis Table Table 14 presents the full gap-analysis matrix used to position HugSelect relative to related work. The original visual matrix has been converted into grouped descriptors to improve readability. Joonbakhsh et al.: Preprint submitted to ElsevierPage 25 of 29 HugSelect: Explainable foundation-model selection Table 14: Comparison of studies on decision-making and recommendation support for foundation-model-related domains. Study Year Domain Alt. Crit. Approach Data sources DC mode Feature / evidence collection Decisionmodel MCDM method Evaluation Weighting Leščinskait ̇ e, P. et al. 2026 ML Artifacts 139 7 Overview, Benchmarking Public literature, OpenAI Semi- automated Textual attributes,Benchmarking MCDM WSM/SAW, AHP, TOPSIS Comparative,Other – Olabanjo, O. et al. 2026 AI System Designs 10 8 Overview, Selection Public literature Automated Surveying, Expert opinion, Benchmarking MCDM WSM/SAW, AHP, TOPSIS, Other Comparative,Other Value-based,Statistical Farshidi, S. et al. 2025 Software Packages 39,841 3 Overview, Taxonomy,Selection GitHub, Public literature,Public dataset, StackOverflow, Gray literature Automated Metadata extraction, Textual attributes,Entity/relationextraction, Surveying,Standard mapping,Machine learning MCDM WSM/SAW, Fuzzy MCDM Comparative,User study,Qualitative Graph-based,Value-based Chen, Q. et al. 2025 foundation-models 121,404 14 Taxonomy, Selection,Benchmarking Hugging Face Automated Metadata extraction, Textual attributes,Entity/relation extraction KB/ED,DDDM – Comparative AI-based Mienye, I. et al. 2025 Large Language Models 31 4 Overview, Taxonomy Public literature – Surveying None – None – Minaee, S. et al. 2025 Large Language Models 48 3 Overview, Taxonomy,Benchmarking Public literature, Publicdataset – Benchmarking None – None – Radulescu, C. et al. 2025 GenAI Models 5 7 Selection Public literature – – MCDM Other Other – Gonzalez, A. et al. 2025 Pre-Trained Models 2,205 5 Taxonomy, Selection Public literature,Hugging Face – Metadata extraction, Textual attributes KB/ED – None – Jain, A. et al. 2025 Pre-Trained Models 5 4 Selection, Benchmarking Public literature Manual Metadata extraction, Textual attributes,Machine learning,Benchmarking MCDM TOPSIS Comparative, Other – Alsalem, M.A. et al. 2024 AI Applications 50 7 Selection, Benchmarking Public literature Manual Expert opinion, Standard mapping MCDM Fuzzy MCDM, Other Other Value-based Drissi, N. et al. 2024 ML Models 8 7 Overview, Benchmarking Public literature Manual Textual attributes, Expert opinion,Benchmarking AI/ML,MCDM WSM/SAW, AHP Comparative, Other Value-based Bhol, S.G. et al. 2024 MLaaS 4 17 Selection Public literature, Gray literature Manual Metadata extraction, Textual attributes,Entity/relationextraction, Surveying MCDM AHP, TOPSIS Comparative Value-based Di Sipio, C. et al. 2024 Pre-Trained Models 3 6 Taxonomy, Selection Public literature,Hugging Face Semi-automated Metadata extraction,Textual attributes KB/ED – User study Value-based Suryani, M. et al. 2024 foundation-models 1,090 2 Overview, Taxonomy GitHub, Public literature,Hugging Face – Metadata extraction, Entity/relation extraction None – None – Chen, Z. et al. 2024 foundation-models 43 6 Overview Public literature – Surveying None – None – Zhou, J. et al. 2024 Agents 97 9 Taxonomy Public literature, Gray literature Manual Surveying None Other Qualitative – Ding, Y. et al. 2024 Vision-Language Models 17 10 Selection, Benchmarking Public dataset Manual Machine learning, Benchmarking AI/ML,DDDM Other Comparative – Lu, Q. et al. 2023 foundation-models 72 3 Taxonomy Public literature – Surveying None – None – Azad, B. et al. 2023 foundation-models 40 22 Overview, Taxonomy Public literature Manual Surveying None – None – Continued on next page Joonbakhsh et al.: Preprint submitted to ElsevierPage 26 of 29 HugSelect: Explainable foundation-model selection Study Year Domain Alt. Crit. Approach Data sources DC mode Feature / evidence collection Decisionmodel MCDM method Evaluation Weighting Awais, M. et al. 2023 foundation-models 97 9 Overview, Taxonomy Public literature Manual Surveying None – None – Shen, Y. et al. 2023 foundation-models 23 3 Selection GitHub, Hugging Face, OpenAI – – AI/ML – Comparative – Zhou, C. et al. 2023 foundation-models 67 3 Overview, Taxonomy Public literature – Surveying None – None – Chakrabortty, R. etal. 2023 Chatbots 8 9 Selection Public literature Manual Expert opinion MCDM TOPSIS, Fuzzy MCDM Other Value-based Hu, Z. et al. 2023 Pre-Trained Models 6 3 Selection, Benchmarking GitHub Manual Benchmarking AI/ML Other Comparative – Zhao, W. et al. 2023 Large Language Models 57 5 Overview Public literature, Stack Overflow – Surveying None – None – Tsay, J. et al. 2023 Machine Learning Models 7,900 8 Overview GitHub, Public literature, Public dataset Automated Metadata extraction DDDM – Comparative – Liu, M. et al. 2023 Machine Learning Models 25,718 11 Taxonomy, Selection GitHub, Stack Overflow – Textual attributes, Entity/relation extraction KB/ED – User study – Khan, S. et al. 2022 Transformer Models 60 6 Overview Public literature – Surveying None – None – Şeker andKahraman 2021 Software Packages 4 8 Selection Public literature Manual Expert opinion MCDM TOPSIS, Fuzzy MCDM Comparative,Other Performance-based,Statistical Cao, X. et al. 2021 Machine Learning 6,239 3 Selection, Benchmarking GitHub, Public dataset – Textual attributes, Entity/relation extraction AI/ML,DDDM – Comparative Graph-based HugSelect 2026 foundation-models 71,274 3 Overview,Taxonomy,Selection GitHub, Public literature,Public dataset, HuggingFace, Stack Overflow,Gray literature Automated Metadata extraction,Textual attributes,Entity/relationextraction, Surveying,Expert opinion, Standardmapping, Machinelearning MCDM WSM/SAW Comparative,User study,Qualitative Graph-based,Value-based Notes. Alt. = number of alternatives; Crit. = number of criteria; DC mode = data collection mode; KB/ED = knowledge-based or expert-driven decision model; DDDM = data-driven decision-making; WSM/SAW = Weighted Sum Model / Simple Additive Weighting. The original visual matrix used x-marks; this table converts the same information into grouped descriptors to improve readability. Joonbakhsh et al.: Preprint submitted to ElsevierPage 27 of 29 HugSelect: Explainable foundation-model selection References Adalı, A., Joonbakhsh, A., Farshidi, S., 2026. Hugselect dataset and replication package. URL: https://doi.org/10.17632/9kxtkvyv5m.2, doi:10.17632/9kxtkvyv5m.2. Adomavicius, G., Manouselis, N., Kwon, Y., 2010. Multi-criteria recommender systems, in: Recommender systems handbook. Springer, p. 769– 803. Ahmadjee, S., Mera-Gómez, C., Farshidi, S., Bahsoon, R., Kazman, R., 2025. Decision support model for selecting the optimal blockchain oracle platform: An evaluation of key factors. ACM Transactions on Software Engineering and Methodology 34, 23:1–23:35. doi:10.1145/3697011. Alsalem, M.A., Alamoodi, A.H., Albahri, O.S., Albahri, A.S., Martínez, L., Yera, R., Duhaim, A.M., Sharaf, I.M., 2024. Evaluation of trustworthy artificial intelligent healthcare applications using multi-criteria decision-making approach. Expert Systems with Applications 246, 123066. doi:10.1016/j.eswa.2023.123066. Awais, M., Naseer, M., Khan, S., Anwer, R.M., Cholakkal, H., Shah, M., Yang, M.H., Khan, F.S., 2025. Foundation models defining a new era in vision: a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence . Azad, B., Azad, R., Eskandari, S., Bozorgpour, A., Kazerouni, A., Rekik, I., Merhof, D., 2023. Foundational models in medical imaging: A comprehensive survey and future vision. arXiv preprint arXiv:2310.18689 . Behzadian, M., Otaghsara, S.K., Yazdani, M., Ignatius, J., 2012. A state-of the-art survey of topsis applications. Expert Systems with applications 39, 13051–13069. Bhol, S.G., Mohanty, S., Pattnaik, P.K., 2024. Machine learning as a service cloud selection: An mcdm approach for optimal decision making. Procedia Computer Science 233, 909–918. URL: https://w.sciencedirect.com/science/article/pii/S1877050924006409, doi:https://doi.org/10.1016/j.procs.2024.03.280. 5th International Conference on Innovative Data Communication Technologies and Application (ICIDCA 2024). Bommasani, R., 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 . Cao, X., Shi, Y., Yu, H., Wang, J., Wang, X., Yan, Z., Chen, Z., 2021. Dekr: description enhanced knowledge graph for machine learning method recommendation, in: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, p. 203–212. Cazzaniga, M., Jaumotte, M.F., Li, L., Melina, M.G., Panton, A.J., Pizzinelli, C., Rockall, E.J., Tavares, M.M.M., 2024. Gen-AI: Artificial intelligence and the future of work. International Monetary Fund. Chakrabortty, R.K., Abdel-Basset, M., Ali, A.M., 2023. A multi-criteria decision analysis model for selecting an optimum customer service chatbot under uncertainty. Decision Analytics Journal 6, 100168. Chen, Q., Huang, K., Zhou, X., Luo, W., Cui, Y., Cheng, G., 2025. Benchmarking recommendation, classification, and tracing based on hugging face knowledge graph, in: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 3433–3443. Chen, Z., Xu, L., Zheng, H., Chen, L., Tolba, A., Zhao, L., Yu, K., Feng, H., 2024. Evolution and prospects of foundation models: From large language models to large multimodal models. Computers, Materials & Continua 80. Clegg, D., Barker, R., 1994. Case method fast-track: a RAD approach. Addison-Wesley Longman Publishing Co., Inc. Davis, F.D., 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly , 319–340. DeepSeek AI, 2025. Deepseek-r1: Reasoning-optimized language model. URL: https://huggingface.co/deepseek-ai/DeepSeek-R1. accessed: 2025-01-15. Di Sipio, C., Rubei, R., Di Rocco, J., Di Ruscio, D., Nguyen, P.T., 2024. Automated categorization of pre-trained models in software engineering: A case study with a hugging face dataset, in: Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering, p. 351–356. Ding, Y., Jiang, B., Zheng, A., Xu, Q., Liang, J., 2024. Learning to rank pre-trained vision-language models for downstream tasks. arXiv preprint arXiv:2412.20682 . Drissi, N., El-Kassabi, H., Serhani, M., 2024. A multi-criteria decision analysis framework for evaluating deep learning models in healthcare research. Decision Analytics Journal doi:10.1016/j.dajour.2024.100523. Farshidi, S., 2020. Multi-criteria decision-making in software production. Ph.D. thesis. Utrecht University. Farshidi, S., Jansen, S., Deldar, M., 2021. A decision model for programming language ecosystem selection: Seven industry case studies. Information and software technology 139, 106640. Farshidi, S., Jansen, S., España, S., Verkleij, J., 2020. Decision support for blockchain platform selection: Three industry case studies. IEEE transactions on Engineering management 67, 1109–1128. Farshidi, S., Jansen, S., de Jong, R., Brinkkemper, S., 2018. A decision support system for software technology selection. Journal of Decision systems 27, 98–110. Farshidi, S., Saberhabibi, A., Eskafi, B., Nikfarjam, N., Eskandari, S., Jansen, S., Chaudron, M., Tekinerdogan, B., 2025. Empirical evaluation of ai-assisted software package selection: A knowledge graph approach. arXiv preprint arXiv:2508.05693 . Faulconbridge, J., Sarwar, A., Spring, M., 2023. How professionals adapt to artificial intelligence: The role of intertwined boundary work. Journal of Management Studies . González, A., Franch, X., Lo, D., Martínez-Fernández, S., 2025. How do pre-trained models support software engineering? an empirical study in hugging face. arXiv preprint arXiv:2506.03013 . Google DeepMind, 2023. Gemini: A family of highly capable multimodal models. URL: https://deepmind.google/technologies/gemini/. technical Report. Hevner, A.R., March, S.T., Park, J., Ram, S., 2004. Design science in information systems research. MIS quarterly , 75–105. Honnibal, M., Montani, I., 2017. spacy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing, To appear. Hugging Face, 2023. Hugging face hub api. URL: https://huggingface.co/docs/hub/api. accessed: 2024-01-15. Joonbakhsh et al.: Preprint submitted to ElsevierPage 28 of 29 HugSelect: Explainable foundation-model selection Jain, S.M., 2022. Hugging face, in: Introduction to transformers for NLP: With the hugging face library and models to solve problems. Springer, p. 51–67. Jansen, S., Farshidi, S., Gousios, G., van der Storm, T., Visser, J., Bruntink, M., 2020. SearchSECO: A worldwide index of the open source software ecosystem, in: Proceedings of the 19th Belgium-Netherlands Software Evolution Workshop (BENEVOL 2020), CEUR-WS.org. URL: https://ceur-ws.org/Vol-2912/paper3.pdf. Joonbakhsh, A., Rostami, A., Kamalinia, A., Nazeri, A., Khunjush, F., Tekinerdogan, B., Farshidi, S., 2025. Evidence-driven decision support for ai model selection in research software engineering. arXiv preprint arXiv:2512.11984 . Leščinskait ̇ e, P., Paulavičius, R., Filatovas, E., 2026. A mcdm-based framework for quantifying reproducibility readiness in machine learning research. Mathematics 14. URL: https://w.mdpi.com/2227-7390/14/9/1536, doi:10.3390/math14091536. Li, N., Farshidi, S., Bianchi, R., Koulouzis, S., Zhao, Z., 2022. Context-aware notebook search in a jupyter-based virtual research environment, in: 2022 IEEE 18th International Conference on e-Science (e-Science), IEEE. p. 393–394. doi:10.1109/eScience55777.2022.00054. Liu, M., Zhao, C., Peng, X., Yu, S., Wang, H., Sha, C., 2023. Task-oriented ml/dl library recommendation based on a knowledge graph. IEEE Transactions on Software Engineering 49, 4081–4096. Lü, L., Medo, M., Yeung, C.H., Zhang, Y.C., Zhang, Z.K., Zhou, T., 2012. Recommender systems. Physics reports 519, 1–49. Martin, W., Sarro, F., Jia, Y., Zhang, Y., Harman, M., 2017. A survey of app store analysis for software engineering. IEEE Transactions on Software Engineering 43, 817–847. doi:10.1109/TSE.2016.2630689. Mienye, I.D., Jere, N., Obaido, G., Ogunruku, O.O., Esenogho, E., Modisane, C., 2025. Large language models: an overview of foundational architectures, recent trends, and a new taxonomy. Discover Applied Sciences 7, 1027. Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., Gao, J., 2024. Large language models: A survey. arXiv preprint arXiv:2402.06196 . Olabanjo, O., Honenberger, P., 2026. Mcda4ai: A framework for managing n>2 criteria problems in decisions about artificial intelligence. Array 29, 100723. URL: https://w.sciencedirect.com/science/article/pii/S2590005626000469, doi:https://doi.org/10.1016/j. array.2026.100723. Radulescu, C.Z., Radulescu, M., 2025. Criteria analysis for the selection of a generative artificial intelligence tool for academic research based on an improved group dematel method. Applied Sciences 15, 5416. Reddit Inc., 2023. Reddit api documentation. URL: https://w.reddit.com/dev/api/. accessed: 2024-01-15. Ricci, F., Rokach, L., Shapira, B., 2010. Introduction to recommender systems handbook, in: Recommender systems handbook. Springer, p. 1–35. Seker, S., Kahraman, C., 2021. A Pythagorean cubic fuzzy methodology based on TOPSIS and TODIM methods and its application to software selection problem. Soft Computing - A Fusion of Foundations, Methodologies and Applications URL: https://w.researchsquare.com/ article/rs-709398/latest.pdf, doi:10.1007/s00500-021-06469-8. Stack Exchange Inc., 2023. Stack exchange api v2.3. URL: https://api.stackexchange.com/docs. accessed: 2024-01-15. Suryani, M.A., Karmakar, S., Mathiak, B., 2024. Exploration of hugging face models by heterogeneous information network and linking across scholarly repositories, in: International Conference on Advances in Social Networks Analysis and Mining, Springer. p. 371–386. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al., 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 . Triantaphyllou, E., 2000. Multi-criteria decision making methods, in: Multi-criteria decision making methods: A comparative study. Springer, p. 5–21. Vaidya, O.S., Kumar, S., 2006. Analytic hierarchy process: An overview of applications. European Journal of operational research 169, 1–29. Venkatesh, V., Davis, F.D., 2000. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management science 46, 186–204. Wohlin, C., 2014. Guidelines for snowballing in systematic literature studies and a replication in software engineering, in: Proceedings of the 18th international conference on evaluation and assessment in software engineering, p. 1–10. Zhang, Y., Ding, H., Shui, Z., Ma, Y., Zou, J., Deoras, A., Wang, H., 2021. Language models as recommender systems: Evaluations and limitations . Zhao, Z., Fan, W., Li, J., Liu, Y., Mei, X., Wang, Y., Wen, Z., Wang, F., Zhao, X., Tang, J., et al., 2024. Recommender systems in the era of large language models (llms). IEEE Transactions on Knowledge and Data Engineering . Zhou, J., Lu, Q., Chen, J., Zhu, L., Xu, X., Xing, Z., Harrer, S., 2024. A taxonomy of architecture options for foundation model-based agents: Analysis and decision model. arXiv preprint arXiv:2408.02920 . Joonbakhsh et al.: Preprint submitted to ElsevierPage 29 of 29