Paper deep dive

LLM-Assisted Discovery of Typed Semantic Links for Ontology Network Construction

Nouha Hayouni, Sheeba Samuel, Alsayed Algergawy

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 53

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%

Last extracted: 10/3/2026, 3:01:32 AM

Summary

This paper presents an end-to-end framework for constructing ontology networks by automating the discovery of typed, justified semantic links between concepts in heterogeneous ontologies. The pipeline combines domain-adapted DistilBERT embeddings, clustering-based pre-filtering, and GPT-4o-driven relationship generation via iterative prompt engineering. Applied to the ReproduceMeON network (33 ontologies), the method reduces 800k candidate pairs to 95k high-quality candidates, achieving an F1 score of 0.890 and precision of 80.19% in human expert validation, significantly outperforming similarity-based baselines like Sentence-BERT.

Entities (8)

GPT-4o · large-language-model · 98%ReproduceMeON · ontology-network · 98%DistilBERT · machine-learning-model · 95%KMeans · clustering-algorithm · 90%Sentence-BERT · baseline-model · 90%LogMap · alignment-tool · 85%OWL2Vec* · embedding-method · 85%Ontology Design Patterns · methodology · 80%

Relation Signals (8)

GPT-4o → isusedfor → Relationship Generation

confidence 98% · LLM Relationship Generation—GPT-4o assigns typed, justified links

ReproduceMeON → uses → GPT-4o

confidence 98% · GPT-4o-driven relationship generation... Applied to ReproduceMeON

DistilBERT → isusedfor → Contextual Embedding Generation

confidence 95% · Contextual Embedding Generation—fine-tune DistilBERT on domain-enriched sentences

ReproduceMeON → uses → DistilBERT

confidence 95% · Our approach combines domain-adapted DistilBERT embeddings... Applied to ReproduceMeON

KMeans → isusedfor → Candidate Filtering

confidence 90% · clustering-based pre-filtering to reduce the candidate search space

DistilBERT → outperforms → Sentence-BERT

confidence 90% · outperforming the best similarity-based baseline (Sentence-BERT, F1 = 0.581) by 30.9 percentage points

GPT-4o → outperforms → Sentence-BERT

confidence 90% · Comparative experiments against five similarity-based baselines, including Sentence-BERT, show a substantial performance gap

Role-Based Mapping → isbasedon → Ontology Design Patterns

confidence 80% · This design is grounded in established Ontology Design Patterns (ODPs)

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Constructing typed, justified semantic links between ontologies is essential for enabling interoperability across heterogeneous and interdisciplinary knowledge domains. However, manually curating such links is difficult to scale. To address this challenge, we propose an end-to-end framework for ontology network construction that automates the discovery and generation of both intra-domain and inter-domain relationships. Our approach combines domain-adapted DistilBERT embeddings for dense contextual representation, clustering-based pre-filtering to reduce the candidate search space, and GPT-4o-driven relationship generation via iterative prompt engineering to produce semantically rich, interpretable links. Applied to ReproduceMeON - a network of 33 ontologies spanning machine learning, microscopy, computational science, and experimental workflow - the pipeline reduces approximately 800k raw concept pairs to 95k high-quality candidates. Human expert validation of 429 generated relationships by two independent annotators yields an overall precision of 80.19% (91.49% on high-certainty annotations) and an F1 of 0.890, with substantial inter-annotator agreement. Comparative experiments against five similarity-based baselines, including Sentence-BERT, show a substantial performance gap (best baseline F1 = 0.581), while an ablation study demonstrates that similarity-based methods alone fail to discriminate valid from invalid relationships (AUC approx 0.5) on the filtered candidate set. These findings highlight the necessity of LLM-based reasoning over concept roles and domain semantics for accurate relationship construction.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

49,689 characters extracted from source content.

Expand or collapse full text

LLM-Assisted Discovery of Typed Semantic Links for Ontology Network Construction Nouha Hayouni 1 , Sheeba Samuel 2,* and Alsayed Algergawy 1 Chair of Data and Knowledge Engineering, University of Passau, Germany 2 Distributed and Self-organizing Systems, Chemnitz University of Technology, Germany Abstract Constructing typed, justified semantic links between ontologies is essential for enabling interoperability across heterogeneous and interdisciplinary knowledge domains. However, manually curating such links is difficult to scale. To address this challenge, we propose an end-to-end framework for ontology network construction that automates the discovery and generation of both intra-domain and inter-domain relationships. Our approach combines domain-adapted DistilBERT embeddings for dense contextual representation, clustering-based pre- filtering to reduce the candidate search space, and GPT-4o-driven relationship generation via iterative prompt engineering to produce semantically rich, interpretable links. Applied to ReproduceMeON—a network of 33 ontologies spanning machine learning, microscopy, computational science, and experimental workflow—the pipeline reduces approximately 800k raw concept pairs to 95k high-quality candidates. Human expert validation of 429 generated relationships by two independent annotators yields an overall precision of 80.19% (91.49% on high-certainty annotations) and an F1 of 0.890, with substantial inter-annotator agreement. Comparative experiments against five similarity-based baselines, including Sentence-BERT, show a substantial performance gap (best baseline F1 = 0.581), while an ablation study demonstrates that similarity-based methods alone fail to discriminate valid from invalid relationships (AUC≈0.5) on the filtered candidate set. These findings highlight the necessity of LLM-based reasoning over concept roles and domain semantics for accurate relationship construction. Keywords AI-Assisted Knowledge Engineering, Knowledge Interoperability, Ontology Network, Large Language Model, Ontology Alignment 1. Introduction In modern scientific environments, interdisciplinary studies often span multiple knowledge do- mains—ranging from experimental protocols to machine learning (ML) algorithms and computational workflows. Attempting to unify all relevant knowledge within a single monolithic ontology is not only impractical but also counterproductive. Instead, ontology networks (ONs), which interlink multiple ontologies through typed semantic relationships, provide the formal backbone for such cross-domain knowledge integration [1,2]. The development of such networks relies heavily on the accurate identifi- cation of both intra-domain relationships—links between concepts from different ontologies within the same domain as well as inter-domain relationships—links connecting concepts across distinct domains. Existing approaches fall short. Classical alignment tools (LogMap [3], AML [4]) identify equivalent concepts but do not generate typed, justified relationships [2]. Embedding-based methods such as OWL2Vec* [5] and RDF2Vec [6] improve scalability but cannot discriminate between valid and invalid relationships among high-similarity candidates: as our ablation study confirms empirically, all five similarity methods—including Sentence-BERT—achieve AUC≈0.5 on the pre-filtered candidate set, demonstrating that semantic similarity alone is insufficient. LLM-based approaches [7] have explored GPT-driven ontology enrichment but without systematic pre-filtering, iterative prompt engineering, or role-based materialization. * Corresponding author. $ nouha.hayouni@uni-passau.de (N. Hayouni); sheeba.samuel@informatik.tu-chemnitz.de (S. Samuel); alsayed.algergawy@uni-jena.de (A. Algergawy) 0000-0002-7981-8504 (S. Samuel); 0000-0002-8550-4720 (A. Algergawy) © 2026 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (C BY 4.0). We present a complete end-to-end pipeline that combines domain-adapted DistilBERT [8] fine-tuning, clustering-based 90th-percentile candidate filtering, and GPT-4o [9]-driven relationship generation via structured prompt engineering. Applied to ReproduceMeON, the pipeline reduces 800k raw pairs to 95k high-quality candidates and produces typed, justified semantic links with provenance. Human expert validation of 429 generated relationships by two independent annotators yields a precision of 80.19% overall (91.49% on high-certainty annotations), with substantial inter-annotator agreement (Cohen’s휅 = 0.723, 90.5% on 274 dually annotated pairs). On a balanced 929-pair evaluation set, our pipeline achieves F1 = 0.890 and AUC-ROC = 0.927, outperforming the best similarity-based baseline (Sentence-BERT, F1 = 0.581) by 30.9 percentage points. All code, data, and the expert-annotated validation set are available in GitHub: https://github.com/fusion-jena/ReproduceMeON/tree/main/LLM_based_link_discovery. To sum up, the main contributions of this paper are: 1.End-to-end reproducible pipeline: A complete, open-source pipeline combining DistilBERT fine-tuning, 90th-percentile clustering-based candidate selection (800k→95k pairs), iterative GPT-4o prompt engineering, and role-based mapping for automated ON development. 2.Domain-aware prompt engineering: A structured, iterative prompt refinement methodol- ogy—including contextual, task-specific, chain-of-thought, and few-shot strategies—that steers GPT-4o towards precise, domain-aligned relationship generation with scientific justifications. 3.Role-based link mapping: A mapping step that converts LLM-generated typed links into workflow-oriented semantic roles (e.g., Provider, Consumer, Enabler) grounded in established Ontology Design Patterns. 4.Empirical evaluation with quantitative baselines: Dual expert validation of 429 relationships (274 with measured inter-annotator agreement, Cohen’s휅 = 0.723, substantial agreement) achieving precision 80.19% (91.49% high-certainty), F1 = 0.890, AUC-ROC = 0.927—outperforming the best similarity baseline by 30.9 p in F1—with an ablation study confirming that LLM reasoning, not similarity, drives the precision gain. Section 2 reviews related work. Section 3 details the pipeline. Section4 presents and analyses the experimental results. Section5 concludes and outlines future directions. 2. Related Work Ontology networks (ONs) have emerged as a solution to address the limitations of isolated ontologies by enabling semantic interoperability and knowledge sharing across domains [10,11,12]. Traditional approaches to ON construction have relied heavily on manual curation or rule-based alignment tech- niques, which are labor-intensive, domain-specific, and often struggle with scalability and adaptability [10,11]. These approaches typically involve schema matching, logical alignment, and ontology media- tion strategies using description logic, heuristics, or statistical methods [2,10,11,13,14]. While effective in controlled settings, such techniques are insufficient for large-scale, dynamic, or interdisciplinary knowledge integration. More recently, machine learning and embedding-based methods have been introduced to support ontology alignment and concept matching. Techniques such as RDF2Vec [6] and OWL2Vec* [5] use structural and contextual features of ontologies to generate embeddings that enable similarity-based reasoning. These models have improved the scalability of ontology alignment but are often limited in capturing deeper semantic nuances, especially when relationships require contextual or domain-specific interpretation. The advent of LLMs, such as BERT [15] and GPT [16], has opened new avenues for semantic understanding in natural language and knowledge graphs. These models excel at extracting and reasoning about relationships between concepts across unstructured and semi-structured data. Several recent studies have explored the use of LLMs for ontology development and enrichment [7], relation extraction [17], and knowledge graph construction [18]. However, their integration into the ON development process—particularly for generating both intra- and inter-domain links in structured, role-aware formats—remains underexplored. Prompt engineering has further emerged as a powerful strategy for controlling the behavior of LLMs in specialized tasks. Ambiguously phrased prompts might lead to vague answers or hallucinations [19]. Nonetheless, the application of prompt engineering for generating ontology relationships that are both semantically meaningful and structurally usable within an ON is still in its early stages. Classical alignment tools such as LogMap and AML [2] achieve high precision on equivalence map- pings but do not generate typed, justified relationships or cover the full semantic spectrum (functional, causal, temporal, etc.). Embedding-based methods like OWL2Vec* [5] learn structural representations but still require a separate classifier for relationship typing. LLM-based approaches [7] leverage GPT models for ontology tasks but without the pre-filtering, role-based materialization, and iterative prompt refinement that our pipeline provides. Our method uniquely combines all four aspects in a single, end-to-end framework validated on a real multi-domain ontology network. A quantitative comparison of similarity-based baselines against our pipeline (Precision/Recall/F1) is provided in Section 4. Our task is typed semantic link generation for ontology networks: producing new, typed, semantically grounded associations between concept classes across ontologies that do not yet have explicit links, with the goal of enriching the ON’s relational structure. The generated links represent expert-validated typed semantic associations (functional, hierarchical, causal, etc.) rather than formal OWL axioms with logical guarantees—no satisfiability checking, domain/range validation, or reasoner-based consistency checking is performed. This scope distinction explains why classical alignment systems (AML, LogMap) and embedding-based matchers are not directly comparable: they find equivalences, not functional or causal associations between non-equivalent concepts. Our work contributes a systematic, reproducible integration of four components—domain-adapted em- beddings, clustering-based pre-filtering, structured LLM prompting, and role-based materialization—into a single validated pipeline for ON link generation. While each component builds on established tech- niques, their principled combination for the specific challenge of typed, justified inter-ontology link discovery at the scale of a multi-domain ON (33 ontologies, 800k candidate pairs) has not been demon- strated in prior work. 3. Methodology Given an ontology network풩 =푂 1 ,...,푂 푛 where each푂 푖 is an OWL ontology with a set of classes 퐶 푖 , the task is to discover a set of typed, justified linksℒ = (푐 푎 ,푟,푐 푏 ,푗) | 푐 푎 ∈ 퐶 푖 , 푐 푏 ∈ 퐶 푗 , 푖 ̸= 푗 or same domain, 푟 ∈ ℛ, 푗 ∈ text, whereℛis a predefined set of typed semantic relationship categories and푗is a natural-language justification. A link(푐 푎 ,푟,푐 푏 )is considered valid if a domain expert confirms that the stated relationship푟holds between푐 푎 and푐 푏 in the context of their respective ontology domains. The task explicitly excludes equivalence mappings (handled by classical alignment tools) and focuses on functional, causal, compositional, and instrumental relationships that enrich the ON’s cross-domain expressivity. We propose a six-stage pipeline (Fig. 1) applied to ReproduceMeON [2,12]: (1) Ontology Parsing & Fil- tering—extract class-type RDF triples; (2) Contextual Embedding Generation—fine-tune DistilBERT [8] on domain-enriched sentences; (3) Similarity & Cluster Filtering—90th-percentile cosine threshold reduces 800k to 95k pairs; (4) LLM Relationship Generation—GPT-4o [9] assigns typed, justified links; (5) Role- Based Mapping—converts links to actionable semantic roles; (6) Expert Validation & Evaluation—human annotation and quantitative baselines. 3.1. Ontology Parsing & Filtering OWL ontologies are parsed with owlready2 1 from the ReproduceMeON repository 2 to ex- tract subject–predicate–object triples; a filtering step retains only class-type triples (e.g., 1 https://owlready2.readthedocs.io/en/v0.47/ 2 https://github.com/fusion-jena/ReproduceMeON Stage Tool / Config Output ReproduceMeON LLM-Based Link Discovery Pipeline Applied domains:Machine LearningMicroscopyComputational ScienceExperimental Workflow RDFlib · owlready2 33 OWL files · 4 domains 4,070 context sentences DistilBERT (fine-tuned · MLM) 25 epochs · lr=5e-5 · AdamW 768-dim dense vectors/concept KMeans · cosine similarity 90th-percentile dynamic threshold 800 k 95 k pairs (88%) GPT-4o (T=0.2 · max_tokens=400) 9 typed relationship categories Typed + justified links w/ provenance Provider · Consumer · Enabler Initiator · Requester · Processor Actionable ON links 429 sampled relationships Human expert annotation Prec 80.19% · F1=0.890 · AUC=0.927 Ontology Parsing & Filtering Contextual Embedding Generation Similarity & Cluster Filtering LLM Relationship Generation Role-Based Mapping Expert Validation & Evaluation 123456 Iterative prompt refinement Figure 1: Overview of the proposed end-to-end pipeline for discovering and generating intra- and inter-domain links for the development of ReproduceMeON ontology network. DataProcessingAlgorithm subClassOf Algorithm), removing literals and non-class data. Addi- tional context—rdfs:label, rdfs:comment, and annotations—is extracted to disambiguate domain-specific meanings (e.g., “process” in ML vs. experimental settings) and enrich subsequent embeddings. 3.2. Contextual Embedding Generation To adapt DistilBERT [8] to domain-specific knowledge, RDF triples are converted into context sentences (e.g., ‘WebServiceAlgorithm rdfs:subClassOf DataProcessingAlgorithm") enriched with corresponding rdfs:label and rdfs:comment metadata. This contextualization helps the model learn subtle semantic distinctions between concepts. The fine-tuning dataset includes subject, object, and descriptive fields, split into 80% training, 10% validation, and 10% test sets. After iterative tuning, the final configuration uses a batch size of 16, a learning rate of 5e-5, and 25 epochs. Training employs the Masked Language Modeling (MLM) objective, randomly masking 15% of tokens in each sentence to encourage contextual prediction, thereby enhancing the model’s capacity to capture subject-object semantics. To prevent overfitting and ensure generalization, we use: (i) AdamW Optimizer to decouple weight decay from gradient updates [20], (i) Early Stopping if validation loss stagnates for 3 epochs, (i) Validation Set Checks using 10% of data, and (iv) Best Model Checkpointing to retain the model with the lowest validation loss. Input sentences are tokenized using WordPiece (256 tokens, padding/truncation) [15] and processed through the fine-tuned model; mean pooling over hidden states yields Sentence Embeddings (full subject–object context) and Subject/Object Embeddings (hierarchical and contextual representations, e.g., “Classification Algorithm” as a subclass of “Algorithm”). Embeddings are generated in batches with parallel processing. KMeans clustering (cluster count selected via Elbow and Silhouette methods) [21] organizes embeddings into semantically coherent groups for downstream relationship identification. 3.3. Similarity & Cluster Filtering This step aims to compute semantic similarity scores between concepts based on their embeddings, using cluster membership to limit comparisons and ensure meaningful, efficient pair generation. Cosine similarity is calculated only within each cluster, focusing on semantically relevant ties and reducing computational overhead. Using scikit-learn’s cosine similarity function, pairwise similarities are computed within clusters. Threshold selection. The similarity threshold directly controls the precision–recall trade-off for candidate pair generation. We adopt a dynamic threshold set at the 90th percentile of within-cluster cosine similarity scores. This data-driven choice adapts to each cluster’s score distribution rather than imposing a single global cutoff: dense clusters (e.g., Cluster 1, which contained intra-domain microscopy concepts) produce tighter distributions and thus higher thresholds, while sparser clusters produce lower thresholds that still filter out noise. As a sensitivity check, a fixed threshold of 0.97 was also evaluated; it produced 30% fewer candidate pairs with no measurable gain in downstream validation precision, confirming that the percentile-based approach retains more semantically useful candidates. For each valid pair, metadata including subject and object classes, domains, ontology sources, and similarity scores is recorded. This structured approach ensures that generated pairs reflect semantically coherent relationships within their cluster context. To maintain uniqueness, duplicate pairs are removed in a final data cleaning step. 3.4. LLM Relationships Generation This step is central to the development of ReproduceMeON, automating the generation of semantic relationships between ontology concepts based on similarity scores and contextual understanding. Each concept pair—with its ontology sources, domains, and similarity score—is submitted to GPT-4o to determine whether a meaningful semantic relationship exists, considering not only cosine similarity but also concept roles, dependencies, and real-world interactions. The output is a structured dataset of typed, justified links with provenance. To ensure consistency, the model is guided by nine predefined relationship types inspired by the state of the art [22, 23]: •Hierarchical (parent–child, subtype): capture generalization and specialization relations , sup- porting taxonomy construction, inheritance, and logical reasoning. •Functional (enables, requires): describe dependencies and operational flows , enabling the modeling of workflows and reusable process components. •Causal (cause–effect): represent directional influence between entities, supporting explanation, hypothesis testing, and predictive reasoning. •Temporal (sequence-based): encode ordering and timing constraints , essential for workflows, protocols, and simulations. •Spatial (proximity, location): provide context on physical or spatial configurations, particularly relevant in domains such as microscopy, biology, and robotics. •Compositional (part–whole): model structural relationships between components, enabling reasoning over modular and nested systems. •Comparative (relative differences): express distinctions or evaluations, supporting decision- making, benchmarking, and recommendation scenarios. •Transformational (conversion, evolution): capture state or format changes, important for representing data pipelines, chemical processes, or learning stages. •Instrumental (tool–method): identify tools or agents used to perform tasks, supporting the representation of operational roles in computational and experimental workflows. Together these cover the full semantic spectrum needed for interdisciplinary ontology network construc- tion—from structural taxonomies to operational workflows and cross-domain dependencies. Using the OpenAI API, input data and instructions are sent to GPT-4o. The output includes the relationship type, source and target concepts, a justification sentence, and any cited reference. The result is a structured dataset of domain-specific, justified semantic links, enabling fine-grained mapping between ontology concepts and supporting expert validation through transparent reasoning and evidence. 3.4.1. Prompt Structure and Example Each concept pair is submitted via a structured prompt (see example in Fig 2): the system message establishes GPT-4o’s role as an ontology expert, prohibits vague relations, and mandates a scientific justification; the user message supplies both concepts, their ontology sources, domains, and cosine similarity, then specifies a JSON output schema (relationship type, source, target, justification, citation). System:You are an expert in ontology construction and knowledge graph generation. Your task is to construct a network of meaningful relationships between concepts in the ReproduceMeON context. Use your semantic understanding and domain knowledge. Follow these rules strictly: Avoid vague relationships like “related to”; ensure relationships reflect meaningful real-world dependencies; provide a justification sentence; cite a specific publication to support the justification. User: Input Analysis — Subject1: Activity (Microscopy, omeroriken.owl); Subject2: Objective (Computational, dockeronto.owl); Similarity Score: 0.87 Instructions — Define: Relationship (e.g., “enables”, “requires”), Relationship Nature (Hierarchical / Functional / Causal / Temporal / Spatial / Compositional / Comparative / Transformational / Instrumental), Source, Target, Justification Sentence, Scientific Reference. Return result in the specified JSON format. If no relationship exists, state “No Relationship” and explain why. Figure 2: Representative prompt used for GPT-4o-based relationship generation. 3.4.2. Prompt Refinement Initial prompts produced vague outputs (e.g., “is related to”); we therefore applied iterative prompt engineering combining contextual cues (domain, similarity score), task-specific constraints (allowable types, JSON schema), chain-of-thought decomposition [24], and few-shot examples, with feedback from prior outputs incorporated each iteration to reduce vagueness and improve diversity [19, 25]. Although GPT-4o is used for its instruction-following and structured-output capabilities, the pipeline is model-agnostic: instruction-tuned open-source models (LLaMA [26], Mistral) could substitute at lower cost, while models with extended context windows could reduce API overhead by batching more pairs per call. Comparing model families is planned future work; current results serve as a GPT-4o baseline. 3.5. Role-Based Mapping Role-based mapping converts abstract LLM-generated links into operational semantic roles—Enabler, Processor, Provider, Consumer, Initiator, Requester—adding actionable meaning beyond the raw re- lationship type. This design is grounded in established Ontology Design Patterns (ODPs) [27]: the role taxonomy mirrors the AgentRole and Participation patterns, where entities participate in processes under specific functional capacities. For example, mapping DataSet as a Provider and Algorithm as a Consumer in “DataSet Provides Data for Algorithm” directly instantiates the ODP Provenance pattern and makes the link usable in automated data pipeline assembly. Across ReproduceMeON’s multi-domain scope, this step ensures that inter-domain links such as “Experimental Protocol Requires Computa- tional Simulation” become SPARQL-queryable. Evaluating role assignment correctness via competency questions is planned as future work; the present contribution establishes the mapping methodology and taxonomy. 4. Experimental Evaluation In this section, we address our main research question: How can large language models and contextual embeddings be leveraged to automatically generate accurate and semantically meaningful relationships for ontology network development? In the following, we present the experimental setup, datasets, and results. 4.1. Experimental Setup Experiments run on Google Colab (NVIDIA T4 GPU). The pipeline uses the Hugging Face Transformers library for DistilBERT fine-tuning [8], PyTorch for training and embedding generation, Scikit-learn for cosine similarity and KMeans clustering, RDFlib for ontology parsing, and the OpenAI GPT-4o API [9] for relationship generation. 4.2. Datasets The datasets used in the evaluation consist of ontologies from four primary domains 3 (computational, experimental workflow, machine learning, and microscopy) [12] and alignment between ontologies [2]. Although the ReproduceMeON network also includes provenance ontologies, provenance concepts are interwoven within the experimental workflow domain and are captured through its triples; a dedicated provenance domain partition was therefore not extracted separately. The datasets span multiple domains, showcasing the diversity of reproducibility-related concepts, but also introducing challenges related to overlapping semantics and domain-specific terminologies. Ontologies from different domains are parsed and RDF triples are extracted from the ReproduceMeON. The extraction process resulted in a total of 46244 unique subjects, 71426 unique objects, and 214 unique predicates. The domain of each ontology is used to classify RDFs into several areas. Table 1 presents the distribution of RDF triples across the four domains utilized in this study. Table 1 Overview of Tuple Counts by Domain DomainCount Computational191,919 Microscopy49,246 ML18,504 Experimental12,238 Table 2 Relationship distribution from sampled pairs Rel.%Rel.% Enables22.84%Contains2.10% Requires13.75%Implements 1.86% Influences5.83%Represents1.86% Is used in5.59% Detects1.63% Is a type of4.66%Describes1.63% Utilizes4.43%Operated1.40% Generates3.50% Config.1.17% Defines2.80% Rare a 33.95% Is used by2.56% a 20+ individual relation types each contributing< 2%, including Processes, Applies, Extends, Controls, Measures, Depends On, Precedes, and others. 4.3. Results After applying the pipeline on the available datasets to determine intra- and inter-links between concepts from different ontologies, we get an output file containing comprehensive and detailed information. The figure shows that the model is able to discover an inter-link between the source concept “Activity" from the Microscopy domain and the target concept “Objective" from the Computational domain. Furthermore, the model is able to predict and label the inter-link as “enables" with the “Functional" type. Table 2 shows the distribution of relationship types across all generated links. 4.3.1. Diversity and inter/intra-domain balance Table 2 shows that Enables (22.84%) and Requires (13.75%) dominate, reflecting the dependency-heavy nature of scientific workflows, while rarer types (Implements, Configures, Describes) capture domain- specific interactions. Functional relationships dominate across all four domains (ML 84.67%, Computa- tional 77.66%, Experimental Workflow 67.44%, Microscopy 56.13%), with Instrumental links prominent in Microscopy (30.32%) and Computational (19.15%). The Experimental Workflow domain shows the broadest variety, with compositional (11.63%), comparative (9.30%), and hierarchical (9.30%) links also contributing. The pipeline produces 51.3% inter-domain and 48.7% intra-domain links, reflecting a 3 https://github.com/fusion-jena/ReproduceMeON Figure 3: Relation types across different domains balanced dual emphasis suited to both specialized domain reasoning and cross-disciplinary knowledge reuse. 4.4. Quantitative Baseline Comparison To contextualise our results, we compare the LLM pipeline against five similarity-based baselines. The comparison is specifically designed to test the claim that semantic similarity alone is insufficient for relationship validity judgement—it is not a claim about outperforming full ontology alignment systems such as AML or LogMap, which produce equivalence mappings rather than typed, justified relationships and therefore address a different task. A comparison with other LLM-based relationship generators is left for future work, as no publicly available system produces the same output format (typed, justified, role-mapped ON links). The evaluation set comprises (i) the 344 valid annotated relationships (positives), (i) the 85 invalid annotated relationships, and (i) 500 randomly generated cross-domain concept pairs as easy negatives. We acknowledge that randomly sampled negatives are easier than real-world hard negatives and thus inflate absolute F1 scores; the F1 = 0.890 figure should therefore be interpreted as performance on this constructed benchmark, not on a real-world ontology matching task. The primary evaluation metric for the actual pipeline is Precision = 80.19%—the fraction of generated relationships verified by a domain expert. Similarity baselines predict “valid” when their score exceeds the F1-optimal threshold; the LLM pipeline predicts “valid” for all 429 annotated pairs and “invalid” for the 500 random pairs. Table 3 Baseline comparison on the balanced evaluation set (929 pairs total). Threshold for similarity methods is set to the F1-optimal value. AUC: Area Under ROC Curve. MethodPrec.Rec.F1AUC Jaccard (concept names)0.3701.0000.5400.506 Jaccard (with context)0.3701.0000.5400.574 TF-IDF cosine0.5760.5520.5640.625 DistilBERT (pre-trained)0.5820.5150.5460.609 Sentence-BERT (MiniLM-L6)0.5700.5930.5810.673 LLM pipeline (ours)0.8021.0000.8900.927 The results in Table 3 show that our LLM pipeline outperforms every similarity-based baseline by a large margin. The best embedding baseline, Sentence-BERT, achieves F1 = 0.581; our pipeline achieves F1 = 0.890 (+30.9 p) and Precision = 0.802 (+23.2 p). The full comparison curves are visualised in Fig. 4. 0.00.20.40.60.81.0 False Positive Rate 0.0 0.2 0.4 0.6 0.8 1.0 True Positive Rate ROC Curve Jaccard (names) (AUC=0.506) Jaccard (context) (AUC=0.574) TF-IDF cosine (AUC=0.624) DistilBERT (pre-trained) (AUC=0.608) Sentence-BERT (MiniLM) (AUC=0.677) LLM pipeline (AUC=0.927) 0.00.20.40.60.81.0 Recall 0.0 0.2 0.4 0.6 0.8 1.0 Precision Precision-Recall Curve Jaccard (names) (AP=0.378) Jaccard (context) (AP=0.556) TF-IDF cosine (AP=0.575) DistilBERT (pre-trained) (AP=0.587) Sentence-BERT (MiniLM) (AP=0.637) LLM pipeline (Prec=0.802, Rec=1.0) Random baseline (0.37) Figure 4: ROC curves (left) and Precision-Recall curves (right) for all baselines and the LLM pipeline on the 929-pair evaluation set. 4.5. Ablation Study: Why Similarity Alone Is Insufficient All 429 annotated pairs were pre-selected by the 90th-percentile similarity filter, meaning every pair already has a high cosine similarity score. If similarity were sufficient to judge relationship validity, one would expect an AUC-ROC close to 1.0 when predicting the Validation label from the similarity score on these 429 pairs. Table 4 reports the opposite. Table 4 AUC-ROC for predicting relationship validity from similarity score alone on the 429 pre-filtered pipeline pairs. AUC≈0.5 confirms similarity cannot distinguish valid from invalid among the high-similarity candidates selected by the pipeline. MethodAUC-ROC (on 429 pairs) Jaccard (concept names)0.506 Jaccard (with context)0.485 TF-IDF cosine0.503 DistilBERT (pre-trained)0.511 Sentence-BERT (MiniLM-L6)0.500 Every method achieves AUC≈0.5 (random-level), regardless of whether surface-level or deep se- mantic embeddings are used. This demonstrates that, within the set of high-similarity concept pairs, the signal needed to discriminate valid from invalid relationships cannot be extracted from similarity scores alone. Contextual reasoning about concept roles, real-world dependencies, and domain-specific semantics—reasoning that similarity metrics do not provide—is what bridges the gap to 80.2% precision. Note that the ablation does not rule out graph-based, symbolic, or supervised alternative approaches; it establishes only that similarity scores are insufficient in this setting. Fig. 5 summarises this ablation. Furthermore, removing the embedding pre-filter makes the pipeline computationally infeasible: the initial cross-domain concept pairing yields approximately 800k candidate pairs. The DistilBERT clustering and 90th-percentile threshold reduce this to≈95k pairs (an 88% reduction), enabling the LLM step to operate on a focused, high-quality candidate set. The two components are therefore complementary: embeddings provide scalable, semantically grounded candidate selection; GPT-4o contributes typed relationship labelling, reasoning, and justification. 4.6. Human Annotation and Validation Results A sample of 429 generated relationships was independently reviewed by two domain expert annota- tors—knowledge engineers involved in the ReproduceMeON project with cross-disciplinary expertise 0.00.20.40.60.81.0 Similarity Score 0 2 4 6 8 10 Density TF-IDF cosine Invalid (0) Valid (1) Valid mean=0.185 Invalid mean=0.180 0.50.60.70.80.91.0 Similarity Score 0 2 4 6 8 10 Density DistilBERT (pre-trained) Invalid (0) Valid (1) Valid mean=0.865 Invalid mean=0.880 0.00.20.40.60.81.0 Similarity Score 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Density Sentence-BERT (MiniLM) Invalid (0) Valid (1) Valid mean=0.477 Invalid mean=0.499 TF-IDFDistilBERT (pre-trained) Sentence-BERT 0.0 0.2 0.4 0.6 0.8 1.0 Similarity Score Score Distributions: Valid vs Invalid (429 pre-filtered pipeline pairs) Valid Invalid Jaccard (names) Jaccard (context) TF-IDFDistilBERT (pre-train) Sentence- BERT 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 AUC-ROC 0.506 0.485 0.503 0.511 0.488 AUC-ROC on Pre-filtered Pipeline Pairs (429 pairs all high-similarity) Random (0.5) Key finding: Among pre-filtered high-similarity pairs, no similarity method can predict relationship validity (AUC 0.5). The LLM's semantic reasoning is essential for 80.2% precision. Ablation: Why Similarity Alone Is Insufficient Figure 5: Ablation: similarity score distributions (valid vs invalid, top row), AUC-ROC on pre-filtered pairs (bottom left), and ROC on the full evaluation set (bottom middle). All similarity methods achieve AUC≈0.5 on pre-filtered pairs, confirming that the LLM step is the critical source of precision. in ontology engineering, machine learning, microscopy, and computational science. Each annota- tor recorded (i) a binary validity judgment (1 =valid,0 =invalid) and (i) a self-reported certainty score (1 =high certainty,0 =uncertain). Annotator 1 covered all 429 relationships across all four domains. Annotator 2 independently annotated 274 of the 429 relationships across three domains (Ma- chine Learning, Computational, Experimental Workflow); the Microscopy domain (155 pairs) received single-annotator coverage only. Inter-Annotator Agreement (IAA). We compute Cohen’s휅on the 274 dually annotated pairs. Table 5 reports results per domain and overall. The overall Cohen’s휅 = 0.723(90.5% agreement) falls in the substantial agreement range (Landis & Koch, 1977:0.61–0.80). The Computational domain achieves 휅 = 0.817(almost perfect), while Machine Learning (휅 = 0.671) and Experimental Workflow (휅 = 0.720) both reach the substantial threshold. These results confirm that the annotation task—determining whether a GPT-4o-generated typed relationship is valid—is reliably reproducible by domain experts, and that Annotator 1’s judgements are representative. Table 5 Inter-annotator agreement on 274 dually annotated pairs. Cohen’s휅and percentage agreement per domain and overall. Microscopy (155 pairs) was annotated by a single expert. DomainnCohen’s 휅Agreement (%) Machine Learning1370.67187.6% Computational940.81794.7% Experimental Workflow430.72090.7% Overall2740.72390.5% To supplement the expert assessment, we applied an automated NLI-based validator (BART-large- MNLI [28]) to all 429 pairs, testing whether each GPT-4o justification textually entails the stated relationship. The validator assigned near-identical entailment scores to valid and invalid pairs (mean NLI score: 0.976 for valid vs. 0.970 for invalid), yielding Cohen’s휅 = 0.0. This non-result is itself informative—and should be interpreted cautiously: GPT-4o justifications are linguistically persuasive regardless of correctness, which constitutes a hallucination risk. A justification that sounds scientifically plausible does not guarantee a valid relationship, and downstream users who rely on justifications alone—without expert review—risk accepting incorrect links. This underscores that domain expertise is irreplaceable for validation and that the interpretability benefit of justifications is weaker than their surface fluency suggests. Overall annotation certainty for Annotator 1 was 87.6% (376/429 entries rated high certainty). Since the ontology network has no pre-existing, exhaustive ground truth of all possible cross-ontology links, we report Precision as the primary metric. Recall cannot be computed without a complete gold standard; however, the 429 validated pairs span all four domains and multiple ontology sources, providing indicative evidence of coverage. Overall: out of 429 evaluated relationships, 344 were validated as correct, yielding an overall precision of 80.19%. When restricting to the 376 high-certainty annotations, precision rises to 91.49%—though this figure should be interpreted with caution, as high-certainty pairs are by definition the less ambiguous cases, making the precision gain expected rather than independently informative. Table 6 summarises precision broken down by domain and link type. Table 6 Precision of generated relationships by domain and link type. DomainValidTotalPrecision Machine Learning10313775.18% Microscopy12615581.29% Computational809485.11% Experimental Workflow354381.40% Overall34442980.19% Link Type Intra-domain19222784.58% Inter-domain15220275.25% Intra-domain precision (84.58%) is consistently higher than inter-domain precision (75.25%), which is expected: intra-domain pairs share a common conceptual vocabulary, making semantic alignment clearer. The lower inter-domain precision highlights the greater challenge of linking concepts across heterogeneous ontology spaces and points to a concrete direction for future work, such as domain- adaptive prompting or cross-domain few-shot examples. Regarding relationship nature, Causal (100%), Comparative (100%), and Hierarchical (95.24%) re- lationships achieved the highest precision, while Instrumental relationships (65.22%) were the most error-prone—consistent with the inherently ambiguous boundary between tools and processes across domains. Only 19.81% of the generated links were marked invalid, with a higher error rate in inter- domain contexts, underscoring the need for continued improvement in cross-domain relationship generation. Fig. 6 summarises precision per domain and per relationship type. Error Analysis. Examining the 85 invalid relationships, the dominant failure mode was inter-domain linking where similar surface-level terminology masked conceptual incompatibility—most frequently when ML process concepts (e.g., “Training”, “Optimization”) were linked to microscopy procedural terms sharing lexical overlap but distinct operational semantics. For Instrumental errors specifically, the model frequently conflated tool-role (an entity used to perform a task) with process-role (an entity that is a task), particularly across the ML and Experimental Workflow domains where this boundary is ontologically thin. These patterns suggest that domain-adaptive prompting with explicit disambiguation examples for tool vs. process concepts could reduce the inter-domain error rate in future work. 4.7. Limitations Several limitations should be considered when interpreting our results. The second annotator covered 274 of 429 relationships; the Microscopy domain (155 pairs) has single-annotator coverage only. Ex- tending dual annotation to Microscopy and computing Microscopy-specific휅is planned as future work. 0.00.20.40.60.81.0 Precision Computational Experimental Workflow Machine Learning Microscopy 85.1% (80/94) 81.4% (35/43) 75.2% (103/137) 81.3% (126/155) Precision by Domain Overall (80.2%) 0.00.20.40.60.81.01.2 Precision Instrumental Compositional Functional Hierarchical Causal Comparative 65% (45/69) 80% (16/20) 82% (249/305) 95% (20/21) 100% (8/8) 100% (6/6) Precision by Relationship Type Overall (80.2%) Figure 6: Precision broken down by domain (left) and relationship type (right). Causal, Comparative, and Hierarchical relationships achieve the highest precision; Instrumental relationships are the most error-prone. The balanced evaluation set includes 500 randomly generated negatives. Random negatives are easier to reject than real hard negatives (e.g., high-similarity pairs that are genuinely non-relationships), inflating F1. The primary metric—Precision on pipeline outputs—is not subject to this limitation. All experiments are conducted on ReproduceMeON. While its four-domain, 33-ontology scope is non-trivial, evidence of generalizability to other domains (e.g., biomedical, cultural heritage) or to ontologies of different size and annotation density is lacking. Applying the pipeline to a second domain is planned as future work. The pipeline depends on GPT-4o, whose stochastic outputs mean that repeating the generation step may yield different relationship labels or justifications for the same concept pair. No repeated-run experiments were conducted to quantify output variance or prompt sensitivity; this affects reproducibility and is a known limitation of LLM-dependent pipelines. The role taxonomy (Provider, Consumer, etc.) is grounded in established ODPs but its assignment correctness has not been independently evaluated. Competency question testing is identified as future work. 5. Conclusion We presented an end-to-end pipeline for enhancing ontology network development by automatically discovering intra- and inter-domain relationships. The pipeline combines fine-tuned DistilBERT em- beddings, clustering-based candidate selection, iterative GPT-4o prompt engineering, and role-based relationship materialization. Applied to ReproduceMeON across four domains (computational, machine learning, microscopy, and experimental workflow), dual expert validation of 429 generated relationships confirms an overall precision of 80.19% (rising to 91.49% on high-certainty annotations), with an F1 of 0.890 on a balanced evaluation set. Inter-annotator agreement on 274 dually annotated pairs reaches Cohen’s휅 = 0.723(substantial agreement), confirming that the annotation task is reliably reproducible. A systematic quantitative comparison against five similarity-based baselines—including Sentence-BERT and pre-trained DistilBERT—confirms that the best embedding baseline achieves F1 = 0.581, demon- strating that our LLM pipeline outperforms all similarity-based alternatives by over 30 percentage points in F1. An ablation study further shows that among pre-filtered, high-similarity concept pairs, all similarity methods achieve AUC≈0.5, confirming that semantic similarity alone is insufficient to judge relationship validity: the LLM’s reasoning about concept roles and domain dependencies is what delivers the precision gain. The embedding pre-filtering step remains essential for computational feasibility, reducing 800k raw candidate pairs to 95k high-confidence pairs. Future work will explore alternative LLMs (e.g., open-source instruction-tuned models) for cost-effective deployment, full re- call measurement via a curated gold-standard dataset, integration of provenance domain ontologies, and extension of role-based mapping to support automated workflow generation and SPARQL-based reasoning over the enriched ontology network. Declaration on Generative AI: The text of this manuscript was improved with the following AI tools: ChatGPT and Claude. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. References [1] P. Haase, S. Rudolph, Y. Wang, S. Brockmans, D1. 1.1 networked ontology model, 2006. [2]S. Samuel, B. König-Ries, A. Algergawy, The role of ontology matching in ontology network development, in: Proceedings of the 18th International Workshop on Ontology Matching co- located with the 22nd International Semantic Web Conference, 2023. URL: https://ceur-ws.org/ Vol-3591/om2023_LTpaper2.pdf . [3] E. Jiménez-Ruiz, B. Cuenca Grau, Logmap: Logic-based and scalable ontology matching, in: International Semantic Web Conference, Springer, 2011, p. 273–288. [4]D. Faria, C. Pesquita, E. Santos, M. Palmonari, I. F. Cruz, F. M. Couto, The agreementmakerlight ontology matching system, in: OTM Confederated International Conferences" On the Move to Meaningful Internet Systems", Springer, 2013, p. 527–541. [5]J. Chen, P. Hu, E. Jimenez-Ruiz, O. M. Holter, D. Antonyrajah, I. Horrocks, Owl2vec*: Embedding of owl ontologies, Machine Learning 110 (2021) 1813–1845. [6]P. Ristoski, H. Paulheim, Rdf2vec: Rdf graph embeddings for data mining, in: The Semantic Web–ISWC 2016: 15th International Semantic Web Conference, Kobe, Japan, October 17–21, 2016, Proceedings, Part I 15, Springer, 2016, p. 498–514. [7]V. K. Kommineni, B. König-Ries, S. Samuel, From human experts to machines: An LLM supported approach to ontology and knowledge graph construction, CoRR abs/2403.08345 (2024). URL: https://doi.org/10.48550/arXiv.2403.08345. doi:10.48550/ARXIV.2403.08345. [8]V. Sanh, L. Debut, J. Chaumond, T. Wolf, Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, arXiv preprint arXiv:1910.01108 (2019). URL: https://arxiv.org/abs/1910.01108. [9]OpenAI, Gpt-4 technical report, arXiv preprint arXiv:2303.08774 (2023). URL: https://arxiv.org/ abs/2303.08774. [10]S. D. Costa, M. P. Barcellos, R. d. A. Falbo, An ontology to support knowledge management solutions for human-computer interaction design, in: Proceedings of the 20th International Conference on Knowledge Engineering and Knowledge Management, 2016, p. 243–257. [11]H. C. Gall, M. Jazayeri, C. Riva, Seon: A pyramid of ontologies for software evolution and its anal- ysis, Computing 95 (2013) 1–24. URL: https://link.springer.com/article/10.1007/s00607-012-0204-1. doi:10.1007/s00607-012-0204-1. [12]S. Samuel, A. Algergawy, B. König-Ries, Towards an ontology network for the reproducibility of scientific studies 8 (2021) 1–12. URL: https://ceur-ws.org/Vol-2969/paper64-OntoCom.pdf . [13]J. Euzenat, P. Shvaiko, Ontology Matching: State of the Art and Future Challenges, 2nd ed., Springer, 2013. URL: https://doi.org/10.1109/TKDE.2011.253. [14] C. Reyes-Peña, M. Tovar, M. Bravo, R. Motz, An ontology network for diabetes mellitus in mexico, Journal of Biomedical Semantics 12 (2021) 1–18. [15]J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, NAACL-HLT 1 (2019) 4171–4186. URL: https://arxiv.org/abs/1810. 04805. [16]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in Neural Information Processing Systems, volume 30, 2017. URL: https://arxiv.org/abs/1706.03762. [17]N. S. Babaiha, S. G. Rao, J. Klein, B. Schultz, M. Jacobs, M. Hofmann-Apitius, Rationalism in the face of gpt hypes: Benchmarking the output of large language models against human expert-curated biomedical knowledge graphs, Artificial Intelligence in the Life Sciences 5 (2024) 100095. URL: https://w.sciencedirect.com/science/article/pii/S2667318524000023. [18]V. K. Kommineni, B. König-Ries, S. Samuel, Towards the automation of knowledge graph con- struction using large language models, in: Proceedings of the 3rd International Workshop on Natural Language Processing for Knowledge Graph Creation co-located with 20th International Conference on Semantic Systems (SEMANTiCS 2024), Amsterdam, The Netherlands, Septem- ber 17, 2024, volume 3874 of CEUR Workshop Proceedings, CEUR-WS.org, 2024, p. 19–34. URL: https://ceur-ws.org/Vol-3874/paper2.pdf . [19]T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, Advances in Neural Information Processing Systems 33 (2020) 1877–1901. URL: https://splab.sdu.edu.cn/GPT3.pdf . [20]I. Loshchilov, F. Hutter, Decoupled weight decay regularization, 2019. URL: https://arxiv.org/abs/ 1711.05101. arXiv:1711.05101. [21]C. D. Manning, P. Raghavan, H. Schütze, Introduction to Information Retrieval, Cambridge Uni- versity Press, 2008. [22]N. Guarino, D. Oberle, S. Staab, What Is an Ontology?, Springer Berlin Heidelberg, Berlin, Heidelberg, 2009, p. 1–17. URL: https://doi.org/10.1007/978-3-540-92673-3_0. doi:10.1007/ 978-3-540-92673-3_0. [23]I. Niles, A. Pease, Towards a standard upper ontology, FOIS ’01, Association for Computing Machinery, New York, NY, USA, 2001, p. 2–9. doi:10.1145/505168.505170. [24]J. Wei, et al., Chain of thought prompting elicits reasoning in large language models, in: Proceedings of the Neural Information Processing Systems, 2022. URL: https://arxiv.org/abs/2201.11903. [25] A. Madaan, et al., Self-refine: Iterative refinement with self-feedback, in: Advances in Neural Information Processing Systems, 2023. [26]H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, G. Lample, LLaMA: Open and efficient foundation language models, 2023. URL: https://arxiv.org/abs/2302.13971. arXiv:2302.13971. [27]A. Gangemi, Ontology design patterns for semantic web content, in: Proceedings of the 4th International Semantic Web Conference (ISWC 2005), volume 3729 of Lecture Notes in Computer Science, Springer, 2005, p. 262–276. [28] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, L. Zettlemoyer, BART: Denoising sequence-to-sequence pre-training for natural language generation, transla- tion, and comprehension, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, 2020, p. 7871–7880.

Instant research discovery