Paper deep dive
RAG-Based Auto-Configuration for Industrial Fieldbus Devices
Aadil Gani Ganie, Saad Ezzini, Naveed Farooz Marazi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/12/2026, 2:09:55 AM
Summary
The paper introduces CLARITY, a production-oriented pipeline for automating the configuration of industrial fieldbus devices (Modbus RTU, OPC-UA, Profibus DP, CANopen). It utilizes a hybrid retrieval system combining dense (BGE-M3) and sparse (BM25) indexing with an ontology graph (ECLASS, AAS, SOSA/SSN) to retrieve relevant manual passages. A local LLM generates JSON-LD configurations, which are validated via SHACL and protected by a two-stage abstention gate to prevent unsafe deployments. The system demonstrates high accuracy and safety, with zero silent failures in benchmarks.
Entities (16)
Relation Signals (15)
CLARITY → supportsprotocol → Profibus DP
confidence 95% · automates device configuration end-to-end for Modbus RTU, OPC-UA, Profibus DP, and CANopen
CLARITY → supportsprotocol → Modbus RTU
confidence 95% · automates device configuration end-to-end for Modbus RTU, OPC-UA, Profibus DP, and CANopen
CLARITY → supportsprotocol → OPC UA
confidence 95% · automates device configuration end-to-end for Modbus RTU, OPC-UA, Profibus DP, and CANopen
CLARITY → supportsprotocol → CANopen
confidence 95% · automates device configuration end-to-end for Modbus RTU, OPC-UA, Profibus DP, and CANopen
CLARITY → usesmodel → bge-m3
confidence 92% · uses a 1024-dimensional BGE-M3 encoder
CLARITY → usesvalidationstandard → SHACL
confidence 92% · filters low-coverage configurations before SHACL validation
CLARITY → generatesformat → JSON-LD
confidence 90% · generates ontology-aligned JSON-LD configurations
CLARITY → testedondevice → Universal Robots UR5e
confidence 90% · case study commissions a physics-simulated Universal Robots UR5e robot
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Industrial device commissioning requires engineers to manually extract hundreds of protocol-specific parameters from heterogeneous PDF manuals and transcribe them into supervisory control systems, a time-intensive, error-prone workflow. This paper presents SysName, a production-oriented pipeline that automates device configuration end-to-end for Modbus RTU, OPC-UA, Profibus DP, and CANopen. It builds a hybrid dense-sparse retrieval index augmented by an ontology graph derived from ECLASS, AAS, and SOSA/SSN, using a BGE-M3 encoder with a cross-encoder reranker to surface relevant manual passages. A local LLM (T=0.1) generates ontology-aligned JSON-LD configurations via protocol-specific prompts and a four-step repair pipeline. A two-stage abstention gate, combining a reranker-score threshold and an IRI resolution ratio, blocks unsafe LLM invocations and filters low-coverage configurations before SHACL validation. On a gold set of 28 field-level queries, the hybrid retriever reaches 0.96 HitRate@10, and the reranker raises MRR@10 from 0.56 to 0.63 with perfect score separation for abstention. The generator attains field-level F1=0.87 with exact match on 9 of 12 runs. End-to-end runs on an H100 GPU complete in 2.6-6.6s per device with zero unsafe writes and zero silent failures on a five-device benchmark; every unsuccessful run is flagged by abstention or deployment verification. Component-wise evaluation localises the single systematic failure to OPC-UA generation, invisible to end-to-end metrics alone. A case study commissions a physics-simulated Universal Robots UR5e robot from unmodified vendor documentation (254-page manual, 8-page register list, 496 chunks), reaching field-level F1=1.0 over three runs with read-back and joint-consistency verification. An ablation study and comparison with five industrial-LLM systems complete the analysis.
Tags
Links
- Source: https://arxiv.org/abs/2608.08618v1
- Canonical: https://arxiv.org/abs/2608.08618v1
Trouble viewing inline? Open PDF directly →
Full Text
33,429 characters extracted from source content.
Expand or collapse full text
RAG-Based Auto-Configuration for Industrial Fieldbus Devices Aadil Gani Ganie1, Saad Ezzini1§, and Naveed Farooz Marazi2 Abstract Industrial device commissioning demands that automation engineers manually extract hundreds of protocol-specific parameters from heterogeneous PDF manuals and transcribe them into supervisory control systems, a workflow that is time-intensive and prone to transcription errors. This paper presents CLARITY , a production-oriented pipeline that automates device configuration end-to-end for Modbus RTU, OPC-UA, Profibus DP, and CANopen. CLARITY builds a hybrid dense-sparse retrieval index augmented by an ontology graph derived from ECLASS, Asset Administration Shell (AAS), and SOSA/SSN, and uses a 1024-dimensional BGE-M3 encoder with a cross-encoder reranker to surface relevant manual passages. A local LLM (T=0.1T=0.1) then generates ontology-aligned JSON-LD configurations via protocol-specific prompts and a four-step JSON repair pipeline. A two-stage abstention gate, combining a reranker-score threshold and an IRI resolution ratio, prevents unsafe LLM invocations and filters configurations with insufficient ontology coverage before SHACL validation. We evaluate every stage against a gold dataset of 28 field-level queries with evidence-grounded relevance labels: the hybrid retriever reaches 0.96 HitRate@10 and the cross-encoder raises MRR@10 from 0.56 to 0.63 while providing perfect score separation for abstention; the generator attains field-level F1=0.87F_1=0.87 with exact-match on 9 of 12 runs; and end-to-end runs on an H100 GPU complete in 2.6–6.6 s per device with zero unsafe writes and zero silent failures on this five-device proof-of-concept benchmark, every unsuccessful run being flagged by abstention or deployment verification. The component-wise evaluation localises the single systematic failure to OPC-UA JSON-LD generation, a defect invisible to end-to-end metrics alone. Beyond the synthetic benchmark, a case study commissions a physics-simulated Universal Robots UR5e collaborative robot directly from its unmodified vendor documentation (a 254-page user manual and an 8-page register list, 496 chunks), reaching field-level F1=1.0F_1=1.0 over three runs with read-back and joint-consistency verification. An ablation study and a comparison with five industrial-LLM systems complete the analysis. I Introduction The proliferation of Industrial Internet of Things (IIoT) devices in manufacturing, process control, and building automation has made integration of heterogeneous field devices – thousands of sensors, actuators, drives, and PLCs on Modbus RTU, OPC-UA, PROFIBUS DP, and CANopen – a critical bottleneck in digital transformation. Commissioning a device requires a skilled engineer to locate the manual, identify the correct register map, baud rate, station address, function codes, and PDO mappings, and transcribe these into the SCADA/DCS configuration – a process that can consume hundreds of engineering hours per cycle and up to 30% of project cost in greenfield deployments [2]. Manual transcription is also error-prone: an incorrect register address or baud rate can silence a sensor or trigger unsafe actuator behaviour. Recent advances in LLMs [3, 19, 22] and retrieval-augmented generation (RAG) [16, 10] open a route to automating this task by grounding generation in retrieved manual content rather than parametric knowledge alone. However, three challenges limit direct application in safety-critical settings: (i) LLMs hallucinate with high confidence [14], and a plausible but invalid register address can damage field hardware; (i) device descriptions must preserve semantic structure from ECLASS, the Asset Administration Shell (AAS) [11], and SOSA/SSN [13] for downstream interoperability; (i) the system must abstain rather than guess when a device is undocumented, since a silent wrong configuration is more dangerous than a refusal. This paper presents CLARITY, a modular six-stage pipeline (ingestion, hybrid retrieval, abstention gating, LLM generation, SHACL validation, protocol deployment) that addresses all three challenges and is evaluated end-to-end on a four-protocol benchmark. The contributions are: 1. Hybrid retrieval with ontology-graph boosting: dense BGE-M3 [5] embeddings, BM25, and ontology-term boosting fused via Reciprocal Rank Fusion [7] and reranked with a cross-encoder [18]. 2. Protocol-specific JSON-LD generation with automatic repair, aligning all output to ECLASS, AAS, and SOSA/SSN IRIs. 3. A two-stage, formally motivated abstention gate whose calibration we characterise empirically: retrieval scores of documented and undocumented queries separate by 8.6×8.6×, so a wide threshold band achieves perfect abstention F1F_1. 4. Four protocol adapters (Modbus RTU, OPC-UA, Profibus DP, CANopen) enforcing deterministic safety constraints before any network I/O. 5. A released component-wise evaluation harness: gold data with evidence-grounded relevance labels, and separate retriever, generator, and end-to-end evaluations with field-level F1F_1, rank metrics, latency breakdown, and a retrieval-depth (k) sensitivity study, plus ablation and security analyses. I Related Work RAG and hybrid retrieval. Lewis et al. [16] introduced RAG; Izacard and Grave [12] extended it with Fusion-in-Decoder; Gao et al. [10] survey naive, advanced, and modular RAG variants. CLARITY extends advanced RAG with ontology-graph boosting, SHACL validation, and safety-motivated abstention not addressed in prior surveys. BM25 [4] remains the standard sparse baseline; CLARITY fuses it with dense retrieval via RRF [7], adding an ontology-term boost as a domain-specific third signal. For embeddings, CLARITY uses BGE-M3 [5] (extending Sentence-BERT-style encoders [20]) and reranks with a cross-encoder [18]. Structured LLM output and abstention. Few-shot prompting [3, 19] and open-weight models such as LLaMA-2 [22], built on the Transformer [23], enable structured JSON generation but still frequently emit malformed JSON below 8B parameters, motivating CLARITY’s repair pipeline. Ji et al. [14] taxonomise hallucination as intrinsic or extrinsic; extrinsic hallucination of protocol parameters is the primary risk CLARITY targets. The abstention gate is motivated by conformal prediction [1], treating reranker score as a conformity measure, and by LLM cascade research [6] showing that routing low-confidence queries away from expensive inference reduces cost without accuracy loss. Ontologies and validation. SOSA/SSN [13] and AAS [11] provide the vocabulary and submodel structure for CLARITY’s JSON-LD [21] output; SHACL [15] provides the formal validation layer. Barnaghi et al. [2] motivate ontology-based IoT interoperability. LLMs in industrial automation. LLM4PLC [8], Agents4PLC [17], and Vendor-Aware Agents [9] apply RAG/LLMs to PLC code generation with compiler or formal-verification feedback, but none targets fieldbus device configuration, incorporates ontology-aligned SHACL-validated output, or implements a formally motivated abstention gate (Table I). CLARITY is thus the only system combining hybrid retrieval, ontology-aligned output, formal (SHACL) validation, multi-protocol deployment, and an abstention gate; the closest prior work, Vendor-Aware Agents [9], uses single-signal dense retrieval with free-form JSON and no validation or abstention. TABLE I: Comparison with Related Industrial LLM Systems System Hybrid Ontol. Formal Multi- Abst. Retr. Out. Valid. Proto. Gate LLM4PLC [8] × × Compiler × × Agents4PLC [17] × × Formal × × Vendor-Aware [9] × × × × × Adv. RAG [10] Partial × × × Partial CLARITY (this work) ✓ ✓ SHACL ✓ ✓ I System Architecture CLARITY is a six-stage pipeline: ingestion, hybrid retrieval, abstention gating, LLM generation, SHACL validation, and protocol deployment, sketched end-to-end in Fig. 1 and formalised in Algorithm 1. 1 Ingestionchunk, BGE-M3,BM25, ont. graph2 RetrievalRRF+graph boost,rerank top-53 Gatesmax≥τs_ ≥τ,ρiri≥0.80 _iri≥0.804 LLM gen.JSON-LD+ repair5 SHACLformalvalidation6 Deployadapter +read-back Figure 1: End-to-end CLARITY pipeline. A PDF manual and a natural-language query enter at Stage 1; a SHACL-valid JSON-LD configuration is deployed and verified by read-back at Stage 6, or the pipeline abstains at Stage 3 (pre-LLM) or Stage 4 (post-LLM). Algorithm 1 CLARITY End-to-End Pipeline 0: PDF manual ℳM, query q, protocol p, SHACL shapes pS_p, τ=0.72τ=0.72, ρ=0.80ρ=0.80 0: Deployed JSON-LD configuration or abstention signal 1: // Stage 1: Ingestion 2: ←chunk(ℳ,512,64)C← chunk(M,512,64); ℰ←BGE-M3()E← BGE-M3(C) 3: ←ontology_graph(ECLASS,AAS,SOSA)G← ontology\_graph(ECLASS,AAS,SOSA) 4: // Stage 2: Hybrid Retrieval 5: Rrrf←RRF(dense(q),BM25(q),krrf=60)R_rrf← RRF( dense(q), BM25(q),\,k_rrf=60); boost matches to G by +0.1+0.1 6: Rtop5←CrossEncoder.rerank(q,Rrrf,5)R_top5← CrossEncoder.rerank(q,R_rrf,5); smax←σ(maxri.score)s_ ←σ( r_i.score) 7: // Stage 3: Pre-LLM Gate 8: if smax<τs_ <τ or ρiri<1−0.20 _iri<1-0.20 then 9: return ABSTAIN 10: end if 11: // Stage 4: LLM Generation + Repair 12: y^←Ollama(π(q,p,Rtop5),T=0.1) y← Ollama(π(q,p,R_top5),T=0.1); y^json←JSONRepair(y^) y_json← JSONRepair( y) 13: // Stage 5: Post-LLM Gate 14: if uncertainty_markers(strip_json(y y)) then 15: return ABSTAIN 16: end if 17: // Stage 6: SHACL Validation + Deployment 18: if not pyshacl.validate(y^json,p y_json,S_p) then 19: return INVALID 20: end if 21: ProtocolAdapter[p].deploy(y^json y_json); 22: return y^json y_json Ingestion. PDF manuals are chunked at 512 tokens with 64-token overlap to avoid splitting register-map rows, embedded with BGE-M3 [5] into 1024-d dense vectors stored in Qdrant (cosine similarity), and indexed in parallel by BM25 for exact-term matching of register addresses and part numbers. An ontology graph G is built from ECLASS, AAS, and SOSA/SSN labels via rdflib. Hybrid retrieval. Dense (top-20) and sparse (top-20) results are fused with RRF (Eq. 1, constant k=60k=60), each chunk receiving a +0.1+0.1 boost if an ontology label appears verbatim, before cross-encoder reranking [18] to the top-5 context passed to the LLM. Writing sis_i for the raw cross-encoder relevance score of candidate chunk i and σ for the logistic sigmoid, the scalar smax=σ(maxisi)∈(0,1)s_ =σ( _is_i)∈(0,1) is the abstention statistic used in Sec. IV. RRF(d)=∑l∈dense,sparse160+rankl(d)RRF(d)= _l∈\ dense, sparse\ 160+rank_l(d) (1) Generation and repair. The LLM (llama3.1:8b via Ollama, T=0.1T=0.1, 2048 tokens) fills a protocol-specific template with q, Rtop5R_top5, and the target JSON-LD @context. A four-step repair pass (strip JS comments, remove trailing commas, normalise Python literals, convert quoting) fixes malformed output before json.loads; at least one step fired in 11 of 15 benchmark runs. Validation and deployment. Repaired JSON-LD is parsed to RDF and validated with pyshacl against protocol-specific SHACL shapes encoding mandatory properties, datatypes, and value ranges (e.g. [1,65534][1,65534] for Modbus registers). Only configurations passing SHACL reach the protocol adapter. IV Abstention Gate Let risk event ℛR be deployment of a configuration containing at least one extrinsic hallucination. The gate is a classifier h:→PROCEED,ABSTAINh:X→\PROCEED,ABSTAIN\ targeting Pr[ℛ∣h=PROCEED]<ϵ [R h=PROCEED]<ε, following conformal prediction [1] with reranker score as conformity measure. Pre-LLM gate evaluates two conjunctive criteria: (C1) smax=σ(maxisi)≥τ=0.72s_ =σ( _is_i)≥τ=0.72 (Eq. 2); (C2) IRI resolution ratio ρiri=Nresolved/Niri≥0.80 _iri=N_resolved/N_iri≥ 0.80, where NiriN_iri is the number of ontology IRIs the retrieved context references and NresolvedN_resolved the number resolving against the loaded ECLASS/AAS/SOSA vocabularies. C1 catches undocumented devices or vague queries; C2 catches passages using nonstandard terminology the LLM would have to hallucinate IRIs for. smax=σ(maxisi)≥τ=0.72s_ =σ\! ( _is_i )≥τ=0.72 (2) Post-LLM gate scans only the prose portion of the LLM response (outside the JSON block) for uncertainty markers (“not specified”, “unknown”, “I do not know”, etc.); scanning the full string caused false positives on legitimate field values such as "firmware":"not specified". On the benchmark, the undocumented “Legacy XYZ” device stays far below threshold, so the gate abstains in under 1 s without invoking the LLM, while documented device-level queries reach smax≥0.99s_ ≥ 0.99; Section VI calibrates the threshold empirically over 33 gold queries. Top-5 chunks, smaxs_ smax≥0.72s_ \!≥\!0.72? ABSTAINscore low ρiri≥0.80 _iri\!≥\!0.80? ABSTAINIRI gapLLM gen. + repair Uncertainty marker? ABSTAINuncertainSHACL + deployyesnoyesnonoyes Figure 2: Two-stage abstention gate. The pre-LLM gate (top two diamonds) fires before inference; the post-LLM gate inspects only the prose portion of the LLM response. V Protocol Adapters Each of the four adapters enforces a deterministic safety layer before any network I/O, deploys validated JSON-LD against a live protocol simulator (pymodbus on :5020, asyncua on :4840) or structural simulation for Profibus/CANopen, and verifies every written value by read-back. Hard constraints include Modbus register addresses in [1,65534][1,65534] with function codes in 1,2,3,4,5,6,15,16\1,2,3,4,5,6,15,16\ (0 and 65535 permanently blocked), OPC-UA Node-ID syntax and datatype checks, Profibus station addresses in [0,125][0,125], and CANopen node IDs in [1,127][1,127] with PDO format and range checks. VI Evaluation Industrial adoption requires component-wise, reproducible evidence, not a single end-to-end demonstration. We therefore evaluate the retriever, the generator, and the full pipeline separately against gold data, and release the harness with the code. VI-A Methodology and Gold Dataset The corpus is produced by the pipeline’s own chunker (512-token windows, 64 overlap) over the four benchmark manuals plus a parsed 8-page PDF: 71 chunks, with deliberate cross-device confusability (three devices specify baud/bit rates, two expose node IDs). The gold set has 28 field-level queries (7 per device) and 5 out-of-corpus queries expected to abstain. Relevance labels are evidence-grounded: a chunk is relevant iff it contains the exact manual statement of the gold value, making labels reproducible rather than judgement-based. Each device carries a gold configuration (3–6 mandatory fields) for generator scoring. Retrieval runs on a CPU (Xeon) node; generation uses llama3.1:8b on Ollama on one NVIDIA H100, three runs per device at T=0.1T=0.1. VI-B Retriever Evaluation Table I compares the four retrieval configurations. Hybrid RRF fusion achieves the best coverage (HitRate@10 0.9640.964 vs. 0.9290.929 for either signal alone), confirming that dense and sparse retrieval fail on different queries: BM25 wins exact identifiers (P0918, 0x18A) while dense embeddings win paraphrases. The cross-encoder improves early precision – MRR@10 rises from 0.5570.557 to 0.6250.625, nDCG@10 from 0.6340.634 to 0.6790.679, and the mean first-relevant rank drops from 3.503.50 to 3.183.18 (rank 27 to rank 9 in the extreme case). Reranking is not uniformly positive (one query’s chunk left the top-5, HitRate@5 0.893→0.8570.893\!→\!0.857), but its decisive benefit is calibration: raw RRF and BM25 scores are unbounded and corpus-dependent, whereas the sigmoid cross-encoder score supports the fixed abstention threshold evaluated next. Retrieval is cheap: embedding 59 ms, search <1<1 ms, reranking 20 candidates 321 ms on CPU. TABLE I: Retriever Evaluation (28 gold queries, 71 chunks; MRR/nDCG at cutoff 10) Variant HR@1 HR@5 HR@10 MRR nDCG Dense (BGE-M3) 0.357 0.857 0.929 0.524 0.561 Sparse (BM25) 0.429 0.857 0.929 0.623 0.647 Hybrid (RRF+graph) 0.321 0.893 0.964 0.557 0.634 + Cross-encoder 0.429 0.857 0.964 0.625 0.679 VI-C Abstention-Gate Calibration On the gold set, every documented query reaches smax≥0.526s_ ≥ 0.526 and every out-of-corpus query stays at smax≤0.061s_ ≤ 0.061 – an 8.6×8.6× separation with no overlap, so any threshold τ∈(0.061,0.526)τ∈(0.061,0.526) attains abstention precision == recall =1.0=1.0. The production threshold τ=0.72τ=0.72 sits above this band: on hard field-level queries it falsely abstains once (precision 0.8330.833, recall 1.01.0), which errs in the safe direction, and on the device-level commissioning queries used by the pipeline (smax≥0.99s_ ≥ 0.99) it causes no false abstention, giving operators an empirical basis for tuning τ. VI-D Generator Evaluation Table I scores generated JSON-LD against gold fields (a field is correct iff its key appears anywhere in the output with the normalised gold value). Micro-averaged over 12 runs: precision 0.8300.830, recall 0.9070.907, F1F_1 0.8670.867, exact-match 9/12. Modbus, Profibus, and CANopen are perfect in all runs; OPC-UA fails in all three. The component-wise design localises the fault precisely: the prompt demonstrably contained the required node identifiers (five occurrences of ns=2;i=1001), yet llama3.1:8b emitted only the server endpoint and omitted the mandated clarity:nodes array – a model limitation on the most deeply nested schema, not a retrieval failure. The OPC-UA SHACL shape does not yet mandate a non-empty node array (a gap this evaluation exposed), so the incomplete configuration was caught one layer later by deployment verification. No incorrect value was ever written to a device. TABLE I: Generator Field-Level Scores (3 runs/device) Protocol Prec. Rec. F1F_1 Exact match Modbus RTU 1.000 1.000 1.000 3/3 OPC-UA 0.000 0.000 0.000 0/3 Profibus DP 1.000 1.000 1.000 3/3 CANopen 1.000 1.000 1.000 3/3 Micro avg. 0.830 0.907 0.867 9/12 VI-E End-to-End Results Table IV reports full-pipeline outcomes over 15 runs. All Modbus, Profibus, and CANopen deployments succeed except one VFD300 run in which the post-LLM gate falsely abstained on an uncertainty marker; OPC-UA never deploys, for the generator reason above; the undocumented device abstains in 0.5 s (pre-LLM gate, no inference cost). Crucially for industrial trust, failure detection is 100%: every unsuccessful run terminated in an explicit abstention or a deployment-verification error, never a silently wrong configuration, and zero safety-constraint violations occurred. All configurations passed SHACL (validity rate 1.0); end-to-end abstention precision/recall is 0.75/1.00.75/1.0. TABLE IV: End-to-End Results on H100 (3 runs/device) Device Protocol Deploy Abstain Lat. (s) TH200 Temp. Modbus RTU 3/3 0/3 6.5 PS500 Press. OPC-UA 0/3† 0/3 3.6 VFD300 Drive Profibus DP 2/3 1/3‡ 3.4 ENC100 Enc. CANopen 3/3 0/3 3.1 Legacy XYZ Modbus RTU 0/3 3/3 (correct) 0.5 †Incomplete generation detected by deployment verification (Sec. VI-D). ‡False abstention by the post-LLM uncertainty-marker gate. VI-F Latency and Retrieval Depth Mean per-stage latency on the H100: query embedding 248 ms, dense ++ BM25 search <2<2 ms, reranking 449 ms, LLM generation 4.69 s (≈ 87%), SHACL 10 ms, deployment 29 ms – documented devices complete in 2.6–6.6 s, versus 35–93 min on CPU-only inference (0.8 vs. ∼120 120 tokens/s), a two-to-three-orders-of-magnitude reduction that makes interactive commissioning practical. Because the LLM dominates, prompt size (retrieval depth k) is the main lever: sweeping k∈1,3,5,10k∈\1,3,5,10\ on TH200, latency triples (2.42.4 to 7.37.3 s) while field F1F_1 is non-monotonic – at k=3k=3 a distractor chunk displaced a needed one and cost a field (F1F_1 0.6670.667), whereas k=5k=5 holds the evidence in a ∼ 1.4k-token prompt at F1F_1 1.01.0. Passing many low-ranked chunks is thus both slower and riskier; the pre-LLM gate adds the complementary saving of 0.5 s per undocumented query instead of a full round-trip. VI-G Ablation Table V isolates each component by disabling it while holding the rest fixed (one run per configuration on the CPU host, in a session where all four documented devices deployed). Removing BM25 fails VFD300, whose station-address field is retrievable only by exact-term matching; removing dense retrieval fails two devices where BM25 cannot bridge the semantic query manual gap – both consistent with the per-signal results in Table I. Removing the pre-LLM gate is a cost failure: the undocumented device consumes a full LLM round-trip before rejection (73 min on the CPU host; ∼7 7 s on the H100) instead of abstaining in under a second. Removing the four-step JSON repair drops deployment from 4/4 to 1/4, confirming that sub-8B models often emit malformed JSON that a lightweight deterministic repair recovers without a second LLM call. Removing SHACL still deploys here but forfeits formal verification: a reserved register address of 0 would pass undetected to the field, precisely the silent error the system exists to prevent. TABLE V: Ablation: Deployment Success (Single CPU-Host Run/Config.) Configuration Dep. Failure mode when removed Full CLARITY 4/4 — −- BM25 (sparse) 3/4 Exact-ID field unretrievable −- Dense retrieval 2/4 Semantic query–manual gap −- Pre-LLM gate 4/4 +73 min wasted on undocumented −- JSON repair 1/4 Malformed JSON unparsed −- SHACL validation 4/4 No formal constraint guarantee VI-H Case Study: Unmodified Vendor Documentation To probe generalisation beyond the synthetic benchmark, we ran the unchanged pipeline on a Universal Robots UR5e collaborative robot. Its knowledge base is the real, unaltered vendor documentation – a 254-page user manual and an 8-page Modbus register list (496 chunks total, no manual re-formatting) – and its actuation is a MuJoCo physics simulation exposed through the robot’s documented Modbus TCP server, so every deployed write and read-back is exercised against a moving robot. From the natural-language request “initialise the PLC data-exchange handshake: write 0,0,0,1 to holding registers 128–131 using function code 6 (UINT16)”, CLARITY generated and deployed the correct configuration in all three runs (field-level F1=1.0F_1=1.0, exact match) in 4.2 and 4.7 s per warm run (14.0 s including one-off model load). A sentinel preload (60000) confirmed each write originated from CLARITY; post-deployment read-back returned the gold values (128:0,129:0,130:0,131:1128:0,129:0,130:0,131:1); the robot-mode register (258) read 7 (Running); and across 18 joint-angle comparisons the milliradian-encoded joint registers matched the simulator ground truth within tolerance, values changing between reads taken 1 s apart (liveness). No SHACL or safety-constraint violation occurred. The real manual also stresses retrieval and calibration. Hybrid HitRate@10 falls to 0.400.40 on the layout-heavy 254-page manual, and the documented/undocumented smaxs_ separation narrows to [0.070,0.156][0.070,0.156], so the production τ=0.72τ=0.72 would falsely abstain on five valid field queries; recalibrating τ to the separation midpoint (0.1130.113) restores correct decisions on all 12 gold queries. This is direct evidence for the per-corpus threshold tuning the sweep prescribes, and for the layout-aware ingestion larger brownfield catalogues will need. VII Security Analysis Industrial RAG systems face manual-content prompt injection: adversarial text in ingested chunks that attempts to steer the LLM into unsafe output. For an attacker with write access to one or more indexed chunks, we consider three vectors – A1 direct instruction override; A2 plausible-but-incorrect parameter values planted for the LLM to copy; A3 retrieval-score manipulation via keyword stuffing. CLARITY layers defences against all three: the SHACL and protocol-adapter constraints reject safety-violating output regardless of LLM instruction-following (A1); the reranker penalises incoherent stuffed text and SHACL bounds catch out-of-range values (A2); the bounded +0.1+0.1 graph boost cannot dominate reranker relevance (A3). The main residual risk is a plausible in-range value that both reranks highly and evades SHACL shapes – to be mitigated by document signing and chunk-level provenance attestation; CLARITY already logs a provenance_trail (document, page, confidence) per chunk for IEC 62443 auditability. VIII Discussion and Conclusion The component-wise evaluation surfaced two concrete defects that end-to-end metrics alone would have conflated: a llama3.1:8b generation weakness on the deeply nested OPC-UA schema (fixable by schema-constrained decoding or few-shot node examples) and an OPC-UA SHACL shape that must additionally mandate a non-empty node array. The core benchmark remains synthetic and small; the UR5e case study (Sec. VI-H) is a first step onto real vendor documentation, but scaling to full brownfield catalogues with multi-column tables and scanned pages will require stronger document-layout extraction (e.g. LayoutLM [24]) and a larger gold set. The threshold sweep gives an empirical band for τ, and the UR5e run shows it must be recalibrated per corpus; a conformal calibration on a held-out set would add a formal 1−α1-α coverage guarantee [1]. Two deployment concerns remain open. When multiple manual revisions disagree, CLARITY currently ranks purely by relevance; the logged provenance_trail (document, page, confidence) already carries the metadata needed to add a revision- or reliability-weighted prior to RRF, so a newer or vendor-authoritative document outranks a stale one. Extension is additive: new devices, manuals, or ontology revisions are onboarded by re-chunking and re-embedding the affected documents and reloading the ECLASS/AAS/SOSA graph, with no retraining, so post-deployment maintenance is an ingestion operation rather than a model update. Future work includes quantisation benchmarking, brownfield evaluation, additional protocol adapters (HART 7, EtherNet/IP, IO-Link), and AAS Technical Data submodel integration. This paper presented CLARITY, a production-oriented pipeline combining hybrid ontology-boosted retrieval, LLM generation with automatic JSON repair, a two-stage abstention gate, SHACL validation, and multi-protocol deployment adapters, evaluated component-wise against gold data. The hybrid retriever reaches HitRate@10 0.9640.964 (MRR@10 0.6250.625 with reranking); the generator attains field-level F1=0.867F_1=0.867 (perfect on three of four protocols); and the full pipeline commissions documented devices in 2.6–6.6 s on a single H100 with zero unsafe writes and 100% failure detection across all 15 benchmark runs, and commissions a UR5e from unmodified vendor documentation. By grounding generation in retrieved evidence, abstaining when that evidence is insufficient, and verifying every deployment, CLARITY provides the transparent, reproducible safety case that industrial adoption demands. References [1] A. N. Angelopoulos and S. Bates (2023) Conformal prediction: a gentle introduction. Foundations and Trends in Machine Learning 16 (4), p. 494–591. Cited by: §I, §IV, §VIII. [2] P. Barnaghi, W. Wang, C. Henson, and K. Taylor (2012) Semantics for the Internet of Things: early progress and back to the future. International Journal on Semantic Web and Information Systems 8 (1), p. 1–21. External Links: Document Cited by: §I, §I. [3] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020) Language models are few-shot learners. In Advances in Neural Information Processing Systems, Vol. 33, p. 1877–1901. Cited by: §I, §I. [4] D. Chen, A. Fisch, J. Weston, and A. Bordes (2017) Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1870–1879. External Links: Document Cited by: §I. [5] J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu (2024) BGE M3-Embedding: multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216. Cited by: item 1, §I, §I. [6] L. Chen, M. Zaharia, and J. Zou (2023) FrugalGPT: how to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176. Cited by: §I. [7] G. V. Cormack, C. L. A. Clarke, and S. Buettcher (2009) Reciprocal rank fusion outperforms Condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 758–759. External Links: Document Cited by: item 1, §I. [8] M. Fakih, R. Dharmaji, Y. Moghaddas, G. Q. Araya, O. Ogundare, and M. A. A. Faruque (2024) LLM4PLC: harnessing large language models for verifiable programming of PLCs in industrial control systems. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), Cited by: TABLE I, §I. [9] S. Feld et al. (2025) Vendor-aware industrial agents: RAG-enhanced LLMs for secure on-premise PLC code generation. arXiv preprint arXiv:2511.09122. Cited by: TABLE I, §I. [10] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, and H. Wang (2023) Retrieval-augmented generation for large language models: a survey. arXiv preprint arXiv:2312.10997. Cited by: §I, TABLE I, §I. [11] IEC (2023) IEC 63278-1:2023, asset administration shell for industrial applications, part 1: asset administration shell structure. International Standard International Electrotechnical Commission. Cited by: §I, §I. [12] G. Izacard and E. Grave (2021) Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, p. 874–880. External Links: Document Cited by: §I. [13] K. Janowicz, A. Haller, S. J. D. Cox, D. L. Phuoc, and M. Lefrançois (2019) SOSA: a lightweight ontology for sensors, observations, samples, and actuators. Journal of Web Semantics 56, p. 1–10. External Links: Document Cited by: §I, §I. [14] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. Bang, A. Madotto, and P. Fung (2023) Survey of hallucination in natural language generation. ACM Computing Surveys 55 (12), p. 248:1–248:38. External Links: Document Cited by: §I, §I. [15] H. Knublauch and D. Kontokostas (2017) Shapes constraint language (SHACL). W3C Recommendation World Wide Web Consortium (W3C). Note: Available: https://w.w3.org/TR/shacl/ Cited by: §I. [16] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela (2020) Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems, Vol. 33, p. 9459–9474. Cited by: §I, §I. [17] Z. Liu et al. (2024) Agents4PLC: automating closed-loop PLC code generation and verification in industrial control systems using LLM-based agents. arXiv preprint arXiv:2410.14209. Cited by: TABLE I, §I. [18] R. Nogueira and K. Cho (2019) Passage re-ranking with BERT. arXiv preprint arXiv:1901.04085. Cited by: item 1, §I, §I. [19] OpenAI (2023) GPT-4 technical report. arXiv preprint arXiv:2303.08774. Cited by: §I, §I. [20] N. Reimers and I. Gurevych (2019) Sentence-BERT: sentence embeddings using siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), p. 3982–3992. External Links: Document Cited by: §I. [21] M. Sporny, D. Longley, G. Kellogg, M. Lanthaler, and N. Lindström (2014) JSON-LD 1.0: a JSON-based serialization for linked data. W3C Recommendation World Wide Web Consortium (W3C). Note: Available: https://w.w3.org/TR/json-ld/ Cited by: §I. [22] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. (2023) Llama 2: open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. Cited by: §I, §I. [23] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30, p. 5998–6008. Cited by: §I. [24] Y. Xu, M. Li, L. Cui, S. Huang, F. Wei, and M. Zhou (2020) LayoutLM: pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, p. 1192–1200. External Links: Document Cited by: §VIII.