Paper deep dive
Multi-tenant Kubernetes Use Cases for AI, Secure Computing and Data Services, and More
Jake Watson, Sadaf R Alam, Christopher Woods, Abdelwahab Kawafi, Thomas Green, Ian Johnson, Ellis Pires, Jessica R. Jones, Utz-Uwe Haus
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/4/2026, 4:59:30 AM
Summary
This paper evaluates the deployment of Kubernetes-as-a-Service on the HPE Cray EX supercomputer Isambard-AI to support multi-tenant AI, secure computing, and data services. It addresses the limitations of traditional batch schedulers like Slurm by integrating Kubernetes with the HPE Slingshot interconnect for network isolation. The study presents two primary use cases: a Trusted Research Environment (TRE) via the FRIDGE project for sensitive medical data, and a distributed AI model hosting service using KubeRay, Ray, and vLLM for confidential computing. The authors detail the technical adaptations required for bare-metal Kubernetes, including CXI driver modifications for VNI-based isolation, and discuss challenges in bridging HPC and cloud-native ecosystems.
Entities (14)
Relation Signals (9)
Isambard-AI â uses â HPE Slingshot
confidence 96% · HPE Cray EX supercomputer with HPE Slingshot interconnect, called Isambard-AI
Isambard-AI â runs â Kubernetes
confidence 95% · We evaluate Kubernetes deployment on a Hewlett Packard Enterprise (HPE) Cray EX supercomputer... called Isambard-AI
KubeRay â orchestrates â Ray
confidence 94% · combines KubeRay, Ray, and vLLM... KubeRay operator... integrates with Kubernetes
Ray â runs â vLLM
confidence 93% · vLLM... integrates natively with Ray and KubeRay
Slurm â complements â Kubernetes
confidence 92% · Kubernetes-based services... are intended to complement, not replace, the Slurm batch scheduler
FRIDGE â implements â Trusted Research Environment
confidence 92% · FRIDGE... a Kubernetes-based, Standard Architecture for Trusted Research Environments (SATRE)-compliant
HPE Slingshot â provides â RDMA
confidence 90% · Remote Direct Memory Access (RDMA) is supported via character devices... HPE Slingshot is a high-speed network fabric
FRIDGE â runson â Isambard-AI
confidence 90% · Deploying FRIDGE on IS-AI would expose its massive-scale GPU compute resources
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Kubernetes, as a container orchestration engine, has been widely used in cloud-native ecosystems for several years. In supercomputing ecosystems, especially where bare-metal performance for compute and network devices are considered, the adoption is somewhat limited. However, with the increasing diversity of use cases such as AI, secure and confidential computing for sensitive data, and mixed workload orchestration, a traditional, single-tenant batch computing system does not offer the flexibility and reproducibility to which public cloud users are accustomed. Note that Kubernetes is not considered a replacement for batch scheduling systems, which have powerful features for large-scale MPI jobs with thousands of network end points. Rather, it is a complementary service provided as part of a national AI Research Resource. We evaluate Kubernetes deployment on a Hewlett Packard Enterprise (HPE) Cray EX supercomputerwith HPE Slingshot interconnect, called Isambard-AI, with co-design use cases. One is a Trusted Research Environment used for medical and health sciences. The other combines KubeRay, Ray, and vLLM to provide a distributed, sandboxed, persistent AI model hosting service targeting multi-tenant confidential computing. We discuss challenges and lessons learned, and where further development is needed to offer a production Kubernetes-as-a-Service on HPE Cray EX (and later) platforms.
Tags
Links
- Source: https://arxiv.org/abs/2608.00742v1
- Canonical: https://arxiv.org/abs/2608.00742v1
Trouble viewing inline? Open PDF directly â
Full Text
63,755 characters extracted from source content.
Expand or collapse full text
Multi-tenant Kubernetes Use Cases for AI, Secure Computing and Data Services, and More Jake Watson Bristol Centre for Supercomputing University of BristolBristolUnited Kingdom jake.watson@bristol.ac.uk , Sadaf R. Alam Bristol Centre for Supercomputing University of BristolBristolUnited Kingdom sadaf.alam@bristol.ac.uk 0000-0002-2534-5078 , Christopher Woods Bristol Centre for Supercomputing University of BristolBristolUnited Kingdom christopher.woods@bristol.ac.uk 0000-0001-6563-9903 , Abdelwahab Kawafi Bristol Centre for Supercomputing University of BristolBristolUnited Kingdom a.kawafi@bristol.ac.uk 0000-0002-1369-6698 , Thomas Green Bristol Centre for Supercomputing University of BristolBristolUnited Kingdom thomas.green@bristol.ac.uk 0009-0001-8622-3664 , Ian Johnson HPE HPC/AI EMEA Research Lab Hewlett Packard EnterpriseBristolUnited Kingdom ian.johnson@hpe.com , Ellis Pires HPE HPC/AI EMEA Research Lab Hewlett Packard EnterpriseBristolUnited Kingdom ellis.pires@hpe.com 0000-0002-6531-8269 , Jessica R. Jones HPE HPC/AI EMEA Research Lab Hewlett Packard EnterpriseBristolUnited Kingdom j.r.jones@hpe.com 0009-0003-7579-7245 and Utz-Uwe Haus HPE HPC/AI EMEA Research Lab Hewlett Packard EnterpriseZĂŒrichSwitzerland utz-uwe.haus@hpe.com 0000-0001-7292-9984 (2026) Abstract. Kubernetes, as a container orchestration engine, has been widely used in cloud-native ecosystems for several years. In supercomputing ecosystems, especially where bare-metal performance for compute and network devices are considered, the adoption is somewhat limited. However, with the increasing diversity of use cases such as AI, secure and confidential computing for sensitive data, and mixed workload orchestration, a traditional, single-tenant batch computing system does not offer the flexibility and reproducibility to which public cloud users are accustomed. Note that Kubernetes is not considered a replacement for batch scheduling systems, which have powerful features for large-scale MPI jobs with thousands of network end points. Rather, it is a complementary service provided as part of a national AI Research Resource. We evaluate Kubernetes deployment on a Hewlett Packard Enterprise (HPE) Cray EX supercomputer with HPE Slingshot interconnect, called Isambard-AI, with co-design use cases. One is a Trusted Research Environment used for medical and health sciences. The other combines KubeRay, Ray, and vLLM to provide a distributed, sandboxed, persistent AI model hosting service targeting multi-tenant confidential computing. We discuss challenges and lessons learned, and where further development is needed to offer a production Kubernetes-as-a-Service on HPE Cray EX (and later) platforms. Kubernetes, Multi-tenancy, Slingshot, RDMA, Trusted Research Environments, Confidential Computing, Distributed LLMs â copyright: acmlicensedâ journalyear: 2026â doi: 10.1145/3837730.3837733â conference: Cray User Group; April 26-30, 2026; Nice, Franceâ ccs: Computing methodologies Distributed computing methodologiesâ ccs: Security and privacy Distributed systems securityâ ccs: Security and privacy Network security 1. Introduction Traditionally, High-Performance Computing (HPC) systems have been designed for running tightly-coupled, large-scale batch workloads. There is now an increasing demand from end-users that HPC systems support increasingly diverse workloads including AI model training and inference, secure AI system sandboxing, and analysis of sensitive data with strict governance requirements. Batch schedulers like Slurm are incredibly powerful for bulk workloads but they often lack the flexibility, service abstractions, and multi-tenant isolation that is needed to address these advanced use cases. The integration of cloud-native technologies into HPC environments presents a solution to address these use cases but carries a new set of challenges. Kubernetes is the pre-eminent technology for multi-tenant workload orchestration in cloud environments. The Kubernetes ecosystem is mature and feature-rich, offering tools for autoscaling, load balancing, namespace isolation, identity federation, software-defined networking, and infrastructure-as-code. The integration of Kubernetes onto bare-metal HPC systems, particularly those with advanced high-speed fabrics, is non-trivial and requires modifications that are not well-supported in the community. We document the design, deployment, and evaluation of Kubernetes-as-a-Service on Isambard-AI (IS-AI), the UKâs national sovereign AI Research Resource (AIRR), which comprises an HPE Cray EX framework with the Slingshot interconnect and NVIDIA Grace (ARM64) Hopper superchips. The Kubernetes-based services presented here on IS-AI are intended to complement, not replace, the Slurm batch scheduler; Slurm is still expected to schedule the largest tenants and partitions, while Kubernetes services target niche and ever-evolving AI and sensitive data handling use cases. The increasing demand of Slurm and Kubernetes represents a growing effort to bridge traditional HPC environments with modern cloud-native workloads, though significant technical and organisational challenges remain. Projects like Slinky [SchedMD, 2026a], developed by SchedMD and NVIDIA, aim to natively integrate Kubernetes with Slurm, allowing containerized workloads to be scheduled alongside traditional batch jobs; this approach introduces complexity in maintaining consistent resource accounting across two distinct scheduling domains. Similarly, the Slurm Operator project [SchedMD, 2026b] provides a Kubernetes operator that can submit and manage Slurm jobs from within a Kubernetes cluster, yet it struggles with latency overhead and the impedance mismatch between Kubernetesâ declarative, eventually-consistent model, and Slurmâs tightly controlled, synchronous job lifecycle. The Cloud Native Computing Foundation (CNCF) Batch System Initiative Working Group [CNCF, 2026] has worked to standardise batch scheduling interfaces, but achieving consensus across diverse HPC and cloud-native stakeholders has proven slow and difficult. Tools like Volcano [Volcano, 2026] and Armada [Armada, 2026] extend Kubernetes with HPC-style scheduling primitives, such as gang scheduling and queue management, but often lack the maturity and production-hardening that Slurm has accumulated over decades. A deeper challenge across all these approaches is the fundamental tension between Kubernetesâ dynamic, ephemeral resource model; and Slurmâs assumption of stable, dedicated hardware; making seamless interoperability difficult to achieve in practice, particularly for MPI-based workloads, high-speed interconnects like SlingShot, and bare-metal GPU access upon which AI, ML, and HPC applications critically depend. We therefore implement the multi-tenancy Slingshot network isolation solution from Friese et al. [Friese et al., 2025] on IS-AI [McIntosh-Smith et al., 2024]. We build upon this adaptation to deploy Kubernetes-as-a-Service to address two use cases. First, Federated Research Infrastructure by Data Governance Extension (FRIDGE) [Watson, 2025]: a Kubernetes-based, Standard Architecture for Trusted Research Environments (SATRE)-compliant [SATRE, ] âsatelliteâ Trusted Research Environment (TRE) for researchers analysing data with strict governance requirements. Subsequently, a combination of KubeRay, Ray [Moritz et al., 2018], and vLLM [et al., 2023] is used to serve a distributed, sandboxed, persistent, AI model with dynamic resource allocation to accommodate the UK AI Security Institute (AISI) user base and the emerging requirement to isolate model execution environments for both security and reproducibility concerns. We evaluate the suitability of this deployment for multi-tenant computing at scale. The implementation and evaluation of the use cases highlight several areas of improvement that we have identified for further investigation and development. These include reducing the operational complexity by automating and integrating independent components that are required for a multi-tenant Kubernetes implementation. A sustainable solution may require a CXI Kubernetes Dynamic Resource Allocation (DRA) driver [Kubernetes, 2026a] that could be maintained as a community. In short, the work presented in this paper provides a solid baseline for hardening the solution for additional Kubernetes use cases for operational environments on existing EX and future GX infrastructures, based on the Slingshot interconnect, which are expected to maintain compatibility with open standards. The remainder of this paper is structured as follows. Section 2 provides background on the three areas of this work; bare-metal Kubernetes and multi-tenancy, TREs, and secure and multi-node model hosting. Section 3 details the implementation of each of these and the specific adaptations required for IS-AI. Section 4 presents results and evaluations for each area, including benchmarking, assessment of the TRE deployment, and analysis of secure model hosting configurations. Finally, Section 5 summarises the contributions of this work and discusses directions for ongoing and future research and development. 2. Background 2.1. Bare-metal Kubernetes and Multi-tenancy A challenge with bare-metal Kubernetes (K8s) deployments is that they require manual integrations for storage, networking, and load balancing; unlike cloud distributions which benefit from provider-managed integrations. Distributions like K3s and Rancher Kubernetes Engine 2 (RKE2) [Rancher, 2026] address this by offering a ânear-out-of-the-boxâ experience. However, HPC environments still require site-specific adaptations including: support for immutable operating systems, provisioning load balancer IPs, and integrating high-speed interconnects like Slingshot. We outline the adaptations required to deploy FRIDGE efficiently and securely in IS-AI, using a portable, multi-platform architecture. HPE Slingshot is a high-speed network fabric that fundamentally comprises Cassini ASIC NICs and Rosetta switches. Ethernet communication is supported through netdev Linux devices and Remote Direct Memory Access (RDMA) is supported via character devices. The RDMA plane bypasses the kernel network and does not adhere to standard OS-level network isolation mechanisms. Instead, it uses a separate access control model that relies on the Virtual Network Identifier (VNI) domains and the Cray eXascale Interconnect (CXI) services. VNIs are integer labels that define isolated communication domains on the fabric, analogous to VLANs. When enforcement is enabled, Rosetta switches only route packets tagged with a given VNI to a destination if it belongs to the same VNI domain. The CXI services are configured on each node by a privileged service, such as the Slurm Workload Manager (WLM) plugin, and define which principals are allowed to use the RDMA interface and which VNI s they are allowed to operate on. When an application attempts to open an RDMA endpoint through libfabric, for example, the CXI provider must produce an authorisation key comprising an (svc_id, vni) tuple that corresponds to a CXI service and the key bearer must be an authorised member of that service. If either of these conditions are not met, then endpoint creation for that service fails and the provider initiates a series of fallbacks to identify a service it is permitted to use. When endpoint creation is successful for a given service, the NIC tags all outgoing packets with the VNI for that service. The Rosetta switch enforces whether the traffic can be directed toward the receiving NIC. The receiving NIC will only deliver the packets to the receiving endpoint if it matches the addressing context and VNI. Kubernetes network isolation operates on the IP layer through plugins and network policies, which have no knowledge of the RDMA plane. Without adaptations, containers on the same node share access to the host CXI devices and do not get assigned distinct CXI services, precluding fabric isolation between tenants. Friese et al. [Friese et al., 2025] address this by modifying the CXI driver, the CXI library, and libfabric to scope CXI services to Linux network namespaces on top of existing user and group scopes. Unlike user and group IDs, network namespace IDs are governed from outside of the containers and cannot be modified from within. This provides a tamper-resistant CXI service binding and allows each container its own CXI service and VNI assignment, enabling container-node multi-tenancy. 2.2. Trusted Research Environments Researchers working with sensitive data, such as medical records, require computing environments that enforce strict data governance; access and data egress must be restricted and auditable. TREs, sometimes referred to as Sensitive Data Environments (SDEs), have risen to prominence in recent years to provide these guarantees. However, current TRE deployments are often limited to cloud-based infrastructure and carry associated drawbacks such as compliance, cost and procurement, and vendor lock-in. A local TRE service on IS-AI would mitigate many of these issues and provide researchers with accessible and powerful confidential AI computing. An illustration of the FRIDGE architecture Figure 1. The TRE architecture developed by the FRIDGE project team from The Alan Turing Institute, University College London, University of Bristol, and University of Cambridge for public-cloud, on-premises and AIRR supercomputers including Isambard-AI at Bristol and Dawn at Cambridge. Figure reproduced from Alan Turing Institute [Turing Institute, 2026]. SATRE [SATRE, ] is a compliance framework for TREs developed by Data and Analytics Research Environments UK (DARE UK) [DARE UK, ]; it defines a set of architectural principles, requirements, and recommendations for modern TREs. It provides an objective and robust baseline to measure TRE deployments and is rapidly becoming the industry standard. FRIDGE [FRIDGE, ] is a DARE UK project that seeks to provide a SATRE-compliant, Kubernetes-based TRE service across AIRRâs production compute infrastructure. The project adopts a container-based tenancy model for maximum portability across different systems. It also offers a satellite model wherein a host TRE can dispatch workloads into an isolated compute environment and securely retrieve the processed results. Deploying FRIDGE on IS-AI would expose its massive-scale GPU compute resources to sensitive data research, while IS-AIâs unique ARM64 hardware profile serves to validate the portability and reusability objectives of the FRIDGE project itself. The FRIDGE architecture diagram in Figure 1 illustrates how an existing Home TRE extends its governance boundary to an external AIRR system, effectively borrowing compute resources while maintaining the TREs security controls. The Home TRE contains two internal networks. The Operator Network hosts an API through which TRE Operators manage and configure the FRIDGE instance. The Researcher Network hosts a dedicated API and a âWorkspaceâ used by Job Submitters, a privileged set of Safe Researchers, to create and dispatch computational jobs. Both networks share a single public IP entry point for all outbound communication with external systems. The AIRR system is managed by the FRIDGE Hosting Organisation and its Hosting Provider Administrators. Within AIRR, a TRE Tenancy carves out a secure enclave that is opaque to both the host system and any other tenancies. This tenancy is divided into two sub-networks. The Access Network contains the Access Cluster, which exposes a Kubernetes API, a Kube Proxy for routing, a FRIDGE Proxy that mediates communication with the Isolated Network, and a Container Repository that stores approved container images. The Isolated Network contains the Isolated Cluster, where actual computation takes place, which exposes its own Kube Application Programming Interface (API) and the core FRIDGE API â the heart of the satellite TRE. The FRIDGE API receives requests forwarded from the FRIDGE Proxy and orchestrates workloads within the Isolated Cluster. Notably, the Isolated Cluster has no direct public-facing connection, reinforcing the layered security model. External to the AIRR tenancy, two additional components feed into the system: Custom Images, supplied by the TRE Operator Organisation, are pushed into the Container Repository, while Public Repositories supply base container images. An Effective Connection represents the logical end-to-end path from the Home TREâs Workspace through to the FRIDGE API in the Isolated Cluster, abstracting over the network hops and proxy layers in between. The diagram also shows Other Projects within AIRR, indicating that the AIRR system may host multiple independent tenancies simultaneously, each isolated from one another. Overall, Figure 1 captures how FRIDGE achieves governance boundary extension: the TRE Operator Organisation retains control over data, workloads, and user access, while the FRIDGE Hosting Organisation provides the underlying compute infrastructure under a shared responsibility model, without need for data sharing agreements. 2.3. Secure and Multi-node Model Hosting Driven by demand from the UKâs AI researchers, particularly AISI user base, we aim to support persistent AI model hosting where Kubernetes orchestrates shared-access inference services against common models, each with a single endpoint. The advantages of persistent model hosting include strong sandboxing, standardisation and reproducibility, and efficient cluster resource sharing. We demonstrate the feasibility of this hosting pattern on IS-AI and Kubernetes. An illustration of the architectural layers required for distributed AI model training and workloads; compute substrate, container orchestration, distributed compute engine, training and inference frameworks, and AI workload. The technologies highlighted for use in this paper are Kubernetes, Ray, vLLM and PyTorch. Figure 2. A layered architecture stack for AI infrastructure, comprising (bottom to top): a Compute Substrate of GPUs and CPUs; a Container Orchestrator, primarily Kubernetes; a Distributed Compute Engine; a Training & Inference Framework spanning transformer-focused and general distributed training libraries; and an AI Workload layer. Figure reproduced from Anyscale [Anyscale, 2026]. Building a modern inference platform for Large Language Models (LLMs) requires a layered stack that balances high-performance model execution with robust infrastructure orchestration . The gold standard architecture for this purpose comprises vLLM [et al., 2023] as the LLM engine, Ray [Moritz et al., 2018] as the distributed runtime, and KubeRay [et al., 2021] as the Kubernetes-native orchestrator. This architecture, illustrated in Figure 2, is garnering increasing interest from IS-AI users. vLLM provides a state-of-the-art, low latency, high throughput, memory efficient engine for serving production-level LLMs. It integrates natively with Ray and KubeRay and its features include: token generation; PagedAttention for memory optimized; distributed and coordinated Key-Value (KV) cache management; optimized hardware support for distributed inference techniques like tensor, pipeline and expert parallelism; and integration with popular open-source model repositories like Hugging Face [Wolf et al., 2019]. Ray is an open-source distributed runtime framework that simplifies deployment and scaling of AI applications by providing features like complex model and workload distribution strategies, request routing, and autoscaling. It provides functionality for AI use cases expected by IS-AI users such as training and inference, and an abstract view of compute resources so that applications can be deployed declaratively, reducing the distributed-compute overhead of users. Ray integrates with Kubernetes through the KubeRay operator, which provides custom resources for cloud-native deployments and application management; including, Role-Based Access Control (RBAC), Kubernetes network security features, and dynamic resource allocation including node and pod lifecycles. The architecture addresses both production-level inference performance and enterprise-grade security controls for multi-tenant sensitive workloads. We demonstrate it is possible to fulfill user ambitions by deploying a multi-node, distributed, AI model service using KubeRay, Ray, and vLLM. 3. Implementation The software versions used in this work, for the Kubernetes cluster baseline and the two use cases, are listed in Table 1. Table 1. Software Versions Baseline Software Version HPE CSM Recipe 25.3.3 libfabric 2.1.xâ OpenMPI 5.0.6 OSU Micro-Benchmarks 7.5.2 CNI Plugins v1.6.2 VNI Service Commit 95869 RKE2 v1.34.4+rke2r1 Metacontroller v4.12.5 SmarterDeviceManager 0.0.10 FRIDGE Commit dbf74 Argo Workflows 4.0.4 KubeRay 1.5.1 Ray 2.54.0 vLLM 0.15.0 NCCL 2.26.6 aws-ofi-nccl 1.17.1 â Patched to support Slingshot-Kubernetes integration. 3.1. Bare-metal Kubernetes and Multi-tenancy On IS-AI, Kubernetes multi-tenancy requires Slingshot fabric network isolation to prevent workloads from interfering with one another. We implement the solution of Friese et al. [Friese et al., 2025] including extension of the CXI driver, CXI library, and libfabric to support creation of CXI services scoped to Linux network namespaces; use of the CXI Container Network Interface (CNI) plugin to provide Kubernetesâ container-level network integration; and use of the VNI service to allow Kubernetes to manage RDMA endpoints. In the solution [Friese et al., 2025], users with workloads requiring Slingshot fabric isolation add a VNI annotation to the metadata of their manifests. The VNI service watches for annotated resources and creates VNI Custom Resource Definition (CRD) reservations. Subsequently, when containers are created, the CXI CNI plugin is invoked, which checks the owning resource for a VNI annotation and extracts the network namespace inode of the container being constructed. The plugin then fetches the reserved VNI and creates a CXI service scoped to the network namespace and VNI. In container deletion, the plugin tears down the CXI service and VNI service releases VNI reservations that are no longer in use by a CXI service. Two VNI allocation models are presented by Friese et al. In the Per-Resource model, each workload annotated with vni: "true" receives an exclusive VNI CRD instance providing isolation between different jobs. In the VNI Claim model, a named VniClaim is first created independently. Each workload annotated with vni: "<claim-name>" is then granted access to the same VNI, enabling communication between jobs. We note that the latter allocation model had not been implemented in the version of CXI CNI plugin available to us, and so we rely on the former. In order to provision a Kubernetes cluster for this work, we isolate a group of compute nodes into a Slurm reservation that prevents user job scheduling. Subsequently, we deploy RKE2 with Longhorn [Longhorn, 2026], the NVIDIA GPU Operator [NVIDIA, 2026], and the KubeRay Operator [Project, 2026a] onto this node group. The CXI CNI chained plugin was compiled and added to Flannel following the procedure outlined by Friese et al. [Friese et al., 2025]. We followed the procedure to deploy the VNI service including: installing Metacontroller [Metacontroller, 2026], installing SmarterDeviceManager [SmarterDeviceManager, ], installing the CRD and controller, compiling the endpoint binary with Golang, and building the endpoint image with Podman. All binaries and container images were built for ARM64 on IS-AIâs compute nodes themselves. We modify the solution for IS-AIâs software environment and, notably, for its ARM64 architecture. Inside Kubernetes containers, the libfabric CXI provider discovery routine attempts to locate the High-Speed Network (HSN) netdev associated with each CXI character device to glean metadata required for initialisation e.g. hsn0 for cxi0. However, the SmarterDeviceManager [SmarterDeviceManager, ] used to mount the network devices does not mount the HSN netdev devices and the sysfs entries are not visible inside the containers. This causes the netdev lookup method to fail leaving libfabric without usable interfaces and unable to initialise CXI providers. We add a small patch to the netdev_lookup() method of libfabric that reads two new environment variables, CXIP_DEFAULT_LINK and CXIP_DEFAULT_SPEED, to supply the link state and interface speed, respectively, that would normally be read from the HSN netdev sysfs entries. In combination with CXIP_SKIP_AMA_CHECK, these changes enable the CXI provider to successfully enumerate and open the CXI character devices without requiring netdev entries present in the container. Note, this patch is best understood as a pragmatic workaround rather than a desirable long-term solution. We also extend SmarterDeviceManager to allow gdrdrv character device mounts to support GDRCopy inside containers. We discuss alternatives to these solutions and potential replacements for SmarterDeviceManager such as the HPE CXI Kubernetes Device Plugin [HPE CXI Kubernetes Device Plugin, ] and Akri [Akri, ] in Section 5. The immutability of SquashFS precludes ad hoc modifications to the booted Slingshot Host Software (SHS) stack. As such, the CXI driver, CXI library, and libfabric modifications are baked into a new image artifact and deployed with Cray System Management (CSM) onto our node group. Principally, the SHS Cray Framework Service (CFS) layer was changed to point toward a Nexus repository containing the patched SHS RPMs. Figure 3 represents a minimal Kubernetes manifest containing the necessary modifications for the Per-Resource VNI allocation model, including the VNI annotation, CXI device mount, host library mounts, and environment variables. A Kubernetes manifest illustrating the changes required to implement Linux namespace-based VNI isolation. Notably, the addition of "vni: true", inclusion of CXIP_DEFAULT_LINK, CXIP_DEFAULT_SPEED, CXIP_SKIP_AMA_CHECK environment variables, and inclusion of libcxi and libfabric mounts from the host. ⏠apiVersion: apps/v1 kind: Deployment metadata: annotations: vni: "true" labels: app: demo name: demo spec: replicas: 2 selector: matchLabels: app: demo strategy: template: metadata: labels: app: demo spec: containers: - image: busybox name: busybox command: ["sleep","infinity"] env: - name: LD_LIBRARY_PATH value: "/usr/lib64/:/extras/lib/:/hostlibs" - name: CXIP_DEFAULT_LINK value: "1" - name: CXIP_DEFAULT_SPEED value: "200000" - name: CXIP_SKIP_AMA_CHECK value: "1" resources: requests: smarter-devices/cxi0: "1" limits: smarter-devices/cxi0: "1" volumeMounts: - name: hostlibs mountPath: /hostlibs - name: libfabric mountPath: /extras volumes: - name: hostlibs hostPath: path: /usr/lib64/ type: Directory - name: libfabric hostPath: path: /opt/shs-libfabric/build type: Directory Figure 3. Minimum example of a Kubernetes deployment manifest for testing network namespace-based Slingshot fabric isolation between containers. The CXI device, environment variables, and libraries required to execute fi_pingpong tests are provided to the containers. 3.2. Trusted Research Environments We deploy a FRIDGE instance on IS-AI using the Python implementation of Pulumi infrastructure-as-code [Pulumi, 2026], which is publicly available [Watson, 2025]. The Pulumi code provisions all Kubernetes namespaces, RBAC bindings, applications, Custom Resource Definitions, and network policies required for a FRIDGE instance. FRIDGE requires load balancer IPs for its API and internal services. On IS-AI, Border Gateway Protocol (BGP) peering is not readily available on either Rosetta or edge switches, precluding standard BGP-based load balancer addresses using Layer 2 Address Resolution Protocol (ARP) announcement. Modifications to these switches are not desirable on a production system. We therefore configure MetalLB [MetalLB, 2026] to advertise load balancer addresses using Layer 2 ARP announcement. Container networking is provided by Cilium [Cilium, 2026], which enforces per-namespace NetworkPolicies to ensure pod traffic within on TRE tenancy cannot reach pods from another. In future work, we will investigate dedicated load balancer nodes hosted outside of the Slingshot fabric to enable BGP support. Since RDMA traffic does not traverse the standard Linux kernel networking stack, conventional network policies have limited effect on traffic flows. By ensuring that workloads intending to use RDMA, specifically those that mount CXI devices, are assigned CXI services with dedicated VNIs we can guarantee that such workloads remain isolated from the rest of the Slingshot fabric. Longhorn [Longhorn, 2026] is used in combination with node-local SSDs to provide encrypted volumes to meet the requirement of data encryption at rest. FRIDGE uses Argo Workflows [Argo, 2026] for job orchestration. We provide WorkflowTemplates pre-configured to mount the CXI devices and host library dependencies described in Section 3.1, ensuring that all jobs submitted through the FRIDGE API have access to the Slingshot fabric without requiring manual low-level interventions from users. The WorkflowTemplate metadata carries the vni: "true" annotation, which is inherited by each Workflow it instantiates. We also update the Metacontroller DecaratorController [Metacontroller, 2026] of the VNI service to look for Workflow creation events, as shown in Figure 4. Each job submitted through the FRIDGE API is then guaranteed to have all its pods given the same VNI allocation. Modifications to the Metacontroller DecaratorController of the @glossaries@gls@linkacronymVNIVNI service that enable it to listen for Argo Workflow and RayCluster creation events for the purposes of @glossaries@gls@linkacronymVNIVNI allocation. ⏠apiVersion: v1 kind: List items: - apiVersion: metacontroller.k8s.io/v1alpha1 kind: DecoratorController metadata: name: vni-autovni-controller spec: attachments: - apiVersion: horizon-opencube.eu/v1 resource: vnis hooks: finalize: version: v1 webhook: url: http://vni-endpoint-service.vni-management:8842/finalize sync: version: v1 webhook: url: http://vni-endpoint-service.vni-management:8842/sync resources: - annotationSelector: matchExpressions: - key: vni operator: Exists apiVersion: apps/v1 resource: deployments #---snip--- - annotationSelector: matchExpressions: - key: vni operator: Exists apiVersion: argoproj.io/v1alpha1 resource: workflows - annotationSelector: matchExpressions: - key: vni operator: Exists apiVersion: ray.io/v1 resource: rayclusters - apiVersion: horizon-opencube.eu/v1 resource: vniclaims Figure 4. Modifications to the Metacontroller DecaratorController of the VNI service that enable it to listen for Argo Workflow and RayCluster creation events for the purposes of VNI allocation. The argo-workflows namespace is configured with the Kubernetes privileged pod security standard to permit the host library mounts required for CXI access. This is a broad grant and we explore less intrusive solutions in Section 5. 3.3. Secure and Multi-node Model Hosting We deploy the KubeRay, Ray, and vLLM stack to show the multi-node AI sandboxing use case discussed in section 2. Ray provides an ARM64 compatible container image with functionality for cluster configuration and management which is readily deployable within Kubernetes. The Ray base image does not contain components for vLLM inference or ARM64 and CUDA compatible PyTorch libraries. The Ray documentation recommends adding dependencies at runtime by applying a runtime_env flag to the application configuration which encapsulates packages and installation instructions. This is unsuitable in our deployment for two reasons: firstly, because the required libraries are large and would add unreasonable overhead to pod initialisation; secondly, the increased startup overhead would require changing Kubernetes timeout configurations so that pods are not prematurely put into unusable states by Kubernetes cluster management. To enable deployment on IS-AI we construct a multi-stage custom image build pipeline that incorporates fabric-specific and inference dependencies at image build time. This pipeline has added benefit of reduing reliance on a compatible host operating system and isolates software within the container. The first stage produces a fabric-aware base image. Starting from the official Ray 2.54.0-py312-cu126-aarch64 image [Project, 2026b], a Spack [Spack, 2026, Gamblin et al., 2015] environment is constructed containing the following components, compiled for CUDA compute architecture sm_90: libfabric with netns-aware CXI provider enabled, built with CUDA-aware and GDRCopy-enabled options, and linked against netns-aware libcxi and cxi-driver; NVIDIA Collective Communications Library (NCCL) with CUDA support; aws-ofi-nccl transport plugin; OpenMPI compiled with Open Fabrics Interfaces (OFI) support. The second stage builds the inference image on top of the first stage image. PyTorch with CUDA support is installed from the PyTorch binary wheel index. vLLM is then installed alongside mistral-common for Mistral tokenizer support. Within the resultant image, Ray can be executed after the Spack environment is sourced and activated. KubeRay provides two patterns for application deployment: Imperative using RayJob and Declarative using RayService. We adopt the declarative approach for cluster and application deployment as it enables user control over cluster creation and scaling; continuous, persistent and reliable inference endpoints guaranteed by Kubernetes lifecycle management; and the ability to declare application configurations using patterns and syntax familiar to users. The declarative approach is recommended by Ray for use cases that require persistence and reliability guarantees. The model sandboxing use case aligns with this because it requires persistent hosting. The declarative approach also conforms to the CXI provision discussed in section 3.1, as it allows libraries and environment variables to be declared within the application configuration in the same way shown in Figure 3. This can be achieved at vLLM application runtime using the runtime_env deployment flag mentioned above, or at pod initialisation time using Kubernetes-native volume and volumeMount components. Using this approach, we deploy a Kubernetes manifest with Ray head and worker pods as required, and with the pre-cached model weights injected via a volumeMount. Mutating Admission Policy code to annotate RayCluster resources with gray!10 @@@set@language @@@set@numbers @@@set@frame @@@set@rulecolor @@@set@language\@listingGroupltx_lst_identifiervni:\@listingGroupltx_lst_space \@listingGroupltx_lst_string [rgb].75,0,.25 [named]pgfstrokecolorrgb.75,0,.25"true" in order for @glossaries@gls@linkacronymVNIVNI allocation. ⏠apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicy metadata: name: add-raycluster-annotations spec: reinvocationPolicy: IfNeeded failurePolicy: Fail matchConstraints: resourceRules: - apiGroups: ["ray.io"] apiVersions: ["v1"] operations: ["CREATE"] resources: ["rayclusters"] matchConditions: - name: has-no-annotation expression: "!has(object.metadata.annotations) || !(âvniâ in object.metadata.annotations)" mutations: - patchType: JSONPatch jsonPatch: expression: > [ JSONPatch op: "add", path: "/metadata/annotations/vni", value: "true" ] --- apiVersion: admissionregistration.k8s.io/v1beta1 kind: MutatingAdmissionPolicyBinding metadata: name: add-raycluster-annotations-binding spec: policyName: add-raycluster-annotations Figure 5. Mutating Admission Policy code to annotate RayCluster resources with vni: "true" in order for the RayCluster to receive VNI allocations. VNI allocation requires top-level resource owners to be annotated with vni: "true". In our implementation this is the RayCluster. The KubeRay declarative pattern creates the RayCluster dynamically on initialization of each RayService, we use a combination of Mutating Admission Policy (MAP) [Kubernetes, 2026b] (see Figure 3.3) and the Metacontroller DecaratorController to watch for RayCluster creation events (see Figure 4). Each pod spawned under the RayCluster triggers the CXI CNI chained plugin workflow which checks the pod ownerReferences to find the parent VNI custom resource and create a CXI service for the podâs netns with the parent VNI. The mechanism of claiming a CXI service ID for each pod under the RayCluster leads to a resource bottleneck in the underlying Cassini hardware because each CXI service claims a Triggered List Entry pool of which there are only three available to users per each Network Interface Card (NIC). 4. Evaluation and Results 4.1. Bare-metal Kubernetes and Multi-tenancy We have successfully reproduced the Slingshot namespace-based network isolation implementation of Friese et al. on IS-AIâs ARM64 architecture and validated it through a series of confidence tests using fi_pingpong. First, we confirmed baseline fabric connectivity between two containers on the same host, each with a distinct CXI service scoped to their respective network namespace operating with the same VNI. Second, we confirmed the same connectivity between two containers on different host nodes. Third, we verified VNI isolation enforcement by manually adjusting the VNI on one side of the interaction and observing expected communication loss. Finally, as a preliminary penetration test, we attempted to use a third container to snoop on traffic between two containers by supplying it with one of the victimâs CXI service IDs and VNIs. As expected, endpoint creation fails because the malicious container does not possess the correct namespace. Two line charts showing OSU Micro-Benchmark results for inter-node point-to-point communication over the Slingshot @glossaries@gls@linkacronymCXICXI fabric. Chart (a) shows bandwidth in MB/s against message size in bytes on a logarithmic scale. Chart (b) shows latency in microseconds against message size in bytes on a logarithmic scale. Both charts contain three lines: bare-metal, Kubernetes without @glossaries@gls@linkacronymVNIVNI isolation, and Kubernetes with @glossaries@gls@linkacronymVNIVNI isolation. In both charts the two Kubernetes lines are indistinguishable from one another across all message sizes. However, both Kubernetes lines differ from the bare-metal baseline by a fixed offset of increased latency and decreased bandwidth, which is only visible for small messages. (a) Point-to-point bandwidth (osu_bw) (b) Point-to-point latency (osu_latency) Figure 6. OSU Micro-Benchmarks inter-node point-to-point (a) bandwidth and (b) latency on bare-metal, Kubernetes without network namespace-based VNI isolation, and Kubernetes with Linux network namespace-based VNI isolation. Figures 5(a) and 5(b) show OSU Micro-Benchmarks [OSU, ] point-to-point bandwidth (osu_bw) and latency (osu_latency) results for three different test configurations; bare-metal, and Kubernetes containers with and without network namespace-based VNI isolation, between different nodes. We average 10 runs per test using 10,000 and 20,000 for the iteration counts for bandwidth and latency measurements, respectively. Both Kubernetes configurations, with and without namespace-based VNI isolation, express statistically identical latency profiles that converge to ~3.11ÎŒ for small messages. We conclude that namespace-based VNI isolation adds no meaningful latency penalty. These latency profiles carry a fixed overhead of ~0.89ÎŒ compared to those observed for both bare-metal and Podman containers launched directly via Slurm, which both converge to ~2.22ÎŒ for small messages. We attribute this latency overhead to Kubernetes network namespace boundary traversal on the data path. The latency overhead manifests a similar reduction in bandwidth for small messages in Kubernetes environments, but in all cases the peak bandwidth converges to the theoretical maximum of the Slingshot fabric of 200Gb/s as message sizes increase. These fixed overheads do not scale with message size and so are unlikely to significantly affect throughput-bound workloads. Moreover, factors like rendezvous protocol parameters play far more of a role in performance, which should be noted by users of the system. 4.2. Trusted Research Environments We evaluate the FRIDGE deployment, executed over the Slingshot-isolated Kubernetes stack described in Section 3, using a representative Argo Workflow submitted by a TRE researcher. The WorkflowTemplate encapsulates the necessary resources; mounts, libraries, environment variables, and annotations; required for a workload to securely use the Slingshot network. This allows users to submit jobs without managing low-level CXI configuration directly. A key operational concern is workflow lifetime; Argo Workflows are not automatically deleted after their pods have completed. In our deployment, VNI reservations are only released once the owning Workflow custom resource is deleted. This gradually leads to exhaustion of VNIs from the available pool. Therefore, we enforce a ttlStrategy in the template configuration. Moving forward, we will investigate the Workflow Archive to maintain auditability. We also identify several issues specific to the bare-metal Slingshot networking stack. Cilium and MetalLB provide the container networking and load-balancer functionality required by FRIDGE. Ciliumâs default handling of ARP replies interfered with routing on HSN interfaces, which carry CIDR-based Slingshot configurations rather than conventional host IPs, preventing access to the Kube API from outside the nodes themselves. We resolve this by manually enabling ARP replies on the interfaces and assigning an address on the HSN to the default-route interface. These issues stem from hosting the control-plane and Slingshot workload traffic on the same nodes and could be addressed by decoupling the control-plane from the Slingshot fabric and providing external load-balancer IPs. The CXI CNI plugin itself required only minimal changes to operate alongside Flannel and Cilium, suggesting the approach is portable across different CNI providers. Storage integration was comparatively simple. Longhorn required minor adjustments to work on IS-AI with local node storage, and was sufficient for demonstration and evaluation workloads. A limitation of this approach is that storage capacity scales only with number of nodes in the isolated FRIDGE cluster. Future work will investigate scalable shared multi-tenant storage. FRIDGE relies on Kubernetes pod security standards, but workloads requiring Slingshot access currently execute in a privileged namespace. This presents a broad attack surface and undermines the intent of the isolation model of the cluster. A more secure approach could involve workload controls through runtime classes, Pod Security Admissions, Validating Admission Policies, or with tools like Kyverno [Kyverno, 2026] and KubeArmour [KubeArmor, 2026]. 4.3. Secure and Multi-node Model Hosting We evaluate multi-node LLM serving performance for models deployed to the FRIDGE cluster with Slingshot namespace VNI isolation as the communication layer between nodes. The AI sandboxing use case deployed to Kubernetes allows for pipeline agnostic deployment of models, meaning model pipelines can be split between cluster compute components in a way that flexibly allocates resources. We measure the inter-node communication overhead from the integration of VNI isolation with Slingshot. The results presented here are not for the purpose of evaluating LLM serving optimization, but to show that our AI sandboxing stack does not degrade user experience and performance. First, as a confidence test, we measure the Time to First Token (TTFT) and Tokens per Second (TPS) [BentoML, 2026] with a custom Python benchmarking script using the requests library to directly call the API endpoint on the deployment without the Slingshot namespace-based VNI isolation. The script uses a single streaming request to the endpoint and records the time elapsed between request and first response using the time module. Subsequent tokens are captured and counted to obtain a TPS estimate. Prior to measurement requests, a warmup request is issued to prime the model. We measure the average TTFT and TPS to be 120ms and 22, respectively, over 10 runs each with a batch size of one with the model distributed between two nodes. The TTFT compares favourably to that of 240ms reported by Cerebras for the same model on their wafer hardware [Cerebras, 2024]. The TPS is comparable to 36 TPS reported by Oracle for the same model running on eight AMD MI300X GPUs with a batch size of one [Oracle, 2024]. For a comprehensive evaluation of the AI sandboxing stack we use the vLLM bench framework [vLLM, 2026]. It allows simulation of real-world scenarios for deployed models. The user specifies the modelâs API endpoint along with a set of parameters that simulates the expected workload and a dataset of sample prompts. In order to simplify benchmarking, we maintain vLLM default settings for a number of parameters. For example, the layer split strategy is set to evenly split model layers between GPUs and nodes. GPU utilization is set to the default 0.9. We change the ârequest-rate parameter to 20 requests per second, this is recommended by vLLM when attempting to simulate realistic traffic patterns. We also set the âmax-ongoing-requests parameter in Ray to 128 to limit queuing overhead. The metrics of greatest interest are; Output TPS to measure system capacity, and Time per Output Token (TPOT) and Inter Token Latency (ITL) to measure responsiveness. Output TPS is a measure of the models token generation throughput across all active requests, TPOT is the average time between generating output tokens excluding the first, while ITL refers to the average time between each consecutive token generated within a sequence. (a) Mean ITL (b) Mean TPOT (c) Output TPS by Median ITL Figure 7. Benchmarking results on the ShareGPT_Vicuna_unfiltered dataset for the Meta-Llama-3.1-405B-Instruct-FP8 model deployed in the four infrastructure setups. (a) and (b) show percentile performance for the ITL and TPOT metrics, a lower time in milliseconds (ms) is better. P99 refers to he value below which 99% of requests fall, it reveals worst-case performance for the slowest 1% of requests. (c) shows the relationship between Output TPS and median ITL, a higher TPS is better. We use the ShareGPT_Vicuna_unfiltered dataset which contains 53K example conversations from user prompts to ChatGPT [anon8231489123, 2023]. We deploy the Meta-Llama-3.1-405B-Instruct-FP8[Meta Llama, 2026] model in a tensor_parallel=4, pipeline_parallel=2 configuration. Meaning the model is deployed using two IS-AI nodes, utilizing 4 GPUs on each node. We compare four separate infrastructure configurations; the Bristol Centre for Supercomputing (BriCS) guidelines on how to perform distributed vLLM inference on IS-AI [BriCS, 2026], the vLLM tutorial on using Ray for multi-node inference [vLLM Authors, 2026], along with our AI Sandboxing stack with the VNI component turned off and on. We perform 10 runs of vLLM bench for each of our hardware configuration setups. Figure 7 shows the performance of each of the hardware configurations we evaluate across the ITL, TPOT and Output TPS metrics. The two setups that use our stack perform favorably compared to the BriCS and vLLM alternatives. For both ITL and TPOT the vni: "true" setup achieves mean (ms) performance of less-than half the BriCS and vLLM setups. The performance improvement of the K8s and VNI deployments are also shown in output TPS where the vni: "true" setup achieves approximately a TPS speed-up of three times over the BriCS configuration. We observe that although our work does not concentrate on LLM serving optimization, we have inadvertently optimized the performance of our K8s and VNI deployment configurations. We followed the provided setup steps for both the BriCS and vLLM configurations, which includes optimizations for NCCL and RDMA, and for BriCS optimizations for Slingshot and IS-AI. 5. Summary and future work Using real world use cases that represent the rapidly evolving secure model hosting and sensitive data environments we have demonstrated a reproducible approach for deploying multi-tenant, bare-metal Kubernetes platforms on IS-AI. Another important aspect that we confirmed is portability of cloud-native architecture and playbooks on IS-AIâs unique hardware profile, which uses Cray HPE EX with NVIDIA Grace-Hopper GH200 and Slingshot interconnect, for both use cases. In other words, supercomputing platforms built on Ethernet-compatible interconnects such as Slingshot and cloud-like provisioning stacks such as CSM can support multi-tenant services without requiring fully bespoke, vertically integrated solutions. However, several challenges remain that we would like to continue working on for hardening the solutions and for reducing the complexity for scale-out operations where we would welcome contributions and feedback from the community. The current Slingshot isolation stack is function, but operationally complex. It relies upon several independent and loosely-coupled components including; Metacontroller, SmarterDeviceManager, the VNI service, and the chained CNI CXI plugin. Many of these components have been subject to custom modifications as documented in Section 3. Also, workloads requiring Slingshot access require several manifest augmentations, including VNI annotations, hosts library and driver mounts, and environment variables describing device metadata. Reducing this operational burden for both administrators and end users is therefore an important direction for future work. A more robust solution would involve simplified implementation details that are elided from the user as much as possible. We outline suggestions and areas of future work to achieve this. The CXI driver, CXI library, and libfabric patches supplied by Friese et al. are being upstreamed into SHS version 14 release candidates, which will hopefully reduce administration overhead of future Slingshot isolation solutions. The supplemental patch to libfabric introduced in this paper should be superseded by mounting CXI character devices along with their associated HSN netdev devices. SmarterDeviceManager, originally developed for agricultural IoT applications, has not seen project activity in more than a year and is likely no longer maintained. This makes it difficult to request or implement feature and security updates. Future support for netdev mounts is therefore unlikely. Akri [Akri, ] emerges from the IoT space and presents a potential candidate replacement for device handling. It is actively maintained with a large community and supports both device discovery by IP and by udev rules. A custom discovery handler could mount the CXI character devices, gdrdrv character devices, and HSN netdev devices as part of a single resource request. CXI Kubernetes DRA driver presents another viable candidate to address many of these problems. With some development, a purpose-built DRA driver for CXI devices could allow implicit netdev discovery and mounting, as well as automatic mounting host dependencies. This approach could also eliminate the need for some of the modifications to components, like to those of Metacontroller to allow listening for custom resource creation. The CNI CXI plugin and the VNI service represent a robust solution and may benefit from minor optimisations. The CNI CXI plugin currently only supports direct parent resource discovery and does not traverse the ownership chain in its entirety. In Section 3.3, we present Kubernetes MAPs as a solution to address this problem. MAPs could be leveraged much more heavily to either warn about missing manifest parameters or to inject them automatically, providing a much more seamless user experience. As discussed in section 3.3 the plugin approach triggers resource constraints caused by hardcoded limits in the underlying Cassini hardware. This limit could be overcome by removing the one-to-one dependency between VNI enabled pods and allocated CXI services. This is feasible because individual CXI services have the capacity for managing multiple VNIs, however it would require more work to understand how AI applications utilize the NIC resources allocated to each CXI. The VNI service could also be modified to represent VNIs as a cluster-wide ResourceSlice. This would couple the VNI lifecycle of workloads more natively into Kubernetes and avoiding the issue of VNI exhaustion by granting the scheduler greater visibility into the CXI allocation stack. Also, the VNI service currently tracks VNIs using an SQLite database, which could potentially be replaced with Custom Resources metadata to improve Kubernetes integration. Alternatively, the service could be integrated into the Dynamic Resource Control 2 (DRC2) VNI allocation service provided by User Services Software (USS) [USS, ]. The AI sandboxing use case could be extended to cover the full lifecycle of AI model development, especially relevant is the fine-tuning use case in which models are finetuned on data that requires the governance and access security restrictions provided by our deployment stack. The AI user experience could be improved by extending FRIDGE and VNI isolation to other AI frameworks such as Kubeflow for finetuning [The Kubeflow Authors, 2026] and KServe for inference [The KServe Authors, 2026]. This would involve finding a new way to assign VNI information to applications because they have different primitives and abstractions around how they treat model deployments. Acknowledgments We would like to thank teams at the BriCS and HPE for their contributions and support that involves not only setting up the infrastructure for the Kubernetes services but also the wider AI software stack. The FRIDGE project partners at the Alan Turing institute, University College London, and University of Cambridge have been instrumental in co-designing the architecture on the UK AIRR resources including Isambard-AI and public cloud ecosystems. Isambard-AI is funded by the UK Governmentâs Department of Science, Innovation and Technology (DSIT) via UK Research and Innovation (UKRI) and Science and Technology Facilities Council (STFC). The FRIDGE project has been funded by the DARE UK Early Adopters programme. References SchedMD [2026a] SchedMD. Slinky: SchedMDâs set of projects to enable interoperability between Slurm and Kubernetes, 2026a. URL https://slurm.schedmd.com/slinky.html. [Online; accessed March 16, 2026]. SchedMD [2026b] SchedMD. Kubernetes Operator for Slurm Clusters, 2026b. URL https://slinky.schedmd.com/projects/slurm-operator/en/release-1.0/. [Online; accessed March 16, 2026]. CNCF [2026] CNCF. Batch System Initiative Working Group, 2026. URL https://tag-runtime.cncf.io/wgs/bsi/. [Online; accessed March 16, 2026]. Volcano [2026] Volcano. Cloud-native Container Batch Scheduler for HPC Workloads, 2026. URL https://volcano.sh/en/docs/. [Online; accessed March 16, 2026]. Armada [2026] Armada. Multi-Kubernetes Cluster Batch Job Meta-Scheduler, 2026. URL https://armadaproject.io/. [Online; accessed March 16, 2026]. Friese et al. [2025] Philipp A. Friese, Ahmed Eleliemy, Utz-Uwe Haus, and Martin Schulz. Closing the hpc-cloud convergence gap: Multi-tenant slingshot rdma for kubernetes, 2025. URL https://arxiv.org/abs/2508.09663. McIntosh-Smith et al. [2024] Simon McIntosh-Smith, Sadaf Alam, and Christopher Woods. Isambard-AI: a leadership-class supercomputer optimised specifically for artificial intelligence. In Proceedings of the Cray User Group, pages 44â54. ACM, 2024. Watson [2025] Jake Watson. FRIDGE on Isambard-AI, 2025. URL https://github.com/isambard-sc/fridge. [9] SATRE. SATRE, 2026. URL https://satre-specification.readthedocs.io/en/stable/. Moritz et al. [2018] Philipp Moritz et al. Ray: A distributed framework for emerging AI applications. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 561â577. USENIX Association, 2018. et al. [2023] Woosuk Kwon et al. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. Kubernetes [2026a] Kubernetes. Dynamic Resource Allocation, 2026a. URL https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/. [Online; accessed March 20, 2026]. Rancher [2026] SUSE Rancher. Rancher kubernetes engine 2, 2026. URL https://docs.rke2.io/. [Online; accessed March 30, 2026]. Turing Institute [2026] Alan Turing Institute. FRIDGE, 2026. URL https://github.com/alan-turing-institute/fridge/blob/main/docs/architecture/architecture.md. [Online; accessed March 20, 2026]. [15] DARE UK. DARE UK, 2026. URL https://dareuk.org.uk/. [16] FRIDGE. FRIDGE, 2026. URL https://dareuk.org.uk/how-we-work/ongoing-activities/dare-uk-early-adopters/fridge/. Anyscale [2026] Anyscale. Anyscale, 2026. URL https://w.anyscale.com/blog/ai-compute-open-source-stack-kubernetes-ray-pytorch-vllm. [Online; accessed March 23, 2026]. et al. [2021] Ali Kanso et al. Designing a kubernetes operator for machine learning applications. In Proceedings of the Seventh International Workshop on Container Technologies and Container Clouds, 2021. URL https://api.semanticscholar.org/CorpusID:244663710. Wolf et al. [2019] Thomas Wolf et al. Huggingfaceâs transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019. doi: 10.48550/arXiv.1910.03771. URL https://arxiv.org/abs/1910.03771. Longhorn [2026] Longhorn. Longhorn, 2026. URL https://longhorn.io/. [Online; accessed March 24, 2026]. NVIDIA [2026] NVIDIA. NVIDIA GPU Operator, 2026. URL https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/overview.html/. [Online; accessed March 24, 2026]. Project [2026a] Ray Project. KubeRay Operator, 2026a. URL https://github.com/ray-project/kuberay. [Online; accessed March 30, 2026]. Metacontroller [2026] Metacontroller. Metacontroller, 2026. URL https://github.com/metacontroller/metacontroller. [Online; accessed March 30, 2026]. [24] SmarterDeviceManager. SmarterDeviceManager, 2026. URL https://github.com/smarter-project/smarter-device-manager. [25] HPE CXI Kubernetes Device Plugin. HPE CXI Kubernetes Device Plugin, 2026. URL https://github.com/HewlettPackard/cxi-k8s-device-plugin. [26] Akri. Akri, 2026. URL https://docs.akri.sh/. Pulumi [2026] Pulumi. Pulumi Infrastructure-as-Code, 2026. URL https://w.pulumi.com/. MetalLB [2026] MetalLB. Metallb, 2026. URL https://metallb.io/. [Online; accessed March 23, 2026]. Cilium [2026] Cilium. Cilium, 2026. URL https://cilium.io/. [Online; accessed March 23, 2026]. Argo [2026] Argo. Argo, 2026. URL https://argoproj.github.io/workflows/. [Online; accessed March 23, 2026]. Project [2026b] Ray Project. Ray Project Docker, 2026b. URL https://hub.docker.com/layers/rayproject/ray/2.54.0-py312-cu126-aarch64/images/sha256-c64c52747e1540949f2340d1bee36c062a964123d7d601a3f803333e1b2d059. [Online; accessed March 24, 2026]. Spack [2026] Spack. Spack, 2026. URL https://spack.io/. [Online; accessed March 24, 2026]. Gamblin et al. [2015] Todd Gamblin, Matthew LeGendre, Michael R. Collette, Gregory L. Lee, Adam Moody, Bronis R. de Supinski, and Scott Futral. The spack package manager: bringing order to hpc software chaos. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC â15, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450337236. doi: 10.1145/2807591.2807623. URL https://doi.org/10.1145/2807591.2807623. Kubernetes [2026b] Kubernetes. Mutating Admission Policy, 2026b. URL https://kubernetes.io/docs/reference/access-authn-authz/mutating-admission-policy/. [Online; accessed March 20, 2026]. [35] OSU. OSU Micro-Benchmarks, 7.5. URL https://mvapich.cse.ohio-state.edu/benchmarks/. Kyverno [2026] Kyverno. Kyverno, 2026. URL https://kyverno.io/. [Online; accessed March 23, 2026]. KubeArmor [2026] KubeArmor. KubeArmor, 2026. URL https://kubearmor.io/. [Online; accessed March 23, 2026]. BentoML [2026] BentoML. Key metrics for LLM inference, 2026. URL https://bentoml.com/llm/inference-optimization/llm-inference-metrics. [Online; accessed March 12, 2026]. Cerebras [2024] Cerebras. Llama 3.1 405b now runs at 969 tokens/s on cerebras inference, 2024. URL https://w.cerebras.ai/blog/llama-405b-inference. Oracle [2024] Oracle. Serving llama 3.1 405b model with amd instinct mi300x accelerators, 2024. URL https://blogs.oracle.com/cloud-infrastructure/serving-llama-31-405b-model-with-amd-mi300x-gpus. vLLM [2026] vLLM. vLLM: Benchmark CLI, 2026. URL https://docs.vllm.ai/en/latest/benchmarking/cli/. [Online; accessed March 13, 2026]. anon8231489123 [2023] anon8231489123. Sharegpt_vicuna_unfiltered, 2023. URL https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered. Accessed: 2026-04-17. Meta Llama [2026] Meta Llama. Meta-llama-3.1-405b-instruct, 2026. URL https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct. BriCS [2026] BriCS. Distributed vllm inference, 2026. URL https://docs.isambard.ac.uk/user-documentation/tutorials/distributed-inference/. Accessed: 2026-04-20. vLLM Authors [2026] vLLM Authors. vllm parallelism and scaling, 2026. URL https://docs.vllm.ai/en/stable/serving/parallelism_scaling/. Accessed: 2026-04-20. [46] USS. USS â User Services Software, 2026. URL https://support.hpe.com/hpesc/public/docDisplay?docId=dp00005611en_us&page=install/About_USS.html&docLocale=en_US. The Kubeflow Authors [2026] The Kubeflow Authors. Kubeflow. https://w.kubeflow.org/, 2026. [Online; accessed 20-April-2026]. The KServe Authors [2026] The KServe Authors. KServe. https://kserve.github.io/website/, 2026. [Online; accessed 20-April-2026]. Glossary Acronyms AIRR AI Research Resource AISI AI Security Institute API Application Programming Interface ARP Address Resolution Protocol BGP Border Gateway Protocol BriCS Bristol Centre for Supercomputing CFS Cray Framework Service CNCF Cloud Native Computing Foundation CNI Container Network Interface CRD Custom Resource Definition CSM Cray System Management CXI Cray eXascale Interconnect DARE UK Data and Analytics Research Environments UK DRA Dynamic Resource Allocation DRC2 Dynamic Resource Control 2 DSIT Department of Science, Innovation and Technology FRIDGE Federated Research Infrastructure by Data Governance Extension HPC High-Performance Computing HPE Hewlett Packard Enterprise HSN High-Speed Network IS-AI Isambard-AI ITL Inter Token Latency K8s Kubernetes KV Key-Value LLM Large Language Model MAP Mutating Admission Policy NCCL NVIDIA Collective Communications Library NIC Network Interface Card OFI Open Fabrics Interfaces RBAC Role-Based Access Control RDMA Remote Direct Memory Access RKE2 Rancher Kubernetes Engine 2 SATRE Standard Architecture for Trusted Research Environments SDE Sensitive Data Environment SHS Slingshot Host Software STFC Science and Technology Facilities Council TPOT Time per Output Token TPS Tokens per Second TRE Trusted Research Environment TTFT Time to First Token UKRI UK Research and Innovation USS User Services Software VNI Virtual Network Identifier WLM Workload Manager