This is a single-node Kubernetes setup running on an NVIDIA DGX Spark (GB10, aarch64). The K3s control plane runs as a systemd service (k3s.service) and its embedded agent doubles as the worker, so one node hosts both the API server and the workloads. Cilium is the CNI, installed with kubeProxyReplacement=true and the L7 (Envoy) proxy enabled. GPU workloads run as normal pods: K3s's embedded containerd uses the NVIDIA container runtime, and the GB10 is shared across pods via the device plugin's time-slicing (the GB10 does not support MIG, so there is no hardware partitioning).
Ingress is handled by the Cilium Gateway API. The Gateway is fronted by a type: LoadBalancer service that gets a stable IP (192.168.100.240) from a CiliumLoadBalancerIPPool — no L2/BGP announcement is needed because nothing on the LAN reaches that IP directly. Instead, tailscale serve on the host terminates HTTPS and proxies to the Gateway locally (http://192.168.100.240:80, reachable on the host via Cilium's socket LB). Remote clients on the tailnet — e.g. the laptop — reach everything through the host's MagicDNS name (https://<device>.<tailnet>.ts.net), with routing done by path in HTTPRoutes.
┌────────────────────────┐
│ Laptop (macbook) │
│ on tailnet │
└───────────┬────────────┘
│ https://<device>.<tailnet>.ts.net
│ (Tailscale / WireGuard, encrypted)
▼
╔══════════════════════════════════════════════════════════════╗
║ DGX Spark (GB10, aarch64) ║
║ ║
║ tailscaled ── serve :443 (TLS termination) ║
║ │ ║
║ │ http://192.168.100.240:80 (local, via socket LB) ║
║ ▼ ║
║ Cilium Gateway [LoadBalancer svc + LB-IPAM] ║
║ │ 192.168.100.240:80 (no hostname → path routing) ║
║ ▼ ║
║ HTTPRoute (path-based) ║
║ │ ║
║ ▼ ║
║ Backend Pods ───────────────▶ GB10 GPU ║
║ (containerd + nvidia runtime, device-plugin time-slicing) ║
║ ║
║ ── k3s server (systemd) + embedded agent ── ║
║ ── Cilium CNI (kube-proxy replacement, Envoy L7) ── ║
╚══════════════════════════════════════════════════════════════╝
The GB10 has no MIG support, so it is shared via the device plugin's time-slicing. The plugin is configured with replicas: 4 (see nvidia-device-plugin-config.yaml), which makes the one physical GPU appear to the scheduler as nvidia.com/gpu: 4. Up to four pods can then each request one "GPU" and run concurrently. This is oversubscription, not isolation: all four slices are the same chip, kernels are context-switched (serialized) onto it, and the pods share the GB10's unified LPDDR5X memory with no per-pod cap. Treat it as cooperative sharing among your own workloads, not multi-tenant isolation.
Each GPU pod must set runtimeClassName: nvidia and request nvidia.com/gpu: 1.
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ each pod:
│ Pod A │ │ Pod B │ │ Pod C │ │ Pod D │ runtimeClassName: nvidia
│ gpu: 1 │ │ gpu: 1 │ │ gpu: 1 │ │ gpu: 1 │ limits: nvidia.com/gpu: 1
└────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘
│ │ │ │
└────────────┴─────┬──────┴────────────┘
▼
┌────────────────────────────────────────────────────┐
│ NVIDIA device plugin (timeSlicing replicas: 4) │
│ advertises nvidia.com/gpu: 4 │
└───────────────────────┬────────────────────────────┘
│ 4 "GPUs" = 4 handles to the SAME device
▼
┌────────────────────────────────────────────────────┐
│ 1 × physical GB10 GPU │
│ • time-shared: kernels context-switch, serialized │
│ • shared unified LPDDR5X memory — NO isolation │
└────────────────────────────────────────────────────┘
You should have tailscale installed and a tailscale network in place.
You should also have the NVIDIA Container Toolkit installed on the host. K3s detects it at startup and auto-creates the nvidia RuntimeClass and containerd runtime that the GPU device plugin relies on. Without it, the device-plugin pod fails and the GB10 is never advertised as nvidia.com/gpu.
When you know the tailscale network, you can configure the gateway.yaml file and specify the correct wildcard hostname.
Simply execute the installer as sudo:
sudo ./installer.sh