Kubernetes DRA for GPU Inference Pools

Kubernetes DRA can describe GPU inference needs more precisely than a generic device count, but precision only helps when claims, device classes, topology, admission, and recovery agree.

This guide builds a production baseline around full GPUs and existing MIG devices while labeling Dynamic MIG, MPS, consumable shares, health checks, and custom time slicing as alpha in NVIDIA GPU Operator 26.7.

Kubernetes DRA begins with an inference shape

Describe model memory, compute class, replica behavior, topology, startup time, and isolation before selecting a GPU. Tokens per second alone cannot distinguish an interactive replica needing stable memory from an offline worker that can tolerate relocation or queueing. In the accelerator scheduler, an inference resource envelope is not supporting material; it is the decision record. The platform operator looks there for model digest, weights, KV budget, batch range, precision, topology, isolation, and SLO.

Begin with the model's physical shape rather than a vendor label. Weights, KV cache, adapters, precision, batch range, topology, and recovery target determine whether a replica needs an exclusive accelerator or can tolerate a partition.

This envelope is also the explanation when a scheduler correctly refuses an impossible placement. The accelerator scheduler puts Dynamic Resource Allocation in front of the platform operator as the opening constraint on Kubernetes DRA. A schedulable request is one whose tradeoffs can be named before a device is chosen.

Define device classes as platform policy

Create platform-owned DeviceClasses for supported full GPUs and existing MIG profiles, with selectors based on stable driver-advertised attributes. Application teams should choose a service class without embedding vendor trivia or node names into every deployment.

Rehearse one uncomfortable move: remove one device generation and confirm unsatisfied claims remain explainable. If workloads match mutable node labels as their allocation contract, the accelerator scheduler tells the platform operator to stop.

A DeviceClass is platform policy expressed as selection. The platform team can map a stable service name to supported hardware attributes while application manifests avoid node names and generation-specific labels.

Removing a device family then becomes a catalog change with visible unsatisfied claims instead of a silent workload migration. Instead of accepting Kubernetes DRA at face value, the platform operator reads the accelerator scheduler through ResourceClaim. Stable class names protect workloads from hardware churn without hiding platform policy.

Version and maturity claims should be checked against Kubernetes Dynamic Resource Allocation, Allocate devices with DRA, and NVIDIA GPU Operator 26.7 DRA driver. Kubernetes documentation defines DRA objects and allocation flow, while the NVIDIA 26.7 guide documents its driver and maturity table. DynamicMIG, MPSSupport, ConsumableShares, NVMLDeviceHealthCheck, PassthroughSupport, DeviceMetadata, and TimeSlicingSettings are alpha and disabled by default in that release.

Model allocation through ResourceClaims

Use a ResourceClaim or template whose lifetime matches the inference pod and whose request names the required device class and constraints. Decide whether a claim is exclusive to one pod or intentionally shared among containers before deployment. For the platform operator, GPU inference scheduling is what makes the accelerator scheduler's Kubernetes DRA boundary measurable. Claim lifetime is capacity management, not a cleanup detail.

ResourceClaims make allocation an owned object rather than a scheduling side effect. The claim can follow a pod template, record the chosen device, and disappear according to an explicit lifetime rule. Scale and deletion tests reveal whether replicas accidentally share a scarce device or strand capacity after their consumers vanish.

Compare a claim ownership map against a fresh attempt to scale, reschedule, delete, and recreate the consuming pod. The platform operator uses the gap, not the prettier artifact, to revise the accelerator scheduler. Ownership tests should also cover stuck finalizers and node loss.

Keep alpha sharing outside the baseline

Before another step, the accelerator scheduler requires feature, driver version, Kubernetes gate, default, compatibility conflicts, test cluster, and promotion owner to live in a feature-maturity ledger. The platform operator treats an empty field as an unresolved decision. NVIDIA GPU Operator 26.7 marks Dynamic MIG, MPS support, consumable shares, custom time-slicing settings, and several related capabilities alpha and disabled by default. Evaluate each behind an explicit feature gate instead of presenting all DRA allocation as equally mature.

The practical Kubernetes DRA test in this accelerator scheduler is NVIDIA DRA driver, and the result belongs to the platform operator. Alpha sharing belongs in an experiment lane with its own recovery evidence.

GPU Operator 26.7 draws an important maturity line. Full GPU and existing MIG allocation have a different operational posture from Dynamic MIG, MPS support, consumable shares, and custom time slicing marked alpha. Keep those experimental paths disabled in the baseline pool so a feature-gate rollback still leaves a schedulable serving tier.

Join scheduling to admission control

A precise claim does not protect the serving tier from too many models, cold starts, or unbounded contexts. Admission should price weights, reserved KV cache, expected concurrency, and recovery headroom before a deployment consumes the allocated GPU.

Allocation does not replace admission control. A claim may fit model weights while the projected context length exhausts KV memory at target concurrency.

Price cache, adapters, fragmentation, cold-start duplication, and failover headroom before admitting another replica, then explain a refusal in workload terms the model owner can change. A change in Dynamic Resource Allocation forces the platform operator to revisit how the accelerator scheduler handles Kubernetes DRA. The best rejection tells the model owner which dimension exceeded the serving budget.

Try to break the boundary by choosing to submit a model that fits weights but exhausts cache under target concurrency. The platform operator reopens the accelerator scheduler design as soon as a valid allocation is treated as guaranteed serving capacity.

Preserve topology through the serving stack

Carry device UUID, NUMA and PCI context, MIG profile, claim UID, pod, model, and router pool into telemetry. Multi-GPU or NVLink-sensitive workloads need a topology statement that the scheduler and runtime can both verify. Before approval, the platform operator uses ResourceClaim to expose a hidden Kubernetes DRA premise in the accelerator scheduler. Request traces become actionable when they retain the allocated device's lineage across every scheduler and runtime hop.

Repeat move a replica across nodes and compare the lineage without manual lookup and capture what changed in a device-to-request lineage graph. That contrast gives the platform operator a practical debugging view of the accelerator scheduler.

Topology information must survive beyond the scheduler. Device UUID, ResourceSlice, claim UID, NUMA and PCI placement, MIG profile, container device, model digest, router pool, and trace identifier form the useful lineage. Without that join, a latency regression becomes a manual node-forensics exercise. Dashboards should preserve those identifiers across retention tiers.

Design health and drain behavior

Failure drills should make claims move through uncomfortable states: device disappearance, driver restart, node drain, stuck allocation, pod eviction, and partial pool loss. Observe whether the scheduler, DRA driver, CDI injection, runtime, and serving router converge on the same ownership story or each retains a different ghost. The accelerator scheduler makes GPU inference scheduling an operating control rather than an implied part of Kubernetes DRA for the platform operator. A recovery test passes only when every layer agrees that the old ownership is gone.

Driver availability, DCGM telemetry, model-worker health, and allocation health are related but distinct signals. In Operator 26.7, alpha NVML device health is not a production baseline, so define how telemetry drains traffic and how claims recover without overstating automatic remediation.

The durable pair is a GPU drain state machine plus health source, threshold, router action, pod action, claim state, replacement result, and operator escalation. It lets the platform operator audit the accelerator scheduler after the original session has disappeared.

Upgrade the driver and API deliberately

Pin Kubernetes, GPU Operator, DRA driver, runtime, and CDI compatibility, then rehearse upgrades on a representative pool. Resource API changes and driver feature gates can alter allocation even when the deployment manifest is unchanged.

Run the hostile case until it can preserve an allocated workload while cycling controller and kubelet plugin versions. If the cluster upgrades without replaying inference claims, the accelerator scheduler has handed the platform operator a specific redesign task. The platform operator keeps Kubernetes DRA open until the accelerator scheduler has tested NVIDIA DRA driver under pressure. Different service classes can share infrastructure without pretending they share risk tolerance.

Pool policy translates business traffic into placement rules. Interactive replicas may reserve known-good full devices and stricter recovery budgets, while batch workers accept queues or tested partitions. The distinction should live in explicit classes and router behavior rather than an operator's memory of which nodes are special.

The serving-control neighborhood includes inference gateway live-load routing, multi-LoRA serving, KV cache optimization, and vLLM vs SGLang agent workloads. Gateway routing, adapter placement, cache budgets, and serving-engine behavior consume the allocated device after scheduling. Their receipts should carry claim and device identity so a latency incident can be joined to actual placement.

The fixture chooses a stable preconfigured MIG device and refuses an alpha MPS path unless explicitly allowed.

Runnable artifact — gpu-claim-selector.test.mjs

import assert from "node:assert/strict";
const devices=[{id:"gpu-a",class:"full-h100",memory:80,mode:"full",zone:"a",alpha:false},{id:"mig-b",class:"mig-2g",memory:20,mode:"mig",zone:"b",alpha:false},{id:"mps-c",class:"shared-alpha",memory:40,mode:"mps",zone:"a",alpha:true}];
const allocate=(claim,{allowAlpha=false}={})=>{const device=devices.find(d=>d.class===claim.deviceClass&&d.memory>=claim.memory&&d.zone===claim.zone&&(allowAlpha||!d.alpha))??null;return device?{claimUID:claim.uid,deviceID:device.id,mode:device.mode,zone:device.zone,cdi:"nvidia.com/"+device.id}:{claimUID:claim.uid,rejected:"unsatisfied-claim"}};
const stable={uid:"claim-17",deviceClass:"mig-2g",memory:18,zone:"b"};assert.deepEqual(allocate(stable),{claimUID:"claim-17",deviceID:"mig-b",mode:"mig",zone:"b",cdi:"nvidia.com/mig-b"});
assert.equal(allocate({...stable,zone:"a"}).rejected,"unsatisfied-claim");assert.equal(allocate({uid:"c-2",deviceClass:"shared-alpha",memory:30,zone:"a"}).rejected,"unsatisfied-claim");assert.equal(allocate({uid:"c-2",deviceClass:"shared-alpha",memory:30,zone:"a"},{allowAlpha:true}).deviceID,"mps-c");
console.log("PASS: GPU claim honors workload constraints");

Run node gpu-claim-selector.test.mjs. Expected receipt: PASS: GPU claim honors workload constraints.

Release with allocation receipts

No accelerator scheduler release is complete with a GPU pool release packet alone. The platform operator also needs a run that can reconstruct one slow request from router through model worker to claim and hardware and an explanation for any mismatch. Release means the accelerator scheduler can show the platform operator a reproducible Dynamic Resource Allocation result for Kubernetes DRA. Publish only combinations the team can reproduce from an empty cluster.

Promotion requires a narrow compatibility matrix: Kubernetes version, DRA feature gates, GPU Operator, driver, container runtime, hardware generation, claim pattern, and serving image. Rehearse install, upgrade, rollback, and recovery for each supported row. A configuration that only works after hand-edited nodes is not a production profile.

Ship claims, class definitions, admission thresholds, alpha-feature state, topology traces, dashboards, and recovery drills beside the serving deployment. A rollout is complete when an operator can explain why each replica received its device and what happens when that device disappears.

Use Kubernetes DRA first for explicit full-GPU and preconfigured MIG allocation with tested device classes, claims, CDI runtime, and recovery. Treat alpha sharing and dynamic-partition features as experiments behind separate gates until their maturity and your failure drills justify production authority.