I’ve published Your GPU Is Up. Is Your AI? The Inference Resilience Profile, a whitepaper proposing a way to define and test AI service continuity across Kubernetes and NVIDIA GPU failure domains.
A healthy accelerator and a running inference service are useful signals. The harder question is what happens to the work already in progress when a GPU, node, network, scheduler, or serving component fails.
Download the whitepaper
The permanent Zenodo record provides the archived paper and citation details. The whitepaper is available under Creative Commons Attribution 4.0 International (CC BY 4.0).
What happens to the request already in flight?
A restart can restore a process while leaving an accepted request incomplete. A retry can duplicate streamed output. A recovered agent can repeat an action if its state and side effect history do not agree. These are the kinds of service outcomes that infrastructure availability alone does not describe.
The proposed Inference Resilience Profile makes those expectations explicit. It describes the workload, failure domains, state continuity policy, isolation requirements, reserve capacity, service objectives, and evidence needed to evaluate recovery. A companion benchmark design, ResilientInferenceBench, describes controlled tests of those requirements.
What the paper covers
- Request survival and output integrity: whether accepted requests complete and streamed responses remain valid and complete.
- State continuity and recovery: how sessions, caches, retrieval context, and agent state behave during disruption.
- Isolation and fault propagation: whether a failure affects other workloads or tenants.
- Performance under failure: retained throughput, latency degradation, recovery time, and reserve capacity consumption.
- Cost, energy, and evidence: the resources recovery consumes and the records needed to reproduce and assess a result.
Who should read it
The paper is written for enterprise architects, platform engineers, AI SRE teams, infrastructure operators, security teams, and researchers working on resilient inference systems.
A public proposal for review
This is a public technical proposal. It has not undergone formal external peer review or empirical platform validation. Numerical examples are synthetic illustrations. Companion software, schemas, and executable test tooling are planned and are not included in the publication.
I’m sharing the design to invite technical feedback before implementation and platform validation. I’m particularly interested in missing failure scenarios, practical measurement challenges, and the evidence operators would need to trust a continuity claim. Share your perspective in the comments.
The post New whitepaper: Your GPU Is Up. Is Your AI? appeared first on Digital Thought Disruption.