AI demand rarely arrives as a clean infrastructure request.
It usually starts as a conversation. A data science team has a model they want to test. A business unit wants an internal assistant. A security team wants retrieval-augmented generation close to sensitive data. A platform team is asked whether the existing private cloud can “just add GPUs.”
That is where many infrastructure teams get pulled into the wrong first decision.
The first decision is not which GPU server to buy, whether the workload should run on Kubernetes or a VM, or whether the cluster should be VxRail, Dell vSAN Ready Nodes, or a dedicated GPU island.
The first decision is the operating model.
VCF 9.1 gives infrastructure teams more options for private AI, GPU acceleration, Kubernetes, vSAN efficiency, observability, and cost management. Broadcom has positioned VCF 9.1 as a private cloud platform for modern workloads, including production AI, with support for AI-oriented infrastructure, Kubernetes, multi-tenant isolation, high-speed networking, and accelerator ecosystem integrations.
That does not remove the design work. It actually makes the design work more important.
If the platform team does not define placement, lifecycle, tenancy, storage, network, and cost models before the first GPU cluster lands, the first successful AI workload can become the pattern everyone else inherits.
That is dangerous.
A GPU cluster is expensive, scarce, operationally sensitive, and politically visible. Once it becomes shared infrastructure, every missing decision turns into a ticket, an exception, or an outage conversation.
This article lays out the decision model infrastructure teams should build before deploying the first GPU-ready VCF cluster.
TL;DR
VCF 9.1 can support a stronger private AI architecture, but the GPU cluster should be the output of the design process, not the starting point.
Before deploying the first GPU cluster, infrastructure teams need to decide:
Where AI workloads belong: existing workload domain, dedicated AI workload domain, or separate expansion path
How GPUs are consumed: full-device passthrough, vGPU, MIG-style partitioning, CPU inference, or Kubernetes node pools
Who owns lifecycle risk: platform, virtualization, VxRail, data science, application, or shared platform engineering
How tenants are isolated: VCF Automation project, Kubernetes namespace, NSX boundary, workload domain, or physical cluster
Which storage classes are appropriate for models, datasets, checkpoints, vector databases, logs, and ephemeral scratch
Whether network design supports GPU-to-GPU, storage, management, and tenant separation
How GPU cost is measured, allocated, reclaimed, and justified
The practical mental model is this:
Do not build “a GPU cluster.” Build an AI service contract, then place the cluster behind it.
Architecture at a Glance: The GPU Cluster Is Not the Service
The diagram below shows the model this article uses. The important part is the order.
The AI request should flow through a service contract first. That contract defines placement, tenancy, accelerator access, storage policy, network path, lifecycle ring, observability, and chargeback. The VCF workload domain and GPU cluster are implementation choices behind that contract.
What to notice: the GPU cluster sits below the service contract. That is intentional. If the platform team cannot describe the service contract, the cluster will absorb every decision by accident.
Scenario: The First AI Request Lands
Assume a realistic enterprise scenario.
The infrastructure team already runs VCF on VxRail with vSAN. The environment supports traditional VM workloads, some Kubernetes use cases, and standard lifecycle operations. A new AI initiative appears with three near-term requests:
A data science team wants GPU-backed notebooks and model testing.
An application team wants an internal inference endpoint for a business assistant.
A governance team wants sensitive data to stay on-premises or inside the private cloud boundary.
The initial instinct may be to create a GPU cluster and hand access to the requesting team.
That works for a lab.
It does not work as an enterprise pattern.
By the third or fourth tenant, the team will need answers to questions that should have been resolved before the first host was ordered:
Can two tenants share the same GPUs?
Who gets priority when GPU capacity is constrained?
Are GPU drivers and firmware patched with the rest of the VCF fleet?
Are training workloads allowed to consume the same storage fabric as inference workloads?
Can a tenant request full-GPU passthrough for performance reasons?
How is idle GPU capacity reclaimed?
Which costs are visible to the business unit?
Who owns model endpoint performance when the bottleneck is storage, network, or token throughput rather than raw GPU utilization?
VCF 9.1 adds capabilities that help with these questions, but it does not answer the operating model for you. Broadcom’s VCF 9.1 material highlights AI observability, multi-tenant AI infrastructure, Kubernetes support, high-speed networking, and accelerator ecosystem options. Those are platform ingredients. The infrastructure team still has to define the service boundaries.
Scope and Terminology Guardrails
This article is not a bill of materials, a sizing guide, or a deployment runbook. It is a decision framework.
TermHow this article uses itVCF instanceThe broader VMware Cloud Foundation environment, including management, workload domains, lifecycle, operations, automation, networking, and storage services.Workload domainA major placement and lifecycle boundary. For AI, this may become the most important decision after the hardware platform.GPU clusterA vSphere cluster with hosts that include supported GPU and network hardware for AI workloads.AI service classA platform-facing contract that describes how an AI workload consumes compute, GPU, storage, network, lifecycle, and cost controls.Tenant boundaryThe isolation model for teams, applications, projects, namespaces, networks, storage policies, and lifecycle blast radius.Accelerator access modelHow a workload gets GPU capacity: CPU-only, shared GPU, vGPU, MIG-style partitioning, DirectPath, or multi-GPU / multi-host acceleration.Lifecycle ringA grouping that controls how quickly hardware, firmware, drivers, ESXi, Kubernetes, and AI platform components move through updates.
A key guardrail: Kubernetes is not automatically the right interface for every AI workload. VKS may be the right model for platform-managed Kubernetes and containerized inference or AI services. Deep learning VMs may be better for teams that need a VM-centric workflow, specific drivers, or tighter control over runtime packaging. Lightweight inference may not need GPUs at all.
VCF Private AI Services documentation and announcements for VCF 9.1 discuss GPU-backed deep learning VMs, DirectPath-style access, and CPU-based inference options such as Llama.cpp in the context of observability and private AI services. The practical point is simple: decide the workload pattern before choosing the accelerator pattern.
Assumptions
This decision model assumes:
The organization is already using, or planning to use, VCF 9.1.
The infrastructure team is considering AI workloads on-premises or in a private cloud boundary.
VxRail and vSAN are part of the existing or target platform.
Some AI workloads may run on VKS, while others may run on VMs.
The team wants a repeatable operating model, not a one-off lab.
GPU supply is constrained enough that cost allocation and reclamation matter.
Security, data locality, and tenant separation are part of the design conversation.
Exact GPU, NIC, firmware, driver, server, vSAN, and VCF compatibility must be validated against the current Broadcom and Dell support matrices before procurement or deployment. VCF 9.1 expands options, but supportability still depends on the exact hardware and software combination.
Decision Criteria Before the First GPU Cluster
Before discussing placement, storage, and lifecycle, the infrastructure team needs a shared set of decision criteria.
A GPU-ready VCF design should be evaluated against these questions:
Decision criterionWhy it mattersPerformance isolationAI workloads can be noisy. A training job, inference burst, or data pipeline can consume GPU, CPU, memory, storage, and network resources in ways traditional VM teams may not expect.Lifecycle blast radiusGPU drivers, firmware, NICs, ESXi versions, Kubernetes versions, and AI runtime dependencies may move at different cadences.Tenant isolationAI workloads often mix sensitive data, proprietary models, shared platforms, and experimental code. Tenant separation must be explicit.Data gravityModels may be small, but datasets, embeddings, checkpoints, vector indexes, and logs can be large and persistent.SupportabilityHardware flexibility is useful only if support boundaries remain clean.Cost visibilityGPU capacity is too expensive to treat as generic cluster capacity.Operational ownershipAI infrastructure touches virtualization, storage, network, Kubernetes, security, automation, FinOps, and data science teams.Developer interfaceThe right abstraction may be a VM, Kubernetes namespace, API endpoint, catalog item, notebook, or managed model service.
The mistake is to optimize one column too early.
For example, a full GPU passthrough model may look ideal for performance but poor for sharing. A shared vGPU model may improve utilization but complicate support, scheduling, and tenant expectations. A dedicated AI workload domain may simplify lifecycle isolation but increase cost. A shared domain may reduce initial spend but create lifecycle and performance conflicts later.
Main Model: Define AI Service Classes First
A practical VCF AI architecture starts with service classes.
Service classes give the platform team a way to translate messy AI demand into infrastructure patterns. They also prevent every request from becoming a custom design.
A starting model might look like this:
AI service classTypical use caseDefault placementAccelerator modelLifecycle postureAI SandboxExperiments, notebooks, proof-of-concept modelsShared non-production domain or small GPU poolCPU, shared GPU, vGPU, or small GPU quotaFast-moving, reclaimableShared InferenceInternal assistants, small model endpoints, RAG servicesShared AI workload domainvGPU, MIG-style partitioning, or CPU inference if acceptableStable but flexibleDedicated InferenceProduction endpoint with predictable latency requirementsDedicated GPU cluster or dedicated resource poolFull GPU, vGPU, or pinned accelerator classControlled change windowFine-Tuning / TrainingModel tuning, batch training, larger distributed jobsDedicated AI workload domainMulti-GPU or multi-host GPU designIsolated lifecycle ringRegulated AISensitive data, strict audit, legal or compliance scopeDedicated tenant boundary, possibly dedicated domainDetermined by risk and performanceSlowest, most controlled
This does two things.
First, it gives tenants a language for requesting AI capacity without naming hardware. Second, it gives infrastructure teams a way to map requests into repeatable placement, lifecycle, storage, network, and cost policies.
Placement Model
Placement is the first major architectural decision because it affects every other decision.
There are three common patterns.
Add GPU Capacity to an Existing Workload Domain
This is attractive when demand is early, budget is limited, and the team wants to validate AI use cases quickly.
It works best when:
Workloads are experimental or low-risk.
GPU capacity is limited.
Lifecycle changes can align with the existing domain.
Tenants do not need strong physical or lifecycle isolation.
Storage and network patterns are not materially different from existing workloads.
The risk is that early convenience becomes permanent architecture. Once production AI workloads land in a shared domain, GPU driver updates, host maintenance, resource contention, and tenant expectations become harder to separate.
Create a Dedicated AI Workload Domain
A dedicated AI workload domain gives the team a cleaner lifecycle and placement boundary.
It works best when:
GPU hardware has a different lifecycle cadence.
AI workloads have distinct storage and network requirements.
Multiple tenants will consume a shared AI platform.
The business expects production inference or training.
Chargeback and governance need a visible boundary.
This is often the better long-term pattern for serious AI adoption. It may cost more up front, but it reduces ambiguity.
Use a Mixed VCF-on-VxRail Expansion Pattern Where Supported
For organizations already running VCF on VxRail, VCF 9.1 introduces a useful design consideration: Broadcom and Dell describe support for mixed hardware within a VCF-on-VxRail deployment using VxRail clusters and Dell vSAN Ready Node clusters, with guardrails. Dell’s VCF 9.1 material states that workload domains can use VxRail clusters or Dell vSAN Ready Node clusters, but hardware within a workload domain remains homogeneous; the management domain remains VxRail when VxRail is present; third-party vSAN Ready Nodes are not supported in VCF on VxRail.
That matters for AI.
If the existing VxRail environment is stable, but GPU-optimized expansion is easier through Dell vSAN Ready Nodes, the team may be able to build a dedicated AI workload domain without turning the entire environment into a one-size-fits-all hardware design. The tradeoff is operational: VxRail lifecycle remains integrated for VxRail clusters, while Dell vSAN Ready Nodes use standard VCF lifecycle management.
The decision is not simply “VxRail or Ready Node.” It is:
Which support and lifecycle boundary do we want AI hardware to live inside?
GPU Consumption Model
The second major decision is how workloads consume GPU capacity.
This is where infrastructure teams need to be careful. Not every AI workload deserves a full GPU. Not every AI workload belongs on shared GPU capacity. Not every workload needs Kubernetes.
VCF 9.1 Private AI Services material highlights DirectPath enablement for NVIDIA AI infrastructure, where a GPU can be assigned exclusively to a single VM. vSphere material for VCF 9.1 also describes Enhanced DirectPath I/O improvements, including support for certain operations without tearing down the AI workload, and notes that NVIDIA vGPU can use time slicing and MIG.
A practical decision table looks like this:
GPU access modelBest fitWatch forCPU-only inferenceLightweight models, dev/test, small internal tools, low-throughput servicesAvoid assuming every AI workload needs GPU. Validate latency and throughput first.vGPU / shared GPUShared inference, notebooks, smaller tenants, utilization-focused platformsRequires clear quota, scheduling, profile, and support rules.MIG-style partitioningBetter isolation on supported GPUs where tenants need partitioned accelerator capacityMust validate GPU model, driver, profile, and management support.DirectPath / full GPUHigh-performance inference, specialized workloads, dedicated model endpoints, driver-sensitive workloadsStrong performance isolation but lower sharing efficiency.Multi-GPU / multi-host accelerationTraining, fine-tuning, distributed workloads, large batch jobsRequires network, storage, scheduling, and lifecycle planning from day one.
The default platform stance should be:
Use CPU inference where it meets the service-level objective.
Use shared or partitioned GPU where utilization matters.
Use full GPU assignment where performance, isolation, or runtime constraints justify it.
Use dedicated placement for multi-GPU or multi-host workloads.
The wrong pattern is letting every tenant ask for “a GPU VM” without defining what class of GPU service they are actually buying.
Lifecycle Model
GPU infrastructure changes the lifecycle conversation.
Traditional VCF lifecycle planning already requires coordination across ESXi, vCenter, NSX, vSAN, firmware, drivers, and workload dependencies. AI adds more moving parts: GPU drivers, accelerator libraries, container images, model runtimes, Kubernetes versions, NIC drivers, RoCE settings, observability agents, and sometimes vendor-specific AI software.
VCF 9.1 includes lifecycle improvements such as ESX live patch behavior, firmware and driver validation against compatibility lists for vSAN clusters, and DRS improvements that help avoid contention during evacuation. Those are important platform improvements, but they do not remove the need for lifecycle rings.
A GPU-ready VCF environment should define at least four lifecycle rings:
Lifecycle ringPurposeLab / validation ringValidate GPU drivers, firmware, ESXi updates, vSAN behavior, VKS versions, and AI runtime dependencies before production exposure.Shared AI ringSupports common inference and sandbox workloads with predictable update windows.Dedicated production ringSupports production AI services with tighter change control and tenant communication.Regulated ringSupports workloads with audit, compliance, or data sensitivity constraints. Slower cadence, more validation, stricter rollback planning.
For VxRail environments, the lifecycle model must also account for the hardware boundary. In the VCF-on-VxRail mixed hardware model, VxRail clusters and Dell vSAN Ready Node clusters have different lifecycle paths, even when they are part of the same broader VCF deployment.
The practical rule:
Do not place AI workloads into a lifecycle boundary you cannot patch, evacuate, validate, and explain.
Tenancy Model
AI tenancy is not just authentication.
Tenancy includes:
Who can request AI capacity
Which projects can see which models
Which tenants can share GPU hardware
Which datasets can be mounted or accessed
Which networks a workload can reach
Which logs and metrics are visible
Which costs are charged back
Which tenant gets priority during contention
Who approves full-GPU or multi-GPU requests
VCF 9.1’s AI positioning includes multi-tenant infrastructure for AI workloads, with isolation and policy controls as part of the platform story. But the implementation decision is still yours.
A useful model is to define tenancy at five possible layers:
Tenancy layerWhen it is enoughWhen it is not enoughCatalog / projectGood for request routing, approvals, ownership, and quotasNot enough for runtime isolation by itselfKubernetes namespaceGood for app-level separation inside a shared clusterNot enough for strong lifecycle or physical isolationNSX network boundaryGood for traffic control and microsegmentationDoes not solve GPU scheduling or storage ownershipCluster boundaryGood for stronger performance and lifecycle separationMay be inefficient if every tenant gets a clusterWorkload domain boundaryStrongest operational and lifecycle boundaryMore expensive and heavier to operate
Most teams will need more than one layer.
For example:
AI sandbox: project + namespace + quota
Shared inference: project + namespace + NSX segmentation + GPU quota
Dedicated inference: project + dedicated resource pool or cluster + explicit GPU assignment
Regulated AI: dedicated project + network + storage policy + cluster or workload domain
The tenancy model should be visible in the service catalog. Tenants should not need to understand the entire VCF topology to request the right class of AI service.
Storage and Data Model
AI storage design is easy to underestimate.
The model file may not be the largest part of the environment. The real storage demand can come from:
Training datasets
Fine-tuning datasets
Embedding stores
Vector databases
Model checkpoints
Feature stores
Intermediate outputs
Inference logs
Audit records
Container images
Notebook workspaces
Temporary scratch space
VCF 9.1 includes notable vSAN updates, including global deduplication becoming generally available for supported vSAN HCI and storage clusters, a new compression approach, and expanded persistent volume scale for Kubernetes-oriented workloads. Broadcom’s vSAN material also describes dedupe and compression as cost-reduction capabilities, with specific support limitations such as exclusions for stretched clusters and two-node topologies.
That matters, but storage efficiency is not the whole design.
A GPU-ready VCF storage model should define storage classes by workload behavior:
Storage classExample dataDesign priorityModel artifact storageModel weights, runtime packages, approved versionsVersioning, integrity, access controlDataset storageTraining data, fine-tuning data, evaluation setsCapacity, throughput, governanceCheckpoint storageTraining checkpoints, intermediate outputsWrite performance, recoverabilityVector database storageEmbeddings, indexes, metadataLatency, consistency, backup strategyInference log storagePrompts, responses, telemetry, audit eventsRetention, privacy, cost controlScratch storageTemporary pipeline outputsPerformance and automated cleanup
Do not use one generic vSAN policy for all AI workloads.
Some workloads need performance. Some need capacity efficiency. Some need retention. Some need encryption and audit. Some need fast deletion because temporary AI data can become a governance problem if left unmanaged.
The storage policy should be part of the AI service class.
Network Model
AI network design has two separate conversations that often get collapsed into one.
The first is tenant connectivity: how users, applications, APIs, model endpoints, databases, and storage systems communicate.
The second is accelerator and data-path connectivity: how GPUs, hosts, storage, and high-speed NICs interact for performance-sensitive workloads.
VCF 9.1 networking material describes EVPN-VXLAN interoperability, flexible connectivity, Enhanced DirectPath I/O for NVIDIA ConnectX and BlueField devices, and GPUDirect RDMA for AI, ML, and distributed computing use cases. VKS 3.6 with VCF 9.1 also adds declarative multi-NIC support for Kubernetes nodes on supported backends, with primary and secondary NIC roles and important constraints such as control plane reachability and interface-change limitations.
A practical AI network model should separate at least these paths:
Network pathPurposeDesign questionManagementESXi, VCF components, lifecycle, monitoringDoes AI hardware introduce new management or support networks?Tenant accessApp-to-model, user-to-service, API trafficIs this isolated by tenant, application, or environment?Storage/dataDataset access, checkpoint writes, vector DB trafficDoes storage traffic compete with inference traffic?GPU / accelerator fabricGPU-to-GPU, RDMA, distributed workload pathsDoes the design require RoCE, GPUDirect RDMA, or specialized NIC behavior?Observability/exportMetrics, logs, traces, billing, model telemetryWhere do AI metrics leave the cluster, and who can see them?
The key decision is whether the first GPU cluster is for simple inference or a future distributed AI platform.
Those are different networks.
If the first cluster is designed only for basic VM-based inference, it may not support the traffic patterns needed for multi-host training later. If it is overdesigned for distributed training from day one, it may be too expensive and complex for the first wave of use cases.
Cost and Showback Model
GPU cost must be visible early.
If it is not, the platform team becomes the hidden subsidy for every AI experiment.
VCF 9.1 Operations material highlights enhanced cost and operations capabilities, including VKS cost details for showback and chargeback, real-time pricing, billing data aligned to FOCUS, and infrastructure observability improvements. Broadcom also describes FOCUS-normalized cost data as a way to compare private cloud cost with other environments and track AI/GPU workloads in a consistent format.
For AI, chargeback should not be limited to cluster ownership.
A useful cost model should include:
Cost signalWhy it mattersGPU-hourBaseline unit for scarce accelerator consumptionAllocated GPU versus used GPUShows waste when tenants reserve but do not consume capacityEndpoint-hourUseful for always-on inference servicesToken throughput and latency metricsHelps connect infrastructure cost to model-serving valueStorage consumed by AI classPrevents embeddings, checkpoints, and logs from becoming invisible costNetwork egress or high-speed fabric useMatters for distributed workloads and data-heavy pipelinesIdle capacity by tenantSupports reclamation policyService class costHelps business units choose sandbox, shared, dedicated, or regulated patterns
VCF Private AI Services material for VCF 9.1 describes AI observability dashboards with model-level and infrastructure-level metrics, including token throughput, time to first token, latency, GPU utilization, temperature, power, and memory usage. Those metrics are useful beyond troubleshooting. They are also the foundation for deciding whether a tenant needs more GPU, a different model profile, better storage, or a different service class.
A simple rule helps:
No tenant should receive scarce GPU capacity without an owner, quota, lease, utilization target, and cost signal.
GPU-Ready VCF Decision Matrix
Use this matrix before approving the first GPU cluster. It is intentionally operational, not just architectural.
Decision areaQuestion to answer before deploymentDefault recommendationRed flagPlacementDoes AI belong in an existing workload domain, a dedicated AI domain, or a separate hardware expansion path?Use a dedicated AI workload domain when production AI, multi-tenant GPU use, or separate lifecycle is expected.“We will add GPUs to the existing cluster and figure out governance later.”Hardware boundaryIs the GPU cluster VxRail, Dell vSAN Ready Node, or another supported platform path?Keep support boundaries clean; validate VCF-on-VxRail mixed hardware rules before ordering.Mixed hardware inside a workload domain without support validation.GPU accessWill workloads use CPU inference, shared GPU, vGPU, MIG, DirectPath, or multi-GPU?Define accelerator classes in the service catalog.Every tenant requests a full GPU by default.Workload interfaceWill tenants consume VMs, VKS clusters, namespaces, notebooks, APIs, or catalog items?Match interface to team maturity and workload pattern.Forcing Kubernetes for every workload or allowing unmanaged VM sprawl.Tenant boundaryIs isolation handled by project, namespace, network, cluster, or workload domain?Use layered tenancy: project + network + quota + runtime boundary.Tenants share GPUs and networks with no documented policy.Lifecycle ringHow are drivers, firmware, ESXi, VKS, GPU operators, and AI runtimes validated?Build lab, shared, production, and regulated lifecycle rings.Production AI runs on unvalidated driver and firmware changes.Storage classWhere do models, datasets, checkpoints, vector data, logs, and scratch data live?Create AI-specific storage policies and retention rules.All AI data lands on one generic datastore policy.Network pathAre management, tenant, storage, GPU fabric, and observability flows separated?Design network paths by traffic type before cluster deployment.Training, inference, storage, and management traffic compete on unclear paths.ObservabilityWhich model, GPU, storage, network, and tenant metrics are required?Track GPU utilization, latency, TTFT, token throughput, storage, and tenant cost.Platform team only monitors host CPU and memory.Cost modelHow are GPU, storage, endpoint, and idle costs allocated?Start with showback, move to chargeback where governance requires it.GPU capacity is free to tenants and invisible to finance.ReclamationWhat happens when a tenant stops using allocated GPU capacity?Require leases, expiration, and utilization review.Idle GPUs remain reserved because nobody owns reclamation.OwnershipWho owns runtime, model platform, infrastructure, cost, and security policy?Publish a RACI before onboarding tenants.Every issue becomes an infrastructure ticket.
Practical Service Contract Example
The platform team does not need to expose every VCF detail to tenants. It needs to expose enough structure to make requests governable.
This example is not a VCF API object. It is a practical worksheet the infrastructure team can translate into catalog items, project policies, namespace templates, storage policies, or approval workflows.
ai_service_class: shared-inference-standard
business_owner: claims-automation-team
technical_owner: ml-platform-team
environment: production
placement:
workload_domain: ai-prod-wld
cluster_policy: shared-gpu-cluster
lifecycle_ring: production-ai
accelerator:
access_model: shared-gpu
max_gpu_quota: 2
allow_full_gpu_directpath: false
gpu_lease_days: 30
reclamation_review: monthly
runtime:
interface: vks
namespace_model: tenant-namespace
approved_runtimes:
– model-serving-runtime-standard
– rag-api-runtime-standard
storage:
model_artifacts_policy: vsan-ai-model-standard
vector_database_policy: vsan-ai-latency-standard
logs_policy: vsan-ai-retention-30d
scratch_policy: vsan-ai-ephemeral
network:
tenant_segment: nsx-ai-claims-prod
secondary_data_network: enabled
external_access: approved-egress-only
observability:
required_metrics:
– gpu_utilization
– token_throughput
– time_to_first_token
– endpoint_latency
– storage_consumed
– idle_gpu_allocation
cost:
showback_unit:
– gpu_hour
– endpoint_hour
– storage_gb_month
chargeback_enabled: true
The important part is not the syntax. The important part is the decision coverage.
A service class like this forces the conversation before the cluster becomes a dumping ground for every AI experiment.
Operational Implications
A GPU-ready VCF design changes the daily operating model.
Capacity Planning Becomes Reservation-Aware
Traditional virtualization capacity planning often focuses on CPU, memory, datastore capacity, and cluster headroom. GPU capacity is different.
A tenant may reserve a GPU and use it poorly. Another may need short bursts. A production inference endpoint may need consistent latency rather than high average utilization. A training job may need multi-GPU scheduling for a limited window.
The platform team needs to track both allocation and actual use.
Lifecycle Planning Becomes More Cross-Functional
The infrastructure team may own VCF lifecycle. The data science team may own containers and models. The security team may own base images and vulnerability policy. The application team may own the endpoint. The platform engineering team may own the catalog.
If those ownership lines are not explicit, patching becomes negotiation.
Storage Operations Become Part of AI Governance
Model artifacts, prompts, responses, vector data, and logs may contain sensitive information. Retention and deletion matter.
AI storage is not only a performance problem. It is also a governance problem.
Network Teams Need to Join Earlier
Distributed AI, high-speed storage access, RDMA, and secondary Kubernetes interfaces are not afterthoughts. If the first GPU cluster may become the foundation for future training or high-throughput inference, network design needs to be part of the initial decision process.
FinOps Moves into the Private Cloud
Many organizations are used to cloud GPU cost being visible because the hyperscaler bill makes it visible. Private cloud GPU cost can disappear into capital expense unless the platform exposes showback or chargeback.
VCF Operations and FOCUS-aligned cost data can help, but the operating model still needs owners, service classes, and accountability.
Before the First GPU Cluster: Infrastructure Team Checklist
Use this checklist as the readiness gate.
AreaReadiness questionDemand intakeDo we know the first three AI workload patterns, not just the first hardware request?Service classesHave we defined sandbox, shared inference, dedicated inference, training, and regulated AI patterns?PlacementHave we chosen existing domain, dedicated AI domain, or supported hardware expansion path?Hardware supportHave GPU, NIC, firmware, driver, ESXi, vSAN, and OEM support matrices been validated?LifecycleDo we have a validation ring before production AI workloads land?TenancyDo we know whether isolation is by project, namespace, network, cluster, or workload domain?StorageAre model, dataset, checkpoint, vector, log, and scratch policies separate?NetworkAre management, tenant, storage, accelerator, and observability paths understood?ObservabilityCan we see GPU utilization, endpoint latency, TTFT, token throughput, storage growth, and idle allocation?CostCan we showback or chargeback GPU-hour, endpoint-hour, and storage consumption?ReclamationDo leases and utilization reviews exist for GPU allocations?OwnershipIs there a published RACI across infrastructure, platform, data science, app, security, and FinOps teams?
If the answer to most of these is “not yet,” the team is not ready for a production GPU cluster. It may be ready for a lab, but the lab should be labeled as a lab.
Conclusion: Build the AI Operating Model Before the GPU Island
VCF 9.1 gives infrastructure teams a stronger platform foundation for private AI workloads. The release brings more AI-oriented options across GPU access, Kubernetes, networking, observability, storage efficiency, and cost visibility.
But the platform does not make the operating model decision for you.
The infrastructure team still has to decide where AI workloads live, how GPUs are consumed, how tenants are isolated, how lifecycle risk is contained, how storage policies map to AI data types, how network paths support accelerator-heavy workloads, and how costs are exposed.
The first GPU cluster will set expectations.
If it is deployed as a one-off capacity answer, every future tenant will inherit ambiguity. If it is deployed behind a clear AI service contract, it becomes the first building block of a private AI platform.
That is the decision that matters.
Do not start with the GPU cluster.
Start with the service model the cluster is supposed to serve.
External Sources
Use these sources for publish-time validation of the VCF 9.1, VxRail, vSAN, VKS, NSX, Private AI, lifecycle, observability, and cost-management claims referenced in the article.
Broadcom: Broadcom Announces VMware Cloud Foundation 9.1https://news.broadcom.com/releases/broadcom-announces-vmware-cloud-foundation-9-1Used for: VCF 9.1 positioning, private AI claims, Kubernetes and multi-tenant AI infrastructure messaging, cost and operational improvement claims.
VMware Cloud Foundation Blog: Streamline, Simplify, and Protect All Your AI Workloads with VCF 9.1https://blogs.vmware.com/cloud-foundation/2026/05/05/streamline-simplify-and-protect-all-your-ai-workloads-with-vcf-9-1/Used for: VCF Private AI Services, AI observability dashboards, deep learning VM patterns, GPU-backed workloads, DirectPath-style access, CPU inference references, and AI infrastructure metrics.
VMware Cloud Foundation Blog: What’s New with vSphere 9.1https://blogs.vmware.com/cloud-foundation/2026/05/12/whats-new-with-vsphere-9-1/Used for: vSphere 9.1 lifecycle updates, Enhanced DirectPath I/O, GPU-related platform behavior, DRS improvements, and firmware/driver validation references.
VMware Cloud Foundation Blog: VMware Cloud Foundation 9.1 Path for Dell VxRail Customershttps://blogs.vmware.com/cloud-foundation/2026/06/16/vmware-cloud-foundation-9-1-path-for-dell-vxrail-customers/Used for: VCF-on-VxRail mixed hardware guidance, VxRail and Dell vSAN Ready Node workload domain boundaries, homogeneous hardware requirements inside workload domains, management domain constraints, and lifecycle/support model differences.
VMware Cloud Foundation Blog: vSAN Compression and Global Deduplication in VCF 9.1https://blogs.vmware.com/cloud-foundation/2026/05/07/vsan-compression-and-global-deduplication-in-vcf-9-1/Used for: vSAN global deduplication, compression, supported topology limitations, storage efficiency claims, and persistent volume scale references.
VMware Cloud Foundation Blog: Simplify Workload Connectivity and Enhance Network Scale and Performance with VCF 9.1https://blogs.vmware.com/cloud-foundation/2026/05/05/simplify-workload-connectivity-and-enhance-network-scale-and-performance-with-vcf-9-1/Used for: NSX and VCF 9.1 networking updates, EVPN-VXLAN interoperability, Enhanced DirectPath I/O for NVIDIA ConnectX and BlueField devices, and GPUDirect RDMA references.
VMware Cloud Foundation Blog: VCF 9.1 Brings Multi-Network Support to vSphere Kubernetes Servicehttps://blogs.vmware.com/cloud-foundation/2026/06/02/vcf-9-1-brings-multi-network-support-to-vsphere-kubernetes-service/Used for: VKS 3.6 multi-network support, Kubernetes node multi-NIC behavior, primary and secondary NIC roles, and operational constraints.
VMware Cloud Foundation Blog: Deploy Modern Apps Faster, Scale Smarter, and Lower Your TCO with VKS on VCF 9.1https://blogs.vmware.com/cloud-foundation/2026/05/05/deploy-modern-apps-faster-scale-smarter-and-lower-your-tco-with-vks-on-vcf-9-1/Used for: VKS 3.6 and VCF 9.1 modern application platform claims, Kubernetes placement, node pool improvements, and TCO-related vendor claims.
VMware Cloud Foundation Blog: Scale, Simplify, and Secure Your Private Cloud Operations with VCF 9.1https://blogs.vmware.com/cloud-foundation/2026/05/05/scale-simplify-and-secure-your-private-cloud-operations-with-vcf-9-1/Used for: VCF Operations updates, cost visibility, showback and chargeback, real-time pricing, VKS cost details, observability, and private cloud operations claims.
VMware Cloud Foundation Blog: Unified Cloud Value: Accelerating Cloud FinOps with VCF 9.1 and FOCUShttps://blogs.vmware.com/cloud-foundation/2026/06/03/unified-cloud-value-accelerating-cloud-fin-ops-with-vcf-9-1-and-focus/Used for: FOCUS-aligned cost data, FinOps normalization, private cloud cost comparison, and AI/GPU workload cost-tracking context.
Introduction to VMware Cloud Foundation Operations: Why VCF Operations Is Critical Today
TL;DR VCF Operations is no longer just an observability tool sitting beside the VMware Cloud Foundation stack. In the VCF 9.x operating…
The post VCF 9.1 for AI Workloads: The Decisions Infrastructure Teams Need to Make Before the First GPU Cluster appeared first on Digital Thought Disruption.