AI demand rarely arrives as a clean infrastructure request.

It usually starts as a conversation. A data science team has a model they want to test. A business unit wants an internal assistant. A security team wants retrieval-augmented generation close to sensitive data. A platform team is asked whether the existing private cloud can “just add GPUs.”

That is where many infrastructure teams get pulled into the wrong first decision.

The first decision is not which GPU server to buy, whether the workload should run on Kubernetes or a VM, or whether the cluster should be VxRail, Dell vSAN Ready Nodes, or a dedicated GPU island.

The first decision is the operating model.

VCF 9.1 gives infrastructure teams more options for private AI, GPU acceleration, Kubernetes, vSAN efficiency, observability, and cost management. Broadcom has positioned VCF 9.1 as a private cloud platform for modern workloads, including production AI, with support for AI-oriented infrastructure, Kubernetes, multi-tenant isolation, high-speed networking, and accelerator ecosystem integrations.

That does not remove the design work. It actually makes the design work more important.

If the platform team does not define placement, lifecycle, tenancy, storage, network, and cost models before the first GPU cluster lands, the first successful AI workload can become the pattern everyone else inherits.

That is dangerous.

A GPU cluster is expensive, scarce, operationally sensitive, and politically visible. Once it becomes shared infrastructure, every missing decision turns into a ticket, an exception, or an outage conversation.

This article lays out the decision model infrastructure teams should build before deploying the first GPU-ready VCF cluster.

TL;DR

VCF 9.1 can support a stronger private AI architecture, but the GPU cluster should be the output of the design process, not the starting point.

Before deploying the first GPU cluster, infrastructure teams need to decide:

• Where AI workloads belong: existing workload domain, dedicated AI workload domain, or separate expansion path
• How GPUs are consumed: full-device passthrough, vGPU, MIG-style partitioning, CPU inference, or Kubernetes node pools
• Who owns lifecycle risk: platform, virtualization, VxRail, data science, application, or shared platform engineering
• How tenants are isolated: VCF Automation project, Kubernetes namespace, NSX boundary, workload domain, or physical cluster
• Which storage classes are appropriate for models, datasets, checkpoints, vector databases, logs, and ephemeral scratch
• Whether network design supports GPU-to-GPU, storage, management, and tenant separation
• How GPU cost is measured, allocated, reclaimed, and justified

The practical mental model is this:

Do not build “a GPU cluster.” Build an AI service contract, then place the cluster behind it.

Architecture at a Glance: The GPU Cluster Is Not the Service

The diagram below shows the model this article uses. The important part is the order.

The AI request should flow through a service contract first. That contract defines placement, tenancy, accelerator access, storage policy, network path, lifecycle ring, observability, and chargeback. The VCF workload domain and GPU cluster are implementation choices behind that contract.

What to notice: the GPU cluster sits below the service contract. That is intentional. If the platform team cannot describe the service contract, the cluster will absorb every decision by accident.

Scenario: The First AI Request Lands

Assume a realistic enterprise scenario.

The infrastructure team already runs VCF on VxRail with vSAN. The environment supports traditional VM workloads, some Kubernetes use cases, and standard lifecycle operations. A new AI initiative appears with three near-term requests:

• A data science team wants GPU-backed notebooks and model testing.
• An application team wants an internal inference endpoint for a business assistant.
• A governance team wants sensitive data to stay on-premises or inside the private cloud boundary.

The initial instinct may be to create a GPU cluster and hand access to the requesting team.

That works for a lab.

It does not work as an enterprise pattern.

By the third or fourth tenant, the team will need answers to questions that should have been resolved before the first host was ordered:

• Can two tenants share the same GPUs?
• Who gets priority when GPU capacity is constrained?
• Are GPU drivers and firmware patched with the rest of the VCF fleet?
• Are training workloads allowed to consume the same storage fabric as inference workloads?
• Can a tenant request full-GPU passthrough for performance reasons?
• How is idle GPU capacity reclaimed?
• Which costs are visible to the business unit?
• Who owns model endpoint performance when the bottleneck is storage, network, or token throughput rather than raw GPU utilization?

VCF 9.1 adds capabilities that help with these questions, but it does not answer the operating model for you. Broadcom’s VCF 9.1 material highlights AI observability, multi-tenant AI infrastructure, Kubernetes support, high-speed networking, and accelerator ecosystem options. Those are platform ingredients. The infrastructure team still has to define the service boundaries.

Scope and Terminology Guardrails

This article is not a bill of materials, a sizing guide, or a deployment runbook. It is a decision framework.

Term	How this article uses it
VCF instance	The broader VMware Cloud Foundation environment, including management, workload domains, lifecycle, operations, automation, networking, and storage services.
Workload domain	A major placement and lifecycle boundary. For AI, this may become the most important decision after the hardware platform.
GPU cluster	A vSphere cluster with hosts that include supported GPU and network hardware for AI workloads.
AI service class	A platform-facing contract that describes how an AI workload consumes compute, GPU, storage, network, lifecycle, and cost controls.
Tenant boundary	The isolation model for teams, applications, projects, namespaces, networks, storage policies, and lifecycle blast radius.
Accelerator access model	How a workload gets GPU capacity: CPU-only, shared GPU, vGPU, MIG-style partitioning, DirectPath, or multi-GPU / multi-host acceleration.
Lifecycle ring	A grouping that controls how quickly hardware, firmware, drivers, ESXi, Kubernetes, and AI platform components move through updates.

A key guardrail: Kubernetes is not automatically the right interface for every AI workload. VKS may be the right model for platform-managed Kubernetes and containerized inference or AI services. Deep learning VMs may be better for teams that need a VM-centric workflow, specific drivers, or tighter control over runtime packaging. Lightweight inference may not need GPUs at all.

VCF Private AI Services documentation and announcements for VCF 9.1 discuss GPU-backed deep learning VMs, DirectPath-style access, and CPU-based inference options such as Llama.cpp in the context of observability and private AI services. The practical point is simple: decide the workload pattern before choosing the accelerator pattern.

Assumptions

This decision model assumes:

• The organization is already using, or planning to use, VCF 9.1.
• The infrastructure team is considering AI workloads on-premises or in a private cloud boundary.
• VxRail and vSAN are part of the existing or target platform.
• Some AI workloads may run on VKS, while others may run on VMs.
• The team wants a repeatable operating model, not a one-off lab.
• GPU supply is constrained enough that cost allocation and reclamation matter.
• Security, data locality, and tenant separation are part of the design conversation.

Exact GPU, NIC, firmware, driver, server, vSAN, and VCF compatibility must be validated against the current Broadcom and Dell support matrices before procurement or deployment. VCF 9.1 expands options, but supportability still depends on the exact hardware and software combination.

Decision Criteria Before the First GPU Cluster

Before discussing placement, storage, and lifecycle, the infrastructure team needs a shared set of decision criteria.

A GPU-ready VCF design should be evaluated against these questions:

Decision criterion	Why it matters
Performance isolation	AI workloads can be noisy. A training job, inference burst, or data pipeline can consume GPU, CPU, memory, storage, and network resources in ways traditional VM teams may not expect.
Lifecycle blast radius	GPU drivers, firmware, NICs, ESXi versions, Kubernetes versions, and AI runtime dependencies may move at different cadences.
Tenant isolation	AI workloads often mix sensitive data, proprietary models, shared platforms, and experimental code. Tenant separation must be explicit.
Data gravity	Models may be small, but datasets, embeddings, checkpoints, vector indexes, and logs can be large and persistent.
Supportability	Hardware flexibility is useful only if support boundaries remain clean.
Cost visibility	GPU capacity is too expensive to treat as generic cluster capacity.
Operational ownership	AI infrastructure touches virtualization, storage, network, Kubernetes, security, automation, FinOps, and data science teams.
Developer interface	The right abstraction may be a VM, Kubernetes namespace, API endpoint, catalog item, notebook, or managed model service.

The mistake is to optimize one column too early.

For example, a full GPU passthrough model may look ideal for performance but poor for sharing. A shared vGPU model may improve utilization but complicate support, scheduling, and tenant expectations. A dedicated AI workload domain may simplify lifecycle isolation but increase cost. A shared domain may reduce initial spend but create lifecycle and performance conflicts later.

Main Model: Define AI Service Classes First

A practical VCF AI architecture starts with service classes.

Service classes give the platform team a way to translate messy AI demand into infrastructure patterns. They also prevent every request from becoming a custom design.

A starting model might look like this:

AI service class	Typical use case	Default placement	Accelerator model	Lifecycle posture
AI Sandbox	Experiments, notebooks, proof-of-concept models	Shared non-production domain or small GPU pool	CPU, shared GPU, vGPU, or small GPU quota	Fast-moving, reclaimable
Shared Inference	Internal assistants, small model endpoints, RAG services	Shared AI workload domain	vGPU, MIG-style partitioning, or CPU inference if acceptable	Stable but flexible
Dedicated Inference	Production endpoint with predictable latency requirements	Dedicated GPU cluster or dedicated resource pool	Full GPU, vGPU, or pinned accelerator class	Controlled change window
Fine-Tuning / Training	Model tuning, batch training, larger distributed jobs	Dedicated AI workload domain	Multi-GPU or multi-host GPU design	Isolated lifecycle ring
Regulated AI	Sensitive data, strict audit, legal or compliance scope	Dedicated tenant boundary, possibly dedicated domain	Determined by risk and performance	Slowest, most controlled

This does two things.

First, it gives tenants a language for requesting AI capacity without naming hardware. Second, it gives infrastructure teams a way to map requests into repeatable placement, lifecycle, storage, network, and cost policies.

Placement Model

Placement is the first major architectural decision because it affects every other decision.

There are three common patterns.

Add GPU Capacity to an Existing Workload Domain

This is attractive when demand is early, budget is limited, and the team wants to validate AI use cases quickly.

It works best when:

• Workloads are experimental or low-risk.
• GPU capacity is limited.
• Lifecycle changes can align with the existing domain.
• Tenants do not need strong physical or lifecycle isolation.
• Storage and network patterns are not materially different from existing workloads.

The risk is that early convenience becomes permanent architecture. Once production AI workloads land in a shared domain, GPU driver updates, host maintenance, resource contention, and tenant expectations become harder to separate.

Create a Dedicated AI Workload Domain

A dedicated AI workload domain gives the team a cleaner lifecycle and placement boundary.

It works best when:

• GPU hardware has a different lifecycle cadence.
• AI workloads have distinct storage and network requirements.
• Multiple tenants will consume a shared AI platform.
• The business expects production inference or training.
• Chargeback and governance need a visible boundary.

This is often the better long-term pattern for serious AI adoption. It may cost more up front, but it reduces ambiguity.

Use a Mixed VCF-on-VxRail Expansion Pattern Where Supported

For organizations already running VCF on VxRail, VCF 9.1 introduces a useful design consideration: Broadcom and Dell describe support for mixed hardware within a VCF-on-VxRail deployment using VxRail clusters and Dell vSAN Ready Node clusters, with guardrails. Dell’s VCF 9.1 material states that workload domains can use VxRail clusters or Dell vSAN Ready Node clusters, but hardware within a workload domain remains homogeneous; the management domain remains VxRail when VxRail is present; third-party vSAN Ready Nodes are not supported in VCF on VxRail.

That matters for AI.

If the existing VxRail environment is stable, but GPU-optimized expansion is easier through Dell vSAN Ready Nodes, the team may be able to build a dedicated AI workload domain without turning the entire environment into a one-size-fits-all hardware design. The tradeoff is operational: VxRail lifecycle remains integrated for VxRail clusters, while Dell vSAN Ready Nodes use standard VCF lifecycle management.

The decision is not simply “VxRail or Ready Node.” It is:

Which support and lifecycle boundary do we want AI hardware to live inside?

GPU Consumption Model

The second major decision is how workloads consume GPU capacity.

This is where infrastructure teams need to be careful. Not every AI workload deserves a full GPU. Not every AI workload belongs on shared GPU capacity. Not every workload needs Kubernetes.

VCF 9.1 Private AI Services material highlights DirectPath enablement for NVIDIA AI infrastructure, where a GPU can be assigned exclusively to a single VM. vSphere material for VCF 9.1 also describes Enhanced DirectPath I/O improvements, including support for certain operations without tearing down the AI workload, and notes that NVIDIA vGPU can use time slicing and MIG.

A practical decision table looks like this:

GPU access model	Best fit	Watch for
CPU-only inference	Lightweight models, dev/test, small internal tools, low-throughput services	Avoid assuming every AI workload needs GPU. Validate latency and throughput first.
vGPU / shared GPU	Shared inference, notebooks, smaller tenants, utilization-focused platforms	Requires clear quota, scheduling, profile, and support rules.
MIG-style partitioning	Better isolation on supported GPUs where tenants need partitioned accelerator capacity	Must validate GPU model, driver, profile, and management support.
DirectPath / full GPU	High-performance inference, specialized workloads, dedicated model endpoints, driver-sensitive workloads	Strong performance isolation but lower sharing efficiency.
Multi-GPU / multi-host acceleration	Training, fine-tuning, distributed workloads, large batch jobs	Requires network, storage, scheduling, and lifecycle planning from day one.

The default platform stance should be:

• Use CPU inference where it meets the service-level objective.
• Use shared or partitioned GPU where utilization matters.
• Use full GPU assignment where performance, isolation, or runtime constraints justify it.
• Use dedicated placement for multi-GPU or multi-host workloads.

The wrong pattern is letting every tenant ask for “a GPU VM” without defining what class of GPU service they are actually buying.

Lifecycle Model

GPU infrastructure changes the lifecycle conversation.

Traditional VCF lifecycle planning already requires coordination across ESXi, vCenter, NSX, vSAN, firmware, drivers, and workload dependencies. AI adds more moving parts: GPU drivers, accelerator libraries, container images, model runtimes, Kubernetes versions, NIC drivers, RoCE settings, observability agents, and sometimes vendor-specific AI software.

VCF 9.1 includes lifecycle improvements such as ESX live patch behavior, firmware and driver validation against compatibility lists for vSAN clusters, and DRS improvements that help avoid contention during evacuation. Those are important platform improvements, but they do not remove the need for lifecycle rings.

A GPU-ready VCF environment should define at least four lifecycle rings:

Lifecycle ring	Purpose
Lab / validation ring	Validate GPU drivers, firmware, ESXi updates, vSAN behavior, VKS versions, and AI runtime dependencies before production exposure.
Shared AI ring	Supports common inference and sandbox workloads with predictable update windows.
Dedicated production ring	Supports production AI services with tighter change control and tenant communication.
Regulated ring	Supports workloads with audit, compliance, or data sensitivity constraints. Slower cadence, more validation, stricter rollback planning.

For VxRail environments, the lifecycle model must also account for the hardware boundary. In the VCF-on-VxRail mixed hardware model, VxRail clusters and Dell vSAN Ready Node clusters have different lifecycle paths, even when they are part of the same broader VCF deployment.

The practical rule:

Do not place AI workloads into a lifecycle boundary you cannot patch, evacuate, validate, and explain.

Tenancy Model

AI tenancy is not just authentication.

Tenancy includes:

• Who can request AI capacity
• Which projects can see which models
• Which tenants can share GPU hardware
• Which datasets can be mounted or accessed
• Which networks a workload can reach
• Which logs and metrics are visible
• Which costs are charged back
• Which tenant gets priority during contention
• Who approves full-GPU or multi-GPU requests

VCF 9.1’s AI positioning includes multi-tenant infrastructure for AI workloads, with isolation and policy controls as part of the platform story. But the implementation decision is still yours.

A useful model is to define tenancy at five possible layers:

Tenancy layer	When it is enough	When it is not enough
Catalog / project	Good for request routing, approvals, ownership, and quotas	Not enough for runtime isolation by itself
Kubernetes namespace	Good for app-level separation inside a shared cluster	Not enough for strong lifecycle or physical isolation
NSX network boundary	Good for traffic control and microsegmentation	Does not solve GPU scheduling or storage ownership
Cluster boundary	Good for stronger performance and lifecycle separation	May be inefficient if every tenant gets a cluster
Workload domain boundary	Strongest operational and lifecycle boundary	More expensive and heavier to operate

Most teams will need more than one layer.

For example:

• AI sandbox: project + namespace + quota
• Shared inference: project + namespace + NSX segmentation + GPU quota
• Dedicated inference: project + dedicated resource pool or cluster + explicit GPU assignment
• Regulated AI: dedicated project + network + storage policy + cluster or workload domain

The tenancy model should be visible in the service catalog. Tenants should not need to understand the entire VCF topology to request the right class of AI service.

Storage and Data Model

AI storage design is easy to underestimate.

The model file may not be the largest part of the environment. The real storage demand can come from:

• Training datasets
• Fine-tuning datasets
• Embedding stores
• Vector databases
• Model checkpoints
• Feature stores
• Intermediate outputs
• Inference logs
• Audit records
• Container images
• Notebook workspaces
• Temporary scratch space

VCF 9.1 includes notable vSAN updates, including global deduplication becoming generally available for supported vSAN HCI and storage clusters, a new compression approach, and expanded persistent volume scale for Kubernetes-oriented workloads. Broadcom’s vSAN material also describes dedupe and compression as cost-reduction capabilities, with specific support limitations such as exclusions for stretched clusters and two-node topologies.

That matters, but storage efficiency is not the whole design.

A GPU-ready VCF storage model should define storage classes by workload behavior:

Storage class	Example data	Design priority
Model artifact storage	Model weights, runtime packages, approved versions	Versioning, integrity, access control
Dataset storage	Training data, fine-tuning data, evaluation sets	Capacity, throughput, governance
Checkpoint storage	Training checkpoints, intermediate outputs	Write performance, recoverability
Vector database storage	Embeddings, indexes, metadata	Latency, consistency, backup strategy
Inference log storage	Prompts, responses, telemetry, audit events	Retention, privacy, cost control
Scratch storage	Temporary pipeline outputs	Performance and automated cleanup

Do not use one generic vSAN policy for all AI workloads.

Some workloads need performance. Some need capacity efficiency. Some need retention. Some need encryption and audit. Some need fast deletion because temporary AI data can become a governance problem if left unmanaged.

The storage policy should be part of the AI service class.

Network Model

AI network design has two separate conversations that often get collapsed into one.

The first is tenant connectivity: how users, applications, APIs, model endpoints, databases, and storage systems communicate.

The second is accelerator and data-path connectivity: how GPUs, hosts, storage, and high-speed NICs interact for performance-sensitive workloads.

VCF 9.1 networking material describes EVPN-VXLAN interoperability, flexible connectivity, Enhanced DirectPath I/O for NVIDIA ConnectX and BlueField devices, and GPUDirect RDMA for AI, ML, and distributed computing use cases. VKS 3.6 with VCF 9.1 also adds declarative multi-NIC support for Kubernetes nodes on supported backends, with primary and secondary NIC roles and important constraints such as control plane reachability and interface-change limitations.

A practical AI network model should separate at least these paths:

Network path	Purpose	Design question
Management	ESXi, VCF components, lifecycle, monitoring	Does AI hardware introduce new management or support networks?
Tenant access	App-to-model, user-to-service, API traffic	Is this isolated by tenant, application, or environment?
Storage/data	Dataset access, checkpoint writes, vector DB traffic	Does storage traffic compete with inference traffic?
GPU / accelerator fabric	GPU-to-GPU, RDMA, distributed workload paths	Does the design require RoCE, GPUDirect RDMA, or specialized NIC behavior?
Observability/export	Metrics, logs, traces, billing, model telemetry	Where do AI metrics leave the cluster, and who can see them?

The key decision is whether the first GPU cluster is for simple inference or a future distributed AI platform.

Those are different networks.

If the first cluster is designed only for basic VM-based inference, it may not support the traffic patterns needed for multi-host training later. If it is overdesigned for distributed training from day one, it may be too expensive and complex for the first wave of use cases.

Cost and Showback Model

GPU cost must be visible early.

If it is not, the platform team becomes the hidden subsidy for every AI experiment.

VCF 9.1 Operations material highlights enhanced cost and operations capabilities, including VKS cost details for showback and chargeback, real-time pricing, billing data aligned to FOCUS, and infrastructure observability improvements. Broadcom also describes FOCUS-normalized cost data as a way to compare private cloud cost with other environments and track AI/GPU workloads in a consistent format.

For AI, chargeback should not be limited to cluster ownership.

A useful cost model should include:

Cost signal	Why it matters
GPU-hour	Baseline unit for scarce accelerator consumption
Allocated GPU versus used GPU	Shows waste when tenants reserve but do not consume capacity
Endpoint-hour	Useful for always-on inference services
Token throughput and latency metrics	Helps connect infrastructure cost to model-serving value
Storage consumed by AI class	Prevents embeddings, checkpoints, and logs from becoming invisible cost
Network egress or high-speed fabric use	Matters for distributed workloads and data-heavy pipelines
Idle capacity by tenant	Supports reclamation policy
Service class cost	Helps business units choose sandbox, shared, dedicated, or regulated patterns

VCF Private AI Services material for VCF 9.1 describes AI observability dashboards with model-level and infrastructure-level metrics, including token throughput, time to first token, latency, GPU utilization, temperature, power, and memory usage. Those metrics are useful beyond troubleshooting. They are also the foundation for deciding whether a tenant needs more GPU, a different model profile, better storage, or a different service class.

A simple rule helps:

No tenant should receive scarce GPU capacity without an owner, quota, lease, utilization target, and cost signal.

GPU-Ready VCF Decision Matrix

Use this matrix before approving the first GPU cluster. It is intentionally operational, not just architectural.

Decision area	Question to answer before deployment	Default recommendation	Red flag
Placement	Does AI belong in an existing workload domain, a dedicated AI domain, or a separate hardware expansion path?	Use a dedicated AI workload domain when production AI, multi-tenant GPU use, or separate lifecycle is expected.	“We will add GPUs to the existing cluster and figure out governance later.”
Hardware boundary	Is the GPU cluster VxRail, Dell vSAN Ready Node, or another supported platform path?	Keep support boundaries clean; validate VCF-on-VxRail mixed hardware rules before ordering.	Mixed hardware inside a workload domain without support validation.
GPU access	Will workloads use CPU inference, shared GPU, vGPU, MIG, DirectPath, or multi-GPU?	Define accelerator classes in the service catalog.	Every tenant requests a full GPU by default.
Workload interface	Will tenants consume VMs, VKS clusters, namespaces, notebooks, APIs, or catalog items?	Match interface to team maturity and workload pattern.	Forcing Kubernetes for every workload or allowing unmanaged VM sprawl.
Tenant boundary	Is isolation handled by project, namespace, network, cluster, or workload domain?	Use layered tenancy: project + network + quota + runtime boundary.	Tenants share GPUs and networks with no documented policy.
Lifecycle ring	How are drivers, firmware, ESXi, VKS, GPU operators, and AI runtimes validated?	Build lab, shared, production, and regulated lifecycle rings.	Production AI runs on unvalidated driver and firmware changes.
Storage class	Where do models, datasets, checkpoints, vector data, logs, and scratch data live?	Create AI-specific storage policies and retention rules.	All AI data lands on one generic datastore policy.
Network path	Are management, tenant, storage, GPU fabric, and observability flows separated?	Design network paths by traffic type before cluster deployment.	Training, inference, storage, and management traffic compete on unclear paths.
Observability	Which model, GPU, storage, network, and tenant metrics are required?	Track GPU utilization, latency, TTFT, token throughput, storage, and tenant cost.	Platform team only monitors host CPU and memory.
Cost model	How are GPU, storage, endpoint, and idle costs allocated?	Start with showback, move to chargeback where governance requires it.	GPU capacity is free to tenants and invisible to finance.
Reclamation	What happens when a tenant stops using allocated GPU capacity?	Require leases, expiration, and utilization review.	Idle GPUs remain reserved because nobody owns reclamation.
Ownership	Who owns runtime, model platform, infrastructure, cost, and security policy?	Publish a RACI before onboarding tenants.	Every issue becomes an infrastructure ticket.

Practical Service Contract Example

The platform team does not need to expose every VCF detail to tenants. It needs to expose enough structure to make requests governable.

This example is not a VCF API object. It is a practical worksheet the infrastructure team can translate into catalog items, project policies, namespace templates, storage policies, or approval workflows.

ai_service_class: shared-inference-standard

business_owner: claims-automation-team
technical_owner: ml-platform-team
environment: production

placement:
  workload_domain: ai-prod-wld
  cluster_policy: shared-gpu-cluster
  lifecycle_ring: production-ai

accelerator:
  access_model: shared-gpu
  max_gpu_quota: 2
  allow_full_gpu_directpath: false
  gpu_lease_days: 30
  reclamation_review: monthly

runtime:
  interface: vks
  namespace_model: tenant-namespace
  approved_runtimes:
    - model-serving-runtime-standard
    - rag-api-runtime-standard

storage:
  model_artifacts_policy: vsan-ai-model-standard
  vector_database_policy: vsan-ai-latency-standard
  logs_policy: vsan-ai-retention-30d
  scratch_policy: vsan-ai-ephemeral

network:
  tenant_segment: nsx-ai-claims-prod
  secondary_data_network: enabled
  external_access: approved-egress-only

observability:
  required_metrics:
    - gpu_utilization
    - token_throughput
    - time_to_first_token
    - endpoint_latency
    - storage_consumed
    - idle_gpu_allocation

cost:
  showback_unit:
    - gpu_hour
    - endpoint_hour
    - storage_gb_month
  chargeback_enabled: true

The important part is not the syntax. The important part is the decision coverage.

A service class like this forces the conversation before the cluster becomes a dumping ground for every AI experiment.

Operational Implications

A GPU-ready VCF design changes the daily operating model.

Capacity Planning Becomes Reservation-Aware

Traditional virtualization capacity planning often focuses on CPU, memory, datastore capacity, and cluster headroom. GPU capacity is different.

A tenant may reserve a GPU and use it poorly. Another may need short bursts. A production inference endpoint may need consistent latency rather than high average utilization. A training job may need multi-GPU scheduling for a limited window.

The platform team needs to track both allocation and actual use.

Lifecycle Planning Becomes More Cross-Functional

The infrastructure team may own VCF lifecycle. The data science team may own containers and models. The security team may own base images and vulnerability policy. The application team may own the endpoint. The platform engineering team may own the catalog.

If those ownership lines are not explicit, patching becomes negotiation.

Storage Operations Become Part of AI Governance

Model artifacts, prompts, responses, vector data, and logs may contain sensitive information. Retention and deletion matter.

AI storage is not only a performance problem. It is also a governance problem.

Network Teams Need to Join Earlier

Distributed AI, high-speed storage access, RDMA, and secondary Kubernetes interfaces are not afterthoughts. If the first GPU cluster may become the foundation for future training or high-throughput inference, network design needs to be part of the initial decision process.

FinOps Moves into the Private Cloud

Many organizations are used to cloud GPU cost being visible because the hyperscaler bill makes it visible. Private cloud GPU cost can disappear into capital expense unless the platform exposes showback or chargeback.

VCF Operations and FOCUS-aligned cost data can help, but the operating model still needs owners, service classes, and accountability.

Before the First GPU Cluster: Infrastructure Team Checklist

Use this checklist as the readiness gate.

Area	Readiness question
Demand intake	Do we know the first three AI workload patterns, not just the first hardware request?
Service classes	Have we defined sandbox, shared inference, dedicated inference, training, and regulated AI patterns?
Placement	Have we chosen existing domain, dedicated AI domain, or supported hardware expansion path?
Hardware support	Have GPU, NIC, firmware, driver, ESXi, vSAN, and OEM support matrices been validated?
Lifecycle	Do we have a validation ring before production AI workloads land?
Tenancy	Do we know whether isolation is by project, namespace, network, cluster, or workload domain?
Storage	Are model, dataset, checkpoint, vector, log, and scratch policies separate?
Network	Are management, tenant, storage, accelerator, and observability paths understood?
Observability	Can we see GPU utilization, endpoint latency, TTFT, token throughput, storage growth, and idle allocation?
Cost	Can we showback or chargeback GPU-hour, endpoint-hour, and storage consumption?
Reclamation	Do leases and utilization reviews exist for GPU allocations?
Ownership	Is there a published RACI across infrastructure, platform, data science, app, security, and FinOps teams?

If the answer to most of these is “not yet,” the team is not ready for a production GPU cluster. It may be ready for a lab, but the lab should be labeled as a lab.

Conclusion: Build the AI Operating Model Before the GPU Island

VCF 9.1 gives infrastructure teams a stronger platform foundation for private AI workloads. The release brings more AI-oriented options across GPU access, Kubernetes, networking, observability, storage efficiency, and cost visibility.

But the platform does not make the operating model decision for you.

The infrastructure team still has to decide where AI workloads live, how GPUs are consumed, how tenants are isolated, how lifecycle risk is contained, how storage policies map to AI data types, how network paths support accelerator-heavy workloads, and how costs are exposed.

The first GPU cluster will set expectations.

If it is deployed as a one-off capacity answer, every future tenant will inherit ambiguity. If it is deployed behind a clear AI service contract, it becomes the first building block of a private AI platform.

That is the decision that matters.

Do not start with the GPU cluster.

Start with the service model the cluster is supposed to serve.

External Sources

Use these sources for publish-time validation of the VCF 9.1, VxRail, vSAN, VKS, NSX, Private AI, lifecycle, observability, and cost-management claims referenced in the article.

• Broadcom: Broadcom Announces VMware Cloud Foundation 9.1
  https://news.broadcom.com/releases/broadcom-announces-vmware-cloud-foundation-9-1
  Used for: VCF 9.1 positioning, private AI claims, Kubernetes and multi-tenant AI infrastructure messaging, cost and operational improvement claims.
• VMware Cloud Foundation Blog: Streamline, Simplify, and Protect All Your AI Workloads with VCF 9.1
  https://blogs.vmware.com/cloud-foundation/2026/05/05/streamline-simplify-and-protect-all-your-ai-workloads-with-vcf-9-1/
  Used for: VCF Private AI Services, AI observability dashboards, deep learning VM patterns, GPU-backed workloads, DirectPath-style access, CPU inference references, and AI infrastructure metrics.
• VMware Cloud Foundation Blog: What’s New with vSphere 9.1
  https://blogs.vmware.com/cloud-foundation/2026/05/12/whats-new-with-vsphere-9-1/
  Used for: vSphere 9.1 lifecycle updates, Enhanced DirectPath I/O, GPU-related platform behavior, DRS improvements, and firmware/driver validation references.
• VMware Cloud Foundation Blog: VMware Cloud Foundation 9.1 Path for Dell VxRail Customers
  https://blogs.vmware.com/cloud-foundation/2026/06/16/vmware-cloud-foundation-9-1-path-for-dell-vxrail-customers/
  Used for: VCF-on-VxRail mixed hardware guidance, VxRail and Dell vSAN Ready Node workload domain boundaries, homogeneous hardware requirements inside workload domains, management domain constraints, and lifecycle/support model differences.
• VMware Cloud Foundation Blog: vSAN Compression and Global Deduplication in VCF 9.1
  https://blogs.vmware.com/cloud-foundation/2026/05/07/vsan-compression-and-global-deduplication-in-vcf-9-1/
  Used for: vSAN global deduplication, compression, supported topology limitations, storage efficiency claims, and persistent volume scale references.
• VMware Cloud Foundation Blog: Simplify Workload Connectivity and Enhance Network Scale and Performance with VCF 9.1
  https://blogs.vmware.com/cloud-foundation/2026/05/05/simplify-workload-connectivity-and-enhance-network-scale-and-performance-with-vcf-9-1/
  Used for: NSX and VCF 9.1 networking updates, EVPN-VXLAN interoperability, Enhanced DirectPath I/O for NVIDIA ConnectX and BlueField devices, and GPUDirect RDMA references.
• VMware Cloud Foundation Blog: VCF 9.1 Brings Multi-Network Support to vSphere Kubernetes Service
  https://blogs.vmware.com/cloud-foundation/2026/06/02/vcf-9-1-brings-multi-network-support-to-vsphere-kubernetes-service/
  Used for: VKS 3.6 multi-network support, Kubernetes node multi-NIC behavior, primary and secondary NIC roles, and operational constraints.
• VMware Cloud Foundation Blog: Deploy Modern Apps Faster, Scale Smarter, and Lower Your TCO with VKS on VCF 9.1
  https://blogs.vmware.com/cloud-foundation/2026/05/05/deploy-modern-apps-faster-scale-smarter-and-lower-your-tco-with-vks-on-vcf-9-1/
  Used for: VKS 3.6 and VCF 9.1 modern application platform claims, Kubernetes placement, node pool improvements, and TCO-related vendor claims.
• VMware Cloud Foundation Blog: Scale, Simplify, and Secure Your Private Cloud Operations with VCF 9.1
  https://blogs.vmware.com/cloud-foundation/2026/05/05/scale-simplify-and-secure-your-private-cloud-operations-with-vcf-9-1/
  Used for: VCF Operations updates, cost visibility, showback and chargeback, real-time pricing, VKS cost details, observability, and private cloud operations claims.
• VMware Cloud Foundation Blog: Unified Cloud Value: Accelerating Cloud FinOps with VCF 9.1 and FOCUS
  https://blogs.vmware.com/cloud-foundation/2026/06/03/unified-cloud-value-accelerating-cloud-fin-ops-with-vcf-9-1-and-focus/
  Used for: FOCUS-aligned cost data, FinOps normalization, private cloud cost comparison, and AI/GPU workload cost-tracking context.

The post VCF 9.1 for AI Workloads: The Decisions Infrastructure Teams Need to Make Before the First GPU Cluster appeared first on Digital Thought Disruption.