NVIDIA has released two new tools — KAI Scheduler and vCluster — that let AI and machine learning teams share graphics processing units more efficiently inside Kubernetes environments. The tools are designed to help organizations get more out of their GPU investments by allowing multiple users to run isolated workloads on the same hardware.
How the tools work together
KAI Scheduler handles job scheduling and resource allocation across a Kubernetes cluster. It decides which jobs get access to which GPUs and for how long. vCluster, on the other hand, creates lightweight, isolated Kubernetes clusters that run inside a shared physical cluster. Each vCluster gets its own set of resources, including GPUs, but they all draw from the same underlying pool.
Together, the two tools give teams the isolation they need for security and compliance while still pooling GPUs. That means a data science team can run a training job on one vCluster while another team runs inference on a different vCluster — all using the same GPU hardware.
Why GPU sharing matters
GPUs are expensive and often sit idle when not in use. In many organizations, each team buys its own dedicated GPUs, leading to low utilization and high costs. By sharing GPUs across teams, companies can run more workloads without buying additional hardware.
NVIDIA's approach uses Kubernetes, the popular container orchestration system, to manage the sharing. KAI Scheduler and vCluster are built on top of existing Kubernetes APIs, so teams don't need to learn new tools. They just deploy their jobs as they normally would, and the scheduler handles the rest.
What this means for AI/ML teams
For AI and ML engineers, the new tools could mean faster access to GPUs and lower costs for their projects. Instead of waiting for a dedicated GPU to become free, they can submit jobs to a shared pool and get results sooner. The isolation provided by vCluster also means one team's workload won't interfere with another's — a common problem in shared environments.
NVIDIA is positioning the tools as part of its AI Enterprise platform, which includes a suite of software for managing AI workloads. The company says the tools are available now, and teams can start testing them with the latest release of the platform.
