If you're building AI infrastructure right now, you're paying for GPUs while the hardware and software around them keep changing, and a stack you buy whole today can be out of date in months. I spent an hour at AI Infrastructure Field Day arguing for a different way to build: an AI kit, made of independent layers. You add the ones you need and don't have today, and you replace one when the hardware or the workload changes.
It was the first time CIQ presented at Tech Field Day, and the delegates asked hard questions. This post summarizes the three parts of the session and the questions I thought were most useful. You can watch the full session or go to each part below.
Build from layers you can replace
Every layer of the AI stack is changing at once, some on one-month to three-month cycles, and the hardware is revised within a year.
I showed what idle GPUs cost with one customer's numbers. Their GPUs arrived before the rest of the data center was ready. By my calculation, roughly 9,000 nodes cost about $15.6 million in GPU hours for every 30 days they sat idle [VERIFY: Brian's calculation from the session; no claim file exists for this figure].
Teams under pressure to get to production ask one vendor for a complete AI factory. That gets them running, and it also commits them to one vendor's fixed view of the stack at one point in time.
One delegate asked whether a single stack can cover all of this. I said it takes a suite of independent products, each one useful without the others.
An AI kit is the set of layers you're missing, run next to the infrastructure and tools you already operate.
Watch part one: why an AI kit.
Tune the operating system under your GPUs
Most of the AI stack diagrams I see in conference talks leave out the operating system, even though the kernel controls access to the GPUs on the node. Rocky Linux from CIQ Pro AI (RLC Pro AI) is Enterprise Linux with a current long-term kernel, tested NVIDIA drivers, CUDA, and AI frameworks already working together. Happy-path node deployment takes 3 minutes 44 seconds, with nothing to assemble first.
The best challenge came from a delegate who asked whether configuration management tools could do the same tuning. They can do a lot of it. They can't change parameters that are set when the kernel and drivers are compiled, and someone still has to redo the work every time a driver, CUDA, or a framework releases.
Another delegate asked who buys RLC Pro AI, since many developers never think past pulling a container. My answer was the teams counting GPU time: anyone with a large GPU investment, and anyone for whom a few percent more throughput changes the result.
Watch part two: maximizing your GPUs with a tuned OS.
Run one workflow across the clusters you have
High-performance computing (HPC) teams keep telling me they now have to run their cluster, general Kubernetes, and generative AI, and they're building a separate stack for each. Fuzzball is the workflow orchestration layer of the kit. It runs on top of an existing Slurm or PBS deployment and uses that deployment as a resource manager, so nothing has to move on day one. Fuzzball deploys on a Kubernetes cluster, so if you don't run Kubernetes yet, that comes first.
One workflow definition, run on-premises and in the cloud, means a new kind of workload doesn't require a new stack.
Watch part three: distributed AI workflows with Fuzzball.
Start with one layer
You don't have to take the whole kit. In a market changing this fast, I don't think anyone should be asked to replace a whole stack at once.
Start with RLC Pro AI, the operating system layer for GPU nodes.
