Fuzzball and CIQ's David Godlove Part 2
In this short excerpt from a CIQ webinar, David Godlove clarifies a common misunderstanding about Fuzzball's relationship with Kubernetes. Kubernetes orchestrates the microservices that make up Fuzzball Orchestrate, but the jobs themselves do not run on Kubernetes. Orchestrate uses a CSI driver to create a volume and mount it into a job at runtime.
On the compute side sits Substrate, a lightweight container platform in the spirit of Apptainer that talks to Orchestrate and runs jobs as processes. Because everything is containerized, Substrate can run on cloud instances or on bare-metal nodes in a cluster, and jobs simply land wherever it is running.
Key takeaways
- Kubernetes only orchestrates the microservices behind Fuzzball Orchestrate; it does not run the HPC jobs themselves.
- Orchestrate uses a CSI driver to create volumes and mount them into containerized jobs at runtime.
- Substrate is a lightweight, job-oriented container platform on compute resources, comparable to Apptainer, that receives work from Orchestrate.
- Because jobs are containerized, Substrate can run on cloud resources or bare-metal cluster nodes with no change to the workflow.
Questions this video answers
Does Fuzzball run HPC jobs inside Kubernetes?
No. Kubernetes hosts the microservices that make up Fuzzball Orchestrate, the control plane, but jobs run on compute resources through Substrate, a lightweight container platform. Orchestrate mounts storage into those containerized jobs via a CSI driver, and Substrate can sit on cloud instances or bare-metal nodes.
This video is part of the Fuzzball playlist. Browse every CIQ video by product and topic.
Transcript
all this is running on kubernetes right this part here but what we're going to do with that it's it's running on top of but it's not using kubernetes right basically kubernetes is just orchestrating all these microservices that make up um that makeup orchestrate this part of this part of fuzzball so when I say storage what I basically mean is fuzzball itself is going to take this CSI driver and it's going to be able to figure out how to mount how to create a volume and mount that volume into your jobs at runtime that's cool and so let's go over here so now we're on
a compute resource and the compute resource has another thing running called substrate so we've got orchestrate and we've got substrate and substrate you can think of really it's just kind of a lightweight uh container platform that also talks to orchestrate that also understands things that orchestrate is telling it says C is like Apptainer and that it's a very lightweight container solution um that's based on kind of like jobs and processes running in things um so this is the thing that uh you know orchestrate would use a CSI driver to mount your storage into your job into your containerized job running on substrate here and critically
this can run either you know on the cloud or you can have substrate running on a bunch of bare metal nodes in your cluster and either way because everything's containerized we can just land those jobs on there and just kick them off and start them
Built for scale. Chosen by the world’s best.
2.75M+
Rocky Linux instances
Being used world wide
90%
Of fortune 100 companies
Use CIQ supported technologies
250k
Avg. monthly downloads
Rocky Linux
9
Enterprise products
Spanning the kernel to the orchestrator
Have questions about your infrastructure?
Talk to a CIQ engineer about Rocky Linux, HPC, and AI infrastructure.