This overview introduces the on-premise release of Fuzzball as of Q3 2024 and CIQ's vision of performance-intensive computing. Workflow orchestration is the core capability: a custom workflow definition system built for HPC gives each job dedicated resources and its own environment, and a workflow handles storage volumes, data ingress and egress, container image preparation, and running jobs in parallel and in dependency order.
The web interface acts as a single pane of glass for monitoring workflows, reading logs, opening an interactive terminal beside a running job, building workflows in a visual editor or importing them from a file, and managing secrets, shared-account members, and persistent volumes. Administrators define shared accounts, user access, organization-wide secrets, and storage classes. Jobs use ephemeral or persistent volumes via storage classes, and context-aware secrets can fetch data from S3, download from a website, pull from a registry, or act as environment variables.
Fuzzball is developed API first, with a documented customer-accessible API and a Python SDK, so the CLI runs from anywhere without SSH. Orchestrate deploys into Kubernetes through a custom operator, while the Substrate runtime executes jobs outside Kubernetes in optimized container images for effectively bare-metal performance. Substrate can be deployed with Ascender or provisioned with Warewulf.
Key takeaways
- Each job in a Fuzzball workflow gets dedicated computational resources and a customized environment specific to that job.
- The web interface monitors workflows, retrieves logs, offers an interactive terminal next to running jobs, and includes a visual workflow editor.
- Workflows create and attach volumes, manage data ingress and egress, prepare container images, and run jobs in dependency order.
- Context-aware secrets can be scoped to an organization, a group account, or a user, and used for S3, websites, registries, or environment variables.
- Fuzzball Orchestrate runs on Kubernetes via a custom operator, while Substrate runs jobs outside Kubernetes for effectively bare-metal performance.
- Substrate can be deployed on Enterprise Linux with Ascender or other tools, or provisioned with Warewulf as part of a cluster.
Questions this video answers
What does a Fuzzball workflow actually manage?
A workflow handles everything around a workload: it creates and attaches storage volumes, manages data ingress and egress, downloads and prepares container images, and runs compute jobs in parallel and in order according to their dependencies and available resources. Jobs can request ephemeral or persistent volumes through storage classes.
How does Fuzzball deliver bare-metal performance if it runs on Kubernetes?
Fuzzball Orchestrate, the control plane, runs on Kubernetes and is deployed with a custom operator. Compute jobs, however, run outside the Kubernetes cluster on the Substrate runtime in optimized container images, so they get effectively bare-metal performance while keeping the portability and reproducibility of containers.
Can I use Fuzzball without SSH?
Yes. All access is API-driven, so the CLI can run from anywhere with no SSH required, and the web interface provides an interactive terminal alongside a running compute job. Both interfaces use a documented, customer-accessible API, and a Python SDK is available for building custom automations.
About this video
Fuzzball is the future of high performance computing (HPC). This video provides an overview of the on-premise release of Fuzzball as of Q3 2024.
Fuzzball offers workflow orchestration with a custom workflow definition system that is built for performance intensive computing (PIC). It has an innovative graphical user interface (GUI) that allows researchers to perform HPC/PIC workflow setup and execution without SSH or complex Linux scripting.
Batch computing and interactive computing, including virtual desktop access. Get bare-metal performance for your HPC workloads. Fuzzball runs on Kubernetes, offering a containerized API-first architecture.
This video is part of the Fuzzball playlist. Browse every CIQ video by product and topic.
Transcript
with fuzzball we're building the future of performance intensive Computing workflow orchestration is a core capability of fuzzball with a custom workflow definition system that's built for high performance Computing each job in a fuzzball workflow is allocated dedicated computational resources and can run in a customized environment specific to that job fuzzball supports a modern container first approach to Performance Computing it's capable of traditional commandline driven batch Computing interactive web-based vdi and even an API driven orchestration as part of your existing or future automation infrastructure the fuzzball web interface provides a single paint of glass for making full use of your compute resources here you can
monitor and review the status of your workflows retrieve log output and even get an immediate access to an interactive terminal alongside a running compute job you can construct new workflows from scratch using the visual workflow editor or import an existing workflow from a file you can also manage Secrets members of your shared accounts and persistent volumes administrators can use the web interface to Define shared accounts manage user access Define organization wide shared secrets and to Define storage classes fuzzball users belong to one or more fuzzball accounts these accounts allow fuzzball users to easily collaborate on workflows and to access shared storage a fuzzball workflow manages
all the necessary activities of your workload it creates and attaches storage volumes it manages data Ingress and egress it downloads and prepares contain container images and it runs your compute jobs in parallel and in order according to their dependency requirements and the resources available Fuzzball compute jobs can use both ephemeral and persistent storage by requesting a volume using an existing storage class these volumes are then attached to the file system of your compute jobs fuzzball manages context aware secrets for use in your fuzzball workflows secrets can be defined for the entire organization shared with members of a group account or set individually by user and
defin Secrets can then be used to fetch data from S3 to download data from a website to pull a container from a registry or just as an environment variable within a job Fuzzball is just as accessible from the command line as it is via its web interface and because all access to fuzzball is apid driven you can run the CLI from anywhere no SSH required fuzzball is developed API first and both the command line and web interfaces communicate with fuzzball using a documented customer accessible API for example you can use a python SDK to access fuzzball via its open API endpoint to build custom automations
View full transcriptHide full transcript
and Integrations with any fuzzball process you can deploy fuzzball orchestrate into your existing kubernetes environment using our custom kubernetes operator you define a fuzzball object to automatically create and configure a fully functional instance of fuzzball orchestrate and adjust that definition later to make realtime changes to your deployment over its lifetime fuzzball orchestrate runs on kubernetes but our custom container runtime substrate is optimized for performance intensive work compute jobs run outside of the kubernetes cluster in optimized CF container images ensuring effectively bare metal performance while maintaining the flexibility portability and reproducibility of a modern container first architecture fuzzball substrate can be deployed on Enterprise Linux with
Ascender or other industry standard tools and it can also be provisioned using Warewulf as part of an integrated compute cluster environment
Built for scale. Chosen by the world’s best.
2.75M+
Rocky Linux instances
Being used world wide
90%
Of fortune 100 companies
Use CIQ supported technologies
250k
Avg. monthly downloads
Rocky Linux
9
Enterprise products
Spanning the kernel to the orchestrator
Have questions about your infrastructure?
Talk to a CIQ engineer about Rocky Linux, HPC, and AI infrastructure.