Cache, store, and run container images inside Fuzzball

Cache, store, and run container images inside Fuzzball

Contributors

Wolfgang Resch, Research Computing Engineer

You can run Fuzzball workflows without an external container registry. Fuzzball stores container images in its built-in object cache as ready-to-run Singularity Image Format (SIF) files. Images your workflows pull are converted and cached there. You can also explicitly push your own images to the object cache and reference them in your workflows.

Earlier releases cached images differently. Storing them as objects makes that caching more reliable and flexible.

Fuzzball stores your container images inside the platform in converted SIF format, so workflows can run without reaching out to an external registry every time.

Pulled images are cached as SIF, ready to run

When a workflow pulls an image from an external registry like Docker Hub, Fuzzball converts it to a SIF container image and stores it in the cache. The conversion happens once. The next workflow that needs the image uses the stored SIF directly, from any node in the cluster.

You can browse cached images like any other object. And because caching no longer needs a shared file system between Substrate instances, it works the same way on every node in your deployment.

Push your own images directly

You can push your own SIF images into the object cache and reference them from your workflow. To keep a pushed image available indefinitely, set its time-to-live to zero so the cache never purges it.

This lets a site run Fuzzball workflows with no outside registry dependency, including in air-gapped environments. The images your workflows need live in the platform you operate.

Setting up your image pipeline? See how the platform handles performance-intensive workloads in the Fuzzball solution brief.

What this changes for your workflows

Picture a researcher who needs a standard base image with two extra libraries. The older path takes three steps across three systems: build the image on a workstation, push it to an external registry, then reference the registry URI from the workflow. Every run depends on the registry being reachable.

Fuzzball trims this. The researcher builds the image on a workstation and pushes the SIF straight into the object cache. The workflow references it directly, and every run reads from the cache; no external registry in the loop.

Keep your container supply chain inside the platform

Fuzzball caches the images your workflows pull from external registries, and it stores the SIF images you push directly. Once an image is in the cache, the workflows run without an external registry dependency in the loop.

See what Fuzzball can do for your cluster. Explore the Fuzzball product page.

Ready to learn more about what CIQ can do for you?

Get in touch

Related posts

AI workflow orchestration: why separate platforms fail

AI workflow orchestration: why separate platforms fail

CIQ 2024 Predictions: The HPC Evolution Boosted by AI and Open Source

CIQ 2024 Predictions: The HPC Evolution Boosted by AI and Open Source

CIQ Fuzzball and Nvidia NIM for Voice-to-Text Processing

CIQ Fuzzball and Nvidia NIM for Voice-to-Text Processing

CIQ's Partnership with NVIDIA: Transforming Enterprise GPU Infrastructure

CIQ's Partnership with NVIDIA: Transforming Enterprise GPU Infrastructure

Built for scale. Chosen by the world’s best.

2.75M+

Rocky Linux instances

Being used world wide

90%

Of fortune 100 companies

Use CIQ supported technologies

250k

Avg. monthly downloads

Rocky Linux