Fuzzball modernizes HPC for R&D on-prem

HPC (High Performance Computing) has long been the backbone of scientific research, financial modeling, complex simulations, and data-intensive tasks like AI (artificial intelligence) and ML (machine learning). However, traditional HPC environments often come with significant challenges: legacy infrastructure, complex job scheduling, and resource management that can slow down the pace of R&D (research and development). Successful HPC has long involved juggling various software environments, managing dependencies, moving large datasets, and efficiently using resources.

Such performance-intensive compute workloads often require researchers and scientists to involve system administrators, data admins, cluster managers, and even compliance officers. It’s time-consuming, even to the point of frustration.

Fuzzball by CIQ accelerates innovation by streamlining these concerns and reducing the back-and-forth communication between system, cluster, and data administrators and compliance officers. With Fuzzball, researchers can focus on what they do best: research.

Fuzzball is a modern solution for HPC workloads

Fuzzball modernizes traditional HPC by offering a fresh, container-first, API-driven architecture with automated data ingress and egress management, a user-friendly GUI for workflow design and execution, and browser-based cluster administration.

It’s a workflow orchestration platform that integrates with existing infrastructure while providing a powerful, flexible environment for running complex workloads. Whether you’re working with traditional command-line batch jobs, interactive web-based interfaces, or API-driven automation, Fuzzball offers the tools and features needed to modernize and streamline your HPC workflows.

By bringing all these capabilities into a single platform, Fuzzball redefines what’s possible in HPC, making it more accessible, efficient, and aligned with today’s use cases:

• AI/ML Training: Automatically allocates data, network, storage, GPU, and CPU resources tailored to the needs of AI/ML workloads
• Scientific simulations: Workloads run on bare-metal resources, ensuring the highest possible performance and lowest latency
• Financial modeling: Portable and reproducible workloads and results, ensuring maximum compliance with regulatory requirements
• Genomic sequencing: Automates the end-to-end sequencing of workflows, from raw data processing to variant analysis, allowing workflows to be ported to different research institutions and platforms

Built-in security and compliance features for performance-intensive workloads

Performance-intensive workloads often have compliance and governance requirements that are challenging to uphold in traditional environments. For example, it’s difficult to audit SSH activity to see exactly what commands an HPC user executed. It’s also difficult to ensure they did not inadvertently (or intentionally) access sensitive data.

For regulated industries, these aren’t mere “challenges” but risks to their business operations.

Fuzzball manages everything from end-to-end, including execution logs, data access controls, storage volume mounting, and secrets management. It fully automates and alleviates the security risks of whether a local Linux account was configured with the right access and execution privileges. Instead, users can configure, execute, and monitor jobs within a web-based interface. All actions are logged and auditable.

All users within Fuzzball belong to built-in accounts, allowing them to easily collaborate on workflows, access shared storage, and stay within the appropriate security bounds for organizational governance.

See logs for every workflow execution and access the CLI within Fuzzball’s web interface.

Auditable, compliant data ingress and egress

In traditional HPC environments, managing data ingress and egress can be challenging. Data must often be manually transferred, staged, and prepared for use in complex workflows. It’s time-consuming and prone to errors that can lead to inefficiencies and non-compliance for organizations bound to stringent data handling regulations and governance. Ensuring that all data transfers are logged, traceable, and compliant typically requires additional tools and manual oversight, further complicating the workflow.

Fuzzball automates and streamlines the entire data ingress and egress process, making it auditable and compliant by design. All data interactions within a workflow are managed, removing complexity from the equation. Whether you’re pulling data from S3, downloading files from external sites, or importing data from a database, Fuzzball ensures that every action is logged and traceable. No more worrying about whether the HPC user is potentially accessing sensitive data in a private SSH session.

With its built-in secrets management, Fuzzball guarantees that other sensitive information is only accessible to authorized components within the workflow. This puts an end to manual configurations and the risk of unauthorized access.

Easily manage data storage volumes and usage within Fuzzball’s web interface.

Meeting modern research demands with modern solutions

Fuzzball is an automated workload management platform for modern needs. It doesn’t just manage storage volumes, it has native integrations with popular systems like AWS S3. It supports context-aware secrets via S3, HTTP(S), OCI, and job-specific environment variables. The CLI is entirely API-driven, which means no SSH access is required and every action is logged. Users can even leverage the built-in APIs to build their own SDKs for custom automations and integrations with any Fuzzball process.

Whereas traditional HPC is built on patterns developed decades ago, Fuzzball is designed for modern compute architectures and deployment patterns. For example, Fuzzball deploys into a Kubernetes environment using CIQ’s custom Kubernetes Operator. Simply define a Fuzzball object and it will automatically create, configure, and deploy a fully-functional instance of Fuzzball’s orchestration system.

Want to learn more about Fuzzball? Watch our Fuzzball deployment webinar with HPC experts Brian Phan and Jonathon Anderson. If you’d like a demo, let us know: info@ciq.com.