CIQ Glossary

Pipeline

Table of contents

Date

January 19, 2024

The primary processing component of a workflow, i.e., the main sequence of computational jobs some HPC-based task will perform as a part of execution. Generally refers to execution outside of a workflow engine’s control–so, for example, if you have a bunch of scripts you’re running that are coordinated by some master script, you have and are running a pipeline; but if all those scripts, their execution order, necessary resources, container image, etc., are codified with some kind of workflow engine, maybe with a YAML-based DSL, and the execution of this codification is then done by the workflow engine, then you have and are running a workflow.

Built for Scale. Chosen by the World’s Best.

1.4M+

Rocky Linux instances

Being used world wide

90%

Of fortune 100 companies

Use CIQ supported technologies

250k

Avg. monthly downloads

Rocky Linux