Fuzzball HPC 2.0 Workflow Demonstrations

There has been a lot of interest in HPC-2.0, the next generation of High Performance Computing and we are very excited to provide glimpses into what it looks like and how to use it. In this demonstration, Forrest Burt from CIQ explains the workflow definition and running LAMMPS (a molecular dynamics suite) across four nodes in parallel using MPI.

Webinar Synopsis:

• Setting Up Example Fuzzball Workflow

• Running LAMMPS in a Container

• Getting Results of the Workflow

Speakers:

• Forrest Burt, High Performance Computing Systems Engineer, CIQ
  • Linkedin

---

Note: This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors.

Full Webinar Transcript:

Setting Up Example Fuzzball Workflow [00:00]

Forrest Burt:

Hey everyone. My name is Forrest Burt from CIQ, and I'm here today with a Fuzzball workflow demo. In this workflow, we’re going to be looking at running the LAMMPS molecular dynamic suite across four nodes at once using MPI. First off, I'll go ahead and show you exactly what our workflow definition for this looks like, so we can know exactly what this workflow is going to run andhow we define that. The versions field at the top defines the version of our DSL that this workflow definition is written in. The volumes field here defines the data volumes that are going to be set up for this workflow, so it can run correctly. We're setting up one volume called v1. It's an ephemeral type volume, meaning that it'll only exist for the lifetime of this workflow.

And we’re going to ingress a file from this URL into this volume and place it at the top level of that volume with this name. We then have the jobs field, which is the section that defines the computational jobs that this workflow should actually run. In the case of this workflow, we only have one computational job, and that's ‘run-lammps.’ The next field here, MPI, defines how we want MPI to be wired up for this workflow job. In this case, we're defining that we want four compute nodes wired up with an open MPI implementation of our MPI wrapper. I'll explain here, in just a moment, how we determine the resources that we want on each one of those nodes. We’re going tobe running out of this container image pulled from this URL. You can see this is a LAMMPS container that we're pulling down from NVIDIA's NGC.

Running LAMMPS in a Container [01:34]

We're going to be running this command inside that container, so this essentially represents what this computational job is meant to do. You can see that we’re basically just  running LAMMPS with an input file at this location. I'll explain where ‘/data’ comes from here in a moment as well. We have a current working directory, meaning this command is going to be run inside of this directory here. We also have a policy set for this job that essentially means that if it starts and fails to complete within 10 hours, it'll automatically time-out and stop. We then have the resource requirements for each one of the nodes that we're requesting for this workflow. In this case, we're looking for eight cores pulled from the minimal number of NUMA nodes on a given compute node that those cores are coming from, and then 30 GiB of RAM also available on that compute node. 

Because we requested four nodes here, we're going to be looking for four separate nodes, each one with these resources available on it for a grand total of 32 cores and 120 GiB of memory available to this entire workflow job to be able to run an MPI over. We also have the data volume I discussed earlier being mounted to this workflow job. You can see that we have the name of the volume we're mounting, which is v1 and matches the dot right there, and then the location where we want to make this volume available inside the container app. In this case, we’re going to be mounting this volume to the container at ‘/data.’ You can see that's why up here when we specify the input file location, we ‘/data/[the name of the file we ingressed here].’

As I mentioned, this essentially places it at the top level directory of this volume, so you can see that it's essentially just right inside that volume at the location we attach it. We'll go ahead and start this workflow, ’fuzzball workflow start,’ and then we'll provide a name for it, so do ‘ciq.co/lammps-mpi-demo/1.’. We'll go ahead and provide the path to the workflow demo we're going to use or parts of the workflow definition file we'll be using for this. It's in the same directory, so I basically just have to specify the name. You'll see we get a message saying that this workflow has started, along with the name of it. I'll go ahead and pull up the Fuzzball workflow status for this workflow, and I'll add the watch flag, so we can watch it live and have it print the status continuously. Go ahead and paste that in there.

Getting Results of the Workflow [04:08]

We can see that this workflow is started. The volume has already been set up. The image is already finished downloading. It's essentially cached from other workflows that have been run, and then this file ingress that we defined here is currently happening. Then once we see this finish, we'll see the nodes for this job that we have start to be provisioned. That'll look like this finishing and this staying as pending. This will fast-forward while we wait a few minutes for those nodes to be provisioned. This workflow is being run on a public cloud provider on a Fuzzball cluster that’s been stood up there. So when we request these four compute nodes, matching these requirements right here, it’s going to take a definition that we've set up within Fuzzball of a certain type of compute node or the definitions that we've set up, find which one best matches this, and then spin up four of those and then start this workflow job on them with our MPI wrapper. We'll give this a few minutes to run. This will probably speed up for a second, and then we'll return once these nodes have been provisioned and this job is started.

Now we can see that this workflow is started in the job status while ‘run-lammps’ has started, so we can see that job and it finished there really quickly. But you can see that job changed status from pending to started. That means those four nodes completed provisioning, and the command that we specified up here started running on those nodes.

We should now be able to view the logs from that run of LAMMPS. We'll do the Fuzzball workflow log, and we'll specify the name of the workflow, and we should get some output here showing that we were running over 32 cores at once. That'll indicate that this workflow job ran successfully, and thus the workflow finished successfully. You can see we indeed have some LAMMPS output here. Because we're running this in CPU-only mode, you’ll see that we get some CUDA errors, but as mentioned, those were just running in CPU-only mode, so we expect that. You can see down here that we have 32 MPI tasks, which is the 32 cores we were expecting. We get a measure of the performance coming out of this. We can see that, based on this, the workflow ran successfully, and we were able torun LAMMPS as a Fuzzball workflow. 

Thank you for joining this demo, and please feel free to look at our other materials surrounding Fuzzball or the other services and products that CIQ provides. Thank you very much. Have a great day.