CIQ

How to Use AWS ParallelCluster 3.8.0 with Rocky Linux 8

How to Use AWS ParallelCluster 3.8.0 with Rocky Linux 8
Yoshiaki SendaDecember 20, 2023

AWS ParallelCluster 3.8.0 was just released on December 19th and added support for Rocky Linux 8, thanks to Enrico Usai -san and the AWS ParallelCluster team! Rocky Linux 8 is supported as a CustomAmi build through the pcluster build-image command. This article shows you how to build an HPC cluster by using AWS ParallelCluster 3.8.0 with Rocky Linux 8.

Disclaimer

This article doesn’t cover aws CLI and pcluster CLI installation and configuration. For those procedures, please see the following:

After following the instructions on those 4 pages, you are ready to go!

Install AWS ParallelCluster CLI on Rocky Linux 8

I played with AWS ParallelCluster 3.8.0 using the Rocky Linux 8 system. This is an example procedure from when I installed the CLI. (For details, please see the above link.)

$ wget https://us-east-1-aws-parallelcluster.s3.amazonaws.com/parallelcluster/3.8.0/installer/pcluster-installer-bundle-3.8.0.480-node-v18.17.1-Linux_x86_64-signed.zip
$ unzip pcluster-installer-bundle-3.8.0.480-node-v18.17.1-Linux_x86_64-signed.zip -d pcluster-installer-bundle
$ cd pcluster-installer-bundle
$ bash install_pcluster.sh

Check version:

$ pcluster version
{
  "version": "3.8.0"
}

Build Rocky Linux 8.8 image for AWS ParallelCluster

Create rocky-8.8.yaml with the following contents:

ℹ️ The minimum RootVolume size for this example is about 42GB. Installing additional packages using Component will require more space. We will explain how to use Component in the next article.

Region: ap-northeast-1
Image:
  Name: Rocky Linux 8.8 Custom AMI for PCluster 3.8.0
  RootVolume:
    Size: 100
Build:
  ParentImage: ami-069b05da577cf3215
  InstanceType: c5.4xlarge

Please modify Region and ParentImage if you use different regions. You can search the region and corresponding image ID here.

ℹ️ Rocky Linux 8.8 AMI from RESF has 2 types. Standard (non-LVM) and LVM. Please use the standard one for AWS ParallelCluster. The following table shows Rocky Linux 8.8 x86_64 standard AMI.

ap-south-1ami-036c222371722b4ec
eu-north-1ami-0a2f2b07a5d58843a
eu-west-3ami-0618721d17eff62b0
eu-west-2ami-09912feb78f61ee4a
eu-west-1ami-0359e98212b83eac4
ap-northeast-3ami-0fbd5e58a55eda78d
ap-northeast-2ami-01b1e81dca9091b25
ap-northeast-1ami-069b05da577cf3215
ca-central-1ami-07e049a0d688b1be0
sa-east-1ami-0f24c9c8b0690494a
ap-southeast-1ami-00e2abffdd8991a00
ap-southeast-2ami-0fa590a1419f1e05a
eu-central-1ami-06557f879780f3afc
us-east-1ami-093bfdaaa5ea85b41
us-east-2ami-03cf1b432c549f1b4
us-west-1ami-0a516be6b65225b41
us-west-2ami-00f575b0315210392

Build the image using the pcluster build-image command. It takes about an hour.

$ pcluster build-image --image-id rocky-88 --image-configuration rocky-8.8.yaml
{
  "image": {
    "imageId": "rocky-88",
    "imageBuildStatus": "BUILD_IN_PROGRESS",
    "cloudformationStackStatus": "CREATE_IN_PROGRESS",
    "cloudformationStackArn": "arn:aws:cloudformation:ap-northeast-1:******",
    "region": "ap-northeast-1",
    "version": "3.8.0"
  }
}

Check the build status:

$ pcluster describe-image --image-id rocky-88
{
  "imageConfiguration": {
    "url": "https://parallelcluster-*******
  },
  "imageId": "rocky-88",
  "imagebuilderImageStatus": "BUILDING",
  "imageBuildStatus": "BUILD_IN_PROGRESS",
  "cloudformationStackStatus": "CREATE_IN_PROGRESS",
  "cloudformationStackArn": "arn:aws:cloudformation:ap-northeast-1:******",
  "region": "ap-northeast-1",
  "version": "3.8.0",
...

Once the Rocky Linux 8 image is successfully built:

$ pcluster describe-image --image-id rocky-88
{
  "imageConfiguration": {
    "url": "https://parallelcluster-******"
  },
  "imageId": "rocky-88",
  "creationTime": "2023-12-20T15:42:37.000Z",
  "imageBuildStatus": "BUILD_COMPLETE",
  "region": "ap-northeast-1",
  "ec2AmiInfo": {
    "amiName": "Rocky Linux 8.8 Custom AMI for PCluster 3.8.0 2023-12-20T15-10-40.342Z",
    "amiId": "ami-******",
    "description": "AWS ParallelCluster AMI for rocky8, kernel-4.18.0-477.10.1.el8_8.x86_64, lustre-2.12.8-1.fsx7.el8.x86_64, efa-2.6.0-1.el8.x86_64, dcv-2023.0.15487-1.el7.x86_64",
    "state": "AVAILABLE",

Check AMI ID. We use this ID at the next step.

$ pcluster describe-image --image-id rocky-88 --query 'ec2AmiInfo.amiId'
"ami-******"

Or you can list available images:

$ pcluster list-images --image-status AVAILABLE
{
  "images": [
    {
      "imageId": "rocky-88",
      "imageBuildStatus": "BUILD_COMPLETE",
      "ec2AmiInfo": {
        "amiId": "ami-******"
      },
      "region": "ap-northeast-1",
      "version": "3.8.0"
    }
  ]
}

The next step is building an HPC cluster using AWS ParallelCluster.

Build Rocky Linux 8 HPC Cluster using AWS ParallelCluster

Create a config file for the Rocky Linux 8 HPC Cluster using the pcluster configure command.

$ pcluster configure --config rocky8-cluster.yaml

After the config file is generated, add CustomAmi at line 4. We use the AMI ID value we got at the previous step. The following is an example rocky8-cluster.yaml file. 

Region: ap-northeast-1
Image:
  Os: rocky8
  CustomAmi: ami-******
HeadNode:
  InstanceType: t2.xlarge
  Networking:
    SubnetId: subnet-******
  Ssh:
    KeyName: yoshisenda-tokyo
  LocalStorage:
    RootVolume:
      Size: 100
Scheduling:
  Scheduler: slurm
  SlurmQueues:
  - Name: c5n9
    ComputeResources:
    - Name: c5n9xlarge
      Instances:
      - InstanceType: c5n.9xlarge
      MinCount: 0
      MaxCount: 16
      Efa:
        Enabled: true
    Networking:
      PlacementGroup:
        Enabled: true
      SubnetIds:
      - subnet-******
  - Name: c5n18
    ComputeResources:
    - Name: c5n18xlarge
      Instances:
      - InstanceType: c5n.18xlarge
      MinCount: 0
      MaxCount: 16
      Efa:
        Enabled: true
    Networking:
      PlacementGroup:
        Enabled: true
      SubnetIds:
      - subnet-******

Build a Rocky Linux 8 HPC Cluster:

$ pcluster create-cluster \
  --cluster-configuration rocky8-cluster.yaml \
  --cluster-name rocky8-cluster \
  --region ap-northeast-1

Check the cluster build status: 

$ pcluster describe-cluster --cluster-name rocky8-cluster
{
  "creationTime": "2023-12-20T16:19:01.897Z",
  "version": "3.8.0",
  "clusterConfiguration": {
    "url": "https://parallelcluster-******"
  },
  "tags": [
    {
      "value": "3.8.0",
      "key": "parallelcluster:version"
    },
    {
      "value": "rocky8-cluster",
      "key": "parallelcluster:cluster-name"
    }
  ],
  "cloudFormationStackStatus": "CREATE_IN_PROGRESS",
  "clusterName": "rocky8-cluster",
  "computeFleetStatus": "UNKNOWN",
  "cloudformationStackArn": "arn:aws:cloudformation:ap-northeast-1:******",
  "lastUpdatedTime": "2023-12-20T16:19:01.897Z",
  "region": "ap-northeast-1",
  "clusterStatus": "CREATE_IN_PROGRESS",
  "scheduler": {
    "type": "slurm"
  }
}

The Rocky Linux 8 HPC Cluster has been successfully built by using AWS ParallelCluster

$ pcluster describe-cluster --cluster-name rocky8-cluster | grep -i status
  "cloudFormationStackStatus": "CREATE_COMPLETE",
  "computeFleetStatus": "RUNNING",
  "clusterStatus": "CREATE_COMPLETE",

and SSH-ing to head node:

$ pcluster ssh --cluster-name rocky8-cluster -i ~/.ssh/yoshisenda-tokyo.pem
Activate the web console with: systemctl enable --now cockpit.socket

Last login: Wed Dec 20 16:27:54 2023
[rocky@ip-10-0-0-230 ~]$ cat /etc/os-release | grep -i pretty 
PRETTY_NAME="Rocky Linux 8.8 (Green Obsidian)"
[rocky@ip-10-0-0-230 ~]$ sinfo
PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
c5n9*        up   infinite     16  idle~ c5n9-dy-c5n9xlarge-[1-16]
c5n18        up   infinite     16  idle~ c5n18-dy-c5n18xlarge-[1-16]

Create a sample job script job.sh with the following content:

#!/bin/bash
#SBATCH --partition=c5n9
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1

cat /etc/os-release | grep -i pretty

Submit job:

$ sbatch job.sh

Check output:

$ cat slurm-1.out
PRETTY_NAME="Rocky Linux 8.8 (Green Obsidian)"

Voila!

Related posts

2023 Holiday Gift Guide for Rocky Linux Users

2023 Holiday Gift Guide for Rocky Linux Users

Dec 19, 2023

Rocky Linux

Why Rocky Linux Is a Rock-Solid Choice in an Economic Downturn

Why Rocky Linux Is a Rock-Solid Choice in an Economic Downturn

Jan 18, 2023

Rocky Linux

6 Signs That It's Time to Move to Rocky Linux

6 Signs That It's Time to Move to Rocky Linux

Feb 23, 2023

Rocky Linux

123
38
>>>