How to Use AWS ParallelCluster 3.8.0 with Rocky Linux 8
AWS ParallelCluster 3.8.0 was just released on December 19th and added support for Rocky Linux 8, thanks to Enrico Usai -san and the AWS ParallelCluster team! Rocky Linux 8 is supported as a CustomAmi build through the pcluster build-image command. This article shows you how to build an HPC cluster by using AWS ParallelCluster 3.8.0 with Rocky Linux 8.
Disclaimer
This article doesn’t cover aws CLI and pcluster CLI installation and configuration. For those procedures, please see the following:
-
Setting up an AWS account: https://docs.aws.amazon.com/parallelcluster/latest/ug/setting-up.html
-
Creating a key pair: https://docs.aws.amazon.com/parallelcluster/latest/ug/set-up-keypair.html
-
Installing the AWS ParallelCluster: https://docs.aws.amazon.com/parallelcluster/latest/ug/install-v3-parallelcluster.html (I’ll show you what I have done this time at the next section.)
-
Configuring and creating a cluster with the AWS ParallelCluster command line interface: https://docs.aws.amazon.com/parallelcluster/latest/ug/install-v3-configuring.html
After following the instructions on those 4 pages, you are ready to go!
Install AWS ParallelCluster CLI on Rocky Linux 8
I played with AWS ParallelCluster 3.8.0 using the Rocky Linux 8 system. This is an example procedure from when I installed the CLI. (For details, please see the above link.)
$ wget https://us-east-1-aws-parallelcluster.s3.amazonaws.com/parallelcluster/3.8.0/installer/pcluster-installer-bundle-3.8.0.480-node-v18.17.1-Linux_x86_64-signed.zip
$ unzip pcluster-installer-bundle-3.8.0.480-node-v18.17.1-Linux_x86_64-signed.zip -d pcluster-installer-bundle
$ cd pcluster-installer-bundle
$ bash install_pcluster.sh
Check version:
$ pcluster version
{
"version": "3.8.0"
}
Build Rocky Linux 8.8 image for AWS ParallelCluster
Create rocky-8.8.yaml with the following contents:
ℹ️ The minimum RootVolume size for this example is about 42GB. Installing additional packages using Component will require more space. We will explain how to use Component in the next article.
Region: ap-northeast-1
Image:
Name: Rocky Linux 8.8 Custom AMI for PCluster 3.8.0
RootVolume:
Size: 100
Build:
ParentImage: ami-069b05da577cf3215
InstanceType: c5.4xlarge
Please modify Region and ParentImage if you use different regions. You can search the region and corresponding image ID here.
ℹ️ Rocky Linux 8.8 AMI from RESF has 2 types. Standard (non-LVM) and LVM. Please use the standard one for AWS ParallelCluster. The following table shows Rocky Linux 8.8 x86_64 standard AMI.
| ap-south-1 | ami-036c222371722b4ec |
| eu-north-1 | ami-0a2f2b07a5d58843a |
| eu-west-3 | ami-0618721d17eff62b0 |
| eu-west-2 | ami-09912feb78f61ee4a |
| eu-west-1 | ami-0359e98212b83eac4 |
| ap-northeast-3 | ami-0fbd5e58a55eda78d |
| ap-northeast-2 | ami-01b1e81dca9091b25 |
| ap-northeast-1 | ami-069b05da577cf3215 |
| ca-central-1 | ami-07e049a0d688b1be0 |
| sa-east-1 | ami-0f24c9c8b0690494a |
| ap-southeast-1 | ami-00e2abffdd8991a00 |
| ap-southeast-2 | ami-0fa590a1419f1e05a |
| eu-central-1 | ami-06557f879780f3afc |
| us-east-1 | ami-093bfdaaa5ea85b41 |
| us-east-2 | ami-03cf1b432c549f1b4 |
| us-west-1 | ami-0a516be6b65225b41 |
| us-west-2 | ami-00f575b0315210392 |
Build the image using the pcluster build-image command. It takes about an hour.
$ pcluster build-image --image-id rocky-88 --image-configuration rocky-8.8.yaml
{
"image": {
"imageId": "rocky-88",
"imageBuildStatus": "BUILD_IN_PROGRESS",
"cloudformationStackStatus": "CREATE_IN_PROGRESS",
"cloudformationStackArn": "arn:aws:cloudformation:ap-northeast-1:******",
"region": "ap-northeast-1",
"version": "3.8.0"
}
}
Check the build status:
$ pcluster describe-image --image-id rocky-88
{
"imageConfiguration": {
"url": "https://parallelcluster-*******
},
"imageId": "rocky-88",
"imagebuilderImageStatus": "BUILDING",
"imageBuildStatus": "BUILD_IN_PROGRESS",
"cloudformationStackStatus": "CREATE_IN_PROGRESS",
"cloudformationStackArn": "arn:aws:cloudformation:ap-northeast-1:******",
"region": "ap-northeast-1",
"version": "3.8.0",
...
Once the Rocky Linux 8 image is successfully built:
$ pcluster describe-image --image-id rocky-88
{
"imageConfiguration": {
"url": "https://parallelcluster-******"
},
"imageId": "rocky-88",
"creationTime": "2023-12-20T15:42:37.000Z",
"imageBuildStatus": "BUILD_COMPLETE",
"region": "ap-northeast-1",
"ec2AmiInfo": {
"amiName": "Rocky Linux 8.8 Custom AMI for PCluster 3.8.0 2023-12-20T15-10-40.342Z",
"amiId": "ami-******",
"description": "AWS ParallelCluster AMI for rocky8, kernel-4.18.0-477.10.1.el8_8.x86_64, lustre-2.12.8-1.fsx7.el8.x86_64, efa-2.6.0-1.el8.x86_64, dcv-2023.0.15487-1.el7.x86_64",
"state": "AVAILABLE",
Check AMI ID. We use this ID at the next step.
$ pcluster describe-image --image-id rocky-88 --query 'ec2AmiInfo.amiId'
"ami-******"
Or you can list available images:
$ pcluster list-images --image-status AVAILABLE
{
"images": [
{
"imageId": "rocky-88",
"imageBuildStatus": "BUILD_COMPLETE",
"ec2AmiInfo": {
"amiId": "ami-******"
},
"region": "ap-northeast-1",
"version": "3.8.0"
}
]
}
The next step is building an HPC cluster using AWS ParallelCluster.
Build Rocky Linux 8 HPC Cluster using AWS ParallelCluster
Create a config file for the Rocky Linux 8 HPC Cluster using the pcluster configure command.
$ pcluster configure --config rocky8-cluster.yaml
After the config file is generated, add CustomAmi at line 4. We use the AMI ID value we got at the previous step. The following is an example rocky8-cluster.yaml file.
Region: ap-northeast-1
Image:
Os: rocky8
CustomAmi: ami-******
HeadNode:
InstanceType: t2.xlarge
Networking:
SubnetId: subnet-******
Ssh:
KeyName: yoshisenda-tokyo
LocalStorage:
RootVolume:
Size: 100
Scheduling:
Scheduler: slurm
SlurmQueues:
- Name: c5n9
ComputeResources:
- Name: c5n9xlarge
Instances:
- InstanceType: c5n.9xlarge
MinCount: 0
MaxCount: 16
Efa:
Enabled: true
Networking:
PlacementGroup:
Enabled: true
SubnetIds:
- subnet-******
- Name: c5n18
ComputeResources:
- Name: c5n18xlarge
Instances:
- InstanceType: c5n.18xlarge
MinCount: 0
MaxCount: 16
Efa:
Enabled: true
Networking:
PlacementGroup:
Enabled: true
SubnetIds:
- subnet-******
Build a Rocky Linux 8 HPC Cluster:
$ pcluster create-cluster \
--cluster-configuration rocky8-cluster.yaml \
--cluster-name rocky8-cluster \
--region ap-northeast-1
Check the cluster build status:
$ pcluster describe-cluster --cluster-name rocky8-cluster
{
"creationTime": "2023-12-20T16:19:01.897Z",
"version": "3.8.0",
"clusterConfiguration": {
"url": "https://parallelcluster-******"
},
"tags": [
{
"value": "3.8.0",
"key": "parallelcluster:version"
},
{
"value": "rocky8-cluster",
"key": "parallelcluster:cluster-name"
}
],
"cloudFormationStackStatus": "CREATE_IN_PROGRESS",
"clusterName": "rocky8-cluster",
"computeFleetStatus": "UNKNOWN",
"cloudformationStackArn": "arn:aws:cloudformation:ap-northeast-1:******",
"lastUpdatedTime": "2023-12-20T16:19:01.897Z",
"region": "ap-northeast-1",
"clusterStatus": "CREATE_IN_PROGRESS",
"scheduler": {
"type": "slurm"
}
}
The Rocky Linux 8 HPC Cluster has been successfully built by using AWS ParallelCluster
$ pcluster describe-cluster --cluster-name rocky8-cluster | grep -i status
"cloudFormationStackStatus": "CREATE_COMPLETE",
"computeFleetStatus": "RUNNING",
"clusterStatus": "CREATE_COMPLETE",
and SSH-ing to head node:
$ pcluster ssh --cluster-name rocky8-cluster -i ~/.ssh/yoshisenda-tokyo.pem
Activate the web console with: systemctl enable --now cockpit.socket
Last login: Wed Dec 20 16:27:54 2023
[rocky@ip-10-0-0-230 ~]$ cat /etc/os-release | grep -i pretty
PRETTY_NAME="Rocky Linux 8.8 (Green Obsidian)"
[rocky@ip-10-0-0-230 ~]$ sinfo
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST
c5n9* up infinite 16 idle~ c5n9-dy-c5n9xlarge-[1-16]
c5n18 up infinite 16 idle~ c5n18-dy-c5n18xlarge-[1-16]
Create a sample job script job.sh with the following content:
#!/bin/bash
#SBATCH --partition=c5n9
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
cat /etc/os-release | grep -i pretty
Submit job:
$ sbatch job.sh
Check output:
$ cat slurm-1.out
PRETTY_NAME="Rocky Linux 8.8 (Green Obsidian)"
Voila!
