Containerizing MoonRay Using Apptainer and Rocky Linux 9

MoonRay is DreamWorks Animation’s in-house 3D renderer, used to render DreamWorks movies. DreamWorks has released MoonRay as open source, under the Apache 2.0 license. This is major news for the animation community, with many contributors already onboard with MoonRay.

The official document provides two ways of building MoonRay: (1) Building MoonRay in a Docker container and (2) Building MoonRay on a CentOS 7 machine. This article provides a third option: (3) Building MoonRay using Apptainer and Rocky Linux 9. Apptainer makes the build process of MoonRay drastically easier and simpler.

Apptainer (formerly Singularity) is the most widely used container system for HPC. It is designed to execute applications at bare-metal performance while being secure, portable, and 100% reproducible.

Setup Procedures

We tested these procedures on Rocky Linux 8, Rocky Linux 9, and Ubuntu 22.04 (20.04 also should work, but was not tested this time). The machine we used for testing has 1 GPU and an NVIDIA driver already installed.

Apptainer

Let’s install Apptainer. This is the only procedure that is OS dependent and needs root privilege. We will show you a Rocky Linux version and an Ubuntu version here.

For Rocky Linux 8 and Rocky Linux 9:

sudo dnf install -y epel-release

sudo dnf install -y apptainer

For Ubuntu 20.04 and 22.04:

sudo add-apt-repository -y ppa:apptainer/ppa

sudo apt install -y apptainer

Apptainer v1.1.6 was used for this article.

apptainer --version

apptainer version 1.1.6-1.el8

NVIDIA OptiX

NVIDIA OptiX is one of MoonRay’s dependencies. Download NVIDIA OptiXTM SDK 7.3.0 from https://developer.nvidia.com/designworks/optix/downloads/legacy. To download this file, you need an NVIDIA account.

MoonRay

Download the MoonRay source code from the GitHub repository. This time we used a forked version of MoonRay, since our PR (pull request) has not yet merged to the official MoonRay. Our PR includes three definition files that we use here and a build guide.

git clone --recurse-submodules -b rockylinux9 https://github.com/panda1100/openmoonray.git

Move NVIDIA-OptiX-SDK-7.3.0-linux64-x86_64.sh from the Download directory to the building directory. 

mv ~/Downloads/NVIDIA-OptiX-SDK-7.3.0-linux64-x86_64.sh . 

Build MoonRay using Apptainer via the helper script called moonray.sh. This helper script will build a base image, dependencies, and MoonRay. The build process will take about 15 minutes or more with 8 cores, so please have a coffee here:

export WORKDIR=$(pwd)

bash ./openmoonray/building/apptainer/moonray.sh

Now you have a MoonRay container called moonray-rocky9.sif . The Apptainer container is a single file and executable. Move this container to ~/bin .

For Rocky Linux 8 and Rocky Linux 9:

cd ./openmoonray/building/apptainer

mv moonray-rocky9.sif ~/bin/moonray-rocky9.sif

For Ubuntu Ubuntu 20.04 and 22.04:

mkdir ~/bin

cd ./openmoonray/building/apptainer

mv moonray-rocky9.sif ~/bin/moonray-rocky9.sif

echo 'export PATH=~/bin:$PATH' >> ~/.bashrc

source ~/.bashrc

Usage

Now we are ready to use MoonRay for rendering!

cd ~/

Execute the container to get the MoonRay shell. The environment variables required to use MoonRay are already set in this shell. 

apptainer shell --nv ~/bin/moonray-rocky9.sif

MoonRay> 

Download the official examples using the wget and unzip downloaded archive and then unzip the glass-of-water example archive. Change the directory to the glass-of-water example directory. 

MoonRay> wget https://docs.openmoonray.org/assets/test-scenes/example_scenes.zip

MoonRay> unzip example_scenes.zip

MoonRay> cd pbrt_scenes

MoonRay> unzip glass-of-water.zip

MoonRay> cd glass-of-water

MoonRay> moonray_gui -info -exec_mode xpu -in scene.rdla -in scene.rdlb -out glass.exr

MoonRay> exit

Launch moonray_gui with xpu execution mode with the input files. You will get a similar image as follows. Press h for displaying the menu, and press n for denoising.

Animal Logic ALab data (1) (about 40GB)

Animal Logic shares ALab datasets under ASWF Digital Assets License v1.1. We used a version of Animal Logic’s ALab dataset that includes pre-converted data from USD to MoonRay-native RDL2 scene format and also includes ready-to-render .tx format textures provided by the MoonRay team. This archive size is about 40GB.

cd ~/

wget https://d2k39ng9pbbkxu.cloudfront.net/ALab.zip

unzip ALab.zip

Rendering RDL2 scene format

Render the pre-converted ALab scene data using moonray_gui:

apptainer shell --nv ~/bin/moonray-rocky9.sif

MoonRay> moonray_gui -info -exec_mode xpu -in ALab.rdla -in ALab.rdlb -out ALab.exr

(1) Animal Logic ALab, Copyright 2022, Animal Logic Pty Limited. All rights reserved.

Rendering Animation from USD format datasets

To render the animation, first convert USD format to RDL2 format using hd_usd2rdl, and render with moonray and denoise with the denoise command. The following script is a modified version of the script that is shared on the MoonRay community’s GitHub here by the user @samhodge:

cd ~/ALab

for i in {997..1271}

do

  echo "frame ${i}"

  date

  hd_usd2rdl -in entry.usda -purpose render -time ${i} -camera root/camera01/GEO/renderCam_hrc/renderCam_buffer/renderCam_srt/renderCam -out stoatrender.${i}.rdlb

  moonray -exec_mode xpu -in stoatrender.${i}.rdlb -out stoatrender.${i}.exr

  denoise -mode optix -in stoatrender.${i}.exr -out stoat.${i}.exr

  echo "frame ${i} end"

  rm stoatrender.${i}.rdlb stoatrender.${i}.exr

  date

done

Rendering result

To review, we built MoonRay using Apptainer and Rocky Linux, rendered an RDL2 format scene with moonray_gui, and rendered USD format scenes with hd_usd2rdl, moonray and denoise.

Appendix: Container Registry

After you build a MoonRay container, you can upload your container to a container registry for later use. This section will explain how to upload to Docker Hub as an example. You need a Docker Hub account prior to the following procedures.

After you login to Docker Hub, click your username on the top right corner and open Account Settings > Security > Access Token. Click New Access Token, fill out a description, and then click Generate. Save your displayed Access Token for later procedures.

Use the above Access Token to login to Docker Hub from Apptainer.

apptainer remote login -u DOCKERHUB_USERNAME oras://docker.io

After successfully logging in, push your container to the Docker Hub registry. The container size is about 7GB, so it will take time to upload.

apptainer push ~/bin/moonray-rocky9.sif oras://docker.io/DOCKERHUB_USERNAME/moonray:1.0.0

Once you push your container to the registry, let’s pull this image from your remote machines (for example, your cloud resources) for rendering.

apptainer pull oras://docker.io/DOCKERHUB_USERNAME/moonray:1.0.0

Now you can use this image on your remote machines.

apptainer shell --nv ./moonray_1.0.0.sif

MoonRay> 

Voila!