Compute nodes provisioned by Warewulf are stateless. They boot from an image served over the network, so anything a node writes to its root filesystem at runtime, including the result of a realm join, is gone on the next reboot. To keep nodes joined to Active Directory, the join has to live on the Warewulf server and reach each node at boot. Warewulf Pro ships overlays that do exactly that.
The warewulf-overlays-sssd package provides two overlays for AD:
-
sssd, a system overlay that configures SSSD and the Kerberos default realm on each node
-
krb5.keytab, a runtime overlay that delivers each node's Kerberos keytab from the Warewulf server
The AD join itself happens once per node, on the Warewulf server. Nodes never run realm join.
This guide assumes Warewulf Pro 4.7 with at least one node booting a Rocky Linux 9 image, and that the Warewulf server and the nodes can resolve and reach your domain controllers through DNS. The examples use the domain my.ad-domain.com and a node named n1.
Prepare the node image
The image needs the SSSD and AD client packages, with SSSD selected as the authentication profile. The cleanest way to get that is a Containerfile. It keeps the image definition version-controlled and reproducible:
FROM ghcr.io/warewulf/warewulf-rockylinux:9
RUN dnf -y install sssd sssd-ad adcli krb5-workstation authselect \
&& authselect select sssd --force \
&& systemctl enable sssd \
&& dnf clean all
Save it as Containerfile in its own directory, then build it there and import it into Warewulf:
[root@wwctl ~]# podman build . --file Containerfile --tag rockylinux-9-ad
[root@wwctl ~]# wwctl image import $(podman image mount localhost/rockylinux-9-ad) rockylinux-9-ad
[root@wwctl ~]# podman image unmount localhost/rockylinux-9-ad
[root@wwctl ~]# wwctl image build rockylinux-9-ad
Assign the image to your nodes with --image rockylinux-9-ad. If you'd rather add the packages to an image you already have:
[root@wwctl ~]# wwctl image shell rockylinux-9
[rockylinux-9] Warewulf> dnf install -y sssd sssd-ad adcli krb5-workstation authselect
[rockylinux-9] Warewulf> authselect select sssd --force
[rockylinux-9] Warewulf> systemctl enable sssd
[rockylinux-9] Warewulf> exit
Keep authselect in that list. Not every Warewulf node image includes it.
Turn on secure mode
A keytab is a credential. Anyone holding a node's keytab can authenticate to AD as that node. By default, Warewulf serves overlays to anyone on the provisioning network who asks, so turn on secure mode in /etc/warewulf/warewulf.conf first:
warewulf:
secure: true
With secure mode on, Warewulf serves the runtime overlay only to requests from a privileged port, which means only root on the node can fetch it. That's why the keytab lives in a runtime overlay and not a system overlay. Restart warewulfd, then rebuild overlays and reboot the nodes so wwclient switches to a privileged port:
[root@wwctl ~]# systemctl restart warewulfd
[root@wwctl ~]# wwctl overlay build
Create a keytab for each node
Create an AD computer account and keytab for each node from the Warewulf server. adcli prompts for the password of the --login-user account, which needs permission to join computers to the domain.
[root@wwctl ~]# dnf install -y adcli krb5-workstation
[root@wwctl ~]# install -d -m 0700 /etc/warewulf/keytabs
[root@wwctl ~]# adcli join --domain=my.ad-domain.com \
--computer-name=N1 \
--host-fqdn=n1.my.ad-domain.com \
--host-keytab=/etc/warewulf/keytabs/n1.keytab \
--login-user=Administrator
[root@wwctl ~]# base64 -w0 /etc/warewulf/keytabs/n1.keytab > /etc/warewulf/keytabs/n1.keytab.b64
[root@wwctl ~]# chmod 0600 /etc/warewulf/keytabs/n1.keytab*
Because --host-keytab is set, adcli writes the keytab only to that path, and the Warewulf server's own Kerberos configuration isn't touched. The overlay reads the base64 copy, named after the node. Warewulf's template Include function strips a trailing newline byte, so a raw keytab that happened to end in 0x0a would arrive corrupted. Base64 sidesteps that.
Every node needs its own computer account. If you're setting up several nodes, run the join in a loop. adcli join --login-user asks for the password on every run, so get a Kerberos ticket for the join account once and pass it to each join with --login-ccache instead:
[root@wwctl ~]# kinit Administrator@MY.AD-DOMAIN.COM
[root@wwctl ~]# for n in n{1..100}; do
adcli join --domain=my.ad-domain.com \
--computer-name=${n^^} \
--host-fqdn=$n.my.ad-domain.com \
--host-keytab=/etc/warewulf/keytabs/$n.keytab \
--login-ccache
base64 -w0 /etc/warewulf/keytabs/$n.keytab > /etc/warewulf/keytabs/$n.keytab.b64
done
[root@wwctl ~]# chmod 0600 /etc/warewulf/keytabs/*
[root@wwctl ~]# kdestroy
Replace n{1..100} with your node names. ${n^^} is the node name in upper case, used as the AD computer name, which is limited to 15 characters.
Configure the overlays
Install the package, put the two overlays in their own profile, and configure it for your domain:
[root@wwctl ~]# dnf install -y warewulf-overlays-sssd
[root@wwctl ~]# wwctl profile add ad
[root@wwctl ~]# wwctl profile set ad \
--system-overlays sssd \
--runtime-overlays krb5.keytab \
--tagadd sssd_domain=my.ad-domain.com
[root@wwctl ~]# wwctl node set n1 --profile default,ad
[root@wwctl ~]# wwctl overlay build
Overlays from multiple profiles are combined on each node, so this leaves your default profile alone.
sssd_domain is the only required tag. The realm defaults to the domain in upper case, home directories default to /home/%u@%d, and users log in with fully qualified names. Each of these can be changed with a tag. To see them all, run:
[root@wwctl ~]# wwctl overlay info sssd etc/sssd/sssd.conf.ww
To check what a node will get before rebooting it:
[root@wwctl ~]# wwctl overlay show -r n1 sssd /etc/sssd/sssd.conf.ww
Boot and verify
[root@wwctl ~]# wwctl power cycle n1
[root@wwctl ~]# ssh n1
[root@n1 ~]# kinit -k 'N1$' && klist && kdestroy
[root@n1 ~]# getent passwd user@my.ad-domain.com
[root@n1 ~]# id user@my.ad-domain.com
The kinit -k line confirms the node can authenticate to AD with its own keytab. After that, log in over SSH as a domain user, reboot the node, and check again. Nothing needs to be redone after a reboot.
The sssd overlay also configures SSSD to start after wwclient. With secure mode on, the keytab arrives with the runtime overlay a few seconds into boot, and without that ordering SSSD could start before the keytab exists.
Home directories
Compute nodes often can't create home directories themselves. Warewulf's default profile mounts /home over NFS from the Warewulf server, exported with root_squash, so root on a node can't write there. That's a sensible default. Leave it alone and create each user's home directory on the system that hosts /home (the Warewulf server, with the default profile) before their first login. Without it, the login fails with "Could not chdir to home directory".
Copy /etc/skel and give it to the UID and GID that id user@my.ad-domain.com reports on a node:
[root@wwctl ~]# cp -a /etc/skel /home/user@my.ad-domain.com
[root@wwctl ~]# chown -R <uid>:<gid> /home/user@my.ad-domain.com
[root@wwctl ~]# chmod 0700 /home/user@my.ad-domain.com
[root@wwctl ~]# restorecon -R /home/user@my.ad-domain.com
cp -a also copies the mode of /etc/skel (0755), so the chmod keeps the directory private. SSSD maps AD users to the same IDs on every machine with the same domain settings, so they match across the cluster. If the Warewulf server is joined to the domain too, mkhomedir_helper user@my.ad-domain.com 0077 does all of this in one step.
Tip: If /home is local to each node, or the nodes can write to it as root, they can create home directories at first login instead: add oddjob-mkhomedir to the image, use authselect select sssd with-mkhomedir --force, and enable oddjobd. On a stateless node, a local /home lives in memory and is cleared on every reboot unless a local disk is mounted there.
Rotating machine passwords
The overlay turns off machine password rotation on the nodes. The keytab on the Warewulf server is the source of truth, and wwclient reapplies the runtime overlay every minute by default, so a keytab rotated on a node would be overwritten with the server's copy. That would leave the node holding a password AD no longer accepts.
If your policy requires rotation, rotate on the server and push the new keytab out:
[root@wwctl ~]# adcli update --domain=my.ad-domain.com \
--computer-name=N1 \
--host-fqdn=n1.my.ad-domain.com \
--host-keytab=/etc/warewulf/keytabs/n1.keytab \
--computer-password-lifetime=0
[root@wwctl ~]# base64 -w0 /etc/warewulf/keytabs/n1.keytab > /etc/warewulf/keytabs/n1.keytab.b64
[root@wwctl ~]# wwctl overlay build n1
--computer-password-lifetime=0 forces the change. Without it, adcli skips accounts whose password is less than 30 days old. Don't skip the overlay build: Warewulf rebuilds overlays when overlay files change, but it doesn't notice changes to the keytab files the overlay reads. Once the overlay is rebuilt, the running node picks up the new keytab within a minute, with no reboot.
Tip: until the new keytab arrives, the node still holds the old one, which AD no longer accepts, so logins on that node can fail for up to a minute. To deliver it right away, have wwclient on the node pull its overlays now:
[root@wwctl ~]# wwctl ssh n1 systemctl reload wwclient
To rotate automatically, use warewulf-krb5-rotate, which the package installs. It runs the steps above for you, and each node authenticates with its own keytab, so no administrator credentials are stored on the Warewulf server. To rotate nodes now:
[root@wwctl ~]# warewulf-krb5-rotate --reload n1 n2
To rotate on a schedule, tag the nodes and enable the timer. Once a day, it rotates each tagged node whose keytab file is older than krb5_rotate_days (30 by default), rebuilds its overlays, and reloads wwclient on it. This example sets a 90-day period; leave out the krb5_rotate_days tag to keep the default:
[root@wwctl ~]# wwctl profile set ad --tagadd krb5_rotate=true --tagadd krb5_rotate_days=90
[root@wwctl ~]# systemctl enable --now warewulf-krb5-rotate.timer
Adding nodes
For each new node, run the adcli join and base64 steps with that node's name, add it to the ad profile, and run wwctl overlay build.
The full setup guide, including every supported tag, is in the Warewulf Pro training curriculum under Pro overlays. If your site uses FreeIPA or a plain LDAP directory instead, the same package covers them; see the companion posts.
