Keystone SystemsKS Systems

Disk encryption

Keystone encrypts all storage devices by default using LUKS2. The encryption architecture differs between the ZFS and ext4 storage backends, but both integrate with TPM2 hardware for automatic unlock during normal boots and provide recovery credentials for emergency access. This page covers the encryption model, the credstore pattern, TPM enrollment, and remote unlock for headless servers.

Encryption architecture

ZFS: the credstore pattern

When using the ZFS backend, Keystone employs a two-layer encryption scheme called the credstore pattern. This design separates the LUKS unlock mechanism from the ZFS encryption keys, allowing TPM2 hardware to manage the unlock while ZFS handles the actual data encryption.

The boot process follows this sequence:

  1. Pool import. The ZFS pool rpool is imported without mounting any datasets.
  2. Encryption root validation. Each boot-critical ZFS dataset is checked to confirm its encryption root is rpool/crypt, preventing the mounting of fraudulent or tampered filesystems.
  3. Credstore unlock. The LUKS-encrypted credstore volume (/dev/zvol/rpool/credstore) is unlocked. The TPM2 module attempts automatic unlock first; if that fails, the system prompts for a passphrase.
  4. Key loading. The ZFS encryption key is read from the mounted credstore (/etc/credstore/zfs-sysroot.mount) and loaded into ZFS.
  5. Filesystem mounting. Encrypted ZFS datasets are mounted and the boot continues.

The credstore itself is a small (100 MB by default) ZFS volume formatted as ext4 inside a LUKS container. During initial deployment, a random 32-byte key is generated and stored in the credstore. This key encrypts all ZFS datasets under rpool/crypt using AES-256-GCM.

Physical Disk
  └── ZFS Pool (rpool)
        ├── rpool/credstore (zvol)
        │     └── LUKS2 container
        │           └── ext4 filesystem
        │                 └── zfs-sysroot.mount (32-byte key)
        └── rpool/crypt (encrypted dataset, key from credstore)
              ├── rpool/crypt/system (/)
              ├── rpool/crypt/system/nix (/nix)
              └── rpool/crypt/system/var (/var)

This separation of concerns means that re-enrolling TPM or changing the LUKS passphrase does not require re-encrypting the ZFS data. Only the credstore LUKS keyslots change; the ZFS encryption key inside remains the same.

ext4: direct LUKS encryption

The ext4 backend uses a simpler model. The root partition is directly encrypted with LUKS2:

Physical Disk
  ├── ESP (FAT32, /boot)
  ├── LUKS2 container (cryptroot)
  │     └── ext4 filesystem (/)
  └── Swap partition
        └── Random encryption (or LUKS2 if hibernation is enabled)

TPM2 automatic unlock is configured directly on the cryptroot LUKS device. There is no credstore intermediary.

TPM2 PCR measurements

Both backends include tpm2-measure-pcr=yes in the LUKS crypttab options. This instructs systemd-cryptsetup to extend a TPM2 Platform Configuration Register (PCR) after unlocking, recording that the disk was unlocked. This measurement prevents replay attacks where an attacker might try to re-use a captured TPM state.

TPM enrollment

TPM (Trusted Platform Module) enrollment allows the system to unlock encrypted disks automatically during boot without manual passphrase entry. The TPM seals the LUKS unlock key against specific PCR values, meaning automatic unlock only succeeds when the system boots in the expected state.

Prerequisites

Before enrolling TPM, two conditions must be met:

  1. Secure Boot must be enabled in User Mode. TPM enrollment binds to the Secure Boot state via PCR 7. If Secure Boot is not active, the PCR values will not match after enrollment.

    bootctl status | grep "Secure Boot"
    # Expected: Secure Boot: enabled (user)
  2. A TPM 2.0 device must be present.

    ls /dev/tpm*
    # Expected: /dev/tpm0  /dev/tpmrm0

Enrollment commands

After a fresh Keystone installation, a login banner indicates that TPM enrollment is not configured. Two enrollment methods are available:

Recovery key (recommended):

sudo keystone-enroll-recovery

This generates a cryptographic recovery key (256-bit, formatted as a dash-separated string). The recovery key must be saved immediately in a secure location outside the encrypted disk, such as a password manager with offline backup or a printed copy in a physical safe.

Custom password:

sudo keystone-enroll-password

This prompts for a custom passphrase (12 to 64 characters). The password must not be "keystone" (the publicly known default).

Both commands perform the same underlying steps:

  1. Add the chosen credential (recovery key or password) to a LUKS keyslot.
  2. Enroll TPM2 automatic unlock bound to the configured PCRs.
  3. Remove the default "keystone" password from LUKS.
  4. Create an enrollment marker file at /var/lib/keystone/tpm-enrollment-complete.

After enrollment, the system unlocks automatically during normal boots. The recovery credential is needed only when TPM unlock fails.

PCR binding

By default, Keystone binds TPM unlock to PCRs 1 and 7:

  • PCR 1: Firmware configuration. Changes when BIOS/UEFI settings are modified.
  • PCR 7: Secure Boot certificates. Changes when Secure Boot keys are modified or Secure Boot is disabled.

This combination protects against firmware tampering and Secure Boot bypass while remaining resilient to normal software updates. Kernel updates, NixOS rebuilds, and bootloader updates do not affect PCRs 1 or 7.

The PCR list is configurable via keystone.os.tpm.pcrs:

keystone.os.tpm = {
  enable = true;
  pcrs = [ 7 ];  # Secure Boot only (most update-resilient)
};

| PCR list | Security | Update resilience | Use case | | --------- | -------- | ----------------- | --------------------------------------- | | [7] | Good | Excellent | Frequent firmware updates | | [1 7] | Better | Good | Default balanced approach | | [0 1 7] | Best | Poor | Maximum security, rare firmware changes |

Re-enrollment after changes

When firmware or Secure Boot configuration changes intentionally, the TPM unlock will fail and the system will prompt for the recovery credential. After booting with the recovery credential:

# Remove the old TPM keyslot
sudo systemd-cryptenroll --wipe-slot=tpm2 /dev/zvol/rpool/credstore

# Enroll a new TPM keyslot with current PCR values
sudo keystone-enroll-tpm

# Reboot to verify
sudo reboot

When recovery credentials are needed

The recovery key or custom password is required in the following situations:

  • Hardware changes: Motherboard replacement, TPM hardware failure, or changes that affect firmware PCR measurements.
  • Firmware changes: Secure Boot key re-enrollment, Secure Boot disabled, or BIOS/UEFI reflash.
  • Intentional reconfiguration: Any change to the firmware or boot chain that alters the measured PCR values.

Normal NixOS updates, kernel upgrades, and bootloader updates do not require re-enrollment when using the default PCR list of [1, 7].

LUKS keyslot layout

After enrollment, the typical LUKS keyslot configuration is:

| Slot | Type | Purpose | | ---- | -------------------- | ------------------------------------------------------- | | 0 | (removed) | Default "keystone" password (removed during enrollment) | | 1 | password or recovery | Recovery key or custom password | | 2 | tpm2 | TPM-sealed automatic unlock key |

LUKS supports up to 32 keyslots. Additional credentials can be added with systemd-cryptenroll:

# Add a second recovery key
sudo systemd-cryptenroll --recovery-key /dev/zvol/rpool/credstore

# View current keyslots
sudo systemd-cryptenroll /dev/zvol/rpool/credstore

Remote unlock

Headless servers cannot enter a passphrase at the console during boot. Keystone provides remote unlock via SSH in the initrd, allowing an administrator to unlock the disk over the network.

Configuration

keystone.os.remoteUnlock = {
  enable = true;
  authorizedKeys = [ "ssh-ed25519 AAAAC3... admin@workstation" ];
  port = 22;              # SSH port in initrd (default: 22)
  networkModule = "e1000e"; # Network driver for initrd (default: "virtio_net")
  dhcp = true;            # Use DHCP for network config (default: true)
};

The networkModule must match the network interface card in the server. Common values include e1000e (Intel), igb (Intel server), r8169 (Realtek), and virtio_net (virtual machines).

How it works

When remote unlock is enabled, Keystone configures the initrd with:

  1. Network support. The specified network driver module is loaded, and DHCP is used (by default) to obtain an IP address.
  2. SSH server. An SSH daemon starts in the initrd, listening on the configured port with the specified authorized keys.
  3. Password agent. The root shell in the initrd runs systemd-tty-ask-password-agent, which prompts for and processes the disk unlock passphrase.

To unlock remotely:

ssh -p 22 root@<server-ip>
# The systemd password agent will prompt for the LUKS passphrase
# Enter the passphrase, and the boot will continue

Host key management

The initrd SSH server uses a dedicated host key at /etc/ssh/initrd_ssh_host_ed25519_key. This key must be pre-generated during deployment:

ssh-keygen -t ed25519 -N '' -f /etc/ssh/initrd_ssh_host_ed25519_key

Using a separate host key for the initrd avoids SSH host key mismatch warnings, since the initrd environment and the fully booted system present different host keys.

Combining with TPM

Remote unlock and TPM enrollment are complementary. On a server with both enabled:

  • Normal boots: TPM unlocks the disk automatically with no human intervention.
  • After firmware changes: TPM unlock fails, and an administrator connects via SSH to enter the recovery credential.
  • TPM hardware failure: Same SSH-based recovery path.

This combination provides unattended normal operation with a network-accessible recovery path.

Security considerations

Default password removal

The default LUKS password ("keystone") is publicly documented and provides no security. It exists solely to enable automated deployments with nixos-anywhere. The enrollment commands remove this password after adding a secure credential. Systems that have not completed enrollment are effectively unencrypted from a security standpoint.

TPM and Secure Boot dependency

Keystone requires Secure Boot to be enabled before TPM enrollment (keystone.os.tpm.enable implies keystone.os.secureBoot.enable). Without Secure Boot, an attacker could boot an unsigned operating system, extract the TPM-sealed key, and access the encrypted data. The Secure Boot verification chain ensures that only trusted code runs before the TPM unseals the disk encryption key.

Recovery credential storage

Recovery credentials must be stored outside the encrypted disk. If the only copy of a recovery credential is on the encrypted disk, a TPM failure renders the data permanently unrecoverable. Recommended storage locations include password managers with offline backups, printed copies in physical safes, or safety deposit boxes for critical systems.

See also

  • Secure Boot for details on the verified boot chain that TPM enrollment depends on
  • Storage backends for the differences between ZFS and ext4 encryption models
  • Getting started for configuring encryption during initial deployment
  • Secrets management for application-level secret management with agenix (distinct from disk encryption keys)