ZFS on NixOS: Snapshots and Management

NixOS provides first-class ZFS support, making it straightforward to leverage ZFS's powerful snapshot and replication features.

Enabling ZFS

Basic Configuration

{ config, pkgs, ... }: {
  boot.supportedFilesystems = [ "zfs" ];
  boot.zfs.forceImportRoot = false;

  # Required: unique 8-character hex string
  networking.hostId = "a1b2c3d4";
}

Generate a host ID:

head -c 8 /etc/machine-id
# or
head -c 4 /dev/urandom | xxd -p

Kernel Considerations

ZFS modules are built for specific kernel versions. For stability:

{
  # Use LTS kernel for better ZFS compatibility
  boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;
}

Pool and Dataset Setup

Creating a Pool

# Single disk
zpool create tank /dev/sda

# Mirror
zpool create tank mirror /dev/sda /dev/sdb

# RAIDZ1 (single parity)
zpool create tank raidz1 /dev/sda /dev/sdb /dev/sdc

Creating Datasets

# Create datasets
zfs create tank/home
zfs create tank/var
zfs create tank/nix

# Set properties
zfs set compression=lz4 tank
zfs set atime=off tank
zfs set xattr=sa tank/home

Declarative Pool Management

You can manage mounts declaratively in NixOS:

{
  fileSystems."/" = {
    device = "tank/root";
    fsType = "zfs";
  };

  fileSystems."/home" = {
    device = "tank/home";
    fsType = "zfs";
  };

  fileSystems."/nix" = {
    device = "tank/nix";
    fsType = "zfs";
  };
}

Snapshot Basics

Manual Snapshots

# Create snapshot
zfs snapshot tank/home@2024-01-15

# Recursive snapshot (all child datasets)
zfs snapshot -r tank@before-upgrade

# List snapshots
zfs list -t snapshot

# List snapshots for specific dataset
zfs list -t snapshot -r tank/home

Accessing Snapshot Data

Snapshots are accessible via hidden .zfs directory:

# Browse snapshot
ls /home/.zfs/snapshot/2024-01-15/

# Recover a file
cp /home/.zfs/snapshot/2024-01-15/important-file.txt ~/

Rollback

# Rollback to snapshot (destroys newer data!)
zfs rollback tank/home@2024-01-15

# Force rollback, destroying intermediate snapshots
zfs rollback -r tank/home@2024-01-15

Destroying Snapshots

# Delete single snapshot
zfs destroy tank/home@old-snapshot

# Delete range
zfs destroy tank/home@snap1%snap5

Automated Snapshots with sanoid

sanoid automates snapshot creation and retention:

{
  services.sanoid = {
    enable = true;
    datasets = {
      "tank/home" = {
        autosnap = true;
        autoprune = true;
        hourly = 24;
        daily = 7;
        weekly = 4;
        monthly = 12;
        yearly = 1;
      };
      "tank/var" = {
        autosnap = true;
        autoprune = true;
        hourly = 12;
        daily = 7;
        weekly = 0;
        monthly = 0;
      };
    };
  };
}

This creates snapshots on schedule and automatically prunes old ones according to retention policy.

Send/Receive for Backups

ZFS can efficiently transmit snapshots to remote systems.

Full Send

# Send to file
zfs send tank/home@snap1 > /backup/home-snap1.zfs

# Send to remote system
zfs send tank/home@snap1 | ssh backup-server zfs recv backup/home

Incremental Send

# Send only changes between snapshots
zfs send -i @snap1 tank/home@snap2 | ssh backup-server zfs recv backup/home

# Send all intermediates
zfs send -I @snap1 tank/home@snap5 | ssh backup-server zfs recv backup/home

Automated Replication with syncoid

syncoid (companion to sanoid) automates replication:

{
  services.syncoid = {
    enable = true;
    commands = {
      "home-backup" = {
        source = "tank/home";
        target = "backup-server:backup/home";
        sendOptions = "w";  # Raw send for encrypted datasets
        recursive = true;
      };
    };
  };
}

NixOS-Specific Patterns

Boot Environments

Create a snapshot before major changes:

# Before nixos-rebuild
zfs snapshot -r tank@pre-rebuild-$(date +%Y%m%d)
nixos-rebuild switch

# If something breaks
zfs rollback -r tank@pre-rebuild-20240115

Impermanence Pattern

Run with tmpfs root and persist only specific paths:

{
  # Root is tmpfs, cleared on reboot
  fileSystems."/" = {
    device = "none";
    fsType = "tmpfs";
    options = [ "defaults" "size=2G" "mode=755" ];
  };

  # Persistent data on ZFS
  fileSystems."/nix" = {
    device = "tank/nix";
    fsType = "zfs";
  };

  fileSystems."/persist" = {
    device = "tank/persist";
    fsType = "zfs";
    neededForBoot = true;
  };

  # Symlink or bind-mount specific paths
  environment.persistence."/persist" = {
    directories = [
      "/var/lib"
      "/etc/nixos"
    ];
    files = [
      "/etc/machine-id"
    ];
  };
}

Requires the impermanence module.

Monitoring and Maintenance

Regular Scrubs

{
  services.zfs.autoScrub = {
    enable = true;
    interval = "weekly";
  };
}

Check Pool Health

zpool status
zpool status -v  # Verbose, shows errors

Automatic Snapshots Before Updates

Create a hook for nixos-rebuild:

# In a wrapper script
zfs snapshot -r tank@pre-rebuild-$(date +%Y%m%d-%H%M%S)
nixos-rebuild "$@"

Troubleshooting

Pool Won't Import

# Force import
zpool import -f tank

# Import with different root
zpool import -R /mnt tank

Snapshot Holds

# List holds preventing destruction
zfs holds tank/home@snapshot

# Release hold
zfs release tag tank/home@snapshot

Space Issues

# Check space usage
zfs list -o name,used,avail,refer,mountpoint

# Find large snapshots
zfs list -t snapshot -o name,used,refer -s used