Zrepl – ZFS replication

  Job: prod_to_backups
  Type: push
  Replication:
      Attempt #1
      Status: fan-out-filesystems
      Progress: [=========================----] 246.7 MiB / 264.7 MiB @ 11.5 MiB/s
        zroot              STEPPING (step 1/2, 624 B/1.2 KiB) next: @a => @b
        zroot/ROOT         DONE (step 2/2, 1.2 KiB/1.2 KiB)
      * zroot/ROOT/default STEPPING (step 1/2, 123.4 MiB/129.3 MiB) next: @a => @b
        zroot/tmp          STEPPING (step 1/2, 29.9 KiB/44.2 KiB) next: @a => @b
        zroot/usr          STEPPING (step 1/2, 624 B/1.2 KiB) next: @a => @b
      * zroot/usr/home     STEPPING (step 1/2, 123.3 MiB/135.3 MiB) next: @a => @b
        zroot/var          STEPPING (step 1/2, 624 B/1.2 KiB) next: @a => @b
        zroot/var/audit    DONE (step 2/2, 1.2 KiB/1.2 KiB)
        zroot/var/crash    DONE (step 2/2, 1.2 KiB/1.2 KiB)
        zroot/var/log      STEPPING (step 1/2, 22.0 KiB/29.2 KiB) next: @a => @b
        zroot/var/mail     STEPPING (step 1/2, 624 B/1.2 KiB) next: @a => @b
  Pruning Sender:
      ...
  Pruning Receiver:
  

Main Features¶

• Filesystem replication
  • [x] Pull & Push mode
  • [x] Multiple transport modes: TCP, TCP + TLS client auth, SSH
  • Advanced replication features
    • [x] Automatic retries for temporary network errors
    • [x] Automatic resumable send & receive
    • [x] Automatic ZFS holds during send & receive
    • [x] Automatic bookmark & hold management for guaranteed incremental send & recv
    • [x] Encrypted raw send & receive to untrusted receivers (OpenZFS native encryption)
    • [x] Properties send & receive
    • [x] Compressed send & receive
    • [x] Large blocks send & receive
    • [x] Embedded data send & receive
    • [x] Resume state send & receive
    • [x] Bandwidth limiting
• Automatic snapshot management
  • [x] Periodic filesystem snapshots
  • [x] Support for pre- and post-snapshot hooks with builtins for MySQL & Postgres
  • [x] Flexible pruning rule system
    • [x] Age-based fading (grandfathering scheme)
    • [x] Bookmarks to avoid divergence between sender and receiver
• Sophisticated Monitoring & Logging
  • [x] Live progress reporting via zrepl status subcommand
  • [x] Comprehensive, structured logging
    • human, logfmt and json formatting
    • stdout, syslog and TCP (+TLS client auth) outlets
  • [x] Prometheus monitoring endpoint
• Maintainable implementation in Go
  • [x] Cross platform
  • [x] Dynamic feature checking
  • [x] Type safe & testable code

Contributing¶

We are happy about any help we can get!

• Financial Support
• Explore the codebase
  • These docs live in the docs/ subdirectory
• Document any non-obvious / confusing / plain broken behavior you encounter when setting up zrepl for the first time
• Check the Issues and Projects sections for things to do.
  The good first issues and docs are suitable starting points.