eblume/blumeops
1
1
Fork 0
Code Issues Pull requests Projects Releases 83 Packages Wiki Activity Actions
blumeops/docs/reference/storage/backups.md
Erich Blume ca0c9354ee Add borgmatic backups for authentik and immich databases (#314) 
## Summary

- Add `authentik` database (blumeops-pg cluster) to borgmatic pg_dump backups
- Add `immich` database (immich-pg cluster) to borgmatic pg_dump backups
- For immich-pg: new borgmatic managed role with `pg_read_all_data`, ExternalSecret, Tailscale LoadBalancer service, and Caddy L4 TCP proxy on port 5433
- Update backup docs to reflect all four CNPG databases + mealie SQLite

## Deploy plan

Deploy order matters — k8s resources must exist before ansible can route to them:

1. **ArgoCD (databases app):** sync to pick up immich-pg borgmatic role, ExternalSecret, and Tailscale service
   ```
   argocd app set blumeops-pg --revision feature/borgmatic-all-pg-backups
   argocd app sync blumeops-pg
   ```
2. **Wait** for `immich-pg-tailscale` service to get a Tailscale IP and `immich-pg.tail8d86e.ts.net` to resolve
3. **Ansible (caddy):** deploy Caddy L4 route for port 5433
   ```
   mise run provision-indri -- --tags caddy
   ```
4. **Ansible (borgmatic):** deploy updated config and .pgpass
   ```
   mise run provision-indri -- --tags borgmatic
   ```
5. **Verify:** trigger a manual borgmatic run and check all four pg_dump streams succeed
   ```
   borgmatic --verbosity 1 2>&1 | grep -E '(Dumping|ERROR)'
   ```

## Test plan

- [x] `kubectl kustomize` builds cleanly
- [x] `ansible --check --diff` for borgmatic and caddy show expected changes
- [ ] ArgoCD sync succeeds for databases app
- [ ] `immich-pg.tail8d86e.ts.net` resolves
- [ ] `pg.ops.eblu.me:5433` accepts connections
- [ ] `borgmatic --verbosity 1` dumps all four databases without errors

Reviewed-on: #314
2026-03-27 16:59:58 -07:00

2.4 KiB
Raw Blame History

title modified tags
Backups 2026-03-27
storage
backup

Backup Policy

Daily automated backups from indri to sifaka NAS.

Schedule

Time Frequency System
2:00 AM Daily borgmatic

What Gets Backed Up

Directories

Path Description Priority
~/code/personal/zk Zettelkasten notes Critical
/opt/homebrew/var/forgejo Git repositories Critical
~/.config/borgmatic Backup config High
~/Documents Personal documents (includes 1password encrypted export) High

Databases

Database Cluster Host Method
miniflux blumeops-pg [[postgresql pg.ops.eblu.me:5432]]
teslamate blumeops-pg [[postgresql pg.ops.eblu.me:5432]]
authentik blumeops-pg [[postgresql pg.ops.eblu.me:5432]]
immich immich-pg [[postgresql pg.ops.eblu.me:5433]]
mealie — (SQLite) k8s pod kubectl exec sqlite3 .backup

Sifaka-Native Data

Some data lives directly on sifaka rather than being backed up to it (photos via immich, music via navidrome, video via jellyfin). See sifaka for data protection details.

What Is NOT Backed Up

Data Reason
ZIM archives (~/transmission/) Re-downloadable via torrent
Prometheus metrics Ephemeral, in k8s PVC
Loki logs Ephemeral, in k8s PVC
devpi cache Re-fetchable from PyPI

Retention Policy

Period Retention
Daily 7 backups
Monthly 12 backups
Yearly 1000 backups

Backup Targets

Repository Location Label
/Volumes/backups/borg/ sifaka (local NAS) —
ssh://u3ugi1x1@u3ugi1x1.repo.borgbase.com/./repo BorgBase (offsite) borgbase-offsite

Monitoring

Metrics exposed to prometheus:

  • borgmatic_up - Repository accessible
  • borgmatic_last_archive_timestamp - Last backup time
  • borgmatic_repo_deduplicated_size_bytes - Disk usage

Dashboard: "Borgmatic Backups" in grafana

Related