Erich Blume
e4a8405de7
## Summary - Remove stale `/opt/homebrew/var/loki` from borgmatic backup (Loki migrated to k8s) - Add Alloy k8s DaemonSet for automatic pod log collection with auto-discovery - Add blackbox probes for miniflux, kiwix, transmission, devpi, argocd - Add transmission-exporter sidecar for full metrics (speed, torrent counts, ratios) - Replace stale devpi dashboard with probe-based metrics (status, response time, uptime) - Add unified "K8s Services Health" dashboard for service uptime/response monitoring ## Manual cleanup already performed - Deleted stale textfile metrics on indri: `devpi.prom`, `transmission.prom` - Deleted stale data directories on indri: `/opt/homebrew/var/loki/`, `/opt/homebrew/var/prometheus/` ## Deployment and Testing - [x] Sync `apps` application to pick up new alloy-k8s app - [x] Deploy alloy-k8s on feature branch: `argocd app set alloy-k8s --revision feature/observability-cleanup && argocd app sync alloy-k8s` - [x] Deploy torrent on feature branch (for transmission exporter): `argocd app set torrent --revision feature/observability-cleanup && argocd app sync torrent` - [x] Deploy prometheus on feature branch (for new scrape config): `argocd app set prometheus --revision feature/observability-cleanup && argocd app sync prometheus` - [x] Deploy grafana-config on feature branch (for dashboards): `argocd app set grafana-config --revision feature/observability-cleanup && argocd app sync grafana-config` - [x] Verify pod logs appear in Loki/Grafana - [x] Verify transmission metrics appear in Prometheus - [x] Verify service probe metrics appear in Prometheus - [x] Run `mise run provision-indri -- --tags borgmatic` to update borgmatic config - [ ] After merge, reset apps to main and resync 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.tail8d86e.ts.net/eblume/blumeops/pulls/43
48 lines
1.4 KiB
YAML
48 lines
1.4 KiB
YAML
---
|
|
borgmatic_config: /Users/erichblume/.config/borgmatic/config.yaml
|
|
borgmatic_config_dir: /Users/erichblume/.config/borgmatic
|
|
borgmatic_log_dir: /Users/erichblume/Library/Logs
|
|
|
|
# Full path to borg binary since LaunchAgent doesn't have homebrew in PATH
|
|
borgmatic_local_path: /opt/homebrew/bin/borg
|
|
|
|
# Schedule: runs daily at 2:00 AM
|
|
borgmatic_schedule_hour: 2
|
|
borgmatic_schedule_minute: 0
|
|
|
|
# Source directories to back up
|
|
borgmatic_source_directories:
|
|
- /Users/erichblume/code/personal/zk
|
|
- /opt/homebrew/var/forgejo
|
|
- /Users/erichblume/.config/borgmatic
|
|
- /Users/erichblume/Documents
|
|
- /Users/erichblume/Pictures
|
|
|
|
# Backup repository
|
|
borgmatic_repositories:
|
|
- path: /Volumes/backups/borg/
|
|
label: sifaka-borg-backups
|
|
encryption: repokey
|
|
append_only: true
|
|
|
|
# Exclude patterns
|
|
borgmatic_exclude_patterns: []
|
|
|
|
# Encryption passcommand (reads borg passphrase)
|
|
borgmatic_encryption_passcommand: cat /Users/erichblume/.borg/config.yaml
|
|
|
|
# Retention policy
|
|
borgmatic_keep_daily: 7
|
|
borgmatic_keep_monthly: 12
|
|
borgmatic_keep_yearly: 1000
|
|
|
|
# PostgreSQL databases to backup (streamed via pg_dump)
|
|
# Password is read from ~/.pgpass (managed by this role)
|
|
# pg_dump_command must be full path since LaunchAgent doesn't have homebrew in PATH
|
|
borgmatic_pg_dump_command: /opt/homebrew/opt/postgresql@18/bin/pg_dump
|
|
borgmatic_postgresql_databases:
|
|
# k8s PostgreSQL (CloudNativePG)
|
|
- name: miniflux
|
|
hostname: pg.tail8d86e.ts.net
|
|
port: 5432
|
|
username: borgmatic
|