blumeops/docs/changelog.d
Erich Blume dc69b8c68b C1: fix borgmatic shower SQLite dump (ssh to ringtail) (#357)
## Summary

Nightly borgmatic backups have been failing for 2 days. Root cause: the
shower SQLite dump `before_backup` hook (added in PR #349) referenced
`kubectl --context=k3s-ringtail`, but indri's kubeconfig deliberately
doesn't carry the ringtail credentials. The hook's failure aborted the
entire run, taking out *both* the local sifaka repo and the BorgBase
offsite. Verified the last good archive was `indri-2026-05-11T02:00`.

## Approach

ssh into ringtail and run `k3s kubectl` there — no indri-side
kubeconfig needed. `/etc/rancher/k3s/k3s.yaml` is mode 644 so no sudo
required, and the existing ssh access from indri to ringtail works.

Inline-shell quoting got hairy fast (fish on ringtail rejected `POD=...`
bash syntax; the nix shower image lacks `tar` so `kubectl cp` fails).
Pulled the dump logic into `~/bin/borgmatic-k8s-sqlite-dump`, deployed
by the ansible role. Each dump entry now declares a `target`:

- `local:<context>` — local kubectl with explicit context (mealie)
- `ssh:<user@host>` — ssh + `k3s kubectl` on the cluster host (shower)

Bytes come back via `kubectl exec ... -- cat` instead of `kubectl cp`
since `cp` needs `tar` in the pod (nix-built containers don't bundle it).

## Test plan

- [x] `mise run provision-indri -- --tags borgmatic --check --diff` shows expected diff
- [x] Apply, helper script deployed at `~/bin/borgmatic-k8s-sqlite-dump`
- [x] Helper invoked directly with `ssh:eblume@ringtail` produces a valid 288 KB SQLite file
- [x] Full `borgmatic create` completes without errors — both mealie.db (1.7 MB) and shower.db (288 KB) appear in `~/.local/share/borgmatic/k8s-dumps/`, archive `indri-2026-05-13T17:31:02` written to sifaka borg repo

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #357
2026-05-13 18:55:50 -07:00
..
+agent-file-neutralization.ai.md C0: adopt AGENTS.md as canonical agent config 2026-04-18 20:15:30 -07:00
+alloy-main-sha-rebuild.infra.md C0: alloy — bump kustomization tags to main-branch SHA 2026-05-01 08:31:27 -07:00
+alloy-native-macos-v1.16.0.infra.md C0: alloy native macOS on indri — upgrade to v1.16.0 2026-05-01 10:36:38 -07:00
+argocd-resource-limits.infra.md Add resource limits to ArgoCD pods to prevent unbounded consumption 2026-04-18 13:04:27 -07:00
+blumeops-tasks-due-recurrence.feature.md C0: blumeops-tasks — replace ambiguous due:+N with "Nd overdue" 2026-04-27 11:49:46 -07:00
+claude-md-import-agents.ai.md C0: CLAUDE.md — import AGENTS.md instead of redirecting to it 2026-04-27 11:41:13 -07:00
+compliance-mute-categories.doc.md C0: docs — explanation article on compliance mute categories 2026-05-04 18:19:53 -07:00
+container-build-suggest-runner-logs.misc.md C0: suggest mise run runner-logs in container-build-and-release 2026-04-21 10:12:00 -07:00
+fix-forge-static-assets.bugfix.md Fix forge.eblu.me static assets by adding missing Host header 2026-04-18 16:00:56 -07:00
+frigate-notify-local.infra.md Add frigate-notify nix container build (#339) 2026-04-21 09:28:02 -07:00
+homepage-config-perms-fix.bugfix.md C0: fix homepage container /app/config write permissions 2026-05-10 20:49:22 -07:00
+prowler-rebuild-on-main.infra.md C0: prowler — rebuild image against main HEAD 2026-04-29 10:49:27 -07:00
+remove-devpi-container-build.misc.md C0: remove containers/devpi/ build artifact 2026-04-29 13:40:45 -07:00
+review-cc-ephemeral-privileged-jobs.misc.md C0: review CC ephemeral-privileged-jobs 2026-04-29 11:09:34 -07:00
+review-cc-init-container-isolation.misc.md C0: review CC init-container-isolation — defer retirement to post-ringtail 2026-05-04 18:31:13 -07:00
+review-cc-trusted-ci-only.misc.md C0: review compensating control trusted-ci-only 2026-05-01 10:49:22 -07:00
+review-compliance-image-iac.feature.md C0: review-compliance-reports — summarize image and IaC scans 2026-04-27 12:18:06 -07:00
+review-contributing-doc.doc.md Review contributing tutorial: add last-reviewed, .ai.md fragment type, prek provenance 2026-04-21 08:53:41 -07:00
+review-index-doc.doc.md C0: doc review — index.md, add ringtail to infra overview 2026-05-06 06:14:40 -07:00
+review-navidrome-doc.doc.md C0: review navidrome reference doc 2026-04-18 20:25:19 -07:00
+review-ollama-doc.doc.md C0: review ollama doc — refresh image, models, last-reviewed 2026-05-01 10:42:33 -07:00
+ringtail-sway-fuzzel.bugfix.md C0: ringtail — restore sway default keybindings, fix fuzzel border config 2026-04-23 12:16:02 -07:00
+rotate-fly-deploy-token-shell-examples.doc.md C0: rotate-fly-deploy-token — fish+bash one-shot, op validator gotcha 2026-05-04 13:42:57 -07:00
+runner-logs-auth.feature.md Fetch job logs via SSH to indri instead of Forgejo web endpoint 2026-04-18 17:08:46 -07:00
+shower-main-sha-rebuild.infra.md C0: shower — rebuild from main SHA (post-PR-349 retag) 2026-05-11 13:55:25 -07:00
+shower-rebuild-from-main-sha.misc.md C0: shower — rebuild from main SHA (post-merge retag) 2026-05-11 20:20:39 -07:00
+tailscale-main-sha-rebuild.infra.md C0: tailscale main-SHA rebuild for ringtail proxyclass 2026-05-06 06:52:39 -07:00
+transmission-doc-review.doc.md C0: transmission doc — review and correct storage/monitoring details 2026-04-29 11:00:01 -07:00
+valkey-main-tag-bump.infra.md C0: valkey — bump kustomization tags to main-branch SHA 2026-05-01 17:47:16 -07:00
+zot-v2.1.16.infra.md C0: zot — upgrade indri registry to v2.1.16 2026-05-04 17:41:07 -07:00
.gitkeep Add towncrier changelog system (#86) 2026-02-03 11:48:13 -08:00
alloy-v1.16.0.infra.md Alloy V1.16.0 (#345) 2026-05-01 08:05:37 -07:00
cleanup-cv-docs-minikube-artifacts.misc.md C1: clean up cv + docs minikube artifacts (#343) 2026-04-29 15:18:39 -07:00
dagger-0-20-6-runner-image-alpine.infra.md Bump Dagger to 0.20.6 and migrate runner-job-image to Alpine container.py 2026-04-21 08:28:18 -07:00
doc-review-replicating-blumeops.doc.md C1: doc review — replicating-blumeops tutorial (#350) 2026-05-11 16:11:35 -07:00
fix-borgmatic-shower-via-ssh.bugfix.md C1: fix borgmatic shower SQLite dump (ssh to ringtail) (#357) 2026-05-13 18:55:50 -07:00
forgejo-runner-v12-8-server-connections.infra.md Upgrade forgejo-runner to v12.8, adopt server.connections, and clean up docs (#338) 2026-04-20 09:03:54 -07:00
homepage-to-ringtail.infra.md C1: migrate homepage dashboard to ringtail k3s 2026-05-10 20:37:03 -07:00
migrate-cv-docs-to-indri.infra.md C1: migrate cv + docs from minikube to indri-native (#342) 2026-04-29 14:55:11 -07:00
migrate-devpi-to-indri.infra.md Migrate devpi from minikube to indri (launchd) (#341) 2026-04-29 13:38:36 -07:00
migrate-immich-to-ringtail.infra.md C2: migrate immich from minikube to ringtail (mikado chain) (#356) 2026-05-13 16:46:17 -07:00
mirror-tailscale-container.infra.md C1: mirror tailscale container locally for ringtail proxyclass (#347) 2026-05-06 06:50:31 -07:00
prowler-iac-mutelist.infra.md Address 6 critical Prowler IaC findings (mute + grafana RBAC tighten) (#340) 2026-04-29 10:43:32 -07:00
review-cc-observability-stack-audit-2026-05-11.infra.md C1: review CC observability-stack-audit (extend to k3s) (#353) 2026-05-11 16:10:39 -07:00
review-ringtail-flake-2026-05-11.infra.md C1: ringtail — weekly flake.lock update (#352) 2026-05-11 16:13:07 -07:00
ringtail-static-ip.infra.md fix(ringtail): explicitly enable net.ipv4.ip_forward 2026-05-12 09:51:16 -07:00
service-review-mealie-2026-05-11.infra.md C1: service review — mealie (v3.12.0 deployed; upstream v3.17.0) (#351) 2026-05-11 16:12:36 -07:00
shower-app-deploy.bugfix.md C1: deploy adelaide-baby-shower-app to ringtail k3s (#349) 2026-05-11 13:47:18 -07:00
shower-app-deploy.feature.md C1: deploy adelaide-baby-shower-app to ringtail k3s (#349) 2026-05-11 13:47:18 -07:00
shower-app-deploy.infra.md C1: deploy adelaide-baby-shower-app to ringtail k3s (#349) 2026-05-11 13:47:18 -07:00
shower-v1.1.0.feature.md C1: deploy shower v1.1.0 (phases + guest memories) (#354) 2026-05-11 20:08:03 -07:00
update-tooling-deps-2026-04.doc.md C1: SHA-pin tooling dependencies (2026-04 cycle) (#344) 2026-04-30 16:51:43 -07:00
update-tooling-deps-2026-04.infra.md C1: SHA-pin tooling dependencies (2026-04 cycle) (#344) 2026-04-30 16:51:43 -07:00
valkey-mirror.infra.md Mirror valkey 8.1 locally for paperless and immich (#346) 2026-05-01 17:40:03 -07:00