Erich Blume 14ca0160ba Migrate devpi from minikube to indri (launchd) (#341 )

## Summary

Devpi was crash-looping under memory pressure on the minikube StatefulSet, breaking the Python toolchain across the repo (`mise run docs-mikado`, `prek`, every `uv pip install`). It moves to indri as a native LaunchAgent.

## What changed

- **New ansible role** `ansible/roles/devpi/`: installs `devpi-server` + `devpi-web` into a uv-managed venv, initializes the server-dir on first run via 1Password root password, runs as a LaunchAgent (`mcquack.eblume.devpi`) bound to `127.0.0.1:3141`. Bootstraps from upstream PyPI (so devpi can install itself on a fresh box).
- **Caddy**: `pypi.ops.eblu.me` now proxies to `http://localhost:3141`.
- **Playbook**: `indri.yml` gains pre_tasks for the root password and the new role.
- **service-versions.yaml**: devpi flipped from `type: argocd` to `type: ansible`.
- **ArgoCD**: removed `apps/devpi.yaml` and `manifests/devpi/`. The in-cluster Application, namespace, and PVC have been deleted.
- **Docs**: new how-to `docs/how-to/operations/devpi-on-indri.md`; `restart-indri.md` lists devpi in the LaunchAgent stop list.

## Already deployed (live on indri)

- Service running: `launchctl list mcquack.eblume.devpi` → PID 53888
- `curl https://pypi.ops.eblu.me/+api` returns 200 ✅
- `mise run docs-mikado` works again ✅
- 1.0G of cached PyPI data was migrated from the PVC to `~erichblume/devpi/server-dir/`
- Minikube namespace and PVC fully reclaimed

## Test plan

- [ ] `mise run services-check` (after merge)
- [ ] CI workflows that use devpi succeed
- [ ] No regressions in tools that depend on `pypi.ops.eblu.me` (prek, uv-script tasks, dagger pipelines)

## Context

This is the C1 prelude to a planned C2 chain (`mikado/retire-minikube-indri`) to retire minikube on indri entirely. Doing devpi as a standalone C1 was the right call because (a) it was urgent — it was breaking the toolchain — and (b) it shakes out the migration recipe before we commit to a multi-leaf chain.

Reviewed-on: #341

2026-04-29 13:38:36 -07:00

3.2 KiB

Raw Blame History

title

modified

Backup Policy

Daily automated backups from indri to sifaka NAS.

Schedule

Time	Frequency	System
2:00 AM	Daily	borgmatic

What Gets Backed Up

Directories

Path	Description	Priority
`~/code/personal/zk`	Zettelkasten notes	Critical
`/opt/homebrew/var/forgejo`	Git repositories	Critical
`~/.config/borgmatic`	Backup config	High
`~/Documents`	Personal documents (includes 1password encrypted export)	High

Databases

Database	Cluster	Host	Method
miniflux	blumeops-pg	[[postgresql	pg.ops.eblu.me:5432]]
teslamate	blumeops-pg	[[postgresql	pg.ops.eblu.me:5432]]
authentik	blumeops-pg	[[postgresql	pg.ops.eblu.me:5432]]
immich	immich-pg	[[postgresql	pg.ops.eblu.me:5433]]
mealie	— (SQLite)	k8s pod	kubectl exec sqlite3 .backup

Immich Photo Library (Offsite Only)

The immich photo library lives on sifaka at /volume1/photos (SMB-mounted on indri as /Volumes/photos). Since sifaka is already the local backup target, photos are backed up to BorgBase offsite only — not back to sifaka.

Property	Value
Config	`~/.config/borgmatic/photos.yaml`
Schedule	Daily at 4:00 AM (offset from main backup)
Source	`/Volumes/photos` (sifaka SMB mount)
Target	BorgBase `borgbase-immich-photos` repo
Size	~128 GB

Uses the same encryption passphrase and SSH key as the main borgmatic config.

Sifaka-Native Data

Other data lives directly on sifaka (music via navidrome, video via jellyfin). See sifaka for data protection details.

What Is NOT Backed Up

Data	Reason
ZIM archives (`~/transmission/`)	Re-downloadable via torrent
Prometheus metrics	Ephemeral, in k8s PVC
Loki logs	Ephemeral, in k8s PVC
devpi cache (`~/devpi/server-dir/` on indri)	Re-fetchable from PyPI on first request

Retention Policy

Period	Retention
Daily	7 backups
Monthly	12 backups
Yearly	1000 backups

Backup Targets

Repository	Location	Label	Backs up
`/Volumes/backups/borg/`	sifaka (local NAS)	`sifaka-borg-backups`	indri data
`ssh://u3ugi1x1@...repo.borgbase.com/./repo`	BorgBase (offsite)	`borgbase-offsite`	indri data
`ssh://xcrtl5tg@...repo.borgbase.com/./repo`	BorgBase (offsite)	`borgbase-immich-photos`	immich photos

Monitoring

Metrics exposed to prometheus:

borgmatic_up - Repository accessible
borgmatic_last_archive_timestamp - Last backup time
borgmatic_repo_deduplicated_size_bytes - Disk usage

Dashboard: "Borgmatic Backups" in grafana

borgmatic - Backup system details
sifaka - Backup storage
postgresql - Database backups
restore-1password-backup - Recover 1Password from backup

3.2 KiB Raw Blame History