- Nix 32.5%
- Jinja 21.5%
- Python 17.9%
- Shell 11.8%
- Go 8.1%
- Other 8.2%
Erich Blume
ca0c9354ee
## Summary - Add `authentik` database (blumeops-pg cluster) to borgmatic pg_dump backups - Add `immich` database (immich-pg cluster) to borgmatic pg_dump backups - For immich-pg: new borgmatic managed role with `pg_read_all_data`, ExternalSecret, Tailscale LoadBalancer service, and Caddy L4 TCP proxy on port 5433 - Update backup docs to reflect all four CNPG databases + mealie SQLite ## Deploy plan Deploy order matters — k8s resources must exist before ansible can route to them: 1. **ArgoCD (databases app):** sync to pick up immich-pg borgmatic role, ExternalSecret, and Tailscale service ``` argocd app set blumeops-pg --revision feature/borgmatic-all-pg-backups argocd app sync blumeops-pg ``` 2. **Wait** for `immich-pg-tailscale` service to get a Tailscale IP and `immich-pg.tail8d86e.ts.net` to resolve 3. **Ansible (caddy):** deploy Caddy L4 route for port 5433 ``` mise run provision-indri -- --tags caddy ``` 4. **Ansible (borgmatic):** deploy updated config and .pgpass ``` mise run provision-indri -- --tags borgmatic ``` 5. **Verify:** trigger a manual borgmatic run and check all four pg_dump streams succeed ``` borgmatic --verbosity 1 2>&1 | grep -E '(Dumping|ERROR)' ``` ## Test plan - [x] `kubectl kustomize` builds cleanly - [x] `ansible --check --diff` for borgmatic and caddy show expected changes - [ ] ArgoCD sync succeeds for databases app - [ ] `immich-pg.tail8d86e.ts.net` resolves - [ ] `pg.ops.eblu.me:5433` accepts connections - [ ] `borgmatic --verbosity 1` dumps all four databases without errors Reviewed-on: #314 |
||
|---|---|---|
| .claude | ||
| .dagger | ||
| .forgejo/workflows | ||
| .github | ||
| ansible | ||
| argocd | ||
| containers | ||
| docs | ||
| fly | ||
| mise-tasks | ||
| nixos/ringtail | ||
| pulumi | ||
| utils/qart | ||
| .ansible-lint | ||
| .gitignore | ||
| .yamllint.yaml | ||
| Brewfile | ||
| CHANGELOG.md | ||
| CLAUDE.md | ||
| dagger.json | ||
| LICENSE | ||
| mise.toml | ||
| prek.toml | ||
| README.md | ||
| service-versions.yaml | ||
| towncrier.toml | ||
| update-loki-3.6.7.infra.md | ||
blumeops
aka "Blue Mops"
Tools and configuration for Erich Blume's personal infrastructure, orchestrated across a Tailscale tailnet.
This is a homelab, but it's also a testing ground for AI-assisted infrastructure development. Much of this codebase was co-authored with Claude Code, and the repo places heavy emphasis on documentation, process, and change classification to make that collaboration work well. I don't know entirely how I feel about LLMs in our current era (there are real concerns about how training data is sourced and energy subsidy) but it felt important to learn how to work with these tools.
The full documentation is published at docs.eblu.me
and lives in the docs/ directory, structured around the
Diataxis framework and designed to be compatible with
Obsidian/Obsidian.nvim.
What runs here
Services are a mix of Kubernetes pods (managed by ArgoCD), macOS LaunchAgent services (managed by Ansible), and NixOS systemd services (managed by Nix flakes), all connected via Tailscale:
- Indri (Mac Mini M1) - primary server. Most services run in Minikube via ArgoCD; Forgejo, Caddy, and others run natively as LaunchAgent services via Ansible.
- Ringtail (NixOS desktop, RTX 4080) - GPU workloads (Frigate NVR, Authentik SSO) on k3s, plus NixOS systemd services.
- Sifaka (Synology NAS) - backup target and bulk storage.
Notable services include Grafana/Prometheus/Loki observability, Immich photos, Jellyfin media, Forgejo git forge, a Zot container registry, and more. Public access is routed through a Fly.io proxy; everything else is tailnet-only.
Project structure
ansible/ Ansible playbooks and roles (indri, sifaka)
argocd/apps/ ArgoCD Application definitions
argocd/manifests/ Kubernetes manifests per service
containers/ Custom container builds (Dockerfile + Nix)
docs/ Diataxis documentation (published at docs.eblu.me)
fly/ Fly.io public proxy configuration
mise-tasks/ Operational scripts run via mise
nixos/ NixOS configuration for ringtail
pulumi/ Pulumi IaC (Tailscale ACLs, Gandi DNS)
.dagger/ Dagger CI pipelines
.forgejo/ Forgejo Actions CI/CD workflows
Getting started
You'll need Homebrew and mise:
brew bundle # install CLI tools (argocd, tea, flyctl, etc.)
mise install # install managed toolchains (ansible, pulumi, dagger, etc.)
prek install # set up git hooks
Git hooks (via prek) enforce secret scanning
(TruffleHog), linting, formatting, and custom checks like doc link validation
and the Mikado branch invariant. They run automatically on git commit.
Operational tasks are driven through mise. Run mise tasks to see what's
available. Key examples:
mise run provision-indri # deploy to indri via Ansible
mise run services-check # verify service health
mise run container-list # list tracked container images
AI-assisted development
This repo is designed to be worked on by both humans and AI agents. The
CLAUDE.md file provides instructions for Claude Code, and the
docs/tutorials/ai-assistance-guide.md
explains the full workflow.
Changes are classified before starting work:
- C0 - quick fixes, committed directly to main
- C1 - feature branch + PR, documentation written before code
- C2 - multi-phase work using the Mikado method for dependency tracking
See the agent change process for details.