C1: shower-specific rate-limit zone for venue-wifi NAT

Default `general` zone (10r/s burst=20) is tuned for internet drive-by
traffic. At the party, 30 guests scanning the splash QR from one
venue-wifi NAT'd public IP would each fetch HTML + ~5 static assets
within a few seconds — easily clearing burst=20, and the second-wave
guests would see 503 with no auto-retry.

New shower_general zone (50r/s burst=200) absorbs that simultaneous-
load spike. Exploit scanners still trip it: the 45.88.138.44 burst
we already saw in Loki fired ~30 req in 2s, well above the new
sustained 50r/s when extrapolated, and burst=200 is still a hard cap
on instantaneous spikes.

Self-healing: `limit_req` is a token bucket — no persistent ban,
nothing to manually flush. A guest who trips it auto-recovers within
~1s; tuning here is about not tripping it on legit traffic in the
first place.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

.gitignore

@@ -1,6 +1,5 @@
 .claude/settings.local.json
 .claude/agent-memory/
-.claude/scheduled_tasks.lock
 
 # Python
 __pycache__/

AGENTS.md

@@ -65,7 +65,7 @@ See [[agent-change-process]] for the full methodology.
 ./pulumi/               # Pulumi IaC (tailnet ACLs, dns, cloud)
 ~/.config/{nvim,fish}   # user's shell config, managed by chezmoi
 ~/code/personal/        # user's projects
-~/code/personal/zk      # user's zettelkasten (Obsidian-sync). Reference-data source; migrating into heph docs (hephaestus).
+~/code/personal/zk      # user's Obsidian-sync managed zettelkasten. Potential source for reference data.
 ~/code/3rd/             # mirrored external projects
 ~/code/work             # FORBIDDEN
 ```
@@ -147,16 +147,10 @@ Create a new spork: `mise run spork-create <mirror-name>`
 
 ## Task Discovery
 
-BlumeOps tasks live in [hephaestus](https://github.com/eblume/hephaestus) (`heph`),
-the user's self-hosted context/task system. Fetch them with the CLI:
-
 ```fish
-heph list --project Blumeops --json  # outstanding Blumeops tasks as JSON
+mise run blumeops-tasks  # fetch from Todoist, sorted by priority
 ```
-
-(This replaced the retired `blumeops-tasks` mise task, which read from Todoist.)
-
-Most operational scripts are stored in `./mise-tasks/`. For scripts with any logic or
+Most tasks are stored in `./mise-tasks/`. For scripts with any logic or
 complexity, use uv run --script 's with explicit dependencies. Complex
 workflows with artifacts should become dagger pipelines. Mise tasks are for
 development processes and operations - tools for the user or the agent.

CHANGELOG.md

@@ -12,259 +12,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 <!-- towncrier release notes start -->
 
-## [v1.17.0] - 2026-06-03
-
-### Features
-
-- Deploy the Adelaide / Heidi / Addie baby shower app — guest splash, raffle
-  picker, and prize assignment console — on ringtail k3s with `shower.eblu.me`
-  as the public entry and `shower.ops.eblu.me` as the tailnet admin host. App
-  source: [`adelaide-baby-shower-app`](https://forge.eblu.me/eblume/adelaide-baby-shower-app).
-- Deploy adelaide-baby-shower-app v1.1.0 to ringtail k3s. Replaces the
-  boolean lock with a four-phase `ShowerState` (`pre_event` → `party` →
-  `prizes_locked` → `event_locked`), adds an append-only "guest memories"
-  panel where guests can leave photos and comments for the baby, and
-  polishes the admin and QR views. Three Django migrations
-  (`0009_shower_phase`, `0010_guest_memories`, `0011_book_description`)
-  run automatically in the entrypoint against the SQLite PV. No config
-  or env-var changes.
-
-  Container build also gains a Forgejo-PyPI workaround: Forgejo's simple
-  index returns absolute file URLs hardcoded to the public ROOT_URL
-  (`forge.eblu.me`), which the Fly edge 403s on `/api/packages/*`. The
-  wheel and sdist are now both pulled via direct `fetchurl` against
-  `forge.ops.eblu.me` (tailnet-only) and the wheel is handed to pip as
-  a local path.
-- `review-compliance-reports` now also fetches and summarizes the weekly Prowler container-image and IaC scans (previously only the K8s CIS in-cluster scan was processed). For each scan it shows status counts, severity breakdown, week-over-week delta, and — for the high-volume image/IaC scans — top-N tables grouped by check ID and resource instead of per-finding listings.
-- runner-logs now authenticates with Forgejo API token and auto-detects the repo from git remote. Job logs are fetched via SSH to indri (reading Forgejo's on-disk zstd log files) instead of the web endpoint, which doesn't support token auth for private repos.
-
-### Bug Fixes
-
-- Fix nightly borgmatic backups failing for 2 days. The shower SQLite
-  dump hook referenced `kubectl --context=k3s-ringtail`, but indri's
-  kubeconfig deliberately doesn't carry the ringtail credentials. The
-  `before_backup` hook's failure aborted the entire run, taking out
-  *both* the local sifaka repo and the BorgBase offsite. Replaced
-  the inline-shell dump with a `~/bin/borgmatic-k8s-sqlite-dump`
-  helper deployed by the ansible role. Each dump entry now declares a
-  `target` of either `local:<context>` (mealie — kubectl uses indri's
-  kubeconfig) or `ssh:<user@host>` (shower — ssh into ringtail and
-  run `k3s kubectl` there, no indri-side kubeconfig needed; k3s.yaml
-  on ringtail is mode 644 so no sudo required). Bytes stream back via
-  `kubectl exec ... -- cat` rather than `kubectl cp`, since `kubectl
-  cp` requires `tar` inside the pod and nix-built images like shower
-  don't bundle it.
-- Shower app container now bakes the wheel + Python deps into the image
-  at build time via `buildPythonPackage` instead of pip-installing on
-  first boot. Boots are deterministic and don't depend on forge PyPI
-  being reachable from the pod. The `wheelHash` in
-  `containers/shower/default.nix` is the sha256 sourced from the
-  [forge PyPI simple index](https://forge.eblu.me/api/packages/eblume/pypi/simple/adelaide-baby-shower-app/);
-  bumping the version means bumping that hash too.
-
-  Borgmatic now covers the shower app: SQLite is dumped from the live
-  pod via `kubectl exec` (mirroring the existing mealie entry, with
-  `context: k3s-ringtail`), and the prize-photo media share is picked up
-  through `/Volumes/shower` (sifaka SMB mount on indri, same pattern as
-  `/Volumes/photos`).
-- Disabled adaptive sync (VRR) on ringtail's DP-1 output. The OMEN 27i IPS panel pumps brightness when its refresh rate swings into the low VRR range during low-framerate content (e.g. game cutscenes), producing a flicker that worsened over a session until a reboot. Pinning the panel to a fixed 165Hz eliminates it.
-- Fixed forge.eblu.me static assets (CSS, JS, images, fonts) not loading — the proxy's static asset cache block was missing the `Host` header, so Caddy couldn't route the requests.
-- Fixed homepage container EACCES on cold start: the nix-built image now chowns
-  `/app/config` to uid 1000 at build time via `fakeRootCommands`, matching the
-  behavior of the old Dockerfile. Without this, homepage couldn't seed missing
-  skeleton configs (proxmox.yaml etc.) or create `/app/config/logs`, crashing on
-  its first uncached request. Caught during the ringtail cutover.
-- Fixed sway keybindings on ringtail — the home-manager `keybindings` block was replacing the module's defaults entirely, leaving only explicit overrides (no workspace switching, focus, move, splits, resize mode, etc). Switched to `lib.mkOptionDefault` with `lib.mkForce` on the conflicting custom binds (`Mod+Return`, `Mod+d`, `Mod+space`, `Mod+l`) so defaults merge back in. Also added `Mod+F1` to show a filterable fuzzel list of current keybindings.
-
-  Fixed fuzzel config errors on launch — `border-radius` and `border-width` were under `[main]`, but fuzzel expects them as `radius`/`width` under a `[border]` section.
-- Pin the Quartz docs build to v4.5.2. The Dagger `build_docs` pipeline cloned Quartz from the default branch unpinned; Quartz v5.0.0 restructured its config layout (`.quartz/plugins`, `../quartz` imports) and broke the docs build against our existing `quartz.config.ts`/`quartz.layout.ts`.
-
-### Infrastructure
-
-- Wire the ringtail `blumeops-pg` cluster (which holds the wave-1-migrated
-  paperless + teslamate databases) into backups and Grafana. Adds a Tailscale
-  LoadBalancer Service (`blumeops-pg-ringtail.tail8d86e.ts.net`) and a Caddy L4
-  route (`pg.ops.eblu.me:5434`), then repoints borgmatic's `teslamate` +
-  `paperless` postgres dumps and the `mealie` SQLite dump at ringtail, and the
-  Grafana TeslaMate datasource at the ringtail DB. Closes the backup gap that
-  opened at cutover (the migrated live data was still being backed up from the
-  now-frozen minikube copies) and unblocks the wave-1 decommission.
-- Migrated homepage dashboard from minikube (indri/arm64) to k3s (ringtail/amd64).
-  The container is now built via nix (`containers/homepage/default.nix`), adapted
-  from nixpkgs `homepage-dashboard` with the upstream Next.js cache patches and
-  wrapped with `dockerTools.buildLayeredImage`. Autodiscovery shifts: services on
-  minikube (ArgoCD, Immich, Kiwix, Mealie, Miniflux, Grafana, Prometheus,
-  Navidrome, Paperless, TeslaMate, Transmission) become explicit static entries
-  in `services.yaml`; ringtail services (Authentik, Frigate/NVR, Ntfy, Ollama)
-  auto-populate via Ingress annotations.
-- Migrated CV (`cv.eblu.me`) and Docs (`docs.eblu.me`) from minikube Deployments to indri-native ansible roles. Caddy now serves the extracted release tarballs directly via a new `kind: static` service-block in the Caddy template — no daemon, no container — replacing the prior nginx-in-a-pod layer. Removes a network hop on every request and shrinks minikube's footprint. See [[cv-on-indri]] and [[docs-on-indri]]. Part of the broader minikube wind-down.
-- Migrated devpi (PyPI mirror at `pypi.ops.eblu.me`) from a minikube StatefulSet to a launchd-managed service on indri. devpi-server now runs in a uv-managed venv with pinned `devpi-server` and `devpi-web` versions, listens on `127.0.0.1:3141`, and is fronted by Caddy. The minikube StatefulSet was crash-looping under memory pressure (and breaking the Python toolchain everywhere); the new layout removes a layer of dependency on cluster health for critical-path tooling. See [[devpi-on-indri]].
-- Move the entire Immich stack — server, machine-learning, valkey,
-  and the PostgreSQL+VectorChord cluster — off `minikube-indri` and
-  onto `k3s-ringtail`. Postgres data migrated zero-loss via CNPG
-  `pg_basebackup` (replica catch-up then promote); row counts on
-  `asset`, `user`, `album`, `smart_search`, `activity`, `asset_face`
-  verified equal between source and replica before cutover. The ML
-  pod now uses ringtail's RTX 4080 via the nvidia-device-plugin
-  (time-slicing bumped 2 → 4 to share with frigate + ollama). Caddy
-  routing at `photos.ops.eblu.me` is unchanged (still
-  `photos.tail8d86e.ts.net`, the device just lives on ringtail now).
-  Borgmatic backups continue against the same `immich-pg` tailnet
-  hostname. First concrete chain in the broader indri-k8s
-  decommission effort.
-- Add local nix container build for `tailscale` (`containers/tailscale/default.nix`) so ringtail's tailscale-operator ProxyClass proxy pods pull from the forge mirror instead of `docker.io/tailscale/tailscale`. Pinned at v1.94.2 to match `service-versions.yaml`. Indri's tailscale-operator continues to use upstream during the k8s-to-ringtail migration.
-- Address the 6 critical Prowler IaC findings against `argocd/manifests/`. Prowler's IaC provider hardcodes `self._mutelist = None` and delegates filtering to Trivy, but doesn't plumb `--ignorefile` through — so the documented "use Trivy filtering" path is actually broken. Added a shim around `trivy` in the Prowler image that injects `--ignorefile $TRIVY_IGNOREFILE` for `trivy fs` invocations when the env var points at a real file. The IaC cronjob now mounts `mutelist/trivyignore.yaml` (Trivy's per-path schema) and sets the env var, muting the `external-secrets` and `kube-state-metrics` Secret-access findings (KSV-0041, KSV-0114). Separately, `grafana-clusterrole` is tightened to remove `secrets` access entirely: the dashboard sidecar already only consumes ConfigMap-labeled dashboards, so its `RESOURCE` env var is now `configmap` instead of `both`.
-- Pin ringtail's wired IP to `192.168.1.21` via NixOS scripted networking; NetworkManager no longer manages `enp5s0`. Removes DHCP lease renewal as a failure mode after a silent lease teardown took ringtail offline. Also explicitly enables `net.ipv4.ip_forward` (previously set implicitly by scripted-DHCP) so k3s pod networking and Tailscale routing continue to work with static networking.
-- Ripped out the compensating-controls (CC) framework: deleted `compensating-controls.yaml`, the `review-compensating-controls` mise task, and the associated how-to / explanation docs. Prowler and Kingfisher continue to run weekly and produce reports; the Prowler mutelist YAML files remain in place but no longer carry `CC: <id>` prefixes — each entry just keeps a free-form `Description` of why the finding is muted. The CC review cadence proved to be more overhead than this single-operator homelab needed.
-- Wire shower app for public exposure: fly nginx `shower.eblu.me` server
-  block as a guest-only surface — splash page, `/prizes/<token>/`, static
-  assets, media. Everything authenticated (`/admin/`, `/host/`,
-  `/accounts/`) returns 403 with a "tailnet only" pointer. Staff hit
-  `shower.ops.eblu.me` for the operator console + admin; the app's
-  v1.0.1 `DJANGO_PUBLIC_URL_BASE` setting makes QR codes generated on
-  the tailnet point back at the WAN host for guests. Plus a Caddy route
-  on indri, Pulumi Gandi CNAME, and a Grafana APM dashboard tracking
-  request rate, error rate, latency, bandwidth, and access logs.
-- Mirror Valkey 8.1 locally as `registry.ops.eblu.me/blumeops/valkey`. Replaces direct pulls of `docker.io/valkey/valkey:8.1-alpine` for paperless and immich sidecars. Built via native Dagger pipeline on Alpine 3.22. Stateless swap — no data migration. Authentik's nix-built Redis remains separate.
-- Add nix-built amd64 valkey for ringtail (`containers/valkey/default.nix`) so immich-ringtail can stop pulling the upstream multi-arch `docker.io/valkey/valkey` image. Existing `container.py` continues to build Alpine arm64 for paperless on indri. Both bump to valkey 8.1.7 (Alpine 3.22 8.1.7-r0 / nixpkgs 8.1.7).
-- Upgrade Grafana Alloy v1.14.0 → v1.16.0 across all four service deployments
-  (alloy-k8s, alloy-ringtail, alloy-tracing-ringtail on k8s; alloy native on
-  indri). Pulls in stable database observability (v1.15) and the OTel Collector
-  v0.147.0 bump. Container build also migrated from Dockerfile to native Dagger
-  `container.py` per the build-container-image migration playbook.
-- Upgraded Dagger from v0.20.1 to v0.20.6 (engine, CLI pin, and SDK regen) and migrated `runner-job-image` from a Debian-based Dockerfile to a native Dagger `container.py` on Alpine 3.23, reusing the shared `alpine_runtime` helper.
-- Decommission the wave-1 services on minikube-indri now that paperless,
-  teslamate, and mealie run on ringtail with their data backed up. Removes the
-  minikube `paperless`/`teslamate`/`mealie` manifest dirs + ArgoCD app
-  definitions (pruning the parked Deployments, Services, and the redundant
-  minikube mealie/paperless PVCs), and drops the `paperless`/`teslamate` roles
-  from the minikube `blumeops-pg` cluster. The `paperless` and `teslamate`
-  databases are dropped from indri's blumeops-pg as the finalization step.
-  miniflux + authentik remain on the minikube cluster (later waves).
-- Upgraded the k8s Forgejo runner to the v12.8 line, switched it from first-boot registration to declarative `server.connections` credentials from 1Password, and consolidated the supporting runner how-to documentation.
-- Move paperless, teslamate, and mealie off `minikube-indri` onto
-  `k3s-ringtail`, shedding ~1.1 GiB of resident load from the
-  OOM-thrashing 8 GiB minikube node (the kernel OOM killer had been
-  killing `kube-apiserver`/`dockerd`/argocd, flapping every
-  minikube-hosted service at once). paperless + teslamate databases
-  move into a fresh CNPG `blumeops-pg` cluster on ringtail via a cold
-  `pg_dump`/`pg_restore` from the quiesced source — row counts verified
-  equal before any routing flip; source DBs dropped only after the
-  ringtail side serves traffic. mealie's SQLite PVC is copied as-is.
-  paperless media stays on sifaka NFS. Downtime-tolerant cold cutover
-  (no streaming replication); rollback is repoint-and-scale-up with the
-  source untouched. Second chain in the indri-k8s decommission after
-  [[migrate-immich-to-ringtail]].
-- Recurring maintenance batch:
-
-  - Ringtail flake inputs refreshed (`disko`, `home-manager`, `nixpkgs`).
-  - Tooling deps bumped: prek hooks (trufflehog v3.95.3, kingfisher v1.101.0, ruff v0.15.14, `ansible-core` 2.21.0); fly proxy base images (nginx 1.30.1-alpine, alloy v1.16.1); `typer==0.26.2` in mise tasks.
-- Updated `nixos/ringtail/flake.lock` (weekly cadence): `disko`, `home-manager`, and `nixpkgs` inputs refreshed. `nixpkgs-services` skipped per overlay convention.
-- Reviewed `mealie` service version freshness; upstream is 5 minor versions ahead (v3.17.0 vs deployed v3.12.0). Marked reviewed; upgrade deferred.
-- Deploy shower v1.1.2 — bump container build to new app release.
-- Upgrade unpoller v2.34.0 → v3.2.0 and migrate container build from Dockerfile to native Dagger (container.py). v3.0.0 carries breaking UniFi API changes; v3.2.0 introduces a 60s background poll (cached scrapes) by default — set `interval = 0` in `up.conf` to restore on-demand polling.
-- Monthly tooling dependency refresh: prek hooks (trufflehog, kingfisher, ruff, shfmt, prettier, actionlint, ansible-lint), fly proxy base images (nginx 1.30.0, tailscale v1.94.2, alloy v1.16.0), normalize pyyaml lower bound in mise-tasks.
-- Add GE-Proton (`pkgs.proton-ge-bin`) to `programs.steam.extraCompatPackages`
-  on ringtail. Subnautica 2 hangs at Mercuna plugin init under Proton
-  Experimental + DXVK D3D12; GE-Proton is available as a Steam per-game
-  compatibility option to work around it.
-- Add `sn2-prelaunch` Steam launch wrapper on ringtail that removes
-  Subnautica 2's stale `Saved/running.dat` and `Saved/beforelobby.dat`
-  lockfiles before each launch. SN2 pops up an invisible (0×0-sized)
-  Error dialog when it detects an unclean exit, blocking GameThread
-  forever; this is observable only as a black screen with a spinning
-  loader. Use via Steam launch option: `sn2-prelaunch %command%`.
-- Add local nix container build for `frigate-notify` (`containers/frigate-notify/default.nix`) so the Frigate→ntfy bridge is rebuilt on ringtail from the forge mirror instead of pulled from `ghcr.io/0x2142/frigate-notify`.
-- Add resource limits to all ArgoCD pods to prevent unbounded resource consumption during node-wide pressure events.
-- Black-hole the `/mirrors/*` repositories at the Fly proxy edge (`return 403` → `forge.ops.eblu.me`). A surprise $29.60 Fly bill traced to ~1.24 TB/30d of egress on `forge.eblu.me`, 99.95% of all proxy egress — of which ~71% was AI scrapers (Meta `meta-externalagent`, OpenAI `GPTBot`, Amazonbot) crawling the near-infinite git-history URL space of the public mirror repos and timing out Forgejo in the process. Mirrors exist for supply-chain control and are consumed over the tailnet, so their public web UI had no legitimate audience. `robots.txt` already disallowed `/mirrors/`, but the offending agents ignore it. Tier-2 mitigations (user-agent denylist, Anubis proof-of-work gateway) are documented in `docs/explanation/ai-scraper-mitigation.md`.
-- Bump paperless and immich kustomizations to the main-SHA-built valkey tag (`v8.1.6-r0-fabca04`). Routine post-merge follow-up to keep production manifests pointing at images built from a commit on main.
-- Bump shower container to v1.1.1 (probe FOD hash).
-- Bumped shower app to v1.1.3 (wheel/sdist + FOD hashes probed on ringtail).
-- Cap systemd-coredump on ringtail (ProcessSizeMax/ExternalSizeMax 1G, MaxUse 2G) so multi-GB Wine/Proton game crash dumps no longer thrash the disk and lock up the desktop.
-- Deploy shower v1.1.1 to ringtail (kustomize newTag bump).
-- Deployed shower v1.1.3 to ringtail (image built and pushed from ringtail; runner bypassed due to indri overload).
-- Fix three follow-ups from the wave-1 decommission: grant the local
-  break-glass `admin` account ArgoCD admin rights (`g, admin, role:admin` —
-  previously only the Authentik `admins` group had access, so admin was
-  locked out whenever its token expired), and repoint the alloy blackbox
-  probe for teslamate from the deleted minikube service to
-  `https://tesla.ops.eblu.me/` (through Caddy over Tailscale). The orphaned
-  paperless/teslamate roles + ExternalSecrets left on the minikube
-  blumeops-pg are also cleaned up.
-- Moved the Immich blackbox health probe from indri's alloy to ringtail's alloy. After the immich migration to ringtail, the probe still targeted `immich-server.immich.svc.cluster.local` on indri's cluster where the service no longer exists, causing a persistent `ServiceProbeFailure` alert.
-- Pin shower v1.1.1 FOD outputHash (probed locally on ringtail).
-- Rebuild Prowler container against main HEAD (v5.23.0-495e45d) after merging the IaC mutelist Dockerfile changes.
-- Rebuild and retag alloy v1.16.0 container images from the main-branch SHA
-  following the squash-merge of #345, per the build-container-image
-  squash-merge convention. Both images (`registry.ops.eblu.me/blumeops/alloy`)
-  now reference `9564435` rather than the branch SHA `26a3ab5`, restoring
-  source traceability after branch cleanup.
-- Rebuild shower from the post-merge commit on main so the container's
-  SHA tag points at a commit that will still exist after the 30-day
-  branch-cleanup window. Functionally identical to the branch-tag image
-  already deployed, just preserves source traceability per
-  [[build-container-image#Squash-merge and container tags]].
-- Rebuild unpoller container from squashed main commit so the image SHA tag matches a commit in main's history (was tagged with the pre-squash branch SHA).
-- Rebuild valkey container from squashed main commit (both arm64 dagger and amd64 nix variants), and update paperless + immich-ringtail kustomizations to the main-SHA tags `v8.1.7-ecded30` and `v8.1.7-ecded30-nix`.
-- Retired the `blumeops-tasks` mise task (Todoist API) in favor of `heph list --project Blumeops --json` from the self-hosted [hephaestus](https://github.com/eblume/hephaestus) system. Updated docs to point task discovery and rotation reminders at heph, and noted that the `~/code/personal/zk` zettelkasten is migrating into heph docs.
-- Switch the Fly proxy deploy strategy from `bluegreen` to `immediate` in `fly/fly.toml`. With a single proxy machine, bluegreen offers little benefit — the green machine routinely failed to reach "started" inside Fly's default 5-minute deploy timeout (the cold-start sequence of `tailscaled` → `tailscale up` → wait-for-MagicDNS → nginx startup eats most of the budget), and the failed deploys would roll back. `immediate` replaces the machine in place with a brief downtime (~5–10s) but actually completes.
-- Switch the ringtail provisioning playbook's blumeops clone URL from `forge.eblu.me` (public, via Fly proxy) to `forge.ops.eblu.me` (tailnet, direct via Caddy on indri). Ringtail is always on the tailnet, so the WAN round-trip is pure overhead — it also made `provision-ringtail` brittle whenever the Fly proxy was slow or down.
-- Switched Grafana's deployment strategy from `RollingUpdate` to `Recreate`. With an RWO PVC holding the SQLite database and Bleve search index, `RollingUpdate` reliably crashloops the new pod on the index lock until rollout timeout. `Recreate` terminates the old pod first so the new one acquires the lock cleanly.
-- Update `tailscale-operator-ringtail` ProxyClass to reference the `0108b68` main-SHA build of the tailscale container. Routine post-merge cleanup so the deployed image traces to a commit that survives PR branch cleanup.
-- Update the ringtail NixOS flake lockfile (`nixos/ringtail/flake.lock`): bump
-  `nixpkgs` (b77b3de → 25f5383) and `disko` (5ba0c95 → 115e521) to latest.
-  `nixpkgs-services` was intentionally left pinned (skipped by the
-  `flake-update` pipeline). Routine recurring maintenance per [[manage-lockfile]].
-- Upgrade native macOS Alloy on indri to v1.16.0. Built on gilbert with Go
-  1.26.2 + CGO (required for the macOS native DNS resolver, which Tailscale
-  MagicDNS depends on), scp'd to `~/.local/bin/alloy` on indri, codesigned,
-  and the LaunchAgent reloaded. Completes the v1.16.0 fleet upgrade started
-  in #345 — all four Alloy services (alloy-k8s, alloy-ringtail,
-  alloy-tracing-ringtail, alloy ansible) now run v1.16.0.
-- Upgraded zot on indri from v2.1.15 to v2.1.16 (security fixes: TLS verification on metrics client, CORS Allow-Credentials suppression on wildcard origins, manifest/API-key body size limits).
-
-### Documentation
-
-- Reviewed `replicating-blumeops` tutorial: fixed "BluemeOps" typos (also in `contributing.md`) and added `last-reviewed` frontmatter.
-- Reviewed [[indri]] reference card: added `devpi`, `cv`, and `docs` to the native-services list; widened the k8s note to reflect the growing set of apps now on ringtail and the planned indri-minikube decommission; added CPU/RAM specs.
-- New how-to: rotate-fly-deploy-token. Documents the 75-day rotation cadence, why we use `org`-scoped tokens (silences the cosmetic metrics-token warning on `fly status` with marginal blast-radius cost given the single-app personal org), and the procedure for rotation + Forgejo Actions secret sync.
-- Add `docs/explanation/ai-scraper-mitigation.md` — the egress-cost / AI-crawler threat model for the public Fly proxy, the tiered mitigation plan (Tier 1: mirror black-hole, shipped; Tier 2: user-agent denylist + Anubis; Tier 3: Cloudflare, rejected on principle), and the data behind it.
-- Fix manage-forgejo-mirrors verify step — sync button is on the repo settings page ("Synchronize now"), not the main repo page.
-- Fixed the `op item edit` invocation in the [[zot]] API-key rotation procedure: the previous `pbpaste | op item edit ... "field[password]=-"` stdin syntax is rejected by op 2.34 as "invalid JSON" (recent op versions treat piped input as a full JSON template, not a single field value). Procedure now reads the clipboard into a local fish variable and passes it as an inline assignment.
-- Fixed the export-filename step in [[run-1password-backup]]: 1Password's desktop app names the export `1PasswordExport-<account-uuid>-<timestamp>.1pux` automatically rather than letting you save to a fixed name, so the procedure now points the task at that glob instead of pretending the default name is `1Password-export.1pux`.
-- Refresh the contributing tutorial: add `last-reviewed`, include the `.ai.md` changelog fragment type, and clarify that `prek` is pinned via `mise`.
-- Review and refresh the Navidrome reference card: add `last-reviewed`, correct the scanner env var name, document the current image/version, and record routing and runtime details from the manifests.
-- Review and refresh the Ollama reference card: add `last-reviewed`, bump the documented image tag to 0.20.4, and add the two `qwen3.5` models now declared in `models.txt`.
-- Reviewed [[1password]] reference card: added the `blumeops` vs `Personal` vault split, noted that `onepassword-connect` runs on both indri and ringtail (not just one cluster), and pulled the `op read` vs `op item get --fields` guidance up from agent memory into the card.
-- Reviewed `index.md`; added ringtail to the infrastructure overview and stamped `last-reviewed`.
-- Reviewed transmission card: corrected storage layout (`/config/` is emptyDir, watch dir disabled) and noted the Prometheus exporter sidecar.
-- rotate-fly-deploy-token: combine mint+store into one command with both fish and bash forms; document the `op item edit` "Password item requires ps value" validator gotcha and the placeholder-password workaround.
-
-### AI Assistance
-
-- Adopt `AGENTS.md` as the canonical agent instruction file, keep `CLAUDE.md` as a compatibility shim, and update docs to reference the neutral file and the correct agent-change-process path.
-- CLAUDE.md now imports AGENTS.md via `@AGENTS.md` instead of telling agents to go read it. Claude Code only auto-loads CLAUDE.md, so the prose shim was easy to skip; the import inlines AGENTS.md into the session prompt unconditionally.
-
-### Miscellaneous
-
-- Removed the dead minikube manifests, container builds, and tooling shims left behind after the cv + docs migration to indri-native (#342). Deletes `argocd/{apps,manifests}/{cv,docs}/`, `containers/{cv,quartz}/`, and the `quartz`→`docs` mapping in `mise-tasks/container-version-check`. Bumps `docs.current-version` to `v1.16.0` (the blumeops release tag) now that the legacy nginx-base version pin is gone.
-- Rebuild shower v1.1.0 container from main HEAD (`3c7967e`) and bump the
-  kustomization tag to `v1.1.0-3c7967e-nix`. The PR was squash-merged, so
-  the branch commit `444ff91` baked into the prior tag isn't reachable
-  from main's history. The new tag points at a commit that exists on
-  main; image content is byte-identical because the FOD output is content
-  addressed and the inputs didn't change.
-- Rebuild shower v1.1.2 from main HEAD (a33fa47) and retag — PR #358 was squash-merged so the branch SHA baked into the prior image tag isn't reachable from main. FOD is content-addressed, so image bytes are identical; only provenance changes.
-- Remove the duplicate Homepage tiles for Mealie, Paperless, Immich, and
-  TeslaMate. Homepage runs on ringtail and autodiscovers ringtail Ingresses via
-  `gethomepage.dev/*` annotations; once these services migrated to ringtail they
-  were discovered automatically, making their leftover static `services.yaml`
-  entries (needed only while they lived on minikube) redundant.
-- Removed the now-unused `containers/devpi/` Dagger build artifact. Devpi runs natively on indri via uv venv; the container image is no longer referenced anywhere. Doc examples in `docs/reference/tools/dagger.md` updated to use `miniflux` as the example container name.
-- `container-build-and-release` now prints the specific `mise run runner-logs <N>` command after dispatching, polling the Forgejo API to resolve the run number for the commit it just triggered.
-- `mise run runner-logs <run> -j <n>` now reports a clear error when the log file doesn't exist on indri (e.g. a runner crash that left `action_task.log_in_storage = 0`). Previously it printed only the header and exited 0, because `zstdcat` exits 0 with a "can't stat … -- ignored" stderr message and ssh+fish on indri swallows the remote exit code.
-
-
 ## [v1.16.0] - 2026-04-18
 
 ### Infrastructure

ansible/playbooks/indri.yml

@@ -260,7 +260,5 @@
       tags: cv
     - role: docs
       tags: docs
-    - role: heph
-      tags: heph
     - role: caddy
       tags: caddy

ansible/playbooks/ringtail.yml

@@ -57,7 +57,7 @@
   tasks:
     - name: Ensure blumeops repo is present
       ansible.builtin.git:
-        repo: "https://forge.ops.eblu.me/eblume/blumeops.git"
+        repo: "https://forge.eblu.me/eblume/blumeops.git"
         dest: /etc/blumeops
         version: "{{ ringtail_commit | default('main') }}"
         force: true

ansible/roles/borgmatic/defaults/main.yml

@@ -56,17 +56,12 @@ borgmatic_k8s_sqlite_dumps:
     namespace: mealie
     label_selector: app=mealie
     db_path: /app/data/mealie.db
-    # migrated to ringtail (wave-1); ssh to ringtail and run k3s kubectl
-    # there, same as shower below.
-    target: ssh:eblume@ringtail
+    context: minikube
   - name: shower
     namespace: shower
     label_selector: app=shower
     db_path: /app/data/db.sqlite3
-    # ssh to ringtail and run k3s kubectl there — avoids needing a
-    # ringtail kubeconfig on indri. k3s.yaml on ringtail is
-    # world-readable (mode 644), so no sudo required.
-    target: ssh:eblume@ringtail
+    context: k3s-ringtail
 
 # Exclude patterns
 borgmatic_exclude_patterns: []
@@ -103,18 +98,17 @@ borgmatic_postgresql_databases:
     hostname: pg.ops.eblu.me
     port: 5432
     username: borgmatic
+  - name: teslamate
+    hostname: pg.ops.eblu.me
+    port: 5432
+    username: borgmatic
   - name: authentik
     hostname: pg.ops.eblu.me
     port: 5432
     username: borgmatic
-  # migrated to ringtail blumeops-pg (wave-1); port 5434 = Caddy L4 route
-  - name: teslamate
-    hostname: pg.ops.eblu.me
-    port: 5434
-    username: borgmatic
   - name: paperless
     hostname: pg.ops.eblu.me
-    port: 5434
+    port: 5432
     username: borgmatic
   # immich-pg cluster (VectorChord) via Caddy L4 on port 5433
   - name: immich

ansible/roles/borgmatic/tasks/main.yml

@@ -19,10 +19,8 @@
   ansible.builtin.copy:
     content: |
       # Managed by ansible (borgmatic role) - k8s PostgreSQL backup credentials
-      # 5432 = minikube blumeops-pg, 5433 = immich-pg, 5434 = ringtail blumeops-pg
       pg.ops.eblu.me:5432:*:borgmatic:{{ borgmatic_db_password }}
       pg.ops.eblu.me:5433:*:borgmatic:{{ borgmatic_db_password }}
-      pg.ops.eblu.me:5434:*:borgmatic:{{ borgmatic_db_password }}
     dest: ~/.pgpass
     mode: '0600'
   no_log: true
@@ -51,20 +49,6 @@
     mode: '0700'
   when: borgmatic_k8s_sqlite_dumps | length > 0
 
-- name: Ensure ~/bin exists
-  ansible.builtin.file:
-    path: "{{ ansible_env.HOME }}/bin"
-    state: directory
-    mode: '0755'
-  when: borgmatic_k8s_sqlite_dumps | length > 0
-
-- name: Deploy k8s SQLite dump helper script
-  ansible.builtin.template:
-    src: k8s-sqlite-dump.sh.j2
-    dest: "{{ ansible_env.HOME }}/bin/borgmatic-k8s-sqlite-dump"
-    mode: '0755'
-  when: borgmatic_k8s_sqlite_dumps | length > 0
-
 - name: Deploy borgmatic configuration
   ansible.builtin.template:
     src: config.yaml.j2

ansible/roles/borgmatic/templates/config.yaml.j2

@@ -32,20 +32,12 @@ exclude_patterns:
 encryption_passcommand: {{ borgmatic_encryption_passcommand }}
 
 {% if borgmatic_k8s_sqlite_dumps %}
-# Pre-backup: dump SQLite databases from k8s pods.
-# Uses sqlite3.backup() for a safe, consistent copy.
-#
-# Quoting/escaping is delegated to ~/bin/borgmatic-k8s-sqlite-dump
-# (deployed by the borgmatic ansible role). Each entry's `target`
-# is either:
-#   - local:<context>  -> local kubectl with --context (mealie etc.)
-#   - ssh:<user@host>  -> ssh + k3s kubectl on the cluster host,
-#                         used for ringtail since indri's kubeconfig
-#                         deliberately doesn't carry that context.
+# Pre-backup: dump SQLite databases from k8s pods
+# Uses sqlite3 .backup for a safe, consistent copy (no corruption from concurrent writes)
 before_backup:
     - mkdir -p {{ borgmatic_k8s_dump_dir }}
 {% for db in borgmatic_k8s_sqlite_dumps %}
-    - {{ ansible_env.HOME }}/bin/borgmatic-k8s-sqlite-dump {{ db.target }} {{ db.namespace }} {{ db.label_selector }} {{ db.db_path }} {{ db.name }} {{ borgmatic_k8s_dump_dir }}/{{ db.name }}.db
+    - /opt/homebrew/bin/kubectl --context={{ db.context }} exec -n {{ db.namespace }} deploy/{{ db.name }} -- python3 -c "import sqlite3; sqlite3.connect('{{ db.db_path }}').backup(sqlite3.connect('/tmp/{{ db.name }}-backup.db'))" && /opt/homebrew/bin/kubectl --context={{ db.context }} cp {{ db.namespace }}/$(/opt/homebrew/bin/kubectl --context={{ db.context }} get pod -n {{ db.namespace }} -l {{ db.label_selector }} -o jsonpath='{.items[0].metadata.name}'):/tmp/{{ db.name }}-backup.db {{ borgmatic_k8s_dump_dir }}/{{ db.name }}.db
 {% endfor %}
 {% endif %}
 

ansible/roles/borgmatic/templates/k8s-sqlite-dump.sh.j2

@@ -1,73 +0,0 @@
-#!/usr/bin/env bash
-# {{ ansible_managed }}
-#
-# Helper script invoked by borgmatic's before_backup hook to capture a
-# k8s pod's SQLite database. Keeps the borgmatic config readable by
-# pulling all the quoting out of YAML.
-#
-# Usage:
-#   borgmatic-k8s-sqlite-dump <target> <namespace> <selector> \
-#                             <db_path> <name> <dump_target>
-#
-# <target> is one of:
-#   local:<context>   - run local kubectl with --context=<context>
-#   ssh:<user@host>   - ssh to host and run k3s kubectl there
-#                       (no indri-side kubeconfig needed)
-#
-# <namespace>      - k8s namespace of the pod
-# <selector>       - label selector to find the pod (e.g. app=shower)
-# <db_path>        - absolute path inside the pod to the SQLite DB
-# <name>           - short name used for temp filenames
-# <dump_target>    - file on this host to receive the dump
-set -euo pipefail
-
-target=${1:?missing target}
-namespace=${2:?missing namespace}
-selector=${3:?missing selector}
-db_path=${4:?missing db path}
-name=${5:?missing name}
-dump_target=${6:?missing dump target}
-
-# Stage the backup next to the source DB (a guaranteed-writable volume);
-# minimal nix images (e.g. mealie) have no /tmp.
-pod_tmp="$(dirname "$db_path")/.borgmatic-backup-${name}.db"
-
-python_backup='import sqlite3; sqlite3.connect("'"$db_path"'").backup(sqlite3.connect("'"$pod_tmp"'"))'
-
-mode=${target%%:*}
-ref=${target#*:}
-
-case "$mode" in
-    local)
-        # Pulls dump bytes out via "kubectl exec -- cat" rather than
-        # "kubectl cp", which would otherwise need tar inside the pod
-        # (nix-built images like shower don't bundle tar).
-        context=$ref
-        kubectl="/opt/homebrew/bin/kubectl --context=$context -n $namespace"
-        pod=$($kubectl get pod -l "$selector" \
-            -o jsonpath='{.items[0].metadata.name}')
-        $kubectl exec "$pod" -- python3 -c "$python_backup"
-        $kubectl exec "$pod" -- cat "$pod_tmp" > "$dump_target"
-        $kubectl exec "$pod" -- rm -f "$pod_tmp"
-        ;;
-    ssh)
-        host=$ref
-        # Force bash on the remote (user's login shell on ringtail is
-        # fish). Pipe the script via stdin to dodge nested quoting.
-        # The dump bytes come back over the ssh stdout stream — no
-        # intermediate scp, no tar requirement in the pod.
-        ssh "$host" bash <<EOF > "$dump_target"
-set -euo pipefail
-export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
-pod=\$(k3s kubectl -n "$namespace" get pod -l "$selector" -o jsonpath='{.items[0].metadata.name}')
-k3s kubectl -n "$namespace" exec "\$pod" -- python3 -c '$python_backup' 1>&2
-k3s kubectl -n "$namespace" exec "\$pod" -- cat "$pod_tmp"
-k3s kubectl -n "$namespace" exec "\$pod" -- rm -f "$pod_tmp" 1>&2
-EOF
-        ;;
-    *)
-        echo "borgmatic-k8s-sqlite-dump: unknown target mode: $mode" >&2
-        echo "  expected local:<context> or ssh:<user@host>" >&2
-        exit 1
-        ;;
-esac

ansible/roles/caddy/defaults/main.yml

@@ -52,9 +52,6 @@ caddy_services:
   - name: devpi
     host: "pypi.{{ caddy_domain }}"
     backend: "http://localhost:3141"
-  - name: heph
-    host: "heph.{{ caddy_domain }}"
-    backend: "http://localhost:8787"  # hephaestus hub (server mode) + PWA shell
   - name: kiwix
     host: "kiwix.{{ caddy_domain }}"
     backend: "https://kiwix.tail8d86e.ts.net"
@@ -120,8 +117,6 @@ caddy_tcp_services:
     backend: "pg.tail8d86e.ts.net:5432"  # PostgreSQL (blumeops-pg)
   - port: 5433
     backend: "immich-pg.tail8d86e.ts.net:5432"  # PostgreSQL (immich-pg)
-  - port: 5434
-    backend: "blumeops-pg-ringtail.tail8d86e.ts.net:5432"  # PostgreSQL (blumeops-pg on ringtail)
   - port: "{{ sifaka_node_exporter_port }}"
     backend: "sifaka:{{ sifaka_node_exporter_port }}"  # Sifaka node_exporter
   - port: "{{ sifaka_smartctl_exporter_port }}"

ansible/roles/docs/defaults/main.yml

@@ -3,8 +3,9 @@
 # Caddy serves docs_content_dir directly via the static-kind service block,
 # with Quartz-style try_files (path → path/ → path.html → 404).
 
-docs_version: "v1.17.0"
+docs_version: "v1.16.0"
 docs_release_url: "https://forge.eblu.me/eblume/blumeops/releases/download/{{ docs_version }}/docs-{{ docs_version }}.tar.gz"
+
 docs_home: /Users/erichblume/blumeops/docs
 docs_content_dir: "{{ docs_home }}/content"
 docs_version_sentinel: "{{ docs_home }}/.installed-version"

ansible/roles/heph/defaults/main.yml

@@ -1,49 +0,0 @@
----
-# hephaestus hub — the canonical heph replica (server mode) on indri.
-# Other devices (e.g. gilbert) are spokes that sync against this hub.
-# See [[set-up-sync-hub]] and [[host-heph-pwa]] in the hephaestus repo.
-
-# Pinned release used for the initial `cargo install` and the PWA shell.
-# After bootstrap, hephd's own --self-update keeps the binary current; this
-# pin only governs the first install and the bundled PWA shell version.
-heph_version: v1.2.1
-
-# Anonymous public HTTPS clone — matches hephd's INSTALL_GIT_URL so the initial
-# install and unattended self-update build from the same source (no ssh-agent).
-heph_repo_url: https://forge.eblu.me/eblume/hephaestus.git
-
-heph_bin_dir: /Users/erichblume/.cargo/bin
-heph_binary: "{{ heph_bin_dir }}/hephd"
-
-# rustc/cargo here are rustup shims. The bare (non-mise) environment that the
-# launchagent and ansible run in falls back to rustup's *default* toolchain,
-# which can lag behind heph's rust-version floor (Cargo.toml: 1.89). Pin the
-# channel explicitly so both the bootstrap build and unattended self-update
-# always use a current toolchain regardless of the host's rustup default.
-heph_rust_toolchain: stable
-
-heph_data_dir: /Users/erichblume/.local/share/heph
-heph_db: "{{ heph_data_dir }}/heph.db"
-heph_socket: "{{ heph_data_dir }}/hephd.sock"
-heph_log_dir: /Users/erichblume/Library/Logs
-
-# Version-pinned source checkout; the PWA static shell is served directly from
-# its heph-pwa/ subdir (no copy), keeping shell and hub in lockstep at heph_version.
-heph_pwa_src_dir: /Users/erichblume/.cache/heph-pwa-src
-heph_web_root: "{{ heph_pwa_src_dir }}/heph-pwa"
-
-# Hub listens on all interfaces so tailnet spokes can reach it directly
-# (http://indri.tail8d86e.ts.net:8787) and Caddy can proxy heph.ops.eblu.me.
-# Access is gated by Authentik OIDC regardless — tailnet reachability is not
-# enough (this is the owner's most sensitive data).
-heph_http_addr: 0.0.0.0:8787
-heph_port: 8787
-heph_external_url: https://heph.ops.eblu.me
-
-# Authentik OIDC — issuer + audience together turn hub auth on. The audience is
-# the device-code client id (see argocd/manifests/authentik heph blueprint).
-heph_oidc_issuer: https://authentik.ops.eblu.me/application/o/heph/
-heph_oidc_audience: heph
-
-# Self-update poll interval (seconds). 10 minutes.
-heph_self_update_interval_secs: 600

ansible/roles/heph/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart heph
-  ansible.builtin.shell: |
-    launchctl unload ~/Library/LaunchAgents/mcquack.eblume.heph.plist 2>/dev/null || true
-    launchctl load ~/Library/LaunchAgents/mcquack.eblume.heph.plist
-  changed_when: true

ansible/roles/heph/tasks/main.yml

@@ -1,82 +0,0 @@
----
-# hephaestus hub (server mode) on indri.
-#
-# DATA SEEDING (one-time, Path A — do this BEFORE the first provision so the hub
-# adopts gilbert's existing data instead of being born empty):
-#
-#   1. On the seed device (gilbert):   heph daemon stop
-#   2. Copy its store to indri:         scp ~/.local/share/heph/heph.db \
-#                                           indri:~/.local/share/heph/heph.db
-#   3. On indri, give the hub its OWN device origin (keeps gilbert's owner_id +
-#      data; hephd regenerates a fresh origin on next start when it is missing):
-#        sqlite3 ~/.local/share/heph/heph.db "DELETE FROM meta WHERE key='origin';"
-#   4. Run this role (installs hephd, stages the PWA, loads the launchagent).
-#
-# hephd auto-creates an empty store on first start if none exists, so seeding is
-# optional — skip it only if you intend a fresh, empty hub.
-
-- name: Ensure heph data directory exists
-  ansible.builtin.file:
-    path: "{{ heph_data_dir }}"
-    state: directory
-    mode: '0700'
-
-- name: Check for installed hephd binary
-  ansible.builtin.stat:
-    path: "{{ heph_binary }}"
-  register: heph_binary_stat
-
-# Bootstrap install only when hephd is absent. Thereafter hephd's own
-# --self-update keeps it current; ansible must not fight (or downgrade) it.
-# This builds from source and can take several minutes on a cold cargo cache.
-- name: Bootstrap-install heph + hephd from the forge ({{ heph_version }})
-  ansible.builtin.command:
-    cmd: >-
-      {{ heph_bin_dir }}/cargo install --locked
-      --git {{ heph_repo_url }}
-      --tag {{ heph_version }}
-      heph hephd
-  environment:
-    PATH: "{{ heph_bin_dir }}:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin"
-    RUSTUP_TOOLCHAIN: "{{ heph_rust_toolchain }}"
-  when: not heph_binary_stat.stat.exists
-  changed_when: true
-  notify: Restart heph
-
-# Checkout provides the PWA shell at {{ heph_web_root }} (heph-pwa/ subdir),
-# served directly by hephd. Static files are read from disk per request, so a
-# version bump needs no restart; the service worker (CACHE = "heph-pwa-vN")
-# evicts stale assets on next load.
-- name: Ensure heph cache parent directory exists
-  ansible.builtin.file:
-    path: "{{ heph_pwa_src_dir | dirname }}"
-    state: directory
-    mode: '0755'
-
-- name: Stage heph-pwa source at {{ heph_version }}
-  ansible.builtin.git:
-    repo: "{{ heph_repo_url }}"
-    dest: "{{ heph_pwa_src_dir }}"
-    version: "{{ heph_version }}"
-    depth: 1
-    single_branch: true
-    force: true
-
-- name: Deploy heph LaunchAgent plist
-  ansible.builtin.template:
-    src: heph.plist.j2
-    dest: ~/Library/LaunchAgents/mcquack.eblume.heph.plist
-    mode: '0644'
-  notify: Restart heph
-
-- name: Check if heph LaunchAgent is loaded
-  ansible.builtin.command: launchctl list mcquack.eblume.heph
-  register: heph_launchctl_check
-  changed_when: false
-  failed_when: false
-
-- name: Load heph LaunchAgent if not loaded
-  ansible.builtin.command: launchctl load ~/Library/LaunchAgents/mcquack.eblume.heph.plist
-  when: heph_launchctl_check.rc != 0
-  changed_when: true
-  failed_when: false

ansible/roles/heph/templates/heph.plist.j2

@@ -1,50 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!-- {{ ansible_managed }} -->
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
-	<key>Label</key>
-	<string>mcquack.eblume.heph</string>
-	<key>ProgramArguments</key>
-	<array>
-		<string>{{ heph_binary }}</string>
-		<string>--mode</string>
-		<string>server</string>
-		<string>--http-addr</string>
-		<string>{{ heph_http_addr }}</string>
-		<string>--db</string>
-		<string>{{ heph_db }}</string>
-		<string>--socket</string>
-		<string>{{ heph_socket }}</string>
-		<string>--web-root</string>
-		<string>{{ heph_web_root }}</string>
-		<string>--oidc-issuer</string>
-		<string>{{ heph_oidc_issuer }}</string>
-		<string>--oidc-audience</string>
-		<string>{{ heph_oidc_audience }}</string>
-		<string>--self-update</string>
-		<string>--self-update-interval-secs</string>
-		<string>{{ heph_self_update_interval_secs }}</string>
-	</array>
-	<key>RunAtLoad</key>
-	<true/>
-	<key>KeepAlive</key>
-	<true/>
-	<key>EnvironmentVariables</key>
-	<dict>
-		<!-- cargo + toolchain on PATH so --self-update can run `cargo install`. -->
-		<key>PATH</key>
-		<string>{{ heph_bin_dir }}:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
-		<key>HOME</key>
-		<string>/Users/erichblume</string>
-		<!-- Pin the rustup channel: the launchagent runs without mise, so a bare
-		     cargo shim would otherwise use rustup's (stale) default toolchain. -->
-		<key>RUSTUP_TOOLCHAIN</key>
-		<string>{{ heph_rust_toolchain }}</string>
-	</dict>
-	<key>StandardOutPath</key>
-	<string>{{ heph_log_dir }}/mcquack.heph.out.log</string>
-	<key>StandardErrorPath</key>
-	<string>{{ heph_log_dir }}/mcquack.heph.err.log</string>
-</dict>
-</plist>

argocd/apps/cloudnative-pg-ringtail.yaml

@@ -1,27 +0,0 @@
-# CloudNativePG Operator for ringtail k3s cluster
-# Deploys the operator only; PostgreSQL clusters are created separately
-#
-# Sibling of cloudnative-pg.yaml (minikube). Same mirror, same release,
-# different destination. Both apps will coexist during the immich
-# migration; the minikube one is removed at the end of the broader
-# indri-k8s decommission.
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: cloudnative-pg-ringtail
-  namespace: argocd
-spec:
-  project: default
-  source:
-    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/mirrors/cloudnative-pg.git
-    targetRevision: v1.27.1
-    path: releases
-    directory:
-      include: 'cnpg-1.27.1.yaml'
-  destination:
-    server: https://ringtail.tail8d86e.ts.net:6443
-    namespace: cnpg-system
-  syncPolicy:
-    syncOptions:
-      - CreateNamespace=true
-      - ServerSideApply=true  # Required for large CRDs that exceed annotation size limit

argocd/apps/databases-ringtail.yaml

@@ -1,26 +0,0 @@
-# Databases on ringtail k3s.
-#
-# Today: only immich-pg (CNPG Cluster) + its borgmatic ExternalSecret.
-# More databases may move here as the indri-k8s decommission proceeds.
-#
-# Prerequisites:
-# - cloudnative-pg-ringtail (operator must exist before the Cluster CR)
-# - external-secrets-ringtail + 1password-connect-ringtail (for the
-#   immich-pg-borgmatic ExternalSecret to sync)
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: databases-ringtail
-  namespace: argocd
-spec:
-  project: default
-  source:
-    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
-    targetRevision: main
-    path: argocd/manifests/databases-ringtail
-  destination:
-    server: https://ringtail.tail8d86e.ts.net:6443
-    namespace: databases
-  syncPolicy:
-    syncOptions:
-      - CreateNamespace=true

argocd/apps/external-secrets-ringtail.yaml

@@ -15,7 +15,7 @@ spec:
   source:
     repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
     targetRevision: main
-    path: argocd/manifests/external-secrets-ringtail
+    path: argocd/manifests/external-secrets
   destination:
     server: https://ringtail.tail8d86e.ts.net:6443
     namespace: external-secrets

argocd/apps/immich-ringtail.yaml

@@ -1,31 +0,0 @@
-# Immich on ringtail k3s.
-#
-# Staging deployment; the minikube `immich` app remains in parallel
-# until cutover. See [[immich-cutover-and-decommission]] for the
-# routing flip + minikube cleanup.
-#
-# Prerequisites:
-# - cnpg-on-ringtail + databases-ringtail (postgres)
-# - 1password-connect-ringtail + external-secrets-ringtail (not used
-#   by this app today — immich-db Secret is created manually,
-#   matching the minikube pattern)
-# - The immich-db Secret in the immich namespace, holding the
-#   password for the `immich` postgres role (copied from the source
-#   immich-pg-app Secret at migration time).
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: immich-ringtail
-  namespace: argocd
-spec:
-  project: default
-  source:
-    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
-    targetRevision: main
-    path: argocd/manifests/immich-ringtail
-  destination:
-    server: https://ringtail.tail8d86e.ts.net:6443
-    namespace: immich
-  syncPolicy:
-    syncOptions:
-      - CreateNamespace=true

argocd/apps/immich.yaml

@@ -0,0 +1,30 @@
+# Immich - Self-hosted photo and video management
+# High-performance Google Photos/iCloud alternative with AI features
+#
+# Kustomize manifests in argocd/manifests/immich/
+# Components: server, machine-learning, valkey (Redis)
+#
+# Prerequisites:
+# 1. Create immich namespace and secrets:
+#    kubectl create namespace immich
+#    kubectl --context=minikube-indri create secret generic immich-db -n immich \
+#      --from-literal=password="$(kubectl --context=minikube-indri -n databases get secret immich-pg-app -o jsonpath='{.data.password}' | base64 -d)"
+# 2. Create immich-pg database and user (see immich-pg app)
+# 3. NFS share on sifaka at /volume1/photos with read/write for indri
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: immich
+  namespace: argocd
+spec:
+  project: default
+  source:
+    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
+    targetRevision: main
+    path: argocd/manifests/immich
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: immich
+  syncPolicy:
+    syncOptions:
+      - CreateNamespace=true

argocd/apps/mealie-ringtail.yaml

@@ -1,26 +0,0 @@
-# Mealie on ringtail k3s.
-#
-# Wave-1 indri-k8s decommission. Staging deployment; the minikube `mealie`
-# app stays in parallel until cutover (copy SQLite PVC, drop the minikube
-# tailscale ingress, flip Caddy). See [[migrate-wave1-ringtail]].
-#
-# Prerequisites:
-# - external-secrets-ringtail (onepassword-blumeops ClusterSecretStore)
-# - mealie-data PVC contents copied from minikube at cutover
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: mealie-ringtail
-  namespace: argocd
-spec:
-  project: default
-  source:
-    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
-    targetRevision: main
-    path: argocd/manifests/mealie-ringtail
-  destination:
-    server: https://ringtail.tail8d86e.ts.net:6443
-    namespace: mealie
-  syncPolicy:
-    syncOptions:
-      - CreateNamespace=true

argocd/apps/mealie.yaml

@@ -0,0 +1,17 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: mealie
+  namespace: argocd
+spec:
+  project: default
+  source:
+    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
+    targetRevision: main
+    path: argocd/manifests/mealie
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: mealie
+  syncPolicy:
+    syncOptions:
+      - CreateNamespace=true

argocd/apps/paperless-ringtail.yaml

@@ -1,28 +0,0 @@
-# Paperless-ngx on ringtail k3s.
-#
-# Wave-1 indri-k8s decommission. Staging deployment; the minikube
-# `paperless` app stays in parallel until cutover (drop the minikube
-# tailscale ingress to free the name, then flip Caddy). See
-# [[migrate-wave1-ringtail]].
-#
-# Prerequisites:
-# - databases-ringtail blumeops-pg (paperless database + role)
-# - external-secrets-ringtail (onepassword-blumeops ClusterSecretStore)
-# - sifaka NFS rule granting ringtail access to /volume1/paperless
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: paperless-ringtail
-  namespace: argocd
-spec:
-  project: default
-  source:
-    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
-    targetRevision: main
-    path: argocd/manifests/paperless-ringtail
-  destination:
-    server: https://ringtail.tail8d86e.ts.net:6443
-    namespace: paperless
-  syncPolicy:
-    syncOptions:
-      - CreateNamespace=true

argocd/apps/paperless.yaml

@@ -0,0 +1,17 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: paperless
+  namespace: argocd
+spec:
+  project: default
+  source:
+    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
+    targetRevision: main
+    path: argocd/manifests/paperless
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: paperless
+  syncPolicy:
+    syncOptions:
+      - CreateNamespace=true

argocd/apps/teslamate-ringtail.yaml

@@ -1,28 +0,0 @@
-# TeslaMate on ringtail k3s.
-#
-# Wave-1 indri-k8s decommission. Staging deployment; the minikube
-# `teslamate` app stays in parallel until cutover (migrate the teslamate
-# database, drop the minikube tailscale ingress, flip Caddy). See
-# [[migrate-wave1-ringtail]].
-#
-# Prerequisites:
-# - databases-ringtail blumeops-pg (teslamate database + role; cube +
-#   earthdistance extensions created by superuser at cutover)
-# - external-secrets-ringtail (onepassword-blumeops ClusterSecretStore)
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: teslamate-ringtail
-  namespace: argocd
-spec:
-  project: default
-  source:
-    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
-    targetRevision: main
-    path: argocd/manifests/teslamate-ringtail
-  destination:
-    server: https://ringtail.tail8d86e.ts.net:6443
-    namespace: teslamate
-  syncPolicy:
-    syncOptions:
-      - CreateNamespace=true

argocd/apps/teslamate.yaml

@@ -0,0 +1,32 @@
+# TeslaMate Tesla Data Logger
+# Requires: CloudNativePG PostgreSQL cluster and manual secret setup
+#
+# Before syncing, create the namespace and secrets:
+#   kubectl create namespace teslamate
+#   op inject -i argocd/manifests/databases/secret-teslamate.yaml.tpl | kubectl apply -f -
+#   op inject -i argocd/manifests/teslamate/secret-encryption-key.yaml.tpl | kubectl apply -f -
+#   op inject -i argocd/manifests/teslamate/secret-db.yaml.tpl | kubectl apply -f -
+#
+# Then create the database:
+#   PGPASSWORD=$(op read "op://blumeops/postgres/password") \
+#     psql -h pg.ops.eblu.me -U eblume -c "CREATE DATABASE teslamate OWNER teslamate;"
+#
+# After syncing, access the TeslaMate UI at https://tesla.tail8d86e.ts.net to complete
+# Tesla API authentication via OAuth flow.
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: teslamate
+  namespace: argocd
+spec:
+  project: default
+  source:
+    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
+    targetRevision: main
+    path: argocd/manifests/teslamate
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: teslamate
+  syncPolicy:
+    syncOptions:
+      - CreateNamespace=true

argocd/manifests/alloy-k8s/config.alloy

@@ -191,9 +191,14 @@ prometheus.exporter.blackbox "services" {
   }
 
   target {
-    // Migrated to ringtail (wave-1); probe through Caddy over Tailscale.
     name    = "teslamate"
-    address = "https://tesla.ops.eblu.me/"
+    address = "http://teslamate.teslamate.svc.cluster.local:4000/"
+    module  = "http_2xx"
+  }
+
+  target {
+    name    = "immich"
+    address = "http://immich-server.immich.svc.cluster.local:2283/api/server/ping"
     module  = "http_2xx"
   }
 

argocd/manifests/alloy-ringtail/config.alloy

@@ -45,26 +45,6 @@ prometheus.scrape "kube_state_metrics" {
   forward_to      = [prometheus.remote_write.prometheus.receiver]
 }
 
-// ============== SERVICE HEALTH PROBES ==============
-
-// Blackbox-style HTTP probes for in-cluster services on ringtail
-prometheus.exporter.blackbox "services" {
-  config = "{ modules: { http_2xx: { prober: http, timeout: 5s } } }"
-
-  target {
-    name    = "immich"
-    address = "http://immich-server.immich.svc.cluster.local:2283/api/server/ping"
-    module  = "http_2xx"
-  }
-}
-
-// Scrape blackbox probe results
-prometheus.scrape "blackbox" {
-  targets         = prometheus.exporter.blackbox.services.targets
-  scrape_interval = "30s"
-  forward_to      = [prometheus.remote_write.prometheus.receiver]
-}
-
 // Push metrics to indri Prometheus
 prometheus.remote_write "prometheus" {
   external_labels = { cluster = "ringtail" }

argocd/manifests/argocd/argocd-rbac-cm-patch.yaml

@@ -2,9 +2,6 @@
 #
 # - workflow-bot: minimal CI/CD permissions (sync, get)
 # - admins: Authentik admins group mapped to ArgoCD admin role
-# - admin: local break-glass account — keeps ArgoCD admin rights for when
-#   Authentik SSO is unavailable (without this it has no permissions, since
-#   policy.default is unset)
 #
 apiVersion: v1
 kind: ConfigMap
@@ -17,4 +14,3 @@ data:
     p, role:workflow-bot, applications, get, *, allow
     g, workflow-bot, role:workflow-bot
     g, admins, role:admin
-    g, admin, role:admin

argocd/manifests/authentik/configmap-blueprint.yaml

@@ -434,93 +434,3 @@ data:
           provider: !KeyOf mealie-provider
           meta_launch_url: https://meals.ops.eblu.me
           policy_engine_mode: all
-
-  heph.yaml: |
-    version: 1
-    metadata:
-      name: BlumeOps Heph SSO
-      labels:
-        blueprints.goauthentik.io/description: "Hephaestus hub OIDC (device-code) provider, application, and device-code flow"
-    entries:
-      # Device-code flow (RFC 8628). authentik ships no default for this, so we
-      # create one and bind it to the brand below. An empty stage_configuration
-      # flow is sufficient: the already-authenticated user just confirms the code.
-      - model: authentik_flows.flow
-        id: device-code-flow
-        identifiers:
-          slug: default-device-code-flow
-        attrs:
-          name: Device code flow
-          title: Device code flow
-          slug: default-device-code-flow
-          designation: stage_configuration
-          authentication: require_authenticated
-
-      # Enable the device-code grant globally by binding the flow to the default
-      # brand (domain authentik-default). Partial update — only sets this field.
-      - model: authentik_brands.brand
-        identifiers:
-          domain: authentik-default
-        attrs:
-          flow_device_code: !KeyOf device-code-flow
-
-      # OAuth2 provider for heph — PUBLIC client (device-code + PKCE, no secret).
-      # client_id doubles as the token audience the hub verifies (--oidc-audience heph),
-      # and the app slug 'heph' is the issuer path (/application/o/heph/).
-      - model: authentik_providers_oauth2.oauth2provider
-        id: heph-provider
-        identifiers:
-          name: Heph
-        attrs:
-          name: Heph
-          authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
-          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
-          client_type: public
-          client_id: heph
-          # CLI/TUI use the device-code grant (no redirect). The heph-pwa browser
-          # login uses Authorization Code + PKCE, which DOES redirect back to the
-          # app's origin — register those here (Authentik also keys token-endpoint
-          # CORS off these origins). Trailing slash matters: the PWA's redirect_uri
-          # is its base dir, e.g. https://heph.ops.eblu.me/.
-          redirect_uris:
-            - matching_mode: strict
-              url: https://heph.ops.eblu.me/
-            - matching_mode: strict
-              url: http://localhost:8787/  # local dev (hephd --web-root)
-          signing_key: !Find [authentik_crypto.certificatekeypair, [name, authentik Self-signed Certificate]]
-          property_mappings:
-            - !Find [authentik_providers_oauth2.scopemapping, [scope_name, openid]]
-            - !Find [authentik_providers_oauth2.scopemapping, [scope_name, email]]
-            - !Find [authentik_providers_oauth2.scopemapping, [scope_name, profile]]
-            # offline_access: heph CLI requests "openid offline_access"; without
-            # this mapping the refresh token is session-bound and hephd's
-            # refresh_token grant 400s once the session lapses (spoke sync dies).
-            - !Find [authentik_providers_oauth2.scopemapping, [scope_name, offline_access]]
-          sub_mode: hashed_user_id
-          include_claims_in_id_token: true
-
-      # Heph application — linked to the OAuth2 provider
-      - model: authentik_core.application
-        id: heph-app
-        identifiers:
-          slug: heph
-        attrs:
-          name: Hephaestus
-          slug: heph
-          provider: !KeyOf heph-provider
-          meta_launch_url: https://heph.ops.eblu.me
-          policy_engine_mode: any
-
-      # Policy binding — restrict heph to admins group (single-owner, sensitive data)
-      - model: authentik_policies.policybinding
-        identifiers:
-          order: 0
-          target: !KeyOf heph-app
-          group: !Find [authentik_core.group, [name, admins]]
-        attrs:
-          target: !KeyOf heph-app
-          group: !Find [authentik_core.group, [name, admins]]
-          order: 0
-          enabled: true
-          negate: false
-          timeout: 30

argocd/manifests/databases-ringtail/blumeops-pg.yaml

@@ -1,97 +0,0 @@
-# PostgreSQL Cluster for blumeops services on ringtail k3s.
-#
-# Wave-1 indri-k8s decommission target (see [[migrate-wave1-ringtail]]).
-# Holds the paperless and teslamate databases migrated off the minikube
-# blumeops-pg via cold pg_dump/pg_restore at cutover. miniflux + authentik
-# stay where they are for now (later waves), so this cluster only carries
-# the wave-1 roles.
-#
-# Apps reach this in-cluster at blumeops-pg-rw.databases.svc.cluster.local
-# — the same name they used on minikube, so teslamate's DATABASE_HOST is
-# unchanged.
-#
-# Database creation is deferred to cutover, mirroring the minikube cluster
-# (where only the bootstrap database is declared and the rest were created
-# out-of-band):
-#   - paperless: the bootstrap database below (restored into at cutover).
-#   - teslamate: created at its cutover by the eblume superuser, because the
-#     dump's `earthdistance` extension is untrusted and CREATE EXTENSION
-#     needs superuser. (cube + earthdistance ownership then transferred to
-#     the teslamate role so it can ALTER EXTENSION UPDATE.)
-apiVersion: postgresql.cnpg.io/v1
-kind: Cluster
-metadata:
-  name: blumeops-pg
-  namespace: databases
-spec:
-  instances: 1
-  imageName: ghcr.io/cloudnative-pg/postgresql:18.3
-
-  storage:
-    size: 10Gi
-    storageClass: local-path
-
-  bootstrap:
-    initdb:
-      database: paperless
-      owner: paperless
-
-  managed:
-    roles:
-      # eblume superuser for admin + privileged restore steps (extensions)
-      - name: eblume
-        login: true
-        superuser: true
-        createdb: true
-        createrole: true
-        connectionLimit: -1
-        ensure: present
-        inherit: true
-        passwordSecret:
-          name: blumeops-pg-eblume
-      # borgmatic read-only user for backups
-      - name: borgmatic
-        login: true
-        connectionLimit: -1
-        ensure: present
-        inherit: true
-        inRoles:
-          - pg_read_all_data
-        passwordSecret:
-          name: blumeops-pg-borgmatic
-      # paperless user (also the bootstrap database owner above; the
-      # managed role sets its password from the 1Password-backed secret)
-      - name: paperless
-        login: true
-        connectionLimit: -1
-        ensure: present
-        inherit: true
-        passwordSecret:
-          name: blumeops-pg-paperless
-      # teslamate user. Extension ownership (cube, earthdistance) is
-      # transferred to this role at cutover so it can ALTER EXTENSION UPDATE.
-      - name: teslamate
-        login: true
-        connectionLimit: -1
-        ensure: present
-        inherit: true
-        passwordSecret:
-          name: blumeops-pg-teslamate
-
-  resources:
-    requests:
-      memory: "256Mi"
-      cpu: "100m"
-    limits:
-      memory: "1Gi"
-      cpu: "500m"
-
-  postgresql:
-    parameters:
-      max_connections: "50"
-      shared_buffers: "128MB"
-      password_encryption: "scram-sha-256"
-    pg_hba:
-      # Password auth from anywhere; network security is via Tailscale.
-      - host all all 0.0.0.0/0 scram-sha-256
-      - host all all ::/0 scram-sha-256

argocd/manifests/databases-ringtail/external-secret-eblume.yaml

@@ -1,30 +0,0 @@
-# ExternalSecret for eblume superuser password
-#
-# Replaces the manual op inject workflow from secret-eblume.yaml.tpl
-#
-# 1Password item: "postgres" in blumeops vault
-# Field: "password"
-#
-apiVersion: external-secrets.io/v1
-kind: ExternalSecret
-metadata:
-  name: blumeops-pg-eblume
-  namespace: databases
-spec:
-  refreshInterval: 1h
-  secretStoreRef:
-    kind: ClusterSecretStore
-    name: onepassword-blumeops
-  target:
-    name: blumeops-pg-eblume
-    creationPolicy: Owner
-    template:
-      type: kubernetes.io/basic-auth
-      data:
-        username: eblume
-        password: "{{ .password }}"
-  data:
-  - secretKey: password
-    remoteRef:
-      key: postgres
-      property: password

argocd/manifests/databases-ringtail/external-secret-immich-borgmatic.yaml

@@ -1,32 +0,0 @@
-# ExternalSecret for borgmatic backup user password on immich-pg cluster
-# (ringtail k3s).
-#
-# Mirror of argocd/manifests/databases/external-secret-immich-borgmatic.yaml.
-# The onepassword-blumeops ClusterSecretStore exists on ringtail via the
-# external-secrets-ringtail app.
-#
-# 1Password item: "borgmatic" in blumeops vault
-# Field: "db-password"
-apiVersion: external-secrets.io/v1
-kind: ExternalSecret
-metadata:
-  name: immich-pg-borgmatic
-  namespace: databases
-spec:
-  refreshInterval: 1h
-  secretStoreRef:
-    kind: ClusterSecretStore
-    name: onepassword-blumeops
-  target:
-    name: immich-pg-borgmatic
-    creationPolicy: Owner
-    template:
-      type: kubernetes.io/basic-auth
-      data:
-        username: borgmatic
-        password: "{{ .password }}"
-  data:
-    - secretKey: password
-      remoteRef:
-        key: borgmatic
-        property: db-password

argocd/manifests/databases-ringtail/immich-pg.yaml

@@ -1,53 +0,0 @@
-# PostgreSQL Cluster for Immich on ringtail k3s.
-#
-# Initially bootstrapped via CNPG pg_basebackup from the minikube
-# immich-pg cluster on 2026-05-13, then promoted to primary. The
-# externalClusters + bootstrap.pg_basebackup blocks have been pruned
-# from this manifest now that the migration is complete — leaving
-# them around is a footgun (re-enabling replica.enabled=true would
-# try to demote this cluster against a stale source). See
-# [[immich-pg-data-migration]] for the procedure used.
-apiVersion: postgresql.cnpg.io/v1
-kind: Cluster
-metadata:
-  name: immich-pg
-  namespace: databases
-spec:
-  instances: 1
-  imageName: ghcr.io/tensorchord/cloudnative-vectorchord:17-0.5.0
-
-  storage:
-    size: 10Gi
-    storageClass: local-path
-
-  # Managed roles
-  managed:
-    roles:
-      - name: borgmatic
-        login: true
-        connectionLimit: -1
-        ensure: present
-        inherit: true
-        inRoles:
-          - pg_read_all_data
-        passwordSecret:
-          name: immich-pg-borgmatic
-
-  resources:
-    requests:
-      memory: "256Mi"
-      cpu: "100m"
-    limits:
-      memory: "1Gi"
-      cpu: "500m"
-
-  postgresql:
-    shared_preload_libraries:
-      - "vchord.so"
-    parameters:
-      max_connections: "50"
-      shared_buffers: "128MB"
-      password_encryption: "scram-sha-256"
-    pg_hba:
-      - host all all 0.0.0.0/0 scram-sha-256
-      - host all all ::/0 scram-sha-256

argocd/manifests/databases-ringtail/kustomization.yaml

@@ -1,16 +0,0 @@
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-
-namespace: databases
-
-resources:
-  - immich-pg.yaml
-  - external-secret-immich-borgmatic.yaml
-  - service-immich-pg-tailscale.yaml
-  # wave-1 indri-k8s decommission: blumeops-pg (paperless + teslamate)
-  - blumeops-pg.yaml
-  - service-blumeops-pg-tailscale.yaml
-  - external-secret-eblume.yaml
-  - external-secret-borgmatic.yaml
-  - external-secret-paperless.yaml
-  - external-secret-teslamate.yaml

argocd/manifests/databases-ringtail/service-blumeops-pg-tailscale.yaml

@@ -1,24 +0,0 @@
-# Tailscale LoadBalancer for the ringtail blumeops-pg cluster.
-# Canonical hostname: blumeops-pg-ringtail.tail8d86e.ts.net (distinct from
-# the minikube blumeops-pg, which still owns pg.tail8d86e.ts.net until the
-# wave-1 decommission). Borgmatic on indri and the Grafana TeslaMate
-# datasource reach it via the Caddy L4 route pg.ops.eblu.me:5434.
-apiVersion: v1
-kind: Service
-metadata:
-  name: blumeops-pg-tailscale
-  namespace: databases
-  annotations:
-    tailscale.com/hostname: "blumeops-pg-ringtail"
-    tailscale.com/proxy-class: "default"
-spec:
-  type: LoadBalancer
-  loadBalancerClass: tailscale
-  selector:
-    cnpg.io/cluster: blumeops-pg
-    role: primary
-  ports:
-    - name: postgresql
-      port: 5432
-      targetPort: 5432
-      protocol: TCP

argocd/manifests/databases/blumeops-pg.yaml

@@ -44,9 +44,18 @@ spec:
           - pg_read_all_data
         passwordSecret:
           name: blumeops-pg-borgmatic
-      # teslamate + paperless roles removed: migrated to ringtail blumeops-pg
-      # (wave-1 decommission). Their databases were dropped from this cluster
-      # after the cutover was verified and backed up.
+      # teslamate user for TeslaMate Tesla data logger
+      # Superuser removed. Extension ownership (cube, earthdistance)
+      # transferred manually so teslamate can ALTER EXTENSION UPDATE.
+      # earthdistance is untrusted — DROP+CREATE needs temporary
+      # superuser escalation during upgrades.
+      - name: teslamate
+        login: true
+        connectionLimit: -1
+        ensure: present
+        inherit: true
+        passwordSecret:
+          name: blumeops-pg-teslamate
       # authentik user for Authentik identity provider (runs on ringtail)
       - name: authentik
         login: true
@@ -56,6 +65,14 @@ spec:
         createdb: true
         passwordSecret:
           name: blumeops-pg-authentik
+      # paperless user for Paperless-ngx document management
+      - name: paperless
+        login: true
+        connectionLimit: -1
+        ensure: present
+        inherit: true
+        passwordSecret:
+          name: blumeops-pg-paperless
 
   # Resource limits for minikube environment
   resources:

argocd/manifests/databases/external-secret-immich-borgmatic.yaml

@@ -1,14 +1,13 @@
-# ExternalSecret for borgmatic backup user password
-#
-# Replaces the manual op inject workflow from secret-borgmatic.yaml.tpl
+# ExternalSecret for borgmatic backup user password on immich-pg cluster
 #
+# Reuses the same 1Password item as blumeops-pg-borgmatic.
 # 1Password item: "borgmatic" in blumeops vault
 # Field: "db-password"
 #
 apiVersion: external-secrets.io/v1
 kind: ExternalSecret
 metadata:
-  name: blumeops-pg-borgmatic
+  name: immich-pg-borgmatic
   namespace: databases
 spec:
   refreshInterval: 1h
@@ -16,7 +15,7 @@ spec:
     kind: ClusterSecretStore
     name: onepassword-blumeops
   target:
-    name: blumeops-pg-borgmatic
+    name: immich-pg-borgmatic
     creationPolicy: Owner
     template:
       type: kubernetes.io/basic-auth

argocd/manifests/databases/immich-pg.yaml

@@ -0,0 +1,69 @@
+# PostgreSQL Cluster for Immich
+# Uses VectorChord (successor to pgvecto.rs) for AI-powered vector search
+# See: https://github.com/immich-app/immich/discussions/9060
+# Managed by CloudNativePG operator
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: immich-pg
+  namespace: databases
+spec:
+  instances: 1
+  # VectorChord image for PostgreSQL 17 with VectorChord 0.5.0
+  # Immich v2.4.1 requires VectorChord >=0.3 <0.6
+  # See: https://github.com/tensorchord/VectorChord
+  imageName: ghcr.io/tensorchord/cloudnative-vectorchord:17-0.5.0
+
+  storage:
+    size: 10Gi
+    storageClass: standard
+
+  # Bootstrap creates initial database and owner
+  bootstrap:
+    initdb:
+      database: immich
+      owner: immich
+      postInitSQL:
+        # Extensions required by Immich
+        - CREATE EXTENSION IF NOT EXISTS vector;
+        - CREATE EXTENSION IF NOT EXISTS vchord CASCADE;
+        - CREATE EXTENSION IF NOT EXISTS cube CASCADE;
+        - CREATE EXTENSION IF NOT EXISTS earthdistance CASCADE;
+
+  # Managed roles
+  # Note: connectionLimit, ensure, inherit are CNPG defaults added to prevent ArgoCD drift
+  managed:
+    roles:
+      # borgmatic read-only user for backups
+      - name: borgmatic
+        login: true
+        connectionLimit: -1
+        ensure: present
+        inherit: true
+        inRoles:
+          - pg_read_all_data
+        passwordSecret:
+          name: immich-pg-borgmatic
+
+  # Resource limits for minikube environment
+  resources:
+    requests:
+      memory: "256Mi"
+      cpu: "100m"
+    limits:
+      memory: "1Gi"
+      cpu: "500m"
+
+  # PostgreSQL configuration
+  postgresql:
+    # VectorChord requires vchord.so in shared_preload_libraries
+    shared_preload_libraries:
+      - "vchord.so"
+    parameters:
+      max_connections: "50"
+      shared_buffers: "128MB"
+      password_encryption: "scram-sha-256"
+    pg_hba:
+      # Allow connections from k8s pods
+      - host all all 0.0.0.0/0 scram-sha-256
+      - host all all ::/0 scram-sha-256

argocd/manifests/databases/kustomization.yaml

@@ -5,8 +5,13 @@ namespace: databases
 
 resources:
   - blumeops-pg.yaml
+  - immich-pg.yaml
   - service-tailscale.yaml
+  - service-immich-pg-tailscale.yaml
   - service-metrics-tailscale.yaml
   - external-secret-eblume.yaml
   - external-secret-borgmatic.yaml
+  - external-secret-immich-borgmatic.yaml
+  - external-secret-teslamate.yaml
   - external-secret-authentik.yaml
+  - external-secret-paperless.yaml

argocd/manifests/databases/service-immich-pg-tailscale.yaml

@@ -1,8 +1,6 @@
-# Tailscale LoadBalancer for immich-pg PostgreSQL access on ringtail.
-# Canonical hostname: immich-pg.tail8d86e.ts.net (claimed from the
-# minikube side after the minikube service was removed during the
-# immich-to-ringtail migration). Borgmatic on indri uses this
-# hostname for nightly backups.
+# Tailscale LoadBalancer for immich-pg PostgreSQL access
+# Canonical hostname: immich-pg.tail8d86e.ts.net
+# Caddy L4 proxies pg.ops.eblu.me:5433 → this service for borgmatic backups
 apiVersion: v1
 kind: Service
 metadata:

argocd/manifests/external-secrets-ringtail/kustomization.yaml

@@ -1,16 +0,0 @@
-# Ringtail (amd64) overlay for external-secrets.
-#
-# Reuses the shared indri manifest as a base and only overrides the controller
-# image to the nix-built amd64 variant (`-nix` tag). The base sets the arm64
-# image (built via containers/external-secrets/container.py on indri's Dagger
-# runner); ringtail's k3s is amd64 and needs the image built by
-# containers/external-secrets/default.nix on the nix-container-builder.
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-
-resources:
-  - ../external-secrets
-
-images:
-  - name: registry.ops.eblu.me/blumeops/external-secrets
-    newTag: v2.2.0-13895bb-nix

argocd/manifests/external-secrets/kustomization.yaml

@@ -12,5 +12,4 @@ resources:
 
 images:
   - name: ghcr.io/external-secrets/external-secrets
-    newName: registry.ops.eblu.me/blumeops/external-secrets
-    newTag: v2.2.0-13895bb
+    newTag: v2.2.0

argocd/manifests/grafana/datasources.yaml

@@ -63,7 +63,5 @@ datasources:
     password: $TESLAMATE_DB_PASSWORD
   type: postgres
   uid: TeslaMate
-  # teslamate DB migrated to ringtail blumeops-pg (wave-1); reached via the
-  # Caddy L4 route on indri (pg.ops.eblu.me:5434 -> blumeops-pg-ringtail).
-  url: pg.ops.eblu.me:5434
+  url: blumeops-pg-rw.databases.svc.cluster.local:5432
   user: teslamate

argocd/manifests/grafana/deployment.yaml

@@ -14,9 +14,7 @@ spec:
       app.kubernetes.io/name: grafana
       app.kubernetes.io/instance: grafana
   strategy:
-    # RWO PVC for SQLite + Bleve index — RollingUpdate spawns the new pod
-    # before the old one terminates, and it crashloops on the index lock.
-    type: Recreate
+    type: RollingUpdate
   template:
     metadata:
       labels:

argocd/manifests/homepage/services.yaml

@@ -71,6 +71,10 @@
           enableBlocks: true
           enableNowPlaying: false
           fields: ["movies", "series", "episodes"]
+    - Mealie:
+        href: https://meals.ops.eblu.me
+        icon: mealie.png
+        description: Recipe manager
     - DJ:
         href: https://dj.ops.eblu.me
         icon: navidrome.png
@@ -81,7 +85,15 @@
           user: "{{HOMEPAGE_VAR_NAVIDROME_USER}}"
           token: "{{HOMEPAGE_VAR_NAVIDROME_TOKEN}}"
           salt: "{{HOMEPAGE_VAR_NAVIDROME_SALT}}"
+    - Paperless:
+        href: https://paperless.ops.eblu.me
+        icon: paperless-ngx.png
+        description: Document management
 - Content:
+    - Immich:
+        href: https://photos.ops.eblu.me
+        icon: immich.png
+        description: Photo management
     - Kiwix:
         href: https://kiwix.ops.eblu.me
         icon: kiwix.png
@@ -126,6 +138,10 @@
         href: https://docs.eblu.me
         icon: mdi-book-open-page-variant
         description: BlumeOps Documentation
+    - TeslaMate:
+        href: https://tesla.ops.eblu.me
+        icon: teslamate.png
+        description: Tesla data logger
     - Transmission:
         href: https://torrent.ops.eblu.me
         icon: transmission.png

argocd/manifests/immich-ringtail/pv-nfs.yaml

@@ -1,29 +0,0 @@
-# NFS PersistentVolume for Immich photo library on ringtail k3s.
-#
-# Mirror of argocd/manifests/immich/pv-nfs.yaml (minikube) but with
-# a distinct name (minikube and ringtail are separate clusters, so PV
-# names don't collide cluster-side, but using the same name in two
-# manifests is confusing).
-#
-# The sifaka NFS export for /volume1/photos already permits
-# 192.168.1.0/24 + 100.64.0.0/10. Ringtail's wired IP (192.168.1.21)
-# falls in the first CIDR, so no DSM rule changes are needed.
-#
-# Verified 2026-05-13: ringtail pod can read existing dirs, write
-# new files, and delete them. DNS resolves sifaka to 192.168.1.203
-# (LAN), so NFS traffic stays off the tailnet — avoids the known
-# sifaka-tailscale-userspace bite.
-apiVersion: v1
-kind: PersistentVolume
-metadata:
-  name: immich-library-nfs-pv-ringtail
-spec:
-  capacity:
-    storage: 2Ti
-  accessModes:
-    - ReadWriteMany
-  persistentVolumeReclaimPolicy: Retain
-  storageClassName: ""
-  nfs:
-    server: sifaka
-    path: /volume1/photos

argocd/manifests/immich/README.md

@@ -0,0 +1,115 @@
+# Immich
+
+Self-hosted photo and video management solution with AI-powered search and face recognition.
+
+## Prerequisites
+
+1. **NFS Share**: Create `/volume1/photos` on sifaka with NFS permissions for indri
+2. **PostgreSQL**: The `immich-pg` cluster (with pgvecto.rs) must be healthy
+3. **Secrets**: Create the database password secret
+
+## Deployment Order
+
+1. Sync `blumeops-pg` (to get CloudNativePG operator if not already running)
+2. Wait for `immich-pg` cluster to be healthy
+3. Create secrets (see below)
+4. Sync `immich` (deploys all resources: storage, services, deployments)
+5. Run `mise run provision-indri -- --tags caddy` to update Caddy config
+
+## Components
+
+| Component | Deployment | Service | Port |
+|-----------|------------|---------|------|
+| Server (web/API) | `immich-server` | `immich-server` | 2283 |
+| Machine Learning | `immich-machine-learning` | `immich-machine-learning` | 3003 |
+| Valkey (Redis) | `immich-valkey` | `immich-valkey` | 6379 |
+
+## Secret Setup
+
+The `immich-db` secret contains the database password, which is auto-generated by CloudNativePG
+in the `immich-pg-app` secret. To create or regenerate the secret:
+
+```bash
+# Create namespace if needed
+kubectl --context=minikube-indri create namespace immich
+
+# Copy password from CNPG secret to immich namespace
+kubectl --context=minikube-indri create secret generic immich-db -n immich \
+  --from-literal=password="$(kubectl --context=minikube-indri -n databases get secret immich-pg-app -o jsonpath='{.data.password}' | base64 -d)"
+```
+
+Note: This secret is not managed by ExternalSecrets since the source of truth is the CNPG-generated secret.
+
+## Access
+
+- **URL**: https://photos.ops.eblu.me (after Caddy is updated)
+- **Tailscale**: https://photos.tail8d86e.ts.net (direct)
+
+## First-Time Setup
+
+1. Navigate to https://photos.ops.eblu.me
+2. Create an admin account
+3. Configure external library (optional - for importing existing photos)
+
+## External Library (iCloud Photos)
+
+To import existing photos from iCloud sync on indri:
+
+1. In Immich Admin > External Libraries, create a new library
+2. Set the import path to the location where iCloud photos sync
+3. Configure scan schedule or trigger manual scan
+
+## Architecture
+
+```
+┌─────────────────┐     ┌─────────────────┐
+│  immich-server  │────▶│  immich-pg      │
+│  (web/api)      │     │  (PostgreSQL    │
+└────────┬────────┘     │   + pgvecto.rs) │
+         │              └─────────────────┘
+         │
+┌────────▼────────┐     ┌─────────────────┐
+│  immich-ml      │     │  valkey         │
+│  (ML inference) │     │  (Redis cache)  │
+└─────────────────┘     └─────────────────┘
+         │
+┌────────▼────────┐
+│  sifaka NFS     │
+│  /volume1/photos│
+└─────────────────┘
+```
+
+## Version Management
+
+Image versions are controlled via `kustomization.yaml`:
+
+```yaml
+images:
+  - name: ghcr.io/immich-app/immich-server
+    newTag: v2.6.3
+  - name: ghcr.io/immich-app/immich-machine-learning
+    newTag: v2.6.3
+  - name: docker.io/valkey/valkey
+    newTag: "8.1-alpine"
+```
+
+To upgrade, update `newTag` values and sync via ArgoCD.
+
+## Troubleshooting
+
+```bash
+# Check pods
+kubectl --context=minikube-indri -n immich get pods
+
+# Check immich-pg cluster
+kubectl --context=minikube-indri -n databases get cluster immich-pg
+
+# View server logs
+kubectl --context=minikube-indri -n immich logs -l app=immich,component=server
+
+# View ML logs
+kubectl --context=minikube-indri -n immich logs -l app=immich,component=machine-learning
+
+# Check PVC binding
+kubectl --context=minikube-indri -n immich get pvc
+```

argocd/manifests/immich/deployment-ml.yaml

@@ -16,16 +16,11 @@ spec:
         app: immich
         component: machine-learning
     spec:
-      runtimeClassName: nvidia
       securityContext:
         seccompProfile:
           type: RuntimeDefault
       containers:
         - name: machine-learning
-          # ringtail uses the -cuda tag (set in kustomization.yaml)
-          # to take advantage of the RTX 4080 via the nvidia
-          # device plugin. Time-slicing is configured for 4 replicas
-          # so frigate + ollama + this pod can share.
           image: ghcr.io/immich-app/immich-machine-learning:kustomized
           ports:
             - name: http
@@ -62,7 +57,6 @@ spec:
               cpu: "100m"
             limits:
               memory: "4Gi"
-              nvidia.com/gpu: "1"
       volumes:
         - name: cache
           persistentVolumeClaim:

argocd/manifests/immich/ingress-tailscale.yaml

@@ -1,9 +1,6 @@
-# Tailscale ProxyGroup Ingress for Immich on ringtail.
-#
-# Production hostname: photos.tail8d86e.ts.net
-# (during the cutover window this was photos-ringtail; the minikube
-# ingress was torn down before this was renamed to photos to avoid
-# the Tailscale device-name collision.)
+# Tailscale Ingress for Immich
+# Exposes Immich at photos.tail8d86e.ts.net
+# Caddy will proxy photos.ops.eblu.me to this endpoint
 apiVersion: networking.k8s.io/v1
 kind: Ingress
 metadata:
@@ -19,6 +16,12 @@ metadata:
     gethomepage.dev/description: "Photo management"
     gethomepage.dev/href: "https://photos.ops.eblu.me"
     gethomepage.dev/pod-selector: "app=immich,component=server"
+    # TODO: Add Immich widget - requires API key from Account Settings > API Keys
+    # See: https://gethomepage.dev/widgets/services/immich/
+    # gethomepage.dev/widget.type: "immich"
+    # gethomepage.dev/widget.url: "https://photos.ops.eblu.me"
+    # gethomepage.dev/widget.key: "{{HOMEPAGE_VAR_IMMICH_API_KEY}}"
+    # gethomepage.dev/widget.version: "2"
 spec:
   ingressClassName: tailscale
   rules:

argocd/manifests/immich/kustomization.yaml

@@ -1,8 +1,7 @@
+---
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
-
 namespace: immich
-
 resources:
   - deployment-server.yaml
   - deployment-ml.yaml
@@ -14,16 +13,11 @@ resources:
   - pv-nfs.yaml
   - pvc.yaml
   - ingress-tailscale.yaml
-
 images:
   - name: ghcr.io/immich-app/immich-server
     newTag: v2.6.3
   - name: ghcr.io/immich-app/immich-machine-learning
-    # CUDA variant of the same release — ringtail has an RTX 4080
-    newTag: v2.6.3-cuda
-  # amd64 valkey built via nix on the ringtail nix-container-builder
-  # (see containers/valkey/default.nix). The Alpine container.py build
-  # is arm64-only and serves paperless on indri.
+    newTag: v2.6.3
   - name: docker.io/valkey/valkey
     newName: registry.ops.eblu.me/blumeops/valkey
-    newTag: v8.1.7-ecded30-nix
+    newTag: v8.1.6-r0-fabca04

argocd/manifests/immich/pv-nfs.yaml

@@ -0,0 +1,22 @@
+# NFS PersistentVolume for Immich photo library
+# Requires: NFS share on sifaka at /volume1/photos with NFS permissions for indri
+#
+# To create on Synology:
+# 1. Control Panel > Shared Folder > Create
+# 2. Name: photos, Location: Volume 1
+# 3. Control Panel > File Services > NFS > NFS Rules
+# 4. Add rule for "photos" share: Hostname=indri, Privilege=Read/Write, Squash=No mapping
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: immich-library-nfs-pv
+spec:
+  capacity:
+    storage: 2Ti
+  accessModes:
+    - ReadWriteMany
+  persistentVolumeReclaimPolicy: Retain
+  storageClassName: ""
+  nfs:
+    server: sifaka
+    path: /volume1/photos

argocd/manifests/immich/pvc.yaml

@@ -1,5 +1,5 @@
-# PersistentVolumeClaim for Immich photo library on ringtail.
-# Binds to immich-library-nfs-pv-ringtail (sifaka:/volume1/photos).
+# PersistentVolumeClaim for Immich photo library
+# Binds to the NFS PV for sifaka:/volume1/photos
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
@@ -9,7 +9,7 @@ spec:
   accessModes:
     - ReadWriteMany
   storageClassName: ""
-  volumeName: immich-library-nfs-pv-ringtail
+  volumeName: immich-library-nfs-pv
   resources:
     requests:
       storage: 2Ti

argocd/manifests/mealie/deployment.yaml

@@ -1,9 +1,3 @@
-# Mealie on ringtail k3s — Nix image.
-#
-# Single gunicorn process (the Nix image's default `mealie-run` entrypoint
-# runs init_db then gunicorn), serving the prebuilt frontend. DB is SQLite
-# on the mealie-data PVC; its contents are copied from the minikube PVC at
-# cutover. See [[migrate-wave1-ringtail]].
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -11,8 +5,6 @@ metadata:
   namespace: mealie
 spec:
   replicas: 1
-  strategy:
-    type: Recreate
   selector:
     matchLabels:
       app: mealie

argocd/manifests/mealie/kustomization.yaml

@@ -12,4 +12,4 @@ resources:
 
 images:
   - name: registry.ops.eblu.me/blumeops/mealie
-    newTag: v3.16.0-e0057b4-nix
+    newTag: v3.12.0-613f05d

argocd/manifests/mealie/pvc.yaml

@@ -1,5 +1,4 @@
-# SQLite data volume for Mealie on ringtail. Contents copied from the
-# minikube mealie-data PVC at cutover (recipes, meal plans, uploaded media).
+---
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
@@ -8,7 +7,7 @@ metadata:
 spec:
   accessModes:
     - ReadWriteOnce
-  storageClassName: local-path
+  storageClassName: standard
   resources:
     requests:
       storage: 2Gi

argocd/manifests/nvidia-device-plugin/kustomization.yaml

@@ -10,4 +10,4 @@ resources:
 
 images:
   - name: nvcr.io/nvidia/k8s-device-plugin
-    newTag: v0.19.2
+    newTag: v0.19.0

argocd/manifests/nvidia-device-plugin/time-slicing-config.yaml

@@ -11,4 +11,4 @@ data:
       timeSlicing:
         resources:
           - name: nvidia.com/gpu
-            replicas: 4
+            replicas: 2

argocd/manifests/paperless-ringtail/pv-nfs.yaml

@@ -1,22 +0,0 @@
-# NFS PersistentVolume for the Paperless document library, mounted from
-# ringtail. Same sifaka export (/volume1/paperless) as the minikube PV,
-# but a distinct PV name so both clusters can declare it during the
-# parallel-run before cutover.
-#
-# Prerequisite: sifaka must have an NFS rule granting ringtail Read/Write
-# (Squash=No mapping) on the paperless share — the same step done for
-# immich. See [[sifaka-nfs-from-ringtail]].
-apiVersion: v1
-kind: PersistentVolume
-metadata:
-  name: paperless-media-nfs-pv-ringtail
-spec:
-  capacity:
-    storage: 500Gi
-  accessModes:
-    - ReadWriteMany
-  persistentVolumeReclaimPolicy: Retain
-  storageClassName: ""
-  nfs:
-    server: sifaka
-    path: /volume1/paperless

argocd/manifests/paperless/deployment.yaml

@@ -1,17 +1,3 @@
-# Paperless-ngx on ringtail k3s — Nix image, multi-process.
-#
-# The upstream s6 image ran web + worker + scheduler + consumer (and DB
-# migrations) in one container. The Nix image (containers/paperless/
-# default.nix) ships the binaries but no supervisor, so we run those as
-# four containers in one pod, sharing the local data/consume dirs
-# (emptyDir) and the NFS media volume; redis is colocated so
-# PAPERLESS_REDIS=localhost works for all. A migrate initContainer runs
-# DB migrations once before the app containers start.
-#
-# DB points in-cluster at the ringtail blumeops-pg (was pg.ops.eblu.me on
-# indri). PAPERLESS_{DATA_DIR,MEDIA_ROOT,CONSUMPTION_DIR} are set
-# explicitly because the Nix package does not default to the upstream
-# /usr/src/paperless paths.
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -19,8 +5,6 @@ metadata:
   namespace: paperless
 spec:
   replicas: 1
-  strategy:
-    type: Recreate
   selector:
     matchLabels:
       app: paperless
@@ -32,38 +16,27 @@ spec:
       securityContext:
         seccompProfile:
           type: RuntimeDefault
-      initContainers:
-        # redis as a native sidecar (restartPolicy: Always): starts before
-        # the migrate init and stays running for the app containers, so all
-        # of them reach PAPERLESS_REDIS=localhost:6379.
-        - name: redis
-          image: docker.io/library/redis:kustomized
-          restartPolicy: Always
-          ports:
-            - containerPort: 6379
-          volumeMounts:
-            - name: redis-data
-              mountPath: /data
-          resources:
-            requests:
-              memory: "32Mi"
-              cpu: "10m"
-            limits:
-              memory: "128Mi"
-        - name: migrate
+      containers:
+        - name: paperless
           image: registry.ops.eblu.me/blumeops/paperless:kustomized
-          command: ["paperless-ngx", "migrate", "--no-input"]
-          env: &paperless-env
+          ports:
+            - containerPort: 8000
+              name: http
+          env:
             - name: PAPERLESS_URL
               value: "https://paperless.ops.eblu.me"
             - name: PAPERLESS_REDIS
               value: "redis://localhost:6379"
             - name: PAPERLESS_DBHOST
-              value: "blumeops-pg-rw.databases.svc.cluster.local"
+              value: "pg.ops.eblu.me"
             - name: PAPERLESS_DBPORT
               value: "5432"
             - name: PAPERLESS_DBNAME
               value: "paperless"
+            # Explicit port to override k8s-injected PAPERLESS_PORT env var
+            # (k8s sets PAPERLESS_PORT=tcp://... for a service named 'paperless')
+            - name: PAPERLESS_PORT
+              value: "8000"
             - name: PAPERLESS_DBUSER
               value: "paperless"
             - name: PAPERLESS_DBPASS
@@ -71,16 +44,6 @@ spec:
                 secretKeyRef:
                   name: paperless-secrets
                   key: db-password
-            # Explicit port to override the k8s-injected PAPERLESS_PORT
-            # (service named 'paperless' would set PAPERLESS_PORT=tcp://...)
-            - name: PAPERLESS_PORT
-              value: "8000"
-            - name: PAPERLESS_DATA_DIR
-              value: "/usr/src/paperless/data"
-            - name: PAPERLESS_MEDIA_ROOT
-              value: "/usr/src/paperless/media"
-            - name: PAPERLESS_CONSUMPTION_DIR
-              value: "/usr/src/paperless/consume"
             - name: PAPERLESS_SECRET_KEY
               valueFrom:
                 secretKeyRef:
@@ -92,6 +55,7 @@ spec:
               value: "eng"
             - name: PAPERLESS_TASK_WORKERS
               value: "1"
+            # Admin account (created on first startup)
             - name: PAPERLESS_ADMIN_USER
               value: "eblume"
             - name: PAPERLESS_ADMIN_PASSWORD
@@ -101,6 +65,8 @@ spec:
                   key: admin-password
             - name: PAPERLESS_ADMIN_MAIL
               value: "blume.erich@gmail.com"
+            # OIDC via Authentik
+            # Full JSON blob pulled from 1Password (includes client secret)
             - name: PAPERLESS_APPS
               value: "allauth.socialaccount.providers.openid_connect"
             - name: PAPERLESS_SOCIALACCOUNT_PROVIDERS
@@ -116,27 +82,19 @@ spec:
               value: "false"
             - name: PAPERLESS_REDIRECT_LOGIN_TO_SSO
               value: "false"
-          volumeMounts: &paperless-mounts
+          volumeMounts:
             - name: data
               mountPath: /usr/src/paperless/data
             - name: media
               mountPath: /usr/src/paperless/media
             - name: consume
               mountPath: /usr/src/paperless/consume
-      containers:
-        - name: web
-          image: registry.ops.eblu.me/blumeops/paperless:kustomized
-          ports:
-            - containerPort: 8000
-              name: http
-          env: *paperless-env
-          volumeMounts: *paperless-mounts
           resources:
             requests:
               memory: "256Mi"
               cpu: "100m"
             limits:
-              memory: "1Gi"
+              memory: "2Gi"
               cpu: "1000m"
           livenessProbe:
             httpGet:
@@ -151,42 +109,16 @@ spec:
             initialDelaySeconds: 30
             periodSeconds: 10
 
-        - name: worker
-          image: registry.ops.eblu.me/blumeops/paperless:kustomized
-          command: ["celery", "--app", "paperless", "worker", "--loglevel", "INFO"]
-          env: *paperless-env
-          volumeMounts: *paperless-mounts
+        - name: redis
+          image: docker.io/library/redis:kustomized
+          ports:
+            - containerPort: 6379
           resources:
             requests:
-              memory: "256Mi"
-              cpu: "100m"
+              memory: "32Mi"
+              cpu: "10m"
             limits:
-              memory: "1Gi"
-              cpu: "1000m"
-
-        - name: beat
-          image: registry.ops.eblu.me/blumeops/paperless:kustomized
-          command: ["celery", "--app", "paperless", "beat", "--loglevel", "INFO"]
-          env: *paperless-env
-          volumeMounts: *paperless-mounts
-          resources:
-            requests:
-              memory: "64Mi"
-              cpu: "20m"
-            limits:
-              memory: "256Mi"
-
-        - name: consumer
-          image: registry.ops.eblu.me/blumeops/paperless:kustomized
-          command: ["paperless-ngx", "document_consumer"]
-          env: *paperless-env
-          volumeMounts: *paperless-mounts
-          resources:
-            requests:
               memory: "128Mi"
-              cpu: "50m"
-            limits:
-              memory: "512Mi"
 
       volumes:
         - name: data
@@ -196,6 +128,3 @@ spec:
             claimName: paperless-media
         - name: consume
           emptyDir: {}
-        - name: redis-data
-          emptyDir:
-            sizeLimit: 1Gi

argocd/manifests/paperless/kustomization.yaml

@@ -13,9 +13,7 @@ resources:
 
 images:
   - name: registry.ops.eblu.me/blumeops/paperless
-    newTag: v2.20.15-fcac8e5-nix
-  # amd64 valkey built via nix (the v8.1.7-ecded30 tag without -nix is the
-  # arm64 Alpine build for indri and fails on ringtail with exec format error)
+    newTag: v2.20.13-07f52e9
   - name: docker.io/library/redis
     newName: registry.ops.eblu.me/blumeops/valkey
-    newTag: v8.1.7-ecded30-nix
+    newTag: v8.1.6-r0-fabca04

argocd/manifests/paperless/pv-nfs.yaml

@@ -0,0 +1,22 @@
+# NFS PersistentVolume for Paperless document library
+# Requires: NFS share on sifaka at /volume1/paperless with NFS permissions for indri
+#
+# To create on Synology:
+# 1. Control Panel > Shared Folder > Create
+# 2. Name: paperless, Location: Volume 1
+# 3. Control Panel > File Services > NFS > NFS Rules
+# 4. Add rule for "paperless" share: Hostname=indri, Privilege=Read/Write, Squash=No mapping
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: paperless-media-nfs-pv
+spec:
+  capacity:
+    storage: 500Gi
+  accessModes:
+    - ReadWriteMany
+  persistentVolumeReclaimPolicy: Retain
+  storageClassName: ""
+  nfs:
+    server: sifaka
+    path: /volume1/paperless

argocd/manifests/paperless/pvc.yaml

@@ -1,5 +1,5 @@
-# PersistentVolumeClaim for the Paperless document library on ringtail.
-# Binds the NFS PV for sifaka:/volume1/paperless.
+# PersistentVolumeClaim for Paperless document library
+# Binds to the NFS PV for sifaka:/volume1/paperless
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
@@ -9,7 +9,7 @@ spec:
   accessModes:
     - ReadWriteMany
   storageClassName: ""
-  volumeName: paperless-media-nfs-pv-ringtail
+  volumeName: paperless-media-nfs-pv
   resources:
     requests:
       storage: 500Gi

argocd/manifests/prowler/mutelist/apiserver.yaml

@@ -6,48 +6,48 @@ Mutelist:
         "apiserver_always_pull_images_plugin":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Only the operator has cluster access; all images pulled from private zot registry."
+          Description: "CC: single-user-cluster, local-registry. Only the operator has cluster access; all images pulled from private zot registry."
         "apiserver_audit_log_maxage_set":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Alloy/Loki provides pod-level audit trail."
+          Description: "CC: observability-stack-audit. Alloy/Loki provides pod-level audit trail."
         "apiserver_audit_log_maxbackup_set":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Alloy/Loki provides pod-level audit trail."
+          Description: "CC: observability-stack-audit. Alloy/Loki provides pod-level audit trail."
         "apiserver_audit_log_maxsize_set":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Alloy/Loki provides pod-level audit trail."
+          Description: "CC: observability-stack-audit. Alloy/Loki provides pod-level audit trail."
         "apiserver_audit_log_path_set":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Alloy/Loki provides pod-level audit trail."
+          Description: "CC: observability-stack-audit. Alloy/Loki provides pod-level audit trail."
         "apiserver_deny_service_external_ips":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "No external IPs routable; cluster only reachable via tailnet."
+          Description: "CC: tailscale-network-isolation. No external IPs routable; cluster only reachable via tailnet."
         "apiserver_disable_profiling":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Profiling endpoint unreachable from public internet."
+          Description: "CC: tailscale-network-isolation. Profiling endpoint unreachable from public internet."
         "apiserver_encryption_provider_config_set":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Etcd not network-exposed; only operator has node access."
+          Description: "CC: tailscale-network-isolation, single-user-cluster. Etcd not network-exposed; only operator has node access."
         "apiserver_kubelet_cert_auth":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Kubelet API not exposed outside the node; minikube auto-generates certificates."
+          Description: "CC: tailscale-network-isolation. Kubelet API not exposed outside the node; minikube auto-generates certificates."
         "apiserver_request_timeout_set":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "API server only reachable via tailnet; DoS risk limited to trusted clients."
+          Description: "CC: tailscale-network-isolation. API server only reachable via tailnet; DoS risk limited to trusted clients."
         "apiserver_service_account_lookup_true":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "Only operator manages service accounts; no revoked tokens in circulation."
+          Description: "CC: single-user-cluster. Only operator manages service accounts; no revoked tokens in circulation."
         "apiserver_strong_ciphers_only":
           Regions: ["*"]
           Resources: ["^kube-apiserver-minikube$"]
-          Description: "API server traffic encrypted by WireGuard at the network layer."
+          Description: "CC: tailscale-network-isolation. API server traffic encrypted by WireGuard at the network layer."

argocd/manifests/prowler/mutelist/control-plane.yaml

@@ -6,12 +6,12 @@ Mutelist:
         "controllermanager_disable_profiling":
           Regions: ["*"]
           Resources: ["^kube-controller-manager-minikube$"]
-          Description: "Profiling endpoint unreachable from public internet."
+          Description: "CC: tailscale-network-isolation. Profiling endpoint unreachable from public internet."
         "scheduler_profiling":
           Regions: ["*"]
           Resources: ["^kube-scheduler-minikube$"]
-          Description: "Profiling endpoint unreachable from public internet."
+          Description: "CC: tailscale-network-isolation. Profiling endpoint unreachable from public internet."
         "kubelet_tls_cert_and_key":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "Kubelet API not exposed outside node; minikube auto-generates certificates."
+          Description: "CC: tailscale-network-isolation, single-user-cluster. Kubelet API not exposed outside node; minikube auto-generates certificates."

argocd/manifests/prowler/mutelist/core-pod-security.yaml

@@ -17,8 +17,9 @@ Mutelist:
             - "^kindnet-"
             - "^storage-provisioner$"
           Description: >-
-            Control-plane and networking pods require hostNetwork by design.
-            Host network itself is only reachable via tailnet.
+            CC: tailscale-network-isolation. Control-plane and networking
+            pods require hostNetwork by design. Host network itself is
+            only reachable via tailnet.
         "core_minimize_privileged_containers":
           Regions: ["*"]
           Resources:
@@ -30,6 +31,7 @@ Mutelist:
             # Forgejo runner
             - "^forgejo-runner-"
           Description: >-
+            CC: single-user-cluster, operator-managed-pods, trusted-ci-only.
             kube-proxy: system pod, single-user cluster. ts-*/ingress-*:
             Tailscale operator-managed. forgejo-runner: DinD limited to
             trusted private forge repos.
@@ -47,24 +49,25 @@ Mutelist:
             - "^nameserver-"
             - "^ingress-"
           Description: >-
-            System pods managed by minikube and Tailscale operator;
-            seccomp profiles set by upstream. Single-user cluster limits
-            exploit surface.
+            CC: single-user-cluster, operator-managed-pods. System pods
+            managed by minikube and Tailscale operator; seccomp profiles
+            set by upstream. Single-user cluster limits exploit surface.
         "core_minimize_hostPID_containers":
           Regions: ["*"]
           Resources:
             - "^prowler-"
           Description: >-
-            Prowler CIS scanner requires hostPID for file permission
-            checks. Runs as CronJob with 7-day TTL, not a persistent
-            workload.
+            CC: ephemeral-privileged-jobs. Prowler CIS scanner requires
+            hostPID for file permission checks. Runs as CronJob with
+            7-day TTL, not a persistent workload.
         "core_minimize_root_containers_admission":
           Regions: ["*"]
           Resources:
             - "^grafana-"
           Description: >-
-            Root limited to init-chown-data container; all runtime
-            containers run as UID 472 with caps dropped.
+            CC: init-container-isolation. Root limited to init-chown-data
+            container; all runtime containers run as UID 472 with caps
+            dropped.
         "core_minimize_containers_added_capabilities":
           Regions: ["*"]
           Resources:
@@ -74,9 +77,10 @@ Mutelist:
             # Grafana init-chown-data
             - "^grafana-"
           Description: >-
-            System pods: capabilities required by function
-            (minikube-managed). Grafana: CHOWN limited to init phase;
-            runtime containers drop ALL.
+            CC: single-user-cluster, init-container-isolation. System
+            pods: capabilities required by function (minikube-managed).
+            Grafana: CHOWN limited to init phase; runtime containers
+            drop ALL.
         "core_minimize_containers_capabilities_assigned":
           Regions: ["*"]
           Resources:
@@ -84,4 +88,5 @@ Mutelist:
             - "^kindnet-"
             - "^grafana-"
           Description: >-
-            See core_minimize_containers_added_capabilities.
+            CC: single-user-cluster, init-container-isolation. See
+            core_minimize_containers_added_capabilities.

argocd/manifests/prowler/mutelist/manual-node-checks.yaml

@@ -1,7 +1,7 @@
 # Node-level and RBAC checks that Prowler reports as MANUAL because it
-# cannot evaluate them from inside a pod. Verified out-of-band by the
-# node-verification block in `mise run review-compliance-reports`, which
-# SSHes into the minikube node and checks each condition directly.
+# cannot evaluate them from inside a pod. Compensated by automated
+# verification in `mise run review-compliance-reports`, which SSHes into
+# the minikube node and checks each condition directly every week.
 Mutelist:
   Accounts:
     "*":
@@ -9,51 +9,51 @@ Mutelist:
         "etcd_unique_ca":
           Regions: ["*"]
           Resources: ["^etcd-minikube$"]
-          Description: "Etcd CA fingerprint verified different from cluster CA by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. Etcd CA fingerprint verified different from cluster CA by review-compliance-reports."
         "kubelet_conf_file_ownership":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "File ownership verified root:root by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. File ownership verified root:root by review-compliance-reports."
         "kubelet_conf_file_permissions":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "File permissions verified 600 by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. File permissions verified 600 by review-compliance-reports."
         "kubelet_config_yaml_ownership":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "File ownership verified root:root by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. File ownership verified root:root by review-compliance-reports."
         "kubelet_config_yaml_permissions":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "File permissions verified 644 by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. File permissions verified 644 by review-compliance-reports."
         "kubelet_service_file_ownership_root":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "File ownership verified root:root by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. File ownership verified root:root by review-compliance-reports."
         "kubelet_service_file_permissions":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "File permissions verified 644 by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. File permissions verified 644 by review-compliance-reports."
         "kubelet_disable_read_only_port":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "readOnlyPort absence (defaults to 0) verified by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. readOnlyPort absence (defaults to 0) verified by review-compliance-reports."
         "kubelet_event_record_qps":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "eventRecordQPS absence (defaults to 5) verified by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. eventRecordQPS absence (defaults to 5) verified by review-compliance-reports."
         "kubelet_manage_iptables":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "makeIPTablesUtilChains absence (defaults to true) verified by review-compliance-reports."
+          Description: "CC: node-config-automated-verification. makeIPTablesUtilChains absence (defaults to true) verified by review-compliance-reports."
         "kubelet_strong_ciphers_only":
           Regions: ["*"]
           Resources: ["^kubelet-config$"]
-          Description: "Go default ciphers used; all traffic WireGuard-encrypted via tailnet."
+          Description: "CC: node-config-automated-verification, tailscale-network-isolation. Go default ciphers used; all traffic WireGuard-encrypted via tailnet."
         "rbac_cluster_admin_usage":
           Regions: ["*"]
           Resources:
             - "^cluster-admin$"
             - "^kubeadm:cluster-admins$"
             - "^minikube-rbac$"
-          Description: "Only built-in/minikube cluster-admin bindings present; verified by review-compliance-reports."
+          Description: "CC: node-config-automated-verification, single-user-cluster. Only built-in/minikube cluster-admin bindings present; verified by review-compliance-reports."

argocd/manifests/prowler/mutelist/rbac.yaml

@@ -13,8 +13,9 @@ Mutelist:
             # ArgoCD
             - "^argocd-"
           Description: >-
-            Built-in K8s roles: only operator can bind them. ArgoCD:
-            requires broad access but is SSO-gated via Authentik OIDC.
+            CC: single-user-cluster, sso-gated-admin-tools. Built-in
+            K8s roles: only operator can bind them. ArgoCD: requires
+            broad access but is SSO-gated via Authentik OIDC.
         "rbac_minimize_pod_creation_access":
           Regions: ["*"]
           Resources:
@@ -25,12 +26,14 @@ Mutelist:
             # CloudNativePG operator
             - "^cnpg-manager$"
           Description: >-
-            Built-in K8s roles and CNPG operator. Only the operator can
-            assign these roles; no untrusted users have cluster access.
+            CC: single-user-cluster. Built-in K8s roles and CNPG
+            operator. Only the operator can assign these roles; no
+            untrusted users have cluster access.
         "rbac_minimize_service_account_token_creation":
           Regions: ["*"]
           Resources:
             - "^system:"
           Description: >-
-            kube-controller-manager requires token creation for SA
-            management. Only operator manages service accounts.
+            CC: single-user-cluster. kube-controller-manager requires
+            token creation for SA management. Only operator manages
+            service accounts.

argocd/manifests/prowler/mutelist/trivyignore.yaml

@@ -14,24 +14,26 @@ misconfigurations:
     paths:
       - "argocd/manifests/external-secrets/rbac.yaml"
     statement: >-
-      external-secrets-operator's entire function is to read and
-      synthesize Secret objects; ClusterRole over secrets is its
-      purpose. Both the controller and cert-controller are
+      CC: operator-purpose-bound-rbac. external-secrets-operator's entire
+      function is to read and synthesize Secret objects; ClusterRole over
+      secrets is its purpose. Both the controller and cert-controller are
       upstream-defined.
   - id: KSV-0041
     paths:
       - "argocd/manifests/kube-state-metrics/rbac.yaml"
       - "argocd/manifests/kube-state-metrics-ringtail/rbac.yaml"
     statement: >-
-      KSM exposes only Secret metadata (name, namespace, type, labels),
-      never the data field. list/watch on secrets is required for
-      kube_secret_info / kube_secret_labels metrics.
+      CC: kube-state-metrics-metadata-only. KSM exposes only Secret
+      metadata (name, namespace, type, labels), never the data field.
+      list/watch on secrets is required for kube_secret_info /
+      kube_secret_labels metrics.
   - id: KSV-0114
     paths:
       - "argocd/manifests/external-secrets/rbac.yaml"
     statement: >-
-      cert-controller manages the external-secrets validating webhook
-      configurations to inject its own rotating CA bundle. RBAC is
-      scoped to two named webhooks (secretstore-validate,
-      externalsecret-validate) via resourceNames; KSV-0114 doesn't see
-      the resourceNames restriction so reports the full ClusterRole.
+      CC: operator-purpose-bound-rbac. cert-controller manages the
+      external-secrets validating webhook configurations to inject its
+      own rotating CA bundle. RBAC is scoped to two named webhooks
+      (secretstore-validate, externalsecret-validate) via resourceNames;
+      KSV-0114 doesn't see the resourceNames restriction so reports the
+      full ClusterRole.

argocd/manifests/shower/kustomization.yaml

@@ -14,4 +14,4 @@ resources:
 
 images:
   - name: registry.ops.eblu.me/blumeops/shower
-    newTag: v1.1.3-3645098-nix
+    newTag: v1.0.2-039d9b9-nix

argocd/manifests/tailscale-operator-base/kustomization.yaml

@@ -6,11 +6,8 @@ namespace: tailscale
 
 # Upstream Tailscale operator manifest from forge mirror.
 # To upgrade: update the ref in the URL AND the newTag below.
-# Must use the tailnet host forge.ops.eblu.me — the public forge.eblu.me
-# black-holes /mirrors/ at the Fly edge (AI-scraper mitigation), which the
-# in-cluster ArgoCD repo-server would otherwise hit and fail with a 403.
 resources:
-  - https://forge.ops.eblu.me/mirrors/tailscale/raw/tag/v1.94.2/cmd/k8s-operator/deploy/manifests/operator.yaml
+  - https://forge.eblu.me/mirrors/tailscale/raw/tag/v1.94.2/cmd/k8s-operator/deploy/manifests/operator.yaml
   - proxyclass.yaml
   - dnsconfig.yaml
 

argocd/manifests/teslamate/README.md

@@ -0,0 +1,69 @@
+# TeslaMate
+
+TeslaMate is a self-hosted Tesla data logger that collects and visualizes vehicle data.
+
+## Prerequisites
+
+### 1. Create 1Password Secrets
+
+Create two items in the blumeops 1Password vault:
+
+1. **TeslaMate DB Password**
+   - Generate a secure password for the teslamate PostgreSQL user
+   - Add a field named `password` with the generated value
+
+2. **TeslaMate Encryption Key**
+   - Generate with: `openssl rand -base64 32`
+   - Add a field named `key` with the generated value
+   - This encrypts Tesla API tokens at rest in the database
+
+### 2. Apply Kubernetes Secrets
+
+```bash
+# Create namespace
+kubectl create namespace teslamate
+
+# Apply database user secret (for CNPG)
+op inject -i argocd/manifests/databases/secret-teslamate.yaml.tpl | kubectl apply -f -
+
+# Apply teslamate secrets
+op inject -i argocd/manifests/teslamate/secret-encryption-key.yaml.tpl | kubectl apply -f -
+op inject -i argocd/manifests/teslamate/secret-db.yaml.tpl | kubectl apply -f -
+```
+
+### 3. Create Database
+
+After the teslamate user exists in PostgreSQL (sync blumeops-pg first):
+
+```bash
+PGPASSWORD=$(op read "op://blumeops/postgres/password") \
+  psql -h pg.ops.eblu.me -U eblume -c "CREATE DATABASE teslamate OWNER teslamate;"
+```
+
+## Deployment
+
+```bash
+# Sync ArgoCD apps
+argocd app sync apps
+argocd app sync blumeops-pg teslamate grafana grafana-config
+```
+
+## Tesla API Setup
+
+1. Access TeslaMate UI at https://tesla.tail8d86e.ts.net
+2. Click "Sign in with Tesla"
+3. Complete OAuth flow in browser
+4. Tokens are encrypted and stored in database
+5. Verify vehicle appears and data collection starts
+
+## Grafana Dashboards
+
+TeslaMate dashboards are available in Grafana at https://grafana.tail8d86e.ts.net
+
+They use the "TeslaMate" PostgreSQL datasource (not Prometheus).
+
+## Notes
+
+- MQTT is disabled (can be enabled later for Home Assistant integration)
+- Timezone is set to America/Los_Angeles
+- Encryption key protects Tesla API tokens at rest

argocd/manifests/teslamate/deployment.yaml

@@ -1,10 +1,3 @@
-# TeslaMate on ringtail k3s — Nix image.
-#
-# The Nix image's Entrypoint waits for postgres, runs migrations
-# (TeslaMate.Release.migrate), then starts the release — so no command
-# override is needed. Stateless; all data lives in the teslamate database
-# on the ringtail blumeops-pg (DATABASE_HOST already an in-cluster name,
-# unchanged from minikube). See [[migrate-wave1-ringtail]].
 apiVersion: apps/v1
 kind: Deployment
 metadata:

argocd/manifests/teslamate/kustomization.yaml

@@ -12,4 +12,4 @@ resources:
 
 images:
   - name: registry.ops.eblu.me/blumeops/teslamate
-    newTag: v3.0.0-fcac8e5-nix
+    newTag: v3.0.0-08c698e

argocd/manifests/unpoller/kustomization.yaml

@@ -10,7 +10,7 @@ resources:
 
 images:
   - name: registry.ops.eblu.me/blumeops/unpoller
-    newTag: v3.2.0-4d1f4af
+    newTag: v2.34.0-613f05d
 
 configMapGenerator:
   - name: unpoller-config

compensating-controls.yaml

@@ -0,0 +1,206 @@
+# Compensating Controls
+#
+# Documents controls that mitigate risks from suppressed or accepted security
+# findings. Referenced by security tools (Prowler mutelist, Kingfisher config,
+# etc.) via "CC: <id>" in finding descriptions or suppression notes.
+#
+# Used by `mise run review-compensating-controls` to surface stale controls.
+#
+# Fields:
+#   id              - kebab-case unique identifier, referenced from tool configs
+#   description     - what the control actually does to mitigate risk
+#   created         - date (YYYY-MM-DD) the control was documented
+#   last-reviewed   - date (YYYY-MM-DD) or null
+#   notes           - optional context
+
+controls:
+  - id: single-user-cluster
+    description: >-
+      Only the cluster operator (eblume) has kubectl access. No untrusted
+      users can create pods, access cached images, or bind RBAC roles.
+    created: 2026-03-30
+    last-reviewed: 2026-04-01
+    notes: >-
+      Verify by checking kubeconfig distribution and Tailscale ACLs.
+      If additional users gain cluster access, re-evaluate all findings
+      muted under this control.
+
+  - id: tailscale-network-isolation
+    description: >-
+      Cluster is not internet-exposed. All access requires Tailscale
+      identity with ACL enforcement. Profiling endpoints, debug ports,
+      and control-plane APIs are unreachable from the public internet.
+    created: 2026-03-30
+    last-reviewed: 2026-04-06
+    notes: >-
+      Verify with 'tailscale serve status --json' on indri and review
+      Tailscale ACLs in pulumi/tailscale/. Only tag:flyio-target services
+      are publicly routable.
+
+  - id: local-registry
+    description: >-
+      Operator-built services use a private zot registry
+      (registry.ops.eblu.me) for supply-chain control. Remaining
+      images are pulled from public registries without stored
+      credentials. No shared registry secrets are cached on cluster
+      nodes.
+    created: 2026-03-30
+    last-reviewed: 2026-04-12
+    notes: >-
+      Verify by checking image prefixes in kustomization.yaml files.
+      Known external-image categories: (1) upstream apps not yet
+      mirrored — immich, ollama, frigate, frigate-notify, valkey;
+      (2) infrastructure components — tailscale operator/proxy,
+      external-secrets, 1password-connect, forgejo-runner, docker
+      DinD, nvidia-device-plugin; (3) utility base images — busybox,
+      alpine (grafana init containers). Track upstream versions in
+      service-versions.yaml. Goal is to progressively mirror these
+      into zot.
+
+  - id: sso-gated-admin-tools
+    description: >-
+      ArgoCD requires SSO authentication via Authentik OIDC. Wildcard
+      RBAC roles are mitigated by requiring authenticated identity
+      before any API access.
+    created: 2026-03-30
+    last-reviewed: 2026-04-14
+    notes: >-
+      Verify Authentik OIDC provider config for ArgoCD and that
+      anonymous access is disabled. Check ArgoCD --auth-token isn't
+      leaked. The workflow-bot API key account is scoped to sync/get
+      only.
+
+  - id: operator-managed-pods
+    description: >-
+      Tailscale operator manages proxy pod specs (ts-*, ingress-*,
+      operator-*, nameserver-*). Pod security settings are set by the
+      operator, not user manifests. Operator is tracked in
+      service-versions.yaml and regularly updated.
+    created: 2026-03-30
+    last-reviewed: 2026-04-21
+    notes: >-
+      Verify operator version is current via 'mise run service-review'.
+      Check Tailscale changelog for security fixes. If operator adds
+      seccomp support, remove these mutes. As of 2026-04-21: still no
+      default seccomp on operator-generated pods (upstream issue #7359
+      open). A ProxyClass + generic device plugin can downgrade proxies
+      from privileged to NET_ADMIN+NET_RAW and set seccompProfile —
+      potential future remediation to remove the seccomp mute without
+      waiting for upstream defaults.
+
+  - id: ephemeral-privileged-jobs
+    description: >-
+      Prowler CIS scanner runs as a CronJob with 7-day TTL
+      auto-deletion, not as a persistent privileged workload. hostPID
+      exposure is time-bounded to scan duration (~20s).
+    created: 2026-03-30
+    last-reviewed: 2026-04-29
+    notes: >-
+      Verify TTL is set in cronjob.yaml. Check that no persistent
+      pods run with hostPID on the scanned cluster (indri). The
+      alloy-tracing DaemonSet on ringtail also uses hostPID but is
+      out of scope — Prowler only scans indri. Tracked in Todoist:
+      "prowler scan against ringtail" — once that lands, the
+      DaemonSet's hostPID+privileged posture will surface as a CIS
+      finding and need its own CC or remediation.
+
+  - id: trusted-ci-only
+    description: >-
+      Forgejo runner only executes workflows from repos on the private
+      forge (forge.ops.eblu.me). No external or untrusted repos can
+      trigger privileged CI jobs.
+    created: 2026-03-30
+    last-reviewed: 2026-05-01
+    notes: >-
+      Verification: (1) Runner config (argocd/manifests/forgejo-runner/
+      config.yaml) connects only to https://forge.ops.eblu.me/. (2) Forge
+      app.ini has DISABLE_REGISTRATION=true and ALLOW_ONLY_EXTERNAL_REGISTRATION
+      =true (ansible/roles/forgejo/defaults/main.yml) — no untrusted users
+      can sign up or create repos. The runner registers at instance scope
+      (repo_id=0/owner_id=0 in action_runner table), but the instance itself
+      is closed, so no per-repo allow-list is needed. Re-evaluate if the
+      forge ever opens to additional users or if the runner is repointed
+      to an external forge.
+
+  - id: init-container-isolation
+    description: >-
+      Root privileges and added capabilities (CHOWN) are limited to
+      init containers that run once at pod startup. All runtime
+      containers run as non-root (UID 472) with all capabilities
+      dropped.
+    created: 2026-03-30
+    last-reviewed: 2026-05-04
+    notes: >-
+      Verify by inspecting grafana deployment.yaml securityContext
+      for both init and runtime containers. If fsGroup alone can
+      handle PVC ownership, remove init-chown-data and this control.
+      Retirement deferred until grafana lands on ringtail's k3s
+      (see [[indri-k8s-migration]]) — storage backend will change,
+      and removing init-chown-data right before that migration
+      trades a real safety net for marginal cleanup. Revisit
+      post-migration.
+
+  - id: node-config-automated-verification
+    description: >-
+      Prowler reports certain node-level checks as MANUAL because it runs
+      inside a pod and cannot evaluate kubelet file permissions, kubelet
+      config arguments, etcd CA separation, or cluster-admin RBAC bindings.
+      The review-compliance-reports script SSHes into the minikube node
+      weekly and programmatically verifies each condition, failing loudly
+      if any check deviates from expected values.
+    created: 2026-04-14
+    last-reviewed: 2026-04-14
+    notes: >-
+      Verification runs as part of 'mise run review-compliance-reports'.
+      If minikube node is unreachable, all checks report as FAIL. If new
+      MANUAL findings appear in Prowler, add corresponding verification
+      logic to the script and update the mutelist.
+
+  - id: operator-purpose-bound-rbac
+    description: >-
+      Operators whose entire function is to manage a sensitive resource
+      legitimately need RBAC over that resource. external-secrets-operator
+      manages Secret objects (its purpose) and the cert-controller mutates
+      its own ValidatingWebhookConfigurations to inject rotating CA bundles.
+      Risk is bounded by: (1) the operator code being upstream open-source
+      and reviewed; (2) RBAC scoped to specific named webhooks where
+      possible; (3) supply chain controls on the operator image (mirrored
+      to local registry, version tracked in service-versions.yaml).
+    created: 2026-04-27
+    last-reviewed: 2026-04-27
+    notes: >-
+      Verify by checking that the operators in question still match their
+      stated purpose (i.e. external-secrets is still the only consumer of
+      these ClusterRoles) and that upstream hasn't published advisories
+      for credential-handling bugs. Re-evaluate if a non-secrets-managing
+      ClusterRole appears under this control.
+
+  - id: kube-state-metrics-metadata-only
+    description: >-
+      kube-state-metrics holds list/watch on Secrets cluster-wide but only
+      exposes Secret object *metadata* (name, namespace, type, creation
+      timestamp, labels) via the kube_secret_info / kube_secret_labels
+      metrics. Secret data fields are never read into KSM's exposed
+      metrics by upstream design. Mitigation rests on KSM's metric
+      schema, the version pin in service-versions.yaml, and the metrics
+      endpoint being reachable only on the cluster network.
+    created: 2026-04-27
+    last-reviewed: 2026-04-27
+    notes: >-
+      Verify by inspecting the /metrics endpoint output for any series
+      that include secret data (only *_info and *_labels metrics should
+      reference secrets, and labels should be limited to user-applied
+      labels — never the data:). Re-evaluate on KSM version bumps.
+
+  - id: observability-stack-audit
+    description: >-
+      Alloy collects pod logs and ships them to Loki, providing an
+      audit trail for cluster activity. Compensates for missing
+      apiserver audit logging which minikube does not configure.
+    created: 2026-03-30
+    last-reviewed: 2026-03-30
+    notes: >-
+      Verify Alloy DaemonSet is running and Loki is receiving logs.
+      Note this is weaker than native apiserver audit logs — it
+      captures pod stdout/stderr, not API request-level auditing.
+      Consider enabling minikube audit logging if supported.

containers/external-secrets/container.py

@@ -1,51 +0,0 @@
-"""External Secrets Operator — native Dagger build.
-
-Two-stage build: Go binary (all providers), Alpine runtime.
-Source cloned from forge mirror.
-
-A single binary serves as the controller, webhook, and cert-controller; the
-Deployments select the role via a subcommand passed in `args:`, so the image
-ENTRYPOINT must be the binary itself (matching upstream's distroless image).
-"""
-
-import dagger
-
-from blumeops.containers import (
-    alpine_runtime,
-    clone_from_forge,
-    go_build,
-    oci_labels,
-)
-
-VERSION = "v2.2.0"
-
-
-async def build(src: dagger.Directory) -> dagger.Container:
-    source = clone_from_forge("external-secrets", VERSION)
-
-    # Upstream `make build` compiles every secret provider into a single
-    # static binary (`-tags all_providers`, CGO disabled). Mirror that so the
-    # local image is functionally identical to ghcr.io/.../external-secrets.
-    backend = go_build(
-        source,
-        "/external-secrets",
-        tags="all_providers",
-    )
-
-    runtime = alpine_runtime(
-        extra_apk=["ca-certificates"],
-        create_user=False,
-    )
-    runtime = oci_labels(
-        runtime,
-        title="External Secrets Operator",
-        description=(
-            "Kubernetes operator that integrates external secret management systems"
-        ),
-        version=VERSION,
-    )
-    return (
-        runtime.with_file("/bin/external-secrets", backend.file("/external-secrets"))
-        .with_user("65534")
-        .with_entrypoint(["/bin/external-secrets"])
-    )

containers/external-secrets/default.nix

@@ -1,56 +0,0 @@
-# Nix-built External Secrets Operator (amd64, for ringtail k3s).
-# Builds v2.2.0 from the forge mirror with all secret providers compiled in,
-# faithful to upstream's `make build` (-tags all_providers). The container.py
-# sibling builds the arm64 image for indri's minikube; this default.nix builds
-# the amd64 image on ringtail's nix-container-builder.
-{ pkgs ? import <nixpkgs> { } }:
-
-let
-  version = "2.2.0";
-
-  src = pkgs.fetchgit {
-    url = "https://forge.ops.eblu.me/mirrors/external-secrets.git";
-    rev = "v${version}";
-    hash = "sha256-eAocOAp5s4CFRrpKfQr2lf3Ji+6nQQ1A5/eTw5B7v9U=";
-  };
-
-  # external-secrets v2.2.0 requires Go >= 1.26.1; nixpkgs default go is 1.25.x.
-  external-secrets = (pkgs.buildGoModule.override { go = pkgs.go_1_26; }) {
-    inherit src version;
-    pname = "external-secrets";
-    vendorHash = "sha256-0xuBK3fjAplPLAElHvKB6d+2lDz+De/s91fV4dPZwjE=";
-
-    doCheck = false;
-
-    subPackages = [ "." ];
-
-    tags = [ "all_providers" ];
-
-    ldflags = [ "-s" "-w" ];
-
-    meta = with pkgs.lib; {
-      description = "Kubernetes operator that integrates external secret management systems";
-      homepage = "https://github.com/external-secrets/external-secrets";
-      license = licenses.asl20;
-      mainProgram = "external-secrets";
-    };
-  };
-in
-
-pkgs.dockerTools.buildLayeredImage {
-  name = "blumeops/external-secrets";
-  contents = [
-    external-secrets
-    pkgs.cacert
-    pkgs.tzdata
-  ];
-
-  config = {
-    Entrypoint = [ "${external-secrets}/bin/external-secrets" ];
-    Env = [
-      "SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt"
-      "TZDIR=${pkgs.tzdata}/share/zoneinfo"
-    ];
-    User = "65534";
-  };
-}

containers/mealie/Dockerfile

@@ -0,0 +1,145 @@
+# Mealie — self-hosted recipe manager
+# Built from source via forge mirror of mealie-recipes/mealie
+# Based on upstream docker/Dockerfile (multi-stage: Node frontend + Python backend)
+
+ARG CONTAINER_APP_VERSION=v3.12.0
+
+###############################################
+# Frontend Build
+###############################################
+FROM node:24-slim AS frontend-builder
+
+ARG CONTAINER_APP_VERSION
+RUN apt-get update && apt-get install --no-install-recommends -y git ca-certificates && rm -rf /var/lib/apt/lists/*
+
+RUN git clone --depth 1 --branch ${CONTAINER_APP_VERSION} \
+    https://forge.ops.eblu.me/mirrors/mealie.git /src
+
+WORKDIR /src/frontend
+
+RUN yarn install \
+    --prefer-offline \
+    --frozen-lockfile \
+    --non-interactive \
+    --production=false \
+    --network-timeout 1000000
+
+RUN yarn generate
+
+###############################################
+# Python Base
+###############################################
+FROM python:3.12-slim AS python-base
+
+ENV MEALIE_HOME="/app"
+ENV PYTHONUNBUFFERED=1 \
+    PYTHONDONTWRITEBYTECODE=1 \
+    PIP_NO_CACHE_DIR=off \
+    PIP_DISABLE_PIP_VERSION_CHECK=on \
+    PIP_DEFAULT_TIMEOUT=100 \
+    VENV_PATH="/opt/mealie"
+
+ENV PATH="$VENV_PATH/bin:$PATH"
+
+RUN useradd -u 911 -U -d $MEALIE_HOME -s /bin/bash abc \
+    && usermod -G users abc \
+    && mkdir $MEALIE_HOME
+
+###############################################
+# Backend Package Build
+###############################################
+FROM python-base AS backend-builder
+
+ARG CONTAINER_APP_VERSION
+RUN apt-get update \
+    && apt-get install --no-install-recommends -y curl git ca-certificates \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN pip install uv
+
+RUN git clone --depth 1 --branch ${CONTAINER_APP_VERSION} \
+    https://forge.ops.eblu.me/mirrors/mealie.git /src
+
+WORKDIR /src
+
+COPY --from=frontend-builder /src/frontend/dist ./mealie/frontend
+
+RUN uv build --out-dir dist
+
+RUN uv export --no-editable --no-emit-project --extra pgsql --format requirements-txt --output-file dist/requirements.txt \
+    && MEALIE_VERSION=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml', 'rb'))['project']['version'])") \
+    && echo "mealie[pgsql]==${MEALIE_VERSION} \\" >> dist/requirements.txt \
+    && pip hash dist/mealie-${MEALIE_VERSION}-py3-none-any.whl | tail -n1 | tr -d '\n' >> dist/requirements.txt \
+    && echo " \\" >> dist/requirements.txt \
+    && pip hash dist/mealie-${MEALIE_VERSION}.tar.gz | tail -n1 >> dist/requirements.txt
+
+###############################################
+# Python Venv Build
+###############################################
+FROM python-base AS venv-builder
+
+RUN apt-get update \
+    && apt-get install --no-install-recommends -y \
+    build-essential \
+    libpq-dev \
+    libwebp-dev \
+    ffmpeg \
+    libsasl2-dev libldap2-dev libssl-dev \
+    gnupg gnupg2 gnupg1 \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN python3 -m venv --upgrade-deps $VENV_PATH
+
+COPY --from=backend-builder /src/dist /dist
+
+RUN . $VENV_PATH/bin/activate \
+    && pip install --require-hashes -r /dist/requirements.txt --find-links /dist
+
+###############################################
+# Production Image
+###############################################
+FROM python-base AS production
+
+ENV PRODUCTION=true
+ENV TESTING=false
+
+RUN apt-get update \
+    && apt-get install --no-install-recommends -y \
+    curl \
+    ffmpeg \
+    gosu \
+    iproute2 \
+    libldap-common \
+    libldap2 \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN mkdir -p /run/secrets
+
+COPY --from=venv-builder $VENV_PATH $VENV_PATH
+
+ENV NLTK_DATA="/nltk_data/"
+RUN mkdir -p $NLTK_DATA
+RUN python -m nltk.downloader -d $NLTK_DATA averaged_perceptron_tagger_eng
+
+VOLUME ["$MEALIE_HOME/data/"]
+ENV APP_PORT=9000
+
+EXPOSE ${APP_PORT}
+
+COPY --from=backend-builder /src/docker/healthcheck.sh $MEALIE_HOME/healthcheck.sh
+RUN chmod +x $MEALIE_HOME/healthcheck.sh
+HEALTHCHECK CMD $MEALIE_HOME/healthcheck.sh
+
+ENV HOST=0.0.0.0
+
+COPY --from=backend-builder /src/docker/entry.sh $MEALIE_HOME/run.sh
+RUN chmod +x $MEALIE_HOME/run.sh
+
+ARG CONTAINER_APP_VERSION
+LABEL org.opencontainers.image.title="Mealie"
+LABEL org.opencontainers.image.description="Self-hosted recipe manager"
+LABEL org.opencontainers.image.version="${CONTAINER_APP_VERSION}"
+LABEL org.opencontainers.image.source="https://forge.eblu.me/eblume/blumeops"
+LABEL org.opencontainers.image.vendor="blumeops"
+
+ENTRYPOINT ["/app/run.sh"]

containers/mealie/default.nix

@@ -1,69 +0,0 @@
-# Nix-built Mealie for ringtail (amd64).
-#
-# Replaces the from-source Dockerfile build (Node frontend + Python venv)
-# with nixpkgs' mealie, which ships a single `mealie` gunicorn entrypoint
-# serving the prebuilt frontend + backend — so this is a clean single-
-# process wrap (unlike paperless, which is multi-process).
-#
-# Mealie stores its DB as SQLite under DATA_DIR (the mealie-data PVC at
-# /app/data); there is no postgres. The run wrapper mirrors the nixpkgs
-# mealie NixOS module: run `libexec/init_db` (Alembic migrations) first,
-# then exec gunicorn.
-#
-# Self-pins nixos-unstable: stable nixpkgs lags at 3.9.2, unstable carries
-# 3.16.0. This is a forward 4-minor bump from the v3.12.0 Dockerfile build
-# (the deferred upgrade) — mealie auto-migrates the SQLite DB forward on
-# startup via init_db; the source PVC is retained for rollback. The version
-# assertion makes nix-build fail if a pin bump changes the version.
-let
-  nixpkgs = fetchTarball {
-    url = "https://github.com/NixOS/nixpkgs/archive/331800de5053fcebacf6813adb5db9c9dca22a0c.tar.gz";
-    sha256 = "1p54fm6dkbq62kpi55cr4wyx7b1nsajpsnjgs64cmp073fwi15f7";
-  };
-  pkgs = import nixpkgs { system = "x86_64-linux"; };
-
-  version = "3.16.0";
-
-  app = pkgs.mealie;
-
-  # Mirror the NixOS module's mealie service: init_db (Alembic) then
-  # gunicorn bound to the app port. DATA_DIR/env come from the image +
-  # k8s manifest.
-  mealie-run = pkgs.writeShellScriptBin "mealie-run" ''
-    set -e
-    ${app}/libexec/init_db
-    exec ${pkgs.lib.getExe app} -b 0.0.0.0:9000
-  '';
-in
-
-assert app.version == version;
-
-pkgs.dockerTools.buildLayeredImage {
-  name = "blumeops/mealie";
-
-  contents = [
-    app
-    mealie-run
-    pkgs.bashInteractive
-    pkgs.coreutils
-    pkgs.cacert
-    pkgs.tzdata
-    # python3 (stdlib sqlite3) for the borgmatic k8s-sqlite-dump helper,
-    # which runs `python3 -c "...sqlite3...backup..."` inside the pod.
-    # Same nixpkgs python mealie is built against, so ~no added closure.
-    pkgs.python3
-  ];
-
-  config = {
-    Cmd = [ "${mealie-run}/bin/mealie-run" ];
-    Env = [
-      "DATA_DIR=/app/data"
-      "SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt"
-      "PYTHONUNBUFFERED=1"
-      "PRODUCTION=true"
-    ];
-    ExposedPorts = {
-      "9000/tcp" = { };
-    };
-  };
-}

containers/paperless/Dockerfile

@@ -0,0 +1,156 @@
+# syntax=docker/dockerfile:1
+# Paperless-ngx — self-hosted document management
+# Built from source via forge mirror of paperless-ngx/paperless-ngx
+# Closely follows upstream Dockerfile structure with git clone instead of COPY
+
+ARG CONTAINER_APP_VERSION=v2.20.13
+
+###############################################
+# Stage 1: Clone source (reused by later stages)
+###############################################
+FROM docker.io/library/alpine:3.22 AS source
+
+ARG CONTAINER_APP_VERSION
+RUN apk add --no-cache git
+RUN git clone --depth 1 --branch ${CONTAINER_APP_VERSION} \
+    https://forge.ops.eblu.me/mirrors/paperless-ngx.git /src
+
+###############################################
+# Stage 2: Compile frontend
+###############################################
+FROM --platform=$BUILDPLATFORM docker.io/node:20-trixie-slim AS compile-frontend
+
+COPY --from=source /src/src-ui /src/src-ui
+WORKDIR /src/src-ui
+
+RUN set -eux \
+    && npm update -g pnpm \
+    && npm install -g corepack@latest \
+    && corepack enable \
+    && pnpm install
+
+RUN set -eux \
+    && ./node_modules/.bin/ng build --configuration production
+
+###############################################
+# Stage 3: s6-overlay base
+###############################################
+FROM ghcr.io/astral-sh/uv:0.9.15-python3.12-trixie-slim AS s6-overlay-base
+
+WORKDIR /usr/src/s6
+
+ENV S6_BEHAVIOUR_IF_STAGE2_FAILS=2 \
+    S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0 \
+    S6_VERBOSITY=1 \
+    PATH=/command:$PATH
+
+ARG TARGETARCH
+ARG TARGETVARIANT
+ARG S6_OVERLAY_VERSION=3.2.1.0
+
+RUN set -eux \
+    && apt-get update \
+    && apt-get install --yes --quiet --no-install-recommends curl xz-utils \
+    && S6_ARCH="" \
+    && if [ "${TARGETARCH}${TARGETVARIANT}" = "amd64" ]; then S6_ARCH="x86_64"; \
+    elif [ "${TARGETARCH}${TARGETVARIANT}" = "arm64" ]; then S6_ARCH="aarch64"; fi \
+    && if [ -z "${S6_ARCH}" ]; then echo "Error: Cannot determine arch"; exit 1; fi \
+    && curl --fail --silent --show-error --location --remote-name-all --parallel \
+        "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz" \
+        "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz.sha256" \
+        "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${S6_ARCH}.tar.xz" \
+        "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${S6_ARCH}.tar.xz.sha256" \
+    && sha256sum --check ./*.sha256 \
+    && tar --directory / -Jxpf s6-overlay-noarch.tar.xz \
+    && tar --directory / -Jxpf s6-overlay-${S6_ARCH}.tar.xz \
+    && rm ./*.tar.xz ./*.sha256 \
+    && apt-get --yes purge curl xz-utils \
+    && apt-get --yes autoremove --purge \
+    && rm -rf /var/lib/apt/lists/*
+
+# Copy rootfs (s6 service definitions, init scripts)
+COPY --from=source /src/docker/rootfs /
+
+###############################################
+# Stage 4: Main application
+###############################################
+FROM s6-overlay-base AS main-app
+
+ARG CONTAINER_APP_VERSION
+ARG DEBIAN_FRONTEND=noninteractive
+ARG TARGETARCH
+ARG JBIG2ENC_VERSION=0.30
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    PYTHONWARNINGS="ignore:::django.http.response:517" \
+    PNGX_CONTAINERIZED=1 \
+    UV_LINK_MODE=copy \
+    UV_CACHE_DIR=/cache/uv/
+
+# Runtime packages
+RUN set -eux \
+    && apt-get update \
+    && apt-get install --yes --quiet --no-install-recommends \
+        curl gosu tzdata fonts-liberation gettext ghostscript gnupg \
+        icc-profiles-free imagemagick postgresql-client \
+        tesseract-ocr tesseract-ocr-eng tesseract-ocr-deu tesseract-ocr-fra \
+        tesseract-ocr-ita tesseract-ocr-spa unpaper pngquant jbig2dec \
+        libxml2 libxslt1.1 qpdf file libmagic1 media-types zlib1g \
+        libzbar0 poppler-utils \
+    && curl --fail --silent --show-error --location --remote-name-all \
+        "https://github.com/paperless-ngx/builder/releases/download/jbig2enc-trixie-v${JBIG2ENC_VERSION}/jbig2enc_${JBIG2ENC_VERSION}-1_${TARGETARCH}.deb" \
+    && dpkg --install ./jbig2enc_${JBIG2ENC_VERSION}-1_${TARGETARCH}.deb \
+    && cp /etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml \
+    && rm --force *.deb \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /usr/src/paperless/src/
+
+# Python dependencies
+COPY --from=source /src/pyproject.toml /src/uv.lock /usr/src/paperless/src/
+
+RUN --mount=type=cache,target=${UV_CACHE_DIR},id=python-cache \
+    set -eux \
+    && apt-get update \
+    && apt-get install --yes --quiet --no-install-recommends \
+        build-essential default-libmysqlclient-dev pkg-config \
+    && uv export --quiet --no-dev --all-extras --format requirements-txt --output-file requirements.txt \
+    && uv pip install --system --no-python-downloads --python-preference system --requirements requirements.txt \
+    && python3 -W ignore::RuntimeWarning -m nltk.downloader -d "/usr/share/nltk_data" snowball_data \
+    && python3 -W ignore::RuntimeWarning -m nltk.downloader -d "/usr/share/nltk_data" stopwords \
+    && python3 -W ignore::RuntimeWarning -m nltk.downloader -d "/usr/share/nltk_data" punkt_tab \
+    && apt-get --yes purge build-essential default-libmysqlclient-dev pkg-config \
+    && apt-get --yes autoremove --purge \
+    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+
+# Copy backend source
+COPY --from=source /src/src ./
+
+# Copy compiled frontend
+COPY --from=compile-frontend /src/src/documents/static/frontend/ ./documents/static/frontend/
+
+# Create user and finalize
+RUN set -eux \
+    && addgroup --gid 1000 paperless \
+    && useradd --uid 1000 --gid paperless --home-dir /usr/src/paperless paperless \
+    && mkdir -p /usr/src/paperless/data /usr/src/paperless/media \
+                /usr/src/paperless/consume /usr/src/paperless/export \
+    && chown -R paperless:paperless /usr/src/paperless \
+    && s6-setuidgid paperless python3 manage.py collectstatic --clear --no-input --link \
+    && s6-setuidgid paperless python3 manage.py compilemessages
+
+VOLUME ["/usr/src/paperless/data", "/usr/src/paperless/media", \
+        "/usr/src/paperless/consume", "/usr/src/paperless/export"]
+
+ENTRYPOINT ["/init"]
+EXPOSE 8000
+
+HEALTHCHECK --interval=30s --timeout=10s --retries=5 \
+    CMD [ "curl", "-fs", "-S", "-L", "--max-time", "2", "http://localhost:8000" ]
+
+LABEL org.opencontainers.image.title="Paperless-ngx"
+LABEL org.opencontainers.image.description="Self-hosted document management system"
+LABEL org.opencontainers.image.version="${CONTAINER_APP_VERSION}"
+LABEL org.opencontainers.image.source="https://forge.eblu.me/eblume/blumeops"
+LABEL org.opencontainers.image.vendor="blumeops"

containers/paperless/default.nix

@@ -1,77 +0,0 @@
-# Nix-built Paperless-ngx for ringtail (amd64).
-#
-# Replaces the from-source Dockerfile build (s6-overlay) with nixpkgs'
-# paperless-ngx, which already bundles the full OCR/imaging closure
-# (tesseract, ghostscript, imagemagick, qpdf, poppler, jbig2enc) and the
-# NLTK data via wrappers — so the image stays lean.
-#
-# Unlike the upstream s6 image, this image does NOT run all processes
-# itself. Paperless is multi-process; on ringtail it runs as four
-# containers sharing this one image, each with a different command:
-#   web      -> paperless-web        (granian, the wrapper below)
-#   worker   -> celery --app paperless worker
-#   beat     -> celery --app paperless beat
-#   consumer -> paperless-ngx document_consumer
-# plus a redis/valkey sidecar. The PYTHONPATH/granian invocation mirrors
-# the nixpkgs paperless NixOS module's paperless-web service exactly.
-#
-# Self-pins nixos-unstable: stable nixpkgs lags at 2.19.6, while unstable
-# carries 2.20.15 — a same-minor forward patch bump from the previous
-# Dockerfile build (v2.20.13). The version assertion makes nix-build fail
-# if a pin bump changes the version, forcing an explicit acknowledgment
-# here and in service-versions.yaml (enforced by container-version-check).
-let
-  nixpkgs = fetchTarball {
-    url = "https://github.com/NixOS/nixpkgs/archive/331800de5053fcebacf6813adb5db9c9dca22a0c.tar.gz";
-    sha256 = "1p54fm6dkbq62kpi55cr4wyx7b1nsajpsnjgs64cmp073fwi15f7";
-  };
-  pkgs = import nixpkgs { system = "x86_64-linux"; };
-
-  version = "2.20.15";
-
-  app = pkgs.paperless-ngx;
-
-  # Mirror the NixOS module's paperless-web service: granian serving the
-  # ASGI app with the package's propagated deps + src on PYTHONPATH.
-  pythonPath =
-    "${app.python.pkgs.makePythonPath app.propagatedBuildInputs}:${app}/lib/paperless-ngx/src";
-
-  paperless-web = pkgs.writeShellScriptBin "paperless-web" ''
-    export PYTHONPATH="${pythonPath}"
-    export PAPERLESS_NLTK_DIR="${app.nltkDataDir}"
-    exec ${app.python.pkgs.granian}/bin/granian \
-      --interface asginl --ws \
-      --host 0.0.0.0 --port 8000 \
-      "paperless.asgi:application"
-  '';
-in
-
-assert app.version == version;
-
-pkgs.dockerTools.buildLayeredImage {
-  name = "blumeops/paperless";
-
-  contents = [
-    app
-    paperless-web
-    pkgs.bashInteractive
-    pkgs.coreutils
-    pkgs.cacert
-    pkgs.tzdata
-  ];
-
-  config = {
-    # Default command is the web server; worker/beat/consumer containers
-    # override `command` in their k8s manifests.
-    Cmd = [ "${paperless-web}/bin/paperless-web" ];
-    Env = [
-      "PAPERLESS_NLTK_DIR=${app.nltkDataDir}"
-      "SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt"
-      "PYTHONUNBUFFERED=1"
-      "PNGX_CONTAINERIZED=1"
-    ];
-    ExposedPorts = {
-      "8000/tcp" = { };
-    };
-  };
-}