C2(migrate-immich-to-ringtail): plan capture two discovered concerns

1. Registering new ArgoCD apps from a feature branch: the app-of-apps
   "apps" Application is self-managing (re-reads apps.yaml on every
   sync, which pins targetRevision: main). So setting its revision to
   a branch doesn't stick across syncs, and new app definitions on a
   branch are invisible to the cluster via the normal flow. The goal
   card now documents the kubectl-apply + per-new-app `argocd app set
   --revision <branch>` workaround.

2. Tailscale device-name collision on cutover. The minikube immich
   ingress claims tailnet hostname "photos" (tls.hosts: [photos]).
   The ringtail ingress can't claim the same name while minikube's is
   alive (Tailscale enforces uniqueness). Staging uses
   tls.hosts: [photos-ringtail], with the rename to "photos" baked
   into immich-cutover-and-decommission step 2 + step 5.

Card dependency graph unchanged; no new cards.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Erich Blume 2026-05-13 12:21:57 -07:00
commit db37e7cc3e
3 changed files with 76 additions and 8 deletions

View file

@ -39,6 +39,14 @@ in [[immich-cutover-and-decommission]].
- `ingress-tailscale.yaml` — ProxyGroup ingress, **must not** set
an explicit `host:` (or use `host: *`) per the lesson on
ProxyGroup VIP routing.
**Hostname collision warning:** the minikube ingress claims the
Tailscale device name `photos` (`tls.hosts: [photos]`). Two
devices on the tailnet cannot share that name. While the
ringtail deployment is being staged it must use a *different*
`tls.hosts` value (e.g. `photos-ringtail`) so it can coexist
with the running minikube one. The flip to `photos` happens at
cutover time, *after* the minikube ingress has been removed.
See [[immich-cutover-and-decommission#Cutover sequence]].
- `kustomization.yaml` — same `images:` block (server, ML, valkey).
- New ArgoCD app `argocd/apps/immich-ringtail.yaml` targeting
ringtail, namespace `immich`. **Manual sync only** until the

View file

@ -35,22 +35,36 @@ real cutover.
scale deploy/immich-server --replicas=0` and same for ML. Leave
valkey + pg running. Confirm no client traffic on the source pg
via `pg_stat_activity`.
2. **Final sync.** Per chosen method in
2. **Tear down the minikube Tailscale ingress.** The `photos`
Tailscale device name must be freed before ringtail's ingress can
claim it (Tailscale enforces uniqueness across the tailnet).
`kubectl --context=minikube-indri -n immich delete ingress
immich-tailscale` and wait for the corresponding `tailscale`-LB
StatefulSet pod to terminate. Verify the `photos` device is gone:
`tailscale status | grep -i photos` from any tailnet host.
3. **Final sync.** Per chosen method in
[[immich-pg-data-migration]]:
- Option A: promote the ringtail replica.
- Option B: take final `pg_dump`, restore to ringtail
`immich-pg`.
3. **Verify.** Run the row-count and schema-diff checks from
4. **Verify.** Run the row-count and schema-diff checks from
[[immich-pg-data-migration#Verification on the real run]].
4. **Bring up ringtail immich** against the now-promoted pg
5. **Flip the ringtail ingress to `photos`.** Update
`argocd/manifests/immich-ringtail/ingress-tailscale.yaml`:
`tls.hosts: [photos]` (was `[photos-ringtail]` during staging per
[[immich-app-on-ringtail]]). Commit, `argocd app sync
immich-ringtail`. Wait for the `photos` device to register on the
tailnet again.
6. **Bring up ringtail immich** against the now-promoted pg
(`argocd app sync immich-ringtail`). Wait for Ready.
5. **Flip routing.** Update Caddy on indri
7. **Flip routing.** Update Caddy on indri
(`ansible/roles/caddy/defaults/main.yml`): `photos.ops.eblu.me`
upstream changes to the ringtail Tailscale ingress hostname.
`mise run provision-indri -- --tags caddy`.
6. **Smoke test.** Open `photos.ops.eblu.me` in a browser. Sign in.
upstream changes to the ringtail Tailscale ingress hostname
(`photos` — same MagicDNS name, now pointing to the ringtail
proxy). `mise run provision-indri -- --tags caddy`.
8. **Smoke test.** Open `photos.ops.eblu.me` in a browser. Sign in.
Scroll the timeline. Open an album. Trigger an ML search.
7. **Update borgmatic.** If the Tailscale hostname for pg changed,
9. **Update borgmatic.** If the Tailscale hostname for pg changed,
update `borgmatic.cfg` on indri to point at the ringtail
`immich-pg-tailscale` service. Run a manual backup to verify.

View file

@ -83,6 +83,52 @@ This is a C2 Mikado chain. The prerequisite cards each represent a
distinct surface that has to work before cutover. See
[[agent-change-process#C2 — Mikado Chain]] for the discipline.
## Workflow note: registering new ArgoCD apps during the chain
This chain adds three new ArgoCD `Application` definitions in
`argocd/apps/`: `cloudnative-pg-ringtail`, `databases-ringtail`,
and (later) `immich-ringtail`. The usual C1/C2 pattern of
`argocd app set <app> --revision <branch> && argocd app sync <app>`
does NOT work for the app-of-apps `apps` Application itself, because
`apps` self-manages: it re-reads `apps.yaml` (which declares
`targetRevision: main`) on every sync and reverts the override. As a
result, new app definitions added on a feature branch are never
visible to the cluster via `apps`.
**Use `kubectl apply` to register each new Application directly:**
```fish
kubectl --context=minikube-indri apply -f argocd/apps/<new-app>.yaml
```
This creates the Application resource out-of-band, bypassing `apps`.
For apps whose source lives in **this** repo (e.g.
`databases-ringtail`, `immich-ringtail` — manifest paths exist only
on the branch until merge), follow the apply with a branch override:
```fish
argocd app set <new-app> --revision mikado/migrate-immich-to-ringtail
argocd app sync <new-app>
```
For apps whose source is an **external** repo at a pinned tag (e.g.
`cloudnative-pg-ringtail``mirrors/cloudnative-pg` `v1.27.1`), no
override is needed — the source revision is independent of this PR.
After PR merge:
```fish
argocd app set <new-app> --revision main
argocd app sync <new-app>
```
`apps` itself, on its next sync from `main`, will discover the new
Application definitions in `argocd/apps/` and adopt the already-running
resources without disruption — provided their in-cluster spec matches
the on-disk definitions (which it does because we applied the same
file).
## Related
- [[shower-on-ringtail]] — a previous migration to ringtail (simpler: