From fcac8e5a7290bac54b25f82895c8120ef81367ff Mon Sep 17 00:00:00 2001 From: Erich Blume Date: Wed, 3 Jun 2026 10:34:00 -0700 Subject: [PATCH] =?UTF-8?q?Wave=201=20indri=E2=86=92ringtail=20migration:?= =?UTF-8?q?=20paperless,=20teslamate,=20mealie=20(#363)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Migrate paperless, teslamate, and mealie off the OOM-saturated minikube-indri node onto ringtail k3s, shedding ~1.1 GiB of resident load. Second chain in the indri-k8s decommission after immich. **Containers ported to Nix (default.nix), build-verified on ringtail:** - paperless → wraps nixpkgs paperless-ngx 2.20.15 (pinned unstable); runs as web/worker/beat/consumer - mealie → wraps nixpkgs mealie 3.16.0 (forward 4-minor bump, breaking-change reviewed); single gunicorn, SQLite - teslamate → from-scratch beamPackages mixRelease (not in nixpkgs); erlang_27+elixir_1_18, npm assets, ex_cldr locales pre-fetched **Data:** cold downtime-tolerant cutover. paperless+teslamate postgres dump/restore from quiesced source into a new ringtail blumeops-pg CNPG cluster; mealie SQLite PVC copied. Source DBs untouched until verified (rollback = repoint). **Also:** ringtail blumeops-pg cluster + ExternalSecrets scaffold; fixes pre-existing shower version-check drift. Runbook: docs/how-to/ringtail/migrate-wave1-ringtail.md. Deploy-from-branch + cutover happens before merge; container images rebuilt from main after merge. Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/363 --- argocd/apps/mealie-ringtail.yaml | 26 +++ argocd/apps/paperless-ringtail.yaml | 28 +++ argocd/apps/teslamate-ringtail.yaml | 28 +++ .../databases-ringtail/blumeops-pg.yaml | 97 +++++++++ .../external-secret-borgmatic.yaml | 30 +++ .../external-secret-eblume.yaml | 30 +++ .../external-secret-paperless.yaml | 28 +++ .../external-secret-teslamate.yaml | 30 +++ .../databases-ringtail/kustomization.yaml | 6 + .../manifests/mealie-ringtail/deployment.yaml | 102 +++++++++ .../mealie-ringtail/external-secret.yaml | 23 ++ .../ingress-tailscale.yaml | 0 .../mealie-ringtail/kustomization.yaml | 15 ++ argocd/manifests/mealie-ringtail/pvc.yaml | 14 ++ argocd/manifests/mealie-ringtail/service.yaml | 13 ++ argocd/manifests/mealie/deployment.yaml | 4 +- argocd/manifests/mealie/kustomization.yaml | 2 +- .../paperless-ringtail/deployment.yaml | 201 ++++++++++++++++++ .../paperless-ringtail/external-secret.yaml | 31 +++ .../ingress-tailscale.yaml | 0 .../paperless-ringtail/kustomization.yaml | 21 ++ .../manifests/paperless-ringtail/pv-nfs.yaml | 22 ++ argocd/manifests/paperless-ringtail/pvc.yaml | 15 ++ .../manifests/paperless-ringtail/service.yaml | 13 ++ argocd/manifests/paperless/deployment.yaml | 5 +- argocd/manifests/paperless/kustomization.yaml | 2 +- .../teslamate-ringtail/deployment.yaml | 72 +++++++ .../external-secret-db.yaml | 25 +++ .../external-secret-encryption-key.yaml | 27 +++ .../ingress-tailscale.yaml | 0 .../teslamate-ringtail/kustomization.yaml | 15 ++ .../manifests/teslamate-ringtail/service.yaml | 12 ++ argocd/manifests/teslamate/deployment.yaml | 5 +- argocd/manifests/teslamate/kustomization.yaml | 2 +- containers/mealie/Dockerfile | 145 ------------- containers/mealie/default.nix | 65 ++++++ containers/paperless/Dockerfile | 156 -------------- containers/paperless/default.nix | 77 +++++++ containers/teslamate/container.py | 104 --------- containers/teslamate/default.nix | 122 +++++++++++ containers/teslamate/entrypoint.sh | 23 -- .../migrate-wave1-ringtail.infra.md | 13 ++ .../immich/migrate-immich-to-ringtail.md | 2 + .../how-to/ringtail/migrate-wave1-ringtail.md | 176 +++++++++++++++ service-versions.yaml | 40 +++- 45 files changed, 1422 insertions(+), 445 deletions(-) create mode 100644 argocd/apps/mealie-ringtail.yaml create mode 100644 argocd/apps/paperless-ringtail.yaml create mode 100644 argocd/apps/teslamate-ringtail.yaml create mode 100644 argocd/manifests/databases-ringtail/blumeops-pg.yaml create mode 100644 argocd/manifests/databases-ringtail/external-secret-borgmatic.yaml create mode 100644 argocd/manifests/databases-ringtail/external-secret-eblume.yaml create mode 100644 argocd/manifests/databases-ringtail/external-secret-paperless.yaml create mode 100644 argocd/manifests/databases-ringtail/external-secret-teslamate.yaml create mode 100644 argocd/manifests/mealie-ringtail/deployment.yaml create mode 100644 argocd/manifests/mealie-ringtail/external-secret.yaml rename argocd/manifests/{mealie => mealie-ringtail}/ingress-tailscale.yaml (100%) create mode 100644 argocd/manifests/mealie-ringtail/kustomization.yaml create mode 100644 argocd/manifests/mealie-ringtail/pvc.yaml create mode 100644 argocd/manifests/mealie-ringtail/service.yaml create mode 100644 argocd/manifests/paperless-ringtail/deployment.yaml create mode 100644 argocd/manifests/paperless-ringtail/external-secret.yaml rename argocd/manifests/{paperless => paperless-ringtail}/ingress-tailscale.yaml (100%) create mode 100644 argocd/manifests/paperless-ringtail/kustomization.yaml create mode 100644 argocd/manifests/paperless-ringtail/pv-nfs.yaml create mode 100644 argocd/manifests/paperless-ringtail/pvc.yaml create mode 100644 argocd/manifests/paperless-ringtail/service.yaml create mode 100644 argocd/manifests/teslamate-ringtail/deployment.yaml create mode 100644 argocd/manifests/teslamate-ringtail/external-secret-db.yaml create mode 100644 argocd/manifests/teslamate-ringtail/external-secret-encryption-key.yaml rename argocd/manifests/{teslamate => teslamate-ringtail}/ingress-tailscale.yaml (100%) create mode 100644 argocd/manifests/teslamate-ringtail/kustomization.yaml create mode 100644 argocd/manifests/teslamate-ringtail/service.yaml delete mode 100644 containers/mealie/Dockerfile create mode 100644 containers/mealie/default.nix delete mode 100644 containers/paperless/Dockerfile create mode 100644 containers/paperless/default.nix delete mode 100644 containers/teslamate/container.py create mode 100644 containers/teslamate/default.nix delete mode 100644 containers/teslamate/entrypoint.sh create mode 100644 docs/changelog.d/migrate-wave1-ringtail.infra.md create mode 100644 docs/how-to/ringtail/migrate-wave1-ringtail.md diff --git a/argocd/apps/mealie-ringtail.yaml b/argocd/apps/mealie-ringtail.yaml new file mode 100644 index 0000000..2f014a9 --- /dev/null +++ b/argocd/apps/mealie-ringtail.yaml @@ -0,0 +1,26 @@ +# Mealie on ringtail k3s. +# +# Wave-1 indri-k8s decommission. Staging deployment; the minikube `mealie` +# app stays in parallel until cutover (copy SQLite PVC, drop the minikube +# tailscale ingress, flip Caddy). See [[migrate-wave1-ringtail]]. +# +# Prerequisites: +# - external-secrets-ringtail (onepassword-blumeops ClusterSecretStore) +# - mealie-data PVC contents copied from minikube at cutover +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: mealie-ringtail + namespace: argocd +spec: + project: default + source: + repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git + targetRevision: main + path: argocd/manifests/mealie-ringtail + destination: + server: https://ringtail.tail8d86e.ts.net:6443 + namespace: mealie + syncPolicy: + syncOptions: + - CreateNamespace=true diff --git a/argocd/apps/paperless-ringtail.yaml b/argocd/apps/paperless-ringtail.yaml new file mode 100644 index 0000000..bec98e9 --- /dev/null +++ b/argocd/apps/paperless-ringtail.yaml @@ -0,0 +1,28 @@ +# Paperless-ngx on ringtail k3s. +# +# Wave-1 indri-k8s decommission. Staging deployment; the minikube +# `paperless` app stays in parallel until cutover (drop the minikube +# tailscale ingress to free the name, then flip Caddy). See +# [[migrate-wave1-ringtail]]. +# +# Prerequisites: +# - databases-ringtail blumeops-pg (paperless database + role) +# - external-secrets-ringtail (onepassword-blumeops ClusterSecretStore) +# - sifaka NFS rule granting ringtail access to /volume1/paperless +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: paperless-ringtail + namespace: argocd +spec: + project: default + source: + repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git + targetRevision: main + path: argocd/manifests/paperless-ringtail + destination: + server: https://ringtail.tail8d86e.ts.net:6443 + namespace: paperless + syncPolicy: + syncOptions: + - CreateNamespace=true diff --git a/argocd/apps/teslamate-ringtail.yaml b/argocd/apps/teslamate-ringtail.yaml new file mode 100644 index 0000000..b7b3491 --- /dev/null +++ b/argocd/apps/teslamate-ringtail.yaml @@ -0,0 +1,28 @@ +# TeslaMate on ringtail k3s. +# +# Wave-1 indri-k8s decommission. Staging deployment; the minikube +# `teslamate` app stays in parallel until cutover (migrate the teslamate +# database, drop the minikube tailscale ingress, flip Caddy). See +# [[migrate-wave1-ringtail]]. +# +# Prerequisites: +# - databases-ringtail blumeops-pg (teslamate database + role; cube + +# earthdistance extensions created by superuser at cutover) +# - external-secrets-ringtail (onepassword-blumeops ClusterSecretStore) +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: teslamate-ringtail + namespace: argocd +spec: + project: default + source: + repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git + targetRevision: main + path: argocd/manifests/teslamate-ringtail + destination: + server: https://ringtail.tail8d86e.ts.net:6443 + namespace: teslamate + syncPolicy: + syncOptions: + - CreateNamespace=true diff --git a/argocd/manifests/databases-ringtail/blumeops-pg.yaml b/argocd/manifests/databases-ringtail/blumeops-pg.yaml new file mode 100644 index 0000000..3a37249 --- /dev/null +++ b/argocd/manifests/databases-ringtail/blumeops-pg.yaml @@ -0,0 +1,97 @@ +# PostgreSQL Cluster for blumeops services on ringtail k3s. +# +# Wave-1 indri-k8s decommission target (see [[migrate-wave1-ringtail]]). +# Holds the paperless and teslamate databases migrated off the minikube +# blumeops-pg via cold pg_dump/pg_restore at cutover. miniflux + authentik +# stay where they are for now (later waves), so this cluster only carries +# the wave-1 roles. +# +# Apps reach this in-cluster at blumeops-pg-rw.databases.svc.cluster.local +# — the same name they used on minikube, so teslamate's DATABASE_HOST is +# unchanged. +# +# Database creation is deferred to cutover, mirroring the minikube cluster +# (where only the bootstrap database is declared and the rest were created +# out-of-band): +# - paperless: the bootstrap database below (restored into at cutover). +# - teslamate: created at its cutover by the eblume superuser, because the +# dump's `earthdistance` extension is untrusted and CREATE EXTENSION +# needs superuser. (cube + earthdistance ownership then transferred to +# the teslamate role so it can ALTER EXTENSION UPDATE.) +apiVersion: postgresql.cnpg.io/v1 +kind: Cluster +metadata: + name: blumeops-pg + namespace: databases +spec: + instances: 1 + imageName: ghcr.io/cloudnative-pg/postgresql:18.3 + + storage: + size: 10Gi + storageClass: local-path + + bootstrap: + initdb: + database: paperless + owner: paperless + + managed: + roles: + # eblume superuser for admin + privileged restore steps (extensions) + - name: eblume + login: true + superuser: true + createdb: true + createrole: true + connectionLimit: -1 + ensure: present + inherit: true + passwordSecret: + name: blumeops-pg-eblume + # borgmatic read-only user for backups + - name: borgmatic + login: true + connectionLimit: -1 + ensure: present + inherit: true + inRoles: + - pg_read_all_data + passwordSecret: + name: blumeops-pg-borgmatic + # paperless user (also the bootstrap database owner above; the + # managed role sets its password from the 1Password-backed secret) + - name: paperless + login: true + connectionLimit: -1 + ensure: present + inherit: true + passwordSecret: + name: blumeops-pg-paperless + # teslamate user. Extension ownership (cube, earthdistance) is + # transferred to this role at cutover so it can ALTER EXTENSION UPDATE. + - name: teslamate + login: true + connectionLimit: -1 + ensure: present + inherit: true + passwordSecret: + name: blumeops-pg-teslamate + + resources: + requests: + memory: "256Mi" + cpu: "100m" + limits: + memory: "1Gi" + cpu: "500m" + + postgresql: + parameters: + max_connections: "50" + shared_buffers: "128MB" + password_encryption: "scram-sha-256" + pg_hba: + # Password auth from anywhere; network security is via Tailscale. + - host all all 0.0.0.0/0 scram-sha-256 + - host all all ::/0 scram-sha-256 diff --git a/argocd/manifests/databases-ringtail/external-secret-borgmatic.yaml b/argocd/manifests/databases-ringtail/external-secret-borgmatic.yaml new file mode 100644 index 0000000..ee600e3 --- /dev/null +++ b/argocd/manifests/databases-ringtail/external-secret-borgmatic.yaml @@ -0,0 +1,30 @@ +# ExternalSecret for borgmatic backup user password +# +# Replaces the manual op inject workflow from secret-borgmatic.yaml.tpl +# +# 1Password item: "borgmatic" in blumeops vault +# Field: "db-password" +# +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: blumeops-pg-borgmatic + namespace: databases +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: onepassword-blumeops + target: + name: blumeops-pg-borgmatic + creationPolicy: Owner + template: + type: kubernetes.io/basic-auth + data: + username: borgmatic + password: "{{ .password }}" + data: + - secretKey: password + remoteRef: + key: borgmatic + property: db-password diff --git a/argocd/manifests/databases-ringtail/external-secret-eblume.yaml b/argocd/manifests/databases-ringtail/external-secret-eblume.yaml new file mode 100644 index 0000000..a324c7d --- /dev/null +++ b/argocd/manifests/databases-ringtail/external-secret-eblume.yaml @@ -0,0 +1,30 @@ +# ExternalSecret for eblume superuser password +# +# Replaces the manual op inject workflow from secret-eblume.yaml.tpl +# +# 1Password item: "postgres" in blumeops vault +# Field: "password" +# +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: blumeops-pg-eblume + namespace: databases +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: onepassword-blumeops + target: + name: blumeops-pg-eblume + creationPolicy: Owner + template: + type: kubernetes.io/basic-auth + data: + username: eblume + password: "{{ .password }}" + data: + - secretKey: password + remoteRef: + key: postgres + property: password diff --git a/argocd/manifests/databases-ringtail/external-secret-paperless.yaml b/argocd/manifests/databases-ringtail/external-secret-paperless.yaml new file mode 100644 index 0000000..e5742be --- /dev/null +++ b/argocd/manifests/databases-ringtail/external-secret-paperless.yaml @@ -0,0 +1,28 @@ +# ExternalSecret for Paperless database user password +# +# 1Password item: "Paperless (blumeops)" in blumeops vault +# Field: "postgresql-password" +# +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: blumeops-pg-paperless + namespace: databases +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: onepassword-blumeops + target: + name: blumeops-pg-paperless + creationPolicy: Owner + template: + type: kubernetes.io/basic-auth + data: + username: paperless + password: "{{ .password }}" + data: + - secretKey: password + remoteRef: + key: Paperless (blumeops) + property: postgresql-password diff --git a/argocd/manifests/databases-ringtail/external-secret-teslamate.yaml b/argocd/manifests/databases-ringtail/external-secret-teslamate.yaml new file mode 100644 index 0000000..0c52e0b --- /dev/null +++ b/argocd/manifests/databases-ringtail/external-secret-teslamate.yaml @@ -0,0 +1,30 @@ +# ExternalSecret for TeslaMate database user password +# +# Replaces the manual op inject workflow from secret-teslamate.yaml.tpl +# +# 1Password item: "TeslaMate" in blumeops vault +# Field: "db_password" +# +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: blumeops-pg-teslamate + namespace: databases +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: onepassword-blumeops + target: + name: blumeops-pg-teslamate + creationPolicy: Owner + template: + type: kubernetes.io/basic-auth + data: + username: teslamate + password: "{{ .password }}" + data: + - secretKey: password + remoteRef: + key: TeslaMate + property: db_password diff --git a/argocd/manifests/databases-ringtail/kustomization.yaml b/argocd/manifests/databases-ringtail/kustomization.yaml index 971e2d4..2bc2af3 100644 --- a/argocd/manifests/databases-ringtail/kustomization.yaml +++ b/argocd/manifests/databases-ringtail/kustomization.yaml @@ -7,3 +7,9 @@ resources: - immich-pg.yaml - external-secret-immich-borgmatic.yaml - service-immich-pg-tailscale.yaml + # wave-1 indri-k8s decommission: blumeops-pg (paperless + teslamate) + - blumeops-pg.yaml + - external-secret-eblume.yaml + - external-secret-borgmatic.yaml + - external-secret-paperless.yaml + - external-secret-teslamate.yaml diff --git a/argocd/manifests/mealie-ringtail/deployment.yaml b/argocd/manifests/mealie-ringtail/deployment.yaml new file mode 100644 index 0000000..10d06ab --- /dev/null +++ b/argocd/manifests/mealie-ringtail/deployment.yaml @@ -0,0 +1,102 @@ +# Mealie on ringtail k3s — Nix image. +# +# Single gunicorn process (the Nix image's default `mealie-run` entrypoint +# runs init_db then gunicorn), serving the prebuilt frontend. DB is SQLite +# on the mealie-data PVC; its contents are copied from the minikube PVC at +# cutover. See [[migrate-wave1-ringtail]]. +apiVersion: apps/v1 +kind: Deployment +metadata: + name: mealie + namespace: mealie +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: mealie + template: + metadata: + labels: + app: mealie + spec: + securityContext: + seccompProfile: + type: RuntimeDefault + containers: + - name: mealie + image: registry.ops.eblu.me/blumeops/mealie:kustomized + ports: + - containerPort: 9000 + env: + - name: BASE_URL + value: "https://meals.ops.eblu.me" + - name: ALLOW_SIGNUP + value: "false" + - name: TZ + value: "America/Los_Angeles" + - name: MAX_WORKERS + value: "1" + - name: WEB_CONCURRENCY + value: "1" + # OIDC — Authentik (public client, PKCE) + - name: OIDC_AUTH_ENABLED + value: "true" + - name: OIDC_CONFIGURATION_URL + value: "https://authentik.ops.eblu.me/application/o/mealie/.well-known/openid-configuration" + - name: OIDC_CLIENT_ID + value: "mealie" + - name: OIDC_CLIENT_SECRET + valueFrom: + secretKeyRef: + name: mealie-secrets + key: oidc-client-secret + - name: OIDC_AUTO_REDIRECT + value: "false" + - name: OIDC_PROVIDER_NAME + value: "Authentik" + - name: OIDC_ADMIN_GROUP + value: "admins" + - name: OIDC_SIGNUP_ENABLED + value: "true" + - name: OIDC_USER_CLAIM + value: "email" + # OpenAI — recipe parsing, image OCR, ingredient extraction + - name: OPENAI_API_KEY + valueFrom: + secretKeyRef: + name: mealie-secrets + key: openai-api-key + - name: OPENAI_MODEL + value: "gpt-4o" + - name: OPENAI_REQUEST_TIMEOUT + value: "120" + - name: OPENAI_WORKERS + value: "1" + volumeMounts: + - name: data + mountPath: /app/data + resources: + requests: + memory: "128Mi" + cpu: "50m" + limits: + memory: "1000Mi" + cpu: "500m" + livenessProbe: + httpGet: + path: /api/app/about + port: 9000 + initialDelaySeconds: 30 + periodSeconds: 30 + readinessProbe: + httpGet: + path: /api/app/about + port: 9000 + initialDelaySeconds: 10 + periodSeconds: 10 + volumes: + - name: data + persistentVolumeClaim: + claimName: mealie-data diff --git a/argocd/manifests/mealie-ringtail/external-secret.yaml b/argocd/manifests/mealie-ringtail/external-secret.yaml new file mode 100644 index 0000000..99c2793 --- /dev/null +++ b/argocd/manifests/mealie-ringtail/external-secret.yaml @@ -0,0 +1,23 @@ +--- +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: mealie-secrets + namespace: mealie +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: onepassword-blumeops + target: + name: mealie-secrets + creationPolicy: Owner + data: + - secretKey: oidc-client-secret + remoteRef: + key: "Authentik (blumeops)" + property: mealie-client-secret + - secretKey: openai-api-key + remoteRef: + key: "openai (blumeops)" + property: credential diff --git a/argocd/manifests/mealie/ingress-tailscale.yaml b/argocd/manifests/mealie-ringtail/ingress-tailscale.yaml similarity index 100% rename from argocd/manifests/mealie/ingress-tailscale.yaml rename to argocd/manifests/mealie-ringtail/ingress-tailscale.yaml diff --git a/argocd/manifests/mealie-ringtail/kustomization.yaml b/argocd/manifests/mealie-ringtail/kustomization.yaml new file mode 100644 index 0000000..8428042 --- /dev/null +++ b/argocd/manifests/mealie-ringtail/kustomization.yaml @@ -0,0 +1,15 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +namespace: mealie + +resources: + - deployment.yaml + - service.yaml + - pvc.yaml + - ingress-tailscale.yaml + - external-secret.yaml + +images: + - name: registry.ops.eblu.me/blumeops/mealie + newTag: v3.16.0-1d4cbbf-nix diff --git a/argocd/manifests/mealie-ringtail/pvc.yaml b/argocd/manifests/mealie-ringtail/pvc.yaml new file mode 100644 index 0000000..89c38ef --- /dev/null +++ b/argocd/manifests/mealie-ringtail/pvc.yaml @@ -0,0 +1,14 @@ +# SQLite data volume for Mealie on ringtail. Contents copied from the +# minikube mealie-data PVC at cutover (recipes, meal plans, uploaded media). +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: mealie-data + namespace: mealie +spec: + accessModes: + - ReadWriteOnce + storageClassName: local-path + resources: + requests: + storage: 2Gi diff --git a/argocd/manifests/mealie-ringtail/service.yaml b/argocd/manifests/mealie-ringtail/service.yaml new file mode 100644 index 0000000..4162b96 --- /dev/null +++ b/argocd/manifests/mealie-ringtail/service.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +kind: Service +metadata: + name: mealie + namespace: mealie +spec: + selector: + app: mealie + ports: + - name: http + port: 9000 + targetPort: 9000 + protocol: TCP diff --git a/argocd/manifests/mealie/deployment.yaml b/argocd/manifests/mealie/deployment.yaml index bdcf91e..7cdd275 100644 --- a/argocd/manifests/mealie/deployment.yaml +++ b/argocd/manifests/mealie/deployment.yaml @@ -4,7 +4,9 @@ metadata: name: mealie namespace: mealie spec: - replicas: 1 + # Migrated to ringtail (mealie-ringtail). Scaled to 0; SQLite PVC retained + # for rollback until the decommission PR. See [[migrate-wave1-ringtail]]. + replicas: 0 selector: matchLabels: app: mealie diff --git a/argocd/manifests/mealie/kustomization.yaml b/argocd/manifests/mealie/kustomization.yaml index fb0713b..02563f4 100644 --- a/argocd/manifests/mealie/kustomization.yaml +++ b/argocd/manifests/mealie/kustomization.yaml @@ -7,7 +7,7 @@ resources: - deployment.yaml - service.yaml - pvc.yaml - - ingress-tailscale.yaml + # ingress removed: name 'meals' handed off to mealie-ringtail at cutover - external-secret.yaml images: diff --git a/argocd/manifests/paperless-ringtail/deployment.yaml b/argocd/manifests/paperless-ringtail/deployment.yaml new file mode 100644 index 0000000..de4f456 --- /dev/null +++ b/argocd/manifests/paperless-ringtail/deployment.yaml @@ -0,0 +1,201 @@ +# Paperless-ngx on ringtail k3s — Nix image, multi-process. +# +# The upstream s6 image ran web + worker + scheduler + consumer (and DB +# migrations) in one container. The Nix image (containers/paperless/ +# default.nix) ships the binaries but no supervisor, so we run those as +# four containers in one pod, sharing the local data/consume dirs +# (emptyDir) and the NFS media volume; redis is colocated so +# PAPERLESS_REDIS=localhost works for all. A migrate initContainer runs +# DB migrations once before the app containers start. +# +# DB points in-cluster at the ringtail blumeops-pg (was pg.ops.eblu.me on +# indri). PAPERLESS_{DATA_DIR,MEDIA_ROOT,CONSUMPTION_DIR} are set +# explicitly because the Nix package does not default to the upstream +# /usr/src/paperless paths. +apiVersion: apps/v1 +kind: Deployment +metadata: + name: paperless + namespace: paperless +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: paperless + template: + metadata: + labels: + app: paperless + spec: + securityContext: + seccompProfile: + type: RuntimeDefault + initContainers: + # redis as a native sidecar (restartPolicy: Always): starts before + # the migrate init and stays running for the app containers, so all + # of them reach PAPERLESS_REDIS=localhost:6379. + - name: redis + image: docker.io/library/redis:kustomized + restartPolicy: Always + ports: + - containerPort: 6379 + volumeMounts: + - name: redis-data + mountPath: /data + resources: + requests: + memory: "32Mi" + cpu: "10m" + limits: + memory: "128Mi" + - name: migrate + image: registry.ops.eblu.me/blumeops/paperless:kustomized + command: ["paperless-ngx", "migrate", "--no-input"] + env: &paperless-env + - name: PAPERLESS_URL + value: "https://paperless.ops.eblu.me" + - name: PAPERLESS_REDIS + value: "redis://localhost:6379" + - name: PAPERLESS_DBHOST + value: "blumeops-pg-rw.databases.svc.cluster.local" + - name: PAPERLESS_DBPORT + value: "5432" + - name: PAPERLESS_DBNAME + value: "paperless" + - name: PAPERLESS_DBUSER + value: "paperless" + - name: PAPERLESS_DBPASS + valueFrom: + secretKeyRef: + name: paperless-secrets + key: db-password + # Explicit port to override the k8s-injected PAPERLESS_PORT + # (service named 'paperless' would set PAPERLESS_PORT=tcp://...) + - name: PAPERLESS_PORT + value: "8000" + - name: PAPERLESS_DATA_DIR + value: "/usr/src/paperless/data" + - name: PAPERLESS_MEDIA_ROOT + value: "/usr/src/paperless/media" + - name: PAPERLESS_CONSUMPTION_DIR + value: "/usr/src/paperless/consume" + - name: PAPERLESS_SECRET_KEY + valueFrom: + secretKeyRef: + name: paperless-secrets + key: secret-key + - name: PAPERLESS_TIME_ZONE + value: "America/Los_Angeles" + - name: PAPERLESS_OCR_LANGUAGE + value: "eng" + - name: PAPERLESS_TASK_WORKERS + value: "1" + - name: PAPERLESS_ADMIN_USER + value: "eblume" + - name: PAPERLESS_ADMIN_PASSWORD + valueFrom: + secretKeyRef: + name: paperless-secrets + key: admin-password + - name: PAPERLESS_ADMIN_MAIL + value: "blume.erich@gmail.com" + - name: PAPERLESS_APPS + value: "allauth.socialaccount.providers.openid_connect" + - name: PAPERLESS_SOCIALACCOUNT_PROVIDERS + valueFrom: + secretKeyRef: + name: paperless-secrets + key: socialaccount-providers + - name: PAPERLESS_SOCIALACCOUNT_ALLOW_SIGNUPS + value: "true" + - name: PAPERLESS_SOCIAL_AUTO_SIGNUP + value: "true" + - name: PAPERLESS_ACCOUNT_ALLOW_SIGNUPS + value: "false" + - name: PAPERLESS_REDIRECT_LOGIN_TO_SSO + value: "false" + volumeMounts: &paperless-mounts + - name: data + mountPath: /usr/src/paperless/data + - name: media + mountPath: /usr/src/paperless/media + - name: consume + mountPath: /usr/src/paperless/consume + containers: + - name: web + image: registry.ops.eblu.me/blumeops/paperless:kustomized + ports: + - containerPort: 8000 + name: http + env: *paperless-env + volumeMounts: *paperless-mounts + resources: + requests: + memory: "256Mi" + cpu: "100m" + limits: + memory: "1Gi" + cpu: "1000m" + livenessProbe: + httpGet: + path: / + port: 8000 + initialDelaySeconds: 60 + periodSeconds: 30 + readinessProbe: + httpGet: + path: / + port: 8000 + initialDelaySeconds: 30 + periodSeconds: 10 + + - name: worker + image: registry.ops.eblu.me/blumeops/paperless:kustomized + command: ["celery", "--app", "paperless", "worker", "--loglevel", "INFO"] + env: *paperless-env + volumeMounts: *paperless-mounts + resources: + requests: + memory: "256Mi" + cpu: "100m" + limits: + memory: "1Gi" + cpu: "1000m" + + - name: beat + image: registry.ops.eblu.me/blumeops/paperless:kustomized + command: ["celery", "--app", "paperless", "beat", "--loglevel", "INFO"] + env: *paperless-env + volumeMounts: *paperless-mounts + resources: + requests: + memory: "64Mi" + cpu: "20m" + limits: + memory: "256Mi" + + - name: consumer + image: registry.ops.eblu.me/blumeops/paperless:kustomized + command: ["paperless-ngx", "document_consumer"] + env: *paperless-env + volumeMounts: *paperless-mounts + resources: + requests: + memory: "128Mi" + cpu: "50m" + limits: + memory: "512Mi" + + volumes: + - name: data + emptyDir: {} + - name: media + persistentVolumeClaim: + claimName: paperless-media + - name: consume + emptyDir: {} + - name: redis-data + emptyDir: + sizeLimit: 1Gi diff --git a/argocd/manifests/paperless-ringtail/external-secret.yaml b/argocd/manifests/paperless-ringtail/external-secret.yaml new file mode 100644 index 0000000..750b7c5 --- /dev/null +++ b/argocd/manifests/paperless-ringtail/external-secret.yaml @@ -0,0 +1,31 @@ +--- +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: paperless-secrets + namespace: paperless +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: onepassword-blumeops + target: + name: paperless-secrets + creationPolicy: Owner + data: + - secretKey: db-password + remoteRef: + key: "Paperless (blumeops)" + property: postgresql-password + - secretKey: secret-key + remoteRef: + key: "Paperless (blumeops)" + property: secret-key + - secretKey: admin-password + remoteRef: + key: "Paperless (blumeops)" + property: admin-password + - secretKey: socialaccount-providers + remoteRef: + key: "Paperless (blumeops)" + property: socialaccount-providers diff --git a/argocd/manifests/paperless/ingress-tailscale.yaml b/argocd/manifests/paperless-ringtail/ingress-tailscale.yaml similarity index 100% rename from argocd/manifests/paperless/ingress-tailscale.yaml rename to argocd/manifests/paperless-ringtail/ingress-tailscale.yaml diff --git a/argocd/manifests/paperless-ringtail/kustomization.yaml b/argocd/manifests/paperless-ringtail/kustomization.yaml new file mode 100644 index 0000000..0a691e0 --- /dev/null +++ b/argocd/manifests/paperless-ringtail/kustomization.yaml @@ -0,0 +1,21 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +namespace: paperless + +resources: + - deployment.yaml + - service.yaml + - pv-nfs.yaml + - pvc.yaml + - ingress-tailscale.yaml + - external-secret.yaml + +images: + - name: registry.ops.eblu.me/blumeops/paperless + newTag: v2.20.15-1d4cbbf-nix + # amd64 valkey built via nix (the v8.1.7-ecded30 tag without -nix is the + # arm64 Alpine build for indri and fails on ringtail with exec format error) + - name: docker.io/library/redis + newName: registry.ops.eblu.me/blumeops/valkey + newTag: v8.1.7-ecded30-nix diff --git a/argocd/manifests/paperless-ringtail/pv-nfs.yaml b/argocd/manifests/paperless-ringtail/pv-nfs.yaml new file mode 100644 index 0000000..2990d1a --- /dev/null +++ b/argocd/manifests/paperless-ringtail/pv-nfs.yaml @@ -0,0 +1,22 @@ +# NFS PersistentVolume for the Paperless document library, mounted from +# ringtail. Same sifaka export (/volume1/paperless) as the minikube PV, +# but a distinct PV name so both clusters can declare it during the +# parallel-run before cutover. +# +# Prerequisite: sifaka must have an NFS rule granting ringtail Read/Write +# (Squash=No mapping) on the paperless share — the same step done for +# immich. See [[sifaka-nfs-from-ringtail]]. +apiVersion: v1 +kind: PersistentVolume +metadata: + name: paperless-media-nfs-pv-ringtail +spec: + capacity: + storage: 500Gi + accessModes: + - ReadWriteMany + persistentVolumeReclaimPolicy: Retain + storageClassName: "" + nfs: + server: sifaka + path: /volume1/paperless diff --git a/argocd/manifests/paperless-ringtail/pvc.yaml b/argocd/manifests/paperless-ringtail/pvc.yaml new file mode 100644 index 0000000..8b44660 --- /dev/null +++ b/argocd/manifests/paperless-ringtail/pvc.yaml @@ -0,0 +1,15 @@ +# PersistentVolumeClaim for the Paperless document library on ringtail. +# Binds the NFS PV for sifaka:/volume1/paperless. +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: paperless-media + namespace: paperless +spec: + accessModes: + - ReadWriteMany + storageClassName: "" + volumeName: paperless-media-nfs-pv-ringtail + resources: + requests: + storage: 500Gi diff --git a/argocd/manifests/paperless-ringtail/service.yaml b/argocd/manifests/paperless-ringtail/service.yaml new file mode 100644 index 0000000..cff2972 --- /dev/null +++ b/argocd/manifests/paperless-ringtail/service.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +kind: Service +metadata: + name: paperless + namespace: paperless +spec: + selector: + app: paperless + ports: + - name: http + port: 8000 + targetPort: 8000 + protocol: TCP diff --git a/argocd/manifests/paperless/deployment.yaml b/argocd/manifests/paperless/deployment.yaml index cc2c013..1730486 100644 --- a/argocd/manifests/paperless/deployment.yaml +++ b/argocd/manifests/paperless/deployment.yaml @@ -4,7 +4,10 @@ metadata: name: paperless namespace: paperless spec: - replicas: 1 + # Migrated to ringtail (paperless-ringtail). Scaled to 0 to prevent + # double-writing the now-ringtail-owned database; manifest retained for + # rollback until the decommission PR. See [[migrate-wave1-ringtail]]. + replicas: 0 selector: matchLabels: app: paperless diff --git a/argocd/manifests/paperless/kustomization.yaml b/argocd/manifests/paperless/kustomization.yaml index 3cd0d74..a92a769 100644 --- a/argocd/manifests/paperless/kustomization.yaml +++ b/argocd/manifests/paperless/kustomization.yaml @@ -8,7 +8,7 @@ resources: - service.yaml - pv-nfs.yaml - pvc.yaml - - ingress-tailscale.yaml + # ingress removed: name 'paperless' handed off to paperless-ringtail at cutover - external-secret.yaml images: diff --git a/argocd/manifests/teslamate-ringtail/deployment.yaml b/argocd/manifests/teslamate-ringtail/deployment.yaml new file mode 100644 index 0000000..cf8cc73 --- /dev/null +++ b/argocd/manifests/teslamate-ringtail/deployment.yaml @@ -0,0 +1,72 @@ +# TeslaMate on ringtail k3s — Nix image. +# +# The Nix image's Entrypoint waits for postgres, runs migrations +# (TeslaMate.Release.migrate), then starts the release — so no command +# override is needed. Stateless; all data lives in the teslamate database +# on the ringtail blumeops-pg (DATABASE_HOST already an in-cluster name, +# unchanged from minikube). See [[migrate-wave1-ringtail]]. +apiVersion: apps/v1 +kind: Deployment +metadata: + name: teslamate + namespace: teslamate +spec: + replicas: 1 + selector: + matchLabels: + app: teslamate + template: + metadata: + labels: + app: teslamate + spec: + securityContext: + seccompProfile: + type: RuntimeDefault + containers: + - name: teslamate + image: registry.ops.eblu.me/blumeops/teslamate:kustomized + ports: + - containerPort: 4000 + env: + - name: DATABASE_USER + value: "teslamate" + - name: DATABASE_PASS + valueFrom: + secretKeyRef: + name: teslamate-db + key: password + - name: DATABASE_NAME + value: "teslamate" + - name: DATABASE_HOST + value: "blumeops-pg-rw.databases.svc.cluster.local" + - name: ENCRYPTION_KEY + valueFrom: + secretKeyRef: + name: teslamate-encryption + key: key + - name: DISABLE_MQTT + value: "true" + - name: CHECK_ORIGIN + value: "false" + - name: TZ + value: "America/Los_Angeles" + resources: + requests: + memory: "128Mi" + cpu: "100m" + limits: + memory: "512Mi" + cpu: "500m" + livenessProbe: + httpGet: + path: / + port: 4000 + initialDelaySeconds: 30 + periodSeconds: 30 + readinessProbe: + httpGet: + path: / + port: 4000 + initialDelaySeconds: 10 + periodSeconds: 10 diff --git a/argocd/manifests/teslamate-ringtail/external-secret-db.yaml b/argocd/manifests/teslamate-ringtail/external-secret-db.yaml new file mode 100644 index 0000000..11eeec6 --- /dev/null +++ b/argocd/manifests/teslamate-ringtail/external-secret-db.yaml @@ -0,0 +1,25 @@ +# ExternalSecret for TeslaMate database password +# +# Replaces the manual op inject workflow from secret-db.yaml.tpl +# +# 1Password item: "TeslaMate" in blumeops vault +# Field: "db_password" +# +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: teslamate-db + namespace: teslamate +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: onepassword-blumeops + target: + name: teslamate-db + creationPolicy: Owner + data: + - secretKey: password + remoteRef: + key: TeslaMate + property: db_password diff --git a/argocd/manifests/teslamate-ringtail/external-secret-encryption-key.yaml b/argocd/manifests/teslamate-ringtail/external-secret-encryption-key.yaml new file mode 100644 index 0000000..96938bf --- /dev/null +++ b/argocd/manifests/teslamate-ringtail/external-secret-encryption-key.yaml @@ -0,0 +1,27 @@ +# ExternalSecret for TeslaMate encryption key +# +# Replaces the manual op inject workflow from secret-encryption-key.yaml.tpl +# +# 1Password item: "TeslaMate" in blumeops vault +# Field: "api_enc_key" +# +# This key encrypts Tesla API tokens at rest in the database. +# +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: teslamate-encryption + namespace: teslamate +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: onepassword-blumeops + target: + name: teslamate-encryption + creationPolicy: Owner + data: + - secretKey: key + remoteRef: + key: TeslaMate + property: api_enc_key diff --git a/argocd/manifests/teslamate/ingress-tailscale.yaml b/argocd/manifests/teslamate-ringtail/ingress-tailscale.yaml similarity index 100% rename from argocd/manifests/teslamate/ingress-tailscale.yaml rename to argocd/manifests/teslamate-ringtail/ingress-tailscale.yaml diff --git a/argocd/manifests/teslamate-ringtail/kustomization.yaml b/argocd/manifests/teslamate-ringtail/kustomization.yaml new file mode 100644 index 0000000..f31fe09 --- /dev/null +++ b/argocd/manifests/teslamate-ringtail/kustomization.yaml @@ -0,0 +1,15 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +namespace: teslamate + +resources: + - deployment.yaml + - service.yaml + - ingress-tailscale.yaml + - external-secret-db.yaml + - external-secret-encryption-key.yaml + +images: + - name: registry.ops.eblu.me/blumeops/teslamate + newTag: v3.0.0-191be1b-nix diff --git a/argocd/manifests/teslamate-ringtail/service.yaml b/argocd/manifests/teslamate-ringtail/service.yaml new file mode 100644 index 0000000..b04f45e --- /dev/null +++ b/argocd/manifests/teslamate-ringtail/service.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: Service +metadata: + name: teslamate + namespace: teslamate +spec: + selector: + app: teslamate + ports: + - port: 4000 + targetPort: 4000 + type: ClusterIP diff --git a/argocd/manifests/teslamate/deployment.yaml b/argocd/manifests/teslamate/deployment.yaml index 42859a7..cf7f9bb 100644 --- a/argocd/manifests/teslamate/deployment.yaml +++ b/argocd/manifests/teslamate/deployment.yaml @@ -4,7 +4,10 @@ metadata: name: teslamate namespace: teslamate spec: - replicas: 1 + # Migrated to ringtail (teslamate-ringtail). Scaled to 0 to prevent + # double-writing the now-ringtail-owned database; manifest retained for + # rollback until the decommission PR. See [[migrate-wave1-ringtail]]. + replicas: 0 selector: matchLabels: app: teslamate diff --git a/argocd/manifests/teslamate/kustomization.yaml b/argocd/manifests/teslamate/kustomization.yaml index a00586f..be9d39d 100644 --- a/argocd/manifests/teslamate/kustomization.yaml +++ b/argocd/manifests/teslamate/kustomization.yaml @@ -6,7 +6,7 @@ namespace: teslamate resources: - deployment.yaml - service.yaml - - ingress-tailscale.yaml + # ingress removed: name 'tesla' handed off to teslamate-ringtail at cutover - external-secret-db.yaml - external-secret-encryption-key.yaml diff --git a/containers/mealie/Dockerfile b/containers/mealie/Dockerfile deleted file mode 100644 index 8df38bf..0000000 --- a/containers/mealie/Dockerfile +++ /dev/null @@ -1,145 +0,0 @@ -# Mealie — self-hosted recipe manager -# Built from source via forge mirror of mealie-recipes/mealie -# Based on upstream docker/Dockerfile (multi-stage: Node frontend + Python backend) - -ARG CONTAINER_APP_VERSION=v3.12.0 - -############################################### -# Frontend Build -############################################### -FROM node:24-slim AS frontend-builder - -ARG CONTAINER_APP_VERSION -RUN apt-get update && apt-get install --no-install-recommends -y git ca-certificates && rm -rf /var/lib/apt/lists/* - -RUN git clone --depth 1 --branch ${CONTAINER_APP_VERSION} \ - https://forge.ops.eblu.me/mirrors/mealie.git /src - -WORKDIR /src/frontend - -RUN yarn install \ - --prefer-offline \ - --frozen-lockfile \ - --non-interactive \ - --production=false \ - --network-timeout 1000000 - -RUN yarn generate - -############################################### -# Python Base -############################################### -FROM python:3.12-slim AS python-base - -ENV MEALIE_HOME="/app" -ENV PYTHONUNBUFFERED=1 \ - PYTHONDONTWRITEBYTECODE=1 \ - PIP_NO_CACHE_DIR=off \ - PIP_DISABLE_PIP_VERSION_CHECK=on \ - PIP_DEFAULT_TIMEOUT=100 \ - VENV_PATH="/opt/mealie" - -ENV PATH="$VENV_PATH/bin:$PATH" - -RUN useradd -u 911 -U -d $MEALIE_HOME -s /bin/bash abc \ - && usermod -G users abc \ - && mkdir $MEALIE_HOME - -############################################### -# Backend Package Build -############################################### -FROM python-base AS backend-builder - -ARG CONTAINER_APP_VERSION -RUN apt-get update \ - && apt-get install --no-install-recommends -y curl git ca-certificates \ - && rm -rf /var/lib/apt/lists/* - -RUN pip install uv - -RUN git clone --depth 1 --branch ${CONTAINER_APP_VERSION} \ - https://forge.ops.eblu.me/mirrors/mealie.git /src - -WORKDIR /src - -COPY --from=frontend-builder /src/frontend/dist ./mealie/frontend - -RUN uv build --out-dir dist - -RUN uv export --no-editable --no-emit-project --extra pgsql --format requirements-txt --output-file dist/requirements.txt \ - && MEALIE_VERSION=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml', 'rb'))['project']['version'])") \ - && echo "mealie[pgsql]==${MEALIE_VERSION} \\" >> dist/requirements.txt \ - && pip hash dist/mealie-${MEALIE_VERSION}-py3-none-any.whl | tail -n1 | tr -d '\n' >> dist/requirements.txt \ - && echo " \\" >> dist/requirements.txt \ - && pip hash dist/mealie-${MEALIE_VERSION}.tar.gz | tail -n1 >> dist/requirements.txt - -############################################### -# Python Venv Build -############################################### -FROM python-base AS venv-builder - -RUN apt-get update \ - && apt-get install --no-install-recommends -y \ - build-essential \ - libpq-dev \ - libwebp-dev \ - ffmpeg \ - libsasl2-dev libldap2-dev libssl-dev \ - gnupg gnupg2 gnupg1 \ - && rm -rf /var/lib/apt/lists/* - -RUN python3 -m venv --upgrade-deps $VENV_PATH - -COPY --from=backend-builder /src/dist /dist - -RUN . $VENV_PATH/bin/activate \ - && pip install --require-hashes -r /dist/requirements.txt --find-links /dist - -############################################### -# Production Image -############################################### -FROM python-base AS production - -ENV PRODUCTION=true -ENV TESTING=false - -RUN apt-get update \ - && apt-get install --no-install-recommends -y \ - curl \ - ffmpeg \ - gosu \ - iproute2 \ - libldap-common \ - libldap2 \ - && rm -rf /var/lib/apt/lists/* - -RUN mkdir -p /run/secrets - -COPY --from=venv-builder $VENV_PATH $VENV_PATH - -ENV NLTK_DATA="/nltk_data/" -RUN mkdir -p $NLTK_DATA -RUN python -m nltk.downloader -d $NLTK_DATA averaged_perceptron_tagger_eng - -VOLUME ["$MEALIE_HOME/data/"] -ENV APP_PORT=9000 - -EXPOSE ${APP_PORT} - -COPY --from=backend-builder /src/docker/healthcheck.sh $MEALIE_HOME/healthcheck.sh -RUN chmod +x $MEALIE_HOME/healthcheck.sh -HEALTHCHECK CMD $MEALIE_HOME/healthcheck.sh - -ENV HOST=0.0.0.0 - -COPY --from=backend-builder /src/docker/entry.sh $MEALIE_HOME/run.sh -RUN chmod +x $MEALIE_HOME/run.sh - -ARG CONTAINER_APP_VERSION -LABEL org.opencontainers.image.title="Mealie" -LABEL org.opencontainers.image.description="Self-hosted recipe manager" -LABEL org.opencontainers.image.version="${CONTAINER_APP_VERSION}" -LABEL org.opencontainers.image.source="https://forge.eblu.me/eblume/blumeops" -LABEL org.opencontainers.image.vendor="blumeops" - -ENTRYPOINT ["/app/run.sh"] diff --git a/containers/mealie/default.nix b/containers/mealie/default.nix new file mode 100644 index 0000000..fdb1430 --- /dev/null +++ b/containers/mealie/default.nix @@ -0,0 +1,65 @@ +# Nix-built Mealie for ringtail (amd64). +# +# Replaces the from-source Dockerfile build (Node frontend + Python venv) +# with nixpkgs' mealie, which ships a single `mealie` gunicorn entrypoint +# serving the prebuilt frontend + backend — so this is a clean single- +# process wrap (unlike paperless, which is multi-process). +# +# Mealie stores its DB as SQLite under DATA_DIR (the mealie-data PVC at +# /app/data); there is no postgres. The run wrapper mirrors the nixpkgs +# mealie NixOS module: run `libexec/init_db` (Alembic migrations) first, +# then exec gunicorn. +# +# Self-pins nixos-unstable: stable nixpkgs lags at 3.9.2, unstable carries +# 3.16.0. This is a forward 4-minor bump from the v3.12.0 Dockerfile build +# (the deferred upgrade) — mealie auto-migrates the SQLite DB forward on +# startup via init_db; the source PVC is retained for rollback. The version +# assertion makes nix-build fail if a pin bump changes the version. +let + nixpkgs = fetchTarball { + url = "https://github.com/NixOS/nixpkgs/archive/331800de5053fcebacf6813adb5db9c9dca22a0c.tar.gz"; + sha256 = "1p54fm6dkbq62kpi55cr4wyx7b1nsajpsnjgs64cmp073fwi15f7"; + }; + pkgs = import nixpkgs { system = "x86_64-linux"; }; + + version = "3.16.0"; + + app = pkgs.mealie; + + # Mirror the NixOS module's mealie service: init_db (Alembic) then + # gunicorn bound to the app port. DATA_DIR/env come from the image + + # k8s manifest. + mealie-run = pkgs.writeShellScriptBin "mealie-run" '' + set -e + ${app}/libexec/init_db + exec ${pkgs.lib.getExe app} -b 0.0.0.0:9000 + ''; +in + +assert app.version == version; + +pkgs.dockerTools.buildLayeredImage { + name = "blumeops/mealie"; + + contents = [ + app + mealie-run + pkgs.bashInteractive + pkgs.coreutils + pkgs.cacert + pkgs.tzdata + ]; + + config = { + Cmd = [ "${mealie-run}/bin/mealie-run" ]; + Env = [ + "DATA_DIR=/app/data" + "SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt" + "PYTHONUNBUFFERED=1" + "PRODUCTION=true" + ]; + ExposedPorts = { + "9000/tcp" = { }; + }; + }; +} diff --git a/containers/paperless/Dockerfile b/containers/paperless/Dockerfile deleted file mode 100644 index a7b4e65..0000000 --- a/containers/paperless/Dockerfile +++ /dev/null @@ -1,156 +0,0 @@ -# syntax=docker/dockerfile:1 -# Paperless-ngx — self-hosted document management -# Built from source via forge mirror of paperless-ngx/paperless-ngx -# Closely follows upstream Dockerfile structure with git clone instead of COPY - -ARG CONTAINER_APP_VERSION=v2.20.13 - -############################################### -# Stage 1: Clone source (reused by later stages) -############################################### -FROM docker.io/library/alpine:3.22 AS source - -ARG CONTAINER_APP_VERSION -RUN apk add --no-cache git -RUN git clone --depth 1 --branch ${CONTAINER_APP_VERSION} \ - https://forge.ops.eblu.me/mirrors/paperless-ngx.git /src - -############################################### -# Stage 2: Compile frontend -############################################### -FROM --platform=$BUILDPLATFORM docker.io/node:20-trixie-slim AS compile-frontend - -COPY --from=source /src/src-ui /src/src-ui -WORKDIR /src/src-ui - -RUN set -eux \ - && npm update -g pnpm \ - && npm install -g corepack@latest \ - && corepack enable \ - && pnpm install - -RUN set -eux \ - && ./node_modules/.bin/ng build --configuration production - -############################################### -# Stage 3: s6-overlay base -############################################### -FROM ghcr.io/astral-sh/uv:0.9.15-python3.12-trixie-slim AS s6-overlay-base - -WORKDIR /usr/src/s6 - -ENV S6_BEHAVIOUR_IF_STAGE2_FAILS=2 \ - S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0 \ - S6_VERBOSITY=1 \ - PATH=/command:$PATH - -ARG TARGETARCH -ARG TARGETVARIANT -ARG S6_OVERLAY_VERSION=3.2.1.0 - -RUN set -eux \ - && apt-get update \ - && apt-get install --yes --quiet --no-install-recommends curl xz-utils \ - && S6_ARCH="" \ - && if [ "${TARGETARCH}${TARGETVARIANT}" = "amd64" ]; then S6_ARCH="x86_64"; \ - elif [ "${TARGETARCH}${TARGETVARIANT}" = "arm64" ]; then S6_ARCH="aarch64"; fi \ - && if [ -z "${S6_ARCH}" ]; then echo "Error: Cannot determine arch"; exit 1; fi \ - && curl --fail --silent --show-error --location --remote-name-all --parallel \ - "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz" \ - "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz.sha256" \ - "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${S6_ARCH}.tar.xz" \ - "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${S6_ARCH}.tar.xz.sha256" \ - && sha256sum --check ./*.sha256 \ - && tar --directory / -Jxpf s6-overlay-noarch.tar.xz \ - && tar --directory / -Jxpf s6-overlay-${S6_ARCH}.tar.xz \ - && rm ./*.tar.xz ./*.sha256 \ - && apt-get --yes purge curl xz-utils \ - && apt-get --yes autoremove --purge \ - && rm -rf /var/lib/apt/lists/* - -# Copy rootfs (s6 service definitions, init scripts) -COPY --from=source /src/docker/rootfs / - -############################################### -# Stage 4: Main application -############################################### -FROM s6-overlay-base AS main-app - -ARG CONTAINER_APP_VERSION -ARG DEBIAN_FRONTEND=noninteractive -ARG TARGETARCH -ARG JBIG2ENC_VERSION=0.30 - -ENV PYTHONDONTWRITEBYTECODE=1 \ - PYTHONUNBUFFERED=1 \ - PYTHONWARNINGS="ignore:::django.http.response:517" \ - PNGX_CONTAINERIZED=1 \ - UV_LINK_MODE=copy \ - UV_CACHE_DIR=/cache/uv/ - -# Runtime packages -RUN set -eux \ - && apt-get update \ - && apt-get install --yes --quiet --no-install-recommends \ - curl gosu tzdata fonts-liberation gettext ghostscript gnupg \ - icc-profiles-free imagemagick postgresql-client \ - tesseract-ocr tesseract-ocr-eng tesseract-ocr-deu tesseract-ocr-fra \ - tesseract-ocr-ita tesseract-ocr-spa unpaper pngquant jbig2dec \ - libxml2 libxslt1.1 qpdf file libmagic1 media-types zlib1g \ - libzbar0 poppler-utils \ - && curl --fail --silent --show-error --location --remote-name-all \ - "https://github.com/paperless-ngx/builder/releases/download/jbig2enc-trixie-v${JBIG2ENC_VERSION}/jbig2enc_${JBIG2ENC_VERSION}-1_${TARGETARCH}.deb" \ - && dpkg --install ./jbig2enc_${JBIG2ENC_VERSION}-1_${TARGETARCH}.deb \ - && cp /etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml \ - && rm --force *.deb \ - && rm -rf /var/lib/apt/lists/* - -WORKDIR /usr/src/paperless/src/ - -# Python dependencies -COPY --from=source /src/pyproject.toml /src/uv.lock /usr/src/paperless/src/ - -RUN --mount=type=cache,target=${UV_CACHE_DIR},id=python-cache \ - set -eux \ - && apt-get update \ - && apt-get install --yes --quiet --no-install-recommends \ - build-essential default-libmysqlclient-dev pkg-config \ - && uv export --quiet --no-dev --all-extras --format requirements-txt --output-file requirements.txt \ - && uv pip install --system --no-python-downloads --python-preference system --requirements requirements.txt \ - && python3 -W ignore::RuntimeWarning -m nltk.downloader -d "/usr/share/nltk_data" snowball_data \ - && python3 -W ignore::RuntimeWarning -m nltk.downloader -d "/usr/share/nltk_data" stopwords \ - && python3 -W ignore::RuntimeWarning -m nltk.downloader -d "/usr/share/nltk_data" punkt_tab \ - && apt-get --yes purge build-essential default-libmysqlclient-dev pkg-config \ - && apt-get --yes autoremove --purge \ - && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* - -# Copy backend source -COPY --from=source /src/src ./ - -# Copy compiled frontend -COPY --from=compile-frontend /src/src/documents/static/frontend/ ./documents/static/frontend/ - -# Create user and finalize -RUN set -eux \ - && addgroup --gid 1000 paperless \ - && useradd --uid 1000 --gid paperless --home-dir /usr/src/paperless paperless \ - && mkdir -p /usr/src/paperless/data /usr/src/paperless/media \ - /usr/src/paperless/consume /usr/src/paperless/export \ - && chown -R paperless:paperless /usr/src/paperless \ - && s6-setuidgid paperless python3 manage.py collectstatic --clear --no-input --link \ - && s6-setuidgid paperless python3 manage.py compilemessages - -VOLUME ["/usr/src/paperless/data", "/usr/src/paperless/media", \ - "/usr/src/paperless/consume", "/usr/src/paperless/export"] - -ENTRYPOINT ["/init"] -EXPOSE 8000 - -HEALTHCHECK --interval=30s --timeout=10s --retries=5 \ - CMD [ "curl", "-fs", "-S", "-L", "--max-time", "2", "http://localhost:8000" ] - -LABEL org.opencontainers.image.title="Paperless-ngx" -LABEL org.opencontainers.image.description="Self-hosted document management system" -LABEL org.opencontainers.image.version="${CONTAINER_APP_VERSION}" -LABEL org.opencontainers.image.source="https://forge.eblu.me/eblume/blumeops" -LABEL org.opencontainers.image.vendor="blumeops" diff --git a/containers/paperless/default.nix b/containers/paperless/default.nix new file mode 100644 index 0000000..734d909 --- /dev/null +++ b/containers/paperless/default.nix @@ -0,0 +1,77 @@ +# Nix-built Paperless-ngx for ringtail (amd64). +# +# Replaces the from-source Dockerfile build (s6-overlay) with nixpkgs' +# paperless-ngx, which already bundles the full OCR/imaging closure +# (tesseract, ghostscript, imagemagick, qpdf, poppler, jbig2enc) and the +# NLTK data via wrappers — so the image stays lean. +# +# Unlike the upstream s6 image, this image does NOT run all processes +# itself. Paperless is multi-process; on ringtail it runs as four +# containers sharing this one image, each with a different command: +# web -> paperless-web (granian, the wrapper below) +# worker -> celery --app paperless worker +# beat -> celery --app paperless beat +# consumer -> paperless-ngx document_consumer +# plus a redis/valkey sidecar. The PYTHONPATH/granian invocation mirrors +# the nixpkgs paperless NixOS module's paperless-web service exactly. +# +# Self-pins nixos-unstable: stable nixpkgs lags at 2.19.6, while unstable +# carries 2.20.15 — a same-minor forward patch bump from the previous +# Dockerfile build (v2.20.13). The version assertion makes nix-build fail +# if a pin bump changes the version, forcing an explicit acknowledgment +# here and in service-versions.yaml (enforced by container-version-check). +let + nixpkgs = fetchTarball { + url = "https://github.com/NixOS/nixpkgs/archive/331800de5053fcebacf6813adb5db9c9dca22a0c.tar.gz"; + sha256 = "1p54fm6dkbq62kpi55cr4wyx7b1nsajpsnjgs64cmp073fwi15f7"; + }; + pkgs = import nixpkgs { system = "x86_64-linux"; }; + + version = "2.20.15"; + + app = pkgs.paperless-ngx; + + # Mirror the NixOS module's paperless-web service: granian serving the + # ASGI app with the package's propagated deps + src on PYTHONPATH. + pythonPath = + "${app.python.pkgs.makePythonPath app.propagatedBuildInputs}:${app}/lib/paperless-ngx/src"; + + paperless-web = pkgs.writeShellScriptBin "paperless-web" '' + export PYTHONPATH="${pythonPath}" + export PAPERLESS_NLTK_DIR="${app.nltkDataDir}" + exec ${app.python.pkgs.granian}/bin/granian \ + --interface asginl --ws \ + --host 0.0.0.0 --port 8000 \ + "paperless.asgi:application" + ''; +in + +assert app.version == version; + +pkgs.dockerTools.buildLayeredImage { + name = "blumeops/paperless"; + + contents = [ + app + paperless-web + pkgs.bashInteractive + pkgs.coreutils + pkgs.cacert + pkgs.tzdata + ]; + + config = { + # Default command is the web server; worker/beat/consumer containers + # override `command` in their k8s manifests. + Cmd = [ "${paperless-web}/bin/paperless-web" ]; + Env = [ + "PAPERLESS_NLTK_DIR=${app.nltkDataDir}" + "SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt" + "PYTHONUNBUFFERED=1" + "PNGX_CONTAINERIZED=1" + ]; + ExposedPorts = { + "8000/tcp" = { }; + }; + }; +} diff --git a/containers/teslamate/container.py b/containers/teslamate/container.py deleted file mode 100644 index 519d77d..0000000 --- a/containers/teslamate/container.py +++ /dev/null @@ -1,104 +0,0 @@ -"""TeslaMate — Tesla data logger. - -Two-stage build: Elixir+Node (builder), Debian slim (runtime). -Source cloned from forge mirror. -""" - -import dagger -from dagger import dag - -from blumeops.containers import clone_from_forge, oci_labels - -VERSION = "v3.0.0" - - -async def build(src: dagger.Directory) -> dagger.Container: - source = clone_from_forge("teslamate", VERSION) - - # Stage 1: Build Elixir release with Node.js assets - builder = ( - dag.container() - .from_("elixir:1.19.5-otp-26") - .with_exec( - [ - "bash", - "-c", - "apt-get update" - " && apt-get install -y ca-certificates curl gnupg git zstd brotli" - " && mkdir -p /etc/apt/keyrings" - " && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key" - " | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg" - ' && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg]' - ' https://deb.nodesource.com/node_22.x nodistro main"' - " > /etc/apt/sources.list.d/nodesource.list" - " && apt-get update" - " && apt-get install -y nodejs" - " && apt-get clean" - " && rm -rf /var/lib/apt/lists/*", - ] - ) - .with_exec(["mix", "local.rebar", "--force"]) - .with_exec(["mix", "local.hex", "--force"]) - .with_directory("/opt/app", source) - .with_workdir("/opt/app") - .with_env_variable("MIX_ENV", "prod") - .with_exec(["mix", "deps.get", "--only", "prod"]) - .with_exec(["mix", "deps.compile"]) - .with_exec( - [ - "npm", - "ci", - "--prefix", - "./assets", - "--progress=false", - "--no-audit", - "--loglevel=error", - ] - ) - .with_exec(["mix", "assets.deploy"]) - .with_exec(["mix", "compile"]) - .with_exec( - ["bash", "-c", "SKIP_LOCALE_DOWNLOAD=true mix release --path /opt/built"] - ) - ) - - # Stage 2: Debian slim runtime - entrypoint = src.file("containers/teslamate/entrypoint.sh") - - runtime = ( - dag.container() - .from_("debian:trixie-slim") - .with_exec( - [ - "bash", - "-c", - "apt-get update && apt-get install -y --no-install-recommends" - " libodbc2 libsctp1 libssl3t64 libstdc++6" - " netcat-openbsd tini tzdata" - " && apt-get clean" - " && rm -rf /var/lib/apt/lists/*" - " && groupadd --gid 10001 --system nonroot" - " && useradd --uid 10000 --system --gid nonroot" - " --home-dir /home/nonroot --shell /sbin/nologin nonroot", - ] - ) - ) - runtime = oci_labels( - runtime, - title="TeslaMate", - description="Tesla data logger and visualization", - version=VERSION, - ) - return ( - runtime.with_env_variable("LANG", "C.UTF-8") - .with_env_variable("SRTM_CACHE", "/opt/app/.srtm_cache") - .with_env_variable("HOME", "/opt/app") - .with_workdir("/opt/app") - .with_directory("/opt/app", builder.directory("/opt/built"), owner="nonroot") - .with_exec(["mkdir", "-p", "/opt/app/.srtm_cache"]) - .with_file("/entrypoint.sh", entrypoint, permissions=0o555, owner="nonroot") - .with_user("nonroot") - .with_exposed_port(4000) - .with_entrypoint(["tini", "--", "/bin/dash", "/entrypoint.sh"]) - .with_default_args(args=["bin/teslamate", "start"]) - ) diff --git a/containers/teslamate/default.nix b/containers/teslamate/default.nix new file mode 100644 index 0000000..e126561 --- /dev/null +++ b/containers/teslamate/default.nix @@ -0,0 +1,122 @@ +# Nix-built TeslaMate for ringtail (amd64). +# +# Replaces the Dagger container.py (Elixir+Node builder -> Debian slim). +# TeslaMate is NOT in nixpkgs, so this is a from-scratch beamPackages +# mixRelease: an Elixir/Phoenix release with npm-built assets. +# +# Pinned to the same nixos-unstable rev as paperless/mealie for a +# consistent toolchain. The BEAM combo is pinned to erlang_27 + elixir_1_18 +# (teslamate requires elixir ~> 1.17; upstream's image uses OTP 26, so we +# stay off the default OTP 28 which elixir 1.18 does not target). +# +# Source comes from the forge mirror (supply-chain control), pinned by the +# v3.0.0 tag's commit so builtins.fetchGit needs no hash. +let + nixpkgs = fetchTarball { + url = "https://github.com/NixOS/nixpkgs/archive/331800de5053fcebacf6813adb5db9c9dca22a0c.tar.gz"; + sha256 = "1p54fm6dkbq62kpi55cr4wyx7b1nsajpsnjgs64cmp073fwi15f7"; + }; + pkgs = import nixpkgs { system = "x86_64-linux"; }; + lib = pkgs.lib; + + version = "3.0.0"; + + beamPackages = pkgs.beam.packages.erlang_27; + elixir = beamPackages.elixir_1_18; + + src = builtins.fetchGit { + url = "https://forge.ops.eblu.me/mirrors/teslamate.git"; + ref = "refs/tags/v${version}"; + rev = "3281154d42330786a182c1bbe094ecda0b1c5578"; + }; + + # ex_cldr downloads locale JSON from GitHub at compile time, which the + # build sandbox blocks. teslamate's cldr.ex reads the data dir from the + # LOCALES env var; point it at the pre-fetched elixir-cldr data so no + # download is attempted (with SKIP_LOCALE_DOWNLOAD=true disabling the + # forced refresh). CLDR data version matches the compile-time errors. + cldrData = pkgs.fetchFromGitHub { + owner = "elixir-cldr"; + repo = "cldr"; + rev = "v2.46.0"; + sha256 = "1iwzk9dc754l72vpf8vsisdjncnjx26pz509552b6vnm49xbxyji"; + }; + + teslamate = beamPackages.mixRelease { + pname = "teslamate"; + inherit version src elixir; + + # Keep the build-generated Erlang cookie in the release. mixRelease + # strips it by default (expecting RELEASE_COOKIE at runtime), but the + # start script reads releases/COOKIE. teslamate is single-node (no + # distributed Erlang exposed), so a baked-in cookie is fine. + removeCookie = false; + + mixFodDeps = beamPackages.fetchMixDeps { + pname = "mix-deps-teslamate"; + inherit src version elixir; + hash = "sha256-DDrREiM1BIMgD2qFPTK8QyjOYlnfE3XlnaH/jk7G2go="; + }; + + # Frontend assets. esbuild + sass are devDeps and the esbuild platform + # binary is an optional dep, so npm ci must include both. We run npm ci + # here (not a separate derivation) because assets/package.json has + # file:../deps/phoenix references that only resolve once mixFodDeps has + # populated deps/. npmConfigHook wires up the offline cache from npmDeps; + # then `node scripts/build.js` (custom esbuild) + `mix phx.digest`. + nativeBuildInputs = [ pkgs.nodejs pkgs.npmHooks.npmConfigHook ]; + npmDeps = pkgs.fetchNpmDeps { + name = "teslamate-npm-deps"; + src = src + "/assets"; + hash = "sha256-XyiaUkT/c4rZnNxmxhVLb+vEXnc64A1hjOrnR5fhaEk="; + }; + npmRoot = "assets"; + + preBuild = '' + export SKIP_LOCALE_DOWNLOAD=true + export LOCALES=${cldrData}/priv/cldr + ( cd assets && npm ci --include=dev --include=optional && node scripts/build.js ) + mix phx.digest --no-deps-check + ''; + }; +in + +pkgs.dockerTools.buildLayeredImage { + name = "blumeops/teslamate"; + + contents = [ + teslamate + pkgs.bashInteractive + pkgs.coreutils + pkgs.dash + pkgs.netcat-openbsd + pkgs.cacert + pkgs.tzdata + ]; + + config = { + # Mirror entrypoint.sh: wait for postgres, run migrations, then start. + Entrypoint = [ + "${pkgs.dash}/bin/dash" + "-c" + '' + : "''${DATABASE_HOST:=127.0.0.1}" + : "''${DATABASE_PORT:=5432}" + while ! ${pkgs.netcat-openbsd}/bin/nc -z "$DATABASE_HOST" "$DATABASE_PORT" 2>/dev/null; do + echo "waiting for postgres at $DATABASE_HOST:$DATABASE_PORT"; sleep 1 + done + ${teslamate}/bin/teslamate eval "TeslaMate.Release.migrate" + exec ${teslamate}/bin/teslamate start + '' + ]; + Env = [ + "HOME=/opt/app" + "SRTM_CACHE=/opt/app/.srtm_cache" + "LANG=C.UTF-8" + "SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt" + ]; + ExposedPorts = { + "4000/tcp" = { }; + }; + }; +} diff --git a/containers/teslamate/entrypoint.sh b/containers/teslamate/entrypoint.sh deleted file mode 100644 index f66117e..0000000 --- a/containers/teslamate/entrypoint.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env dash -set -e - -: "${DATABASE_HOST:="127.0.0.1"}" -: "${DATABASE_PORT:=5432}" -: "${ULIMIT_MAX_NOFILE:=65536}" - -# prevent memory bloat in some misconfigured versions of Docker/containerd -# where the nofiles limit is very large. 0 means don't set it. -if test "${ULIMIT_MAX_NOFILE}" != 0 && test "$(ulimit -n)" -gt "${ULIMIT_MAX_NOFILE}"; then - ulimit -n "${ULIMIT_MAX_NOFILE}" -fi - -# wait until Postgres is ready -while ! nc -z "${DATABASE_HOST}" "${DATABASE_PORT}" 2>/dev/null; do - echo waiting for postgres at "${DATABASE_HOST}":"${DATABASE_PORT}" - sleep 1s -done - -# apply migrations -bin/teslamate eval "TeslaMate.Release.migrate" - -exec "$@" diff --git a/docs/changelog.d/migrate-wave1-ringtail.infra.md b/docs/changelog.d/migrate-wave1-ringtail.infra.md new file mode 100644 index 0000000..c44263a --- /dev/null +++ b/docs/changelog.d/migrate-wave1-ringtail.infra.md @@ -0,0 +1,13 @@ +Move paperless, teslamate, and mealie off `minikube-indri` onto +`k3s-ringtail`, shedding ~1.1 GiB of resident load from the +OOM-thrashing 8 GiB minikube node (the kernel OOM killer had been +killing `kube-apiserver`/`dockerd`/argocd, flapping every +minikube-hosted service at once). paperless + teslamate databases +move into a fresh CNPG `blumeops-pg` cluster on ringtail via a cold +`pg_dump`/`pg_restore` from the quiesced source — row counts verified +equal before any routing flip; source DBs dropped only after the +ringtail side serves traffic. mealie's SQLite PVC is copied as-is. +paperless media stays on sifaka NFS. Downtime-tolerant cold cutover +(no streaming replication); rollback is repoint-and-scale-up with the +source untouched. Second chain in the indri-k8s decommission after +[[migrate-immich-to-ringtail]]. diff --git a/docs/how-to/immich/migrate-immich-to-ringtail.md b/docs/how-to/immich/migrate-immich-to-ringtail.md index cd23384..e654b62 100644 --- a/docs/how-to/immich/migrate-immich-to-ringtail.md +++ b/docs/how-to/immich/migrate-immich-to-ringtail.md @@ -122,6 +122,8 @@ file). ## Related +- [[migrate-wave1-ringtail]] — the next chain in the indri-k8s + decommission: paperless, teslamate, and mealie - [[shower-on-ringtail]] — a previous migration to ringtail (simpler: no upstream cluster, SQLite, no GPU) - [[connect-to-postgres]] — getting a psql session against CNPG diff --git a/docs/how-to/ringtail/migrate-wave1-ringtail.md b/docs/how-to/ringtail/migrate-wave1-ringtail.md new file mode 100644 index 0000000..ffb8cdc --- /dev/null +++ b/docs/how-to/ringtail/migrate-wave1-ringtail.md @@ -0,0 +1,176 @@ +--- +title: Migrate Wave 1 (paperless, teslamate, mealie) to Ringtail +modified: 2026-06-03 +last-reviewed: 2026-06-03 +tags: + - how-to + - operations + - ringtail + - migration +--- + +# Migrate Wave 1 to Ringtail + +Move paperless, teslamate, and mealie off `minikube-indri` and onto +`k3s-ringtail`. This is the load-shedding response to minikube going +OOM: the kernel OOM killer was thrashing the 8 GiB node — killing +`kube-apiserver`, `dockerd`, and the argocd application-controller — +which made every minikube-hosted service probe-flap at once. These +three app pods are ~1.1 GiB resident combined and are the heaviest +non-observability tenants left on minikube. Following +[[migrate-immich-to-ringtail]], the first chain in the indri-k8s +decommission. + +## End state + +- `paperless`, `teslamate`, and `mealie` run on ringtail k3s in their + own namespaces, off minikube entirely. +- A CNPG `blumeops-pg` Cluster runs in a `databases` namespace on + ringtail (PostgreSQL, owned by ringtail's `cnpg-system` operator), + holding the `paperless` and `teslamate` databases. Apps reach it + in-cluster via `blumeops-pg-rw.databases.svc.cluster.local`. +- mealie keeps its SQLite database; its 2 GiB `mealie-data` PVC is + copied to a ringtail PVC. +- paperless media still lives on [[sifaka]] via NFS (RWX, 500 GiB), + mounted from ringtail pods. teslamate has no file state. +- Routing: `paperless.ops.eblu.me`, `teslamate.ops.eblu.me`, and + `mealie.ops.eblu.me` (Caddy on indri) proxy to Tailscale + ProxyGroup ingresses on ringtail. Service names are unchanged. +- The minikube manifests and the `paperless`/`teslamate`/`mealie` + databases inside indri's `blumeops-pg` are removed only after + cutover is verified. + +## Non-goals + +- Migrating the rest of `blumeops-pg` (e.g. miniflux) — that is a + later wave. This chain moves only the paperless + teslamate + databases out; the source cluster on indri stays up for the others. +- Version bumps or config changes. Lift-and-shift only. +- Public (Fly) exposure changes. These stay tailnet-only. +- The observability stack (prometheus/loki/tempo/grafana) — deferred; + it carries 50 GiB of local TSDB and is the riskiest move. + +## Critical constraint: no data loss + +**Downtime is acceptable — data loss is not.** We can take each +service fully offline for its cutover, which removes the entire +class of streaming-replication and double-writer hazards. The cold +dump is taken from a *quiesced* source, so it is internally +consistent. + +Data surfaces: + +1. **paperless postgres** — document metadata, tags, correspondents, + the search index state. The document *files* are on NFS and never + move, but losing the DB means files-without-index. This is the + surface to protect most carefully. +2. **teslamate postgres** — drive/charge history. Re-derivable only + from Tesla's API for a limited window; treat as unrecoverable. +3. **mealie SQLite** — recipes, meal plans. On the `mealie-data` PVC. + +The source databases on indri are **never dropped until the ringtail +side is verified and serving**. Rollback is "repoint and scale back +up," not "restore from backup." [[borgmatic]] remains the backstop. + +## Why a fresh CNPG cluster (not cross-cluster pg) + +indri's `blumeops-pg` is already exposed tailnet-wide at +`pg.ops.eblu.me` (Caddy L4), so we *could* leave the DBs on indri and +just move the app pods. We are not, because: + +- The goal is to retire minikube — keeping pg there blocks it and + leaves a cross-host runtime dependency (ringtail apps SPOF on + indri's pg over the tailnet). +- CNPG is the same operator on both clusters; a Cluster CR on ringtail + is mechanically equivalent to the one on minikube. +- Naming the ringtail cluster `blumeops-pg` in `databases` lets apps + use the same in-cluster DNS they would on indri. + +## Cold-cutover procedure (per service) + +Do these one service at a time. paperless first (heaviest, highest +data-sensitivity), then teslamate, then mealie. + +### 0. Prerequisites (once, before any service) + +- Confirm ringtail's `cnpg-system` operator and `databases` namespace + are healthy (immich-pg already runs there). +- Confirm ringtail pods can reach indri's `pg.ops.eblu.me:5432` (used + only to pull the dump) and the sifaka NFS export for paperless + media. See [[sifaka-nfs-from-ringtail]]. +- Define the ringtail `blumeops-pg` CNPG Cluster manifest (model on + `databases-ringtail/immich-pg.yaml`) and its ExternalSecrets for + the per-app roles. Sync it; let it come up empty and healthy. + +### 1. Quiesce the source + +```fish +kubectl --context=minikube-indri -n scale deploy/ --replicas=0 +# confirm 0 running, DB now has no writers +``` + +### 2. Dump from indri, restore to ringtail (postgres apps) + +```fish +# dump the single app DB from the quiesced source +kubectl --context=minikube-indri -n databases exec blumeops-pg-1 -- \ + pg_dump -Fc -d > /tmp/.dump + +# restore into the ringtail cluster +kubectl --context=k3s-ringtail -n databases exec -i blumeops-pg-1 -- \ + pg_restore --no-owner --role= -d < /tmp/.dump +``` + +For **mealie** (SQLite) instead: copy the `mealie-data` PVC contents +to the ringtail PVC (e.g. a one-shot rsync pod mounting both, or +`kubectl cp` via a helper pod). Verify the `.db` file size and that +mealie boots read-only against it. + +### 3. Verify the restore (before any routing flips) + +- Row counts match source for the key tables, scripted: + - paperless: `documents_document`, `documents_tag`, + `documents_correspondent`, `auth_user`. + - teslamate: `cars`, `drives`, `charging_processes`, `positions`. +- `pg_dump --schema-only --no-owner` diff between source and dest is + empty modulo CNPG-managed roles. +- Boot the app against the ringtail DB on its tailnet name *before* + Caddy is flipped, and smoke-test (paperless: documents list + + search; teslamate: dashboard loads recent drives; mealie: recipes + list). + +### 4. Release the service name + +```fish +# delete the minikube tailscale ingress so ringtail can claim the name +kubectl --context=minikube-indri -n delete ingress -tailscale +``` + +### 5. Bring up on ringtail + +- Apply the ringtail manifests (new ArgoCD app `-ringtail`, + `destination.server` = `https://ringtail.tail8d86e.ts.net:6443`). + App points at `blumeops-pg-rw.databases.svc.cluster.local`. +- Sync; wait for healthy + the ProxyGroup ingress to get its name. + +### 6. Flip routing + +- Repoint the Caddy `.ops.eblu.me` upstream at the ringtail + ProxyGroup ingress (provision-indri, caddy role). +- `mise run services-check` — confirm the service flips from FIRING + to OK and no neighbours regressed. + +### 7. Decommission the source (only after verification) + +- Remove the minikube manifests for the app. +- Drop the app DB from indri's `blumeops-pg` (paperless/teslamate) + **last**, once the ringtail side has served real traffic. + +## Rollback + +If a cutover fails verification at any step before §7: + +- Re-create the minikube tailscale ingress (if §4 ran). +- Scale the minikube app back to `1`. +- Repoint Caddy back to the minikube ingress. +- The source DB was never modified or dropped. Document the failure. diff --git a/service-versions.yaml b/service-versions.yaml index 5440f01..699f89c 100644 --- a/service-versions.yaml +++ b/service-versions.yaml @@ -47,7 +47,7 @@ services: - name: shower type: argocd last-reviewed: 2026-05-15 - current-version: "1.1.2" + current-version: "1.1.3" upstream-source: https://forge.eblu.me/eblume/adelaide-baby-shower-app notes: | Django app for Adelaide / Heidi / Addie's baby shower. Wheel @@ -222,9 +222,17 @@ services: - name: teslamate type: argocd - last-reviewed: 2026-04-14 + last-reviewed: "2026-06-03" current-version: "v3.0.0" upstream-source: https://github.com/teslamate-org/teslamate/releases + notes: >- + Tesla data logger. Container ported from Dagger (container.py) to Nix + (containers/teslamate/default.nix) — a from-scratch beamPackages + mixRelease (Elixir/Phoenix release with npm-built assets), since + teslamate is not in nixpkgs. Pins erlang_27 + elixir_1_18 from the + shared nixos-unstable rev; assets via in-release npm ci + esbuild; + ex_cldr locale data pre-fetched (LOCALES env) to avoid sandbox + downloads. Version unchanged (v3.0.0). Build verified on ringtail. - name: transmission type: argocd @@ -328,21 +336,31 @@ services: - name: mealie type: argocd - last-reviewed: 2026-05-11 - current-version: "v3.12.0" + last-reviewed: "2026-06-03" + current-version: "v3.16.0" upstream-source: https://github.com/mealie-recipes/mealie/releases notes: >- - Recipe manager; built from source via forge mirror. - Upstream is at v3.17.0 as of 2026-05-11 (5 minor versions ahead). - Container/manifest still pinned to v3.12.0 — upgrade deferred to a - separate task (build new image, review changelog for breaking changes). + Recipe manager. Container ported from Dockerfile to Nix + (containers/mealie/default.nix wraps nixpkgs mealie from a pinned + nixos-unstable; single gunicorn process, SQLite on the mealie-data + PVC). Bumped v3.12.0 -> v3.16.0 as part of the port (the deferred + upgrade). Breaking-change review v3.13-v3.16: no schema breaking + changes, SQLite auto-migrates forward via init_db; notable items are + minor (OIDC missing-claims log -> DEBUG, NLP parser uses user-defined + units, Nuxt 3->4 frontend, new Announcements feature, path-traversal + patches). Source PVC retained for rollback. Build verified on ringtail. - name: paperless type: argocd - last-reviewed: "2026-04-08" - current-version: "v2.20.13" + last-reviewed: "2026-06-03" + current-version: "v2.20.15" upstream-source: https://github.com/paperless-ngx/paperless-ngx/releases - notes: Document management; built from source via forge mirror + notes: >- + Document management. Container ported from Dockerfile to Nix + (containers/paperless/default.nix wraps nixpkgs paperless-ngx from a + pinned nixos-unstable). Runs as web/worker/beat/consumer containers on + ringtail (multi-process; no s6). Bumped v2.20.13 -> v2.20.15 (the + unstable package version, same-minor patch) as part of the port. - name: unpoller type: argocd