From 8498dbb559f5860e81612cac7fa3ed0b166653c2 Mon Sep 17 00:00:00 2001 From: Erich Blume Date: Wed, 29 Apr 2026 13:30:06 -0700 Subject: [PATCH] docs: refresh devpi reference card and scrub stale references MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - docs/reference/services/devpi.md: rewrite for the new launchd layout (no more namespace/PVC/ArgoCD/image/ExternalSecret) and link to the new how-to. - docs/reference/infrastructure/tailscale.md: drop tag:devpi from the per-service Ingress tag list. - docs/reference/storage/backups.md: clarify the devpi-cache row to call out the new on-indri path. - docs/how-to/operations/rebuild-minikube-cluster.md: trim the "Devpi cold cache after rebuild" section — devpi is no longer in minikube — keep a brief note for the still-possible cold-cache race after fresh devpi init. - docs/how-to/operations/devpi-on-indri.md: correct the Backup section — server-dir is NOT in borgmatic_source_directories. - pulumi/tailscale: remove now-dead tag:devpi ACL rule, tagOwner, and __main__.py comment. Will need `pulumi up` to apply. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/how-to/operations/devpi-on-indri.md | 2 +- .../operations/rebuild-minikube-cluster.md | 20 +---------- docs/reference/infrastructure/tailscale.md | 2 +- docs/reference/services/devpi.md | 34 +++++++++++-------- docs/reference/storage/backups.md | 2 +- pulumi/tailscale/__main__.py | 2 +- pulumi/tailscale/policy.hujson | 9 ++--- 7 files changed, 27 insertions(+), 44 deletions(-) diff --git a/docs/how-to/operations/devpi-on-indri.md b/docs/how-to/operations/devpi-on-indri.md index 85d4cbf..0334d37 100644 --- a/docs/how-to/operations/devpi-on-indri.md +++ b/docs/how-to/operations/devpi-on-indri.md @@ -66,7 +66,7 @@ Edit `devpi_server_version` / `devpi_web_version` in `ansible/roles/devpi/defaul ## Backup -The server-dir is included in the borgmatic file backup that already covers `~erichblume/`. Devpi indexes are recoverable from upstream PyPI on a fresh start; only the local index (`root/dev`) is unique state. +The server-dir is **not** in `borgmatic_source_directories` and is not backed up. The PyPI cache (`+files/`) is re-fetchable from upstream on first request; the local `eblume/dev` index can be republished from source. If retention becomes important, add `/Users/erichblume/devpi/server-dir/` to the borgmatic source list. ## Related diff --git a/docs/how-to/operations/rebuild-minikube-cluster.md b/docs/how-to/operations/rebuild-minikube-cluster.md index e23d027..0d924e9 100644 --- a/docs/how-to/operations/rebuild-minikube-cluster.md +++ b/docs/how-to/operations/rebuild-minikube-cluster.md @@ -235,25 +235,7 @@ mise run services-check ## Post-Rebuild: Cold Cache Failures -### Devpi (PyPI Cache) - -After a rebuild, devpi's package cache is empty. The first Dagger-based container build will trigger a flood of concurrent package downloads. Devpi uses lazy caching — it serves package metadata (simple index) immediately from upstream PyPI but fetches wheel files on demand. Under heavy concurrent load with a cold cache, the upstream fetch can race with the client request, causing devpi to return `no such file` (HTTP 404) for packages it knows about but hasn't finished downloading yet. - -**Why devpi, not PyPI?** The repo's `uv.lock` was generated with devpi as the index, so every package source URL points at `pypi.ops.eblu.me`. Dagger's Python SDK runtime does a locked install (`uv sync`), not fresh resolution — it fetches from whatever URLs are in the lockfile. This is intentional (supply chain control), but means all builds — local and CI — depend on devpi being available and warm. - -**Symptoms:** Forgejo Actions Dagger builds fail during module initialization with errors like: -``` -Failed to download `googleapis-common-protos==1.74.0` -HTTP status client error (404 Not Found) for url (https://pypi.ops.eblu.me/root/pypi/+f/...) -``` - -**Fix:** Re-run the failed build. The first attempt warms the cache; subsequent builds succeed. Alternatively, warm the cache manually before triggering CI builds: - -```bash -# From any machine that can reach pypi.ops.eblu.me, install the Dagger SDK -# to pre-populate the most common packages: -pip install --dry-run --index-url https://pypi.ops.eblu.me/root/pypi/+simple/ dagger-io -``` +Devpi runs natively on indri (see [[devpi-on-indri]]) and is unaffected by minikube rebuilds, so the historical "devpi cold cache after rebuild" failure mode no longer applies. If devpi itself goes cold (fresh server-dir), the same lazy-cache race can still cause `404` on the first Dagger build under concurrent load — re-run the build to warm the cache, or pre-warm with `uv pip install --dry-run --index-url https://pypi.ops.eblu.me/root/pypi/+simple/ dagger-io`. ## Related diff --git a/docs/reference/infrastructure/tailscale.md b/docs/reference/infrastructure/tailscale.md index 2794111..9c15d83 100644 --- a/docs/reference/infrastructure/tailscale.md +++ b/docs/reference/infrastructure/tailscale.md @@ -33,7 +33,7 @@ ACLs managed via Pulumi in `pulumi/tailscale/policy.hujson`. | `tag:loki` | indri | Loki log aggregation | | `tag:k8s-api` | indri | Kubernetes API server (minikube) | | `tag:k8s-operator` | (operator pod) | Tailscale operator for k8s — see [[tailscale-operator]] | -| `tag:k8s` | (Ingress proxy pods) | Kubernetes Tailscale Ingress nodes; each also carries a per-service tag (`tag:grafana`, `tag:kiwix`, `tag:devpi`, `tag:feed`, `tag:pg`) | +| `tag:k8s` | (Ingress proxy pods) | Kubernetes Tailscale Ingress nodes; each also carries a per-service tag (`tag:grafana`, `tag:kiwix`, `tag:feed`, `tag:pg`) | | `tag:ci-gateway` | (ephemeral CI containers) | CI containers pushing images to registry | | `tag:flyio-proxy` | (Fly.io proxy container) | Public reverse proxy | | `tag:flyio-target` | indri, designated Ingress endpoints | Endpoints reachable by the Fly.io proxy (indri for Caddy routing, Ingress pods for Alloy metrics/logs) | diff --git a/docs/reference/services/devpi.md b/docs/reference/services/devpi.md index c6493fe..589a802 100644 --- a/docs/reference/services/devpi.md +++ b/docs/reference/services/devpi.md @@ -1,7 +1,7 @@ --- title: Devpi -modified: 2026-03-23 -last-reviewed: 2026-03-23 +modified: 2026-04-29 +last-reviewed: 2026-04-29 tags: - service - python @@ -9,31 +9,37 @@ tags: # devpi (PyPI Proxy) -PyPI caching proxy and private package index. +PyPI caching proxy and private package index. Runs natively on [[indri]] as a LaunchAgent (not in-cluster). See [[devpi-on-indri]] for deploy and operations. ## Quick Reference | Property | Value | |----------|-------| -| **URL** | https://pypi.ops.eblu.me | -| **Namespace** | `devpi` | -| **ArgoCD App** | `devpi` | -| **Storage** | 50Gi PVC | -| **Image** | `registry.ops.eblu.me/blumeops/devpi` (see `argocd/manifests/devpi/kustomization.yaml` for current tag) | +| **URL** | `https://pypi.ops.eblu.me` | +| **Listen** | `127.0.0.1:3141` (loopback only; reached via Caddy) | +| **Service** | LaunchAgent `mcquack.eblume.devpi` on indri | +| **Server-dir** | `/Users/erichblume/devpi/server-dir/` | +| **Runtime** | uv-managed venv at `/Users/erichblume/devpi/venv/` | +| **Ansible role** | `ansible/roles/devpi/` | +| **Versions** | Pinned in `ansible/roles/devpi/defaults/main.yml` (`devpi_server_version`, `devpi_web_version`) | ## Indices | Index | Purpose | |-------|---------| -| `root/pypi` | PyPI mirror/cache (auto-created) | -| `eblume/dev` | Private packages (inherits from root/pypi) | +| `root/pypi` | PyPI mirror/cache (auto-created by `devpi-init`) | +| `eblume/dev` | Private packages (inherits from `root/pypi`) | ## Credentials -Root password stored in 1Password (blumeops vault), injected via ExternalSecret. +Root password stored in 1Password (`blumeops` vault, item `devpi`, field `root password`). Fetched via `op read` in the `ansible/playbooks/indri.yml` `pre_tasks` and passed to the role on first init. + +## Backup + +The server-dir is **not** backed up. The PyPI cache (`+files/`) is re-fetchable from upstream on first request. The local `eblume/dev` index metadata is small but also not critical to retain — packages can be republished from source. If retention becomes important, add `/Users/erichblume/devpi/server-dir/` to `borgmatic_source_directories`. ## Related -- [[use-pypi-proxy]] - Client configuration and package uploads -- [[argocd]] - Deployment -- [[1password]] - Secrets management +- [[devpi-on-indri]] — Deploy, verify, and version-bump procedures +- [[use-pypi-proxy]] — Client configuration and package uploads +- [[1password]] — Secrets management diff --git a/docs/reference/storage/backups.md b/docs/reference/storage/backups.md index 9ca3bcb..14dbcea 100644 --- a/docs/reference/storage/backups.md +++ b/docs/reference/storage/backups.md @@ -62,7 +62,7 @@ Other data lives directly on [[sifaka]] (music via [[navidrome]], video via [[je | ZIM archives (`~/transmission/`) | Re-downloadable via torrent | | Prometheus metrics | Ephemeral, in k8s PVC | | Loki logs | Ephemeral, in k8s PVC | -| devpi cache | Re-fetchable from PyPI | +| devpi cache (`~/devpi/server-dir/` on indri) | Re-fetchable from PyPI on first request | ## Retention Policy diff --git a/pulumi/tailscale/__main__.py b/pulumi/tailscale/__main__.py index 2f5262b..3acbb62 100644 --- a/pulumi/tailscale/__main__.py +++ b/pulumi/tailscale/__main__.py @@ -37,7 +37,7 @@ acl = tailscale.Acl( # indri - Mac Mini M1, primary homelab server # Hosts forge, loki, zot registry, and the k8s control plane. -# Other services (grafana, kiwix, devpi, etc.) run in k8s with their own Tailscale devices. +# Other services (grafana, kiwix, etc.) run in k8s with their own Tailscale devices. indri = tailscale.get_device(name="indri.tail8d86e.ts.net") indri_tags = tailscale.DeviceTags( "indri-tags", diff --git a/pulumi/tailscale/policy.hujson b/pulumi/tailscale/policy.hujson index 84f1f17..88408ef 100644 --- a/pulumi/tailscale/policy.hujson +++ b/pulumi/tailscale/policy.hujson @@ -20,7 +20,8 @@ }, // --- Members: user-facing services only --- - // Kiwix, Forge, devpi, Miniflux, PostgreSQL + // Kiwix, Forge, Miniflux, PostgreSQL + // (devpi moved off-cluster to indri; reachable via Caddy on tag:flyio-target) { "src": ["autogroup:member"], "dst": ["tag:kiwix"], @@ -31,11 +32,6 @@ "dst": ["tag:forge"], "ip": ["tcp:443", "tcp:22"], }, - { - "src": ["autogroup:member"], - "dst": ["tag:devpi"], - "ip": ["tcp:443"], - }, { "src": ["autogroup:member"], "dst": ["tag:feed"], @@ -152,7 +148,6 @@ "tag:grafana": ["autogroup:admin", "tag:blumeops"], "tag:kiwix": ["autogroup:admin", "tag:blumeops"], "tag:forge": ["autogroup:admin", "tag:blumeops"], - "tag:devpi": ["autogroup:admin", "tag:blumeops"], "tag:loki": ["autogroup:admin", "tag:blumeops"], "tag:pg": ["autogroup:admin", "tag:blumeops"], "tag:feed": ["autogroup:admin", "tag:blumeops"],