diff --git a/plans/k8s-migration/P5_devpi.md b/plans/k8s-migration/P5_devpi.md index 7f24b7b..b5ea9d2 100644 --- a/plans/k8s-migration/P5_devpi.md +++ b/plans/k8s-migration/P5_devpi.md @@ -1,37 +1,190 @@ -# Phase 5: devpi Migration +# Phase 5: devpi Migration to Kubernetes -**Goal**: Migrate devpi to k8s +**Goal**: Migrate devpi PyPI caching proxy from indri to k8s -**Status**: Pending +**Status**: In Progress (2026-01-20) -**Prerequisites**: [Phase 4](P4_miniflux.md) complete +**Prerequisites**: [Phase 4](P4_miniflux.complete.md) complete + +--- + +## Overview + +This phase migrates devpi from mcquack LaunchAgent on indri to Kubernetes: +1. Stop devpi on indri and clear Tailscale service +2. Build custom devpi container image (devpi-server + devpi-web) +3. Deploy as StatefulSet with PVC for data persistence +4. Expose via Tailscale Ingress at `pypi.tail8d86e.ts.net` +5. Initialize fresh (no data migration - just cache and one private package) +6. Validate with pip proxy and mcquack package upload + +**Decision: Fresh start** - The only private package is mcquack v1.0.0 which we'll re-upload. The PyPI cache is re-fetchable. --- ## Steps -### 1. Build devpi container +### 1. Stop devpi on indri (FIRST) -- Dockerfile with devpi-server + devpi-web -- Push to local Zot registry +```bash +# Stop and unload the LaunchAgent +ssh indri 'launchctl unload ~/Library/LaunchAgents/mcquack.eblume.devpi.plist' + +# Also stop the metrics collector +ssh indri 'launchctl unload ~/Library/LaunchAgents/mcquack.eblume.devpi-metrics.plist' + +# Clear any tailscale serve entries (if any) +ssh indri 'tailscale serve off --service=svc:pypi' 2>/dev/null || true +``` + +**User action required**: Delete the `pypi` device/service from Tailscale admin console. + +### 2. Build devpi container image + +Create `argocd/manifests/devpi/Dockerfile`: + +```dockerfile +FROM python:3.12-slim + +# Install devpi-server and devpi-web +RUN pip install --no-cache-dir devpi-server devpi-web + +# Create data directory +RUN mkdir -p /devpi + +# Expose default port +EXPOSE 3141 + +# Use ENTRYPOINT for flexibility +ENTRYPOINT ["devpi-server"] + +# Default args (can be overridden) +CMD ["--serverdir", "/devpi", "--host", "0.0.0.0", "--port", "3141"] +``` + +Build and push to Zot: + +```bash +# From gilbert (has podman) +cd argocd/manifests/devpi +podman build -t registry.tail8d86e.ts.net/blumeops/devpi:latest . +podman push registry.tail8d86e.ts.net/blumeops/devpi:latest +``` + +### 3. Create k8s manifests + +| File | Purpose | +|------|---------| +| `argocd/apps/devpi.yaml` | ArgoCD Application definition | +| `argocd/manifests/devpi/statefulset.yaml` | StatefulSet with PVC | +| `argocd/manifests/devpi/service.yaml` | ClusterIP Service | +| `argocd/manifests/devpi/ingress-tailscale.yaml` | Tailscale Ingress for `pypi.tail8d86e.ts.net` | +| `argocd/manifests/devpi/init-job.yaml` | One-time initialization job | +| `argocd/manifests/devpi/kustomization.yaml` | Kustomize configuration | + +### 4. Deploy via ArgoCD + +```bash +# Point app at feature branch for testing +argocd app create devpi \ + --repo ssh://forgejo@indri.tail8d86e.ts.net:2200/eblume/blumeops.git \ + --path argocd/manifests/devpi \ + --dest-server https://kubernetes.default.svc \ + --dest-namespace devpi \ + --revision feature/p5-devpi + +# Or add to apps/ and let app-of-apps sync +argocd app sync apps +argocd app sync devpi +``` + +### 5. Initialize devpi + +After StatefulSet is running: + +```bash +# Exec into pod to initialize +kubectl -n devpi exec -it devpi-0 -- devpi-init --serverdir /devpi --root-passwd + +# Or use init job with secret +``` + +Create user and index: + +```bash +# From workstation +uvx devpi use https://pypi.tail8d86e.ts.net +uvx devpi login root +uvx devpi user -c eblume email=blume.erich@gmail.com +uvx devpi index -c eblume/dev bases=root/pypi +``` + +### 6. Update pip.conf on gilbert + +The existing `~/.config/pip/pip.conf` should work unchanged since we're keeping the same hostname `pypi.tail8d86e.ts.net`. + +### 7. Remove devpi ansible role + +- Remove `devpi` from `ansible/playbooks/indri.yml` roles list +- Remove `devpi_metrics` from alloy textfile collectors +- Keep devpi ansible role directory for reference (or delete) + +### 8. Validate + +```bash +# Test pip cache proxy +pip install --index-url https://pypi.tail8d86e.ts.net/root/pypi/+simple/ requests + +# Upload mcquack +cd ~/code/personal/mcquack +uv build +uv publish --publish-url https://pypi.tail8d86e.ts.net/eblume/dev/ +``` --- -### 2. Deploy as StatefulSet +## New Files -- PVC for data (50Gi) -- Migrate existing data (excluding PyPI cache) +| Path | Purpose | +|------|---------| +| `argocd/apps/devpi.yaml` | ArgoCD Application definition | +| `argocd/manifests/devpi/Dockerfile` | Container image build | +| `argocd/manifests/devpi/statefulset.yaml` | StatefulSet with PVC | +| `argocd/manifests/devpi/service.yaml` | ClusterIP Service | +| `argocd/manifests/devpi/ingress-tailscale.yaml` | Tailscale Ingress | +| `argocd/manifests/devpi/kustomization.yaml` | Kustomize configuration | +| `argocd/manifests/devpi/README.md` | Setup documentation | + +## Modified Files + +| Path | Change | +|------|--------| +| `ansible/playbooks/indri.yml` | Remove devpi role | +| `ansible/roles/alloy/defaults/main.yml` | Remove devpi textfile collector | +| `ansible/roles/borgmatic/defaults/main.yml` | Remove devpi backup (no longer on indri) | + +## Deleted Files + +| Path | Reason | +|------|--------| +| `ansible/roles/devpi/` | Role no longer needed | +| `ansible/roles/devpi_metrics/` | Metrics collected differently in k8s | --- -### 3. Configure Tailscale LoadBalancer +## Verification Checklist -Tag: `svc:pypi` +- [ ] devpi pod healthy in k8s +- [ ] https://pypi.tail8d86e.ts.net accessible +- [ ] Web interface shows root/pypi index +- [ ] `pip install ` works through proxy +- [ ] mcquack v1.0.0 uploaded to eblume/dev +- [ ] `pip install --index-url https://pypi.tail8d86e.ts.net/eblume/dev/+simple/ mcquack` works +- [ ] Old devpi service removed from indri +- [ ] zk documentation updated --- -### 4. Update pip.conf on gilbert +## Implementation Notes ---- - -### 5. Stop mcquack devpi +*To be filled in during implementation*