blumeops/plans/k8s-migration/P5_devpi.md

Phase 5: devpi Migration to Kubernetes

Goal: Migrate devpi PyPI caching proxy from indri to k8s

Status: In Progress (2026-01-20)

Prerequisites: Phase 4 complete

---

Overview

This phase migrates devpi from mcquack LaunchAgent on indri to Kubernetes:

1. Stop devpi on indri and clear Tailscale service
2. Build custom devpi container image (devpi-server + devpi-web)
3. Deploy as StatefulSet with PVC for data persistence
4. Expose via Tailscale Ingress at pypi.tail8d86e.ts.net
5. Initialize fresh (no data migration - just cache and one private package)
6. Validate with pip proxy and mcquack package upload

Decision: Fresh start - The only private package is mcquack v1.0.0 which we'll re-upload. The PyPI cache is re-fetchable.

---

Steps

1. Stop devpi on indri (FIRST)

# Stop and unload the LaunchAgent
ssh indri 'launchctl unload ~/Library/LaunchAgents/mcquack.eblume.devpi.plist'

# Also stop the metrics collector
ssh indri 'launchctl unload ~/Library/LaunchAgents/mcquack.eblume.devpi-metrics.plist'

# Clear any tailscale serve entries (if any)
ssh indri 'tailscale serve off --service=svc:pypi' 2>/dev/null || true

User action required: Delete the pypi device/service from Tailscale admin console.

2. Build devpi container image

Create argocd/manifests/devpi/Dockerfile:

FROM python:3.12-slim

# Install devpi-server and devpi-web
RUN pip install --no-cache-dir devpi-server devpi-web

# Create data directory
RUN mkdir -p /devpi

# Expose default port
EXPOSE 3141

# Use ENTRYPOINT for flexibility
ENTRYPOINT ["devpi-server"]

# Default args (can be overridden)
CMD ["--serverdir", "/devpi", "--host", "0.0.0.0", "--port", "3141"]

Build and push to Zot:

# From gilbert (has podman)
cd argocd/manifests/devpi
podman build -t registry.tail8d86e.ts.net/blumeops/devpi:latest .
podman push registry.tail8d86e.ts.net/blumeops/devpi:latest

3. Create k8s manifests

File	Purpose
argocd/apps/devpi.yaml	ArgoCD Application definition
argocd/manifests/devpi/statefulset.yaml	StatefulSet with PVC
argocd/manifests/devpi/service.yaml	ClusterIP Service
argocd/manifests/devpi/ingress-tailscale.yaml	Tailscale Ingress for pypi.tail8d86e.ts.net
argocd/manifests/devpi/init-job.yaml	One-time initialization job
argocd/manifests/devpi/kustomization.yaml	Kustomize configuration

4. Deploy via ArgoCD

# Point app at feature branch for testing
argocd app create devpi \
  --repo ssh://forgejo@indri.tail8d86e.ts.net:2200/eblume/blumeops.git \
  --path argocd/manifests/devpi \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace devpi \
  --revision feature/p5-devpi

# Or add to apps/ and let app-of-apps sync
argocd app sync apps
argocd app sync devpi

5. Initialize devpi

After StatefulSet is running:

# Exec into pod to initialize
kubectl -n devpi exec -it devpi-0 -- devpi-init --serverdir /devpi --root-passwd <from-1password>

# Or use init job with secret

Create user and index:

# From workstation
uvx devpi use https://pypi.tail8d86e.ts.net
uvx devpi login root
uvx devpi user -c eblume email=blume.erich@gmail.com
uvx devpi index -c eblume/dev bases=root/pypi

6. Update pip.conf on gilbert

The existing ~/.config/pip/pip.conf should work unchanged since we're keeping the same hostname pypi.tail8d86e.ts.net.

7. Remove devpi ansible role

• Remove devpi from ansible/playbooks/indri.yml roles list
• Remove devpi_metrics from alloy textfile collectors
• Keep devpi ansible role directory for reference (or delete)

8. Validate

# Test pip cache proxy
pip install --index-url https://pypi.tail8d86e.ts.net/root/pypi/+simple/ requests

# Upload mcquack
cd ~/code/personal/mcquack
uv build
uv publish --publish-url https://pypi.tail8d86e.ts.net/eblume/dev/

---

New Files

Path	Purpose
argocd/apps/devpi.yaml	ArgoCD Application definition
argocd/manifests/devpi/Dockerfile	Container image build
argocd/manifests/devpi/statefulset.yaml	StatefulSet with PVC
argocd/manifests/devpi/service.yaml	ClusterIP Service
argocd/manifests/devpi/ingress-tailscale.yaml	Tailscale Ingress
argocd/manifests/devpi/kustomization.yaml	Kustomize configuration
argocd/manifests/devpi/README.md	Setup documentation

Modified Files

Path	Change
ansible/playbooks/indri.yml	Remove devpi role
ansible/roles/alloy/defaults/main.yml	Remove devpi textfile collector
ansible/roles/borgmatic/defaults/main.yml	Remove devpi backup (no longer on indri)

Deleted Files

Path	Reason
ansible/roles/devpi/	Role no longer needed
ansible/roles/devpi_metrics/	Metrics collected differently in k8s

---

Verification Checklist

• devpi pod healthy in k8s
• https://pypi.tail8d86e.ts.net accessible
• Web interface shows root/pypi index
• pip install <package> works through proxy
• mcquack v1.0.0 uploaded to eblume/dev
• pip install --index-url https://pypi.tail8d86e.ts.net/eblume/dev/+simple/ mcquack works
• Old devpi service removed from indri
• zk documentation updated

---

Implementation Notes

To be filled in during implementation