Migrate devpi from minikube to indri (launchd) (#341)
## Summary Devpi was crash-looping under memory pressure on the minikube StatefulSet, breaking the Python toolchain across the repo (`mise run docs-mikado`, `prek`, every `uv pip install`). It moves to indri as a native LaunchAgent. ## What changed - **New ansible role** `ansible/roles/devpi/`: installs `devpi-server` + `devpi-web` into a uv-managed venv, initializes the server-dir on first run via 1Password root password, runs as a LaunchAgent (`mcquack.eblume.devpi`) bound to `127.0.0.1:3141`. Bootstraps from upstream PyPI (so devpi can install itself on a fresh box). - **Caddy**: `pypi.ops.eblu.me` now proxies to `http://localhost:3141`. - **Playbook**: `indri.yml` gains pre_tasks for the root password and the new role. - **service-versions.yaml**: devpi flipped from `type: argocd` to `type: ansible`. - **ArgoCD**: removed `apps/devpi.yaml` and `manifests/devpi/`. The in-cluster Application, namespace, and PVC have been deleted. - **Docs**: new how-to `docs/how-to/operations/devpi-on-indri.md`; `restart-indri.md` lists devpi in the LaunchAgent stop list. ## Already deployed (live on indri) - Service running: `launchctl list mcquack.eblume.devpi` → PID 53888 - `curl https://pypi.ops.eblu.me/+api` returns 200 ✅ - `mise run docs-mikado` works again ✅ - 1.0G of cached PyPI data was migrated from the PVC to `~erichblume/devpi/server-dir/` - Minikube namespace and PVC fully reclaimed ## Test plan - [ ] `mise run services-check` (after merge) - [ ] CI workflows that use devpi succeed - [ ] No regressions in tools that depend on `pypi.ops.eblu.me` (prek, uv-script tasks, dagger pipelines) ## Context This is the C1 prelude to a planned C2 chain (`mikado/retire-minikube-indri`) to retire minikube on indri entirely. Doing devpi as a standalone C1 was the right call because (a) it was urgent — it was breaking the toolchain — and (b) it shakes out the migration recipe before we commit to a multi-leaf chain. Reviewed-on: #341
This commit is contained in:
parent
f4a24595b1
commit
14ca0160ba
24 changed files with 260 additions and 289 deletions
|
|
@ -1,29 +0,0 @@
|
|||
# devpi PyPI Caching Proxy
|
||||
# Provides PyPI cache and private package hosting
|
||||
#
|
||||
# After first deployment, initialize devpi:
|
||||
# kubectl -n devpi exec -it devpi-0 -- devpi-init --serverdir /devpi --root-passwd <password>
|
||||
# kubectl -n devpi rollout restart statefulset devpi
|
||||
#
|
||||
# Then create user/index:
|
||||
# uvx devpi use https://pypi.tail8d86e.ts.net
|
||||
# uvx devpi login root
|
||||
# uvx devpi user -c eblume email=blume.erich@gmail.com
|
||||
# uvx devpi index -c eblume/dev bases=root/pypi
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: devpi
|
||||
namespace: argocd
|
||||
spec:
|
||||
project: default
|
||||
source:
|
||||
repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
|
||||
targetRevision: main
|
||||
path: argocd/manifests/devpi
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
namespace: devpi
|
||||
syncPolicy:
|
||||
syncOptions:
|
||||
- CreateNamespace=true
|
||||
|
|
@ -159,8 +159,10 @@ prometheus.exporter.blackbox "services" {
|
|||
}
|
||||
|
||||
target {
|
||||
// devpi runs natively on indri (LaunchAgent), not in-cluster.
|
||||
// We probe through Caddy (https://pypi.ops.eblu.me) which the cluster can reach via Tailscale.
|
||||
name = "devpi"
|
||||
address = "http://devpi.devpi.svc.cluster.local:3141/+api"
|
||||
address = "https://pypi.ops.eblu.me/+api"
|
||||
module = "http_2xx"
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,72 +0,0 @@
|
|||
# devpi PyPI Caching Proxy
|
||||
|
||||
devpi-server running in Kubernetes, providing:
|
||||
- PyPI caching proxy at `root/pypi`
|
||||
- Private package hosting at `eblume/dev`
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Create the root password secret
|
||||
|
||||
```fish
|
||||
kubectl create namespace devpi
|
||||
op inject -i argocd/manifests/devpi/secret-root.yaml.tpl | kubectl apply -f -
|
||||
```
|
||||
|
||||
### 2. Deploy via ArgoCD
|
||||
|
||||
```fish
|
||||
argocd app sync apps
|
||||
argocd app sync devpi
|
||||
```
|
||||
|
||||
The container will auto-initialize on first startup using the root password from the secret.
|
||||
|
||||
### 3. Create user and index (first time only)
|
||||
|
||||
After the pod is running:
|
||||
|
||||
```fish
|
||||
# Login to devpi as root
|
||||
uvx --from devpi-client devpi use https://pypi.tail8d86e.ts.net
|
||||
uvx --from devpi-client devpi login root
|
||||
# Enter root password when prompted
|
||||
|
||||
# Create eblume user (prompts for password - use the one from 1Password)
|
||||
uvx --from devpi-client devpi user -c eblume email=blume.erich@gmail.com
|
||||
|
||||
# Create private index inheriting from PyPI
|
||||
uvx --from devpi-client devpi index -c eblume/dev bases=root/pypi
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### As pip index (caching proxy)
|
||||
|
||||
Configure `~/.config/pip/pip.conf`:
|
||||
|
||||
```ini
|
||||
[global]
|
||||
index-url = https://pypi.tail8d86e.ts.net/root/pypi/+simple/
|
||||
trusted-host = pypi.tail8d86e.ts.net
|
||||
```
|
||||
|
||||
### Upload private packages
|
||||
|
||||
```fish
|
||||
cd ~/code/personal/your-package
|
||||
uv build
|
||||
uv publish --publish-url https://pypi.tail8d86e.ts.net/eblume/dev/
|
||||
```
|
||||
|
||||
## URLs
|
||||
|
||||
- Web UI: https://pypi.tail8d86e.ts.net
|
||||
- PyPI cache: https://pypi.tail8d86e.ts.net/root/pypi/+simple/
|
||||
- Private index: https://pypi.tail8d86e.ts.net/eblume/dev/+simple/
|
||||
|
||||
## Credentials
|
||||
|
||||
Stored in 1Password vault `blumeops`, item `kyhzfifryqnuk7jeyibmmjvxxm`:
|
||||
- `root password` - devpi root user
|
||||
- `password` - eblume user password
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
# ExternalSecret for devpi root password
|
||||
#
|
||||
# Replaces the manual op inject workflow from secret-root.yaml.tpl
|
||||
#
|
||||
# 1Password item: "devpi" in blumeops vault
|
||||
# Field: "root password"
|
||||
#
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: devpi-root
|
||||
namespace: devpi
|
||||
spec:
|
||||
refreshInterval: 1h
|
||||
secretStoreRef:
|
||||
kind: ClusterSecretStore
|
||||
name: onepassword-blumeops
|
||||
target:
|
||||
name: devpi-root
|
||||
creationPolicy: Owner
|
||||
data:
|
||||
- secretKey: password
|
||||
remoteRef:
|
||||
key: devpi
|
||||
property: root password
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: devpi-tailscale
|
||||
namespace: devpi
|
||||
annotations:
|
||||
tailscale.com/proxy-class: "default"
|
||||
tailscale.com/proxy-group: "ingress"
|
||||
gethomepage.dev/enabled: "true"
|
||||
gethomepage.dev/name: "PyPI"
|
||||
gethomepage.dev/group: "Infrastructure"
|
||||
gethomepage.dev/icon: "pypi.png"
|
||||
gethomepage.dev/description: "PyPI cache"
|
||||
gethomepage.dev/href: "https://pypi.ops.eblu.me"
|
||||
gethomepage.dev/pod-selector: "app=devpi"
|
||||
spec:
|
||||
ingressClassName: tailscale
|
||||
defaultBackend:
|
||||
service:
|
||||
name: devpi
|
||||
port:
|
||||
number: 3141
|
||||
tls:
|
||||
- hosts:
|
||||
- pypi
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
namespace: devpi
|
||||
|
||||
resources:
|
||||
- statefulset.yaml
|
||||
- service.yaml
|
||||
- ingress-tailscale.yaml
|
||||
- external-secret.yaml
|
||||
|
||||
images:
|
||||
- name: registry.ops.eblu.me/blumeops/devpi
|
||||
newTag: v6.19.3-37b8a21
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: devpi
|
||||
namespace: devpi
|
||||
spec:
|
||||
selector:
|
||||
app: devpi
|
||||
ports:
|
||||
- name: http
|
||||
port: 3141
|
||||
targetPort: 3141
|
||||
protocol: TCP
|
||||
|
|
@ -1,64 +0,0 @@
|
|||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: devpi
|
||||
namespace: devpi
|
||||
spec:
|
||||
serviceName: devpi
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: devpi
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: devpi
|
||||
spec:
|
||||
securityContext:
|
||||
fsGroup: 1000
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
containers:
|
||||
- name: devpi
|
||||
image: registry.ops.eblu.me/blumeops/devpi:kustomized
|
||||
env:
|
||||
- name: DEVPI_ROOT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: devpi-root
|
||||
key: password
|
||||
- name: DEVPI_OUTSIDE_URL
|
||||
value: "https://pypi.ops.eblu.me"
|
||||
ports:
|
||||
- containerPort: 3141
|
||||
name: http
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /devpi
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "2Gi" # High limit for initial PyPI index build, reclaimed after
|
||||
cpu: "500m"
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /+api
|
||||
port: 3141
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 30
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /+api
|
||||
port: 3141
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 10
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
resources:
|
||||
requests:
|
||||
storage: 50Gi
|
||||
Loading…
Add table
Add a link
Reference in a new issue