eblume/blumeops
1
1
Fork 0
Code Issues Pull requests Projects Releases 83 Packages Wiki Activity Actions
blumeops/plans/completed/k8s-migration/P6_kiwix.complete.md

31 KiB
Raw Blame History

Phase 6: Kiwix and Transmission Migration

Goal: Migrate kiwix-serve and transmission torrent daemon to k8s with shared storage

Status: Ready to implement

Prerequisites: Phase 5.1 complete (minikube on docker driver)

Blocker: Podman Driver Volume Mount Limitations

First attempt branch: feature/p6-kiwix-transmission

The initial implementation was completed and tested, but all volume mount approaches failed due to the podman driver's rootless container limitations:

Approach Result
NFS volume Failed - CAP_SYS_ADMIN required for NFS mounts
SMB CSI driver Failed - mount.cifs returns EPERM inside rootless container
minikube mount (9p) Failed - permission denied mounting into podman VM
hostPath Failed - path doesn't exist inside minikube container

Root cause: The podman driver runs minikube in a rootless container that lacks kernel capabilities for filesystem mounts. This is a documented limitation of the experimental podman driver.

Solution: Phase 5.1 migrates minikube from podman to QEMU2 driver, which creates an actual VM with full kernel capabilities.

What's preserved:

  • All k8s manifests in feature/p6-kiwix-transmission are complete and tested
  • Prerequisites (SMB share, k8s-smb user, data rsync) are done
  • Can retry P6 immediately after P5.1 completes

Overview

This phase migrates two services that share storage but operate independently:

  1. Transmission - General-purpose BitTorrent daemon (standalone service)
  2. Kiwix - Serves ZIM archives via HTTP

The current architecture on indri:

  • Transmission downloads torrents to ~/transmission/
  • Ansible syncs a declarative torrent list to transmission
  • Completed ZIMs are symlinked to kiwix's serving directory
  • kiwix-serve runs as a LaunchAgent with explicit file arguments

New architecture in k8s:

  • SMB volume on sifaka (/volume1/torrents) for all torrent downloads
  • SMB CSI driver for mounting the Synology share in k8s
  • Transmission as a standalone service with Tailscale ingress (torrent.tail8d86e.ts.net)
  • Kiwix deployment that watches for .zim files among all downloads
  • Declarative ZIM list in kiwix manifest, synced to transmission automatically
  • CronJob to detect new ZIMs and restart kiwix

Key design principles:

  • Transmission is a general-purpose torrent daemon, not just for kiwix
  • Users can add arbitrary torrents via transmission web UI/RPC
  • Kiwix declares which ZIM torrents it wants and handles syncing them to transmission
  • Kiwix watches the shared download directory for any .zim files (regardless of how they were added)

Architecture Decisions

Storage: Direct NFS to Sifaka ✅ TESTED

Solution: Direct NFS volume mounts from pods to sifaka. No SMB CSI driver or minikube mount needed.

With the docker driver, minikube containers NAT outbound traffic through indri's LAN IP (192.168.1.50). Sifaka's NFS exports are configured to allow:

  • 192.168.1.0/24 - Docker containers via indri NAT
  • 100.64.0.0/10 - Tailscale clients

Storage path: /volume1/torrents/ on sifaka (NFS export)

  • General-purpose torrent download directory
  • Contains ZIM files, Linux ISOs, and whatever else users download
  • Accessed via native k8s NFS volume (no credentials needed - IP-based access)

No backup needed:

  • Sifaka is RAID 5/6, already the backup target
  • ZIM files are re-downloadable via torrent
  • Other torrents are typically re-downloadable too
  • Future offsite backups will cover all shares

Torrent Daemon: Transmission (Standalone Service)

Why stick with Transmission:

  • Proven reliability on indri
  • Well-maintained container images (linuxserver/transmission)
  • RPC API for automation
  • DHT/PEX for good peer discovery
  • Web UI for interactive management

Container image: lscr.io/linuxserver/transmission:latest

  • Includes web UI for monitoring and adding torrents
  • Supports environment variable configuration
  • Uses /downloads for completed files

Standalone service:

  • Own namespace: torrent
  • Own Tailscale ingress: torrent.tail8d86e.ts.net
  • Can be used for any torrents, not just ZIM archives
  • Users interact with it directly via web UI

Declarative ZIM Torrent Management

Pattern: Kiwix ConfigMap → Kiwix Sidecar → Transmission RPC

  1. ConfigMap (kiwix-zim-torrents) in kiwix namespace lists desired ZIM torrent URLs
  2. Kiwix sidecar syncs ConfigMap to transmission (adds missing torrents)
  3. Transmission downloads to shared SMB volume
  4. Kiwix watches SMB volume for .zim files

This allows adding new ZIM archives by:

  1. Adding torrent URL to ConfigMap in kiwix's ArgoCD manifest
  2. Syncing the kiwix ArgoCD app
  3. Kiwix sidecar adds torrent to transmission
  4. Waiting for download to complete
  5. Kiwix restarts automatically when ZIM watcher detects the new file

Non-declarative torrents:

  • Users can add any torrent via torrent.tail8d86e.ts.net web UI
  • If someone adds a ZIM torrent manually, kiwix will still pick it up
  • Non-ZIM downloads coexist in the same directory

Kiwix Restart Orchestration

Challenge: kiwix-serve doesn't hot-reload new ZIM files; requires restart.

Solution: CronJob watcher

  • Runs hourly (configurable)
  • Lists completed .zim files in SMB volume (among all downloads)
  • Compares with hash of last-seen list
  • If changed, triggers kubectl rollout restart deployment/kiwix

Graceful handling of incomplete downloads:

  • Transmission stores incomplete files with .part extension
  • Kiwix glob pattern *.zim only matches completed files
  • Kiwix can start immediately with whatever ZIMs exist

Prerequisites (Manual Steps)

1. Configure NFS Export on Sifaka

Status: DONE - The torrents shared folder exists at /volume1/torrents with NFS exports allowing:

  • 192.168.1.0/24 - Docker containers via indri NAT
  • 100.64.0.0/10 - Tailscale clients

2. Copy Existing Downloads to Sifaka

Before migration, copy existing downloads to avoid re-downloading ~138GB:

# From indri - mount the NFS share
sudo mount -t nfs sifaka:/volume1/torrents /Volumes/torrents

# Then rsync (adjust mount path as needed)
rsync -avP ~/transmission/ /Volumes/torrents/

# Verify ZIM files
ls -la /Volumes/torrents/*.zim

Steps

1. Create Shared NFS PersistentVolume

This PV is shared between transmission and kiwix namespaces. Uses direct NFS - no CSI driver needed.

File: argocd/manifests/torrent/pv-nfs.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: torrents-nfs-pv
spec:
  capacity:
    storage: 1Ti
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: ""
  nfs:
    server: sifaka
    path: /volume1/torrents

No secrets needed - NFS uses IP-based access control configured on sifaka.

Transmission Service (Standalone)

3. Create Transmission Namespace Resources

File: argocd/manifests/torrent/pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: torrents-storage
  namespace: torrent
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: ""
  volumeName: torrents-nfs-pv
  resources:
    requests:
      storage: 1Ti

File: argocd/manifests/torrent/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: transmission
  namespace: torrent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: transmission
  template:
    metadata:
      labels:
        app: transmission
    spec:
      containers:
        - name: transmission
          image: lscr.io/linuxserver/transmission:latest
          env:
            - name: PUID
              value: "1000"
            - name: PGID
              value: "1000"
            - name: TZ
              value: "America/Los_Angeles"
          ports:
            - containerPort: 9091
              name: web
            - containerPort: 51413
              name: peer-tcp
            - containerPort: 51413
              protocol: UDP
              name: peer-udp
          volumeMounts:
            - name: downloads
              mountPath: /downloads
            - name: config
              mountPath: /config
          resources:
            requests:
              memory: "256Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
          livenessProbe:
            httpGet:
              path: /transmission/web/
              port: 9091
            initialDelaySeconds: 30
            periodSeconds: 30
          readinessProbe:
            httpGet:
              path: /transmission/web/
              port: 9091
            initialDelaySeconds: 10
            periodSeconds: 10
      volumes:
        - name: downloads
          persistentVolumeClaim:
            claimName: torrents-storage
        - name: config
          emptyDir: {}  # Config is ephemeral; torrents persist in SMB

File: argocd/manifests/torrent/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: transmission
  namespace: torrent
spec:
  selector:
    app: transmission
  ports:
    - name: web
      port: 9091
      targetPort: 9091

File: argocd/manifests/torrent/ingress-tailscale.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: transmission
  namespace: torrent
spec:
  ingressClassName: tailscale
  rules:
    - host: torrent
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: transmission
                port:
                  number: 9091

File: argocd/manifests/torrent/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: torrent
resources:
  - pv-nfs.yaml
  - pvc.yaml
  - deployment.yaml
  - service.yaml
  - ingress-tailscale.yaml

File: argocd/apps/torrent.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: torrent
  namespace: argocd
spec:
  project: default
  source:
    repoURL: ssh://forgejo@indri.tail8d86e.ts.net:2200/eblume/blumeops.git
    targetRevision: main
    path: argocd/manifests/torrent
  destination:
    server: https://kubernetes.default.svc
    namespace: torrent
  syncPolicy:
    syncOptions:
      - CreateNamespace=true

Kiwix Service

2. Create Kiwix PVC (References Same PV)

File: argocd/manifests/kiwix/pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: torrents-storage
  namespace: kiwix
spec:
  accessModes:
    - ReadWriteMany  # Need write for the sync sidecar to work
  storageClassName: ""
  volumeName: torrents-nfs-pv
  resources:
    requests:
      storage: 1Ti

4. Create Declarative ZIM Torrent List ConfigMap

This ConfigMap lists the ZIM archives that kiwix wants. The kiwix sidecar syncs these to transmission.

File: argocd/manifests/kiwix/configmap-zim-torrents.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: kiwix-zim-torrents
  namespace: kiwix
data:
  torrents.txt: |
    # Declarative ZIM archive torrent URLs
    # These are synced to transmission automatically by the kiwix sidecar
    # Format: one URL per line, comments start with #
    #
    # Users can also add ZIM torrents manually via torrent.tail8d86e.ts.net
    # and kiwix will pick them up automatically.

    # Wikipedia - Top 1M English articles (43G)
    https://download.kiwix.org/zim/wikipedia/wikipedia_en_top1m_maxi_2025-09.zim.torrent

    # Project Gutenberg - Public domain books (72G)
    https://download.kiwix.org/zim/gutenberg/gutenberg_en_all_2023-08.zim.torrent

    # iFixit - Repair guides (3.3G)
    https://download.kiwix.org/zim/ifixit/ifixit_en_all_2025-12.zim.torrent

    # Stack Exchange
    https://download.kiwix.org/zim/stack_exchange/superuser.com_en_all_2025-12.zim.torrent
    https://download.kiwix.org/zim/stack_exchange/math.stackexchange.com_en_all_2025-12.zim.torrent

    # LibreTexts - Open educational resources
    https://download.kiwix.org/zim/libretexts/libretexts.org_en_bio_2025-01.zim.torrent
    https://download.kiwix.org/zim/libretexts/libretexts.org_en_chem_2025-01.zim.torrent
    https://download.kiwix.org/zim/libretexts/libretexts.org_en_eng_2025-01.zim.torrent
    https://download.kiwix.org/zim/libretexts/libretexts.org_en_math_2025-01.zim.torrent
    https://download.kiwix.org/zim/libretexts/libretexts.org_en_phys_2025-01.zim.torrent
    https://download.kiwix.org/zim/libretexts/libretexts.org_en_human_2025-01.zim.torrent

    # DevDocs - Programming documentation
    https://download.kiwix.org/zim/devdocs/devdocs_en_bash_2026-01.zim.torrent
    https://download.kiwix.org/zim/devdocs/devdocs_en_python_2026-01.zim.torrent
    https://download.kiwix.org/zim/devdocs/devdocs_en_go_2026-01.zim.torrent
    https://download.kiwix.org/zim/devdocs/devdocs_en_kubernetes_2026-01.zim.torrent
    https://download.kiwix.org/zim/devdocs/devdocs_en_docker_2026-01.zim.torrent
    https://download.kiwix.org/zim/devdocs/devdocs_en_git_2026-01.zim.torrent
    https://download.kiwix.org/zim/devdocs/devdocs_en_postgresql_2026-01.zim.torrent
    # Add more from ansible/roles/kiwix/defaults/main.yml as needed

5. Create Torrent Sync Script ConfigMap

This script syncs the declarative ZIM list to transmission.

File: argocd/manifests/kiwix/configmap-sync-script.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: zim-torrent-sync-script
  namespace: kiwix
data:
  sync-zim-torrents.sh: |
    #!/bin/bash
    # Sync ZIM torrents from kiwix ConfigMap to Transmission
    # Runs as a sidecar in the kiwix deployment
    set -euo pipefail

    TORRENT_LIST="${TORRENT_LIST:-/config/torrents.txt}"
    TRANSMISSION_HOST="${TRANSMISSION_HOST:-transmission.torrent.svc.cluster.local}"
    TRANSMISSION_PORT="${TRANSMISSION_PORT:-9091}"

    echo "Syncing ZIM torrents to transmission at ${TRANSMISSION_HOST}:${TRANSMISSION_PORT}"

    # Wait for transmission to be ready
    echo "Waiting for Transmission RPC..."
    max_attempts=30
    attempt=0
    until curl -sf "http://${TRANSMISSION_HOST}:${TRANSMISSION_PORT}/transmission/rpc" >/dev/null 2>&1; do
        attempt=$((attempt + 1))
        if [[ $attempt -ge $max_attempts ]]; then
            echo "Transmission not ready after ${max_attempts} attempts, will retry next cycle"
            exit 0  # Don't fail, just skip this sync
        fi
        sleep 10
    done
    echo "Transmission is ready"

    # Get current torrents from transmission
    # transmission-remote returns header + data + footer, extract just torrent names
    current=$(transmission-remote "${TRANSMISSION_HOST}:${TRANSMISSION_PORT}" -l 2>/dev/null | \
              tail -n +2 | head -n -1 | awk '{print $NF}' || true)

    added=0
    skipped=0

    while IFS= read -r url || [[ -n "$url" ]]; do
        # Skip empty lines and comments
        [[ -z "$url" || "$url" =~ ^[[:space:]]*# ]] && continue
        # Trim whitespace
        url=$(echo "$url" | xargs)
        [[ -z "$url" ]] && continue

        # Extract base name from URL (remove .torrent extension)
        basename=$(basename "$url" .torrent)
        # Also try without .zim in case transmission reports it differently
        basename_no_zim="${basename%.zim}"

        # Check if already in transmission
        if echo "$current" | grep -qF "$basename_no_zim"; then
            ((skipped++)) || true
        else
            if transmission-remote "${TRANSMISSION_HOST}:${TRANSMISSION_PORT}" -a "$url" 2>/dev/null; then
                echo "Added: $basename"
                ((added++)) || true
            else
                echo "Warning: Failed to add $url" >&2
            fi
        fi
    done < "$TORRENT_LIST"

    echo "Sync complete: $added added, $skipped already present"

6. Deploy Kiwix with Torrent Sync Sidecar

File: argocd/manifests/kiwix/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kiwix
  namespace: kiwix
  annotations:
    # Track ZIM file changes for restart detection
    kiwix.blumeops/zim-hash: ""
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kiwix
  template:
    metadata:
      labels:
        app: kiwix
    spec:
      containers:
        # Main kiwix-serve container
        - name: kiwix-serve
          image: ghcr.io/kiwix/kiwix-serve:3.8.1
          args:
            - --port=80
            - /data/*.zim  # Serves ALL .zim files, regardless of how they were added
          ports:
            - containerPort: 80
              name: http
          volumeMounts:
            - name: torrents
              mountPath: /data
              readOnly: true
          resources:
            requests:
              memory: "256Mi"
              cpu: "100m"
            limits:
              memory: "1Gi"
          livenessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 30
          readinessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 10

        # Sidecar: Syncs declarative ZIM torrents to transmission
        - name: torrent-sync
          image: lscr.io/linuxserver/transmission:latest  # Has transmission-remote CLI
          command: ["/bin/bash", "-c"]
          args:
            - |
              echo "Starting ZIM torrent sync sidecar"
              # Initial sync
              /scripts/sync-zim-torrents.sh || echo "Initial sync failed, will retry"
              # Periodic sync every 30 minutes
              while true; do
                sleep 1800
                /scripts/sync-zim-torrents.sh || echo "Sync failed, will retry"
              done
          env:
            - name: TRANSMISSION_HOST
              value: "transmission.torrent.svc.cluster.local"
            - name: TRANSMISSION_PORT
              value: "9091"
            - name: TORRENT_LIST
              value: "/config/torrents.txt"
          volumeMounts:
            - name: zim-torrents-config
              mountPath: /config/torrents.txt
              subPath: torrents.txt
            - name: sync-script
              mountPath: /scripts
          resources:
            requests:
              memory: "32Mi"
              cpu: "10m"
            limits:
              memory: "64Mi"

      volumes:
        - name: torrents
          persistentVolumeClaim:
            claimName: torrents-storage
        - name: zim-torrents-config
          configMap:
            name: kiwix-zim-torrents
        - name: sync-script
          configMap:
            name: zim-torrent-sync-script
            defaultMode: 0755

File: argocd/manifests/kiwix/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: kiwix
  namespace: kiwix
spec:
  selector:
    app: kiwix
  ports:
    - name: http
      port: 80
      targetPort: 80

7. Create Tailscale Ingress for Kiwix

File: argocd/manifests/kiwix/ingress-tailscale.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kiwix
  namespace: kiwix
spec:
  ingressClassName: tailscale
  rules:
    - host: kiwix
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kiwix
                port:
                  number: 80

8. Create ZIM Watcher CronJob

This CronJob runs hourly to detect new completed ZIMs (from any source) and triggers a kiwix restart.

File: argocd/manifests/kiwix/cronjob-zim-watcher.yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: zim-watcher
  namespace: kiwix
spec:
  schedule: "0 * * * *"  # Every hour
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: zim-watcher
          containers:
            - name: watcher
              image: bitnami/kubectl:latest
              command: ["/bin/bash", "-c"]
              args:
                - |
                  set -euo pipefail

                  # Get current ZIM files (among all downloads)
                  # This picks up ZIMs from both declarative list AND manually added torrents
                  current_zims=$(ls -1 /data/*.zim 2>/dev/null | sort | md5sum | cut -d' ' -f1 || echo "empty")

                  # Get stored hash from deployment annotation
                  stored_hash=$(kubectl get deployment kiwix -n kiwix -o jsonpath='{.metadata.annotations.kiwix\.blumeops/zim-hash}' 2>/dev/null || echo "")

                  echo "Current ZIMs hash: $current_zims"
                  echo "Stored hash: $stored_hash"

                  # Also list what ZIMs we found
                  echo "ZIM files found:"
                  ls -la /data/*.zim 2>/dev/null || echo "  (none)"

                  if [[ "$current_zims" != "$stored_hash" && "$current_zims" != "empty" ]]; then
                    echo "ZIM files changed, restarting kiwix deployment..."
                    kubectl annotate deployment kiwix -n kiwix "kiwix.blumeops/zim-hash=$current_zims" --overwrite
                    kubectl rollout restart deployment/kiwix -n kiwix
                    echo "Restart triggered"
                  else
                    echo "No changes detected"
                  fi
              volumeMounts:
                - name: torrents
                  mountPath: /data
                  readOnly: true
          restartPolicy: OnFailure
          volumes:
            - name: torrents
              persistentVolumeClaim:
                claimName: torrents-storage
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: zim-watcher
  namespace: kiwix
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: zim-watcher
  namespace: kiwix
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: zim-watcher
  namespace: kiwix
subjects:
  - kind: ServiceAccount
    name: zim-watcher
    namespace: kiwix
roleRef:
  kind: Role
  name: zim-watcher
  apiGroup: rbac.authorization.k8s.io

9. Create Kiwix Kustomization

File: argocd/manifests/kiwix/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kiwix
resources:
  - pvc.yaml
  - configmap-zim-torrents.yaml
  - configmap-sync-script.yaml
  - deployment.yaml
  - service.yaml
  - ingress-tailscale.yaml
  - cronjob-zim-watcher.yaml

10. Create Kiwix ArgoCD Application

File: argocd/apps/kiwix.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: kiwix
  namespace: argocd
spec:
  project: default
  source:
    repoURL: ssh://forgejo@indri.tail8d86e.ts.net:2200/eblume/blumeops.git
    targetRevision: main
    path: argocd/manifests/kiwix
  destination:
    server: https://kubernetes.default.svc
    namespace: kiwix
  syncPolicy:
    syncOptions:
      - CreateNamespace=true

Deployment Sequence

Phase A: Storage Setup (Manual)

  1. Configure SMB share on sifaka (see Prerequisites section)
  2. Copy existing downloads: 
    ssh indri 'rsync -avP ~/transmission/ sifaka:/volume1/torrents/'
  3. Verify SMB access from indri: 
    # Test SMB mount via Finder or smbclient
smbclient -L //sifaka -U eblume

Phase B: Deploy Transmission to Kubernetes

Deploy transmission first since kiwix depends on it.

  1. Create feature branch (if not already done)
  2. Add torrent manifests to argocd/manifests/torrent/
  3. Add ArgoCD Application to argocd/apps/torrent.yaml
  4. Push branch to forge
  5. Sync ArgoCD apps: 
    argocd app sync apps
argocd app set torrent --revision feature/p6-kiwix
argocd app sync torrent
  6. Verify transmission deployment: 
    kubectl --context=minikube-indri -n torrent get pods
kubectl --context=minikube-indri -n torrent logs deployment/transmission
  7. Test transmission web UI:

Phase C: Deploy Kiwix to Kubernetes

  1. Add kiwix manifests to argocd/manifests/kiwix/
  2. Add ArgoCD Application to argocd/apps/kiwix.yaml
  3. Push to forge
  4. Sync ArgoCD: 
    argocd app set kiwix --revision feature/p6-kiwix
argocd app sync kiwix
  5. Verify kiwix deployment: 
    kubectl --context=minikube-indri -n kiwix get pods
kubectl --context=minikube-indri -n kiwix logs deployment/kiwix -c kiwix-serve
kubectl --context=minikube-indri -n kiwix logs deployment/kiwix -c torrent-sync

Phase D: Verification

  1. Test kiwix access: 
    curl -s https://kiwix.tail8d86e.ts.net/ | head -20
  2. Verify ZIM files are served:
  3. Check transmission status via k8s: 
    kubectl --context=minikube-indri -n torrent exec deployment/transmission -- transmission-remote -l
  4. Verify torrent sync is working: 
    kubectl --context=minikube-indri -n kiwix logs deployment/kiwix -c torrent-sync
  5. Add a test torrent manually via https://torrent.tail8d86e.ts.net to verify interactive use

Phase E: Cutover

  1. Verify all services working correctly
  2. Stop transmission on indri: 
    ssh indri 'brew services stop transmission-cli'
  3. Stop kiwix on indri: 
    ssh indri 'launchctl unload ~/Library/LaunchAgents/mcquack.eblume.kiwix-serve.plist'
  4. Clear kiwix Tailscale serve entry: 
    ssh indri 'tailscale serve status --json'
ssh indri 'tailscale serve clear svc:kiwix'
  5. Delete svc:kiwix device from Tailscale admin (if needed to free hostname)
  6. Verify k8s services claim the hostnames: 
    curl -s https://kiwix.tail8d86e.ts.net/
curl -s https://torrent.tail8d86e.ts.net/transmission/web/

Phase F: Cleanup

  1. Remove indri transmission/kiwix from ansible:
    • Remove transmission and transmission_metrics roles from indri.yml
    • Remove kiwix role from indri.yml
    • Remove svc:kiwix from tailscale_serve
    • Remove transmission/kiwix log collection from alloy
  2. Run ansible to clean up: 
    mise run provision-indri -- --tags tailscale-serve,alloy
  3. Merge PR after all verification
  4. Reset ArgoCD to main: 
    argocd app set torrent --revision main
argocd app sync torrent
argocd app set kiwix --revision main
argocd app sync kiwix

Adding New ZIM Archives (Declarative)

To add a new ZIM archive via GitOps:

  1. Find torrent URL on https://download.kiwix.org/zim/
  2. Add URL to ConfigMap in argocd/manifests/kiwix/configmap-zim-torrents.yaml
  3. Commit and push to feature branch
  4. Sync ArgoCD: 
    argocd app sync kiwix
  5. Wait for download (check transmission at https://torrent.tail8d86e.ts.net)
  6. Kiwix restarts automatically when ZIM watcher detects the new file (hourly)
    • Or manually: kubectl rollout restart deployment/kiwix -n kiwix

Adding ZIM Archives (Manual/Interactive)

Alternatively, add a ZIM torrent manually:

  1. Open transmission web UI at https://torrent.tail8d86e.ts.net
  2. Add torrent via URL or file upload
  3. Wait for download to complete
  4. Kiwix restarts automatically when ZIM watcher detects the new file (hourly)
    • Or manually: kubectl rollout restart deployment/kiwix -n kiwix

Note: Manually added ZIM torrents are NOT tracked in git. If you want them to persist across cluster rebuilds, add them to the ConfigMap.

Adding Non-ZIM Torrents

The transmission service is general-purpose:

  1. Open transmission web UI at https://torrent.tail8d86e.ts.net
  2. Add any torrent (Linux ISOs, etc.)
  3. Downloads go to /volume1/torrents/ on sifaka SMB share
  4. Access downloads via SMB mount or sifaka's file browser

Non-ZIM downloads don't affect kiwix - it only serves .zim files.

Rollback Plan

If migration fails:

  1. Stop k8s services: 
    argocd app delete kiwix --cascade
argocd app delete torrent --cascade
kubectl delete namespace kiwix
kubectl delete namespace torrent
kubectl delete pv torrents-smb-pv
  2. Restart indri services: 
    ssh indri 'brew services start transmission-cli'
ssh indri 'launchctl load ~/Library/LaunchAgents/mcquack.eblume.kiwix-serve.plist'
  3. Re-enable Tailscale serve: 
    mise run provision-indri -- --tags tailscale-serve
  4. Verify access: 
    curl https://kiwix.tail8d86e.ts.net/

Files Summary

New Files

Path Purpose
Transmission (torrent namespace)
argocd/apps/torrent.yaml ArgoCD Application for transmission
argocd/manifests/torrent/pv-nfs.yaml Shared NFS PersistentVolume
argocd/manifests/torrent/pvc.yaml Transmission PVC
argocd/manifests/torrent/deployment.yaml Transmission deployment
argocd/manifests/torrent/service.yaml Transmission service
argocd/manifests/torrent/ingress-tailscale.yaml Tailscale Ingress for torrent.tail8d86e.ts.net
argocd/manifests/torrent/kustomization.yaml Kustomize configuration
Kiwix (kiwix namespace)
argocd/apps/kiwix.yaml ArgoCD Application for kiwix
argocd/manifests/kiwix/pvc.yaml Kiwix PVC (references shared PV)
argocd/manifests/kiwix/configmap-zim-torrents.yaml Declarative ZIM torrent URL list
argocd/manifests/kiwix/configmap-sync-script.yaml ZIM torrent sync script
argocd/manifests/kiwix/deployment.yaml Kiwix deployment with sync sidecar
argocd/manifests/kiwix/service.yaml Kiwix service
argocd/manifests/kiwix/ingress-tailscale.yaml Tailscale Ingress for kiwix.tail8d86e.ts.net
argocd/manifests/kiwix/cronjob-zim-watcher.yaml ZIM watcher CronJob + RBAC
argocd/manifests/kiwix/kustomization.yaml Kustomize configuration

Modified Files

Path Change
ansible/playbooks/indri.yml Remove transmission, transmission_metrics, kiwix roles
ansible/roles/tailscale_serve/defaults/main.yml Remove svc:kiwix
ansible/roles/alloy/defaults/main.yml Remove transmission/kiwix log collection

Roles Kept (not deleted)

  • ansible/roles/transmission/ - Kept for reference
  • ansible/roles/transmission_metrics/ - Kept for reference
  • ansible/roles/kiwix/ - Kept for reference

Verification Checklist

  • NFS export configured on sifaka (/volume1/torrents)
  • NFS exports allow 192.168.1.0/24 and 100.64.0.0/10
  • Direct NFS mount from pod tested and working
  • Existing downloads copied to sifaka
  • Transmission pod running in k8s (torrent namespace)
  • https://torrent.tail8d86e.ts.net accessible (web UI)
  • Can add torrents manually via web UI
  • Kiwix pod running in k8s (kiwix namespace)
  • https://kiwix.tail8d86e.ts.net accessible
  • All existing ZIM archives visible in kiwix
  • Kiwix torrent-sync sidecar synced ZIMs to transmission
  • ZIM watcher CronJob ran successfully
  • Indri transmission stopped
  • Indri kiwix stopped
  • Tailscale hostname cutover complete (both services)
  • Ansible playbook updated
  • zk documentation updated