eblume/blumeops
1
1
Fork 0
Code Issues Pull requests Projects Releases 83 Packages Wiki Activity Actions

P6: Kiwix and Transmission migration planning #35

Merged
eblume merged 3 commits from feature/p6-kiwix-planning into main 2026-01-20 18:42:11 -08:00
Conversation 0 Commits 3 Files changed 1 +150 -55
1 changed files with 150 additions and 55 deletions
Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit 7c34451f1e - Show all commits

Use SMB over NFS due to lack of CAP_SYS_ADMIN in podman

Prev Next
Erich Blume 2026-01-20 17:53:49 -08:00
commit 7c34451f1e

205
plans/k8s-migration/P6_kiwix.md
View file

@ -1,6 +1,6 @@
# Phase 6: Kiwix and Transmission Migration
**Goal**: Migrate kiwix-serve and transmission torrent daemon to k8s with NFS storage on sifaka
**Goal**: Migrate kiwix-serve and transmission torrent daemon to k8s with SMB storage on sifaka
**Status**: Planning
@ -21,7 +21,8 @@ The current architecture on indri:
- kiwix-serve runs as a LaunchAgent with explicit file arguments
New architecture in k8s:
- **NFS volume** on sifaka (`/volume1/torrents`) for all torrent downloads
- **SMB volume** on sifaka (`/volume1/torrents`) for all torrent downloads
- **SMB CSI driver** for mounting the Synology share in k8s
- **Transmission** as a standalone service with Tailscale ingress (`torrent.tail8d86e.ts.net`)
- **Kiwix** deployment that watches for `.zim` files among all downloads
- **Declarative ZIM list** in kiwix manifest, synced to transmission automatically
@ -37,19 +38,19 @@ New architecture in k8s:
## Architecture Decisions
### Storage: NFS on Sifaka
### Storage: SMB on Sifaka
**Why NFS over SMB:**
- Native k8s NFS support (no CSI driver needed)
- ReadWriteMany/ReadOnlyMany access modes
- Better performance for large files
- Simpler setup than SMB CSI
**Why SMB instead of NFS:**
- Minikube with podman driver lacks CAP_SYS_ADMIN required for NFS mounts
- SMB already works reliably with Synology (used for other shares)
- SMB CSI driver ([csi-driver-smb](https://github.com/kubernetes-csi/csi-driver-smb)) is well-maintained
- Supports ReadWriteMany access mode for concurrent pod access
- Native Synology SMB support with good macOS compatibility
**Storage path:** `/volume1/torrents/` on sifaka
**Storage path:** `/volume1/torrents/` on sifaka (SMB share name: `torrents`)
- General-purpose torrent download directory
- Contains ZIM files, Linux ISOs, and whatever else users download
- Owned by appropriate UID/GID for container access
- Exported to indri's IP range
- Accessed via SMB credentials stored in k8s Secret
**No backup needed:**
- Sifaka is RAID 5/6, already the backup target
@ -83,8 +84,8 @@ New architecture in k8s:
1. **ConfigMap** (`kiwix-zim-torrents`) in kiwix namespace lists desired ZIM torrent URLs
2. **Kiwix sidecar** syncs ConfigMap to transmission (adds missing torrents)
3. Transmission downloads to shared NFS volume
4. Kiwix watches NFS for `.zim` files
3. Transmission downloads to shared SMB volume
4. Kiwix watches SMB volume for `.zim` files
This allows adding new ZIM archives by:
1. Adding torrent URL to ConfigMap in kiwix's ArgoCD manifest
@ -104,7 +105,7 @@ This allows adding new ZIM archives by:
**Solution:** CronJob watcher
- Runs hourly (configurable)
- Lists completed `.zim` files in NFS volume (among all downloads)
- Lists completed `.zim` files in SMB volume (among all downloads)
- Compares with hash of last-seen list
- If changed, triggers `kubectl rollout restart deployment/kiwix`
@ -117,74 +118,162 @@ This allows adding new ZIM archives by:
## Prerequisites (Manual Steps)
### 1. Configure NFS Export on Sifaka (USER ACTION REQUIRED)
### 1. Configure SMB Share on Sifaka (USER ACTION REQUIRED)
On Synology DSM:
1. Create shared folder: `torrents`
- Location: `/volume1/torrents`
- No compression, no encryption
2. Enable NFS service: Control Panel → File Services → NFS → Enable
3. Create NFS rule for `torrents` share:
- Hostname/IP: `100.64.0.0/10` (Tailscale CGNAT range) or specific indri IP
- Privilege: Read/Write
- Squash: Map root to admin
- Security: sys
- Enable async: Yes (better performance)
4. Note the export path (likely `/volume1/torrents`)
2. SMB is enabled by default on Synology; verify at Control Panel → File Services → SMB
3. Set permissions on the `torrents` share:
- Give your user (eblume) Read/Write access
4. Create or note credentials for k8s access:
- Can use existing Synology user credentials
- Store in 1Password for later k8s Secret creation
### 2. Copy Existing Downloads to Sifaka
### 2. Mirror SMB CSI Driver Helm Chart to Forge (USER ACTION REQUIRED)
Mirror the SMB CSI driver chart to forge for GitOps deployment:
```bash
# Clone the upstream chart repo
cd ~/code/3rd
git clone https://github.com/kubernetes-csi/csi-driver-smb.git
cd csi-driver-smb
# Push to forge mirror
git remote add forge ssh://forgejo@indri.tail8d86e.ts.net:2200/eblume/csi-driver-smb.git
git push forge --all --tags
```
### 3. Copy Existing Downloads to Sifaka
Before migration, copy existing downloads to avoid re-downloading ~138GB:
```bash
# From indri - copy everything (ZIMs and any other torrents)
rsync -avP ~/transmission/ sifaka:/volume1/torrents/
# From indri - mount the SMB share via Finder or command line
open smb://sifaka/torrents
# Then rsync (adjust mount path as needed)
rsync -avP ~/transmission/ /Volumes/torrents/
# Verify ZIM files
ssh sifaka 'ls -la /volume1/torrents/*.zim'
ls -la /Volumes/torrents/*.zim
```
### 3. Verify NFS Mount from Indri
### 4. Store SMB Credentials in 1Password
```bash
# Test mount from indri
ssh indri 'sudo mount -t nfs sifaka:/volume1/torrents /mnt/test && ls /mnt/test && sudo umount /mnt/test'
```
Create a 1Password item for Synology SMB credentials:
- Vault: `vg6xf6vvfmoh5hqjjhlhbeoaie` (blumeops vault)
- Item name: `synology-smb-torrents`
- Fields: `username`, `password`
---
## Steps
### 1. Create Shared NFS PersistentVolume
### 1. Deploy SMB CSI Driver via ArgoCD
**File:** `argocd/manifests/smb-csi/values.yaml`
```yaml
# Minimal values - defaults are generally fine
controller:
replicas: 1
```
**File:** `argocd/apps/smb-csi.yaml`
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: smb-csi
namespace: argocd
spec:
project: default
sources:
# Helm chart from forge mirror
- repoURL: ssh://forgejo@indri.tail8d86e.ts.net:2200/eblume/csi-driver-smb.git
targetRevision: v1.17.0
path: charts/csi-driver-smb
helm:
releaseName: csi-driver-smb
valueFiles:
- $values/argocd/manifests/smb-csi/values.yaml
# Values from our git repo
- repoURL: ssh://forgejo@indri.tail8d86e.ts.net:2200/eblume/blumeops.git
targetRevision: main
ref: values
destination:
server: https://kubernetes.default.svc
namespace: kube-system
syncPolicy:
syncOptions:
- CreateNamespace=true
```
### 2. Create Shared SMB PersistentVolume
This PV is shared between transmission and kiwix namespaces.
**File:** `argocd/manifests/torrent/pv-nfs.yaml`
**File:** `argocd/manifests/torrent/pv-smb.yaml`
```yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: torrents-nfs-pv
name: torrents-smb-pv
spec:
capacity:
storage: 1Ti # Logical limit, NFS doesn't enforce
storage: 1Ti
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
storageClassName: "" # Static provisioning
nfs:
server: sifaka # Tailscale hostname
path: /volume1/torrents
storageClassName: ""
mountOptions:
- dir_mode=0777
- file_mode=0777
- uid=1000
- gid=1000
- noperm
- mfsymlinks
- cache=strict
- noserverino # Required to prevent data corruption
csi:
driver: smb.csi.k8s.io
volumeHandle: torrents-smb-pv
volumeAttributes:
source: //sifaka/torrents
nodeStageSecretRef:
name: smbcreds
namespace: torrent
```
**Note:** Using `sifaka` hostname requires Tailscale DNS from k8s pods. This should work with the existing DNSConfig from Phase 1.
**File:** `argocd/manifests/torrent/secret-smb.yaml.tpl`
```yaml
# Template - apply manually with credentials from 1Password
# kubectl --context=minikube create secret generic smbcreds \
# --namespace torrent \
# --from-literal=username=$(op read "op://vg6xf6vvfmoh5hqjjhlhbeoaie/synology-smb-torrents/username") \
# --from-literal=password=$(op read "op://vg6xf6vvfmoh5hqjjhlhbeoaie/synology-smb-torrents/password")
apiVersion: v1
kind: Secret
metadata:
name: smbcreds
namespace: torrent
type: Opaque
stringData:
username: "{{ op://vg6xf6vvfmoh5hqjjhlhbeoaie/synology-smb-torrents/username }}"
password: "{{ op://vg6xf6vvfmoh5hqjjhlhbeoaie/synology-smb-torrents/password }}"
```
---
## Transmission Service (Standalone)
### 2. Create Transmission Namespace Resources
### 3. Create Transmission Namespace Resources
**File:** `argocd/manifests/torrent/pvc.yaml`
@ -198,7 +287,7 @@ spec:
accessModes:
- ReadWriteMany
storageClassName: ""
volumeName: torrents-nfs-pv
volumeName: torrents-smb-pv
resources:
requests:
storage: 1Ti
@ -268,7 +357,7 @@ spec:
persistentVolumeClaim:
claimName: torrents-storage
- name: config
emptyDir: {} # Config is ephemeral; torrents persist in NFS
emptyDir: {} # Config is ephemeral; torrents persist in SMB
```
**File:** `argocd/manifests/torrent/service.yaml`
@ -318,7 +407,8 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: torrent
resources:
- pv-nfs.yaml
- pv-smb.yaml
- secret-smb.yaml.tpl
- pvc.yaml
- deployment.yaml
- service.yaml
@ -365,7 +455,7 @@ spec:
accessModes:
- ReadWriteMany # Need write for the sync sidecar to work
storageClassName: ""
volumeName: torrents-nfs-pv
volumeName: torrents-smb-pv
resources:
requests:
storage: 1Ti
@ -783,14 +873,15 @@ spec:
### Phase A: Storage Setup (Manual)
1. **Configure NFS on sifaka** (see Prerequisites section)
1. **Configure SMB share on sifaka** (see Prerequisites section)
2. **Copy existing downloads:**
```bash
ssh indri 'rsync -avP ~/transmission/ sifaka:/volume1/torrents/'
```
3. **Verify NFS access from indri:**
3. **Verify SMB access from indri:**
```bash
ssh indri 'showmount -e sifaka'
# Test SMB mount via Finder or smbclient
smbclient -L //sifaka -U eblume
```
### Phase B: Deploy Transmission to Kubernetes
@ -930,8 +1021,8 @@ The transmission service is general-purpose:
1. **Open transmission web UI** at https://torrent.tail8d86e.ts.net
2. **Add any torrent** (Linux ISOs, etc.)
3. **Downloads go to** `/volume1/torrents/` on sifaka NFS share
4. **Access downloads** via NFS mount or sifaka's file browser
3. **Downloads go to** `/volume1/torrents/` on sifaka SMB share
4. **Access downloads** via SMB mount or sifaka's file browser
Non-ZIM downloads don't affect kiwix - it only serves `.zim` files.
@ -947,7 +1038,7 @@ If migration fails:
argocd app delete torrent --cascade
kubectl delete namespace kiwix
kubectl delete namespace torrent
kubectl delete pv torrents-nfs-pv
kubectl delete pv torrents-smb-pv
```
2. **Restart indri services:**
```bash
@ -973,7 +1064,10 @@ If migration fails:
|------|---------|
| **Transmission (torrent namespace)** | |
| `argocd/apps/torrent.yaml` | ArgoCD Application for transmission |
| `argocd/manifests/torrent/pv-nfs.yaml` | Shared NFS PersistentVolume |
| `argocd/apps/smb-csi.yaml` | ArgoCD Application for SMB CSI driver |
| `argocd/manifests/smb-csi/values.yaml` | SMB CSI driver Helm values |
| `argocd/manifests/torrent/pv-smb.yaml` | Shared SMB PersistentVolume |
| `argocd/manifests/torrent/secret-smb.yaml.tpl` | SMB credentials secret template |
| `argocd/manifests/torrent/pvc.yaml` | Transmission PVC |
| `argocd/manifests/torrent/deployment.yaml` | Transmission deployment |
| `argocd/manifests/torrent/service.yaml` | Transmission service |
@ -1008,9 +1102,10 @@ If migration fails:
## Verification Checklist
- [ ] NFS export configured on sifaka (`/volume1/torrents`)
- [ ] SMB share configured on sifaka (`/volume1/torrents`)
- [ ] SMB CSI driver deployed to k8s
- [ ] Existing downloads copied to sifaka
- [ ] NFS mount works from indri minikube
- [ ] SMB credentials secret created in k8s
- [ ] Transmission pod running in k8s (`torrent` namespace)
- [ ] https://torrent.tail8d86e.ts.net accessible (web UI)
- [ ] Can add torrents manually via web UI