Deploy Ollama LLM server on ringtail (#277)
## Summary - Deploy Ollama as a new ArgoCD-managed service on ringtail's k3s cluster with GPU acceleration - Declarative model management via `models.txt` + sidecar sync script (mirrors kiwix torrent pattern) - Initial models: `qwen2.5:14b`, `deepseek-r1:14b`, `phi4:14b`, `gemma3:12b` - hostPath PV on `/mnt/storage1/ollama` for fast local model storage (200Gi) - Tailscale ingress at `ollama.ops.eblu.me` for API access from tailnet - Enable GPU time-slicing (`replicas: 2`) on nvidia-device-plugin so Frigate and Ollama share the RTX 4080 ## Deployment and Testing - [ ] Deploy nvidia-device-plugin changes first: `argocd app sync nvidia-device-plugin` - [ ] Verify GPU time-slicing: `kubectl describe node ringtail --context=k3s-ringtail` shows `nvidia.com/gpu: 2` - [ ] Sync `apps` app with `--revision feature/ollama-ringtail` - [ ] Set ollama app to branch: `argocd app set ollama --revision feature/ollama-ringtail && argocd app sync ollama` - [ ] Verify model-sync sidecar pulls models: `kubectl logs -n ollama deploy/ollama -c model-sync --context=k3s-ringtail` - [ ] Test API: `curl https://ollama.ops.eblu.me/api/tags` - [ ] Test inference: `curl https://ollama.ops.eblu.me/api/generate -d '{"model":"qwen2.5:14b","prompt":"Hello"}'` - [ ] Verify Frigate still works after GPU sharing change - [ ] After merge: `argocd app set ollama --revision main && argocd app sync ollama` Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/277
This commit is contained in:
parent
0f79c61c42
commit
31d925814f
15 changed files with 292 additions and 0 deletions
|
|
@ -85,6 +85,9 @@ caddy_services:
|
|||
- name: ntfy
|
||||
host: "ntfy.{{ caddy_domain }}"
|
||||
backend: "https://ntfy.tail8d86e.ts.net"
|
||||
- name: ollama
|
||||
host: "ollama.{{ caddy_domain }}"
|
||||
backend: "https://ollama.tail8d86e.ts.net"
|
||||
- name: sifaka
|
||||
host: "nas.{{ caddy_domain }}"
|
||||
backend: "http://sifaka:5000"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue