eblume/blumeops
1
1
Fork 0
Code Issues Pull requests Projects Releases 83 Packages Wiki Activity Actions
blumeops/docs/reference/kubernetes/cluster.md

53 lines
1.8 KiB
Markdown
Raw Normal View History

Phase 2: Add Reference section with 24 technical reference cards (#88) ## Summary - Create `docs/reference/` section with 24 technical reference cards - Services (16): alloy, argocd, borgmatic, 1password, forgejo, grafana, jellyfin, kiwix, loki, miniflux, navidrome, postgresql, prometheus, teslamate, transmission, zot - Infrastructure (3): hosts, tailscale, routing - Kubernetes (2): cluster, apps - Storage (2): sifaka, backups - Update README to mark Phase 2 as complete - Add towncrier changelog fragment ## Deployment and Testing - [ ] Build docs locally to verify wiki-links resolve - [ ] Deploy via ArgoCD and verify at docs.ops.eblu.me/reference/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/88
2026-02-03 14:27:37 -08:00
---
Update all docs titles to human-readable (#117) ## Summary - Updated frontmatter `title:` in all 63 doc cards from slug-case to human-readable (e.g. `borgmatic` → `Borgmatic`, `ai-assistance-guide` → `AI Assistance Guide`) - Titles now closely match file stems so `[[wiki-links]]` render naturally without alternate anchor text - Corrected titles that diverged from stems (e.g. `host-inventory` → `Hosts`, `grafana-alloy` → `Alloy`, `argocd-applications` → `Apps`) - Deleted `title-test-alpha.md` and `title-test-beta.md` test cards and removed their reference index entry ## Deployment and Testing - [x] `docs-check-links` passes — all wiki-links valid - [x] `docs-check-index` passes - [x] `docs-check-filenames` passes - [ ] Verify titles render correctly on docs site after deploy Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/117
2026-02-07 21:44:57 -08:00
title: Cluster
Recurring review sweep: 4 doc cards + nvidia-device-plugin v0.19.2 (#366) Knocks out the two daily recurring review tasks (doc review + service review) in one PR. ## Doc review (4 never-reviewed reference cards, `last-reviewed: 2026-06-04`) - **cluster.md** — Kubernetes version v1.34.0 → **v1.35.0**; refreshed the stale ringtail workload list and noted the in-progress minikube→k3s migration (points to `[[ringtail]]` as the canonical list). - **ntfy.md / tempo.md / alloy.md** — corrected image references: these are now **locally-built `registry.ops.eblu.me/blumeops/*` nix containers** (ntfy v2.19.2, tempo v2.10.3, alloy-k8s v1.16.0), not upstream Docker Hub. Fly.io alloy binary bumped to v1.16.1. ## Service review - **nvidia-device-plugin** (ringtail GPU): v0.19.0 → **v0.19.2**. Upstream patch releases — CDI/Tegra fixes + dependency bumps, no breaking changes for our manifest-based CDI + RuntimeClass setup (the service-account change in the notes is helm-only). ## Not in this PR (need container rebuilds, deferred) The other stale services are locally-built nix images, so upgrading them is a forge-runner rebuild rather than a clean tag bump — left untouched (not date-bumped, so they resurface): **prometheus** (v3.10.0→v3.12.0), **loki** (3.6.7→3.7.2), **kube-state-metrics**, **homepage**. Happy to do these as a follow-up rebuild PR. ## Deploy / verify Not yet deployed — `nvidia-device-plugin` still points at `main`. After review: ``` argocd app set nvidia-device-plugin --revision reviews-jun4 && argocd app sync nvidia-device-plugin # after merge: argocd app set nvidia-device-plugin --revision main && argocd app sync nvidia-device-plugin ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/366
2026-06-04 13:37:02 -07:00
modified: 2026-06-04
last-reviewed: 2026-06-04
Phase 2: Add Reference section with 24 technical reference cards (#88) ## Summary - Create `docs/reference/` section with 24 technical reference cards - Services (16): alloy, argocd, borgmatic, 1password, forgejo, grafana, jellyfin, kiwix, loki, miniflux, navidrome, postgresql, prometheus, teslamate, transmission, zot - Infrastructure (3): hosts, tailscale, routing - Kubernetes (2): cluster, apps - Storage (2): sifaka, backups - Update README to mark Phase 2 as complete - Add towncrier changelog fragment ## Deployment and Testing - [ ] Build docs locally to verify wiki-links resolve - [ ] Deploy via ArgoCD and verify at docs.ops.eblu.me/reference/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/88
2026-02-03 14:27:37 -08:00
tags:
- kubernetes
---
# Kubernetes Cluster
Remove unused Mosquitto MQTT broker from ringtail Mosquitto has been dormant since frigate-notify switched from MQTT to webapi polling (529ba10). Tear down live infra (ArgoCD app, namespace) and remove all manifests, service-versions entry, services-check, and doc references. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 18:37:31 -07:00
BlumeOps runs two Kubernetes clusters: a Minikube cluster on [[indri]] (most services) and a k3s cluster on [[ringtail]] (GPU workloads, notifications). Both are managed by [[argocd]] on indri.
Phase 2: Add Reference section with 24 technical reference cards (#88) ## Summary - Create `docs/reference/` section with 24 technical reference cards - Services (16): alloy, argocd, borgmatic, 1password, forgejo, grafana, jellyfin, kiwix, loki, miniflux, navidrome, postgresql, prometheus, teslamate, transmission, zot - Infrastructure (3): hosts, tailscale, routing - Kubernetes (2): cluster, apps - Storage (2): sifaka, backups - Update README to mark Phase 2 as complete - Add towncrier changelog fragment ## Deployment and Testing - [ ] Build docs locally to verify wiki-links resolve - [ ] Deploy via ArgoCD and verify at docs.ops.eblu.me/reference/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/88
2026-02-03 14:27:37 -08:00
## Cluster Specifications
| Property | Value |
|----------|-------|
| **Driver** | docker |
| **Container Runtime** | docker |
Recurring review sweep: 4 doc cards + nvidia-device-plugin v0.19.2 (#366) Knocks out the two daily recurring review tasks (doc review + service review) in one PR. ## Doc review (4 never-reviewed reference cards, `last-reviewed: 2026-06-04`) - **cluster.md** — Kubernetes version v1.34.0 → **v1.35.0**; refreshed the stale ringtail workload list and noted the in-progress minikube→k3s migration (points to `[[ringtail]]` as the canonical list). - **ntfy.md / tempo.md / alloy.md** — corrected image references: these are now **locally-built `registry.ops.eblu.me/blumeops/*` nix containers** (ntfy v2.19.2, tempo v2.10.3, alloy-k8s v1.16.0), not upstream Docker Hub. Fly.io alloy binary bumped to v1.16.1. ## Service review - **nvidia-device-plugin** (ringtail GPU): v0.19.0 → **v0.19.2**. Upstream patch releases — CDI/Tegra fixes + dependency bumps, no breaking changes for our manifest-based CDI + RuntimeClass setup (the service-account change in the notes is helm-only). ## Not in this PR (need container rebuilds, deferred) The other stale services are locally-built nix images, so upgrading them is a forge-runner rebuild rather than a clean tag bump — left untouched (not date-bumped, so they resurface): **prometheus** (v3.10.0→v3.12.0), **loki** (3.6.7→3.7.2), **kube-state-metrics**, **homepage**. Happy to do these as a follow-up rebuild PR. ## Deploy / verify Not yet deployed — `nvidia-device-plugin` still points at `main`. After review: ``` argocd app set nvidia-device-plugin --revision reviews-jun4 && argocd app sync nvidia-device-plugin # after merge: argocd app set nvidia-device-plugin --revision main && argocd app sync nvidia-device-plugin ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/366
2026-06-04 13:37:02 -07:00
| **Kubernetes Version** | v1.35.0 |
Phase 2: Add Reference section with 24 technical reference cards (#88) ## Summary - Create `docs/reference/` section with 24 technical reference cards - Services (16): alloy, argocd, borgmatic, 1password, forgejo, grafana, jellyfin, kiwix, loki, miniflux, navidrome, postgresql, prometheus, teslamate, transmission, zot - Infrastructure (3): hosts, tailscale, routing - Kubernetes (2): cluster, apps - Storage (2): sifaka, backups - Update README to mark Phase 2 as complete - Add towncrier changelog fragment ## Deployment and Testing - [ ] Build docs locally to verify wiki-links resolve - [ ] Deploy via ArgoCD and verify at docs.ops.eblu.me/reference/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/88
2026-02-03 14:27:37 -08:00
| **CPUs** | 6 |
| **Memory** | 11GB |
| **Disk** | 200GB |
| **API Server** | https://k8s.tail8d86e.ts.net |
**Prerequisites:** Docker Desktop with at least 12GB memory allocated.
## Volume Mounting
Enforce unique doc filenames and simple wiki-links (#109) ## Summary - Rename section index files to match their titles (tutorials.md, reference.md, how-to.md, explanation.md) so all filenames are unique - Convert all ~47 path-based wiki-links to simple filename format across 15 files - Update doc-filenames task to no longer skip index.md files - Update doc-links task to reject path-based links containing '/' This ensures all wiki-links work correctly in obsidian.nvim by making links resolvable by filename alone. ## Testing - `mise run doc-filenames` - all unique - `mise run doc-links` - no broken or path-based links - `mise run doc-titles` - no duplicates Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/109
2026-02-04 17:21:34 -08:00
Pods mount NFS directly from [[sifaka|Sifaka]]. Docker NATs outbound traffic through indri's LAN IP (192.168.1.50), allowing access to Sifaka's NFS exports.
Phase 2: Add Reference section with 24 technical reference cards (#88) ## Summary - Create `docs/reference/` section with 24 technical reference cards - Services (16): alloy, argocd, borgmatic, 1password, forgejo, grafana, jellyfin, kiwix, loki, miniflux, navidrome, postgresql, prometheus, teslamate, transmission, zot - Infrastructure (3): hosts, tailscale, routing - Kubernetes (2): cluster, apps - Storage (2): sifaka, backups - Update README to mark Phase 2 as complete - Add towncrier changelog fragment ## Deployment and Testing - [ ] Build docs locally to verify wiki-links resolve - [ ] Deploy via ArgoCD and verify at docs.ops.eblu.me/reference/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/88
2026-02-03 14:27:37 -08:00
## Registry Mirror
Convert wiki-link titles to lowercase slugs (#92) ## Summary - Convert all frontmatter titles to lowercase-hyphenated format (e.g., `grafana-alloy` instead of `Grafana Alloy`) - Update all wiki-links to use the new slug format - Update `doc-titles` task to validate slug format (lowercase, hyphens only) Quartz appears to require titles without spaces for wiki-link resolution. ## Deployment and Testing - [x] Pre-commit hooks pass (`doc-titles` and `doc-links`) - [ ] Build docs v1.0.8 and deploy - [ ] Verify wiki-links resolve correctly (e.g., `[[grafana-alloy]]`) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/92
2026-02-03 16:06:35 -08:00
Containerd uses [[zot]] as a pull-through cache at `host.minikube.internal:5050`.
Phase 2: Add Reference section with 24 technical reference cards (#88) ## Summary - Create `docs/reference/` section with 24 technical reference cards - Services (16): alloy, argocd, borgmatic, 1password, forgejo, grafana, jellyfin, kiwix, loki, miniflux, navidrome, postgresql, prometheus, teslamate, transmission, zot - Infrastructure (3): hosts, tailscale, routing - Kubernetes (2): cluster, apps - Storage (2): sifaka, backups - Update README to mark Phase 2 as complete - Add towncrier changelog fragment ## Deployment and Testing - [ ] Build docs locally to verify wiki-links resolve - [ ] Deploy via ArgoCD and verify at docs.ops.eblu.me/reference/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/88
2026-02-03 14:27:37 -08:00
Mirrors configured: `registry.ops.eblu.me`, `docker.io`, `ghcr.io`, `quay.io`
Fix services-check and update docs for Frigate migration to ringtail (#218) ## Summary - Move mosquitto, ntfy, frigate, frigate-notify pod checks from `minikube-indri` to `k3s-ringtail` context in `services-check` - Add `nvidia-device-plugin` pod check for ringtail k3s - Rename "Kubernetes pods" section to "Indri minikube pods" for clarity - Update 8 documentation files to reflect the migration completed in PRs #216/#217 ## Files Changed | File | Change | |------|--------| | `mise-tasks/services-check` | Move 4 pod checks to k3s-ringtail, add nvidia-device-plugin | | `docs/reference/services/frigate.md` | Image→tensorrt, detector→ONNX/CUDA, shm→512Mi | | `docs/reference/infrastructure/ringtail.md` | List actual k3s workloads | | `docs/reference/infrastructure/indri.md` | Note frigate migration | | `docs/explanation/architecture.md` | Add ringtail to diagram + compute layer | | `docs/reference/kubernetes/cluster.md` | Note two clusters, add k3s section | | `docs/reference/reference.md` | Update frigate/ntfy location | | `docs/how-to/plans/completed/operationalize-reolink-camera.md` | Add post-completion migration note | | `CLAUDE.md` | Add k3s-ringtail context guidance | ## Test plan - [ ] `mise run services-check` — all checks pass - [ ] Review each doc for accuracy against deployed state 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/218
2026-02-19 14:38:21 -08:00
## K3s on Ringtail
Single-node k3s cluster for workloads requiring amd64 or GPU access. See [[ringtail]] for cluster specs, workload list, and secrets management.
| Property | Value |
|----------|-------|
| **Context** | `k3s-ringtail` |
| **API Server** | `https://ringtail.tail8d86e.ts.net:6443` |
Recurring review sweep: 4 doc cards + nvidia-device-plugin v0.19.2 (#366) Knocks out the two daily recurring review tasks (doc review + service review) in one PR. ## Doc review (4 never-reviewed reference cards, `last-reviewed: 2026-06-04`) - **cluster.md** — Kubernetes version v1.34.0 → **v1.35.0**; refreshed the stale ringtail workload list and noted the in-progress minikube→k3s migration (points to `[[ringtail]]` as the canonical list). - **ntfy.md / tempo.md / alloy.md** — corrected image references: these are now **locally-built `registry.ops.eblu.me/blumeops/*` nix containers** (ntfy v2.19.2, tempo v2.10.3, alloy-k8s v1.16.0), not upstream Docker Hub. Fly.io alloy binary bumped to v1.16.1. ## Service review - **nvidia-device-plugin** (ringtail GPU): v0.19.0 → **v0.19.2**. Upstream patch releases — CDI/Tegra fixes + dependency bumps, no breaking changes for our manifest-based CDI + RuntimeClass setup (the service-account change in the notes is helm-only). ## Not in this PR (need container rebuilds, deferred) The other stale services are locally-built nix images, so upgrading them is a forge-runner rebuild rather than a clean tag bump — left untouched (not date-bumped, so they resurface): **prometheus** (v3.10.0→v3.12.0), **loki** (3.6.7→3.7.2), **kube-state-metrics**, **homepage**. Happy to do these as a follow-up rebuild PR. ## Deploy / verify Not yet deployed — `nvidia-device-plugin` still points at `main`. After review: ``` argocd app set nvidia-device-plugin --revision reviews-jun4 && argocd app sync nvidia-device-plugin # after merge: argocd app set nvidia-device-plugin --revision main && argocd app sync nvidia-device-plugin ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/366
2026-06-04 13:37:02 -07:00
| **Workloads** | GPU workloads (Frigate, Ollama), notifications (ntfy, frigate-notify), [[authentik]], and services migrated off indri minikube (Immich, Mealie, Paperless, TeslaMate). See [[ringtail]] for the authoritative list. |
Services are being progressively migrated from indri's minikube to ringtail's k3s; the split above reflects an in-progress state, not a fixed boundary.
Fix services-check and update docs for Frigate migration to ringtail (#218) ## Summary - Move mosquitto, ntfy, frigate, frigate-notify pod checks from `minikube-indri` to `k3s-ringtail` context in `services-check` - Add `nvidia-device-plugin` pod check for ringtail k3s - Rename "Kubernetes pods" section to "Indri minikube pods" for clarity - Update 8 documentation files to reflect the migration completed in PRs #216/#217 ## Files Changed | File | Change | |------|--------| | `mise-tasks/services-check` | Move 4 pod checks to k3s-ringtail, add nvidia-device-plugin | | `docs/reference/services/frigate.md` | Image→tensorrt, detector→ONNX/CUDA, shm→512Mi | | `docs/reference/infrastructure/ringtail.md` | List actual k3s workloads | | `docs/reference/infrastructure/indri.md` | Note frigate migration | | `docs/explanation/architecture.md` | Add ringtail to diagram + compute layer | | `docs/reference/kubernetes/cluster.md` | Note two clusters, add k3s section | | `docs/reference/reference.md` | Update frigate/ntfy location | | `docs/how-to/plans/completed/operationalize-reolink-camera.md` | Add post-completion migration note | | `CLAUDE.md` | Add k3s-ringtail context guidance | ## Test plan - [ ] `mise run services-check` — all checks pass - [ ] Review each doc for accuracy against deployed state 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/218
2026-02-19 14:38:21 -08:00
Phase 2: Add Reference section with 24 technical reference cards (#88) ## Summary - Create `docs/reference/` section with 24 technical reference cards - Services (16): alloy, argocd, borgmatic, 1password, forgejo, grafana, jellyfin, kiwix, loki, miniflux, navidrome, postgresql, prometheus, teslamate, transmission, zot - Infrastructure (3): hosts, tailscale, routing - Kubernetes (2): cluster, apps - Storage (2): sifaka, backups - Update README to mark Phase 2 as complete - Add towncrier changelog fragment ## Deployment and Testing - [ ] Build docs locally to verify wiki-links resolve - [ ] Deploy via ArgoCD and verify at docs.ops.eblu.me/reference/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/88
2026-02-03 14:27:37 -08:00
## Related
Enforce unique doc filenames and simple wiki-links (#109) ## Summary - Rename section index files to match their titles (tutorials.md, reference.md, how-to.md, explanation.md) so all filenames are unique - Convert all ~47 path-based wiki-links to simple filename format across 15 files - Update doc-filenames task to no longer skip index.md files - Update doc-links task to reject path-based links containing '/' This ensures all wiki-links work correctly in obsidian.nvim by making links resolvable by filename alone. ## Testing - `mise run doc-filenames` - all unique - `mise run doc-links` - no broken or path-based links - `mise run doc-titles` - no duplicates Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/109
2026-02-04 17:21:34 -08:00
- [[apps|Apps]] - ArgoCD applications
Convert wiki-link titles to lowercase slugs (#92) ## Summary - Convert all frontmatter titles to lowercase-hyphenated format (e.g., `grafana-alloy` instead of `Grafana Alloy`) - Update all wiki-links to use the new slug format - Update `doc-titles` task to validate slug format (lowercase, hyphens only) Quartz appears to require titles without spaces for wiki-link resolution. ## Deployment and Testing - [x] Pre-commit hooks pass (`doc-titles` and `doc-links`) - [ ] Build docs v1.0.8 and deploy - [ ] Verify wiki-links resolve correctly (e.g., `[[grafana-alloy]]`) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/92
2026-02-03 16:06:35 -08:00
- [[argocd]] - GitOps deployment
- [[zot]] - Registry mirror
Reference in a new issue Copy permalink