K8s Migration Phase 2: Grafana to Kubernetes #30

Merged

eblume merged 8 commits from feature/k8s-phase2-grafana into main

2026-01-19 14:40:25 -08:00

eblume commented

2026-01-19 11:38:02 -08:00

Owner

Summary

Migrate Grafana from Homebrew/Ansible to Kubernetes deployment
Switch CloudNativePG to use forge-mirrored Helm chart (HTTPS, no auth needed)
Add Grafana Helm chart deployment via ArgoCD with multi-source pattern
Add Grafana config (Tailscale Ingress, 9 dashboard ConfigMaps)
Update Loki to bind 0.0.0.0 for k8s pod access via host.containers.internal

Key Changes

argocd/apps/grafana.yaml - Grafana Helm chart Application
argocd/apps/grafana-config.yaml - Ingress + dashboard ConfigMaps
argocd/apps/cloudnative-pg.yaml - Now uses forge mirror instead of external Helm repo
ansible/roles/loki/templates/loki-config.yaml.j2 - Bind 0.0.0.0

Deployment and Testing

Deploy Loki config change: mise run provision-indri -- --tags loki
Create namespace: ki create namespace monitoring
Create secret: op inject -i argocd/manifests/grafana-config/secret-admin.yaml.tpl | ki apply -f -
Sync ArgoCD apps (grafana, grafana-config)
Verify Grafana works at https://grafana.tail8d86e.ts.net
Remove svc:grafana from ansible tailscale_serve
Stop brew grafana: ssh indri 'brew services stop grafana'
Delete ansible grafana role

🤖 Generated with Claude Code

## Summary - Migrate Grafana from Homebrew/Ansible to Kubernetes deployment - Switch CloudNativePG to use forge-mirrored Helm chart (HTTPS, no auth needed) - Add Grafana Helm chart deployment via ArgoCD with multi-source pattern - Add Grafana config (Tailscale Ingress, 9 dashboard ConfigMaps) - Update Loki to bind 0.0.0.0 for k8s pod access via `host.containers.internal` ## Key Changes - `argocd/apps/grafana.yaml` - Grafana Helm chart Application - `argocd/apps/grafana-config.yaml` - Ingress + dashboard ConfigMaps - `argocd/apps/cloudnative-pg.yaml` - Now uses forge mirror instead of external Helm repo - `ansible/roles/loki/templates/loki-config.yaml.j2` - Bind 0.0.0.0 ## Deployment and Testing - [x] Deploy Loki config change: `mise run provision-indri -- --tags loki` - [x] Create namespace: `ki create namespace monitoring` - [x] Create secret: `op inject -i argocd/manifests/grafana-config/secret-admin.yaml.tpl | ki apply -f -` - [x] Sync ArgoCD apps (grafana, grafana-config) - [x] Verify Grafana works at https://grafana.tail8d86e.ts.net - [x] Remove svc:grafana from ansible tailscale_serve - [x] Stop brew grafana: `ssh indri 'brew services stop grafana'` - [x] Delete ansible grafana role 🤖 Generated with [Claude Code](https://claude.com/claude-code)

eblume added 1 commit

2026-01-19 11:38:02 -08:00

K8s Migration Phase 2: Grafana to Kubernetes 955253e124

- Migrate Grafana from Homebrew/Ansible to Kubernetes deployment
- Switch CloudNativePG to use forge-mirrored Helm chart (HTTPS)
- Add Grafana Helm chart deployment via ArgoCD
- Add Grafana config (Tailscale Ingress, dashboard ConfigMaps)
- Update Loki to bind 0.0.0.0 for k8s pod access
- Update P2 plan document with detailed implementation steps

Helm chart mirrors created in forge:
- cloudnative-pg-charts (from github.com/cloudnative-pg/charts)
- grafana-helm-charts (from github.com/grafana/helm-charts)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

eblume added 1 commit

2026-01-19 11:41:44 -08:00

Target feature branch for ArgoCD deployments e78b4070fb

Temporarily point ArgoCD apps at feature/k8s-phase2-grafana branch
for testing. Will update back to 'main' before merging PR.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

eblume added 1 commit

2026-01-19 11:46:28 -08:00

Update all apps to target feature/k8s-phase2-grafana f27bbb49d2

Update app-of-apps and all ArgoCD applications to point at
this feature branch for testing. Will change back to main
before merging.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

eblume added 1 commit

2026-01-19 12:18:00 -08:00

Switch chart repos from HTTPS to SSH for cluster egress compatibility d0966ae3f6

- Update cloudnative-pg and grafana apps to use SSH URLs
- Convert repo-forge secret to repo-creds-forge credential template
- The credential template matches all repos under eblume/ using URL prefix
- SSH key is now added to Forgejo user (not as deploy key) for all-repo access

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

eblume added 1 commit

2026-01-19 13:50:05 -08:00

Add indri SSH host key to ArgoCD known_hosts f1547e4a4f

- Add argocd-ssh-known-hosts-cm.yaml patch with forge (indri:2200) host key
- Include upstream ArgoCD default host keys for github, gitlab, bitbucket, etc.
- Required for ArgoCD to connect to forge via SSH

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

eblume added 1 commit

2026-01-19 13:57:16 -08:00

Remove svc:grafana from tailscale_serve - now k8s-hosted f1d4fc0891

Grafana is now running in Kubernetes with its own Tailscale Ingress.
Removing the ansible-managed Tailscale Serve entry frees up the
'grafana' hostname for the k8s ingress to claim.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

eblume added 1 commit

2026-01-19 14:21:06 -08:00

Retire ansible grafana role - now k8s-hosted 5c5251fd32

Grafana is now running in Kubernetes with:
- Helm chart from forge-mirrored grafana-helm-charts repo
- Tailscale Ingress at grafana.tail8d86e.ts.net
- SQLite persistence via PVC
- Dashboards provisioned via ConfigMaps

Removed:
- ansible/roles/grafana/ directory (dashboards migrated to k8s ConfigMaps)
- grafana role from indri.yml playbook

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

eblume added 1 commit

2026-01-19 14:38:32 -08:00

Update all apps to target main branch af1c370556

Removed TODOs and updated targetRevision from feature/k8s-phase2-grafana
to main for all ArgoCD Applications in preparation for PR merge.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>