C2(upgrade-grafana): finalize rewrite cards as historical docs
Rewrite all Mikado cards to historical documentation, remove branch field from goal card, and add changelog fragment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
c57b1fabc1
commit
6b00f15221
4 changed files with 56 additions and 84 deletions
1
docs/changelog.d/upgrade-grafana.feature.md
Normal file
1
docs/changelog.d/upgrade-grafana.feature.md
Normal file
|
|
@ -0,0 +1 @@
|
|||
Upgrade Grafana from 11.4.0 to 12.3.3 with home-built container image and Kustomize manifests, replacing the Helm chart deployment.
|
||||
|
|
@ -9,30 +9,32 @@ tags:
|
|||
|
||||
# Build Grafana Container
|
||||
|
||||
Build a home-built Grafana 12.x container image and publish to the forge registry.
|
||||
Home-built Grafana container image published to `registry.ops.eblu.me/blumeops/grafana`.
|
||||
|
||||
## Context
|
||||
## How It Works
|
||||
|
||||
Grafana currently uses the upstream `docker.io/grafana/grafana:11.4.0` image via the Helm chart. Per supply-chain policy, this should be replaced with a locally built image pushed to `forge.ops.eblu.me/eblume/grafana`.
|
||||
The Dockerfile at `containers/grafana/Dockerfile` downloads the official Grafana OSS tarball for the target architecture (arm64/amd64), installs it into Alpine, and sets up standard paths.
|
||||
|
||||
## Steps
|
||||
To build and push a new version:
|
||||
|
||||
1. Add a Grafana container build to Dagger (or Nix, following existing patterns)
|
||||
2. Base on the official Grafana source or use a Nix derivation
|
||||
3. Tag and push to `forge.ops.eblu.me/eblume/grafana:<version>`
|
||||
4. Add to `mise run container-list` inventory
|
||||
```fish
|
||||
# Update version in Dockerfile
|
||||
# ARG CONTAINER_APP_VERSION=12.3.3
|
||||
|
||||
## Lessons
|
||||
mise run container-build-and-release grafana
|
||||
```
|
||||
|
||||
- **Tarball directory name:** The Grafana OSS tarball extracts to `grafana-<version>` (e.g. `grafana-12.3.3`), *not* `grafana-v<version>`. The `mv` command in the Dockerfile must match this.
|
||||
- **Binary PATH:** The Grafana binary lives at `bin/grafana` inside the extracted directory. The Dockerfile must add the bin directory to `$PATH` (e.g. `ENV PATH="/usr/share/grafana/bin:$PATH"`) or use the full path in CMD.
|
||||
## Gotchas
|
||||
|
||||
## Reference
|
||||
- **Tarball directory name:** Extracts to `grafana-<version>` (e.g. `grafana-12.3.3`), *not* `grafana-v<version>`.
|
||||
- **Binary PATH:** The binary lives at `bin/grafana` inside the extracted directory. The Dockerfile sets `ENV PATH="/usr/share/grafana/bin:$PATH"`.
|
||||
- **UID 472:** Matches the official Grafana image for PVC ownership compatibility.
|
||||
|
||||
- Follow [[build-container-image]] for the standard container build workflow
|
||||
- See existing container builds in `.dagger/` for patterns
|
||||
- The k8s-sidecar image (`quay.io/kiwigrid/k8s-sidecar`) is a secondary concern — address after the main Grafana image
|
||||
## Future Work
|
||||
|
||||
The k8s-sidecar image (`quay.io/kiwigrid/k8s-sidecar`) is still pulled from upstream. Replace with a home-built image when prioritized.
|
||||
|
||||
## Related
|
||||
|
||||
- [[upgrade-grafana]] — Goal card
|
||||
- [[upgrade-grafana]] — Migration context
|
||||
- [[build-container-image]] — Standard container build workflow
|
||||
|
|
|
|||
|
|
@ -8,35 +8,27 @@ tags:
|
|||
|
||||
# Kustomize Grafana Deployment
|
||||
|
||||
Convert Grafana from a Helm chart deployment to plain Kustomize manifests.
|
||||
Grafana is deployed via plain Kustomize manifests in `argocd/manifests/grafana/`, replacing the previous Helm chart.
|
||||
|
||||
## Context
|
||||
## Manifest Structure
|
||||
|
||||
Grafana is currently deployed via ArgoCD using a Helm chart (`grafana-8.8.2`) from a forge mirror. The chart produces: Deployment, Service, PVC, ConfigMaps (grafana.ini, datasources), RBAC resources, and a sidecar container for dashboard provisioning.
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `kustomization.yaml` | Resource list |
|
||||
| `deployment.yaml` | Grafana container + k8s-sidecar for dashboards |
|
||||
| `service.yaml` | ClusterIP on port 80 → 3000 |
|
||||
| `pvc.yaml` | 1Gi SQLite storage |
|
||||
| `configmap.yaml` | `grafana.ini` and datasource provisioning |
|
||||
| `serviceaccount.yaml` | Service account |
|
||||
| `rbac.yaml` | ClusterRole/RoleBinding for sidecar ConfigMap access |
|
||||
|
||||
## Steps
|
||||
## Key Details
|
||||
|
||||
1. Template the current Helm chart to see what it produces:
|
||||
```fish
|
||||
# From the forge mirror, or use argocd app manifests
|
||||
argocd app manifests grafana > /tmp/grafana-helm-output.yaml
|
||||
```
|
||||
2. Create Kustomize equivalents in `argocd/manifests/grafana/`:
|
||||
- `kustomization.yaml`
|
||||
- `deployment.yaml` — Grafana container + k8s-sidecar container
|
||||
- `service.yaml`
|
||||
- `pvc.yaml` — Reuse existing 1Gi PVC
|
||||
- `configmap.yaml` — grafana.ini and datasource provisioning
|
||||
- `rbac.yaml` — ClusterRole, ClusterRoleBinding, Role, RoleBinding
|
||||
3. Update `argocd/apps/grafana.yaml` to use a single kustomize source instead of the Helm multi-source
|
||||
4. Remove the Helm values.yaml (replaced by the kustomize manifests)
|
||||
|
||||
## Notes
|
||||
|
||||
- The existing PVC must not be deleted during the transition — ensure the kustomize PVC matches the existing one's name
|
||||
- The sidecar (`quay.io/kiwigrid/k8s-sidecar`) should also be replaced with a home-built image eventually, but is lower priority — focus on Grafana itself first
|
||||
- Preserve all existing config: Authentik OIDC, datasources, dashboard sidecar labels
|
||||
- **PVC name must remain `grafana`** — changing it would create a new volume and lose the SQLite DB
|
||||
- **Sidecar** watches ConfigMaps with label `grafana_dashboard=1` and reloads dashboards via the Grafana API
|
||||
- **Secrets** come from ExternalSecrets (`grafana-admin`, `grafana-authentik-oauth`, `grafana-teslamate-datasource`) managed by the `grafana-config` ArgoCD app
|
||||
|
||||
## Related
|
||||
|
||||
- [[upgrade-grafana]] — Goal card
|
||||
- [[upgrade-grafana]] — Migration context
|
||||
- [[grafana]] — Service reference card
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
---
|
||||
title: Upgrade Grafana
|
||||
branch: mikado/upgrade-grafana
|
||||
requires:
|
||||
- kustomize-grafana-deployment
|
||||
- build-grafana-container
|
||||
|
|
@ -13,60 +12,38 @@ tags:
|
|||
|
||||
# Upgrade Grafana
|
||||
|
||||
Upgrade Grafana from 11.4.0 (Helm chart 8.8.2) to 12.x, converting from Helm to Kustomize with a home-built container image.
|
||||
Upgraded Grafana from 11.4.0 (Helm chart) to 12.3.3, converting from Helm to Kustomize with a home-built container image.
|
||||
|
||||
## Current State
|
||||
## What Changed
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Helm chart** | `grafana-8.8.2` (from forge mirror of `grafana/helm-charts`) |
|
||||
| **Grafana app** | 11.4.0 |
|
||||
| **Deployment** | Helm via ArgoCD multi-source |
|
||||
| **Namespace** | `monitoring` |
|
||||
| **Storage** | SQLite on 1Gi PVC |
|
||||
- **Image:** `docker.io/grafana/grafana:11.4.0` → `registry.ops.eblu.me/blumeops/grafana:v12.3.3`
|
||||
- **Deployment:** Helm multi-source (chart + values) → single Kustomize directory
|
||||
- **ArgoCD app:** Simplified to one source pointing at `argocd/manifests/grafana/`
|
||||
|
||||
Datasources: [[prometheus]], [[loki]], PostgreSQL (TeslaMate). Dashboard ConfigMaps provisioned via sidecar.
|
||||
|
||||
## Target State
|
||||
|
||||
- Grafana 12.x running from a home-built container (`forge.ops.eblu.me/eblume/grafana`)
|
||||
- Kustomize manifests in `argocd/manifests/grafana/` (no Helm chart dependency)
|
||||
- ArgoCD app simplified to a single source (kustomize directory)
|
||||
- All existing datasources, dashboards, and Authentik OIDC intact
|
||||
All existing datasources ([[prometheus]], [[loki]], TeslaMate), dashboard ConfigMaps, and Authentik OIDC were preserved without changes.
|
||||
|
||||
## Grafana 12 Breaking Changes
|
||||
|
||||
- **Angular plugin removal:** All AngularJS panels force-migrated to React. Our dashboards already use only React panels — no action needed.
|
||||
- **Datasource UID format enforcement:** UIDs must be alphanumeric + dash/underscore, ≤40 chars. Our UIDs (`prometheus`, `loki`, `TeslaMate`) are compliant.
|
||||
- **Annotation table migration:** Full rewrite of the `annotation` table (adds `dashboard_uid` column). Small SQLite DB — should be fast. PVC is disposable if anything goes wrong.
|
||||
None affected us:
|
||||
|
||||
Overall risk: **Low.**
|
||||
- **Angular plugin removal** — our dashboards already used React panels
|
||||
- **Datasource UID format enforcement** — our UIDs were already compliant
|
||||
- **Annotation table migration** — completed automatically on the small SQLite DB
|
||||
|
||||
## Execution
|
||||
## How to Repeat
|
||||
|
||||
Once both prerequisites are complete:
|
||||
To upgrade Grafana again in the future:
|
||||
|
||||
1. Update `argocd/apps/grafana.yaml` to point at the kustomize directory (single source, remove Helm multi-source)
|
||||
2. Update `argocd/manifests/grafana/` with the kustomize manifests using the home-built image
|
||||
3. Deploy on branch, verify with checklist below
|
||||
4. Update `service-versions.yaml` to the new version and today's date
|
||||
1. Update `CONTAINER_APP_VERSION` in `containers/grafana/Dockerfile`
|
||||
2. Build and push via `mise run container-build-and-release grafana`
|
||||
3. Update the image tag in `argocd/manifests/grafana/deployment.yaml`
|
||||
4. Update `service-versions.yaml`
|
||||
5. Sync: `argocd app sync grafana`
|
||||
|
||||
The SQLite PVC is disposable — dashboards are provisioned from ConfigMaps and datasources from config. No backup needed.
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
- [ ] Pod running: `kubectl --context=minikube-indri -n monitoring get pods -l app.kubernetes.io/name=grafana`
|
||||
- [ ] UI loads at `https://grafana.ops.eblu.me`
|
||||
- [ ] Admin login works
|
||||
- [ ] Authentik OIDC login works
|
||||
- [ ] Datasources healthy: Prometheus, Loki, TeslaMate (Settings → Datasources → Test each)
|
||||
- [ ] Key dashboards render: macOS System, Services Health, TeslaMate Overview
|
||||
- [ ] Sidecar loaded all dashboard ConfigMaps
|
||||
- [ ] `mise run services-check` passes
|
||||
- [ ] No errors in pod logs
|
||||
The SQLite PVC is disposable — dashboards come from ConfigMaps and datasources from config.
|
||||
|
||||
## Related
|
||||
|
||||
- [[grafana]] — Service reference card
|
||||
- [[build-grafana-container]] — Prereq: build the container image
|
||||
- [[kustomize-grafana-deployment]] — Prereq: create kustomize manifests
|
||||
- [[build-grafana-container]] — Building the container image
|
||||
- [[kustomize-grafana-deployment]] — Kustomize manifest structure
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue