Erich Blume
cc54b4f565
All checks were successful
Deploy Fly.io Proxy / deploy (push) Successful in 1m16s
## Summary - Embed Grafana Alloy in the Fly.io proxy container to collect nginx JSON access logs (→ Loki) and derive request rate, latency histogram, cache status, and bandwidth metrics (→ Prometheus) - Add nginx `stub_status` endpoint for connection-level metrics (active/reading/writing/waiting) - Create two Grafana dashboards: **Docs APM** (per-service view filtered by `host="docs.eblu.me"`) and **Fly.io Proxy Health** (aggregate proxy health across all upstream services) ## Changed Files | File | Change | |------|--------| | `fly/nginx.conf` | Add JSON `log_format` + `access_log`, add `stub_status` endpoint | | `fly/Dockerfile` | COPY Alloy binary from `grafana/alloy:v1.5.1`, COPY `alloy.river` config | | `fly/alloy.river` | **New** — Alloy config: log tailing, metric extraction, remote_write | | `fly/start.sh` | Start Alloy after Tailscale, before nginx | | `argocd/manifests/grafana-config/dashboards/configmap-docs-apm.yaml` | **New** — Docs APM dashboard | | `argocd/manifests/grafana-config/dashboards/configmap-flyio.yaml` | **New** — Fly.io Proxy Health dashboard | | `argocd/manifests/grafana-config/kustomization.yaml` | Register new dashboard configmaps | | `docs/reference/services/flyio-proxy.md` | Document observability setup | ## Deployment and Testing - [ ] `mise run fly-deploy` — rebuild container with Alloy - [ ] `curl https://docs.eblu.me/` — generate traffic - [ ] `fly logs -a blumeops-proxy` — verify Alloy startup - [ ] Query Prometheus: `flyio_nginx_http_requests_total{instance="flyio-proxy"}` - [ ] Query Loki: `{instance="flyio-proxy", job="flyio-nginx"}` - [ ] `argocd app sync grafana-config` — deploy dashboards - [ ] Verify dashboards show data in Grafana - [ ] `mise run services-check` — no regressions Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/123
40 lines
1.6 KiB
YAML
40 lines
1.6 KiB
YAML
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
|
|
namespace: monitoring
|
|
|
|
resources:
|
|
- ingress-tailscale.yaml
|
|
- external-secret-admin.yaml
|
|
- external-secret-teslamate-datasource.yaml
|
|
# Dashboard ConfigMaps - discovered by Grafana sidecar via label grafana_dashboard=1
|
|
- dashboards/configmap-borgmatic.yaml
|
|
- dashboards/configmap-devpi.yaml
|
|
- dashboards/configmap-loki.yaml
|
|
- dashboards/configmap-macos.yaml
|
|
- dashboards/configmap-minikube.yaml
|
|
- dashboards/configmap-jellyfin.yaml
|
|
- dashboards/configmap-postgresql.yaml
|
|
- dashboards/configmap-services.yaml
|
|
- dashboards/configmap-zot.yaml
|
|
- dashboards/configmap-docs-apm.yaml
|
|
- dashboards/configmap-flyio.yaml
|
|
# TeslaMate dashboards
|
|
- dashboards/configmap-teslamate-overview.yaml
|
|
- dashboards/configmap-teslamate-charges.yaml
|
|
- dashboards/configmap-teslamate-drives.yaml
|
|
- dashboards/configmap-teslamate-efficiency.yaml
|
|
- dashboards/configmap-teslamate-states.yaml
|
|
- dashboards/configmap-teslamate-vampire-drain.yaml
|
|
- dashboards/configmap-teslamate-battery-health.yaml
|
|
- dashboards/configmap-teslamate-statistics.yaml
|
|
- dashboards/configmap-teslamate-charge-level.yaml
|
|
- dashboards/configmap-teslamate-updates.yaml
|
|
- dashboards/configmap-teslamate-trip.yaml
|
|
- dashboards/configmap-teslamate-locations.yaml
|
|
- dashboards/configmap-teslamate-mileage.yaml
|
|
- dashboards/configmap-teslamate-drive-stats.yaml
|
|
- dashboards/configmap-teslamate-charging-stats.yaml
|
|
- dashboards/configmap-teslamate-projected-range.yaml
|
|
- dashboards/configmap-teslamate-timeline.yaml
|
|
- dashboards/configmap-teslamate-visited.yaml
|