diff --git a/docs/reference/services/alloy.md b/docs/reference/services/alloy.md index cc797d3..c7958bf 100644 --- a/docs/reference/services/alloy.md +++ b/docs/reference/services/alloy.md @@ -7,9 +7,10 @@ tags: # Grafana Alloy -Unified observability collector for metrics and logs with two deployments: +Unified observability collector for metrics and logs with three deployments: 1. **Indri (host)** - System metrics and service logs from macOS host 2. **Kubernetes (DaemonSet)** - Automatic pod log collection and service health probes +3. **Fly.io proxy (embedded)** - nginx access log metrics and log forwarding from [[flyio-proxy]] ## Quick Reference @@ -20,6 +21,8 @@ Unified observability collector for metrics and logs with two deployments: | **K8s Namespace** | `alloy` | | **K8s Image** | `grafana/alloy:v1.8.2` | | **ArgoCD App** | `alloy-k8s` | +| **Fly.io Config** | `fly/alloy.river` | +| **Fly.io Image** | `grafana/alloy:v1.5.1` (binary copied into nginx container) | ## Metrics Collected @@ -33,6 +36,13 @@ Unified observability collector for metrics and logs with two deployments: - All pod logs via `loki.source.kubernetes` - Service health probes: miniflux, kiwix, transmission, devpi, argocd +### From Fly.io Proxy +- `flyio_nginx_http_requests_total` — request rate by status/method/host +- `flyio_nginx_http_request_duration_seconds` — latency histogram +- `flyio_nginx_http_response_bytes_total` — response bandwidth +- `flyio_nginx_cache_requests_total` — cache HIT/MISS/EXPIRED counts +- Pushed to [[prometheus]] via remote_write through [[caddy]] + ## Logs Collected **Brew services:** forgejo, tailscale @@ -41,6 +51,8 @@ Unified observability collector for metrics and logs with two deployments: Logs pushed to [[loki]] at `https://loki.tail8d86e.ts.net/loki/api/v1/push`. +**Fly.io proxy:** nginx JSON access logs pushed to [[loki]] at `https://loki.ops.eblu.me/loki/api/v1/push` (via [[caddy]]). + ## Why Built from Source The Homebrew bottle uses `CGO_ENABLED=0`, which breaks Tailscale MagicDNS. Building with `CGO_ENABLED=1` uses the macOS native resolver. diff --git a/docs/reference/services/grafana.md b/docs/reference/services/grafana.md index 092997a..43f2a2c 100644 --- a/docs/reference/services/grafana.md +++ b/docs/reference/services/grafana.md @@ -41,6 +41,8 @@ Optional annotation: `grafana_folder: "FolderName"` - Minikube - Kubernetes cluster overview - Borgmatic Backups - Backup status and trends - Services Health - HTTP probe results +- Docs APM - Request rate, latency, cache for docs.eblu.me +- Fly.io Proxy Health - Aggregate proxy health across all upstream services - TeslaMate (18 dashboards) - Vehicle data ## Related diff --git a/docs/reference/services/loki.md b/docs/reference/services/loki.md index 6515c53..b015802 100644 --- a/docs/reference/services/loki.md +++ b/docs/reference/services/loki.md @@ -36,12 +36,16 @@ Log aggregation system for BlumeOps infrastructure. **From Kubernetes (via Alloy DaemonSet):** - All pods in all namespaces +**From Fly.io proxy (via embedded Alloy):** +- nginx JSON access logs (`{instance="flyio-proxy", job="flyio-nginx"}`) + ## Query Examples (LogQL) ```logql {service="forgejo"} # All forgejo logs {service="borgmatic", stream="stderr"} # Borgmatic errors {host="indri"} |= "error" # All logs containing "error" +{instance="flyio-proxy"} |= "docs.eblu.me" # Fly.io proxy access logs for docs ``` ## Related diff --git a/docs/reference/services/prometheus.md b/docs/reference/services/prometheus.md index 3d3cd44..67e491f 100644 --- a/docs/reference/services/prometheus.md +++ b/docs/reference/services/prometheus.md @@ -25,6 +25,7 @@ Metrics storage and querying for BlumeOps infrastructure. ### Remote Write (from Alloy) - Indri system metrics via [[alloy|Alloy]] remote_write - Textfile metrics: minikube, borgmatic, zot, jellyfin +- [[flyio-proxy]] nginx metrics (`flyio_nginx_*`) via Alloy embedded in Fly.io container ### Scrape Targets