eblume/blumeops
1
1
Fork 0
Code Issues Pull requests Projects Releases 83 Packages Wiki Activity Actions

Update Alloy, Prometheus, Loki, Grafana reference cards for flyio-proxy

Browse source 
Add Fly.io proxy as a third Alloy deployment, document the new
remote_write source in Prometheus, new log source in Loki, and
two new dashboards in Grafana.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Erich Blume 2026-02-08 10:05:13 -08:00
commit 34fceff627
4 changed files with 20 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

14
docs/reference/services/alloy.md
View file

@ -7,9 +7,10 @@ tags:
# Grafana Alloy
Unified observability collector for metrics and logs with two deployments:
Unified observability collector for metrics and logs with three deployments:
1. **Indri (host)** - System metrics and service logs from macOS host
2. **Kubernetes (DaemonSet)** - Automatic pod log collection and service health probes
3. **Fly.io proxy (embedded)** - nginx access log metrics and log forwarding from [[flyio-proxy]]
## Quick Reference
@ -20,6 +21,8 @@ Unified observability collector for metrics and logs with two deployments:
| **K8s Namespace** | `alloy` |
| **K8s Image** | `grafana/alloy:v1.8.2` |
| **ArgoCD App** | `alloy-k8s` |
| **Fly.io Config** | `fly/alloy.river` |
| **Fly.io Image** | `grafana/alloy:v1.5.1` (binary copied into nginx container) |
## Metrics Collected
@ -33,6 +36,13 @@ Unified observability collector for metrics and logs with two deployments:
- All pod logs via `loki.source.kubernetes`
- Service health probes: miniflux, kiwix, transmission, devpi, argocd
### From Fly.io Proxy
- `flyio_nginx_http_requests_total` — request rate by status/method/host
- `flyio_nginx_http_request_duration_seconds` — latency histogram
- `flyio_nginx_http_response_bytes_total` — response bandwidth
- `flyio_nginx_cache_requests_total` — cache HIT/MISS/EXPIRED counts
- Pushed to [[prometheus]] via remote_write through [[caddy]]
## Logs Collected
**Brew services:** forgejo, tailscale
@ -41,6 +51,8 @@ Unified observability collector for metrics and logs with two deployments:
Logs pushed to [[loki]] at `https://loki.tail8d86e.ts.net/loki/api/v1/push`.
**Fly.io proxy:** nginx JSON access logs pushed to [[loki]] at `https://loki.ops.eblu.me/loki/api/v1/push` (via [[caddy]]).
## Why Built from Source
The Homebrew bottle uses `CGO_ENABLED=0`, which breaks Tailscale MagicDNS. Building with `CGO_ENABLED=1` uses the macOS native resolver.

2
docs/reference/services/grafana.md
View file

@ -41,6 +41,8 @@ Optional annotation: `grafana_folder: "FolderName"`
- Minikube - Kubernetes cluster overview
- Borgmatic Backups - Backup status and trends
- Services Health - HTTP probe results
- Docs APM - Request rate, latency, cache for docs.eblu.me
- Fly.io Proxy Health - Aggregate proxy health across all upstream services
- TeslaMate (18 dashboards) - Vehicle data
## Related

4
docs/reference/services/loki.md
View file

@ -36,12 +36,16 @@ Log aggregation system for BlumeOps infrastructure.
**From Kubernetes (via Alloy DaemonSet):**
- All pods in all namespaces
**From Fly.io proxy (via embedded Alloy):**
- nginx JSON access logs (`{instance="flyio-proxy", job="flyio-nginx"}`)
## Query Examples (LogQL)
```logql
{service="forgejo"} # All forgejo logs
{service="borgmatic", stream="stderr"} # Borgmatic errors
{host="indri"} |= "error" # All logs containing "error"
{instance="flyio-proxy"} |= "docs.eblu.me" # Fly.io proxy access logs for docs
```
## Related

1
docs/reference/services/prometheus.md
View file

@ -25,6 +25,7 @@ Metrics storage and querying for BlumeOps infrastructure.
### Remote Write (from Alloy)
- Indri system metrics via [[alloy|Alloy]] remote_write
- Textfile metrics: minikube, borgmatic, zot, jellyfin
- [[flyio-proxy]] nginx metrics (`flyio_nginx_*`) via Alloy embedded in Fly.io container
### Scrape Targets