eblume/blumeops

Erich Blume 2fa536e547 C2(deploy-infra-alerting): impl add textfile staleness and Frigate alerts

- TextfileStale: fires when a .prom textfile on indri hasn't been
  updated in 1 hour (node_textfile_mtime_seconds). Covers borgmatic,
  zot, minikube, jellyfin exporters.
- FrigateCameraDown: fires when frigate_camera_fps drops to 0 for 5m.
- Add runbooks for both alerts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-03-22 13:43:16 -07:00

809 B

Raw Blame History

title

modified

tags

Observability

2026-03-22

operations

Observability

Metrics, logs, traces, and dashboards for BlumeOps infrastructure.

Components

prometheus - Metrics storage and querying
loki - Log aggregation
tempo - Distributed tracing
alloy - Metrics, log, and trace collection
grafana - Dashboards and visualization

Alerting

deploy-infra-alerting - Alerting pipeline (Grafana Unified Alerting → ntfy)
runbook-service-probe-failure - Service health check failure runbook
runbook-postgres-unhealthy - PostgreSQL cluster health runbook
runbook-pod-not-ready - Pod not ready runbook
runbook-textfile-stale - Metrics textfile freshness runbook
runbook-frigate-camera-down - Frigate camera health runbook