---
title: Prometheus
tags:
  - service
  - observability
---

# Prometheus

Metrics storage and querying for BlumeOps infrastructure.

## Quick Reference

| Property | Value |
|----------|-------|
| **URL** | https://prometheus.ops.eblu.me |
| **Tailscale URL** | https://prometheus.tail8d86e.ts.net |
| **Namespace** | `monitoring` |
| **Image** | `prom/prometheus:v3.2.1` |
| **Storage** | 50Gi PVC |

## Data Sources

### Remote Write (from Alloy)
- Indri system metrics via [[alloy\|Alloy]] remote_write
- Textfile metrics: minikube, borgmatic, zot, jellyfin

### Scrape Targets
| Target | Metrics |
|--------|---------|
| `sifaka:9100` | [[storage/sifaka\|Sifaka]] NAS (node_exporter) |
| `cnpg-metrics.tail8d86e.ts.net:9187` | [[postgresql\|CloudNativePG]] metrics |
| `kube-state-metrics.monitoring.svc:8080` | Kubernetes resource metrics |

## Query API

```bash
# Check targets
curl -s https://prometheus.ops.eblu.me/api/v1/targets | jq '.data.activeTargets[].scrapeUrl'
```

## ArgoCD Management

```bash
argocd app sync prometheus
```

Manifests: `argocd/manifests/prometheus/`

## Related

- [[alloy\|Alloy]] - Metrics collector
- [[grafana\|Grafana]] - Visualization
- [[loki\|Loki]] - Logs counterpart