Erich Blume
649ae15204
Remove backslash escaping from pipe characters in wiki-links. Correct: [[services/forgejo|Forgejo]] Wrong: [[services/forgejo\|Forgejo]] Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
54 lines
1.2 KiB
Markdown
54 lines
1.2 KiB
Markdown
---
|
|
title: Prometheus
|
|
tags:
|
|
- service
|
|
- observability
|
|
---
|
|
|
|
# Prometheus
|
|
|
|
Metrics storage and querying for BlumeOps infrastructure.
|
|
|
|
## Quick Reference
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| **URL** | https://prometheus.ops.eblu.me |
|
|
| **Tailscale URL** | https://prometheus.tail8d86e.ts.net |
|
|
| **Namespace** | `monitoring` |
|
|
| **Image** | `prom/prometheus:v3.2.1` |
|
|
| **Storage** | 50Gi PVC |
|
|
|
|
## Data Sources
|
|
|
|
### Remote Write (from Alloy)
|
|
- Indri system metrics via [[alloy|Alloy]] remote_write
|
|
- Textfile metrics: minikube, borgmatic, zot, jellyfin
|
|
|
|
### Scrape Targets
|
|
| Target | Metrics |
|
|
|--------|---------|
|
|
| `sifaka:9100` | [[storage/sifaka|Sifaka]] NAS (node_exporter) |
|
|
| `cnpg-metrics.tail8d86e.ts.net:9187` | [[postgresql|CloudNativePG]] metrics |
|
|
| `kube-state-metrics.monitoring.svc:8080` | Kubernetes resource metrics |
|
|
|
|
## Query API
|
|
|
|
```bash
|
|
# Check targets
|
|
curl -s https://prometheus.ops.eblu.me/api/v1/targets | jq '.data.activeTargets[].scrapeUrl'
|
|
```
|
|
|
|
## ArgoCD Management
|
|
|
|
```bash
|
|
argocd app sync prometheus
|
|
```
|
|
|
|
Manifests: `argocd/manifests/prometheus/`
|
|
|
|
## Related
|
|
|
|
- [[alloy|Alloy]] - Metrics collector
|
|
- [[grafana|Grafana]] - Visualization
|
|
- [[loki|Loki]] - Logs counterpart
|