Erich Blume
c281fb5403
## Summary
Adds the third observability pillar — **distributed tracing** — alongside existing metrics (Prometheus) and logs (Loki).
- **Grafana Tempo 2.10.1** on minikube-indri for trace storage with 7d retention, OTLP receivers, and `metrics_generator` that remote-writes span-metrics (RED) to Prometheus
- **Beyla eBPF auto-instrumentation** via a privileged Alloy DaemonSet on ringtail — instruments HTTP services (Frigate, ntfy, Ollama, Immich) without code changes
- **Grafana integration** — Tempo datasource with trace↔log and trace↔metrics correlation, plus Loki derivedFields for trace ID linking
- **Prometheus** scrapes Tempo operational metrics
### Architecture
```
ringtail (k3s) indri (minikube)
┌──────────────────────┐ ┌─────────────────────┐
│ Alloy+Beyla (eBPF) │──OTLP HTTP────────→ │ Tempo │
│ ↳ Frigate, ntfy, │ via tailnet │ ↳ trace storage │
│ Ollama, Immich │ │ ↳ RED → Prometheus │
└──────────────────────┘ │ │
│ Grafana │
│ ↳ Tempo datasource │
└─────────────────────┘
```
### New files (12)
- `docs/reference/services/tempo.md` — reference doc
- `docs/changelog.d/feature-otel-tracing.feature.md`
- `argocd/apps/tempo.yaml` + `argocd/manifests/tempo/` (6 files)
- `argocd/apps/alloy-tracing-ringtail.yaml` + `argocd/manifests/alloy-tracing-ringtail/` (4 files)
### Modified files (6)
- `argocd/manifests/grafana/datasources.yaml` — Tempo datasource + Loki derivedFields
- `argocd/manifests/prometheus/prometheus.yml` — Tempo scrape target
- `service-versions.yaml` — tempo + alloy-tracing-ringtail entries
- `docs/reference/services/grafana.md` — Tempo in datasources table
- `docs/reference/reference.md` — Tempo in services index
- `docs/reference/operations/observability.md` — Tempo in components list
## Deployment and Testing
- [ ] Sync `apps` app to pick up new Application definitions
- [ ] `argocd app set tempo --revision feature/otel-tracing && argocd app sync tempo`
- [ ] Verify Tempo pod: `kubectl --context=minikube-indri get pods -n monitoring -l app=tempo`
- [ ] Verify Tempo ready: port-forward 3200 and `curl localhost:3200/ready`
- [ ] Verify Tailscale ingresses: `kubectl --context=minikube-indri get ingress -n monitoring`
- [ ] `argocd app set alloy-tracing-ringtail --revision feature/otel-tracing && argocd app sync alloy-tracing-ringtail`
- [ ] Check Beyla discovery in alloy-tracing logs on ringtail
- [ ] Sync grafana-config for updated datasources
- [ ] Sync prometheus for updated scrape config
- [ ] Test Grafana Tempo datasource connection
- [ ] Generate test traffic and search traces in Grafana Explore → Tempo
- [ ] After merge: reset all ArgoCD app revisions back to main
Reviewed-on: #286
53 lines
2.5 KiB
Markdown
53 lines
2.5 KiB
Markdown
---
|
|
title: Apps
|
|
modified: 2026-03-04
|
|
tags:
|
|
- kubernetes
|
|
- argocd
|
|
---
|
|
|
|
# ArgoCD Applications
|
|
|
|
Registry of all applications deployed via [[argocd]].
|
|
|
|
## Application Registry
|
|
|
|
| App | Namespace | Path/Source | Service |
|
|
|-----|-----------|-------------|---------|
|
|
| `apps` | argocd | `argocd/apps/` | App-of-apps root |
|
|
| `argocd` | argocd | `argocd/manifests/argocd/` | [[argocd]] |
|
|
| `tailscale-operator` | tailscale | `argocd/manifests/tailscale-operator/` | [[tailscale-operator]] |
|
|
| `1password-connect` | 1password | `argocd/manifests/1password-connect/` | [[1password]] |
|
|
| `external-secrets` | external-secrets | Helm chart | [[1password]] |
|
|
| `external-secrets-config` | external-secrets | `argocd/manifests/external-secrets-config/` | [[1password]] |
|
|
| `cloudnative-pg` | cnpg-system | `mirrors/cloudnative-pg` release manifest | PostgreSQL operator |
|
|
| `blumeops-pg` | databases | `argocd/manifests/databases/` | [[postgresql]] |
|
|
| `prometheus` | monitoring | `argocd/manifests/prometheus/` | [[prometheus]] |
|
|
| `loki` | monitoring | `argocd/manifests/loki/` | [[loki]] |
|
|
| `grafana` | monitoring | Helm chart (forge mirror) | [[grafana]] |
|
|
| `grafana-config` | monitoring | `argocd/manifests/grafana-config/` | [[grafana]] |
|
|
| `immich` | immich | Helm chart | [[immich]] |
|
|
| `tempo` | monitoring | `argocd/manifests/tempo/` | [[tempo]] |
|
|
| `alloy-k8s` | alloy | `argocd/manifests/alloy-k8s/` | [[alloy|Alloy]] |
|
|
| `alloy-tracing-ringtail` | alloy | `argocd/manifests/alloy-tracing-ringtail/` | [[alloy|Alloy]] (eBPF tracing) |
|
|
| `kube-state-metrics` | monitoring | `argocd/manifests/kube-state-metrics/` | K8s metrics |
|
|
| `miniflux` | miniflux | `argocd/manifests/miniflux/` | [[miniflux]] |
|
|
| `kiwix` | kiwix | `argocd/manifests/kiwix/` | [[kiwix]] |
|
|
| `torrent` | torrent | `argocd/manifests/torrent/` | [[transmission]] |
|
|
| `navidrome` | navidrome | `argocd/manifests/navidrome/` | [[navidrome]] |
|
|
| `teslamate` | teslamate | `argocd/manifests/teslamate/` | [[teslamate]] |
|
|
| `cv` | cv | `argocd/manifests/cv/` | [[cv]] |
|
|
| `forgejo-runner` | forgejo-runner | `argocd/manifests/forgejo-runner/` | [[forgejo]] CI |
|
|
| `ollama` | ollama | `argocd/manifests/ollama/` | [[ollama]] |
|
|
|
|
## Sync Policies
|
|
|
|
| Application | Policy | Rationale |
|
|
|-------------|--------|-----------|
|
|
| `apps` | Automated | Picks up new Application manifests |
|
|
| All others | Manual | Explicit control over deployments |
|
|
|
|
## Related
|
|
|
|
- [[argocd]] - GitOps platform details
|
|
- [[cluster|Cluster]] - Kubernetes infrastructure
|