blumeops/ansible/playbooks/indri.yml
Erich Blume 17023085cb Migrate observability stack to Kubernetes (#42)
Note: the name of this branch was chosen before the scope widened to encompass the entire observability stack.

Summary

  - Fix Grafana data source URLs (docker driver uses host.minikube.internal, not host.containers.internal)
  - Migrate Prometheus and Loki from indri to Kubernetes with Tailscale Ingresses
  - Expose CNPG PostgreSQL metrics via Tailscale and update dashboard to use cnpg_* metrics
  - Update Alloy to push metrics/logs to k8s endpoints (prometheus.tail8d86e.ts.net, loki.tail8d86e.ts.net)
  - Add ACL rule for port 9187 (CNPG metrics)
  - Delete obsolete ansible roles for prometheus and loki

Changes

  - argocd/manifests/prometheus/ - New Prometheus StatefulSet with 20Gi PVC and Tailscale Ingress
  - argocd/manifests/loki/ - New Loki StatefulSet with 20Gi PVC and Tailscale Ingress
  - argocd/apps/prometheus.yaml, argocd/apps/loki.yaml - ArgoCD Applications
  - argocd/manifests/grafana/values.yaml - Data sources now use k8s internal DNS
  - argocd/manifests/databases/service-metrics-tailscale.yaml - CNPG metrics endpoint
  - argocd/manifests/grafana-config/dashboards/configmap-postgresql.yaml - Updated to cnpg_* metrics
  - ansible/roles/alloy/defaults/main.yml - Push to k8s Tailscale endpoints
  - pulumi/policy.hujson - ACL for port 9187
  - Deleted ansible/roles/prometheus/ and ansible/roles/loki/

Deployment and Testing

  - Stop prometheus and loki on indri
  - Sync ArgoCD apps (apps, prometheus, loki, grafana)
  - Run mise run provision-indri -- --tags alloy
  - Verify Grafana dashboards show data

🤖 Generated with https://claude.ai/claude-code

Reviewed-on: https://forge.tail8d86e.ts.net/eblume/blumeops/pulls/42
2026-01-22 12:06:02 -08:00

45 lines
1.2 KiB
YAML

---
- name: Configure indri
hosts: indri
# Fetch 1Password credentials upfront to minimize prompts
# Each role also fetches its own credentials (with 'when: <var> is not defined')
# so they still work when running with --tags
pre_tasks:
- name: Fetch borgmatic database password
ansible.builtin.command:
cmd: op --vault vg6xf6vvfmoh5hqjjhlhbeoaie item get mw2bv5we7woicjza7hc6s44yvy --fields db-password --reveal
delegate_to: localhost
register: _borgmatic_db_pw
changed_when: false
no_log: true
check_mode: false
tags: [borgmatic]
- name: Set borgmatic database password fact
ansible.builtin.set_fact:
borgmatic_db_password: "{{ _borgmatic_db_pw.stdout }}"
no_log: true
tags: [borgmatic]
roles:
- role: alloy
tags: alloy
- role: borgmatic
tags: borgmatic
- role: borgmatic_metrics
tags: borgmatic_metrics
- role: forgejo
tags: forgejo
- role: zot
tags: zot
- role: zot_metrics
tags: zot_metrics
- role: minikube
tags: minikube
- role: minikube_metrics
tags: minikube_metrics
- role: plex_metrics
tags: plex_metrics
- role: tailscale_serve
tags: tailscale-serve