Erich Blume a7d771d945 Add docs/ directory with blumeops zk cards

Move 21 blumeops-tagged zettelkasten cards from ~/code/personal/zk/
to docs/ in this repository. These files are symlinked back into the
zk at ~/code/personal/zk/blumeops for seamless obsidian.nvim integration.

This enables:
- Git-managed documentation in the blumeops repo
- Preserved wiki links between blumeops docs
- obsidian-sync isolation (docs don't sync to other devices)
- Direct editing via obsidian.nvim with the blumeops workspace

Also updates zk-docs mise task to read from local docs/ directory.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-02-02 19:09:19 -08:00

3.3 KiB

Raw Blame History

aliases

Prometheus Management Log

Prometheus provides metrics storage and querying for the 1767747119-YCPO infrastructure, running in Kubernetes (minikube on indri).

Service Details

URL: https://prometheus.tail8d86e.ts.net
Namespace: monitoring
Image: prom/prometheus:v3.2.1
ArgoCD app: prometheus
Storage: 50Gi PVC

Data Sources

Remote Write (from Alloy)

Indri system metrics via alloy remote_write
Textfile metrics: minikube, borgmatic, zot, jellyfin

Scrape Targets

sifaka:9100 - Synology NAS (node_exporter in Docker)
cnpg-metrics.tail8d86e.ts.net:9187 - CloudNativePG PostgreSQL metrics
kube-state-metrics.monitoring.svc:8080 - Kubernetes resource metrics (pods, deployments, etc.)

Useful Commands

# View logs
kubectl --context=minikube-indri -n monitoring logs -f prometheus-0

# Check targets
curl -s https://prometheus.tail8d86e.ts.net/api/v1/targets | jq '.data.activeTargets[].scrapeUrl'

# Sync from ArgoCD
argocd app sync prometheus

ArgoCD Management

Prometheus is deployed via ArgoCD from argocd/manifests/prometheus/:

statefulset.yaml - StatefulSet with 50Gi PVC
configmap.yaml - Prometheus configuration
service.yaml - ClusterIP service
ingress-tailscale.yaml - Tailscale Ingress

Log

Wed Jan 22 2026 (observability cleanup)

Added kube-state-metrics scrape target for k8s resource metrics
Enhanced Minikube dashboard with namespace filtering and resource usage panels
Uses kube_pod_info, kube_pod_container_resource_requests, etc.

Wed Jan 22 2026 (later)

Migrated to Kubernetes - moved from Homebrew on indri to k8s StatefulSet
Exposed via Tailscale Ingress at prometheus.tail8d86e.ts.net
Remote write endpoint now at k8s service, Alloy updated to push there
Retired ansible prometheus role from indri
Added ACL grant for tag:homelab → tag:k8s on port 443 for Alloy access

Wed Jan 22 2026

Added CNPG PostgreSQL metrics scraping. The CloudNativePG operator exposes Prometheus metrics on port 9187. Exposed via Tailscale at cnpg-metrics.tail8d86e.ts.net:9187 and added to scrape_configs as job cnpg-postgres.

Wed Jan 15 2026

Prometheus now accepts metrics via remote_write from alloy. The --web.enable-remote-write-receiver flag was added to enable this.

Indri metrics are no longer scraped - they're pushed by Alloy. Sifaka still uses traditional scraping via node_exporter running in Docker on the Synology.

Mon Jan 13 2026

Prometheus is now managed via ansible in 1767747119-YCPO. Configuration files are templated from the ansible role.

Mon Jan 12 2026 21:56

Prometheus was stood up about a week ago at this point. I am currently renaming localhost to indri in the scrape_configs. While I'm here I'm going to see if I can add Synology stats.

I'm adding Container Manager to Sifaka now. I should probably have a Sifaka management log, but not yet. Downloaded prom/node-exporter and made a container for it. Using the latest tag because I'm nasty.

Done. Adding to scrape configs.

Ok, it didn't like the indri hostname. Could probably fix somehow with either magicdns or /etc/hosts but for now, I'm using relabel_configs. This is working. Gotta go to bed.

3.3 KiB Raw Blame History