eblume/blumeops
1
1
Fork 0
Code Issues Pull requests Projects Releases 83 Packages Wiki Activity Actions
blumeops/docs/zk/1768283761-TRXN.md
Erich Blume b8104d75ad Move zk cards to docs/zk/ for documentation restructuring (#84) 
## Summary
- Move all existing zettelkasten cards from `docs/` to `docs/zk/` as a temporary holding area
- Update `zk-docs` mise task to look in the new location
- Add `docs/README.md` explaining the Diataxis-based restructuring plan and target audiences

## Context
This is phase 1 of a multi-phase documentation restructuring effort. The goal is to reorganize docs to follow the Diataxis framework while serving multiple audiences:
1. Erich (owner) - knowledge graph/zk
2. Claude/AI agents - memory and context enrichment
3. New external readers - high-level overview
4. Potential operators/contributors - onboarding
5. Replicators - people wanting to duplicate the approach

## Testing
- [x] Verified `mise run zk-docs` still works with the new path
- [x] Updated obsidian.nvim config (in ~/.config/nvim) to point to new path

## Note
The obsidian.nvim config change is outside this repo but was made as part of this work.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/84
2026-02-03 09:13:50 -08:00

3.3 KiB
Raw Blame History

id aliases tags
1768283761-TRXN
prometheus
blumeops

Prometheus Management Log

Prometheus provides metrics storage and querying for the 1767747119-YCPO infrastructure, running in Kubernetes (minikube on indri).

Service Details

Data Sources

Remote Write (from Alloy)

  • Indri system metrics via alloy remote_write
  • Textfile metrics: minikube, borgmatic, zot, jellyfin

Scrape Targets

  • sifaka:9100 - Synology NAS (node_exporter in Docker)
  • cnpg-metrics.tail8d86e.ts.net:9187 - CloudNativePG PostgreSQL metrics
  • kube-state-metrics.monitoring.svc:8080 - Kubernetes resource metrics (pods, deployments, etc.)

Useful Commands

# View logs
kubectl --context=minikube-indri -n monitoring logs -f prometheus-0

# Check targets
curl -s https://prometheus.tail8d86e.ts.net/api/v1/targets | jq '.data.activeTargets[].scrapeUrl'

# Sync from ArgoCD
argocd app sync prometheus

ArgoCD Management

Prometheus is deployed via ArgoCD from argocd/manifests/prometheus/:

  • statefulset.yaml - StatefulSet with 50Gi PVC
  • configmap.yaml - Prometheus configuration
  • service.yaml - ClusterIP service
  • ingress-tailscale.yaml - Tailscale Ingress

Log

Wed Jan 22 2026 (observability cleanup)

  • Added kube-state-metrics scrape target for k8s resource metrics
  • Enhanced Minikube dashboard with namespace filtering and resource usage panels
  • Uses kube_pod_info, kube_pod_container_resource_requests, etc.

Wed Jan 22 2026 (later)

  • Migrated to Kubernetes - moved from Homebrew on indri to k8s StatefulSet
  • Exposed via Tailscale Ingress at prometheus.tail8d86e.ts.net
  • Remote write endpoint now at k8s service, Alloy updated to push there
  • Retired ansible prometheus role from indri
  • Added ACL grant for tag:homelab → tag:k8s on port 443 for Alloy access

Wed Jan 22 2026

Added CNPG PostgreSQL metrics scraping. The CloudNativePG operator exposes Prometheus metrics on port 9187. Exposed via Tailscale at cnpg-metrics.tail8d86e.ts.net:9187 and added to scrape_configs as job cnpg-postgres.

Wed Jan 15 2026

Prometheus now accepts metrics via remote_write from alloy. The --web.enable-remote-write-receiver flag was added to enable this.

Indri metrics are no longer scraped - they're pushed by Alloy. Sifaka still uses traditional scraping via node_exporter running in Docker on the Synology.

Mon Jan 13 2026

Prometheus is now managed via ansible in 1767747119-YCPO. Configuration files are templated from the ansible role.

Mon Jan 12 2026 21:56

Prometheus was stood up about a week ago at this point. I am currently renaming localhost to indri in the scrape_configs. While I'm here I'm going to see if I can add Synology stats.

I'm adding Container Manager to Sifaka now. I should probably have a Sifaka management log, but not yet. Downloaded prom/node-exporter and made a container for it. Using the latest tag because I'm nasty.

Done. Adding to scrape configs.

Ok, it didn't like the indri hostname. Could probably fix somehow with either magicdns or /etc/hosts but for now, I'm using relabel_configs. This is working. Gotta go to bed.