eblume/blumeops
1
1
Fork 0
Code Issues Pull requests Projects Releases 83 Packages Wiki Activity Actions
83 releases 160 tags
  • v1.14.3 2cb0ce4289

    BlumeOps v1.14.3 Stable

    eblume released this 2026-03-22 18:20:38 -07:00 | 337 commits to main since this release

    BlumeOps release v1.14.3

    What's Changed

    Features

    • Deploy infrastructure alerting pipeline using Grafana Unified Alerting with ntfy push notifications. 7 alert rules with runbooks covering service health, pod readiness, PostgreSQL, textfile freshness, Frigate cameras, and ArgoCD sync status. services-check now queries the alerting API for covered checks.

    Bug Fixes

    • Fix Frigate NVR crash by re-adding required mqtt config section (disabled) after Mosquitto removal.
    • Fix borgmatic backup failure: use correct kubectl context (minikube) on indri for Mealie SQLite dump hook

    Infrastructure

    • Localize Grafana Alloy container image with dual Dockerfile + Nix builds from forge mirror
    • Upgrade Prometheus from v3.9.1 to v3.10.0 (distroless variants, PromQL fill operators, performance improvements)
    • Bump Frigate recording retention (180d continuous, 30d detections, 730d alerts) and add camera-fps health check to services-check.
    • Improve Frigate health checks in services-check: per-camera FPS validation and NFS storage accessibility check.
    • Increase data retention: Prometheus 15d → 10y, Loki 31d → 365d (PVC sizes unchanged; minikube hostpath doesn't enforce limits)
    • Standardize OCI labels across all container Dockerfiles with consistent title, description, version, source, and vendor metadata.

    Documentation

    • Review and correct Tailscale reference doc: fix ACL path, add missing device tags (ringtail, per-service tags, ci-gateway, flyio-proxy), correct access matrix (PyPI→DevPI, homelab grants), add SSH homelab→homelab rule, document auto approvers, add last-reviewed frontmatter.

    AI Assistance

    • Add four Claude Code subagents: infra-health (background health monitor), doc-reviewer (persistent-memory doc review), change-classifier (C0/C1/C2 triage), and mikado-navigator (C2 chain state advisor).

    Miscellaneous

    • Standardized USAGE pragmas and typer CLI parsing across all mise tasks: added missing #USAGE directive to mikado-branch-invariant-check, converted pr-comments and op-backup from raw sys.argv to typer for consistency with all other uv python scripts.

    Documentation

    Download docs-v1.14.3.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.eblu.me/eblume/blumeops/releases/download/v1.14.3/docs-v1.14.3.tar.gz
    Downloads
  • v1.14.2 1f000c8e39

    BlumeOps v1.14.2 Stable

    eblume released this 2026-03-17 13:24:09 -07:00 | 374 commits to main since this release

    BlumeOps release v1.14.2

    What's Changed

    Features

    • Deploy Mealie recipe manager on minikube-indri for meal planning and prep automation.
    • Add UnPoller deployment to monitor UniFi network metrics via Prometheus

    Bug Fixes

    • Fix Caddy v2.11 breaking change: preserve original Host header for HTTPS upstreams.
    • Fix plan-a-meal random recipe queries — add required paginationSeed parameter

    Infrastructure

    • Externalize Tailscale operator manifest to forge mirror, removing 495 KB vendored file from the repo.
    • Externalize TeslaMate Grafana dashboards to forge mirror, removing 713 KB of ConfigMaps from the repo.
    • Upgrade Caddy from v2.10.2 to v2.11.2 (7 CVE fixes), create caddy-l4 forge mirror, migrate all ~/code/3rd clones on indri to HTTPS forge.ops.eblu.me remotes.
    • Upgrade borgmatic from 2.0.13 to 2.1.3 on indri (improved borg warning handling, memory/performance improvements)

    Documentation

    • Add git last-modified subsort to docs-review script, so ties in review date are broken by least recently updated first.
    • Review jellyfin (10.11.6, current) and automounter (1.11.0) services; add missing frigate share to automounter docs.

    Documentation

    Download docs-v1.14.2.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.eblu.me/eblume/blumeops/releases/download/v1.14.2/docs-v1.14.2.tar.gz
    Downloads
  • v1.14.1 8b3b17d555

    BlumeOps v1.14.1 Stable

    eblume released this 2026-03-14 10:11:03 -07:00 | 397 commits to main since this release

    BlumeOps release v1.14.1

    What's Changed

    Features

    • Add docs-preview mise task: builds docs with Dagger and serves them locally in the production quartz container, opening the browser directly to the specified card. Also adds visual preview hints to the docs-review checklist and the review-documentation how-to.

    Infrastructure

    • Add jobsync to services-check and homepage dashboard; mark as reviewed at v1.1.4
    • Bump Grafana Alloy to v1.14.0 across all deployments (indri, alloy-k8s, alloy-ringtail, alloy-tracing-ringtail)
    • Upgrade zot container registry from v2.1.13 to v2.1.15 (CVE-2025-30204, open redirect fix). Fix trivy CVE DB downloads by adding /usr/local/bin to LaunchAgent PATH.
    • Remove Mosquitto (MQTT broker) — unused since frigate-notify switched to webapi polling. Deleted ArgoCD app, k8s manifests, namespace, and updated all docs.

    Documentation

    • Add how-to card for running the 1Password backup (mise run op-backup), with bidirectional links to restore procedure and service reference.

    Documentation

    Download docs-v1.14.1.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.eblu.me/eblume/blumeops/releases/download/v1.14.1/docs-v1.14.1.tar.gz
    Downloads
  • v1.14.0 0ef5fe5792

    BlumeOps v1.14.0 Stable

    eblume released this 2026-03-09 12:03:26 -07:00 | 411 commits to main since this release

    BlumeOps release v1.14.0

    What's Changed

    Features

    • Deploy JobSync to ringtail k3s — nix-built container, Tailscale Ingress, Caddy route at jobsync.ops.eblu.me, Ollama integration for AI features.

    Bug Fixes

    • Fix 1Password Connect logs showing as errors in Grafana by normalizing numeric log levels (1-5) to standard strings (error/warn/info/debug/trace) in the Alloy log processing pipeline.
    • Fix mikado-branch-invariant-check false positive: close commits without preceding impl commits are valid (e.g., operational tasks with no code changes).

    Infrastructure

    • Disable Quartz SPA mode and remove robots.txt crawler exclusions to fix the Facebook crawler spider trap. Remove hand-curated category index files in favor of Quartz auto-generated folder pages.

    Documentation

    • Add JobSync reference card, update ringtail workloads table, document observability via Loki, and wire RAPIDAPI_KEY through ExternalSecret for job search automation.
    • Relax wiki-link constraints: allow path-based links for disambiguation, drop global filename uniqueness requirement, remove docs-check-filenames and docs-check-index hooks.

    Documentation

    Download docs-v1.14.0.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.eblu.me/eblume/blumeops/releases/download/v1.14.0/docs-v1.14.0.tar.gz
    Downloads
  • v1.13.3 55013db124

    BlumeOps v1.13.3 Stable

    eblume released this 2026-03-06 20:48:56 -08:00 | 426 commits to main since this release

    BlumeOps release v1.13.3

    What's Changed

    Infrastructure

    • Upgrade Dagger engine and CLI from v0.20.0 to v0.20.1.

    Documentation

    • Add how-to guide for upgrading Dagger, documenting the correct phase ordering to avoid chicken-and-egg CI failures.

    Documentation

    Download docs-v1.13.3.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.eblu.me/eblume/blumeops/releases/download/v1.13.3/docs-v1.13.3.tar.gz
    Downloads
  • v1.13.2 a7c21bd8a6

    BlumeOps v1.13.2 Stable

    eblume released this 2026-03-06 19:03:21 -08:00 | 431 commits to main since this release

    BlumeOps release v1.13.2

    What's Changed

    Infrastructure

    • Replace nginx spider-trap 404 guards with robots.txt disallowing /explorer/ to prevent crawler-induced infinite URL trees.

    Documentation

    Download docs-v1.13.2.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.eblu.me/eblume/blumeops/releases/download/v1.13.2/docs-v1.13.2.tar.gz
    Downloads
  • v1.13.1 1537412c09

    BlumeOps v1.13.1 Stable

    eblume released this 2026-03-06 10:00:37 -08:00 | 434 commits to main since this release

    BlumeOps release v1.13.1

    What's Changed

    Infrastructure

    • Add :kustomized sentinel tag to all manifest image references overridden by kustomize, making it clear the real tag lives in kustomization.yaml.
    • Add nginx spider-trap guards to docs.eblu.me Quartz container — blocks recursive crawler paths at /tags/ depth >1 and global depth ≥5.

    Documentation

    Download docs-v1.13.1.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.eblu.me/eblume/blumeops/releases/download/v1.13.1/docs-v1.13.1.tar.gz
    Downloads
  • v1.13.0 46cc3fbc2e

    BlumeOps v1.13.0 Stable

    eblume released this 2026-03-05 11:11:29 -08:00 | 440 commits to main since this release

    BlumeOps release v1.13.0

    What's Changed

    Features

    • Add Authentik OIDC login for ArgoCD — eblume (admins group) gets admin access via SSO while local admin password remains as break-glass.
    • Expose Forgejo publicly at forge.eblu.me via Fly.io reverse proxy with rate limiting, fail2ban, and security hardening.
    • Deploy Ollama LLM server on ringtail with GPU acceleration and declarative model management
    • Add distributed tracing via Grafana Tempo and Beyla eBPF auto-instrumentation. Tempo runs on minikube-indri for trace storage, while a privileged Alloy DaemonSet on ringtail uses Beyla to instrument HTTP services (Frigate, ntfy, Ollama, Immich) without code changes. Grafana gets trace-to-log and trace-to-metrics correlation.
    • Add fly.io nginx proxy observability and application logs to Forgejo dashboard; rename from "Forgejo Repository Health" to "Forgejo".

    Bug Fixes

    • Add per-torrent rate metrics using Transmission's native rate_download/rate_upload fields. Dashboard panels were querying cumulative byte gauges (torrent size) instead of actual transfer rates.
    • Fix Frigate database loss on pod restart by pointing database path to persistent /db volume
    • Fix runner-job-image Dagger version mismatch: bump from 0.19.11 to 0.20.0 to match upgraded Dagger module.

    Infrastructure

    • Home-build grafana-sidecar container image, replacing upstream quay.io/kiwigrid/k8s-sidecar for supply chain control.
    • Add HA (2 replicas + PDB) for CV and Docs services for zero-downtime deploys.
    • Build Loki container image locally instead of pulling from upstream
    • Replace unmaintained metalmatze/transmission-exporter sidecar with homegrown Python exporter using prometheus_client and transmission-rpc. Same metric names, so Grafana dashboards work unchanged.
    • Upgrade Transmission from 4.0.6-r4 to 4.1.1-r1 (Alpine edge community repo)
    • Bump Frigate memory limit from 2Gi to 3Gi to prevent OOMKills under steady-state ONNX + CUDA workload.
    • Add Gandi bookmark to homepage dashboard
    • Allow implicit octals in yamllint and use 0755 directly in k8s manifests instead of decimal or disable-line comments.
    • Upgrade Dagger engine and CLI from v0.19.11 to v0.20.0
    • Upgrade TeslaMate from v2.2.0 to v3.0.0 (dark mode, BRIN index optimization, Elixir 1.19.5, trixie-slim runtime)
    • Add OOMKilled Containers stat panel and Container Restarts timeseries to the Kubernetes Clusters dashboard for persistent OOMKill visibility.
    • Add pre-commit hook to prevent changelog fragments from being placed in subdirectories.
    • Bump kiwix-serve from 3.8.1 to 3.8.2

    Documentation

    • Clarify that changelog fragments apply to all change levels (C0, C1, C2), not just C2.
    • Add reference card for the Ollama LLM inference service.
    • Clarify that all mikado frontmatter is removed during chain finalization; clean up stale frontmatter from closed chains; fix ai-docs exit code after plans directory retirement.
    • Retire docs plans directory: deleted completed/abandoned plans, converted migrate-forgejo-from-brew to a mikado chain root card, removed plans references from tutorials and how-to index.
    • Review and fix upgrade-grafana doc: correct image tag reference to kustomization.yaml, add sidecar cross-reference, update stale service-versions notes.
    • Use towncrier orphan fragment naming (+slug.<type>.md) for C0 changes to avoid main.* collisions.

    Documentation

    Download docs-v1.13.0.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.eblu.me/eblume/blumeops/releases/download/v1.13.0/docs-v1.13.0.tar.gz
    Downloads
  • v1.12.1 7a1875936c

    BlumeOps v1.12.1 Stable

    eblume released this 2026-03-02 18:17:06 -08:00 | 493 commits to main since this release

    BlumeOps release v1.12.1

    What's Changed

    Features

    • Mikado branch invariant hook now rejects impl commits that modify Mikado card files (docs with requires:, status:, or branch: mikado/ frontmatter).

    Infrastructure

    • Switch git hooks from pre-commit to prek, a faster Rust-native drop-in replacement. Adds built-in checks for case conflicts, private key detection, and executable shebangs. Configuration migrated from .pre-commit-config.yaml to prek.toml.

    Documentation

    • Review build-authentik-from-source Mikado chain: fix go-server-derivation path errors, remove stale DRF fork content from mirror doc, add last-reviewed to all cards.

    Documentation

    Download docs-v1.12.1.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.ops.eblu.me/eblume/blumeops/releases/download/v1.12.1/docs-v1.12.1.tar.gz
    Downloads
  • v1.12.0 2a2811d7a5

    BlumeOps v1.12.0 Stable

    eblume released this 2026-03-01 17:24:08 -08:00 | 501 commits to main since this release

    BlumeOps release v1.12.0

    What's Changed

    Bug Fixes

    • Fix authentik 2026.2.0 startup crash caused by Django migration ordering bug (FieldError: Cannot resolve keyword 'group_id'). Patch ensures authentik_core/0056 runs before authentik_rbac/0010.

    Infrastructure

    • Upgrade authentik from 2025.10.1 to 2026.2.0, building core services from source via custom Nix derivations rather than using nixpkgs directly (nixpkgs still provides satellite dependencies like Python, Go, and system libraries). Four components (API client generation, Python backend, web UI, Go server) assembled into a single container image with full supply chain control via forge mirrors.
    • Sync Frigate zone coordinates from live API to manifest (driveway_entrance, driveway)
    • Pin blumeops-pg to PostgreSQL 18.3 (from floating :18 tag at 18.1)

    Documentation

    • Review and update authentik-api-client-generation doc: remove stale patch note, fix test-build.nix section, add last-reviewed date.
    • Review all three forgejo-runner Mikado chain docs: stamp last-reviewed, add cross-links, fix configmap.yaml → config.yaml reference.
    • Review build-grafana-container docs; fix stale grafana.md reference card (Helm → Kustomize).

    Documentation

    Download docs-v1.12.0.tar.gz and configure the quartz container with:

    DOCS_RELEASE_URL=https://forge.ops.eblu.me/eblume/blumeops/releases/download/v1.12.0/docs-v1.12.0.tar.gz
    Downloads