pyroscope.ebpf needs /proc/kallsyms and tracefs access for eBPF
profiling. Mount host /proc and /sys into the container.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
pyroscope.ebpf creates /tmp/symb-cache at startup. Nix-built
container images have a read-only root filesystem, so /tmp needs
an emptyDir volume mount.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
pyroscope.ebpf requires __container_id__ and service_name labels on
targets. Added relabel rules to map from Kubernetes pod metadata.
Removed targets_only which doesn't exist as an argument.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous sed deleted lines which broke the webpack config syntax.
Instead, rewrite the `from:` path to point at the pre-copied icons
directory which webpack can access without following Nix store symlinks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mkYarnPackage hoists most dependencies to the root node_modules/,
with deps/grafana-pyroscope/node_modules/ containing only package-
specific overrides. @grafana/ui lives at ../../node_modules/, not
the local node_modules/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mkYarnPackage symlinks node_modules into the read-only Nix store,
so we can't modify the directory in-place. Instead, pre-copy the
@grafana/ui icons to the webpack output location and sed-remove the
CopyPlugin entry that tries to glob through the symlink.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mkYarnPackage symlinks node_modules from the Nix store, but webpack's
CopyPlugin can't follow symlinks when resolving glob patterns. The
@grafana/ui icons directory is dereferenced (cp -rL) before running
the webpack build. Also fixes binary output path (root, not cmd/).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The webpack build fails on a @grafana/ui icons glob that doesn't
resolve in the Nix sandbox. Since Grafana is the primary UI via the
pyroscope datasource plugin, the standalone web UI isn't needed.
Build with EMBEDASSETS="" (upstream's build-dev path).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The upstream Makefile's `build` target runs frontend via Docker, which
isn't available in the Nix sandbox. Instead, build the frontend
(yarn + webpack) separately via mkYarnPackage, copy assets to
public/build/, then invoke `make go/bin` with EMBEDASSETS=embedassets
to compile the Go binary with embedded frontend.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Go toolchain auto-download (triggered by `toolchain go1.25.8` in
go.mod) puts scripts with shebangs into the module cache. Nix fixup
phase patches these to reference /nix/store paths, violating the
fixed-output derivation contract. dontFixup = true leaves the cache
as-is, which is fine since these files are only consumed as build
inputs by the pyroscope derivation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Nix derivation follows the Alloy pattern: stdenv + pre-fetched Go
modules for multi-module workspace (go.work with ./api, ./lidia).
goModules hash is a placeholder (fakeHash) — first build on ringtail
will produce the real hash. Kustomization updated to use local
registry image. Service-versions entries added for pyroscope and
alloy-profiling-ringtail.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Pyroscope server (StatefulSet on ringtail k3s) and Alloy profiling
DaemonSet with pyroscope.ebpf collection. Grafana datasource with
traces-to-profiles cross-linking. Docs updated across observability
reference card, Alloy, Grafana, apps registry, and README.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Minor release with new widgets (Tracearr, SparklyFitness), Seerr rename,
and dependency bumps. No breaking changes for our config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Upgrade External Secrets Operator from v1.3.2 (helm-chart-2.0.0) to v2.2.0
- Migrate from Helm chart deployment to static kustomize manifests, matching the repo's kustomize-first pattern
- Merge separate `-config` ArgoCD apps into the main operator apps (6 → 4 apps)
- Clean up Helm-specific labels (`helm.sh/chart`, `managed-by: Helm`)
- Update README example from v1beta1 to v1 API
## Breaking changes assessment
Low risk — v2.0.0 removed Alibaba and Device42 providers (we use neither). No templating changes affect us. All ExternalSecrets already use v1 API.
## Deployment steps
1. Sync CRDs first on both clusters (new CRD version)
2. Sync operator apps (now kustomize-based)
3. Verify ClusterSecretStore and all ExternalSecrets are healthy
4. Delete orphaned config apps: `argocd app delete external-secrets-config` and `-config-ringtail`
5. `mise run services-check`
Reviewed-on: #312
## Summary
- Add Snowflake proxy as a native systemd service on ringtail (NixOS)
- Uses `pkgs.snowflake` from nixpkgs (v2.11.0)
- Hardened systemd unit with DynamicUser, ProtectSystem=strict, 512MB memory limit
- Prometheus metrics enabled on localhost:9999
## What is Snowflake?
A Tor pluggable transport that helps censored users reach the Tor network via WebRTC. **This is NOT a Tor exit node** — traffic exits through Tor exit nodes operated by others. The proxy operator cannot see traffic content (double-encrypted) and destination servers never see the proxy's IP.
## Changes
- `nixos/ringtail/configuration.nix` — new systemd service definition
- `docs/reference/services/snowflake-proxy.md` — service reference card
- `docs/reference/infrastructure/ringtail.md` — updated systemd services section
- `service-versions.yaml` — added entry (type: nixos)
## Deploy plan
After review, deploy via `mise run provision-ringtail`. Service starts automatically.
## Test plan
- [ ] `mise run provision-ringtail` succeeds
- [ ] `ssh ringtail 'systemctl status snowflake-proxy'` shows active
- [ ] `ssh ringtail 'journalctl -u snowflake-proxy --no-pager -n 20'` shows broker connections
- [ ] `ssh ringtail 'curl -s localhost:9999/metrics'` returns Prometheus metrics
Reviewed-on: #311
The kubectl image lacks curl/python3. Use the prowler image
(which has Python) with a pure-Python urllib script instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prowler's --registry flag doesn't work (registry args not passed
to ImageProvider constructor, prowler-cloud/prowler PR #10128
regression). Use an init container to enumerate images from the
zot catalog API and generate an image list file instead.
See: https://github.com/eblume/prowler/tree/fix/image-provider-registry-args
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Image scan: add https:// scheme to registry URL.
IaC scan: use --scan-repository-url (Prowler clones the repo
itself), removing the need for an init container. The flag
is --scan-path for local dirs, --scan-repository-url for git.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Clone repo in init container, scan Dockerfiles and K8s manifests
with Prowler's IaC provider (Trivy). Reports written to
sifaka:/volume1/reports/prowler-iac/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Trivy to the Prowler container for image and IaC scanning.
New CronJob (Saturday 3am) scans all blumeops/* images in the
registry for CVEs, embedded secrets, and Dockerfile misconfigs.
Reports written to sifaka:/volume1/reports/prowler-images/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
C0 follow-up to #309: update kustomization newTag for all containers
rebuilt by the merge (authentik, authentik-redis, ntfy, alloy).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Replace upstream `docker.io/library/redis:7-alpine` (Redis 7.4.8) with a nix-built container using Redis 8.2.3 from nixpkgs
- Introduce **attached service pattern**: `parent` field in service-versions.yaml, `<parent>-<component>` naming convention, and `assert pkgs.redis.version == version` in default.nix to prevent silent version drift on `flake.lock` updates
- Document the pattern in [[review-services]] so future attached services slot in cleanly
- Backfill `parent: grafana` on existing `grafana-sidecar` entry
## Version drift protection
1. `flake.lock` update bumps nixpkgs redis → `assert` in `default.nix` breaks `nix-build`
2. Developer updates `version` in `default.nix` → prek's `container-version-check` demands matching `service-versions.yaml` update
3. Both must agree before commit succeeds
## Test plan
- [ ] Build container from branch on ringtail (`mise run container-build-and-release authentik-redis`)
- [ ] Update kustomization `newTag` to branch-built image tag
- [ ] Sync authentik ArgoCD app from branch (`argocd app set authentik --revision localize-redis && argocd app sync authentik`)
- [ ] Verify Authentik login, session persistence, and task queue still work
- [ ] After merge: C0 follow-up to update `newTag` to the main-built image tag
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #309
Bump from RC to latest stable (security fixes for config endpoint and
cross-camera auth). Add new 0.17 motion retention tier at 365 days,
reduce continuous from 180 to 30 days.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Uses the grafana_folder annotation to place the dashboard in the
existing folder created by alert rule provisioning.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two panels: currently firing alerts (firing/pending/noData/error) and
recent state changes. Refreshes every 30s. Uses Grafana's built-in
alertlist panel type — no datasource needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dramatiq defaults to one worker process per CPU core. On ringtail (16 cores)
this spawned 16 processes, each loading the full Django app, exceeding the
1Gi memory limit and causing a crash loop (228 restarts over 7 days).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Merges `build-container.yaml` and `build-container-nix.yaml` into a single workflow
- Detect job classifies each changed container by presence of `Dockerfile` and/or `default.nix`
- Dockerfile containers build on `k8s` (indri) via Dagger; Nix containers build on `nix-container-builder` (ringtail) via nix-build + skopeo
- Containers with both build files (alloy, nettest, ntfy) get built on both runners
## Test plan
- [ ] Push a change to a Dockerfile-only container (e.g. grafana) — verify it builds on k8s only
- [ ] Push a change to a nix-only container (e.g. jobsync) — verify it builds on nix-container-builder only
- [ ] Push a change to a dual container (e.g. ntfy) — verify it builds on both runners
- [ ] Test workflow_dispatch with a specific container name
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #306
Replace hardcoded image tags in Quick Reference tables with pointers to
kustomization manifests (tags drift with every container release). Fix
Prometheus CNPG scrape target, remove misleading .ts.net URLs, expand
external-secrets stub, add backup/disaster-recovery cross-references.
Limit doc-reviewer agent to one doc per cycle.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove `group: ""` from ignoreDifferences in tailscale-operator and
tailscale-operator-ringtail — ArgoCD normalizes away the empty string
field, so the live state never matches git.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
