GitOps repository for personal infrastructure management
  • Nix 32.5%
  • Jinja 21.5%
  • Python 17.9%
  • Shell 11.8%
  • Go 8.1%
  • Other 8.2%
Find a file
Erich Blume cc54b4f565
All checks were successful
Deploy Fly.io Proxy / deploy (push) Successful in 1m16s
Add Fly.io proxy observability via embedded Alloy (#123)
## Summary

- Embed Grafana Alloy in the Fly.io proxy container to collect nginx JSON access logs (→ Loki) and derive request rate, latency histogram, cache status, and bandwidth metrics (→ Prometheus)
- Add nginx `stub_status` endpoint for connection-level metrics (active/reading/writing/waiting)
- Create two Grafana dashboards: **Docs APM** (per-service view filtered by `host="docs.eblu.me"`) and **Fly.io Proxy Health** (aggregate proxy health across all upstream services)

## Changed Files

| File | Change |
|------|--------|
| `fly/nginx.conf` | Add JSON `log_format` + `access_log`, add `stub_status` endpoint |
| `fly/Dockerfile` | COPY Alloy binary from `grafana/alloy:v1.5.1`, COPY `alloy.river` config |
| `fly/alloy.river` | **New** — Alloy config: log tailing, metric extraction, remote_write |
| `fly/start.sh` | Start Alloy after Tailscale, before nginx |
| `argocd/manifests/grafana-config/dashboards/configmap-docs-apm.yaml` | **New** — Docs APM dashboard |
| `argocd/manifests/grafana-config/dashboards/configmap-flyio.yaml` | **New** — Fly.io Proxy Health dashboard |
| `argocd/manifests/grafana-config/kustomization.yaml` | Register new dashboard configmaps |
| `docs/reference/services/flyio-proxy.md` | Document observability setup |

## Deployment and Testing

- [ ] `mise run fly-deploy` — rebuild container with Alloy
- [ ] `curl https://docs.eblu.me/` — generate traffic
- [ ] `fly logs -a blumeops-proxy` — verify Alloy startup
- [ ] Query Prometheus: `flyio_nginx_http_requests_total{instance="flyio-proxy"}`
- [ ] Query Loki: `{instance="flyio-proxy", job="flyio-nginx"}`
- [ ] `argocd app sync grafana-config` — deploy dashboards
- [ ] Verify dashboards show data in Grafana
- [ ] `mise run services-check` — no regressions

Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/123
2026-02-08 10:05:38 -08:00
.claude Add pre-commit hooks for code quality (#19) 2026-01-16 19:33:02 -08:00
.forgejo Add Fly.io public reverse proxy for docs.eblu.me (#120) 2026-02-08 02:36:19 -08:00
.github Rename github readme to avoid display issues on github.com 2026-01-29 11:32:55 -08:00
ansible Fix Fly.io deploy token quoting (#121) 2026-02-08 02:43:45 -08:00
argocd Add Fly.io proxy observability via embedded Alloy (#123) 2026-02-08 10:05:38 -08:00
containers Fix argocd CLI download in forgejo-runner image 2026-02-03 17:11:36 -08:00
docs Add Fly.io proxy observability via embedded Alloy (#123) 2026-02-08 10:05:38 -08:00
fly Add Fly.io proxy observability via embedded Alloy (#123) 2026-02-08 10:05:38 -08:00
mise-tasks Add docs.eblu.me and Fly.io to services-check (#122) 2026-02-08 02:48:15 -08:00
pulumi Add Fly.io proxy observability via embedded Alloy (#123) 2026-02-08 10:05:38 -08:00
.ansible-lint Add pre-commit hooks for code quality (#19) 2026-01-16 19:33:02 -08:00
.gitignore Add pre-commit hooks for code quality (#19) 2026-01-16 19:33:02 -08:00
.pre-commit-config.yaml Remove title slug check and test duplicate titles (#116) 2026-02-07 21:26:18 -08:00
.yamllint.yaml K8s Migration Phase 1: Infrastructure Setup (#29) 2026-01-19 09:49:52 -08:00
Brewfile Add Fly.io public reverse proxy for docs.eblu.me (#120) 2026-02-08 02:36:19 -08:00
CHANGELOG.md Update docs release to v1.5.0 2026-02-08 10:37:30 +00:00
CLAUDE.md Rename indri-services-check to services-check (#103) 2026-02-04 07:49:15 -08:00
mise.toml Review exploring-the-docs and add doc consistency checks (#112) 2026-02-05 21:12:06 -08:00
README.md Add Fly.io public reverse proxy for docs.eblu.me (#120) 2026-02-08 02:36:19 -08:00
towncrier.toml Fix Quartz build to preserve git history for accurate file dates (#105) 2026-02-04 08:25:46 -08:00

blumeops

                    l0K                                k..:k.
                  .:...c.                            ;c....
                    ....'o                          x.....
                      ....k                        x....
                       ... l'                    'c....
                        ....,l                  o'....
                         .....x                k....
                          .....d.             c....
                            ... l            x....
                              .,.d         ;c.c'
                               'c':;      x',c.
                                .:,'o   .x.::.
                                 .;:.k ,:.c'
                                   ,c.c';:.
                                    .,.:;.
                                   ;'.c, l
                                  d',c..:.d.
                                 O.:;.  'c';c
                               ;c.c'     .:;.x
                              o',c.       .;:.k
                             x.::.          'c.l.
                         dOKl.c,             .c,'o
                   0l'...... ..'              .::.ocx.
                 'o ............              o .... :olx;
                x,ox;. ....... .k             ....,dKKo;..x
              'd,OXXXXk:. ...... ;            ;:dXOl;',';l;o;
             x,oXXXXXXXXXkc. ...              .lc,',':dKNNNx;x;
           ;o;0KXXXXXXXXXXXX0l.                .',ckNNNNNNNNNxco0d
          l,d0oOXKOKXXXXKXXXX0.                  kNNNNNNNNNNNNNXxloo::
             .OXxdXKOX0kXXXX0.                   .KNNNNNNNNNNXONX0o.
                ,OdxKldXXXXx.                     ,NNNNNNNNNNNKoc
                   :.OXXkKo                       .kNNNNNNNNXx.
                      ':0c                         .NdNkXkc

Blue Mops — GitOps for Erich Blume's personal computing environment.

What is this?

Infrastructure-as-code for my tailnet (tail8d86e.ts.net). This repo contains ansible playbooks, configuration, and automation for managing my personal infrastructure.

This codebase was heavily co-authored by Claude Code, as an experiment in LLM-assisted development. I want to include a personal note here that I don't know entirely how I feel about LLMs in our current era, but it felt important to learn.

Development

Pre-commit Hooks

This repo uses pre-commit for code quality and consistency. Install hooks with:

uvx pre-commit install

Run all hooks manually:

uvx pre-commit run --all-files

Hooks include:

  • General: trailing whitespace, end-of-file fixer, large files, merge conflicts
  • Secrets: TruffleHog for secret detection
  • YAML: yamllint, ansible-lint
  • Python: ruff (linting + formatting)
  • Shell: shellcheck, shfmt
  • TOML: taplo
  • JSON: prettier

CI/CD

This repo uses Forgejo Actions for CI/CD. Workflows live in .forgejo/workflows/ (not .github/workflows/). The runner executes jobs in host mode within the Kubernetes cluster.

Documentation

Documentation lives in docs/ and follows the Diataxis framework. Published at https://docs.eblu.me.

Docs use Obsidian wiki-link syntax ([[link]]) for cross-references. Edit with any markdown editor, or use obsidian.nvim for enhanced navigation.