blumeops/docs/how-to/configuration/update-documentation.md
Erich Blume a87c997ee1
All checks were successful
Deploy Fly.io Proxy / deploy (push) Successful in 1m28s
Expose Forgejo publicly at forge.eblu.me (#278)
## Summary

Expose Forgejo publicly at `forge.eblu.me` via the Fly.io reverse proxy — the first dynamic, authenticated public-facing service.

- **Forgejo hardening:** Domain changed to forge.eblu.me, SSH stays on forge.ops.eblu.me, reverse proxy trust headers configured, local registration locked to external-only (Authentik SSO)
- **Tailscale Ingress:** ExternalName Service + Ingress in tailscale-operator creates forge.tail8d86e.ts.net endpoint
- **Fly.io proxy:** nginx server block with rate-limited auth endpoints (3r/s), fail2ban with custom nginx-deny action, security headers, /swagger blocked, WebSocket support, 512m body limit
- **Authentik:** OAuth callback updated to forge.eblu.me
- **DNS/TLS:** CNAME record in Pulumi, cert in fly-setup
- **Rename:** ~29 files updated from forge.ops.eblu.me to forge.eblu.me (HTTPS refs only; SSH, container builds, and Caddy table kept as-is)

## Deployment Order

1. `mise run provision-indri -- --tags forgejo` (config changes)
2. Verify forge.ops.eblu.me still works
3. `argocd app set tailscale-operator --revision feature/forge-public && argocd app sync tailscale-operator`
4. Verify `curl https://forge.tail8d86e.ts.net`
5. `cd fly && fly deploy`
6. Verify pre-DNS: `curl -H "Host: forge.eblu.me" https://blumeops-proxy.fly.dev/`
7. `fly certs add forge.eblu.me -a blumeops-proxy`
8. `argocd app set authentik --revision feature/forge-public && argocd app sync authentik`
9. `mise run dns-preview && mise run dns-up`
10. Full verification (see below)
11. Rehearse `mise run fly-shutoff`
12. After merge: reset ArgoCD revisions to main, re-sync

## Verification Checklist

- [ ] forge.eblu.me loads, shows public repos
- [ ] forge.ops.eblu.me still works from tailnet
- [ ] SSH clone via forge.ops.eblu.me:2222 works
- [ ] HTTPS clone via forge.eblu.me works
- [ ] UI shows forge.eblu.me for HTTPS clone, forge.ops.eblu.me for SSH
- [ ] /swagger returns 403
- [ ] Rapid login attempts trigger 429 rate limit
- [ ] fail2ban bans after 5 failed logins in 10 minutes
- [ ] ArgoCD can still sync (SSH unaffected)
- [ ] `mise run fly-shutoff` stops all public traffic
- [ ] `mise run services-check` passes

Reviewed-on: #278
2026-03-03 08:40:41 -08:00

3.9 KiB

title modified last-reviewed tags
Update Documentation 2026-02-19 2026-02-19
how-to
documentation
ci-cd

Update Documentation

How to publish documentation changes to https://docs.eblu.me.

Quick Release

After merging documentation changes to main:

  1. Go to Actions > Build BlumeOps > Run workflow
  2. Select version bump type (patch/minor/major) or enter a specific version
  3. The workflow builds, releases, and deploys automatically

Direct link: https://forge.eblu.me/eblume/blumeops/actions?workflow=build-blumeops.yaml

What the Workflow Does

The build-blumeops workflow (.forgejo/workflows/build-blumeops.yaml):

  1. Resolves version — Uses input or auto-increments from latest release
  2. Builds changelog — Runs towncrier on the runner to update CHANGELOG.md
  3. Builds docs — Calls dagger call build-docs (Quartz build in a container)
  4. Creates release — Uploads docs-<version>.tar.gz to Forgejo releases
  5. Updates deployment — Edits argocd/manifests/docs/deployment.yaml with new URL
  6. Commits changes — Pushes changelog and deployment updates to main
  7. Deploys — Syncs the docs ArgoCD app
  8. Purges cache — Clears the nginx cache on the flyio-proxy so the new docs are served immediately

Changelog Fragments (Towncrier)

When making changes, add a changelog fragment to docs/changelog.d/:

# Format: <identifier>.<type>.md
# Types: feature, bugfix, infra, doc, ai, misc

# Using branch name (preferred)
echo "Add new feature X" > docs/changelog.d/my-feature.feature.md

# Orphan fragment (when no branch fits)
echo "Fix bug Y" > docs/changelog.d/+fix-bug.bugfix.md

Fragments are automatically collected into CHANGELOG.md (at repo root) during release.

Fragment types:

Type Description
feature New features
bugfix Bug fixes
infra Infrastructure changes
doc Documentation updates
ai AI assistance changes
misc Other changes

Runner Environment

The workflow runs on the k8s label, which uses the forgejo-runner in Kubernetes:

  • Runner deployment: argocd/manifests/forgejo-runner/
  • Job image: registry.ops.eblu.me/blumeops/runner-job-image (commit-SHA tagged)
  • Build engine: dagger CLI installed at runtime; Node.js and Python run inside Dagger containers

The job image is built from containers/forgejo-runner/Dockerfile.

Quartz Static Site Generator

Quartz builds the documentation into a static site with:

  • Wiki-link support ([[page]] syntax)
  • Backlinks panel showing what references each page
  • Graph view of document connections
  • Full-text search

Configuration files (in docs/):

  • quartz.config.ts - Site metadata, plugins, theme
  • quartz.layout.ts - Page layout components

Quartz is cloned fresh during each build (not vendored) to use the latest version.

Manual Build (Local)

To test docs locally without triggering a release:

# Build docs tarball (identical to CI)
dagger call build-docs --src=. --version=dev export --path=./docs-dev.tar.gz

# Inspect the output
tar tf docs-dev.tar.gz | head -20

# Debug a Quartz build failure interactively
dagger call --interactive build-docs --src=. --version=dev

Troubleshooting

Workflow fails on "Resolve version":

  • Check if the version already exists as a release
  • Ensure version format is vX.Y.Z

Docs not updating after deploy:

  • Check ArgoCD sync status: argocd app get docs
  • Verify the pod restarted: kubectl --context=minikube-indri -n docs get pods
  • Check pod logs for download errors

Towncrier not finding fragments:

  • Fragments must be in docs/changelog.d/
  • Must have .md extension
  • Must match pattern <name>.<type>.md
  • docs - Documentation service reference
  • dagger - Build engine reference
  • forgejo - Git forge and CI/CD
  • argocd - GitOps deployment