diff --git a/docs/changelog.d/feature-forge-public.feature.md b/docs/changelog.d/feature-forge-public.feature.md new file mode 100644 index 0000000..44be391 --- /dev/null +++ b/docs/changelog.d/feature-forge-public.feature.md @@ -0,0 +1 @@ +Expose Forgejo publicly at forge.eblu.me via Fly.io reverse proxy with rate limiting, fail2ban, and security hardening. diff --git a/docs/how-to/configuration/expose-service-publicly.md b/docs/how-to/configuration/expose-service-publicly.md index b342481..bb6b258 100644 --- a/docs/how-to/configuration/expose-service-publicly.md +++ b/docs/how-to/configuration/expose-service-publicly.md @@ -1,7 +1,7 @@ --- title: Expose a Service Publicly -modified: 2026-02-16 -last-reviewed: 2026-02-16 +modified: 2026-03-03 +last-reviewed: 2026-03-03 tags: - how-to - fly-io @@ -259,7 +259,7 @@ server { } ``` -**Dynamic service template** (e.g., Forgejo — hypothetical, not currently deployed): +**Dynamic service template** (e.g., Forgejo — see `fly/nginx.conf` for the live configuration): ```nginx # --- forge.eblu.me (dynamic, authenticated) --- @@ -440,32 +440,30 @@ see plan history in git). ### fail2ban fail2ban monitors log files for repeated failed authentication attempts -(SSH brute force, bad login passwords, API abuse) and bans IPs via -firewall rules. +and bans offending IPs. **Static sites**: fail2ban does not apply. There is no login surface, no sessions, no credentials to brute force. -**Dynamic services with authentication** (e.g., Forgejo): fail2ban is -relevant and should be configured on **indri**, not on Fly.io. The -nginx proxy is transparent — it forwards requests but does not see -authentication outcomes. fail2ban watches the service's own logs on -indri for patterns like repeated failed logins. +**Dynamic services with authentication** (e.g., Forgejo): fail2ban +runs in the **Fly.io container**, not on indri. Standard iptables +banning won't work in Fly.io because `$remote_addr` is Fly's internal +proxy IP, not the client. Instead, fail2ban uses a custom nginx-based +ban action: -Setup considerations for Forgejo specifically: +1. fail2ban watches the nginx JSON access log for repeated 401/403 + responses to login endpoints, keyed on the `client_ip` field + (populated from the `Fly-Client-IP` header) +2. On ban, it appends the IP to `/etc/nginx/forge-deny.conf` and + reloads nginx +3. nginx uses a `geo` directive keyed on `$http_fly_client_ip` to + check the deny list and return 403 for banned IPs -- Forgejo logs failed auth attempts to its log file -- fail2ban needs a filter matching Forgejo's log format -- Banned IPs are blocked at indri's firewall (the Fly.io proxy IP is - the Tailscale address of the `flyio-proxy` node, not the end user's - IP) -- **Important**: for fail2ban to see real client IPs, the nginx proxy - must pass `X-Real-IP` / `X-Forwarded-For` headers (included in the - dynamic service nginx config above), and Forgejo must be configured - to trust the proxy and log the forwarded IP rather than the proxy's - Tailscale IP -- Disable open user registration before exposing Forgejo publicly — - require explicit invites +Ban lists are **ephemeral across deploys** — nginx rate limiting +provides the persistent baseline; fail2ban adds escalating bans for +active attacks. + +See `fly/fail2ban/` for the filter, jail, and action configuration. ### Break-glass shutoff @@ -504,7 +502,7 @@ dynamic, authenticated service like [[forgejo]]. - [ ] Disable open user registration (require invites or admin approval) - [ ] Audit access controls and permissions - [ ] Configure the service to log the forwarded client IP (not the proxy IP) -- [ ] Set up fail2ban on indri with a filter for the service's log format +- [ ] Set up fail2ban in the Fly.io container with a filter for the service's login endpoints - [ ] Tag the service's Tailscale Ingress with `tag:flyio-target` - [ ] Test the nginx config locally or in staging before deploying - [ ] Rehearse the break-glass shutoff (`mise run fly-shutoff`) diff --git a/docs/reference/infrastructure/routing.md b/docs/reference/infrastructure/routing.md index 092119f..91457e9 100644 --- a/docs/reference/infrastructure/routing.md +++ b/docs/reference/infrastructure/routing.md @@ -1,6 +1,6 @@ --- title: Routing -modified: 2026-02-09 +modified: 2026-03-03 tags: - infrastructure - networking @@ -49,6 +49,7 @@ DNS CNAMEs point to `blumeops-proxy.fly.dev`. TLS via Fly.io-managed Let's Encry | Service | URL | Description | |---------|-----|-------------| | [[docs]] | https://docs.eblu.me | Documentation site | +| [[forgejo]] | https://forge.eblu.me | Git hosting (public) | ## Tailscale-Only Services diff --git a/docs/reference/services/forgejo.md b/docs/reference/services/forgejo.md index 68fd1f4..504107e 100644 --- a/docs/reference/services/forgejo.md +++ b/docs/reference/services/forgejo.md @@ -1,6 +1,6 @@ --- title: Forgejo -modified: 2026-02-20 +modified: 2026-03-03 tags: - service - git @@ -15,7 +15,8 @@ Git forge and CI/CD platform. **Primary source of truth for blumeops** (mirrored | Property | Value | |----------|-------| -| **URL** | https://forge.ops.eblu.me | +| **URL (public)** | https://forge.eblu.me | +| **URL (internal)** | https://forge.ops.eblu.me | | **SSH** | `ssh://forgejo@forge.ops.eblu.me:2222` | | **Local Ports** | 3001 (HTTP), 2200 (SSH) | | **Config** | `ansible/roles/forgejo/templates/app.ini.j2` | @@ -94,23 +95,30 @@ This is a bootstrapping requirement - the PAT enables IaC for all other secrets. **Break-glass:** Local password login always works (with local MFA). Authentik SSO is additive — if Authentik is down, log in with local credentials. -## Future: Public Access +## Public Access -Forgejo can be exposed publicly at `forge.eblu.me` via [[flyio-proxy]]. Since Forgejo runs natively on [[indri]] (not in k8s), the pattern is: +Forgejo is publicly accessible at `https://forge.eblu.me` via [[flyio-proxy]]. This is the first dynamic, authenticated service exposed publicly. -1. Create a k8s ExternalName Service pointing to indri's Tailscale IP -2. Create a Tailscale Ingress with `tailscale.com/tags: "tag:k8s,tag:flyio-target"` -3. Add the nginx server block and DNS CNAME +| Access Method | URL | Reachable From | +|---------------|-----|----------------| +| **HTTPS (public)** | https://forge.eblu.me | Public internet | +| **HTTPS (internal)** | https://forge.ops.eblu.me | Tailnet only | +| **SSH** | `ssh://forgejo@forge.ops.eblu.me:2222` | Tailnet only | -Exposing a dynamic, authenticated service like Forgejo requires a full security review before going live: +The UI shows `forge.eblu.me` for HTTPS clone URLs and `forge.ops.eblu.me` for SSH clone URLs. -- Disable all local registration — only allow login via [[authentik]] (`DISABLE_REGISTRATION = true`, `ALLOW_ONLY_EXTERNAL_REGISTRATION = true`) -- Configure fail2ban on indri with a filter for Forgejo's log format -- Ensure Forgejo logs the forwarded client IP (`X-Real-IP`) rather than the proxy's Tailscale IP -- Audit repository visibility defaults and permissions -- Rehearse the break-glass shutoff (`mise run fly-shutoff`) +### Security Controls -See [[expose-service-publicly]] for the full howto and dynamic service checklist. +- **Registration:** Local registration disabled; only [[authentik]] SSO login allowed (`ALLOW_ONLY_EXTERNAL_REGISTRATION = true`) +- **Reverse proxy trust:** `REVERSE_PROXY_LIMIT = 2`, `REVERSE_PROXY_TRUSTED_PROXIES = *` — Forgejo logs the real client IP from `X-Real-IP` header, not the proxy's Tailscale IP +- **Rate limiting:** nginx rate limits login/signup/forgot-password endpoints (3r/s per client IP via `Fly-Client-IP` header) +- **fail2ban:** Runs in the Fly.io container; bans IPs after 5 failed logins in 10 minutes via nginx deny list (ephemeral across deploys) +- **Swagger:** Blocked at the proxy (`/swagger` returns 403); use forge.ops.eblu.me for API access +- **OAuth dead-end:** "Sign in with Authentik" redirects to the (tailnet-only) Authentik URL — SSO only works from the tailnet + +### Break-glass + +`mise run fly-shutoff` stops all public traffic immediately. forge.ops.eblu.me continues to work from the tailnet. See [[expose-service-publicly#Break-glass shutoff]]. ## Related