blumeops/mise-tasks/container-build-and-release

212 lines
6.7 KiB
Text
Raw Normal View History

#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.12"
C1: SHA-pin tooling dependencies (2026-04 cycle) (#344) ## Summary Monthly tooling dependency refresh, with a one-time conversion from version-tag pins (`rev = "vX.Y.Z"`, `image:tag`, `>=`) to SHA / digest pins everywhere. ## Changes - **prek hooks**: all `rev = "vX.Y.Z"` → commit SHA + `# vX.Y.Z` comment. Bumped trufflehog (3.94.0→3.95.2), kingfisher (1.91.0→1.97.0), ruff (0.15.7→0.15.12), shfmt (3.13.0→3.13.1), prettier (3.8.1→3.8.3), actionlint (1.7.11→1.7.12). - **fly/Dockerfile**: tag pins → `image@sha256:...` digest pins. Bumped nginx (1.29.6→1.30.0-alpine), tailscale (v1.94.1→v1.94.2 — still inside the safe pre-1.96.5 range), alloy (v1.14.1→v1.16.0). - **mise-tasks**: PEP 723 inline deps converted from `>=` to `==` (PEP 508 doesn't support hashes inline). All scripts pinned to current latest: rich 15.0.0, typer 0.25.0, pyyaml 6.0.3, httpx 0.28.1. - **prek `additional_dependencies`**: ansible-lint==26.4.0, ansible-core==2.20.5. - **taplo-lint**: pass `--no-schema`. Upstream's `--default-schema-catalogs` returns a format taplo v0.9.3 can't parse — we don't validate against TOML schemas anyway, so this turns off the broken catalog fetch. - **docs/update-tooling-dependencies**: documents the SHA-pin convention, `docker buildx imagetools inspect` for digest lookup, and `prek clean` before re-verifying (cache grows to several GiB). Forgejo workflow `actions/checkout@v6.0.2` was already at the latest SHA — no change. ## Test plan - [x] `prek run --all-files` passes after `prek clean` - [x] `deploy-fly` workflow builds and deploys the new fly image on merge - [x] `fly status -a blumeops-proxy` healthy after deploy - [x] Spot-check a few mise tasks (`mise run blumeops-tasks`, `mise run docs-check-links`) to confirm pinned deps resolve cleanly Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/344
2026-04-30 16:51:43 -07:00
# dependencies = ["typer==0.25.0", "httpx==0.28.1"]
# ///
#MISE description="Trigger container build workflows via Forgejo API"
#USAGE arg "<container>" help="Container name (directory under containers/)"
#USAGE flag "--ref <ref>" help="Commit SHA or branch to build (defaults to current HEAD)"
#USAGE flag "--dry-run" help="Show what would be done without triggering"
"""Trigger container build workflow via Forgejo API dispatch.
Dispatches the unified build-container workflow, which handles both
Dockerfile and Nix builds in a single workflow.
"""
import subprocess
import sys
import time
from pathlib import Path
import httpx
import typer
REGISTRY = "registry.ops.eblu.me"
Expose Forgejo publicly at forge.eblu.me (#278) ## Summary Expose Forgejo publicly at `forge.eblu.me` via the Fly.io reverse proxy — the first dynamic, authenticated public-facing service. - **Forgejo hardening:** Domain changed to forge.eblu.me, SSH stays on forge.ops.eblu.me, reverse proxy trust headers configured, local registration locked to external-only (Authentik SSO) - **Tailscale Ingress:** ExternalName Service + Ingress in tailscale-operator creates forge.tail8d86e.ts.net endpoint - **Fly.io proxy:** nginx server block with rate-limited auth endpoints (3r/s), fail2ban with custom nginx-deny action, security headers, /swagger blocked, WebSocket support, 512m body limit - **Authentik:** OAuth callback updated to forge.eblu.me - **DNS/TLS:** CNAME record in Pulumi, cert in fly-setup - **Rename:** ~29 files updated from forge.ops.eblu.me to forge.eblu.me (HTTPS refs only; SSH, container builds, and Caddy table kept as-is) ## Deployment Order 1. `mise run provision-indri -- --tags forgejo` (config changes) 2. Verify forge.ops.eblu.me still works 3. `argocd app set tailscale-operator --revision feature/forge-public && argocd app sync tailscale-operator` 4. Verify `curl https://forge.tail8d86e.ts.net` 5. `cd fly && fly deploy` 6. Verify pre-DNS: `curl -H "Host: forge.eblu.me" https://blumeops-proxy.fly.dev/` 7. `fly certs add forge.eblu.me -a blumeops-proxy` 8. `argocd app set authentik --revision feature/forge-public && argocd app sync authentik` 9. `mise run dns-preview && mise run dns-up` 10. Full verification (see below) 11. Rehearse `mise run fly-shutoff` 12. After merge: reset ArgoCD revisions to main, re-sync ## Verification Checklist - [ ] forge.eblu.me loads, shows public repos - [ ] forge.ops.eblu.me still works from tailnet - [ ] SSH clone via forge.ops.eblu.me:2222 works - [ ] HTTPS clone via forge.eblu.me works - [ ] UI shows forge.eblu.me for HTTPS clone, forge.ops.eblu.me for SSH - [ ] /swagger returns 403 - [ ] Rapid login attempts trigger 429 rate limit - [ ] fail2ban bans after 5 failed logins in 10 minutes - [ ] ArgoCD can still sync (SSH unaffected) - [ ] `mise run fly-shutoff` stops all public traffic - [ ] `mise run services-check` passes Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/278
2026-03-03 08:40:41 -08:00
FORGE_URL = "https://forge.eblu.me"
FORGE_API = f"{FORGE_URL}/api/v1"
REPO = "eblume/blumeops"
FORGE_ACTIONS = f"{FORGE_URL}/{REPO}/actions"
WORKFLOW = "build-container.yaml"
app = typer.Typer(add_completion=False)
def git(*args: str) -> str:
result = subprocess.run(
["git", *args], capture_output=True, text=True, check=True
)
return result.stdout.strip()
def get_forge_token() -> str:
result = subprocess.run(
["op", "read", "op://blumeops/w3663ffnvkewbftncqxtcpeavy/api-token"],
capture_output=True,
text=True,
check=True,
)
return result.stdout.strip()
def max_run_number(headers: dict[str, str]) -> int:
"""Return the highest current run_number for WORKFLOW, or 0 if none."""
resp = httpx.get(
f"{FORGE_API}/repos/{REPO}/actions/tasks",
params={"limit": 50},
headers=headers,
timeout=15,
)
if resp.status_code != 200:
return 0
runs = [
t["run_number"]
for t in resp.json().get("workflow_runs", [])
if t.get("workflow_id") == WORKFLOW
]
return max(runs, default=0)
def find_dispatched_run(
ref: str, floor: int, headers: dict[str, str], timeout_s: int = 20
) -> int | None:
"""Poll the tasks endpoint for the run triggered by our dispatch.
Matches by head_sha + workflow + run_number > floor so we don't pick up
an older build of the same commit or a concurrent unrelated dispatch.
"""
deadline = time.monotonic() + timeout_s
while time.monotonic() < deadline:
resp = httpx.get(
f"{FORGE_API}/repos/{REPO}/actions/tasks",
params={"limit": 20},
headers=headers,
timeout=15,
)
if resp.status_code == 200:
for task in resp.json().get("workflow_runs", []):
if (
task.get("head_sha") == ref
and task.get("workflow_id") == WORKFLOW
and task.get("run_number", 0) > floor
):
return task["run_number"]
time.sleep(1)
return None
def list_containers() -> None:
typer.echo("Available containers:")
for d in sorted(Path("containers").iterdir()):
if not d.is_dir():
continue
types = []
Native Dagger container builds + Navidrome v0.61.1 (#330) ## Summary - Move Dagger module from `.dagger/` to repo root (`src/blumeops/`), rename `blumeops-ci` → `blumeops` - Replace opaque `docker_build()` with native Dagger pipelines that surface full build errors per step - Migrate navidrome as the first container (`containers/navidrome/container.py`) - Upgrade navidrome from v0.60.3 to v0.61.1 (major artwork overhaul, SQLite FTS5 search, server-managed transcoding) - Add `dagger call container-version` for CI version extraction without Dockerfile parsing - All mise tasks (`container-list`, `container-version-check`, `container-build-and-release`) updated for hybrid mode - Legacy `docker_build()` fallback preserved for all other containers ## Motivation When navidrome v0.61.0 added a new Go build tag (`sqlite_fts5`), `docker_build()` showed only "exit code: 1". We had to run `docker build --progress=plain` manually to find `undefined: buildtags.SQLITE_FTS5`. Native Dagger pipelines show the full error inline. ## Container build dispatch needed After merge, dispatch container build for navidrome: ``` mise run container-build-and-release navidrome --ref 470b4bd ``` ## Deploy steps 1. Wait for container build to complete 2. Back up navidrome-data PVC (non-reversible DB migrations) 3. `argocd app set navidrome --revision main && argocd app sync navidrome` 4. Verify at https://dj.ops.eblu.me ## Future Remaining containers migrate incrementally in follow-up PRs using the same pattern. Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/330
2026-04-11 17:11:56 -07:00
if (d / "container.py").exists():
types.append("dagger")
if (d / "Dockerfile").exists():
types.append("dockerfile")
if (d / "default.nix").exists():
types.append("nix")
if types:
typer.echo(f" - {d.name} ({', '.join(types)})")
@app.command()
def main(
container: str = typer.Argument(help="Container name (directory under containers/)"),
ref: str = typer.Option("", "--ref", help="Commit SHA to build (defaults to current HEAD)"),
dry_run: bool = typer.Option(False, "--dry-run", help="Show what would be done without triggering"),
) -> None:
"""Trigger container build workflows via Forgejo API dispatch."""
container_dir = Path("containers") / container
Native Dagger container builds + Navidrome v0.61.1 (#330) ## Summary - Move Dagger module from `.dagger/` to repo root (`src/blumeops/`), rename `blumeops-ci` → `blumeops` - Replace opaque `docker_build()` with native Dagger pipelines that surface full build errors per step - Migrate navidrome as the first container (`containers/navidrome/container.py`) - Upgrade navidrome from v0.60.3 to v0.61.1 (major artwork overhaul, SQLite FTS5 search, server-managed transcoding) - Add `dagger call container-version` for CI version extraction without Dockerfile parsing - All mise tasks (`container-list`, `container-version-check`, `container-build-and-release`) updated for hybrid mode - Legacy `docker_build()` fallback preserved for all other containers ## Motivation When navidrome v0.61.0 added a new Go build tag (`sqlite_fts5`), `docker_build()` showed only "exit code: 1". We had to run `docker build --progress=plain` manually to find `undefined: buildtags.SQLITE_FTS5`. Native Dagger pipelines show the full error inline. ## Container build dispatch needed After merge, dispatch container build for navidrome: ``` mise run container-build-and-release navidrome --ref 470b4bd ``` ## Deploy steps 1. Wait for container build to complete 2. Back up navidrome-data PVC (non-reversible DB migrations) 3. `argocd app set navidrome --revision main && argocd app sync navidrome` 4. Verify at https://dj.ops.eblu.me ## Future Remaining containers migrate incrementally in follow-up PRs using the same pattern. Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/330
2026-04-11 17:11:56 -07:00
has_container_py = (container_dir / "container.py").exists()
has_dockerfile = (container_dir / "Dockerfile").exists()
has_nix = (container_dir / "default.nix").exists()
Native Dagger container builds + Navidrome v0.61.1 (#330) ## Summary - Move Dagger module from `.dagger/` to repo root (`src/blumeops/`), rename `blumeops-ci` → `blumeops` - Replace opaque `docker_build()` with native Dagger pipelines that surface full build errors per step - Migrate navidrome as the first container (`containers/navidrome/container.py`) - Upgrade navidrome from v0.60.3 to v0.61.1 (major artwork overhaul, SQLite FTS5 search, server-managed transcoding) - Add `dagger call container-version` for CI version extraction without Dockerfile parsing - All mise tasks (`container-list`, `container-version-check`, `container-build-and-release`) updated for hybrid mode - Legacy `docker_build()` fallback preserved for all other containers ## Motivation When navidrome v0.61.0 added a new Go build tag (`sqlite_fts5`), `docker_build()` showed only "exit code: 1". We had to run `docker build --progress=plain` manually to find `undefined: buildtags.SQLITE_FTS5`. Native Dagger pipelines show the full error inline. ## Container build dispatch needed After merge, dispatch container build for navidrome: ``` mise run container-build-and-release navidrome --ref 470b4bd ``` ## Deploy steps 1. Wait for container build to complete 2. Back up navidrome-data PVC (non-reversible DB migrations) 3. `argocd app set navidrome --revision main && argocd app sync navidrome` 4. Verify at https://dj.ops.eblu.me ## Future Remaining containers migrate incrementally in follow-up PRs using the same pattern. Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/330
2026-04-11 17:11:56 -07:00
if not has_container_py and not has_dockerfile and not has_nix:
typer.echo(f"Error: No container.py, Dockerfile, or default.nix found in '{container_dir}'")
typer.echo()
list_containers()
raise typer.Exit(1)
if not ref:
ref = git("rev-parse", "HEAD")
else:
# Resolve short SHAs or branch names to full SHA
ref = git("rev-parse", ref)
short_sha = ref[:7]
image = f"blumeops/{container}"
# Show expected builds
builds = []
Native Dagger container builds + Navidrome v0.61.1 (#330) ## Summary - Move Dagger module from `.dagger/` to repo root (`src/blumeops/`), rename `blumeops-ci` → `blumeops` - Replace opaque `docker_build()` with native Dagger pipelines that surface full build errors per step - Migrate navidrome as the first container (`containers/navidrome/container.py`) - Upgrade navidrome from v0.60.3 to v0.61.1 (major artwork overhaul, SQLite FTS5 search, server-managed transcoding) - Add `dagger call container-version` for CI version extraction without Dockerfile parsing - All mise tasks (`container-list`, `container-version-check`, `container-build-and-release`) updated for hybrid mode - Legacy `docker_build()` fallback preserved for all other containers ## Motivation When navidrome v0.61.0 added a new Go build tag (`sqlite_fts5`), `docker_build()` showed only "exit code: 1". We had to run `docker build --progress=plain` manually to find `undefined: buildtags.SQLITE_FTS5`. Native Dagger pipelines show the full error inline. ## Container build dispatch needed After merge, dispatch container build for navidrome: ``` mise run container-build-and-release navidrome --ref 470b4bd ``` ## Deploy steps 1. Wait for container build to complete 2. Back up navidrome-data PVC (non-reversible DB migrations) 3. `argocd app set navidrome --revision main && argocd app sync navidrome` 4. Verify at https://dj.ops.eblu.me ## Future Remaining containers migrate incrementally in follow-up PRs using the same pattern. Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/330
2026-04-11 17:11:56 -07:00
if has_container_py or has_dockerfile:
label = "dagger" if has_container_py else "dockerfile"
builds.append(f" {label:12s} -> {REGISTRY}/{image}:v<version>-{short_sha}")
if has_nix:
Native Dagger container builds + Navidrome v0.61.1 (#330) ## Summary - Move Dagger module from `.dagger/` to repo root (`src/blumeops/`), rename `blumeops-ci` → `blumeops` - Replace opaque `docker_build()` with native Dagger pipelines that surface full build errors per step - Migrate navidrome as the first container (`containers/navidrome/container.py`) - Upgrade navidrome from v0.60.3 to v0.61.1 (major artwork overhaul, SQLite FTS5 search, server-managed transcoding) - Add `dagger call container-version` for CI version extraction without Dockerfile parsing - All mise tasks (`container-list`, `container-version-check`, `container-build-and-release`) updated for hybrid mode - Legacy `docker_build()` fallback preserved for all other containers ## Motivation When navidrome v0.61.0 added a new Go build tag (`sqlite_fts5`), `docker_build()` showed only "exit code: 1". We had to run `docker build --progress=plain` manually to find `undefined: buildtags.SQLITE_FTS5`. Native Dagger pipelines show the full error inline. ## Container build dispatch needed After merge, dispatch container build for navidrome: ``` mise run container-build-and-release navidrome --ref 470b4bd ``` ## Deploy steps 1. Wait for container build to complete 2. Back up navidrome-data PVC (non-reversible DB migrations) 3. `argocd app set navidrome --revision main && argocd app sync navidrome` 4. Verify at https://dj.ops.eblu.me ## Future Remaining containers migrate incrementally in follow-up PRs using the same pattern. Reviewed-on: https://forge.eblu.me/eblume/blumeops/pulls/330
2026-04-11 17:11:56 -07:00
builds.append(f" {'nix':12s} -> {REGISTRY}/{image}:v<version>-{short_sha}-nix")
if dry_run:
typer.echo("[dry-run mode]")
typer.echo(f"Container: {container}")
typer.echo(f"Commit: {ref} ({short_sha})")
typer.echo(f"Expected builds:")
for b in builds:
typer.echo(b)
typer.echo()
if dry_run:
typer.echo(f"[dry-run] Would dispatch {WORKFLOW}")
typer.echo()
typer.echo("Monitor builds with: mise run runner-logs")
typer.echo(f" or visit: {FORGE_ACTIONS}")
return
token = get_forge_token()
headers = {
"Authorization": f"token {token}",
"Content-Type": "application/json",
}
# Verify the commit has been pushed to the remote
resp = httpx.get(
f"{FORGE_API}/repos/{REPO}/git/commits/{ref}",
headers=headers,
timeout=15,
)
if resp.status_code != 200:
typer.echo(f"Error: commit {short_sha} not found on remote")
typer.echo("Push your changes before triggering a build: git push origin main")
raise typer.Exit(1)
# Snapshot the highest existing run_number so we can identify the one
# our dispatch creates.
floor = max_run_number(headers)
url = f"{FORGE_API}/repos/{REPO}/actions/workflows/{WORKFLOW}/dispatches"
payload = {
"ref": "main",
"inputs": {
"container": container,
"ref": ref,
},
}
resp = httpx.post(url, json=payload, headers=headers, timeout=30)
if resp.status_code == 204:
typer.echo(f"Dispatched {WORKFLOW}")
else:
typer.echo(f"Error dispatching {WORKFLOW}: {resp.status_code} {resp.text}")
raise typer.Exit(1)
typer.echo()
run_number = find_dispatched_run(ref, floor, headers)
if run_number is not None:
typer.echo(f"Monitor builds with: mise run runner-logs {run_number}")
else:
typer.echo("Monitor builds with: mise run runner-logs")
typer.echo(f" or visit: {FORGE_ACTIONS}")
if __name__ == "__main__":
app()