diff --git a/docs/how-to/dagger/upgrade-dagger.md b/docs/how-to/dagger/upgrade-dagger.md index d41ea09..0ee66a6 100644 --- a/docs/how-to/dagger/upgrade-dagger.md +++ b/docs/how-to/dagger/upgrade-dagger.md @@ -1,6 +1,6 @@ --- title: Upgrade Dagger -modified: 2026-03-06 +modified: 2026-04-11 last-reviewed: 2026-03-06 tags: - how-to @@ -26,7 +26,7 @@ Dagger versions are pinned in multiple places. The runner job image (which execu | `service-versions.yaml` | `runner-job-image` version and `last-reviewed` | 1 | | `mise.toml` | `dagger` tool version | 2 | | `dagger.json` | `engineVersion` | 2 | -| `.dagger/uv.lock` | SDK dependency lock (regenerated automatically) | 2 | +| `uv.lock` | SDK dependency lock (regenerated automatically) | 2 | | `docs/reference/tools/dagger.md` | Version references in documentation | 2 | | `argocd/manifests/forgejo-runner/deployment.yaml` | `RUNNER_LABELS` image tag | 2 | @@ -63,7 +63,7 @@ Once the Phase 1 build completes, upgrade the module engine version and deploy t "engineVersion": "v" ``` -4. Regenerate the SDK lock file — run any `dagger call` command (e.g., `dagger call --help` or `dagger functions`). This updates `.dagger/uv.lock` if SDK dependencies changed. +4. Regenerate the SDK lock file — run any `dagger call` command (e.g., `dagger call --help` or `dagger functions`). This updates `uv.lock` if SDK dependencies changed. 5. Update `docs/reference/tools/dagger.md` — bump the version in the Quick Reference table and any version references in the body text. diff --git a/docs/how-to/deployment/build-container-image.md b/docs/how-to/deployment/build-container-image.md index 4b47b3f..ce746b0 100644 --- a/docs/how-to/deployment/build-container-image.md +++ b/docs/how-to/deployment/build-container-image.md @@ -1,6 +1,6 @@ --- title: Build Container Image -modified: 2026-02-24 +modified: 2026-04-11 last-reviewed: 2026-02-15 tags: - how-to @@ -14,8 +14,8 @@ How to create a custom container image in BlumeOps, build it locally, and releas ## Prerequisites -- [Dagger CLI](https://docs.dagger.io/install) installed locally (for Dockerfile builds) -- A `Dockerfile` and/or `default.nix` for the service +- [Dagger CLI](https://docs.dagger.io/install) installed locally +- A `container.py`, `Dockerfile`, and/or `default.nix` for the service ## 1. Create the container directory @@ -23,16 +23,21 @@ Add build files under `containers//`: ``` containers// -├── Dockerfile (built by Dagger on the k8s runner) +├── container.py (native Dagger pipeline — preferred for new containers) +├── Dockerfile (legacy — built via docker_build() fallback) ├── default.nix (built by nix-build on the ringtail runner) └── (optional scripts, configs) ``` -A container can have one or both build files. The directory name becomes the image name: `registry.ops.eblu.me/blumeops/`. +A container can have one or more build files. The directory name becomes the image name: `registry.ops.eblu.me/blumeops/`. + +**New containers for indri (k8s runner) should use `container.py`** — native Dagger pipelines surface full build errors per step, while `docker_build()` (used for Dockerfiles) swallows errors. See `containers/navidrome/container.py` for the reference pattern. Existing Dockerfile containers are migrated incrementally during [[review-services|service reviews]]. + +**Ringtail containers should continue using `default.nix`** — these are built by `nix-build` on the ringtail runner and don't benefit from the Dagger migration. ## 2. Build locally -**Dockerfile** — test with Dagger: +**Any container** (native `container.py` or legacy Dockerfile) — test with Dagger: ```bash dagger call build --src=. --container-name= @@ -65,10 +70,11 @@ Use `--dry-run` to preview without dispatching. | Build file | Workflow | Runner | Registry tag | |------------|----------|--------|--------------| +| `container.py` | `build-container.yaml` | `k8s` (indri) | `:vX.Y.Z-` | | `Dockerfile` | `build-container.yaml` | `k8s` (indri) | `:vX.Y.Z-` | -| `default.nix` | `build-container-nix.yaml` | `nix-container-builder` ([[ringtail]]) | `:vX.Y.Z--nix` | +| `default.nix` | `build-container.yaml` | `nix-container-builder` ([[ringtail]]) | `:vX.Y.Z--nix` | -The version (`X.Y.Z`) is extracted from `ARG CONTAINER_APP_VERSION=` in the Dockerfile or `version = "..."` in `default.nix`. The SHA is the short (7-char) commit hash. +The version (`X.Y.Z`) is extracted from `VERSION` in `container.py` (via `dagger call container-version`), `ARG CONTAINER_APP_VERSION=` in Dockerfiles, or `version = "..."` in `default.nix`. The SHA is the short (7-char) commit hash. Check available images and tags with: @@ -112,36 +118,36 @@ Existing containers demonstrate several build approaches: | Pattern | Example | Notes | |---------|---------|-------| -| Alpine package install | [[#transmission]] | Simplest — install from apk | -| Go from source | [[#miniflux]] | Clone upstream, `go build` | -| Multi-stage with Node + Go | [[#navidrome]] | Separate UI and backend build stages | -| Multi-stage Elixir | [[#teslamate]] | Elixir release with Node assets | -| Runtime tarball download | [[#kiwix-serve]] | Download pre-built binary with arch detection | -| Nix `dockerTools` | [[#ntfy-nix]] | `buildLayeredImage` with nix-built app | - -### transmission - -`containers/transmission/Dockerfile` — Installs transmission-daemon directly from Alpine packages. Good starting point for services available in apk. - -### miniflux - -`containers/miniflux/Dockerfile` — Two-stage Go build. Clones upstream at a pinned version tag, runs `make`, copies the binary into a minimal Alpine runtime. +| Native Dagger (Go + Node) | [[#navidrome]] | `container.py` with helper functions — preferred for new containers | +| Alpine package install | [[#transmission]] | Simplest Dockerfile — install from apk | +| Go from source | [[#miniflux]] | Dockerfile: clone upstream, `go build` | +| Multi-stage Elixir | [[#teslamate]] | Dockerfile: Elixir release with Node assets | +| Runtime tarball download | [[#kiwix-serve]] | Dockerfile: download pre-built binary with arch detection | +| Nix `dockerTools` | [[#ntfy-nix]] | `buildLayeredImage` with nix-built app (ringtail runner) | ### navidrome -`containers/navidrome/Dockerfile` — Three-stage build with separate Node.js UI compilation, Go backend build with CGO (taglib), and a minimal Alpine runtime with ffmpeg. +`containers/navidrome/container.py` — Native Dagger build. Three-stage pipeline using helper functions: `node_build()` for UI, `go_build()` with CGO/taglib/FTS5 for backend, `alpine_runtime()` with ffmpeg. This is the reference pattern for migrating Dockerfile containers to native Dagger builds. + +### transmission + +`containers/transmission/Dockerfile` — Installs transmission-daemon directly from Alpine packages. Good starting point for services available in apk. (Legacy Dockerfile — migrate to `container.py` during review.) + +### miniflux + +`containers/miniflux/Dockerfile` — Two-stage Go build. Clones upstream at a pinned version tag, runs `make`, copies the binary into a minimal Alpine runtime. (Legacy Dockerfile — migrate to `container.py` during review.) ### teslamate -`containers/teslamate/Dockerfile` — Two-stage Elixir build with Node.js asset compilation. Uses Debian-based images due to Elixir/OTP dependencies. +`containers/teslamate/Dockerfile` — Two-stage Elixir build with Node.js asset compilation. Uses Debian-based images due to Elixir/OTP dependencies. (Legacy Dockerfile — migrate to `container.py` during review.) ### kiwix-serve -`containers/kiwix-serve/Dockerfile` — Downloads a pre-built binary from upstream, with architecture detection for cross-platform support. +`containers/kiwix-serve/Dockerfile` — Downloads a pre-built binary from upstream, with architecture detection for cross-platform support. (Legacy Dockerfile — migrate to `container.py` during review.) ### ntfy (nix) -`containers/ntfy/default.nix` — Builds ntfy from source using `buildGoModule` and packages it with `dockerTools.buildLayeredImage`. Runs alongside the existing Dockerfile; the nix variant is tagged `:version-nix` in the registry. +`containers/ntfy/default.nix` — Builds ntfy from source using `buildGoModule` and packages it with `dockerTools.buildLayeredImage`. Runs alongside the existing Dockerfile; the nix variant is tagged `:version-nix` in the registry. Nix containers should continue using `default.nix`. ## Related diff --git a/docs/how-to/forgejo-runner/validate-workflows-against-v12.md b/docs/how-to/forgejo-runner/validate-workflows-against-v12.md index 1b6cdc0..5f98502 100644 --- a/docs/how-to/forgejo-runner/validate-workflows-against-v12.md +++ b/docs/how-to/forgejo-runner/validate-workflows-against-v12.md @@ -1,6 +1,6 @@ --- title: Validate Workflows Against v12 -modified: 2026-02-27 +modified: 2026-04-11 last-reviewed: 2026-02-27 tags: - how-to @@ -25,7 +25,7 @@ All 6 workflows pass v12.7.0 schema validation with no changes needed: ## Deliverables -1. `validate_workflows` function added to `.dagger/src/blumeops_ci/main.py` +1. `validate_workflows` function added to `src/blumeops/main.py` (formerly `.dagger/src/blumeops_ci/main.py`) - Uses `forgejo-runner validate --directory .` inside the upstream runner container - `runner_version` parameter (default `12.7.0`) pins to deployed version 2. `mise run validate-workflows` task wired to `dagger call validate-workflows` diff --git a/docs/how-to/zot/add-container-version-sync-check.md b/docs/how-to/zot/add-container-version-sync-check.md index ebf1056..ff137fb 100644 --- a/docs/how-to/zot/add-container-version-sync-check.md +++ b/docs/how-to/zot/add-container-version-sync-check.md @@ -1,6 +1,6 @@ --- title: Add Container Version Sync Check -modified: 2026-02-20 +modified: 2026-04-11 tags: - how-to - containers @@ -10,7 +10,7 @@ tags: # Add Container Version Sync Check -Add a prek check that validates version consistency across the three places container versions are declared: Dockerfile ARGs, `service-versions.yaml`, and nix derivations. No VERSION files needed — the existing sources are the source of truth, and the check enforces they agree. +Add a prek check that validates version consistency across the places container versions are declared: `container.py` VERSION constants, Dockerfile ARGs, `service-versions.yaml`, and nix derivations. The check enforces they agree. ## Context @@ -20,13 +20,14 @@ Discovered during analysis of [[adopt-commit-based-container-tags]]: the new com ### 1. Created `mise run container-version-check` task -A typer-based uv-script that iterates over `containers/*/` and validates five rules per container: +A typer-based uv-script that iterates over `containers/*/` and validates six rules per container: -1. Any Dockerfile must declare `ARG CONTAINER_APP_VERSION=` -2. Any `default.nix` must produce a version via `dagger call nix-version` -3. At least one build file must exist (Dockerfile or default.nix) -4. A matching `service-versions.yaml` entry must exist with non-null `current-version` -5. All resolved versions from (1), (2), and (4) must agree (v-prefix stripped for comparison) +1. Any `container.py` must declare `VERSION = ""` +2. Any Dockerfile must declare `ARG CONTAINER_APP_VERSION=` +3. Any `default.nix` must produce a version via `dagger call nix-version` +4. At least one build file must exist (`container.py`, Dockerfile, or `default.nix`) +5. A matching `service-versions.yaml` entry must exist with non-null `current-version` +6. All resolved versions from (1), (2), (3), and (5) must agree (v-prefix stripped for comparison) Scoping: by default only checks containers changed vs main. `--all-files` checks everything. If `service-versions.yaml` itself changed, all containers are checked. diff --git a/docs/how-to/zot/add-dagger-nix-build.md b/docs/how-to/zot/add-dagger-nix-build.md index fa5f261..c84661a 100644 --- a/docs/how-to/zot/add-dagger-nix-build.md +++ b/docs/how-to/zot/add-dagger-nix-build.md @@ -1,6 +1,6 @@ --- title: Add Dagger Nix Build Function -modified: 2026-02-20 +modified: 2026-04-11 tags: - how-to - containers @@ -23,7 +23,7 @@ Currently, nix containers can only be built on ringtail (the `nix-container-buil ### 1. Add `build_nix` Dagger function -A new function in `.dagger/src/blumeops_ci/main.py` that builds a nix container inside a `nixos/nix` container: +A new function in `src/blumeops/main.py` (formerly `.dagger/src/blumeops_ci/main.py`) that builds a nix container inside a `nixos/nix` container: ```python @function @@ -80,7 +80,7 @@ The `flake_lock` function already demonstrates running nix inside Dagger using ` | File | Change | |------|--------| -| `.dagger/src/blumeops_ci/main.py` | Add `build_nix`, `nix_version`, optionally `publish_nix` | +| `src/blumeops/main.py` | Add `build_nix`, `nix_version`, optionally `publish_nix` | ## Verification diff --git a/docs/how-to/zot/adopt-commit-based-container-tags.md b/docs/how-to/zot/adopt-commit-based-container-tags.md index 82c90fc..80a3f37 100644 --- a/docs/how-to/zot/adopt-commit-based-container-tags.md +++ b/docs/how-to/zot/adopt-commit-based-container-tags.md @@ -1,6 +1,6 @@ --- title: Adopt Commit-Based Container Tags -modified: 2026-02-20 +modified: 2026-04-11 tags: - how-to - containers @@ -64,7 +64,7 @@ Where: |------|--------| | `.forgejo/workflows/build-container.yaml` | Replace tag trigger with path + dispatch triggers; compute version and SHA | | `.forgejo/workflows/build-container-nix.yaml` | Same trigger changes; add `-nix` suffix to new tag format | -| `.dagger/src/blumeops_ci/main.py` | Accept SHA parameter; publish with new tag format | +| `src/blumeops/main.py` | Accept SHA parameter; publish with new tag format | | `mise-tasks/container-build-and-release` | New task replacing `container-tag-and-release`; triggers workflow dispatch | | `mise-tasks/container-list` | Updated tag display for new format | | `docs/how-to/deployment/build-container-image.md` | Updated documentation | diff --git a/docs/how-to/zot/harden-zot-registry.md b/docs/how-to/zot/harden-zot-registry.md index 47ca322..d74a5d0 100644 --- a/docs/how-to/zot/harden-zot-registry.md +++ b/docs/how-to/zot/harden-zot-registry.md @@ -1,6 +1,6 @@ --- title: Harden Zot Registry -modified: 2026-02-21 +modified: 2026-04-11 tags: - how-to - zot @@ -32,7 +32,7 @@ Updated `ansible/roles/zot/templates/config.json.j2` with: | `ansible/roles/zot/templates/config.json.j2` | Zot config with auth + access control | | `ansible/roles/zot/defaults/main.yml` | OIDC issuer and external URL variables | | `ansible/roles/zot/templates/oidc-credentials.json.j2` | OIDC client credentials | -| `.dagger/src/blumeops_ci/main.py` | `publish()` with registry auth | +| `src/blumeops/main.py` | `publish()` with registry auth | | `.forgejo/workflows/build-container.yaml` | Dagger push with API key | | `.forgejo/workflows/build-container-nix.yaml` | Skopeo push with API key | diff --git a/docs/how-to/zot/pin-container-versions.md b/docs/how-to/zot/pin-container-versions.md index 4d0a64c..b592728 100644 --- a/docs/how-to/zot/pin-container-versions.md +++ b/docs/how-to/zot/pin-container-versions.md @@ -1,6 +1,6 @@ --- title: Pin Container Versions -modified: 2026-02-20 +modified: 2026-04-11 tags: - how-to - containers @@ -18,13 +18,15 @@ Discovered during analysis of [[adopt-commit-based-container-tags]]: containers ## What Was Done -Every container Dockerfile now declares `ARG CONTAINER_APP_VERSION=X.Y.Z` as its first ARG, providing a uniform parsing target. Containers that use the version in build commands chain it to a semantic ARG: +Every container Dockerfile declares `ARG CONTAINER_APP_VERSION=X.Y.Z` as its first ARG, providing a uniform parsing target. Containers that use the version in build commands chain it to a semantic ARG: ```dockerfile ARG CONTAINER_APP_VERSION=v0.60.3 ARG NAVIDROME_VERSION=${CONTAINER_APP_VERSION} ``` +> **Note:** Containers migrated to native Dagger builds use `VERSION = "X.Y.Z"` in `container.py` instead. See `containers/navidrome/container.py` for the pattern. New containers should use `container.py` rather than Dockerfiles. + Specific changes: - **devpi**: Pinned devpi-server==6.19.1 and devpi-web==5.0.1 - **cv**: `CONTAINER_APP_VERSION=1.0.3` (matches latest Forgejo package release) diff --git a/docs/how-to/zot/wire-ci-registry-auth.md b/docs/how-to/zot/wire-ci-registry-auth.md index cce0655..e2507c9 100644 --- a/docs/how-to/zot/wire-ci-registry-auth.md +++ b/docs/how-to/zot/wire-ci-registry-auth.md @@ -1,6 +1,6 @@ --- title: Wire CI Registry Auth -modified: 2026-02-21 +modified: 2026-04-11 tags: - how-to - zot @@ -36,7 +36,7 @@ Authentication uses a zot API key generated after the service account's first OI | File | Purpose | |------|---------| -| `.dagger/src/blumeops_ci/main.py` | `publish()` accepts optional `registry_password` | +| `src/blumeops/main.py` | `publish()` accepts optional `registry_password` | | `.forgejo/workflows/build-container.yaml` | Passes API key to Dagger | | `.forgejo/workflows/build-container-nix.yaml` | Passes API key to skopeo | | `ansible/playbooks/indri.yml` | Pre_task fetches API key from 1Password | diff --git a/docs/reference/tools/dagger.md b/docs/reference/tools/dagger.md index b07ed78..379c10f 100644 --- a/docs/reference/tools/dagger.md +++ b/docs/reference/tools/dagger.md @@ -1,6 +1,6 @@ --- title: Dagger -modified: 2026-03-06 +modified: 2026-04-11 tags: - reference - ci-cd @@ -15,17 +15,18 @@ Build engine for BlumeOps CI/CD pipelines. Replaces shell-based build scripts wi | Property | Value | |----------|-------| -| **Module** | `blumeops-ci` | +| **Module** | `blumeops` | | **Engine Version** | v0.20.1 | | **SDK** | Python | -| **Source** | `.dagger/src/blumeops_ci/main.py` | -| **Config** | `dagger.json` | +| **Source** | `src/blumeops/main.py` | +| **Config** | `dagger.json` (source: `.`) | ## Functions | Function | Signature | Description | |----------|-----------|-------------| -| `build` | `(src, container_name) → Container` | Build a container from `containers//Dockerfile` | +| `build` | `(src, container_name) → Container` | Build a container — uses native pipeline (`container.py`) if available, falls back to `docker_build()` for Dockerfile containers | +| `container_version` | `(container_name) → str` | Return the `VERSION` from a container's `container.py` (empty string if no `container.py`) | | `publish` | `(src, container_name, version, registry?) → str` | Build and push to registry (default: `registry.ops.eblu.me`) | | `build_nix` | `(src, container_name) → File` | Build a nix container from `containers//default.nix`, return docker-archive tarball | | `nix_version` | `(package) → str` | Extract the version of a nixpkgs package | @@ -33,6 +34,18 @@ Build engine for BlumeOps CI/CD pipelines. Replaces shell-based build scripts wi | `flake_lock` | `(src, flake_path?) → File` | Resolve flake inputs, return updated `flake.lock` | | `flake_update` | `(src, flake_path?) → File` | Update all flake inputs to latest, return `flake.lock` | +## Container Build Types + +Containers can be built in three ways: + +| Build file | How it works | Error visibility | +|------------|-------------|-----------------| +| `container.py` | Native Dagger pipeline (preferred) | Full per-step output | +| `Dockerfile` | `docker_build()` fallback (legacy) | Opaque — errors swallowed | +| `default.nix` | `nix-build` on ringtail runner | Full nix output | + +New containers for indri (k8s runner) should use `container.py`. Ringtail containers should continue using `default.nix`. Existing Dockerfile containers are migrated incrementally during [[review-services|service reviews]]. See `containers/navidrome/container.py` for the reference pattern. + ## CLI Examples ```bash diff --git a/docs/reference/tools/mise-tasks.md b/docs/reference/tools/mise-tasks.md index ae92013..16bc10e 100644 --- a/docs/reference/tools/mise-tasks.md +++ b/docs/reference/tools/mise-tasks.md @@ -1,6 +1,6 @@ --- title: Mise Tasks -modified: 2026-02-24 +modified: 2026-04-11 tags: - reference - tools @@ -47,7 +47,7 @@ Run `mise tasks --sort name` for the live list with descriptions. |------|-------------| | `container-list` | List containers and their recent tags | | `container-build-and-release` | Trigger container build workflows via Forgejo API | -| `container-version-check` | Validate version consistency across Dockerfiles, nix, and manifests | +| `container-version-check` | Validate version consistency across container.py, Dockerfiles, nix, and manifests | | `mirror-create` | Create an upstream mirror in the `mirrors/` Forgejo org | | `mirror-update-pats` | Update GitHub PAT on all mirror repos on indri |