From 0da5d8906c2072545f69d57da97164475f55f038 Mon Sep 17 00:00:00 2001 From: Erich Blume Date: Fri, 20 Feb 2026 19:08:15 -0800 Subject: [PATCH] Add version-infrastructure prereqs to harden-zot-registry Mikado chain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Analysis of adopt-commit-based-container-tags revealed three new prerequisites: - pin-container-versions: add version ARGs to devpi, cv, quartz Dockerfiles - add-dagger-nix-build: Dagger functions for nix container builds and version extraction - add-container-version-sync-check: pre-commit hook enforcing version consistency across Dockerfile ARGs, service-versions.yaml, and nix derivations Eliminated the need for separate VERSION files — existing sources (Dockerfile ARGs, nix derivations, service-versions.yaml) are the source of truth, with a sync check enforcing consistency. Co-Authored-By: Claude Opus 4.6 --- .../changelog.d/harden-zot-mikado-cards.ai.md | 2 +- docs/how-to/how-to.md | 3 + .../zot/add-container-version-sync-check.md | 84 ++++++++++++++++ docs/how-to/zot/add-dagger-nix-build.md | 97 +++++++++++++++++++ .../zot/adopt-commit-based-container-tags.md | 9 +- docs/how-to/zot/pin-container-versions.md | 85 ++++++++++++++++ 6 files changed, 278 insertions(+), 2 deletions(-) create mode 100644 docs/how-to/zot/add-container-version-sync-check.md create mode 100644 docs/how-to/zot/add-dagger-nix-build.md create mode 100644 docs/how-to/zot/pin-container-versions.md diff --git a/docs/changelog.d/harden-zot-mikado-cards.ai.md b/docs/changelog.d/harden-zot-mikado-cards.ai.md index 13c7541..ea0594d 100644 --- a/docs/changelog.d/harden-zot-mikado-cards.ai.md +++ b/docs/changelog.d/harden-zot-mikado-cards.ai.md @@ -1 +1 @@ -Create C2 Mikado cards for harden-zot-registry: root goal and three prerequisite cards (register-zot-oidc-client, wire-ci-registry-auth, enforce-tag-immutability). +Expand harden-zot-registry Mikado chain: add prereqs for container version sync check, pin container versions, and Dagger nix build function. diff --git a/docs/how-to/how-to.md b/docs/how-to/how-to.md index 36e87ba..c8094fc 100644 --- a/docs/how-to/how-to.md +++ b/docs/how-to/how-to.md @@ -73,6 +73,9 @@ Mikado chain for hardening the zot registry. Track progress with `mise run docs- - [[wire-ci-registry-auth]] - [[enforce-tag-immutability]] - [[adopt-commit-based-container-tags]] +- [[add-container-version-sync-check]] +- [[pin-container-versions]] +- [[add-dagger-nix-build]] ## Authentik diff --git a/docs/how-to/zot/add-container-version-sync-check.md b/docs/how-to/zot/add-container-version-sync-check.md new file mode 100644 index 0000000..ef534ca --- /dev/null +++ b/docs/how-to/zot/add-container-version-sync-check.md @@ -0,0 +1,84 @@ +--- +title: Add Container Version Sync Check +modified: 2026-02-20 +status: active +requires: + - pin-container-versions + - add-dagger-nix-build +tags: + - how-to + - containers + - ci + - zot +--- + +# Add Container Version Sync Check + +Add a pre-commit check that validates version consistency across the three places container versions are declared: Dockerfile ARGs, `service-versions.yaml`, and nix derivations. No VERSION files needed — the existing sources are the source of truth, and the check enforces they agree. + +## Context + +Discovered during analysis of [[adopt-commit-based-container-tags]]: the new commit-SHA-based image tags need a reliable version source (`vX.Y.Z-`). Versions are currently scattered across Dockerfile ARGs (varying naming conventions), `service-versions.yaml` entries (many still `null`), and nix derivations (implicit from nixpkgs). A sync check ensures these stay consistent without adding a redundant fourth source. + +## What to Do + +### 1. Create `mise run container-version-check` task + +A uv-script mise task that validates version consistency across sources: + +1. **Dockerfile ARG** — parse `ARG _VERSION=` (strip `v` prefix if present). This is the primary version declaration for Dockerfile containers. +2. **`service-versions.yaml`** — `current-version` field for the matching service name (strip `v` prefix). Must agree with the Dockerfile ARG. +3. **Nix derivation** — for nix-only containers (authentik), extract the version via `dagger call nix-version` (from [[add-dagger-nix-build]]). For dual containers (nettest, ntfy), the Dockerfile ARG is the primary check target; the nix version is informational. + +The check also validates: +- Every `hybrid` service in `service-versions.yaml` has a non-null `current-version` +- Every container with a Dockerfile has a parseable `*_VERSION` ARG (after [[pin-container-versions]] is complete) + +Report mismatches as errors. Exit non-zero if any are found. + +### 2. Add pre-commit hook + +Add a `container-version-check` entry to `.pre-commit-config.yaml` following the existing `docs-check-*` pattern: + +```yaml +- id: container-version-check + name: container-version-check + entry: mise run container-version-check + language: system + files: ^(containers/|service-versions\.yaml) + pass_filenames: false +``` + +### 3. Populate `service-versions.yaml` + +Fill in `current-version` for all hybrid services that currently have `null`. The sync check needs these values to validate against. + +## Version extraction in CI + +The CI workflow (from [[adopt-commit-based-container-tags]]) extracts the version at build time — no VERSION file needed: + +- **Dockerfile builds**: `grep -oP 'ARG \w+_VERSION=\K\S+' containers/$CONTAINER/Dockerfile | head -1` +- **Nix builds**: `dagger call nix-version --src=. --package=` or `nix eval --raw nixpkgs#.version` + +## Key Files + +| File | Change | +|------|--------| +| `mise-tasks/container-version-check` | New: sync validation script | +| `.pre-commit-config.yaml` | Add `container-version-check` hook | +| `service-versions.yaml` | Fill in `current-version` for hybrid services | + +## Verification + +- [ ] `mise run container-version-check` passes with no errors +- [ ] Intentionally changing a Dockerfile ARG without updating `service-versions.yaml` fails the check +- [ ] Pre-commit hook catches mismatches on commit +- [ ] `service-versions.yaml` has `current-version` populated for all hybrid services +- [ ] Nix-only container versions are checked via Dagger (or gracefully skipped with a warning if Dagger unavailable) + +## Related + +- [[pin-container-versions]] — Prereq: containers need parseable version ARGs first +- [[add-dagger-nix-build]] — Prereq: nix version extraction +- [[adopt-commit-based-container-tags]] — Parent: CI uses the same version extraction at build time +- [[harden-zot-registry]] — Root goal diff --git a/docs/how-to/zot/add-dagger-nix-build.md b/docs/how-to/zot/add-dagger-nix-build.md new file mode 100644 index 0000000..2e1e00a --- /dev/null +++ b/docs/how-to/zot/add-dagger-nix-build.md @@ -0,0 +1,97 @@ +--- +title: Add Dagger Nix Build Function +modified: 2026-02-20 +status: active +tags: + - how-to + - containers + - ci + - dagger + - zot +--- + +# Add Dagger Nix Build Function + +Add Dagger functions for building nix container images and extracting version info from nix derivations. This enables local nix container evaluation and provides the version extraction mechanism needed by [[add-container-version-sync-check]]. + +## Context + +Discovered during analysis of [[adopt-commit-based-container-tags]]: nix containers (authentik, ntfy, nettest) derive their bundled app version from the nixpkgs pin, not from an explicit declaration. To validate that a VERSION file matches the actual nix-built version, we need a way to query the version from nix. + +Currently, nix containers can only be built on ringtail (the `nix-container-builder` runner). There is no local build path for developers — the only option is to push and wait for CI. Adding a Dagger-based nix build gives both local evaluation and version extraction. + +## What to Do + +### 1. Add `build_nix` Dagger function + +A new function in `.dagger/src/blumeops_ci/main.py` that builds a nix container inside a `nixos/nix` container: + +```python +@function +async def build_nix( + self, src: dagger.Directory, container_name: str +) -> dagger.File: + """Build a nix container from containers//default.nix. Returns the image tarball.""" + # Uses NIX_IMAGE (nixos/nix:2.33.3) — already defined in the module + # Runs nix-build inside the container + # Returns the docker-archive tarball +``` + +This mirrors the existing `build` function (Dockerfile) but for nix. The result is a docker-archive tarball that can be loaded with `docker load` or pushed with `skopeo`. + +### 2. Add `nix_version` Dagger function + +A function that extracts the version of a specific nix package from the nixpkgs pin: + +```python +@function +async def nix_version( + self, src: dagger.Directory, package: str +) -> str: + """Extract the version of a nixpkgs package. Returns version string.""" + # nix eval --raw nixpkgs#.version +``` + +This lets the version sync check run `dagger call nix-version --src=. --package=authentik` to get the actual version that would be built. + +### 3. Add `publish_nix` Dagger function (optional) + +If useful, a combined build-and-push that mirrors `publish` but for nix images: + +```python +@function +async def publish_nix( + self, src: dagger.Directory, container_name: str, version: str, + registry: str = "registry.ops.eblu.me", +) -> str: + """Build nix container and push to registry via skopeo.""" +``` + +This would give a `dagger call publish-nix` path parallel to the existing `dagger call publish`. + +## Nix in Dagger + +The `flake_lock` function already demonstrates running nix inside Dagger using `nixos/nix:2.33.3`. The nix build function follows the same pattern but needs: + +- `NIX_PATH` set to resolved nixpkgs (same as the CI workflow does) +- `--extra-experimental-features "nix-command flakes"` for `nix eval` +- The full repo source mounted (nix files may reference other files like `test-connectivity.sh`) + +## Key Files + +| File | Change | +|------|--------| +| `.dagger/src/blumeops_ci/main.py` | Add `build_nix`, `nix_version`, optionally `publish_nix` | + +## Verification + +- [ ] `dagger call build-nix --src=. --container-name=nettest` produces a valid docker-archive tarball +- [ ] `dagger call nix-version --src=. --package=ntfy-sh` returns the correct version string +- [ ] `dagger call nix-version --src=. --package=authentik` returns the Authentik version +- [ ] Tarball from `build-nix` can be loaded with `docker load` and run locally + +## Related + +- [[add-container-version-sync-check]] — Parent: needs nix version extraction for sync check +- [[adopt-commit-based-container-tags]] — Grandparent goal +- [[dagger]] — Dagger reference diff --git a/docs/how-to/zot/adopt-commit-based-container-tags.md b/docs/how-to/zot/adopt-commit-based-container-tags.md index aa42018..28afed1 100644 --- a/docs/how-to/zot/adopt-commit-based-container-tags.md +++ b/docs/how-to/zot/adopt-commit-based-container-tags.md @@ -2,6 +2,8 @@ title: Adopt Commit-Based Container Tags modified: 2026-02-20 status: active +requires: + - add-container-version-sync-check tags: - how-to - containers @@ -35,7 +37,12 @@ Both the Dockerfile and Nix workflows fire for each trigger, each bailing out if ### Version Source -Each container declares the version of its primary bundled app. The mechanism for declaring this (e.g., a `VERSION` file, parsing a Dockerfile `ARG`, or a convention per container) should be determined during implementation. +Each container's version is extracted at build time from existing declarations — no separate VERSION file: + +- **Dockerfile builds**: parsed from `ARG _VERSION=` in the Dockerfile +- **Nix builds**: extracted via `dagger call nix-version` or `nix eval` + +The [[add-container-version-sync-check]] pre-commit check ensures these declarations stay in sync with `service-versions.yaml`. See [[pin-container-versions]] for the work to ensure every container has a parseable version. ### Image Tag Format diff --git a/docs/how-to/zot/pin-container-versions.md b/docs/how-to/zot/pin-container-versions.md new file mode 100644 index 0000000..30bce73 --- /dev/null +++ b/docs/how-to/zot/pin-container-versions.md @@ -0,0 +1,85 @@ +--- +title: Pin Container Versions +modified: 2026-02-20 +status: active +tags: + - how-to + - containers + - ci + - zot +--- + +# Pin Container Versions + +Ensure every container has an explicit, parseable version declaration so that [[add-container-version-sync-check]] has something to validate against. + +## Context + +Discovered during analysis of [[adopt-commit-based-container-tags]]: most containers already have version ARGs (miniflux, navidrome, ntfy, etc.), but several do not. Without explicit versions in the build files, there is nothing for a VERSION file to sync against. + +## Containers Needing Work + +### devpi — Pin pip dependencies + +Currently installs `devpi-server` and `devpi-web` without version pins: + +```dockerfile +RUN pip install --no-cache-dir devpi-server devpi-web +``` + +Add version ARGs and pin: + +```dockerfile +ARG DEVPI_SERVER_VERSION=6.12.1 +ARG DEVPI_WEB_VERSION=4.2.2 +RUN pip install --no-cache-dir \ + devpi-server==${DEVPI_SERVER_VERSION} \ + devpi-web==${DEVPI_WEB_VERSION} +``` + +The VERSION file will track `devpi-server` as the primary version. + +### cv — Add internal version ARG + +Thin nginx wrapper that downloads content at runtime. No upstream app to track — the version reflects the container definition itself: + +```dockerfile +ARG CV_VERSION=0.1.0 +``` + +Bump when the Dockerfile or scripts change. + +### quartz — Add internal version ARG + +Same pattern as cv: + +```dockerfile +ARG QUARTZ_VERSION=0.1.0 +``` + +### nettest — Already handled + +Utility container with no upstream. Will use `0.1.0` in VERSION file. No Dockerfile ARG needed since there's nothing to pin — the Dockerfile just installs Alpine packages at whatever version Alpine ships. The sync check can skip the Dockerfile ARG validation for containers without a `*_VERSION` ARG. + +### forgejo-runner — Already handled + +Has `ARG DAGGER_VERSION=0.19.11` as its primary version. Good enough. + +## Key Files + +| File | Change | +|------|--------| +| `containers/devpi/Dockerfile` | Pin devpi-server and devpi-web versions | +| `containers/cv/Dockerfile` | Add `ARG CV_VERSION` | +| `containers/quartz/Dockerfile` | Add `ARG QUARTZ_VERSION` | + +## Verification + +- [ ] Every container Dockerfile either has a `*_VERSION` ARG or is documented as version-exempt (nettest) +- [ ] `devpi` container builds with pinned versions +- [ ] `cv` and `quartz` containers still build and serve correctly + +## Related + +- [[add-container-version-sync-check]] — Parent: needs parseable versions for sync check +- [[adopt-commit-based-container-tags]] — Grandparent goal