Native Dagger container builds + Navidrome v0.61.1 #330

Merged
eblume merged 5 commits from native-dagger-containers into main 2026-04-11 17:11:57 -07:00
11 changed files with 76 additions and 54 deletions
Showing only changes of commit 215ebef73e - Show all commits

Update docs for Dagger module restructure

- Update dagger.md reference card: new module name, source path,
  container build types table, container-version function
- Update build-container-image.md: container.py as preferred for indri,
  default.nix for ringtail, navidrome as reference pattern
- Update upgrade-dagger.md: uv.lock path (no longer under .dagger/)
- Update .dagger/src/blumeops_ci path references across 5 how-to docs
- Update container-version-check and pin-container-versions docs
- Update mise-tasks reference
- Set modified: 2026-04-11 on all changed docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Erich Blume 2026-04-11 17:09:51 -07:00

View file

@ -1,6 +1,6 @@
--- ---
title: Upgrade Dagger title: Upgrade Dagger
modified: 2026-03-06 modified: 2026-04-11
last-reviewed: 2026-03-06 last-reviewed: 2026-03-06
tags: tags:
- how-to - how-to
@ -26,7 +26,7 @@ Dagger versions are pinned in multiple places. The runner job image (which execu
| `service-versions.yaml` | `runner-job-image` version and `last-reviewed` | 1 | | `service-versions.yaml` | `runner-job-image` version and `last-reviewed` | 1 |
| `mise.toml` | `dagger` tool version | 2 | | `mise.toml` | `dagger` tool version | 2 |
| `dagger.json` | `engineVersion` | 2 | | `dagger.json` | `engineVersion` | 2 |
| `.dagger/uv.lock` | SDK dependency lock (regenerated automatically) | 2 | | `uv.lock` | SDK dependency lock (regenerated automatically) | 2 |
| `docs/reference/tools/dagger.md` | Version references in documentation | 2 | | `docs/reference/tools/dagger.md` | Version references in documentation | 2 |
| `argocd/manifests/forgejo-runner/deployment.yaml` | `RUNNER_LABELS` image tag | 2 | | `argocd/manifests/forgejo-runner/deployment.yaml` | `RUNNER_LABELS` image tag | 2 |
@ -63,7 +63,7 @@ Once the Phase 1 build completes, upgrade the module engine version and deploy t
"engineVersion": "v<new-version>" "engineVersion": "v<new-version>"
``` ```
4. Regenerate the SDK lock file — run any `dagger call` command (e.g., `dagger call --help` or `dagger functions`). This updates `.dagger/uv.lock` if SDK dependencies changed. 4. Regenerate the SDK lock file — run any `dagger call` command (e.g., `dagger call --help` or `dagger functions`). This updates `uv.lock` if SDK dependencies changed.
5. Update `docs/reference/tools/dagger.md` — bump the version in the Quick Reference table and any version references in the body text. 5. Update `docs/reference/tools/dagger.md` — bump the version in the Quick Reference table and any version references in the body text.

View file

@ -1,6 +1,6 @@
--- ---
title: Build Container Image title: Build Container Image
modified: 2026-02-24 modified: 2026-04-11
last-reviewed: 2026-02-15 last-reviewed: 2026-02-15
tags: tags:
- how-to - how-to
@ -14,8 +14,8 @@ How to create a custom container image in BlumeOps, build it locally, and releas
## Prerequisites ## Prerequisites
- [Dagger CLI](https://docs.dagger.io/install) installed locally (for Dockerfile builds) - [Dagger CLI](https://docs.dagger.io/install) installed locally
- A `Dockerfile` and/or `default.nix` for the service - A `container.py`, `Dockerfile`, and/or `default.nix` for the service
## 1. Create the container directory ## 1. Create the container directory
@ -23,16 +23,21 @@ Add build files under `containers/<name>/`:
``` ```
containers/<name>/ containers/<name>/
├── Dockerfile (built by Dagger on the k8s runner) ├── container.py (native Dagger pipeline — preferred for new containers)
├── Dockerfile (legacy — built via docker_build() fallback)
├── default.nix (built by nix-build on the ringtail runner) ├── default.nix (built by nix-build on the ringtail runner)
└── (optional scripts, configs) └── (optional scripts, configs)
``` ```
A container can have one or both build files. The directory name becomes the image name: `registry.ops.eblu.me/blumeops/<name>`. A container can have one or more build files. The directory name becomes the image name: `registry.ops.eblu.me/blumeops/<name>`.
**New containers for indri (k8s runner) should use `container.py`** — native Dagger pipelines surface full build errors per step, while `docker_build()` (used for Dockerfiles) swallows errors. See `containers/navidrome/container.py` for the reference pattern. Existing Dockerfile containers are migrated incrementally during [[review-services|service reviews]].
**Ringtail containers should continue using `default.nix`** — these are built by `nix-build` on the ringtail runner and don't benefit from the Dagger migration.
## 2. Build locally ## 2. Build locally
**Dockerfile** — test with Dagger: **Any container** (native `container.py` or legacy Dockerfile) — test with Dagger:
```bash ```bash
dagger call build --src=. --container-name=<name> dagger call build --src=. --container-name=<name>
@ -65,10 +70,11 @@ Use `--dry-run` to preview without dispatching.
| Build file | Workflow | Runner | Registry tag | | Build file | Workflow | Runner | Registry tag |
|------------|----------|--------|--------------| |------------|----------|--------|--------------|
| `container.py` | `build-container.yaml` | `k8s` (indri) | `:vX.Y.Z-<sha>` |
| `Dockerfile` | `build-container.yaml` | `k8s` (indri) | `:vX.Y.Z-<sha>` | | `Dockerfile` | `build-container.yaml` | `k8s` (indri) | `:vX.Y.Z-<sha>` |
| `default.nix` | `build-container-nix.yaml` | `nix-container-builder` ([[ringtail]]) | `:vX.Y.Z-<sha>-nix` | | `default.nix` | `build-container.yaml` | `nix-container-builder` ([[ringtail]]) | `:vX.Y.Z-<sha>-nix` |
The version (`X.Y.Z`) is extracted from `ARG CONTAINER_APP_VERSION=` in the Dockerfile or `version = "..."` in `default.nix`. The SHA is the short (7-char) commit hash. The version (`X.Y.Z`) is extracted from `VERSION` in `container.py` (via `dagger call container-version`), `ARG CONTAINER_APP_VERSION=` in Dockerfiles, or `version = "..."` in `default.nix`. The SHA is the short (7-char) commit hash.
Check available images and tags with: Check available images and tags with:
@ -112,36 +118,36 @@ Existing containers demonstrate several build approaches:
| Pattern | Example | Notes | | Pattern | Example | Notes |
|---------|---------|-------| |---------|---------|-------|
| Alpine package install | [[#transmission]] | Simplest — install from apk | | Native Dagger (Go + Node) | [[#navidrome]] | `container.py` with helper functions — preferred for new containers |
| Go from source | [[#miniflux]] | Clone upstream, `go build` | | Alpine package install | [[#transmission]] | Simplest Dockerfile — install from apk |
| Multi-stage with Node + Go | [[#navidrome]] | Separate UI and backend build stages | | Go from source | [[#miniflux]] | Dockerfile: clone upstream, `go build` |
| Multi-stage Elixir | [[#teslamate]] | Elixir release with Node assets | | Multi-stage Elixir | [[#teslamate]] | Dockerfile: Elixir release with Node assets |
| Runtime tarball download | [[#kiwix-serve]] | Download pre-built binary with arch detection | | Runtime tarball download | [[#kiwix-serve]] | Dockerfile: download pre-built binary with arch detection |
| Nix `dockerTools` | [[#ntfy-nix]] | `buildLayeredImage` with nix-built app | | Nix `dockerTools` | [[#ntfy-nix]] | `buildLayeredImage` with nix-built app (ringtail runner) |
### transmission
`containers/transmission/Dockerfile` — Installs transmission-daemon directly from Alpine packages. Good starting point for services available in apk.
### miniflux
`containers/miniflux/Dockerfile` — Two-stage Go build. Clones upstream at a pinned version tag, runs `make`, copies the binary into a minimal Alpine runtime.
### navidrome ### navidrome
`containers/navidrome/Dockerfile` — Three-stage build with separate Node.js UI compilation, Go backend build with CGO (taglib), and a minimal Alpine runtime with ffmpeg. `containers/navidrome/container.py` — Native Dagger build. Three-stage pipeline using helper functions: `node_build()` for UI, `go_build()` with CGO/taglib/FTS5 for backend, `alpine_runtime()` with ffmpeg. This is the reference pattern for migrating Dockerfile containers to native Dagger builds.
### transmission
`containers/transmission/Dockerfile` — Installs transmission-daemon directly from Alpine packages. Good starting point for services available in apk. (Legacy Dockerfile — migrate to `container.py` during review.)
### miniflux
`containers/miniflux/Dockerfile` — Two-stage Go build. Clones upstream at a pinned version tag, runs `make`, copies the binary into a minimal Alpine runtime. (Legacy Dockerfile — migrate to `container.py` during review.)
### teslamate ### teslamate
`containers/teslamate/Dockerfile` — Two-stage Elixir build with Node.js asset compilation. Uses Debian-based images due to Elixir/OTP dependencies. `containers/teslamate/Dockerfile` — Two-stage Elixir build with Node.js asset compilation. Uses Debian-based images due to Elixir/OTP dependencies. (Legacy Dockerfile — migrate to `container.py` during review.)
### kiwix-serve ### kiwix-serve
`containers/kiwix-serve/Dockerfile` — Downloads a pre-built binary from upstream, with architecture detection for cross-platform support. `containers/kiwix-serve/Dockerfile` — Downloads a pre-built binary from upstream, with architecture detection for cross-platform support. (Legacy Dockerfile — migrate to `container.py` during review.)
### ntfy (nix) ### ntfy (nix)
`containers/ntfy/default.nix` — Builds ntfy from source using `buildGoModule` and packages it with `dockerTools.buildLayeredImage`. Runs alongside the existing Dockerfile; the nix variant is tagged `:version-nix` in the registry. `containers/ntfy/default.nix` — Builds ntfy from source using `buildGoModule` and packages it with `dockerTools.buildLayeredImage`. Runs alongside the existing Dockerfile; the nix variant is tagged `:version-nix` in the registry. Nix containers should continue using `default.nix`.
## Related ## Related

View file

@ -1,6 +1,6 @@
--- ---
title: Validate Workflows Against v12 title: Validate Workflows Against v12
modified: 2026-02-27 modified: 2026-04-11
last-reviewed: 2026-02-27 last-reviewed: 2026-02-27
tags: tags:
- how-to - how-to
@ -25,7 +25,7 @@ All 6 workflows pass v12.7.0 schema validation with no changes needed:
## Deliverables ## Deliverables
1. `validate_workflows` function added to `.dagger/src/blumeops_ci/main.py` 1. `validate_workflows` function added to `src/blumeops/main.py` (formerly `.dagger/src/blumeops_ci/main.py`)
- Uses `forgejo-runner validate --directory .` inside the upstream runner container - Uses `forgejo-runner validate --directory .` inside the upstream runner container
- `runner_version` parameter (default `12.7.0`) pins to deployed version - `runner_version` parameter (default `12.7.0`) pins to deployed version
2. `mise run validate-workflows` task wired to `dagger call validate-workflows` 2. `mise run validate-workflows` task wired to `dagger call validate-workflows`

View file

@ -1,6 +1,6 @@
--- ---
title: Add Container Version Sync Check title: Add Container Version Sync Check
modified: 2026-02-20 modified: 2026-04-11
tags: tags:
- how-to - how-to
- containers - containers
@ -10,7 +10,7 @@ tags:
# Add Container Version Sync Check # Add Container Version Sync Check
Add a prek check that validates version consistency across the three places container versions are declared: Dockerfile ARGs, `service-versions.yaml`, and nix derivations. No VERSION files needed — the existing sources are the source of truth, and the check enforces they agree. Add a prek check that validates version consistency across the places container versions are declared: `container.py` VERSION constants, Dockerfile ARGs, `service-versions.yaml`, and nix derivations. The check enforces they agree.
## Context ## Context
@ -20,13 +20,14 @@ Discovered during analysis of [[adopt-commit-based-container-tags]]: the new com
### 1. Created `mise run container-version-check` task ### 1. Created `mise run container-version-check` task
A typer-based uv-script that iterates over `containers/*/` and validates five rules per container: A typer-based uv-script that iterates over `containers/*/` and validates six rules per container:
1. Any Dockerfile must declare `ARG CONTAINER_APP_VERSION=<value>` 1. Any `container.py` must declare `VERSION = "<value>"`
2. Any `default.nix` must produce a version via `dagger call nix-version` 2. Any Dockerfile must declare `ARG CONTAINER_APP_VERSION=<value>`
3. At least one build file must exist (Dockerfile or default.nix) 3. Any `default.nix` must produce a version via `dagger call nix-version`
4. A matching `service-versions.yaml` entry must exist with non-null `current-version` 4. At least one build file must exist (`container.py`, Dockerfile, or `default.nix`)
5. All resolved versions from (1), (2), and (4) must agree (v-prefix stripped for comparison) 5. A matching `service-versions.yaml` entry must exist with non-null `current-version`
6. All resolved versions from (1), (2), (3), and (5) must agree (v-prefix stripped for comparison)
Scoping: by default only checks containers changed vs main. `--all-files` checks everything. If `service-versions.yaml` itself changed, all containers are checked. Scoping: by default only checks containers changed vs main. `--all-files` checks everything. If `service-versions.yaml` itself changed, all containers are checked.

View file

@ -1,6 +1,6 @@
--- ---
title: Add Dagger Nix Build Function title: Add Dagger Nix Build Function
modified: 2026-02-20 modified: 2026-04-11
tags: tags:
- how-to - how-to
- containers - containers
@ -23,7 +23,7 @@ Currently, nix containers can only be built on ringtail (the `nix-container-buil
### 1. Add `build_nix` Dagger function ### 1. Add `build_nix` Dagger function
A new function in `.dagger/src/blumeops_ci/main.py` that builds a nix container inside a `nixos/nix` container: A new function in `src/blumeops/main.py` (formerly `.dagger/src/blumeops_ci/main.py`) that builds a nix container inside a `nixos/nix` container:
```python ```python
@function @function
@ -80,7 +80,7 @@ The `flake_lock` function already demonstrates running nix inside Dagger using `
| File | Change | | File | Change |
|------|--------| |------|--------|
| `.dagger/src/blumeops_ci/main.py` | Add `build_nix`, `nix_version`, optionally `publish_nix` | | `src/blumeops/main.py` | Add `build_nix`, `nix_version`, optionally `publish_nix` |
## Verification ## Verification

View file

@ -1,6 +1,6 @@
--- ---
title: Adopt Commit-Based Container Tags title: Adopt Commit-Based Container Tags
modified: 2026-02-20 modified: 2026-04-11
tags: tags:
- how-to - how-to
- containers - containers
@ -64,7 +64,7 @@ Where:
|------|--------| |------|--------|
| `.forgejo/workflows/build-container.yaml` | Replace tag trigger with path + dispatch triggers; compute version and SHA | | `.forgejo/workflows/build-container.yaml` | Replace tag trigger with path + dispatch triggers; compute version and SHA |
| `.forgejo/workflows/build-container-nix.yaml` | Same trigger changes; add `-nix` suffix to new tag format | | `.forgejo/workflows/build-container-nix.yaml` | Same trigger changes; add `-nix` suffix to new tag format |
| `.dagger/src/blumeops_ci/main.py` | Accept SHA parameter; publish with new tag format | | `src/blumeops/main.py` | Accept SHA parameter; publish with new tag format |
| `mise-tasks/container-build-and-release` | New task replacing `container-tag-and-release`; triggers workflow dispatch | | `mise-tasks/container-build-and-release` | New task replacing `container-tag-and-release`; triggers workflow dispatch |
| `mise-tasks/container-list` | Updated tag display for new format | | `mise-tasks/container-list` | Updated tag display for new format |
| `docs/how-to/deployment/build-container-image.md` | Updated documentation | | `docs/how-to/deployment/build-container-image.md` | Updated documentation |

View file

@ -1,6 +1,6 @@
--- ---
title: Harden Zot Registry title: Harden Zot Registry
modified: 2026-02-21 modified: 2026-04-11
tags: tags:
- how-to - how-to
- zot - zot
@ -32,7 +32,7 @@ Updated `ansible/roles/zot/templates/config.json.j2` with:
| `ansible/roles/zot/templates/config.json.j2` | Zot config with auth + access control | | `ansible/roles/zot/templates/config.json.j2` | Zot config with auth + access control |
| `ansible/roles/zot/defaults/main.yml` | OIDC issuer and external URL variables | | `ansible/roles/zot/defaults/main.yml` | OIDC issuer and external URL variables |
| `ansible/roles/zot/templates/oidc-credentials.json.j2` | OIDC client credentials | | `ansible/roles/zot/templates/oidc-credentials.json.j2` | OIDC client credentials |
| `.dagger/src/blumeops_ci/main.py` | `publish()` with registry auth | | `src/blumeops/main.py` | `publish()` with registry auth |
| `.forgejo/workflows/build-container.yaml` | Dagger push with API key | | `.forgejo/workflows/build-container.yaml` | Dagger push with API key |
| `.forgejo/workflows/build-container-nix.yaml` | Skopeo push with API key | | `.forgejo/workflows/build-container-nix.yaml` | Skopeo push with API key |

View file

@ -1,6 +1,6 @@
--- ---
title: Pin Container Versions title: Pin Container Versions
modified: 2026-02-20 modified: 2026-04-11
tags: tags:
- how-to - how-to
- containers - containers
@ -18,13 +18,15 @@ Discovered during analysis of [[adopt-commit-based-container-tags]]: containers
## What Was Done ## What Was Done
Every container Dockerfile now declares `ARG CONTAINER_APP_VERSION=X.Y.Z` as its first ARG, providing a uniform parsing target. Containers that use the version in build commands chain it to a semantic ARG: Every container Dockerfile declares `ARG CONTAINER_APP_VERSION=X.Y.Z` as its first ARG, providing a uniform parsing target. Containers that use the version in build commands chain it to a semantic ARG:
```dockerfile ```dockerfile
ARG CONTAINER_APP_VERSION=v0.60.3 ARG CONTAINER_APP_VERSION=v0.60.3
ARG NAVIDROME_VERSION=${CONTAINER_APP_VERSION} ARG NAVIDROME_VERSION=${CONTAINER_APP_VERSION}
``` ```
> **Note:** Containers migrated to native Dagger builds use `VERSION = "X.Y.Z"` in `container.py` instead. See `containers/navidrome/container.py` for the pattern. New containers should use `container.py` rather than Dockerfiles.
Specific changes: Specific changes:
- **devpi**: Pinned devpi-server==6.19.1 and devpi-web==5.0.1 - **devpi**: Pinned devpi-server==6.19.1 and devpi-web==5.0.1
- **cv**: `CONTAINER_APP_VERSION=1.0.3` (matches latest Forgejo package release) - **cv**: `CONTAINER_APP_VERSION=1.0.3` (matches latest Forgejo package release)

View file

@ -1,6 +1,6 @@
--- ---
title: Wire CI Registry Auth title: Wire CI Registry Auth
modified: 2026-02-21 modified: 2026-04-11
tags: tags:
- how-to - how-to
- zot - zot
@ -36,7 +36,7 @@ Authentication uses a zot API key generated after the service account's first OI
| File | Purpose | | File | Purpose |
|------|---------| |------|---------|
| `.dagger/src/blumeops_ci/main.py` | `publish()` accepts optional `registry_password` | | `src/blumeops/main.py` | `publish()` accepts optional `registry_password` |
| `.forgejo/workflows/build-container.yaml` | Passes API key to Dagger | | `.forgejo/workflows/build-container.yaml` | Passes API key to Dagger |
| `.forgejo/workflows/build-container-nix.yaml` | Passes API key to skopeo | | `.forgejo/workflows/build-container-nix.yaml` | Passes API key to skopeo |
| `ansible/playbooks/indri.yml` | Pre_task fetches API key from 1Password | | `ansible/playbooks/indri.yml` | Pre_task fetches API key from 1Password |

View file

@ -1,6 +1,6 @@
--- ---
title: Dagger title: Dagger
modified: 2026-03-06 modified: 2026-04-11
tags: tags:
- reference - reference
- ci-cd - ci-cd
@ -15,17 +15,18 @@ Build engine for BlumeOps CI/CD pipelines. Replaces shell-based build scripts wi
| Property | Value | | Property | Value |
|----------|-------| |----------|-------|
| **Module** | `blumeops-ci` | | **Module** | `blumeops` |
| **Engine Version** | v0.20.1 | | **Engine Version** | v0.20.1 |
| **SDK** | Python | | **SDK** | Python |
| **Source** | `.dagger/src/blumeops_ci/main.py` | | **Source** | `src/blumeops/main.py` |
| **Config** | `dagger.json` | | **Config** | `dagger.json` (source: `.`) |
## Functions ## Functions
| Function | Signature | Description | | Function | Signature | Description |
|----------|-----------|-------------| |----------|-----------|-------------|
| `build` | `(src, container_name) → Container` | Build a container from `containers/<name>/Dockerfile` | | `build` | `(src, container_name) → Container` | Build a container — uses native pipeline (`container.py`) if available, falls back to `docker_build()` for Dockerfile containers |
| `container_version` | `(container_name) → str` | Return the `VERSION` from a container's `container.py` (empty string if no `container.py`) |
| `publish` | `(src, container_name, version, registry?) → str` | Build and push to registry (default: `registry.ops.eblu.me`) | | `publish` | `(src, container_name, version, registry?) → str` | Build and push to registry (default: `registry.ops.eblu.me`) |
| `build_nix` | `(src, container_name) → File` | Build a nix container from `containers/<name>/default.nix`, return docker-archive tarball | | `build_nix` | `(src, container_name) → File` | Build a nix container from `containers/<name>/default.nix`, return docker-archive tarball |
| `nix_version` | `(package) → str` | Extract the version of a nixpkgs package | | `nix_version` | `(package) → str` | Extract the version of a nixpkgs package |
@ -33,6 +34,18 @@ Build engine for BlumeOps CI/CD pipelines. Replaces shell-based build scripts wi
| `flake_lock` | `(src, flake_path?) → File` | Resolve flake inputs, return updated `flake.lock` | | `flake_lock` | `(src, flake_path?) → File` | Resolve flake inputs, return updated `flake.lock` |
| `flake_update` | `(src, flake_path?) → File` | Update all flake inputs to latest, return `flake.lock` | | `flake_update` | `(src, flake_path?) → File` | Update all flake inputs to latest, return `flake.lock` |
## Container Build Types
Containers can be built in three ways:
| Build file | How it works | Error visibility |
|------------|-------------|-----------------|
| `container.py` | Native Dagger pipeline (preferred) | Full per-step output |
| `Dockerfile` | `docker_build()` fallback (legacy) | Opaque — errors swallowed |
| `default.nix` | `nix-build` on ringtail runner | Full nix output |
New containers for indri (k8s runner) should use `container.py`. Ringtail containers should continue using `default.nix`. Existing Dockerfile containers are migrated incrementally during [[review-services|service reviews]]. See `containers/navidrome/container.py` for the reference pattern.
## CLI Examples ## CLI Examples
```bash ```bash

View file

@ -1,6 +1,6 @@
--- ---
title: Mise Tasks title: Mise Tasks
modified: 2026-02-24 modified: 2026-04-11
tags: tags:
- reference - reference
- tools - tools
@ -47,7 +47,7 @@ Run `mise tasks --sort name` for the live list with descriptions.
|------|-------------| |------|-------------|
| `container-list` | List containers and their recent tags | | `container-list` | List containers and their recent tags |
| `container-build-and-release` | Trigger container build workflows via Forgejo API | | `container-build-and-release` | Trigger container build workflows via Forgejo API |
| `container-version-check` | Validate version consistency across Dockerfiles, nix, and manifests | | `container-version-check` | Validate version consistency across container.py, Dockerfiles, nix, and manifests |
| `mirror-create` | Create an upstream mirror in the `mirrors/` Forgejo org | | `mirror-create` | Create an upstream mirror in the `mirrors/` Forgejo org |
| `mirror-update-pats` | Update GitHub PAT on all mirror repos on indri | | `mirror-update-pats` | Update GitHub PAT on all mirror repos on indri |