Native Dagger container builds + Navidrome v0.61.1 (#330)
## Summary
- Move Dagger module from `.dagger/` to repo root (`src/blumeops/`), rename `blumeops-ci` → `blumeops`
- Replace opaque `docker_build()` with native Dagger pipelines that surface full build errors per step
- Migrate navidrome as the first container (`containers/navidrome/container.py`)
- Upgrade navidrome from v0.60.3 to v0.61.1 (major artwork overhaul, SQLite FTS5 search, server-managed transcoding)
- Add `dagger call container-version` for CI version extraction without Dockerfile parsing
- All mise tasks (`container-list`, `container-version-check`, `container-build-and-release`) updated for hybrid mode
- Legacy `docker_build()` fallback preserved for all other containers
## Motivation
When navidrome v0.61.0 added a new Go build tag (`sqlite_fts5`), `docker_build()` showed only "exit code: 1". We had to run `docker build --progress=plain` manually to find `undefined: buildtags.SQLITE_FTS5`. Native Dagger pipelines show the full error inline.
## Container build dispatch needed
After merge, dispatch container build for navidrome:
```
mise run container-build-and-release navidrome --ref 470b4bd
```
## Deploy steps
1. Wait for container build to complete
2. Back up navidrome-data PVC (non-reversible DB migrations)
3. `argocd app set navidrome --revision main && argocd app sync navidrome`
4. Verify at https://dj.ops.eblu.me
## Future
Remaining containers migrate incrementally in follow-up PRs using the same pattern.
Reviewed-on: #330
This commit is contained in:
parent
4fc0192731
commit
c86b5d7772
33 changed files with 422 additions and 929 deletions
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Upgrade Dagger
|
||||
modified: 2026-03-06
|
||||
modified: 2026-04-11
|
||||
last-reviewed: 2026-03-06
|
||||
tags:
|
||||
- how-to
|
||||
|
|
@ -26,7 +26,7 @@ Dagger versions are pinned in multiple places. The runner job image (which execu
|
|||
| `service-versions.yaml` | `runner-job-image` version and `last-reviewed` | 1 |
|
||||
| `mise.toml` | `dagger` tool version | 2 |
|
||||
| `dagger.json` | `engineVersion` | 2 |
|
||||
| `.dagger/uv.lock` | SDK dependency lock (regenerated automatically) | 2 |
|
||||
| `uv.lock` | SDK dependency lock (regenerated automatically) | 2 |
|
||||
| `docs/reference/tools/dagger.md` | Version references in documentation | 2 |
|
||||
| `argocd/manifests/forgejo-runner/deployment.yaml` | `RUNNER_LABELS` image tag | 2 |
|
||||
|
||||
|
|
@ -63,7 +63,7 @@ Once the Phase 1 build completes, upgrade the module engine version and deploy t
|
|||
"engineVersion": "v<new-version>"
|
||||
```
|
||||
|
||||
4. Regenerate the SDK lock file — run any `dagger call` command (e.g., `dagger call --help` or `dagger functions`). This updates `.dagger/uv.lock` if SDK dependencies changed.
|
||||
4. Regenerate the SDK lock file — run any `dagger call` command (e.g., `dagger call --help` or `dagger functions`). This updates `uv.lock` if SDK dependencies changed.
|
||||
|
||||
5. Update `docs/reference/tools/dagger.md` — bump the version in the Quick Reference table and any version references in the body text.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Build Container Image
|
||||
modified: 2026-02-24
|
||||
modified: 2026-04-11
|
||||
last-reviewed: 2026-02-15
|
||||
tags:
|
||||
- how-to
|
||||
|
|
@ -14,8 +14,8 @@ How to create a custom container image in BlumeOps, build it locally, and releas
|
|||
|
||||
## Prerequisites
|
||||
|
||||
- [Dagger CLI](https://docs.dagger.io/install) installed locally (for Dockerfile builds)
|
||||
- A `Dockerfile` and/or `default.nix` for the service
|
||||
- [Dagger CLI](https://docs.dagger.io/install) installed locally
|
||||
- A `container.py`, `Dockerfile`, and/or `default.nix` for the service
|
||||
|
||||
## 1. Create the container directory
|
||||
|
||||
|
|
@ -23,16 +23,21 @@ Add build files under `containers/<name>/`:
|
|||
|
||||
```
|
||||
containers/<name>/
|
||||
├── Dockerfile (built by Dagger on the k8s runner)
|
||||
├── container.py (native Dagger pipeline — preferred for new containers)
|
||||
├── Dockerfile (legacy — built via docker_build() fallback)
|
||||
├── default.nix (built by nix-build on the ringtail runner)
|
||||
└── (optional scripts, configs)
|
||||
```
|
||||
|
||||
A container can have one or both build files. The directory name becomes the image name: `registry.ops.eblu.me/blumeops/<name>`.
|
||||
A container can have one or more build files. The directory name becomes the image name: `registry.ops.eblu.me/blumeops/<name>`.
|
||||
|
||||
**New containers for indri (k8s runner) should use `container.py`** — native Dagger pipelines surface full build errors per step, while `docker_build()` (used for Dockerfiles) swallows errors. See `containers/navidrome/container.py` for the reference pattern. Existing Dockerfile containers are migrated incrementally during [[review-services|service reviews]].
|
||||
|
||||
**Ringtail containers should continue using `default.nix`** — these are built by `nix-build` on the ringtail runner and don't benefit from the Dagger migration.
|
||||
|
||||
## 2. Build locally
|
||||
|
||||
**Dockerfile** — test with Dagger:
|
||||
**Any container** (native `container.py` or legacy Dockerfile) — test with Dagger:
|
||||
|
||||
```bash
|
||||
dagger call build --src=. --container-name=<name>
|
||||
|
|
@ -65,10 +70,11 @@ Use `--dry-run` to preview without dispatching.
|
|||
|
||||
| Build file | Workflow | Runner | Registry tag |
|
||||
|------------|----------|--------|--------------|
|
||||
| `container.py` | `build-container.yaml` | `k8s` (indri) | `:vX.Y.Z-<sha>` |
|
||||
| `Dockerfile` | `build-container.yaml` | `k8s` (indri) | `:vX.Y.Z-<sha>` |
|
||||
| `default.nix` | `build-container-nix.yaml` | `nix-container-builder` ([[ringtail]]) | `:vX.Y.Z-<sha>-nix` |
|
||||
| `default.nix` | `build-container.yaml` | `nix-container-builder` ([[ringtail]]) | `:vX.Y.Z-<sha>-nix` |
|
||||
|
||||
The version (`X.Y.Z`) is extracted from `ARG CONTAINER_APP_VERSION=` in the Dockerfile or `version = "..."` in `default.nix`. The SHA is the short (7-char) commit hash.
|
||||
The version (`X.Y.Z`) is extracted from `VERSION` in `container.py` (via `dagger call container-version`), `ARG CONTAINER_APP_VERSION=` in Dockerfiles, or `version = "..."` in `default.nix`. The SHA is the short (7-char) commit hash.
|
||||
|
||||
Check available images and tags with:
|
||||
|
||||
|
|
@ -112,36 +118,36 @@ Existing containers demonstrate several build approaches:
|
|||
|
||||
| Pattern | Example | Notes |
|
||||
|---------|---------|-------|
|
||||
| Alpine package install | [[#transmission]] | Simplest — install from apk |
|
||||
| Go from source | [[#miniflux]] | Clone upstream, `go build` |
|
||||
| Multi-stage with Node + Go | [[#navidrome]] | Separate UI and backend build stages |
|
||||
| Multi-stage Elixir | [[#teslamate]] | Elixir release with Node assets |
|
||||
| Runtime tarball download | [[#kiwix-serve]] | Download pre-built binary with arch detection |
|
||||
| Nix `dockerTools` | [[#ntfy-nix]] | `buildLayeredImage` with nix-built app |
|
||||
|
||||
### transmission
|
||||
|
||||
`containers/transmission/Dockerfile` — Installs transmission-daemon directly from Alpine packages. Good starting point for services available in apk.
|
||||
|
||||
### miniflux
|
||||
|
||||
`containers/miniflux/Dockerfile` — Two-stage Go build. Clones upstream at a pinned version tag, runs `make`, copies the binary into a minimal Alpine runtime.
|
||||
| Native Dagger (Go + Node) | [[#navidrome]] | `container.py` with helper functions — preferred for new containers |
|
||||
| Alpine package install | [[#transmission]] | Simplest Dockerfile — install from apk |
|
||||
| Go from source | [[#miniflux]] | Dockerfile: clone upstream, `go build` |
|
||||
| Multi-stage Elixir | [[#teslamate]] | Dockerfile: Elixir release with Node assets |
|
||||
| Runtime tarball download | [[#kiwix-serve]] | Dockerfile: download pre-built binary with arch detection |
|
||||
| Nix `dockerTools` | [[#ntfy-nix]] | `buildLayeredImage` with nix-built app (ringtail runner) |
|
||||
|
||||
### navidrome
|
||||
|
||||
`containers/navidrome/Dockerfile` — Three-stage build with separate Node.js UI compilation, Go backend build with CGO (taglib), and a minimal Alpine runtime with ffmpeg.
|
||||
`containers/navidrome/container.py` — Native Dagger build. Three-stage pipeline using helper functions: `node_build()` for UI, `go_build()` with CGO/taglib/FTS5 for backend, `alpine_runtime()` with ffmpeg. This is the reference pattern for migrating Dockerfile containers to native Dagger builds.
|
||||
|
||||
### transmission
|
||||
|
||||
`containers/transmission/Dockerfile` — Installs transmission-daemon directly from Alpine packages. Good starting point for services available in apk. (Legacy Dockerfile — migrate to `container.py` during review.)
|
||||
|
||||
### miniflux
|
||||
|
||||
`containers/miniflux/Dockerfile` — Two-stage Go build. Clones upstream at a pinned version tag, runs `make`, copies the binary into a minimal Alpine runtime. (Legacy Dockerfile — migrate to `container.py` during review.)
|
||||
|
||||
### teslamate
|
||||
|
||||
`containers/teslamate/Dockerfile` — Two-stage Elixir build with Node.js asset compilation. Uses Debian-based images due to Elixir/OTP dependencies.
|
||||
`containers/teslamate/Dockerfile` — Two-stage Elixir build with Node.js asset compilation. Uses Debian-based images due to Elixir/OTP dependencies. (Legacy Dockerfile — migrate to `container.py` during review.)
|
||||
|
||||
### kiwix-serve
|
||||
|
||||
`containers/kiwix-serve/Dockerfile` — Downloads a pre-built binary from upstream, with architecture detection for cross-platform support.
|
||||
`containers/kiwix-serve/Dockerfile` — Downloads a pre-built binary from upstream, with architecture detection for cross-platform support. (Legacy Dockerfile — migrate to `container.py` during review.)
|
||||
|
||||
### ntfy (nix)
|
||||
|
||||
`containers/ntfy/default.nix` — Builds ntfy from source using `buildGoModule` and packages it with `dockerTools.buildLayeredImage`. Runs alongside the existing Dockerfile; the nix variant is tagged `:version-nix` in the registry.
|
||||
`containers/ntfy/default.nix` — Builds ntfy from source using `buildGoModule` and packages it with `dockerTools.buildLayeredImage`. Runs alongside the existing Dockerfile; the nix variant is tagged `:version-nix` in the registry. Nix containers should continue using `default.nix`.
|
||||
|
||||
## Related
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Validate Workflows Against v12
|
||||
modified: 2026-02-27
|
||||
modified: 2026-04-11
|
||||
last-reviewed: 2026-02-27
|
||||
tags:
|
||||
- how-to
|
||||
|
|
@ -25,7 +25,7 @@ All 6 workflows pass v12.7.0 schema validation with no changes needed:
|
|||
|
||||
## Deliverables
|
||||
|
||||
1. `validate_workflows` function added to `.dagger/src/blumeops_ci/main.py`
|
||||
1. `validate_workflows` function added to `src/blumeops/main.py` (formerly `.dagger/src/blumeops_ci/main.py`)
|
||||
- Uses `forgejo-runner validate --directory .` inside the upstream runner container
|
||||
- `runner_version` parameter (default `12.7.0`) pins to deployed version
|
||||
2. `mise run validate-workflows` task wired to `dagger call validate-workflows`
|
||||
|
|
|
|||
|
|
@ -47,6 +47,7 @@ For all service types, start by reading the service's reference card (`docs/refe
|
|||
3. Review the upstream changelog for breaking changes
|
||||
4. If the service uses a custom-built container, also check the base image for security updates and follow [[build-container-image]] to rebuild
|
||||
5. If upgrading, update the manifest and follow [[deploy-k8s-service]]
|
||||
6. If the container still uses a Dockerfile (no `container.py`), consider migrating to a native Dagger build — see the `containers/navidrome/container.py` pattern for reference
|
||||
|
||||
### Ansible Services (`type: ansible`)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Add Container Version Sync Check
|
||||
modified: 2026-02-20
|
||||
modified: 2026-04-11
|
||||
tags:
|
||||
- how-to
|
||||
- containers
|
||||
|
|
@ -10,7 +10,7 @@ tags:
|
|||
|
||||
# Add Container Version Sync Check
|
||||
|
||||
Add a prek check that validates version consistency across the three places container versions are declared: Dockerfile ARGs, `service-versions.yaml`, and nix derivations. No VERSION files needed — the existing sources are the source of truth, and the check enforces they agree.
|
||||
Add a prek check that validates version consistency across the places container versions are declared: `container.py` VERSION constants, Dockerfile ARGs, `service-versions.yaml`, and nix derivations. The check enforces they agree.
|
||||
|
||||
## Context
|
||||
|
||||
|
|
@ -20,13 +20,14 @@ Discovered during analysis of [[adopt-commit-based-container-tags]]: the new com
|
|||
|
||||
### 1. Created `mise run container-version-check` task
|
||||
|
||||
A typer-based uv-script that iterates over `containers/*/` and validates five rules per container:
|
||||
A typer-based uv-script that iterates over `containers/*/` and validates six rules per container:
|
||||
|
||||
1. Any Dockerfile must declare `ARG CONTAINER_APP_VERSION=<value>`
|
||||
2. Any `default.nix` must produce a version via `dagger call nix-version`
|
||||
3. At least one build file must exist (Dockerfile or default.nix)
|
||||
4. A matching `service-versions.yaml` entry must exist with non-null `current-version`
|
||||
5. All resolved versions from (1), (2), and (4) must agree (v-prefix stripped for comparison)
|
||||
1. Any `container.py` must declare `VERSION = "<value>"`
|
||||
2. Any Dockerfile must declare `ARG CONTAINER_APP_VERSION=<value>`
|
||||
3. Any `default.nix` must produce a version via `dagger call nix-version`
|
||||
4. At least one build file must exist (`container.py`, Dockerfile, or `default.nix`)
|
||||
5. A matching `service-versions.yaml` entry must exist with non-null `current-version`
|
||||
6. All resolved versions from (1), (2), (3), and (5) must agree (v-prefix stripped for comparison)
|
||||
|
||||
Scoping: by default only checks containers changed vs main. `--all-files` checks everything. If `service-versions.yaml` itself changed, all containers are checked.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Add Dagger Nix Build Function
|
||||
modified: 2026-02-20
|
||||
modified: 2026-04-11
|
||||
tags:
|
||||
- how-to
|
||||
- containers
|
||||
|
|
@ -23,7 +23,7 @@ Currently, nix containers can only be built on ringtail (the `nix-container-buil
|
|||
|
||||
### 1. Add `build_nix` Dagger function
|
||||
|
||||
A new function in `.dagger/src/blumeops_ci/main.py` that builds a nix container inside a `nixos/nix` container:
|
||||
A new function in `src/blumeops/main.py` (formerly `.dagger/src/blumeops_ci/main.py`) that builds a nix container inside a `nixos/nix` container:
|
||||
|
||||
```python
|
||||
@function
|
||||
|
|
@ -80,7 +80,7 @@ The `flake_lock` function already demonstrates running nix inside Dagger using `
|
|||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `.dagger/src/blumeops_ci/main.py` | Add `build_nix`, `nix_version`, optionally `publish_nix` |
|
||||
| `src/blumeops/main.py` | Add `build_nix`, `nix_version`, optionally `publish_nix` |
|
||||
|
||||
## Verification
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Adopt Commit-Based Container Tags
|
||||
modified: 2026-02-20
|
||||
modified: 2026-04-11
|
||||
tags:
|
||||
- how-to
|
||||
- containers
|
||||
|
|
@ -64,7 +64,7 @@ Where:
|
|||
|------|--------|
|
||||
| `.forgejo/workflows/build-container.yaml` | Replace tag trigger with path + dispatch triggers; compute version and SHA |
|
||||
| `.forgejo/workflows/build-container-nix.yaml` | Same trigger changes; add `-nix` suffix to new tag format |
|
||||
| `.dagger/src/blumeops_ci/main.py` | Accept SHA parameter; publish with new tag format |
|
||||
| `src/blumeops/main.py` | Accept SHA parameter; publish with new tag format |
|
||||
| `mise-tasks/container-build-and-release` | New task replacing `container-tag-and-release`; triggers workflow dispatch |
|
||||
| `mise-tasks/container-list` | Updated tag display for new format |
|
||||
| `docs/how-to/deployment/build-container-image.md` | Updated documentation |
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Harden Zot Registry
|
||||
modified: 2026-02-21
|
||||
modified: 2026-04-11
|
||||
tags:
|
||||
- how-to
|
||||
- zot
|
||||
|
|
@ -32,7 +32,7 @@ Updated `ansible/roles/zot/templates/config.json.j2` with:
|
|||
| `ansible/roles/zot/templates/config.json.j2` | Zot config with auth + access control |
|
||||
| `ansible/roles/zot/defaults/main.yml` | OIDC issuer and external URL variables |
|
||||
| `ansible/roles/zot/templates/oidc-credentials.json.j2` | OIDC client credentials |
|
||||
| `.dagger/src/blumeops_ci/main.py` | `publish()` with registry auth |
|
||||
| `src/blumeops/main.py` | `publish()` with registry auth |
|
||||
| `.forgejo/workflows/build-container.yaml` | Dagger push with API key |
|
||||
| `.forgejo/workflows/build-container-nix.yaml` | Skopeo push with API key |
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Pin Container Versions
|
||||
modified: 2026-02-20
|
||||
modified: 2026-04-11
|
||||
tags:
|
||||
- how-to
|
||||
- containers
|
||||
|
|
@ -18,13 +18,15 @@ Discovered during analysis of [[adopt-commit-based-container-tags]]: containers
|
|||
|
||||
## What Was Done
|
||||
|
||||
Every container Dockerfile now declares `ARG CONTAINER_APP_VERSION=X.Y.Z` as its first ARG, providing a uniform parsing target. Containers that use the version in build commands chain it to a semantic ARG:
|
||||
Every container Dockerfile declares `ARG CONTAINER_APP_VERSION=X.Y.Z` as its first ARG, providing a uniform parsing target. Containers that use the version in build commands chain it to a semantic ARG:
|
||||
|
||||
```dockerfile
|
||||
ARG CONTAINER_APP_VERSION=v0.60.3
|
||||
ARG NAVIDROME_VERSION=${CONTAINER_APP_VERSION}
|
||||
```
|
||||
|
||||
> **Note:** Containers migrated to native Dagger builds use `VERSION = "X.Y.Z"` in `container.py` instead. See `containers/navidrome/container.py` for the pattern. New containers should use `container.py` rather than Dockerfiles.
|
||||
|
||||
Specific changes:
|
||||
- **devpi**: Pinned devpi-server==6.19.1 and devpi-web==5.0.1
|
||||
- **cv**: `CONTAINER_APP_VERSION=1.0.3` (matches latest Forgejo package release)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
title: Wire CI Registry Auth
|
||||
modified: 2026-02-21
|
||||
modified: 2026-04-11
|
||||
tags:
|
||||
- how-to
|
||||
- zot
|
||||
|
|
@ -36,7 +36,7 @@ Authentication uses a zot API key generated after the service account's first OI
|
|||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `.dagger/src/blumeops_ci/main.py` | `publish()` accepts optional `registry_password` |
|
||||
| `src/blumeops/main.py` | `publish()` accepts optional `registry_password` |
|
||||
| `.forgejo/workflows/build-container.yaml` | Passes API key to Dagger |
|
||||
| `.forgejo/workflows/build-container-nix.yaml` | Passes API key to skopeo |
|
||||
| `ansible/playbooks/indri.yml` | Pre_task fetches API key from 1Password |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue