C2: Build authentik from source (Mikado chain) (#274)
All checks were successful
Build Container / detect (push) Successful in 3s
Build Container (Nix) / detect (push) Successful in 1s
Build Container / build (authentik) (push) Successful in 2s
Build Container (Nix) / build (authentik) (push) Successful in 22s

## Mikado Chain: build-authentik-from-source

Replace `pkgs.authentik` from nixpkgs with a custom Nix derivation built from source.
This removes the dependency on the nixpkgs packaging timeline and gives full version control.

Target version: **2025.12.4** (nixpkgs reference, upgrading from deployed 2025.10.1).

### Dependency Graph

```
build-authentik-from-source (goal)
├── authentik-go-server-derivation
│   ├── authentik-api-client-generation  ← IN PROGRESS
│   └── authentik-python-backend-derivation
├── authentik-web-ui-derivation
│   └── authentik-api-client-generation  ← IN PROGRESS
└── authentik-python-backend-derivation
```

### Ready Leaves
- `authentik-api-client-generation` — Go + TypeScript client generation from OpenAPI schema
- `authentik-python-backend-derivation` — Django backend with 60+ deps, 4 in-tree packages

### Architecture
Ported from [nixpkgs `pkgs/by-name/au/authentik/package.nix`](https://github.com/NixOS/nixpkgs/tree/master/pkgs/by-name/au/authentik):
- `source.nix` — shared version/source fetch
- `client-go.nix` — Go API client generation
- `client-ts.nix` — TypeScript API client generation
- `api-go-vendor-hook.nix` — Go vendor directory injection hook
- (more components to follow as leaves are closed)

### Related Cards
- [[build-authentik-from-source]] — Goal card
- [[authentik-api-client-generation]]
- [[authentik-python-backend-derivation]]
- [[authentik-web-ui-derivation]]
- [[authentik-go-server-derivation]]

Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/274
This commit is contained in:
Erich Blume 2026-03-01 13:45:00 -08:00
commit efa9806bfa
21 changed files with 868 additions and 85 deletions

View file

@ -1 +1 @@
Start C2 Mikado chain: build authentik from a custom Nix derivation (from source) to replace nixpkgs dependency and gain full version control.
Build authentik 2026.2.0 from source via custom Nix derivation, replacing the nixpkgs `pkgs.authentik` dependency. Four components (API client generation, Python backend, web UI, Go server) assembled into a single container image with full supply chain control via forge mirrors.

View file

@ -163,7 +163,7 @@ The `mikado-branch-invariant-check` commit-msg hook validates this convention an
2. **Open a PR** after the first card commits so the user can review the Mikado graph
3. **Work leaf nodes** — pick a leaf (a card with `status: active` and no unmet `requires`):
- Commit code changes (`C2(<chain>): impl ...`) that progress toward closing it
- **Verify the change works** (deploy from branch, run tests, etc.) before closing
- **Verify the card's own deliverables** (deploy from branch, run tests, etc.) before closing. "Works" means the card's stated outputs are correct — not that downstream consumers have integrated them. If a downstream card later discovers the output doesn't fit, that's a new prerequisite discovery handled by the normal reset mechanism.
- Commit the card closure (`C2(<chain>): close ...`) — remove `status: active`
- Push to origin — this is the save point
4. **End the cycle** — after pushing a closed leaf node, prompt the user to review the PR and suggest ending the session. Each closed leaf is a natural stopping point; the chain is designed to be resumed later. Don't rush into the next leaf without the user's go-ahead.

View file

@ -1,7 +1,8 @@
---
title: Generate Authentik API Clients
modified: 2026-02-28
status: active
requires:
- mirror-authentik-build-deps
tags:
- how-to
- authentik
@ -32,6 +33,30 @@ Both clients are generated from the same `schema.yml` OpenAPI spec in the main a
- TypeScript client replaces `web/node_modules/@goauthentik/api/` in the web UI build
- The nixpkgs derivation patches the generated Go client (`client-go-config.patch`) — check if still needed
## Testing on Ringtail
Use this ad-hoc `test-build.nix` harness (not committed to the repo):
```nix
# test-build.nix
let
pkgs = (builtins.getFlake "nixpkgs").legacyPackages.x86_64-linux;
sources = import ./sources.nix { inherit pkgs; };
in
{
client-go = import ./client-go.nix { inherit pkgs sources; };
client-ts = import ./client-ts.nix { inherit pkgs sources; };
api-go-vendor-hook = import ./api-go-vendor-hook.nix { inherit pkgs sources; };
}
```
```fish
set tmpdir (ssh ringtail 'mktemp -d /tmp/authentik-test.XXXXXX')
scp containers/authentik/*.nix ringtail:$tmpdir/
ssh ringtail "cd $tmpdir && nix-build test-build.nix -A client-go --extra-experimental-features 'nix-command flakes'"
ssh ringtail "rm -rf $tmpdir"
```
## Related
- [[build-authentik-from-source]] — Parent goal

View file

@ -1,7 +1,6 @@
---
title: Build Authentik Go Server
modified: 2026-02-28
status: active
modified: 2026-03-01
requires:
- authentik-api-client-generation
- authentik-python-backend-derivation

View file

@ -1,7 +1,8 @@
---
title: Build Authentik Python Backend
modified: 2026-02-28
status: active
modified: 2026-03-01
requires:
- mirror-authentik-build-deps
tags:
- how-to
- authentik
@ -14,32 +15,60 @@ Build `authentik-django` — the Python/Django application that forms the core b
## Context
This is the most complex component. The nixpkgs derivation uses `python3.override` with extensive `packageOverrides` to handle authentik's non-standard dependencies:
Authentik 2026.2.0 requires Python 3.14 (`requires-python = "==3.14.*"`). The nixpkgs reference derivation (2025.12.4) builds all 60+ Python deps through nix's `python3.override` with `packageOverrides`. This approach breaks on Python 3.14 because many nixpkgs python314 packages haven't been updated — astor, dacite, exceptiongroup, and pydantic-core all fail to build.
- **4 in-tree Python packages** built from the monorepo: `ak-guardian`, `django-channels-postgres`, `django-dramatiq-postgres`, `django-postgres-cache`
- **Forked `djangorestframework`** from `authentik-community/django-rest-framework` (specific commit)
- **Pinned `dramatiq`** at 1.17.1 (upstream uses newer versions that break authentik)
- **Django 5** forced via `django_5`
- **60+ Python dependencies** from nixpkgs
Instead of carrying individual overrides for each broken package, we use **`uv`** to install Python dependencies from PyPI, where upstream maintainers have already published Python 3.14-compatible wheels. Nix provides only the Python interpreter and system libraries.
Post-install, the derivation patches hardcoded paths in `settings.py`, `default.yml`, `email/utils.py`, and `files/backends/file.py` to reference Nix store paths.
## Approach: uv sync FOD + autoPatchelfHook
Nix builds are sandboxed with no network access. The pattern is:
1. **Fixed-output derivation (FOD)**`uv sync --frozen` fetches and installs all dependencies into a venv. FODs are allowed network access because the output hash is declared upfront. Compiled `.so` files reference Nix store paths (RPATHs to libxml2, krb5, etc.), which FODs must not contain, so we strip references with `remove-references-to` and delete `bin/` and `.pyc` files.
2. **Main derivation** — copies the FOD's `lib/python3.14/site-packages/`, recreates `bin/` with proper python symlinks, restores `pyvenv.cfg`, and runs `autoPatchelfHook` to re-link `.so` files against the correct Nix store libraries.
**Why not `uv pip download` + `uv pip install --no-index`?** `uv pip download` does not exist in uv 0.9.29 (nixpkgs). And the download-only approach has further complications with sdist-only packages (psycopg-c, gssapi) that must be compiled anyway.
## What to Do
1. Create a Python package override set that builds the 4 in-tree packages from source
2. Pin the forked `djangorestframework` and `dramatiq` versions
3. Build `authentik-django` using `hatchling` as the build backend
4. Apply the 4 `substituteInPlace` patches for Nix store path references
5. Copy lifecycle scripts, `manage.py`, blueprints, and web assets into the output
6. Verify: `python -c "import authentik"` succeeds
1. Create the FOD (`python-deps.nix`) that runs `uv sync --frozen --no-install-project --no-install-workspace --no-dev`, then strips all Nix store references from the output
2. Create the main derivation (`authentik-django.nix`) that:
- Copies the FOD's site-packages
- Recreates venv `bin/` and `pyvenv.cfg`
- Runs `autoPatchelfHook` to restore `.so` RPATHs
- Copies 4 in-tree workspace packages directly into site-packages
- Copies `authentik/` and `lifecycle/` into site-packages
- Copies `opencontainers` from `fetchFromGitHub` into site-packages
3. Apply `substituteInPlace` patches for Nix store paths in `settings.py`, `default.yml`, `email/utils.py`
4. Copy lifecycle scripts, `manage.py`, blueprints into the output
5. Verify: `$out/bin/python3.14 -c "import authentik"` succeeds
## Key Details
- Build backend: `hatchling`
- Entry point: `manage.py` (Django management commands)
- Lifecycle scripts: `lifecycle/` directory (used by Go server and `ak` wrapper)
- Blueprints: `blueprints/` directory (YAML IaC definitions)
- The output must include `web/` assets (email templates reference them)
- Nix provides: `python314`, `uv`, system libraries (`libxml2`, `libxslt`, `openssl`, `libffi`, `zlib`, etc.)
- PyPI provides: all Python packages (via pre-built `cp314` wheels where available, sdist builds otherwise)
- The FOD hash must be recomputed when `uv.lock` changes
- `manylinux` wheels bundle some `.so` files — acceptable for a container image
- The 4 in-tree packages are installed from monorepo source, not PyPI
- Standard `djangorestframework` 3.16.1 from PyPI (no longer forked as of 2026.2.0)
## Lessons Learned
Build issues encountered and resolved:
| Issue | Fix |
|-------|-----|
| `pg_config` not found for psycopg-c | Use `pkgs.postgresql.pg_config` (separate derivation), not `pkgs.postgresql` |
| gssapi `gss_acquire_cred_impersonate_name` undeclared | `NIX_CFLAGS_COMPILE="-include gssapi/gssapi_ext.h"` — function is in `gssapi_ext.h`, not auto-included |
| xmlsec linker error `-lltdl` | Add `pkgs.libtool` to buildInputs (provides libltdl) |
| psycopg-c needs `libpq` | Add `pkgs.libpq` to buildInputs |
| Static `refTargets` list missed 6 store refs | Replaced with dynamic discovery: `grep -aohE '/nix/store/...'` finds all refs, `remove-references-to` strips them |
| `xargs grep` exit code 123 under `pipefail` | Wrap pipeline in `{ ... \|\| true; }` — grep returning 1 (no match) causes xargs to return 123 |
| `grep -aoE` includes filename prefix in output | Use `grep -aohE` (`-h` suppresses filenames) to get clean store paths |
| autoPatchelfHook can't find libraries | `buildInputs` in main derivation must include all libraries that `.so` files link against |
The `uv sync` completes in ~3.5 minutes. Dynamic reference discovery finds 19 unique store paths and strips all of them. After stripping, `remove-references-to` mangles hashes to `eeee...` bytes — about 40 files still "contain" `/nix/store/` strings but with invalid hashes, which is expected and harmless. `autoPatchelfHook` in the main derivation resolves all NEEDED entries with 0 unsatisfied dependencies.
Build verified: `$out/bin/python3.14 -c "import authentik"` succeeds, along with all key dependencies (django 5.2.11, lxml, xmlsec, psycopg, guardian, opencontainers).
## Related

View file

@ -1,7 +1,6 @@
---
title: Build Authentik Web UI
modified: 2026-02-28
status: active
modified: 2026-03-01
requires:
- authentik-api-client-generation
tags:
@ -14,30 +13,28 @@ tags:
Build the Lit-based TypeScript web frontend for authentik.
## Context
## Overview
The web UI lives in `web/` in the authentik repo. It's built with Rollup and uses Lit web components. The nixpkgs derivation builds this in two phases:
The web UI lives in `web/` in the authentik repo. As of 2026.2.0, the main build uses **esbuild** (via wireit) and the SFE sub-package uses **rollup**. The Nix build uses a two-phase approach:
1. **`webui-deps`** — Fixed-output derivation that runs `npm ci` to fetch Node dependencies. Uses platform-specific output hashes (aarch64-linux vs x86_64-linux).
2. **`webui`** — Patches in the generated TypeScript API client (`client-ts`), then runs `npm run build`. Output includes `dist/` and `authentik/` static directories.
1. **`webui-deps.nix`** — Fixed-output derivation that runs `npm ci` to fetch Node dependencies. Platform-specific output hash (npm downloads architecture-specific native binaries for esbuild, rollup, and SWC).
2. **`webui.nix`** — Copies deps, patches in the generated TypeScript API client (`client-ts`), patches shebangs, then runs `npm run build` (wireit/esbuild) and `npm run build:sfe` (rollup). Output includes `dist/` and `authentik/` static directories.
There's also a **`website`** derivation (Docusaurus-based API docs at `website/`) that produces the `/help` endpoint. This is optional but included in the nixpkgs build.
## Build Details
## What to Do
- **Node.js:** `nodejs_24` (authentik requires Node >= 24, npm >= 11.6.2)
- **Build time:** ~33s on ringtail (x86_64-linux)
- **FOD hash:** Platform-specific — will need updating on each authentik version bump
- **Output:** `$out/dist/` (JS/CSS bundles) and `$out/authentik/` (static SVG/PNG icons)
- **Consumed by:** Go server (`authentik-server.nix` via `webui` parameter) for static file serving, and `authentik-django.nix` for email template icon paths
- **Docusaurus website** (`/help` endpoint) is not built — optional and can be added later
1. Create a fixed-output derivation for `npm ci` in `web/` (platform-specific hashes)
2. Patch the generated TypeScript client into `web/node_modules/@goauthentik/api/`
3. Build with `npm run build` — produces `dist/` and `authentik/` directories
4. Optionally build the Docusaurus website (`website/`) for the `/help` endpoint
5. Verify: static assets exist and reference correct paths
## Key Lessons
## Key Details
- Build tool: Rollup (via npm scripts)
- Node.js version: `nodejs_24` in current nixpkgs (check upstream requirements)
- The TypeScript API client must be patched in before the build
- Fixed-output hashes break on any npm dependency change — will need updating per release
- Output is consumed by both `authentik-django` (email templates) and the Go server (static serving)
- The 2026.2.0 build switched from rollup to esbuild for the main frontend. Only the SFE sub-package still uses rollup.
- The version string in `packages/core/version/node.js` uses a JSON import-with-assertion that doesn't resolve in the Nix sandbox — must be patched to hardcode the version.
- `NODE_OPTIONS=--openssl-legacy-provider` is needed for compatibility.
- Workspace packages have separate `node_modules/` directories — the FOD must collect all of them via `find`.
## Related

View file

@ -1,8 +1,6 @@
---
title: Build Authentik from Source
modified: 2026-02-28
status: active
branch: mikado/authentik-source-build
modified: 2026-03-01
requires:
- authentik-go-server-derivation
- authentik-web-ui-derivation
@ -15,45 +13,58 @@ tags:
# Build Authentik from Source
Replace `pkgs.authentik` from nixpkgs with a custom Nix derivation that builds authentik from source. This removes the dependency on the nixpkgs packaging timeline and gives full version control.
Custom Nix derivation that builds authentik from source, replacing the `pkgs.authentik` nixpkgs dependency. This gives full version control independent of the nixpkgs release cycle.
## Motivation
The nix-container-builder runner on ringtail resolves `nixpkgs` via the NixOS nix registry, which pins to `nixos-25.11`. That channel lags behind upstream authentik releases — e.g. nixos-25.11 has 2025.10.1 while upstream is at 2025.12.4+. Building from source lets us target any release.
This also serves as practice for packaging services from source using Nix, relying on nixpkgs only for satellite dependencies (Python interpreter, Node.js, Go toolchain, system libraries).
The nix-container-builder runner on ringtail resolves `nixpkgs` via the NixOS nix registry, which pins to `nixos-25.11`. That channel lags behind upstream authentik releases. Building from source lets us target any release by updating `sources.nix`.
## Architecture
Authentik has four build components that must be assembled:
Authentik has four build components assembled by `containers/authentik/default.nix`:
1. **API client generation** — Go and TypeScript bindings generated from `schema.yml` (OpenAPI)
2. **Python backend** (`authentik-django`) — Django application with 60+ Python dependencies, including 4 in-tree packages and a forked `djangorestframework`
3. **Web UI** — Lit-based TypeScript frontend built with Rollup
4. **Go server** — HTTP server binary (`cmd/server`) that serves the web UI and spawns gunicorn for Django
1. **API client generation** (`client-go.nix`, `client-ts.nix`) — Go and TypeScript bindings generated from `schema.yml` (OpenAPI)
2. **Python backend** (`authentik-django.nix`) — Django application with 60+ Python dependencies installed via `uv` from PyPI (see [[authentik-python-backend-derivation]])
3. **Web UI** (`webui.nix`) — Lit-based TypeScript frontend built with esbuild + rollup
4. **Go server** (`authentik-server.nix`) — HTTP server binary that serves the web UI and spawns gunicorn for Django
The final package is the `ak` bash wrapper that orchestrates Go server + Python worker.
The `ak` wrapper script in `default.nix` sets PATH/VIRTUAL_ENV and delegates to `lifecycle/ak`, which dispatches `server` to the Go binary and everything else to Python/Django.
**Python packaging strategy:** Nix provides the Python 3.14 interpreter and system libraries. Python packages are installed from PyPI using `uv`, locked by authentik's `uv.lock`. This avoids nixpkgs' Python 3.14 compatibility issues and aligns with upstream's build process.
## Source
Forge mirror: https://forge.ops.eblu.me/mirrors/authentik (upstream: `goauthentik/authentik`)
All derivations fetch from forge mirrors for supply chain control:
- https://forge.ops.eblu.me/mirrors/authentik (upstream: `goauthentik/authentik`)
- https://forge.ops.eblu.me/mirrors/authentik-client-go (upstream: `goauthentik/client-go`)
Reference derivation: [nixpkgs `pkgs/by-name/au/authentik/package.nix`](https://github.com/NixOS/nixpkgs/tree/master/pkgs/by-name/au/authentik)
Version and hashes are centralized in `containers/authentik/sources.nix`.
## What to Do
## Updating to a New Version
Once all prerequisites are complete:
1. Update `version` in `sources.nix` and `default.nix`
2. Update `src` and `client-go-src` hashes in `sources.nix` (use `nix-prefetch-git` on ringtail)
3. Rebuild `python-deps.nix` FOD — hash changes when `uv.lock` changes
4. Rebuild `webui-deps.nix` FOD — hash changes when `package-lock.json` or platform-specific npm binaries change
5. Recompute `vendorHash` in `authentik-server.nix` if Go dependencies changed
6. Test on ringtail: `nix-build test-build.nix -A assembled`
7. Build and push the container via CI
1. Assemble the component derivations into a final `ak`-wrapped package in `containers/authentik/`
2. Update `containers/authentik/default.nix` to use the custom derivation instead of `pkgs.authentik`
3. Test locally via Dagger before pushing to CI: `dagger call build-nix --src=. --container-name=authentik`
4. Build and push the container: `mise run container-build-and-release authentik`
5. Update `argocd/manifests/authentik/kustomization.yaml` with the new image tag
6. Update `service-versions.yaml` with the new version
7. Verify deployment: ArgoCD sync, UI login, OAuth2 flows
## Testing
Nix derivations target `x86_64-linux`. Test incrementally on ringtail:
```fish
set tmpdir (ssh ringtail 'mktemp -d /tmp/authentik-test.XXXXXX')
scp containers/authentik/*.nix ringtail:$tmpdir/
ssh ringtail "cd $tmpdir && nix-build test-build.nix -A assembled --extra-experimental-features 'nix-command flakes'"
ssh ringtail "rm -rf $tmpdir"
```
`test-build.nix` provides both individual component targets and a fully-wired `assembled` target.
## Related
- [[build-authentik-container]] — Current nixpkgs-based build (to be replaced)
- [[build-authentik-container]] — Container build reference
- [[deploy-authentik]] — Parent deployment goal
- [[agent-change-process]] — C2 methodology

View file

@ -0,0 +1,40 @@
---
title: Mirror Authentik Build Dependencies
modified: 2026-02-28
tags:
- how-to
- authentik
---
# Mirror Authentik Build Dependencies
Mirror the external repositories needed to build authentik from source onto the forge, ensuring full supply chain control.
## Context
Building authentik from source requires fetching code from three GitHub repositories. The main `goauthentik/authentik` repo is already mirrored, but two companion repos are not:
- **`goauthentik/client-go`** — Go API client bindings, versioned in lockstep with authentik (e.g. `v3.2026.2.0` matches `version/2026.2.0`). Used by the Go server build.
- **`authentik-community/django-rest-framework`** — Fork of DRF pinned to a specific commit. Authentik's Python backend requires this custom version. The upstream org name (`authentik-community`) differs from the main repo org (`goauthentik`), so the mirror name must be explicit.
## What to Do
1. Mirror `goauthentik/client-go`:
```fish
mise run mirror-create https://github.com/goauthentik/client-go.git \
--name authentik-client-go \
--description "Go API client for authentik (lockstep versioned)"
```
2. Mirror `authentik-community/django-rest-framework`:
```fish
mise run mirror-create https://github.com/authentik-community/django-rest-framework.git \
--name authentik-django-rest-framework \
--description "Authentik fork of Django REST Framework"
```
3. Verify both mirrors sync: check tags appear on forge
## Related
- [[build-authentik-from-source]] — Parent goal
- [[authentik-api-client-generation]] — Consumes client-go mirror
- [[authentik-python-backend-derivation]] — Consumes django-rest-framework mirror

View file

@ -101,6 +101,7 @@ Mikado chain for deploying Authentik. Track progress with `mise run docs-mikado
Mikado chain for building Authentik from a custom Nix derivation (from source). Track progress with `mise run docs-mikado build-authentik-from-source`.
- [[build-authentik-from-source]]
- [[mirror-authentik-build-deps]]
- [[authentik-api-client-generation]]
- [[authentik-python-backend-derivation]]
- [[authentik-web-ui-derivation]]