diff --git a/argocd/manifests/kingfisher/cronjob.yaml b/argocd/manifests/kingfisher/cronjob.yaml index e895b59..5720810 100644 --- a/argocd/manifests/kingfisher/cronjob.yaml +++ b/argocd/manifests/kingfisher/cronjob.yaml @@ -28,7 +28,9 @@ spec: kingfisher scan gitea \ --api-url https://forge.ops.eblu.me/api/v1/ \ + --clone-url-base https://forge.ops.eblu.me/ \ --user eblume \ + --all-organizations \ --repo-type all \ --no-update-check \ --tls-mode lax \ diff --git a/containers/kingfisher/Dockerfile b/containers/kingfisher/Dockerfile index 13eb25f..32a5014 100644 --- a/containers/kingfisher/Dockerfile +++ b/containers/kingfisher/Dockerfile @@ -1,6 +1,15 @@ -# Kingfisher — built from sporked deploy branch -# Multi-stage: Rust build with vectorscan/Boost, then minimal Alpine runtime -ARG CONTAINER_APP_VERSION=latest +# Kingfisher — deterministic build from sporked feature branches +# +# Builds a fully-pinned kingfisher binary by checking out a specific upstream +# SHA and merging feature branches at specific SHAs on top. Independent of +# the 'deploy' branch, which is a convenience view and may have moved. +# +# Inputs: +# CONTAINER_APP_VERSION — commit on the upstream 'main' branch to base on +# FEATURES — space-separated "branch=sha" pairs to merge on top +# +# The resulting binary includes upstream code + local patches, reproducible +# from the same inputs regardless of when the build runs. # --- Build stage --- FROM rust:1.92-bookworm AS build @@ -11,10 +20,26 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ WORKDIR /build -ARG CONTAINER_APP_VERSION -RUN git clone --branch deploy \ - https://forge.ops.eblu.me/eblume/kingfisher.git . \ - && git checkout "${CONTAINER_APP_VERSION}" +ARG CONTAINER_APP_VERSION=1d37d29 +ARG FEATURES="feature/upstream/clone-url-base=677c7a5" + +RUN git clone https://forge.ops.eblu.me/eblume/kingfisher.git . \ + && git checkout "${CONTAINER_APP_VERSION}" \ + && git config user.name "container-build" \ + && git config user.email "build@blumeops" + +# Merge each pinned feature branch +RUN set -e; \ + for spec in ${FEATURES}; do \ + branch="${spec%%=*}"; \ + sha="${spec##*=}"; \ + echo "Merging ${branch} at ${sha}..."; \ + git fetch origin "${branch}"; \ + git merge --no-ff "${sha}" \ + -m "container-build: merge ${branch} at ${sha}" \ + || { echo "ERROR: merge conflict on ${branch}"; exit 1; }; \ + done; \ + echo "Build tree ready at $(git rev-parse --short HEAD)" RUN cargo build --release \ && install -m 0755 target/release/kingfisher /usr/local/bin/kingfisher diff --git a/docs/explanation/spork-strategy.md b/docs/explanation/spork-strategy.md index ce1c999..f5ac4ea 100644 --- a/docs/explanation/spork-strategy.md +++ b/docs/explanation/spork-strategy.md @@ -60,6 +60,7 @@ Note that a cron-triggered workflow is especially dangerous: it requires no user - [[create-a-spork]] — initial setup with `mise run spork-create` - [[manage-spork-branches]] — feature branches, the deploy branch, handling rebase conflicts +- [[build-spork-container]] — building reproducible containers from pinned SHAs ## See also diff --git a/docs/how-to/configuration/build-spork-container.md b/docs/how-to/configuration/build-spork-container.md new file mode 100644 index 0000000..1c5f979 --- /dev/null +++ b/docs/how-to/configuration/build-spork-container.md @@ -0,0 +1,86 @@ +--- +title: Build a Spork Container +modified: 2026-03-29 +last-reviewed: 2026-03-29 +tags: + - how-to + - containers + - git +--- + +# Build a Spork Container + +How to build a container image from a [[spork-strategy|sporked]] project with fully-pinned, reproducible inputs. + +## Why not use the `deploy` branch directly? + +The `deploy` branch is force-pushed on every mirror-sync. Building from `deploy` is not reproducible — the same Dockerfile run a week later gives different code. Instead, spork containers build their own merge tree from explicit inputs: + +- **`CONTAINER_APP_VERSION`** — the commit on `main` to base on (the upstream version) +- **`FEATURES`** — space-separated `branch=sha` pairs to merge on top + +This makes builds reproducible regardless of when they run. + +## Prerequisites + +- Sporked project set up (see [[create-a-spork]]) +- Container build tooling (`mise run container-build-and-release`) + +## Get the SHAs + +```fish +cd ~/code/3rd/kingfisher +git fetch origin + +# Upstream SHA (what main is based on) +git rev-parse --short origin/main +# e.g., 1d37d29 + +# Feature branch SHAs +git rev-parse --short origin/feature/upstream/clone-url-base +# e.g., 677c7a5 +``` + +## Build the container + +```fish +# The version in service-versions.yaml is the upstream SHA +mise run container-build-and-release kingfisher 1d37d29 \ + --build-arg CONTAINER_APP_VERSION=1d37d29 \ + --build-arg FEATURES="feature/upstream/clone-url-base=677c7a5" +``` + +The container tag will be `1d37d29-`. + +## Update the deployment + +1. Update `argocd/manifests/kingfisher/kustomization.yaml` with the new tag +2. Update `service-versions.yaml` if the upstream SHA changed +3. Sync the ArgoCD app + +## How the Dockerfile works + +The build stage: + +1. Clones the sporked repo from forge +2. Checks out `main` at `CONTAINER_APP_VERSION` +3. For each entry in `FEATURES`, fetches the branch and merges at the pinned SHA +4. Builds from source with `cargo build --release` + +If any merge conflicts, the build fails loudly. + +The runtime stage is minimal: debian-slim + git + the binary. + +## Note on `CONTAINER_APP_VERSION` + +For most blumeops containers, `CONTAINER_APP_VERSION` is an upstream release version like `5.22.0` or `v2.19.2`. For sporked containers it's a git SHA — the upstream commit the build is based on. This is a deliberate abuse of the naming convention to satisfy the `container-version-check` hook. Don't confuse it with an upstream release number. + +## Reproducibility guarantee + +Given the same `CONTAINER_APP_VERSION` and `FEATURES`, the build produces identical source code regardless of what `deploy`, `blumeops`, or `main` currently look like on forge. The only external dependency is the Rust/Boost toolchain version in the `FROM` line. + +## See also + +- [[create-a-spork]] — initial spork setup +- [[manage-spork-branches]] — feature branch workflow +- [[kingfisher]] — first sporked project diff --git a/docs/reference/services/kingfisher.md b/docs/reference/services/kingfisher.md index d6c5cf2..7512d6b 100644 --- a/docs/reference/services/kingfisher.md +++ b/docs/reference/services/kingfisher.md @@ -16,7 +16,7 @@ Secret detection and live validation scanner for Forgejo repositories, using Mon | Property | Value | |----------|-------| | **Namespace** | `kingfisher` | -| **Image** | `ghcr.io/mongodb/kingfisher` (see `argocd/manifests/kingfisher/kustomization.yaml` for current tag) | +| **Image** | `registry.ops.eblu.me/blumeops/kingfisher` (see `argocd/manifests/kingfisher/kustomization.yaml` for current tag) | | **Schedule** | Sunday 4am (after Prowler k8s scan at 3am) | | **Reports** | `sifaka:/volume1/reports/kingfisher/` (NFS) | | **Manifests** | `argocd/manifests/kingfisher/` | @@ -24,7 +24,7 @@ Secret detection and live validation scanner for Forgejo repositories, using Mon ## What it does -Runs as a weekly CronJob that scans all repositories in the `eblume` user on Forgejo for leaked secrets, API keys, and credentials. Produces timestamped HTML and JSON reports on the sifaka NFS share. +Runs as a weekly CronJob that scans all Forgejo repos (eblume + all orgs) for leaked secrets, API keys, and credentials. Produces timestamped HTML reports on the sifaka NFS share. Uses `--clone-url-base` to route git clones via the internal tailnet instead of the public Fly.io proxy. Uses the Forgejo/Gitea API to enumerate repos, then clones and scans each one. Validation is enabled (secrets are tested against their respective APIs to confirm they're live). Reports are HTML only. @@ -46,7 +46,7 @@ kubectl logs -f job/kingfisher-manual -n kingfisher --context=minikube-indri ## Limitations -- Clone URLs come from Forgejo's API response using the instance's public `ROOT_URL` (`forge.eblu.me`), so clones roundtrip through Fly.io. Mirror/org scanning is excluded for now to avoid unnecessary external bandwidth. A clone URL rewrite option would need an upstream contribution. +- Built from a [[spork-strategy|sporked]] fork with a local `--clone-url-base` patch. See [[build-spork-container]] for the build process. - Only one output format per invocation. Currently producing HTML only. ## See also