Add k3s, 1Password Connect, and systemd nix-container-builder to ringtail (#209)

## Summary

  Extends ringtail from a desktop/gaming NixOS box into an infrastructure node with a k3s cluster, secrets management, and a Forgejo Actions
  runner for building containers with Nix.

  ### K3s cluster
  - Single-node k3s with Traefik/ServiceLB/metrics-server disabled (minimal footprint)
  - TLS SAN set to `ringtail.tail8d86e.ts.net` so ArgoCD on indri can manage it via Tailscale
  - Containerd registry mirrors pull through Zot on indri (`k3s-registries.yaml`)
  - Tailscale interface added to `trustedInterfaces` for cross-node ArgoCD access
  - `kubectl` added to system packages

  ### 1Password Connect + External Secrets Operator
  - Four new ArgoCD apps targeting `k3s-ringtail`: `1password-connect-ringtail`, `external-secrets-crds-ringtail`, `external-secrets-ringtail`,
  `external-secrets-config-ringtail`
  - Reuses the same Helm charts/values as indri, just pointed at ringtail's k3s API server
  - Bootstrap secrets (`op-credentials`, `onepassword-token`) provisioned by Ansible pre_tasks via `op read`, then applied to the `1password`
  namespace in post_tasks

  ### Systemd Forgejo Actions runner
  - Native `services.gitea-actions-runner` with `forgejo-runner` package — no DinD, no k8s pod, runs directly on the NixOS host
  - Label `nix-container-builder:host` — jobs execute on the host with `nix`, `skopeo`, `nodejs`, etc. in PATH
  - Registration token fetched from 1Password (`Forgejo Secrets/runner_reg`) by Ansible and written to `/etc/forgejo-runner/token.env`
  - Runner's dynamic user (`gitea-runner`) added to `nix.settings.trusted-users` for nix daemon access

  ### Nix container build workflow
  - New `.forgejo/workflows/build-container-nix.yaml` triggers on `*-nix-v[0-9]*` tags (e.g. `nettest-nix-v1.0.0`)
  - Builds with `nix build -f containers/<name>/default.nix`, pushes to Zot via `skopeo copy`
  - Existing Dockerfile workflow guarded with `if: !contains(github.ref_name, '-nix-v')` to avoid double-triggering

  ### Mise task updates
  - `container-tag-and-release` auto-detects `default.nix` vs `Dockerfile` and uses the appropriate tag format (`-nix-v` vs `-v`)
  - `container-list` shows build type indicator (`[nix]` / `[dockerfile]`)

  ## Post-merge

  1. `mise run provision-ringtail` — deploys k3s token, runner token, NixOS rebuild
  2. Register k3s cluster in ArgoCD (first time only):
     ```fish
     ssh ringtail 'sudo cat /etc/rancher/k3s/k3s.yaml' | \
       sed 's|127.0.0.1|ringtail.tail8d86e.ts.net|' > /tmp/k3s-ringtail.yaml
     set -x KUBECONFIG /tmp/k3s-ringtail.yaml
     argocd cluster add default --name k3s-ringtail
  3. Sync ArgoCD apps in order: 1password-connect-ringtail -> external-secrets-crds-ringtail -> external-secrets-ringtail ->
  external-secrets-config-ringtail
  4. Verify runner: ssh ringtail 'systemctl status gitea-runner-nix-container-builder'
  5. Check Forgejo admin panel for ringtail-nix-builder runner online
  6. Test: create containers/<name>/default.nix, tag with <name>-nix-v0.1.0

Reviewed-on: https://forge.ops.eblu.me/eblume/blumeops/pulls/209
This commit is contained in:
Erich Blume 2026-02-18 21:15:30 -08:00
commit 918df9e642
16 changed files with 499 additions and 17 deletions

View file

@ -96,12 +96,32 @@ in
dedicatedServer.openFirewall = true;
};
# K3s single-node cluster
services.k3s = {
enable = true;
role = "server";
tokenFile = "/etc/k3s/token";
extraFlags = toString [
"--disable=traefik"
"--disable=servicelb"
"--disable=metrics-server"
"--write-kubeconfig-mode=644"
"--tls-san=ringtail.tail8d86e.ts.net"
];
};
# K3s containerd registry mirrors (pull through Zot on indri)
environment.etc."rancher/k3s/registries.yaml".source = ./k3s-registries.yaml;
# Tailscale
services.tailscale = {
enable = true;
extraUpFlags = [ "--accept-routes" "--ssh" ];
};
# Trust Tailscale interface (ArgoCD on indri connects via tailnet)
networking.firewall.trustedInterfaces = [ "tailscale0" ];
# SSH
services.openssh = {
enable = true;
@ -124,6 +144,7 @@ in
# System packages
environment.systemPackages = with pkgs; [
git
kubectl
python3 # required for Ansible
vim
htop
@ -369,9 +390,35 @@ in
"d /mnt/storage2 0755 eblume users -"
];
# Forgejo Actions runner (nix container builder)
services.gitea-actions-runner = {
package = pkgs.forgejo-runner;
instances.nix_container_builder = {
enable = true;
name = "ringtail-nix-builder";
url = "https://forge.ops.eblu.me";
tokenFile = "/etc/forgejo-runner/token.env";
labels = [ "nix-container-builder:host" ];
hostPackages = with pkgs; [
bash coreutils curl gawk gitMinimal gnused nodejs wget
nix skopeo
];
settings = {
log.level = "info";
runner = {
capacity = 1;
timeout = "3h";
};
};
};
};
# Enable nix flakes
nix.settings.experimental-features = [ "nix-command" "flakes" ];
# Allow the runner's dynamic user to access the nix daemon
nix.settings.trusted-users = [ "gitea-runner" ];
# NixOS release
system.stateVersion = "25.11";
}