Deploy JobSync — job search tracker on ringtail k3s (#288)

## Summary

C2 Mikado chain to deploy [JobSync](https://github.com/Gsync/jobsync) — a self-hosted job application tracker — to ringtail's k3s cluster.

### Mikado Graph

```
deploy-jobsync (goal)
├── build-jobsync-container
│   └── mirror-jobsync
└── integrate-jobsync-ollama
```

### What is JobSync?

Next.js app with SQLite for tracking job applications. Features resume management, application pipeline tracking, and AI-powered resume review/job matching.

### Key Decisions

- **Ringtail k3s** (not minikube-indri) — colocates with Ollama for zero-latency AI
- **Nix container** via `buildLayeredImage` — no Dockerfile, mirrors upstream source on forge
- **Ollama for AI** — uses existing deployment, no API keys needed for AI features
- **No upstream fork** — vanilla JobSync, Anthropic AI deferred to future work if needed

### Current Status

Planning phase — cards committed, ready for review before implementation begins.

Reviewed-on: #288

ansible/roles/caddy/defaults/main.yml

@@ -85,6 +85,9 @@ caddy_services:
   - name: ntfy
     host: "ntfy.{{ caddy_domain }}"
     backend: "https://ntfy.tail8d86e.ts.net"
+  - name: jobsync
+    host: "jobsync.{{ caddy_domain }}"
+    backend: "https://jobsync.tail8d86e.ts.net"
   - name: ollama
     host: "ollama.{{ caddy_domain }}"
     backend: "https://ollama.tail8d86e.ts.net"

argocd/apps/jobsync.yaml

@@ -0,0 +1,18 @@
+---
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: jobsync
+  namespace: argocd
+spec:
+  project: default
+  source:
+    repoURL: ssh://forgejo@forge.ops.eblu.me:2222/eblume/blumeops.git
+    targetRevision: main
+    path: argocd/manifests/jobsync
+  destination:
+    server: https://ringtail.tail8d86e.ts.net:6443
+    namespace: jobsync
+  syncPolicy:
+    syncOptions:
+      - CreateNamespace=true

argocd/manifests/jobsync/deployment.yaml

@@ -0,0 +1,73 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: jobsync
+  namespace: jobsync
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: jobsync
+  template:
+    metadata:
+      labels:
+        app: jobsync
+    spec:
+      containers:
+        - name: jobsync
+          image: blumeops/jobsync:kustomized
+          ports:
+            - containerPort: 3000
+              name: http
+          env:
+            - name: DATABASE_URL
+              value: "file:/data/dev.db"
+            - name: NEXTAUTH_URL
+              value: "https://jobsync.ops.eblu.me"
+            - name: AUTH_TRUST_HOST
+              value: "true"
+            - name: NEXT_TELEMETRY_DISABLED
+              value: "1"
+            - name: TZ
+              value: "America/Los_Angeles"
+            - name: OLLAMA_BASE_URL
+              value: "http://ollama.ollama.svc.cluster.local:11434"
+            - name: AUTH_SECRET
+              valueFrom:
+                secretKeyRef:
+                  name: jobsync-secrets
+                  key: auth_secret
+            - name: ENCRYPTION_KEY
+              valueFrom:
+                secretKeyRef:
+                  name: jobsync-secrets
+                  key: encryption_key
+          volumeMounts:
+            - name: data
+              mountPath: /data
+          resources:
+            requests:
+              memory: "256Mi"
+              cpu: "100m"
+            limits:
+              memory: "512Mi"
+              cpu: "500m"
+          livenessProbe:
+            httpGet:
+              path: /
+              port: 3000
+            initialDelaySeconds: 30
+            periodSeconds: 30
+          readinessProbe:
+            httpGet:
+              path: /
+              port: 3000
+            initialDelaySeconds: 10
+            periodSeconds: 10
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: jobsync-data

argocd/manifests/jobsync/external-secret.yaml

@@ -0,0 +1,23 @@
+---
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: jobsync-secrets
+  namespace: jobsync
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: onepassword-blumeops
+  target:
+    name: jobsync-secrets
+    creationPolicy: Owner
+  data:
+    - secretKey: auth_secret
+      remoteRef:
+        key: JobSync
+        property: auth_secret
+    - secretKey: encryption_key
+      remoteRef:
+        key: JobSync
+        property: encryption_key

argocd/manifests/jobsync/ingress-tailscale.yaml

@@ -0,0 +1,26 @@
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: jobsync-tailscale
+  namespace: jobsync
+  annotations:
+    tailscale.com/proxy-class: "default"
+    tailscale.com/proxy-group: "ingress"
+    gethomepage.dev/enabled: "true"
+    gethomepage.dev/name: "JobSync"
+    gethomepage.dev/group: "Apps"
+    gethomepage.dev/icon: "mdi-briefcase-search"
+    gethomepage.dev/description: "Job application tracker"
+    gethomepage.dev/href: "https://jobsync.ops.eblu.me"
+    gethomepage.dev/pod-selector: "app=jobsync"
+spec:
+  ingressClassName: tailscale
+  defaultBackend:
+    service:
+      name: jobsync
+      port:
+        number: 3000
+  tls:
+    - hosts:
+        - jobsync

argocd/manifests/jobsync/kustomization.yaml

@@ -0,0 +1,15 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: jobsync
+resources:
+  - pvc.yaml
+  - external-secret.yaml
+  - deployment.yaml
+  - service.yaml
+  - ingress-tailscale.yaml
+
+images:
+  - name: blumeops/jobsync
+    newName: registry.ops.eblu.me/blumeops/jobsync
+    newTag: "v1.1.4-e51ec83-nix"

argocd/manifests/jobsync/pvc.yaml

@@ -0,0 +1,13 @@
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: jobsync-data
+  namespace: jobsync
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: local-path
+  resources:
+    requests:
+      storage: 5Gi

argocd/manifests/jobsync/service.yaml

@@ -0,0 +1,13 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: jobsync
+  namespace: jobsync
+spec:
+  selector:
+    app: jobsync
+  ports:
+    - name: http
+      port: 3000
+      targetPort: 3000

containers/jobsync/default.nix

@@ -0,0 +1,126 @@
+# Nix-built JobSync container
+# Next.js job application tracker with Prisma/SQLite
+# Built with dockerTools.buildLayeredImage for efficient layer caching
+{ pkgs ? import <nixpkgs> { } }:
+
+let
+  version = "1.1.4";
+
+  prismaEngines = pkgs.prisma-engines;
+
+  src = pkgs.fetchgit {
+    url = "https://forge.ops.eblu.me/mirrors/jobsync.git";
+    rev = "v${version}";
+    hash = "sha256-59W5OF36yD67jEK5xa9jSL4EVN9RG+Ez/w9Mq2VykSA=";
+  };
+
+  jobsync = pkgs.buildNpmPackage {
+    inherit src version;
+    pname = "jobsync";
+    npmDepsHash = "sha256-yRNOxtz66qSlmfjR3QDPUQe0C8sdg06tBbuK1Ws1gEA=";
+
+    nodejs = pkgs.nodejs_20;
+
+    # Patch out Google Fonts import (nix sandbox blocks network access at
+    # build time). Replace with a simple object; app uses system sans-serif.
+    postPatch = ''
+      substituteInPlace src/app/layout.tsx \
+        --replace-fail 'import { Inter } from "next/font/google";' "" \
+        --replace-fail 'const inter = Inter({
+  subsets: ["latin"],
+  variable: "--font-inter",
+});' 'const inter = { variable: "" };'
+    '';
+
+    # Point Prisma at nixpkgs-built engines (no network download in sandbox)
+    env = {
+      PRISMA_QUERY_ENGINE_LIBRARY = "${prismaEngines}/lib/libquery_engine.node";
+      PRISMA_QUERY_ENGINE_BINARY = "${prismaEngines}/bin/query-engine";
+      PRISMA_SCHEMA_ENGINE_BINARY = "${prismaEngines}/bin/schema-engine";
+      PRISMA_FMT_BINARY = "${prismaEngines}/bin/prisma-fmt";
+      PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING = "1";
+      DATABASE_URL = "file:/tmp/build.db";
+      NEXT_TELEMETRY_DISABLED = "1";
+    };
+
+    buildPhase = ''
+      runHook preBuild
+
+      # Generate Prisma client using nixpkgs engines
+      npx prisma generate
+
+      # Build Next.js
+      npm run build
+
+      runHook postBuild
+    '';
+
+    installPhase = ''
+      runHook preInstall
+
+      mkdir -p $out/app
+
+      # Copy Next.js standalone output
+      cp -r .next/standalone/. $out/app/
+      cp -r .next/static $out/app/.next/static
+      cp -r public $out/app/public
+
+      # Copy Prisma schema and migrations for runtime migrate deploy
+      cp -r prisma $out/app/prisma
+
+      # Copy entrypoint
+      cp ${./entrypoint.sh} $out/app/entrypoint.sh
+
+      runHook postInstall
+    '';
+
+    dontNpmBuild = true;
+  };
+
+  entrypoint = pkgs.writeShellScript "jobsync-entrypoint" ''
+    cd ${jobsync}/app
+    exec ${pkgs.bash}/bin/bash entrypoint.sh "$@"
+  '';
+in
+
+pkgs.dockerTools.buildLayeredImage {
+  name = "blumeops/jobsync";
+  tag = "latest";
+
+  contents = [
+    jobsync
+    prismaEngines
+    pkgs.nodejs_20
+    pkgs.cacert
+    pkgs.tzdata
+    pkgs.bash
+    pkgs.coreutils
+  ];
+
+  # Create writable directories and FHS symlinks for nix container
+  extraCommands = ''
+    mkdir -p tmp data usr/bin
+    ln -s ${pkgs.coreutils}/bin/env usr/bin/env
+  '';
+
+  config = {
+    Entrypoint = [ "${entrypoint}" ];
+    WorkingDir = "${jobsync}/app";
+    Env = [
+      "SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt"
+      "TZDIR=${pkgs.tzdata}/share/zoneinfo"
+      "NODE_ENV=production"
+      "PORT=3000"
+      "DATABASE_URL=file:/data/dev.db"
+      "PRISMA_QUERY_ENGINE_LIBRARY=${prismaEngines}/lib/libquery_engine.node"
+      "PRISMA_SCHEMA_ENGINE_BINARY=${prismaEngines}/bin/schema-engine"
+      "PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1"
+    ];
+    ExposedPorts = {
+      "3000/tcp" = { };
+    };
+    Volumes = {
+      "/data" = { };
+    };
+  };
+}

containers/jobsync/entrypoint.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+set -e
+
+# Auto-generate AUTH_SECRET if not provided
+if [ -z "$AUTH_SECRET" ]; then
+  AUTH_SECRET="$(node -e "console.log(require('crypto').randomBytes(32).toString('base64'))")"
+  export AUTH_SECRET
+  echo "AUTH_SECRET was not set — generated a temporary secret for this container."
+fi
+
+# Run Prisma migrations (npx -y downloads prisma if not in local node_modules)
+npx -y prisma@6.19.0 migrate deploy
+
+# Start the Next.js server
+exec node server.js

docs/changelog.d/mikado-jobsync.feature.md

@@ -0,0 +1 @@
+Deploy JobSync to ringtail k3s — nix-built container, Tailscale Ingress, Caddy route at `jobsync.ops.eblu.me`, Ollama integration for AI features.

docs/how-to/how-to.md

@@ -88,6 +88,11 @@ tags:
 
 - [[upgrade-dagger]]
 
+## JobSync
+
+- [[deploy-jobsync]]
+- [[build-jobsync-container]]
+
 ## Forgejo Runner
 
 - [[upgrade-k8s-runner]]

docs/how-to/jobsync/build-jobsync-container.md

@@ -0,0 +1,61 @@
+---
+title: Build JobSync Container
+modified: 2026-03-08
+tags:
+  - how-to
+  - jobsync
+  - nix
+---
+
+# Build JobSync Container
+
+Build and release the JobSync nix container image.
+
+```fish
+mise run container-release jobsync 1.1.4
+```
+
+The derivation is at `containers/jobsync/default.nix`. It uses `buildNpmPackage` for the Next.js app and `dockerTools.buildLayeredImage` for the container. The entrypoint (`containers/jobsync/entrypoint.sh`) runs `prisma migrate deploy` then starts `node server.js`.
+
+## Upgrading JobSync
+
+1. Update the forge mirror: `mise run mirror-sync jobsync`
+2. Update `version` in `default.nix` to match the new upstream tag
+3. Clear `hash` in `fetchgit` (set to `""`), build, grab the correct hash from the error
+4. Clear `npmDepsHash` (set to `""`), build again, grab the correct hash
+5. Check if `postPatch` still applies — the Google Fonts import may change between versions
+6. `mise run container-release jobsync <new-version>`
+7. Update `newTag` in `argocd/manifests/jobsync/kustomization.yaml`
+
+## Nix + Prisma + Next.js Pitfalls
+
+### Prisma engine downloads blocked by sandbox
+
+Prisma tries to download platform-specific engine binaries during `prisma generate`. The nix sandbox blocks network access at build time.
+
+**Fix:** Use `pkgs.prisma-engines` from nixpkgs and set env vars pointing at the nix store paths: `PRISMA_QUERY_ENGINE_LIBRARY`, `PRISMA_QUERY_ENGINE_BINARY`, `PRISMA_SCHEMA_ENGINE_BINARY`, `PRISMA_FMT_BINARY`. Set `PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1` to tolerate minor version mismatch.
+
+### Google Fonts blocked by sandbox
+
+`next/font/google` fetches from `fonts.googleapis.com` during `next build`.
+
+**Fix:** Patch `src/app/layout.tsx` in `postPatch` to replace the Google font import with a no-op object. The app falls back to system sans-serif.
+
+### Missing FHS paths in nix containers
+
+Nix containers lack `/usr/bin/env`, `/tmp`, etc. `npx`-downloaded packages use `#!/usr/bin/env node` shebangs.
+
+**Fix:** In `extraCommands`: `mkdir -p tmp data usr/bin` and `ln -s ${pkgs.coreutils}/bin/env usr/bin/env`.
+
+### Runtime migrations via npx
+
+The nix sandbox blocks network at build time, but runtime has full network access. Use `npx -y prisma@<version> migrate deploy` in the entrypoint — npx downloads the prisma CLI on first run.
+
+### Build on ringtail, not via Dagger
+
+The Dagger `build-nix` pipeline runs in host architecture. On macOS (arm64), this produces arm64 images. Build on ringtail (x86_64) using the CI workflow or `mise run container-release`.
+
+## Related
+
+- [[deploy-jobsync]]
+- [[build-container-image]]

docs/how-to/jobsync/deploy-jobsync.md

@@ -0,0 +1,60 @@
+---
+title: Deploy JobSync
+modified: 2026-03-08
+tags:
+  - how-to
+  - jobsync
+---
+
+# Deploy JobSync
+
+[JobSync](https://github.com/Gsync/jobsync) is a self-hosted job application tracker (Next.js + Prisma/SQLite) running on ringtail's k3s cluster via ArgoCD.
+
+- **URL:** `https://jobsync.ops.eblu.me`
+- **Auth:** Local accounts (email/password), no SSO
+- **Storage:** 5Gi PVC at `/data` (SQLite DB + resume uploads)
+- **AI:** Ollama at `ollama.ollama.svc.cluster.local:11434`
+
+## Manifests
+
+All in `argocd/manifests/jobsync/`:
+
+| File | Purpose |
+|------|---------|
+| `deployment.yaml` | Single-replica deployment |
+| `service.yaml` | ClusterIP on port 3000 |
+| `ingress-tailscale.yaml` | Tailscale Ingress (ProxyGroup) |
+| `pvc.yaml` | 5Gi local-path for `/data` |
+| `external-secret.yaml` | `auth_secret` + `encryption_key` from 1Password |
+| `kustomization.yaml` | Image tag override |
+
+## Environment Variables
+
+| Variable | Source | Purpose |
+|----------|--------|---------|
+| `DATABASE_URL` | Hardcoded | `file:/data/dev.db` |
+| `AUTH_SECRET` | ExternalSecret | NextAuth session signing |
+| `ENCRYPTION_KEY` | ExternalSecret | AES-256-GCM for stored API keys |
+| `NEXTAUTH_URL` | Hardcoded | `https://jobsync.ops.eblu.me` |
+| `AUTH_TRUST_HOST` | Hardcoded | `true` |
+| `TZ` | Hardcoded | `America/Los_Angeles` |
+| `OLLAMA_BASE_URL` | Hardcoded | `http://ollama.ollama.svc.cluster.local:11434` |
+
+## Updating the Container
+
+1. Build and push: `mise run container-release jobsync <version>`
+2. Update `newTag` in `kustomization.yaml` to the full tag (e.g. `v1.1.4-e51ec83-nix`)
+3. Sync: `argocd app sync jobsync`
+
+See [[build-jobsync-container]] for nix build details.
+
+## Notes
+
+- **1Password item:** "JobSync" in blumeops vault, fields `auth_secret` and `encryption_key`
+- **Caddy route:** `jobsync.ops.eblu.me` → `https://jobsync.tail8d86e.ts.net` (in `ansible/roles/caddy/defaults/main.yml`)
+- **`service-versions.yaml`:** Must have a `jobsync` entry or the pre-commit hook rejects container changes
+
+## Related
+
+- [[build-jobsync-container]]
+- [[deploy-k8s-service]]

service-versions.yaml

@@ -155,6 +155,13 @@ services:
     current-version: "2026.2.0"
     upstream-source: https://github.com/goauthentik/authentik/releases
 
+  - name: jobsync
+    type: argocd
+    last-reviewed: null
+    current-version: "1.1.4"
+    upstream-source: https://github.com/Gsync/jobsync/releases
+    notes: Job application tracker; nix container on ringtail k3s
+
   - name: ollama
     type: argocd
     last-reviewed: "2026-03-02"