Set up hephaestus from template and add design + tech spec
Some checks failed
Build / validate (push) Failing after 2s

Customize the generated repo (rename Dagger module to hephaestus_ci /
HephaestusCi, set docs baseUrl, add All-Rights-Reserved LICENSE, update
README/AGENTS), and add the project's foundational design documentation:

- docs/explanation/design.md — rationale + decision-history record
- docs/reference/tech-spec.md — implementation-ready technical spec

These define hephaestus as a self-hosted, client/server + offline-first
system unifying a markdown knowledge base with task management: typed node
graph, the lived priority discipline ("what is next?"), recurrence with
fresh-per-occurrence checklists, op-log/CRDT sync with conflict resolution,
OIDC/Authentik auth, the heph.nvim surface, and a TDD strategy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Erich Blume 2026-05-31 09:37:28 -07:00
commit cbf859b2d7
14 changed files with 768 additions and 36 deletions

View file

@ -0,0 +1,3 @@
"""Hephaestus CI — Dagger build functions."""
from .main import HephaestusCi as HephaestusCi

View file

@ -2,9 +2,8 @@ import dagger
from dagger import dag, function, object_type
# TODO: Rename class to match your project (also rename the src/ directory)
@object_type
class ProjectTemplateCi:
class HephaestusCi:
@function
async def build_docs(self, src: dagger.Directory, version: str) -> dagger.File:
"""Build Quartz docs site. Returns docs tarball."""

View file

@ -1,4 +0,0 @@
"""Hephaestus CI — Dagger build functions."""
# TODO: Rename class to match your project (also rename the src/ directory)
from .main import ProjectTemplateCi as ProjectTemplateCi

View file

@ -8,13 +8,16 @@ Guidance for Claude Code working in this repository. See also [[ai-assistance-gu
## First-Time Setup
This repository is a **Forgejo template**. Most customization points are auto-resolved by Forgejo's template variable expansion when a new repo is created. The remaining manual steps are:
This repository was instantiated from the `project-template` Forgejo template. Setup status:
1. **Set `baseUrl`** in `docs/quartz.config.ts` — this is the hosted docs domain, not the repo URL. localhost if not hosted.
2. **Rename `.dagger/src/project_template_ci/`** directory to match your project, and update the class names inside
3. **Fill in project structure** in the `AGENTS.md` Project Structure section
4. **Fill in license info** in `README.md`
5. **When all TODOs are resolved:** delete this "First-Time Setup" section entirely from `AGENTS.md` — it is only needed once
- [x] `baseUrl` in `docs/quartz.config.ts` set to `localhost` (update once docs are hosted)
- [x] Dagger module renamed to `.dagger/src/hephaestus_ci/` (`HephaestusCi` class)
- [ ] Fill in the Project Structure section below once the Rust workspace is scaffolded
- [ ] Fill in license info in `README.md`
Delete this section once the remaining items are resolved.
> **This is now a generated repo, not the template source.** C1/C2 changes use feature branches + PRs (`tea pr create`); noteworthy changes get changelog fragments in `docs/changelog.d/`.
## Rules
@ -52,7 +55,7 @@ See [[agent-change-process]] for the full methodology.
```
./docs/ # Diataxis docs, Quartz config, and release content
./docs/changelog.d/ # keep only .gitkeep in the template; generated repos add towncrier fragments here
./.dagger/ # Dagger module; rename src/project_template_ci/ when forking
./.dagger/ # Dagger module (src/hephaestus_ci/) backing docs builds and releases
./.forgejo/workflows/ # generic build and release workflows for generated repos
./.forgejo/scripts/ # optional per-project build/release hooks consumed by the workflows
./mise-tasks/ # repo automation via `mise run`

18
LICENSE Normal file
View file

@ -0,0 +1,18 @@
All Rights Reserved
Copyright (c) 2026 Erich Blume
This software and its associated documentation and source files (the
"Software") are the private, proprietary work of the copyright holder.
No license, express or implied, is granted to any party. You may not use,
copy, modify, merge, publish, distribute, sublicense, or sell copies of the
Software, in whole or in part, without the prior written permission of the
copyright holder.
The Software is provided "as is", without warranty of any kind, express or
implied, including but not limited to the warranties of merchantability,
fitness for a particular purpose, and noninfringement.
Open-source release under a permissive license may be considered in the
future, at the sole discretion of the copyright holder.

View file

@ -1,6 +1,8 @@
# project-template
# hephaestus
A personal project template with opinionated infrastructure for documentation, CI, and AI-assisted development.
A personal context management system — a unified, self-hosted application that fuses a wiki-style knowledge base (Zettelkasten) with task management. Built in Rust, offline-capable, and syncs to a central instance hosted in blumeops.
See [the project design document](docs/explanation/design.md) for goals, architecture, and the development roadmap.
## What's Included
@ -11,20 +13,6 @@ A personal project template with opinionated infrastructure for documentation, C
- **AI assistance**`AGENTS.md` + structured docs for Claude Code (C0/C1/C2 change process, Mikado method)
- **Task runner** — [mise](https://mise.jdx.dev/) tasks for docs validation, Mikado chain management, release preview, and runner inspection
## Forking This Template
This is a **Forgejo template repository**. When you create a new repo from this template, Forgejo automatically expands variables like `${REPO_NAME}` and `${REPO_OWNER}` in key files — handling most customization automatically.
After creating your repo, the remaining manual steps are:
1. Set `baseUrl` in `docs/quartz.config.ts` to your docs site domain
2. Rename `.dagger/src/project_template_ci/` directory and update class names to match your project
3. Review and tailor the project structure section in `AGENTS.md`
4. Add license information to `README.md`
5. Remove the "First-Time Setup" section from `AGENTS.md` and this section from `README.md`
If you use Claude Code, it will prompt you to resolve remaining TODOs at the start of your first session.
## Getting Started
```bash
@ -54,4 +42,4 @@ dagger call build-docs --src=. --version=dev export --path=./docs-dev.tar.gz
## License
<!-- TODO: Add license information -->
All rights reserved. This is a personal, private project — not licensed for use, copying, modification, or distribution. Open-sourcing may be considered in the future. See [LICENSE](LICENSE).

View file

@ -0,0 +1 @@
Add the project design document (`docs/explanation/design.md`, rationale + decision history) and the distilled technical specification (`docs/reference/tech-spec.md`, the build artifact) defining hephaestus as a unified, self-hosted, **client/server + offline-first** task + knowledge-base system: typed node graph, the lived priority discipline ("what is next?"), recurrence with fresh-per-occurrence checklists, op-log/CRDT sync with conflict resolution, OIDC/Authentik auth, the heph.nvim surface, and a test-driven development strategy.

View file

@ -0,0 +1 @@
Customize the repository from the `project-template`: rename the Dagger module to `hephaestus_ci` (`HephaestusCi`), set the docs `baseUrl`, add an All-Rights-Reserved `LICENSE`, and update `README.md`/`AGENTS.md` for the hephaestus project.

419
docs/explanation/design.md Normal file
View file

@ -0,0 +1,419 @@
---
title: Design Document
modified: 2026-05-31
tags:
- explanation
- design
---
# Hephaestus — Design Document
> **Status:** Living design record. This is the **rationale + decision-history** document. For the clean, implementation-facing spec the next session builds from, see **[[tech-spec]]**. Sections marked **❓ OPEN** are unresolved; **🔒 DECIDED** are settled.
## 1. Purpose & Vision
**Hephaestus** is a personal context management system that fuses two systems the owner already relies on into a single, purpose-built, self-hosted application:
1. **The Zettelkasten** (`~/code/personal/zk`) — an Obsidian vault of ~600 markdown files acting as a personal wiki, journal, knowledge base, and "task-context" store for long-running projects. Notes use `[[wiki-links]]`, YAML frontmatter (`id`, `aliases`, `tags`), timestamp-based note IDs (`1722897441-MWFE`), and daily notes (`YYYY-MM-DD`).
2. **Todoist** — the owner's primary task manager, accessed today via a bespoke Python mise-task in blumeops (`mise-tasks/blumeops-tasks`) that polls the Todoist v1 API (projects → tasks; fields: `content`, `description`, `priority`, `due`) using a token from 1Password.
Today these are loosely coupled by fragile cross-links (`todoist://<task-id>` in notes, `[[note]]` text in Todoist comments). Hephaestus replaces the fragile coupling with **one unified data model and database** where tasks and knowledge are first-class, linkable entities.
### Guiding principles
- **Context-aware:** A task like "Fix the roof leak" is coupled to the relevant knowledge — the home-repair log, the log of contractor calls, etc. Surfacing that context is a core feature, not an afterthought.
- **Workflow-optimized:** Highly optimized, concise answers to recurring questions — above all **"What is next?"** — are the primary UX. Speed and concision beat feature breadth.
- **Owner-built, owner-hosted:** A standalone codebase replacing a loose stack of tools, hosted out of blumeops.
## 2. Goals & Non-Goals
### Goals (v1 / prototype)
- 🔒 Unified data model linking **notes** and **tasks** as first-class, cross-linkable entities.
- 🔒 SQLite backend.
- 🔒 Rust implementation.
- 🔒 **Distributed dual-mode operation:** useful fully offline, auto-syncs to a central self-hosted instance in blumeops (k3s container) when reachable. The central instance may be unavailable for long periods.
- 🔒 Surfaces over a shared core (via the `hephd` daemon): **CLI** (everyday driver), **nvim plugin** (primary editing UX), **web UI** (hub-hosted).
- 🔒 Optimized workflow queries, especially "What is next?", with context surfacing.
- 🔒 **Auth from the ground up** (OIDC/Authentik), with isolated multi-user accounts — sensitive data.
### Non-Goals (v1) / Later
- ⏳ iOS app + Apple Watch app for quick voice-dispatched task capture — explicitly a **follow-up goal**, not v1.
- ⏳ Sharing/collaboration between users — accounts are isolated for v1 (multi-user *isolation* is in scope; multi-user *collaboration* is not).
- ❌ Obsidian feature parity (canvas, graph view, plugins) — clean break.
- ❌ **Migration / import / Todoist sync in v1** — the prototype is a **clean break**: heph is the system of record from day one. No ZK import, no Todoist bridge. (Migration may return as a later, optional phase.)
- ❌ Encryption at rest / E2E in v1 — security model is **access restriction (OIDC auth) only**; plain SQLite. May revisit.
- ❌ Inferred/semantic context surfacing in v1 — explicit links + search only.
## 3. Data Model
> 🔒 **DECIDED (shape).** SQLite is the canonical store; a node's body is plain markdown text; an `export` mode materializes the whole store as a directory of `.md` files. Clean break from Obsidian-the-app, but markdown stays the portable payload. Tasks are **thin**; rich context lives in documents; everything is connected by typed links.
### Entities
- **Node** — the base entity. Every first-class thing is a node with a stable, sync-safe ID (see §3.1). A node has a `kind` discriminator. Kinds (initial set):
- **`doc`** — a rich markdown context document (the bulk of the knowledge base; replaces ZK articles, journals, logs). Body = markdown text. Has `aliases`, timestamps.
- **`task`** — **thin**: ID + state, plus a few scalar attributes (attention-state, do-date, late-on, recurrence) and links to a project/tags/context. **No prose body of its own** — context is supplied by linked `doc` nodes (or a doc *is* the work-log for the task). This is the key unification: ZK-articles-with-todos and Todoist-tasks-with-context both reduce to *typed links between thin tasks and rich docs*. The full task-attribute model is in §6.2 (it encodes the owner's actual lived prioritization system).
- **`project`** — a grouping node (maps to Todoist projects / ZK project dirs like `payrix/`).
- **`tag`** — a label node (so tags are linkable/queryable, not just strings).
- **`journal`** — a daily-note node keyed by date (`YYYY-MM-DD`).
- **Link** — a typed, directional edge between two nodes. Types (initial set): `wiki` (a `[[link]]` found in a doc body), `context-of` (doc ↔ task), `blocks`, `parent`/`child`, `tagged`, `in-project`. Wiki-links in markdown bodies are parsed and materialized as `wiki` links so the graph and the prose stay consistent.
### 3.1 Identity
- ❓ **Leaning ULID** (sortable, sync-safe, collision-free across offline devices) as the internal primary key, with a human-facing `slug`/`alias` layer on top so `[[wiki-links]]` can be written by name. The ZK's timestamp-IDs (`1722897441-MWFE`) are ULID-like already; a migration can map them.
### 3.2 Scalar task attributes vs. links
**Tags and projects are nodes** (linkable, queryable). **Attention-state, do-date, late-on, recurrence are scalar columns** on the task. See §6.2 for the semantics — these columns are not arbitrary; they encode a specific, battle-tested model. Due dates are *not* first-class nodes (a task has one project/context; "this Friday" doesn't need its own context).
### 3.3 Recurrence model (recurring tasks & checklists) — IN SCOPE for v1
A **recurring task** carries an **RFC-5545 RRULE** and acts as a recurring **definition**.
> ❌ **The anti-pattern we must avoid (Todoist's mistake):** modeling a recurring task's checklist sub-items as standalone, non-recurring tasks that **carry their completion state forward** when the parent recurs. This is the corner; designing around it now is *why* recurring checklists are in v1 rather than deferred.
**heph model (proposed — one point to ratify):**
- The recurring definition holds a **checklist template** (the ordered set of sub-item definitions).
- Each **occurrence** materializes a **fresh checklist instance** — brand-new context items, all `outstanding` — scoped to that occurrence. **Completion state is per-occurrence and never carries forward.**
- Occurrences are retained as history (no hard deletes; ties naturally to the per-task **log**, §6.4). The "what is next?" engine surfaces the **current/active occurrence**; completing it advances to the next per the RRULE.
**The one fork to confirm:**
- **(a) Occurrence instances (recommended):** the definition spawns a lightweight **instance** per occurrence (its own do-date + its own fresh checklist items). Full per-occurrence history; unambiguously "a fresh instance per recurrence"; heavier (more rows).
- **(b) Roll-forward in place:** a single task node; on completion, **log the finished occurrence** (to the per-task log) and **reset the checklist to outstanding** + advance the do-date. Lighter; history is narrative (in the log) rather than queryable per-occurrence.
Both satisfy the correctness requirement (fresh checklist, no carried state). (a) is recommended for clean history and zero ambiguity; (b) is the pragmatic-lite option.
## 4. Architecture
> 🔒 **DECIDED (shape).**
Layers, top to bottom:
- **Surfaces (thin clients):** the **nvim plugin (`heph.nvim`)** is the **primary surface for v1** — a full "org-mode"-style experience (markdown buffers backed by `doc` nodes; agenda / "what is next" / capture / linking / journaling as plugin commands). It is the explicit **successor to obsidian.nvim**, which the owner currently drives the ZK with (telescope picker, `<Leader>o*` verbs, `<Enter>` to follow `[[wiki-links]]`, dailies, multi-state checkboxes) and rarely uses Obsidian-proper alongside. heph.nvim must reach that feature parity (see §6.5) and replace it. The **CLI** (`heph`) is a secondary/utility surface (export, scripting, admin) — explicitly *not* a focus, though it shares the same command surface. The **web UI** is the occasional hub-served surface. A later iOS/Watch client talks to the hub directly.
- ❓ **Dual-mode binary:** a single Rust binary that can run as the **server/daemon** *or* expose the same operations as **CLI subcommands** *or* act as the **backend the nvim plugin drives**. Appealing (one codebase, one command surface) — to confirm as the packaging model.
- **Per-device daemon (`hephd`):** owns the local SQLite handle, the CRDT/op-log state, and background sync. All local surfaces connect to it over a local socket. This is what makes multi-surface access concurrent and safe on one SQLite file, and gives one place to run background sync.
- **Core crate (Rust lib):** data model, query engine, markdown parsing + wiki-link extraction, and sync logic. Linked by both `hephd` and the hub server.
- **Hub:** `hephd` running in "server" mode on blumeops k3s — central SQLite, the sync rendezvous point, and the web UI host. May be offline for long periods; devices keep working and reconcile when it returns.
Open points:
- ❓ nvim ↔ daemon transport: JSON-RPC over a unix socket (nvim has a native RPC channel; allows the daemon to *push* updates when sync brings in remote changes) vs. the plugin shelling out to the `heph` CLI. Leaning socket RPC.
- ❓ Web front-end style: server-rendered (Axum + HTMX/Askama) vs. WASM SPA (Leptos/Dioxus). Leaning **SSR + HTMX** since it's the lowest-traffic surface — favor simplicity.
## 5. Sync & Conflict Resolution
### Requirements (gathered from the owner)
- 🔒 ~99% of interaction today is via the **CLI on Gilbert** (dev laptop). Future split: ~90% Gilbert, ~9% phone/watch, ~1% web.
- 🔒 Must work **seamlessly across tailnet workstations** (Gilbert, ringtail, others): dispatch a change on one, walk to another, see it arrive "in short order."
- 🔒 **Genuine offline conflicts will occur** — e.g. Gilbert edits offline, then ringtail edits, then Gilbert reconnects. Must resolve **gracefully**: auto-merge where possible; for the unresolvable remainder, a **conflict queue + alert** ("you have N merge conflicts, run `heph conflicts`").
- 🔒 **Auth from the ground up** — this is extremely sensitive data. Use **Authentik (OIDC)** as IdP, support **multiple isolated user accounts** (in practice just `eblume`).
- 🔒 A **web UI** is part of the hosted (hub) model.
### Proposed model — *for confirmation*
A **hub-and-spoke, op-log + CRDT** design:
- **Topology:** each device runs `hephd` with a full local SQLite replica; the blumeops **hub** is the rendezvous. Devices push/pull an **append-only operation log** to the hub when reachable. Hub-and-spoke (not P2P mesh) — simpler, matches "central self-hosted app," and the hub is normally up on the tailnet so propagation is prompt. When the hub is down, devices keep working and the op-log drains on reconnect.
- **Merge semantics (so "graceful" is precise):**
- **`doc` bodies (markdown):** a **text CRDT** (e.g. Yrs/Automerge) → concurrent edits *always* merge with no hard conflict. This covers the common case.
- **Scalar task fields (completion, due, priority):** **last-writer-wins via Hybrid Logical Clocks**. Deterministic and cheap.
- **Links / set membership (tags, projects):** add/remove as a CRDT set (OR-Set) → no conflicts.
- **Conflict queue:** because CRDTs auto-merge, true blocking conflicts are rare. We *surface* (not block) the cases that are semantically ambiguous: a discarded LWW value on a meaningful field (e.g. completion flipped both ways, divergent due dates), or concurrent edits to overlapping text regions. These go to `heph conflicts` with an alert; sync never stalls.
- **Identity & ordering:** ULIDs for nodes (§3.1); HLC timestamps stamp every op for causal ordering across offline devices.
- **Auth:** the hub's sync + web endpoints require an **OIDC bearer token from Authentik**. Devices obtain tokens via the **OAuth device-code flow**; refresh token stored in the OS keychain / 1Password. Every node/op is owned by a `user_id`; the hub enforces per-user isolation. Offline devices operate on cached credentials.
### Open points
- ❓ **Hub-only vs. P2P fallback:** if the hub is down but Gilbert and ringtail are both on the tailnet, do they sync *directly*, or wait for the hub? Hub-only is simpler; direct P2P is more available. (Leaning hub-only for v1.)
- ❓ **Encryption at rest:** given "extremely sensitive," should each device's SQLite (and the hub's) be encrypted at rest (e.g. SQLCipher), and/or should op-log payloads be end-to-end encrypted so the hub stores ciphertext it can't read? (E2E complicates server-side search/web UI.)
- ❓ **Todoist's role** during/after migration — two-way bridge during transition, import-once, or retire on cutover?
- ❓ How "short order" is short — is a few seconds of propagation (push-based) needed, or is periodic pull (e.g. every 3060s) fine?
## 6. Workflow Features
> Driven by "highly optimized specific workflows." heph is organized around a small set of **workflows**, each a first-class surface in heph.nvim:
>
> - **Task / "What is next?"** (§6.1§6.3) — the Tactical / Strategic / Organizational task engine. The flagship.
> - **Log** (§6.4) — daily journal + optional append-only per-task work-logs ("narrative > list").
> - **Wiki / KB** (§6.5) — the whole context corpus as a browsable/queryable personal wiki; the direct **obsidian.nvim replacement**.
> - **Calendar** (§6.6) — entering via the calendar, tied to Strategic mode. *Deferred, but scaffolded carefully.*
### "What is next?" — the flagship
> ❓ **ACTIVE DEEP-DIVE.** This is the defining workflow and has ~10 years of the owner's prior art and prior tool attempts behind it. Being designed from that prior art rather than from scratch — see §6.1. The blumeops-tasks ranking (overdue-first, then priority, p3 = backlog) is a known reference point, not the target.
### Context surfacing — v1 scope
> 🔒 **DECIDED for v1.** Explicit links only; no inferred/semantic context yet.
- Every **task** automatically gets **one canonical `doc` (context) node**, created and linked on task creation. That canonical doc is the jumping-off point and can wiki-link onward to other docs/cards.
- nvim commands provide **telescope/fzf/rg-style search** over context docs, optionally with preloaded filters (e.g. scoped to a project).
- Inferred context (shared tags, full-text/semantic similarity) is deferred — large surface area, not the v1 focus.
### 6.1 "What is next?" — prior art & synthesis
> Distilled from ~10 years of the owner's thinking — predating the ZK by ~5 years — chiefly the owner's direct account (this design conversation), plus `zk/mole/20240202113646.todo.md` (the most developed written framework), daily logs (`zk/home/2023-09-01.log.md`, `2023-10-12.log.md`), and `zk/payrix/friction_log.md`. Prior tool code may exist at `github.com/eblume/{mole,hermes}` (check detached-head branches — several restarts); the owner expects little reusable there beyond what's captured here.
#### History: Hermes → Mole → heph (and the lessons)
- **Hermes — the "time accountant" (≈ earliest).** Goal: a *timeline*-based system fed "blueprints" of what a day should look like, with rules like *"check email twice a day, unless a high-priority email comes in — but even then no more than once an hour,"* continuously re-solving the calendar in real time. Built a working prototype using a **SAT solver (OR-tools)**. **Killer feature deduced: a quick, meaningful answer to "what is next" — here, the next thing on the calendar.** **Why it failed:** the calendar became a *battle zone*, swinging wildly every few minutes as the solver re-resolved the dependency graph → impossible to plan around. **Lesson + salvage:** there may be fruit here as a future, *opt-in* heph **planning mode**, but real-time auto-rescheduling of a visible calendar is anti-useful.
- **Mole — "whack-a-mole" rules engine.** A rules engine that constantly re-evaluated rules and materialized Todoist tasks (canonical first test: *"is there an active task 'whack the mole'? if not, make one"*). **"What is next" = "what Todoist says is available."** Worked reasonably. **Why it stalled:** Todoist accumulated **too many tasks**, so *choosing the actual next thing* became the whole problem again. This pushed the owner to design the Todoist project/priority/filter discipline in §6.2. **The deeper lesson:** Mole's automation **wasn't adding much value over the project/priority discipline itself** — so the owner *turned Mole off and just kept using Todoist*. The manual discipline was the real engine.
- **Today.** No automation beyond blumeops' **read-only** `blumeops-tasks` mise poller. Just Todoist + the §6.2 discipline — but as the direct descendant of all the above. **Implication for heph:** the value is in *embodying and strengthening the discipline* (and finally fixing the weak note↔task link), **not** in clever auto-scheduling or a rules engine. Earn any automation only after the discipline is faithfully supported.
#### "What's next?" is three questions, not one
The owner's framework splits the question by **mode**:
- **Tactical** — blank-slate "what do I do *right now*?" Stay in the editor/terminal; capture interruptions and resume in **milliseconds** ("save state and walk away in milliseconds"). This is the mode a single ranked list serves.
- **Strategic** — "I want to do X — what's the *next step*, or *where was I*?" Exploring a goal, generating the sub-tasks to reach it.
- **Organizational** — "I want to do X — help me *plan* the steps." Project-level enumeration of what needs doing, not how.
🔒 **REAFFIRMED.** The owner still strongly identifies with this split (the earlier "maybe" was conflating it with the unrelated CLI-first *process* note on the same page). The clean operational test, in the owner's words: **"looking at a project's list of todos = Organizational; deciding which to work on = Strategic; working a todo = Tactical."** Escalation flows upward (project-infra work can become Strategic, then Tactical).
**How heph embodies it** — as **named modes/views in the nvim plugin**, *not* an inference engine that guesses your mode:
- **Tactical (execution view):** you are "tactically engaging" a specific committed task. Loads its canonical context + sub-items into a **goal stack**; millisecond interrupt-capture pushes new context items onto the stack; "smallest amount of contextual guidance and next steps to stay productive." Includes **Chore Mode** (synthesize a goal stack from a query — e.g. the `Chores` project — and chug through one by one).
- **Strategic (planning view):** editor present but the goal is to *explore/decompose*, not edit. "To do X I must also do Y and Z." Produces a fleshed-out task with ready sub-items so Tactical can grab and go. **When Strategic completes, no work has been done** — only new/changed tasks, docs, links.
- **Organizational (survey view):** project-level enumeration — *what* needs doing, not *how*.
- **Shared primitive — the goal stack** (see §6.3) spans Tactical and Strategic; transitions between modes are first-class (Strategic *produces* → Tactical *consumes*).
#### The real cost is resumption, not ranking
The recurring pain in the logs is **reorienting after a context switch**, not deciding raw priority. Design rules that follow:
- **Resumption hint invariant:** when work pauses on a task/todo, it should leave at least one concrete next-step so future-you can re-enter quickly. heph should make capturing/“leaving a breadcrumb” frictionless and surface it first on return.
- **Narrative > list:** daily logs proved more motivating and contextual than flat task lists ("a log is the way"). Journaling/log nodes are first-class, and "what is next" should be able to lean on recent log narrative, not just task metadata.
#### The "weak link" is heph's reason to exist
The #1 unsolved problem: the **fragile, clunky coupling between Todoist and notes** (manually typing `nb o 45` into task descriptions). heph's thin-task ↔ auto-created canonical-context-doc model is the direct fix: the link is structural, not a hand-typed string.
#### Mapping prior taxonomy → heph entities
| Prior (`nb`/Todoist) | Meaning | heph |
|---|---|---|
| nb-project | grouping; completing all members completes it | `project` node |
| nb-todo | planning complex multi-step work; has "a single unified context log" | `task` + its **canonical context `doc`** |
| nb-task | immediate, short-lived "might eventually be done"; checking off = "no longer might be done" (not necessarily *completed*) | thin `task` node |
| Todoist | "deciding what is next and knowing what needs to be done now" | heph's "what is next?" engine |
| Calendar | scheduling / future | due dates (+ later calendar integration) |
> 🔒 **Preserved.** The nb-task semantic — checking off means "no longer outstanding," *not* "accomplished" — carries into heph. **Context items are `outstanding` / `not-outstanding`** (see §6.3). For **committed tasks**, heph distinguishes end-states: **done** (accomplished) vs. **dropped/dismissed** (let go, e.g. during a Blue review). Both are "not outstanding"; the distinction is kept for honesty and history.
#### Process steer (now historical)
The owner's old rule — "avoid ncurses and interactive UIs; write atomic code and call it manually with subcommands… until a strong pattern emerges" — was a *focus device for hand-building tools*. It no longer constrains heph: we go **straight to the nvim org-mode plugin backed by the Rust server** (the patterns are now well understood; see §6.2/§6.3). The CLI remains as a secondary/utility surface.
#### Distilled requirements for heph's "what is next?" — *to confirm/refine*
1. **Mode-aware** (Tactical / Strategic / Organizational), not a single global sort.
2. **Sub-millisecond capture** and fast state-save (Tactical interrupt handling).
3. **Resumption-first:** on returning to a task, surface the last breadcrumb/next-step before anything else.
4. **Context-coupled:** every task one keystroke away from its canonical context doc and onward links.
5. **Log-aware:** can factor in / surface recent daily-log narrative.
6. **A sane default ranking** for the Tactical blank-slate case — the concrete model is §6.2, not the blumeops-tasks placeholder.
### 6.2 The lived priority discipline (the working basis for heph)
> 🔒 **This is the real model.** The owner has used this Todoist discipline daily for years; Mole was eventually *turned off* because this discipline outvalued the automation. heph should **embody and strengthen** it. Captured verbatim-in-spirit from the owner's account.
**Projects = contexts (a task has exactly one).** Examples: `Life` (catch-all), `Work`, `Chores`, `Maintenance`, `Outside`, `Allison` (wife), `Errands`, `Cooking`, `Health`. A project *contextualizes* a task; it is not a plan.
**Attention-state (thought of by color, not number):**
| State | Meaning |
|---|---|
| **White** (default) | Able to be done **once its do-date arrives**. (See do-date semantics below.) |
| **Orange** | **Top of mind.** Keep **≤ 6**, reassessed *every day*. Goal: you know your top-of-minds *without looking at the tool*. |
| **Red** | Top of mind **+ there will be a consequence if not done ASAP**. Red means **the existence of a consequence, NOT its severity/importance.** Flagging by *importance* is a trap → anxiety (everything due feels important). |
| **Blue** | **On Deck** (backlog). Doable, but deliberately *cooling off*. The pressure-relief valve that keeps the working set light. |
**Do-date, not due-date (key insight):** "due date" means **DO date***don't do it before this date*. Optionally a separate **"Late On"** marker for when it actually becomes a problem. **Due dates are a terrible planning tool** — calendar-style deadlines make everything "yell for attention" constantly. heph must treat the do-date as *earliest actionable*, not a deadline alarm.
**Working-set discipline (the healthy tensions — a feature, not a bug):**
- Keep White+Orange+Red ≤ **~30 total**, or the system collapses into noise. Keep Orange ≤ **6**.
- Push freely to Blue to stay light; periodically **review Blue: keep vs. let go** (this is where "drop/dismiss" happens). Target On-Deck **< 100** (owner is at ~149 and hates it).
- These tensions **map the owner's emotional state to the task system** and should be *surfaced honestly*. heph must avoid **two failure modes**: (1) **"fake happy"** — telling you everything's fine when it isn't; (2) **overwhelm** — yelling that you're buried until you freeze and make no progress. Reflect true state; help maintain the tensions.
**Filters = saved views (queries over the above):**
- **Top of Mind** — all Red + Orange.
- **Tasks / Work Tasks** — Red/Orange/White that are *actionable now* (do-date arrived), scoped by project.
- **Chores** — like Tasks but from `Chores`, *slightly tuned down* so the (near-universally recurring) chores don't become a nuisance.
- **On Deck** — all Blue.
- **Schedule** — recurring checklists from a few projects (e.g. *Morning Routine* → brush teeth, feed birds, coffee, medicine). The owner uses *this particular filter* sparingly, but the underlying **recurring-checklist mechanism is IN SCOPE for v1** (see §3.3): the only way to be sure we avoid Todoist's mistake is to model it up front. **Each recurrence gets a fresh checklist instance; completion never carries forward.**
**heph's default "what is next?" (Tactical blank-slate)** therefore ≈ Top of Mind (Red first — by *consequence*, then Orange) **+** White items whose do-date has arrived, scoped to the current project/context, **with Blue hidden**. Concise, honest, light.
### 6.3 Two kinds of task: commitments vs. context items
> 🔒 **DECIDED (shape).** A **commitment axis** orthogonal to the §6.2 attention-states.
- **Committed task** — long-lived, strongly tied to context (e.g. "Complete PLAT-1234", "Build a nursery"). **Participates in §6.2**: has an attention-state, a project, a do-date; is a candidate for "what is next"; may carry sub-structure and a web of context.
- **Context item** — ephemeral, created mid-flow ("six things before this is done", "oh crap, another" pushed on the stack). **Does *not* enter the §6.2 scheduling system** — it is *part of its containing task's context*, with only **outstanding / not-outstanding** state. Never appears in global "what is next."
- **Promotion** is first-class: a context item can be **promoted to a committed task** when you decide to actually schedule it (the old "nb-todo → Todoist-task on activation").
- **Ephemeral ≠ deleted:** deletes effectively never happen — tombstones + a dedicated cleanup mode only. "Ephemeral" is lifecycle/visibility, not storage. (This also keeps the CRDT sync simple.)
- **The goal stack** is a **session-scoped, synthesized construct**: seeded from a committed task (Tactical) or from a query over contexts (Chore Mode — "grind on some chores"). It holds committed tasks and/or context items for the duration of a work session.
#### Editing surface for context items — *deliberately deferred to the prototype*
> 🔑 **Key realization: this is NOT a data-model fork.** In both options below the **stored representation is identical** — context items are outstanding nodes linked to their container; the daemon owns them; markdown `export` renders them as checkboxes regardless. The choice is *purely the nvim editing affordance*, so the backend is shared and we **prototype both and pick by feel** (ergonomics can't be judged on paper).
- **Option A — checkboxes in the body, derived on save.** Edit the task's canonical context doc as plain markdown; `- [ ]` lines are materialized into context-item state on `:w` (`BufWritePost`), reusing the wiki-link extraction machinery. **No real-time scanning needed** (debounced live updates are optional polish). Remote CRDT edits are *pushed* from the daemon to reconcile an open buffer. Item identity is low-stakes (reword = tombstone-old + add-new) since items are ephemeral; identity is pinned only at **promotion**. Most "personal org-mode"-native; millisecond capture = "type a line."
- **Option B — structured items, rendered.** Items are nodes from the start; the body stays pure prose. Render inline via **virtual text/extmarks** (no pane) or a transient **floating window** (avoids the disliked persistent cutaway); capture via a leader chord + `vim.ui.input`. Cleaner separation and trivial promotion; heavier capture; more UI to build.
- Lean: **A** for capture flow, but **build the shared node backend first** and trial both affordances in the plugin.
> **Prior-art note (multi-state checkboxes):** the owner's obsidian.nvim config uses checkbox states `" ", "x", ">", "~", "!"` (todo / done / forwarded / ~ / important). heph's task/context-item state model should accommodate richer-than-binary states (at least the done/dropped/outstanding set in §6.1; possibly forwarded/deferred).
### 6.4 The Log workflow
> 🔒 **In scope for v1** (daily journal certainly; per-task logs as the model allows). Logs embody the "narrative > list" insight and the resumption-hint invariant from §6.1.
Two related shapes:
- **Daily journal.** Today driven by the `zkd` abbrev → `Obsidian dailies`: a telescope picker to create/open dated journal buffers. heph keeps this as a **first-class** feature: `journal` nodes keyed by date, a picker to jump to today / recent / create. The daily journal is the catch-all narrative stream.
- **Per-task log.** Sometimes a task wants an append-only "here's what I was working on" side-commentary. It **half-belongs to the task's context and half doesn't** — so model it as a **separate buffer/node from the task's canonical context doc**, optionally present only for tasks that need longer-duration commentary. Properties:
- **Append-only** (entries accrete; you don't rewrite history).
- **Dispatchable from an isolated context:** you must be able to fire a log entry *without leaving / losing your place in* the context buffer you're navigating — i.e. append to the log from elsewhere (a quick-capture into the task's log), while the context buffer stays where it is. (This is the ergonomic crux; likely a quick-capture command targeting "the current task's log".)
- The per-task log is a strong candidate to *be* the resumption breadcrumb store (§6.1).
❓ Open: is the per-task log a distinct node `kind` (`log`, append-only), or just a `doc` flagged append-only and linked `log-of` → task? Leaning a dedicated append-only `log` kind to make "dispatch an entry" a cheap, well-defined op.
### 6.5 The Wiki / KB workflow
> 🔒 **First-class in v1.** This is the "view all my contexts as one big personal wiki" surface — the direct replacement for obsidian.nvim.
The owner estimates ~90% of the value comes from a few well-worn primitives, so heph.nvim should reach **obsidian.nvim feature parity** on day one:
- **Follow `[[wiki-links]]` on `<Enter>`** to jump between context docs (the core navigation gesture).
- **Telescope-based** search (`rg`), quick-switch between docs, tag browse, **backlinks**, outgoing **links** — mirroring the owner's `<Leader>o*` verbs (to be rebound under heph's own prefix).
- **Logical layout + tagging** of context docs so the corpus is browsable/queryable as a whole.
- New-doc-from-query and insert-link from the picker (obsidian.nvim's `<C-x>`/`<C-l>`).
heph.nvim **replaces** obsidian.nvim outright — the owner already rarely opens Obsidian itself. This workflow is the knowledge-base analogue of the Organizational task view: surveying and traversing the `doc` graph.
### 6.6 The Calendar workflow
> ⏳ **Deferred (later version) — but scaffold now, don't paint ourselves into a corner.** Ties closely to **Strategic** mode (planning against time).
- The owner uses **Calendar.app** bound to a **Gmail calendar** (almost certainly **CalDAV**; Gmail also exposes ICS/WebCal feeds — to confirm when implemented).
- Eventual goal: **integration between calendar views and task views** — enter the system *via* the calendar, see tasks/do-dates in temporal context.
- ⚠️ **Hard-won gotcha:** enabling Todoist's gcal integration **flooded the calendar with a million events** because of recurring rules. heph must **never explode recurrence into individual calendar events**. This echoes the Hermes "battle zone" failure (§6.1) — calendar integration must be *read-mostly, careful, de-duplicated, and opt-in*, not a firehose.
- **Scaffolding guidance (decisions to keep clean now):** keep a crisp separation between *tasks/do-dates* and *calendar events*; model recurrence as RRULEs that are **expanded lazily for display, never materialized as stored events**; design sync so a future **read-only CalDAV ingestion** (calendar → heph) is easy, and treat any heph → calendar export as a later, explicitly bounded, non-exploding feature.
## 7. Hosting & Deployment
Reuse the established blumeops patterns (🔒 confirmed by repo conventions):
- Containerized service deployed to **k3s** (ringtail cluster) via **ArgoCD app-of-apps** + **Kustomize** raw manifests (no Helm/Flux).
- Secrets via **1Password + external-secrets operator** (`ClusterSecretStore onepassword-blumeops`).
- Image built via **Dagger + Forgejo Actions**, pushed to **Zot** registry (`registry.ops.eblu.me`).
- ❓ Rust build approach for the container: Nix (like kingfisher) vs. Dagger-native `cargo` multi-stage build.
- ❓ Persistence for the central instance's SQLite — NFS PVC (sifaka) vs. local PV. (SQLite over NFS has known locking caveats.)
- ❓ How do clients reach the central instance — Tailscale tailnet only, or public via Caddy/Fly proxy?
## 8. Technology Choices (to decide)
- ❓ nvim plugin stack: Lua + JSON-RPC over a socket to `hephd` (with daemon→plugin push) vs. CLI shell-out. Telescope as the picker dependency.
- ❓ Web framework (Axum + HTMX/Askama? Leptos? Dioxus?).
- ❓ SQLite access layer (sqlx, rusqlite, SeaORM, Diesel) and migrations.
- ❓ CRDT / sync library (Automerge-rs, Yrs, bespoke op-log).
- ❓ Markdown parser/renderer (pulldown-cmark, comrak) and wiki-link / checkbox extraction.
## 9. Open Questions Summary
### Decided
- ✅ Source of truth: **SQLite-primary**, markdown as the body payload, with a directory `export` mode. Clean break from Obsidian-the-app.
- ✅ Note/task model: **typed node graph** — thin `task` nodes, rich `doc` nodes, connected by typed links.
- ✅ Front-end: **per-device `hephd` daemon** with thin clients; **nvim plugin** as primary editing UX, **CLI** as everyday driver, **web UI** on the hub.
- ✅ Sync (shape): **hub-and-spoke, op-log + CRDT**, HLC ordering, OIDC/Authentik auth, conflict queue for the ambiguous remainder. (Details below still open.)
### Decided since (B/C/D)
- ✅ Context surfacing v1: **explicit links only** + auto-created canonical context doc per task + nvim fuzzy search. Inferred/semantic deferred.
- ✅ **v1 is distributed, not local-only:** client/server + **offline-first sync with automatic merge + conflict queue** + **full OIDC/Authentik auth with per-user isolation** are all IN v1 (tech-spec §12/§13). Web UI and k3s deployment are later (hub binary is built deployable).
- ✅ Security v1: **OIDC/Authentik auth + per-user isolation** is the boundary; plain SQLite at rest (no encryption). May revisit.
- ✅ Migration v1: **clean break**, no ZK import / no Todoist bridge. heph is system of record day one.
- ✅ **License: All Rights Reserved / private.** Open-sourcing considered later.
### Decided (workflows)
- ✅ heph organized around first-class workflows (§6): **Task** (what-is-next), **Log**, **Wiki/KB**, **Calendar**.
- ✅ **Log** (§6.4) in scope: daily journal (picker over dated `journal` nodes, à la `zkd`/`Obsidian dailies`) + optional **append-only per-task work-logs** (separate buffer from the canonical context doc; dispatchable without leaving the context buffer).
- ✅ **Wiki/KB** (§6.5) first-class: heph.nvim reaches **obsidian.nvim feature parity** (follow `[[links]]` on `<Enter>`, telescope search/quick-switch/tags/backlinks/links) and **replaces** it.
- ✅ **Calendar** (§6.6) deferred but **scaffolded carefully**: never explode recurrence into stored events (Todoist-gcal-flood / Hermes lesson); keep tasks vs. events separate; lazy RRULE expansion; design for a future read-only CalDAV ingest.
### Decided (workflow model)
- ✅ "What is next?" rests on the **lived priority discipline** (§6.2): projects-as-contexts; attention-states White/Orange/Red/Blue (Red = *consequence*, not severity); **do-date not due-date**; working-set tensions surfaced honestly (avoid "fake happy" *and* overwhelm).
- ✅ **Tactical / Strategic / Organizational** reaffirmed as **named modes/views** in the nvim plugin, sharing a **goal-stack** primitive (§6.1).
- ✅ **Commitment axis** (§6.3): committed tasks (in §6.2 system) vs. ephemeral context items (outstanding/not-outstanding, scoped to a container); **promotion** is first-class; **no hard deletes** (tombstones + cleanup mode).
- ✅ Task end-states: **done** vs. **dropped/dismissed** (both "not outstanding").
- ✅ Prototype goes **straight to the nvim plugin + Rust server**; CLI is secondary.
- ✅ **Rust stack RATIFIED** (§11.2): rusqlite + tokio/JSON-RPC + pulldown-cmark + ulid + rrule + clap.
- ✅ **Recurrence + recurring checklists IN SCOPE for v1** (§3.3): RRULE-driven; **each occurrence gets a fresh checklist instance, completion never carries forward** (avoids Todoist's corner).
### Still open
*Prototype-blocking (resolve at kickoff — see §11 / tech-spec):*
1. **Recurrence model (§3.3): (a) occurrence-instances vs (b) roll-forward-in-place** — both avoid the corner; leaning (a).
2. **CRDT library** for body merge: `yrs` (leaning) vs `automerge`; bespoke op-log/HLC for scalar fields either way.
3. **Hub network transport** (tech-spec §6.1): `axum` HTTP/JSON (leaning) vs gRPC; sync propagation cadence (push vs periodic pull).
4. **Context-item editing surface (§6.3): Option A vs B***decided inside the prototype by feel* (shared backend).
*Not prototype-blocking (later phases):*
5. Per-task **log** representation: dedicated append-only `log` node kind vs. `doc` flagged append-only (§6.4).
6. Calendar protocol confirmation (CalDAV vs ICS/WebCal) and the careful, non-exploding integration (§6.6).
7. Web front-end style (SSR + HTMX vs. WASM SPA).
8. k3s deployment specifics: Rust container build (Nix vs Dagger-native cargo); central SQLite persistence (local PV vs NFS); client reachability (tailnet-only vs public).
9. Encryption at rest / E2E (currently out; may revisit).
## 10. Roadmap (provisional)
- **Phase 0 — Design** (this document + [[tech-spec]]): done enough to build.
- **Phase 1 — v1 prototype (one C1 effort, built in TDD slices):** `heph-core` (model, schema, extraction, recurrence, "what is next", op-log/HLC/CRDT merge) → `hephd` client mode → **hub mode + offline sync + conflict queue****OIDC/Authentik auth + per-user isolation**`heph.nvim` + `heph` CLI. Runnable client/server, offline-capable, on the tailnet.
- **Phase 2 — k3s deployment:** Dagger→Zot image, ArgoCD app + Kustomize manifests, external-secrets; hub on blumeops.
- **Phase 3 — Web UI** on the hub.
- **Phase 4 (later, optional)** — calendar integration (careful CalDAV); migration from ZK / Todoist; iOS / Apple Watch voice capture; Hermes-style planning mode.
## 11. v1 Prototype — Scope, Stack, and Kickoff
> The handoff target for the next context session. Items here are **proposals to ratify**; once ratified, a fresh session can build directly from this.
> **The canonical, implementation-ready scope/stack/schema/API now live in [[tech-spec]].** This section keeps the *intent* and the kickoff process; defer to the spec for details (and update the spec when decisions change).
### 11.1 Scope — *distributed from day one* (ratified)
v1 is **not** local-only. It is a **client/server, offline-first** system:
- **In:** the full data model (§3, §6) incl. **recurrence + recurring checklists** (§3.3); the "what is next?" engine; **client/server architecture** (per-device client `hephd` + a runnable/deployable hub `hephd`); **offline-first op-log + CRDT sync with automatic merge and a conflict queue** (tech-spec §12); **OIDC/Authentik auth with per-user isolation** (tech-spec §13); the `heph.nvim` + `heph` CLI surfaces.
- **Out (later, scaffolded):** the **web UI** (hub serves sync only in v1); **actual k3s deployment** to blumeops (hub binary is built deployable; ArgoCD/Kustomize/Dagger is a fast-follow); calendar; iOS/Watch; inferred context; P2P-over-tailnet fallback; persistent goal stack (an in-plugin session stack suffices).
**Why this bigger v1:** the distributed/offline/merge behavior and the auth model are *architecturally load-bearing* — bolting them on later would mean reworking the core. The owner explicitly wants them proven from the start. (Recurrence-with-checklists is in for the same reason — see §3.3.)
### 11.2 Stack — ✅ RATIFIED
Core: `rusqlite` (bundled) + migrations · `tokio` + JSON-RPC/unix-socket (local) · `ulid` · `rrule` · `pulldown-cmark` · `clap` · `anyhow`/`thiserror` · `tracing`. Distributed/auth additions (some to confirm at kickoff): text CRDT `yrs` (vs `automerge`) for bodies + bespoke op-log/HLC for fields · hub transport `axum`/`reqwest` · OIDC via `openidconnect` + `keyring`. Full detail and the SQLite schema: **tech-spec §4.5, §10, §12, §13**.
### 11.3 First-session kickoff checklist
1. `mise run ai-docs`; read **[[tech-spec]]** (the build spec) and this doc's §3/§6 for rationale.
2. **Classify as C1** — greenfield, no existing system to Mikado-untangle. Single long-lived feature branch + early draft PR, docs-first, push as you go ([[agent-change-process]]).
3. Scaffold the cargo workspace + `heph.nvim` skeleton; **fill the AGENTS.md Project Structure section** (last template TODO).
4. Build outward in testable slices (TDD, tech-spec §2/§9): `heph-core` (schema, model, extraction, recurrence, "what is next", **op-log/HLC/CRDT merge**) → `hephd` client mode (local RPC) → **hub mode + sync + conflict queue****OIDC auth**`heph.nvim` + `heph` CLI.
5. Confirm the kickoff-open picks: recurrence model (a vs b, §3.3), CRDT lib (`yrs` vs `automerge`), hub transport, context-item editing surface (A vs B, §6.3).
## Related
- [[tech-spec]] — clean implementation-facing technical specification distilled from this document
- [[ai-assistance-guide]] — conventions for AI agents in this repo
- [[agent-change-process]] — C0/C1/C2 change methodology

View file

@ -10,4 +10,4 @@ tags:
Background context and design decisions.
<!-- TODO: Add explanation entries as the project grows -->
- [[design]] — Hephaestus design document: vision, data model, architecture, sync, and roadmap

View file

@ -13,7 +13,7 @@ const config: QuartzConfig = {
enablePopovers: true,
analytics: null,
locale: "en-US",
baseUrl: "CHANGEME.example.com", // TODO: Update to your docs site URL
baseUrl: "localhost", // TODO: set to hosted docs domain once published (see blumeops docs.eblu.me pattern)
ignorePatterns: ["private", "templates", ".obsidian"],
defaultDateType: "modified",
theme: {

View file

@ -10,11 +10,15 @@ tags:
Technical reference material for the repository tooling that ships with this project.
## Project
- [[tech-spec]] — Hephaestus technical specification (data model, RPC API, "what is next?" ranking, recurrence, testing strategy, v1 scope)
## Template Surface Area
| Path | Purpose |
|------|---------|
| `.dagger/src/project_template_ci/` | Dagger module that builds the Quartz docs tarball used by releases |
| `.dagger/src/hephaestus_ci/` | Dagger module that builds the Quartz docs tarball used by releases |
| `.forgejo/workflows/build.yaml` | Generic CI validation workflow |
| `.forgejo/workflows/release.yaml` | Manual release workflow that versions, builds docs, and publishes release assets |
| `.forgejo/scripts/` | Optional project-specific hooks consumed by the workflows |
@ -64,5 +68,4 @@ Technical reference material for the repository tooling that ships with this pro
## TODO After Templating
- TODO: Set `baseUrl` in `docs/quartz.config.ts` to the hosted docs domain, or `localhost` if the docs are only previewed locally
- TODO: Rename `.dagger/src/project_template_ci/` and the exported Dagger class to match the generated project
- TODO: Set `baseUrl` in `docs/quartz.config.ts` to the hosted docs domain once published (currently `localhost`)

301
docs/reference/tech-spec.md Normal file
View file

@ -0,0 +1,301 @@
---
title: Technical Specification
modified: 2026-05-31
tags:
- reference
- design
---
# Hephaestus — Technical Specification
> Clean, implementation-facing spec for the v1 prototype. For the *why* behind every choice (history, prior art, decision trail), see [[design]]. Where this spec and the design doc disagree, the design doc's latest decision wins — file an update here.
## 1. Overview
Hephaestus (heph) is a self-hosted personal context-management system that unifies a markdown knowledge base with task management in one database. **v1 is a distributed client/server system**: each device runs a local replica that works fully **offline**, and syncs to a central **hub** when reachable, with **automatic merge + conflict resolution** for concurrent offline edits. Access is authenticated via **OIDC (Authentik)** with per-user data isolation. The web UI and the actual k3s deployment are later phases; the hub ships in v1 as a runnable/deployable binary. Components:
- **`heph-core`** — Rust library: data model, SQLite store, query engine, markdown parsing/extraction, recurrence, and the sync engine (op-log, HLC, CRDT merge, conflict detection).
- **`hephd`** — Rust daemon with two modes:
- **client mode** (per device): owns the local SQLite replica; serves a JSON-RPC API over a unix socket to local surfaces; runs background sync to the hub.
- **server/hub mode**: owns the central SQLite; serves the authenticated sync endpoint (and, later, the web UI) over the network.
- **`heph`** — Rust CLI: utility/admin surface (export, scripting, smoke tests, `heph conflicts`).
- **`heph.nvim`** — Lua Neovim plugin: the primary user surface ("org-mode"-style); a thin client of the local `hephd`.
## 2. Development approach
**Development is test-driven (TDD).** Write the failing test first; implement to green; refactor. No feature is "done" without tests at the appropriate layer(s) in §9. Core logic must be **deterministic and clock-injected** (no ambient wall-clock reads in `heph-core`; the current time is always passed in) so ranking and recurrence are testable.
## 3. Architecture
- Surfaces (`heph.nvim`, `heph` CLI) are thin clients; they never touch SQLite directly. All state operations go through the local `hephd` over the unix socket. Socket path default: `$XDG_RUNTIME_DIR/heph/hephd.sock`.
- Each device's `hephd` (client mode) is the single writer/owner of that device's local SQLite replica (WAL mode), and works **fully offline**. It records every local mutation as an op in an append-only **op-log** and runs **background sync** to the hub.
- The **hub** `hephd` (server mode) is the **hub-and-spoke rendezvous**: devices push/pull ops to it over the network (authenticated, §13). It is *not* required to be online — devices keep working and reconcile when it returns.
- Merge is automatic where possible; the unresolvable remainder goes to a **conflict queue** surfaced to the user (§12). Sync never blocks local work.
- `heph-core` is synchronous and side-effect-light (incl. deterministic merge logic); `hephd` wraps it with async I/O, transport, and auth (`tokio`). DB calls run on a blocking pool.
- See **§12 (Sync & Conflict Resolution)** and **§13 (Authentication)** for the detailed models.
## 4. Data model
All first-class entities are **nodes**; relationships are **links**. Markdown bodies are stored in SQLite; files are an export artifact, not the source of truth.
### 4.1 Node kinds
| kind | meaning | body |
|---|---|---|
| `doc` | rich context document (knowledge base, work-logs, journals) | markdown |
| `task` | thin task or ephemeral context item (see §4.3) | none (context via links) |
| `project` | grouping/context for tasks | optional |
| `tag` | label | optional |
| `journal` | daily note, titled by ISO date | markdown |
### 4.2 Link types
`wiki` (materialized from `[[links]]` in a body), `canonical-context` (task → its auto-created context doc), `context-of`, `log-of` (task → its append-only log), `blocks`, `parent`, `tagged`, `in-project`.
### 4.3 Task semantics
- **Attention-state** (required on committed tasks): `white` (do once do-date arrives), `orange` (top of mind), `red` (top of mind + a consequence exists if late — *consequence, not severity*), `blue` (on-deck/backlog).
- **do-date** = *earliest actionable date* ("do date"), **not a deadline**. Optional **late-on** marks when lateness becomes a problem.
- **Commitment axis:** `committed = 1` tasks participate in scheduling/"what is next"; `committed = 0` are **ephemeral context items** scoped to a container (`container_id`), with only `outstanding`/done states, never surfaced globally. Context items may be **promoted** to committed tasks.
- **States:** `outstanding`, `done`, `dropped` (done and dropped are both "not outstanding"; the distinction is retained).
- **No hard deletes:** everything uses `tombstoned`; physical deletion only in an explicit cleanup mode.
### 4.4 Recurrence (§3.3 of [[design]])
A `task` with a non-null `recurrence` (RFC-5545 RRULE) is a **recurring definition**. Sub-items flagged `is_template = 1` form its **checklist template**. **Each occurrence produces a fresh checklist instance** (new `outstanding` context items copied from the template); **completion never carries forward across occurrences** — this is a hard requirement.
Two candidate implementations (pick at kickoff; (a) is the lean):
- **(a) Occurrence instances:** definition spawns a `task_occurrences` row per occurrence, each with its own do-date and fresh checklist items. Full history.
- **(b) Roll-forward in place:** single node; on completion, log the occurrence, reset the checklist to `outstanding`, advance the do-date.
### 4.5 SQLite schema (starting point)
```
nodes(
id TEXT PRIMARY KEY, -- ULID
owner_id TEXT NOT NULL REFERENCES users(id), -- per-user isolation
kind TEXT NOT NULL, -- doc|task|project|tag|journal
title TEXT NOT NULL,
body TEXT, -- markdown (nullable); materialized view of body_crdt
body_crdt BLOB, -- text-CRDT state for the body (merge), nullable
created_at INTEGER NOT NULL, -- epoch ms
modified_at INTEGER NOT NULL,
hlc TEXT NOT NULL, -- hybrid logical clock of last write (sync ordering)
tombstoned INTEGER NOT NULL DEFAULT 0
)
tasks(
node_id TEXT PRIMARY KEY REFERENCES nodes(id),
attention TEXT, -- white|orange|red|blue (committed tasks)
do_date INTEGER, -- epoch ms, nullable
late_on INTEGER, -- epoch ms, nullable
state TEXT NOT NULL, -- outstanding|done|dropped
committed INTEGER NOT NULL, -- 1 committed task, 0 context item
container_id TEXT REFERENCES nodes(id), -- context item → container
recurrence TEXT, -- RRULE; present = recurring definition
is_template INTEGER NOT NULL DEFAULT 0 -- checklist-template item
)
-- recurrence model (a) only:
task_occurrences(
id TEXT PRIMARY KEY, -- ULID
def_id TEXT NOT NULL REFERENCES nodes(id),
occurrence_date INTEGER NOT NULL,
state TEXT NOT NULL, -- outstanding|done|dropped|skipped
created_at INTEGER NOT NULL,
tombstoned INTEGER NOT NULL DEFAULT 0
)
links(
id TEXT PRIMARY KEY, -- ULID
src_id TEXT NOT NULL REFERENCES nodes(id),
dst_id TEXT NOT NULL REFERENCES nodes(id),
type TEXT NOT NULL,
created_at INTEGER NOT NULL,
tombstoned INTEGER NOT NULL DEFAULT 0
)
aliases(node_id TEXT REFERENCES nodes(id), alias TEXT) -- wiki-link name resolution
nodes_fts -- FTS5 over title, body
-- identity & sync --
users(
id TEXT PRIMARY KEY, -- ULID
oidc_sub TEXT UNIQUE NOT NULL, -- OIDC subject (Authentik)
name TEXT,
created_at INTEGER NOT NULL
)
oplog( -- append-only operation log (the sync unit)
id TEXT PRIMARY KEY, -- ULID
owner_id TEXT NOT NULL REFERENCES users(id),
hlc TEXT NOT NULL, -- hybrid logical clock (causal order)
origin TEXT NOT NULL, -- originating device id
op_type TEXT NOT NULL, -- node.create|node.body_delta|task.set_field|link.add|link.remove|...
target_id TEXT NOT NULL,
payload TEXT NOT NULL, -- JSON (e.g. CRDT delta, field+value, OR-Set add/remove)
applied INTEGER NOT NULL DEFAULT 0
)
sync_state( -- per-peer cursor (device ↔ hub)
peer TEXT PRIMARY KEY, -- 'hub' on a client; device id on the hub
last_pushed_hlc TEXT,
last_pulled_hlc TEXT,
updated_at INTEGER NOT NULL
)
conflicts( -- ambiguous merges surfaced to the user
id TEXT PRIMARY KEY,
owner_id TEXT NOT NULL REFERENCES users(id),
node_id TEXT NOT NULL REFERENCES nodes(id),
field TEXT NOT NULL, -- which field / 'body-region'
local_val TEXT, remote_val TEXT,
local_hlc TEXT, remote_hlc TEXT,
status TEXT NOT NULL, -- open|resolved
created_at INTEGER NOT NULL
)
```
Projects/tags are `nodes`; membership is `links` (`in-project`, `tagged`). All `tasks`/`task_occurrences`/`links` rows inherit ownership via their node(s).
## 5. Markdown handling
- Bodies are stored verbatim. On write (node create/update), `heph-core` extracts:
- `[[wiki-links]]``wiki` links (resolved via `aliases`/title; unresolved links are allowed and recorded).
- GFM task-list items (`- [ ]` / `- [x]`) → context-item state under the node (Option A editing model — see [[design]] §6.3; the alternative is command-driven items, decided in-prototype).
- Extraction is idempotent and diff-based: re-writing an unchanged body is a no-op; reworded checklist lines tombstone-old + add-new (context items are cheap).
- `export` materializes all non-tombstoned nodes to a directory tree of `.md` files (frontmatter + body), reproducing the corpus portably.
## 6. Daemon RPC API (JSON-RPC over unix socket)
Methods (request → response; errors are JSON-RPC errors). Signatures are indicative, not final:
- `node.get(id) → Node`
- `node.create({kind, title, body?}) → Node`
- `node.update({id, title?, body?}) → Node` (body update re-runs extraction)
- `node.tombstone(id) → ok`
- `task.create({title, project?, attention?, do_date?, late_on?, recurrence?, committed?}) → Task` (auto-creates the canonical context `doc` + `canonical-context` link)
- `task.set_state({id, state}) → Task` (recurring: advances per §4.4)
- `task.set_attention({id, attention}) → Task`
- `task.promote({context_item_id, attention?, project?}) → Task`
- `next({scope?, limit?}) → [RankedTask]` (the "what is next?" query, §7)
- `search({query, filters?}) → [Node]` (FTS)
- `links.outgoing(id) → [Link]` / `links.backlinks(id) → [Link]`
- `journal.open_or_create(date) → Node`
- `log.append({task_id, text}) → ok` (append to the task's `log-of` node)
- `export({path}) → {count}`
- `health() → {orange_count, active_count, on_deck_count, conflict_count, sync_status, ...}` (working-set + sync indicators)
- **auth:** `auth.login() → {device_code_flow...}` / `auth.status() → {user, logged_in}` / `auth.logout()` (§13)
- **sync:** `sync.now() → {pushed, pulled, conflicts}` (force a sync cycle; background sync runs automatically) / `sync.status() → {last_pushed, last_pulled, pending_ops, online}`
- **conflicts:** `conflicts.list() → [Conflict]` / `conflicts.resolve({id, choice}) → ok`
Every local mutation method records an op in the **op-log** for sync. The local daemon supports **server-push** notifications (e.g. a node changed by an incoming sync) so an open buffer can reconcile in real time.
### 6.1 Hub (server-mode) sync endpoint
Separate from the local unix-socket RPC, the hub exposes an **authenticated network endpoint** (HTTP/JSON or gRPC — pick at kickoff) for op exchange: clients present an OIDC bearer token (§13); the hub validates, scopes by `owner_id`, accepts pushed ops, and returns ops the client hasn't seen (by HLC cursor). The hub applies the same `heph-core` merge logic.
## 7. "What is next?" ranking
Given an optional `scope` (project/context) and `limit` (default 5):
1. **Candidates:** committed tasks where `state = outstanding`, not tombstoned, `attention ≠ blue`, and actionable now: `do_date IS NULL OR do_date ≤ now`. For recurring definitions, evaluate the current active occurrence's do-date. Apply `scope` if given.
2. **Order:**
1. attention: `red``orange``white`;
2. urgency: tasks past `late_on` first, then most-overdue (smallest `do_date`) first;
3. tie-break: earlier `do_date`, then `created_at`.
3. **Output:** concise rows — title, project, attention, do/late, link to canonical context. `red` items always appear regardless of `limit`.
`blue` (on-deck) is hidden from `next` by design; surfaced only by an explicit on-deck view. `health()` exposes the working-set tensions (orange ≤ 6, active ≤ ~30, on-deck count) honestly — never masking overload nor manufacturing calm.
## 8. heph.nvim surface (v1)
Replaces obsidian.nvim. Telescope-backed. Core commands/gestures:
- Follow `[[wiki-link]]` under cursor on `<Enter>`.
- Search / quick-switch / tags / backlinks / outgoing links (pickers).
- Daily journal picker (create/open dated `journal` nodes).
- Task capture; show "what is next" (`:Heph next`); set attention; mark done/dropped.
- Open a task's canonical context doc; edit context-item checkboxes (Option A) in the buffer (extracted on `:w`).
- Per-task log quick-append without leaving the current buffer.
## 9. Testing strategy (TDD, layered)
All layers are required; CI runs them on every push/PR (extend `.forgejo/scripts/build` to run `cargo test` and the nvim e2e suite; `prek` already runs in `build.yaml`).
- **Unit (`heph-core`):** model invariants; markdown extraction (wiki-links, checkboxes); **RRULE expansion and the fresh-checklist-per-occurrence rule** (assert completion never carries forward); the "what is next?" ranking (table-driven cases); migration up/down.
- **Property tests (`proptest`):** ranking yields a total order; extraction is idempotent; recurrence never leaks completion state across occurrences; tombstones are never resurrected.
- **Integration (`hephd`, real sockets):** start a daemon against a **temp SQLite file**, connect over a **real unix socket**, and exercise the RPC API end-to-end, asserting resulting DB state. Include **multi-client concurrency** tests on the socket and clock-injection for deterministic time.
- **Sync & offline (multi-replica):** spin up **two client `hephd` replicas + a hub `hephd`**, all over **real network sockets** against temp DBs, and assert convergence:
- online round-trip: edit on A → appears on B via the hub;
- **offline → reconcile:** partition A and B from the hub, make divergent edits on each, reconnect, assert both converge;
- **conflict path:** concurrent conflicting scalar edits (e.g. both set a different do-date) land in the **conflict queue** and `conflicts.resolve` settles them deterministically;
- **body CRDT merge:** concurrent edits to the same `doc` body auto-merge without a hard conflict;
- HLC ordering and op-log idempotency (replaying ops is a no-op).
- **Auth:** OIDC token validation on the hub endpoint (reject missing/invalid/expired); **per-user isolation** (user A cannot read/sync user B's nodes); device-code flow happy path against a mock IdP.
- **End-to-end (headless nvim):** drive `heph.nvim` in `nvim --headless` against a real `hephd` + temp DB, running **scripted example workflows** and asserting outcomes (via RPC/DB state and buffer contents). Minimum workflows:
- capture a task → appears in `:Heph next` → open canonical context → add a checklist item → check it → mark task done;
- create today's journal via the picker;
- follow a `[[link]]` on `<Enter>` to the target doc;
- **a recurring task with a checklist: complete it, then assert the next occurrence presents a fresh, all-unchecked checklist;**
- **a sync-driven update arrives while a buffer is open and the buffer reconciles.**
- Harness: `plenary.nvim`/busted, or drive nvim via its msgpack-RPC from the test runner. Keep example workflows as reusable fixtures.
- **CLI tests:** invoke `heph` subcommands against a temp DB; snapshot output; assert `export` round-trips the corpus; `heph conflicts` lists/resolves.
## 10. Technology stack (ratified)
`rusqlite` (bundled) + migration runner · `tokio` + line-delimited JSON-RPC over unix socket · `ulid` · `rrule` · `pulldown-cmark` · `clap` · `anyhow`/`thiserror` · `tracing`. Neovim plugin in Lua, depending on `telescope.nvim`. Cargo workspace: `crates/heph-core`, `crates/hephd`, `crates/heph`, plus `heph.nvim/`.
**Added for v1 client/server + auth (some to confirm at kickoff):**
- **Text CRDT (body merge):** `yrs` (Rust Yjs) — *leaning*; alternative `automerge`. Used for `doc`/`journal`/log bodies. Structured fields use a bespoke op-log + HLC (no library needed).
- **HLC:** small bespoke hybrid-logical-clock (or a crate) — deterministic, clock-injected.
- **Hub network transport:** `axum` (HTTP/JSON) for the sync endpoint — *leaning* (reuses the eventual web-UI server); `reqwest` on the client side.
- **OIDC:** `openidconnect` crate for the Authentik device-code flow; tokens cached in the OS keychain (`keyring`) / 1Password.
## 11. v1 scope
**In:**
- The full data model, markdown handling, "what is next?" ranking, and **recurrence + recurring checklists** (§4§8).
- **Client/server architecture:** per-device client `hephd` + a central hub `hephd` (runnable/deployable binary).
- **Offline-first** operation with **op-log + CRDT sync** and **automatic merge + a conflict queue** (§12).
- **OIDC/Authentik authentication** with **per-user data isolation** (§13).
- `heph.nvim` + `heph` CLI surfaces (incl. `heph conflicts`).
**Out (later phases, scaffolded so as not to block):**
- **Web UI** (the hub serves sync only in v1; reserve `axum` for it later).
- **Actual k3s deployment** to blumeops (Dagger→Zot image, ArgoCD app + Kustomize manifests, external-secrets) — fast-follow once the architecture is proven; the hub binary is built to be deployable.
- **Calendar** integration (read-mostly CalDAV; **never explode recurrence into stored events**), iOS/Watch capture, inferred/semantic context, P2P-over-tailnet sync fallback.
See [[design]] §5§7 for the constraints later phases impose on present choices (keep tasks vs. calendar events separate; expand RRULEs lazily).
## 12. Sync & conflict resolution
**Topology:** hub-and-spoke. Each device holds a full local replica + op-log; the hub is the rendezvous. Devices **push/pull ops by HLC cursor**; the hub never needs to be online for local work.
**Merge semantics (the unit of sync is the op):**
- **`doc`/`journal`/log bodies:** **text CRDT** (`yrs`) → concurrent edits always merge, no hard conflict.
- **Scalar task fields** (attention, do_date, late_on, state, …): **last-writer-wins by HLC**. The losing value, if meaningful, is recorded in `conflicts` (surfaced, not silently dropped).
- **Links / set membership** (tags, project, parent): **OR-Set** add/remove semantics → no conflicts.
- **Tombstones**, never hard deletes → deletion/merge is monotonic and CRDT-friendly.
**Conflict queue:** the unresolvable/ambiguous remainder (a discarded LWW value on a meaningful field; flagged overlapping body regions) becomes an `open` row in `conflicts`. Surfaced via `health().conflict_count`, `conflicts.list`, `heph conflicts`, and a heph.nvim view: *"you have N conflicts."* `conflicts.resolve({id, choice})` settles each. **Sync never blocks** on conflicts.
**Determinism:** HLCs are clock-injected; op application is idempotent and order-independent given HLC. These are the core invariants the sync tests assert.
**Open at kickoff:** CRDT lib confirmation (`yrs` vs `automerge`); hub transport (`axum` HTTP/JSON vs gRPC); propagation cadence (push vs. periodic pull); exactly which fields are "meaningful" enough to enqueue vs. silently LWW.
## 13. Authentication
- **OIDC against Authentik.** Clients authenticate via the **OAuth 2.0 device-code flow** (`auth.login`); the resulting tokens are cached in the OS keychain (`keyring`) / 1Password and refreshed automatically. Offline devices operate on cached credentials.
- **Hub enforcement:** the sync endpoint requires a valid **OIDC bearer token**; the hub maps the token's `sub` to a `users` row and **scopes every op by `owner_id`**. No cross-user reads/writes.
- **Per-user isolation:** all nodes (and their dependent rows) carry `owner_id`; queries and sync are always user-scoped. In practice a single user (`eblume`), but the isolation is real from day one.
- **Local trust:** the local unix-socket RPC trusts the OS user (file-permission-scoped socket); app-level auth is for the **network** boundary (device ↔ hub).
- **At-rest:** plain SQLite in v1 (no encryption) — security boundary is auth + (eventually) network restriction. May revisit (see [[design]]).
## Related
- [[design]] — full design document with rationale and decision history

View file

@ -100,7 +100,7 @@ This project ships two Forgejo workflows:
- `build.yaml` runs on pushes and pull requests targeting `main`, executes `prek run --all-files`, and then runs an optional project hook at `.forgejo/scripts/build` when present.
- `release.yaml` is a manual workflow that computes the next version, optionally builds `CHANGELOG.md` from fragments, packages Quartz docs via Dagger, runs an optional `.forgejo/scripts/release` hook for extra assets, creates the Forgejo release, and pushes changelog updates back to `main` when fragments were consumed.
- TODO: Rename `.dagger/src/project_template_ci/` and the exported Dagger class during first-time setup.
The Dagger module lives at `.dagger/src/hephaestus_ci/` and exports the `HephaestusCi` class.
## Common Pitfalls