diff --git a/.dagger/src/hephaestus_ci/__init__.py b/.dagger/src/hephaestus_ci/__init__.py new file mode 100644 index 0000000..03c30dd --- /dev/null +++ b/.dagger/src/hephaestus_ci/__init__.py @@ -0,0 +1,3 @@ +"""Hephaestus CI — Dagger build functions.""" + +from .main import HephaestusCi as HephaestusCi diff --git a/.dagger/src/project_template_ci/main.py b/.dagger/src/hephaestus_ci/main.py similarity index 94% rename from .dagger/src/project_template_ci/main.py rename to .dagger/src/hephaestus_ci/main.py index df47c43..e129a44 100644 --- a/.dagger/src/project_template_ci/main.py +++ b/.dagger/src/hephaestus_ci/main.py @@ -2,9 +2,8 @@ import dagger from dagger import dag, function, object_type -# TODO: Rename class to match your project (also rename the src/ directory) @object_type -class ProjectTemplateCi: +class HephaestusCi: @function async def build_docs(self, src: dagger.Directory, version: str) -> dagger.File: """Build Quartz docs site. Returns docs tarball.""" diff --git a/.dagger/src/project_template_ci/__init__.py b/.dagger/src/project_template_ci/__init__.py deleted file mode 100644 index e9233a3..0000000 --- a/.dagger/src/project_template_ci/__init__.py +++ /dev/null @@ -1,4 +0,0 @@ -"""Hephaestus CI — Dagger build functions.""" - -# TODO: Rename class to match your project (also rename the src/ directory) -from .main import ProjectTemplateCi as ProjectTemplateCi diff --git a/AGENTS.md b/AGENTS.md index 882e882..6188bef 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,13 +8,16 @@ Guidance for Claude Code working in this repository. See also [[ai-assistance-gu ## First-Time Setup -This repository is a **Forgejo template**. Most customization points are auto-resolved by Forgejo's template variable expansion when a new repo is created. The remaining manual steps are: +This repository was instantiated from the `project-template` Forgejo template. Setup status: -1. **Set `baseUrl`** in `docs/quartz.config.ts` — this is the hosted docs domain, not the repo URL. localhost if not hosted. -2. **Rename `.dagger/src/project_template_ci/`** directory to match your project, and update the class names inside -3. **Fill in project structure** in the `AGENTS.md` Project Structure section -4. **Fill in license info** in `README.md` -5. **When all TODOs are resolved:** delete this "First-Time Setup" section entirely from `AGENTS.md` — it is only needed once +- [x] `baseUrl` in `docs/quartz.config.ts` set to `localhost` (update once docs are hosted) +- [x] Dagger module renamed to `.dagger/src/hephaestus_ci/` (`HephaestusCi` class) +- [ ] Fill in the Project Structure section below once the Rust workspace is scaffolded +- [ ] Fill in license info in `README.md` + +Delete this section once the remaining items are resolved. + +> **This is now a generated repo, not the template source.** C1/C2 changes use feature branches + PRs (`tea pr create`); noteworthy changes get changelog fragments in `docs/changelog.d/`. ## Rules @@ -52,7 +55,7 @@ See [[agent-change-process]] for the full methodology. ``` ./docs/ # Diataxis docs, Quartz config, and release content ./docs/changelog.d/ # keep only .gitkeep in the template; generated repos add towncrier fragments here -./.dagger/ # Dagger module; rename src/project_template_ci/ when forking +./.dagger/ # Dagger module (src/hephaestus_ci/) backing docs builds and releases ./.forgejo/workflows/ # generic build and release workflows for generated repos ./.forgejo/scripts/ # optional per-project build/release hooks consumed by the workflows ./mise-tasks/ # repo automation via `mise run` diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..c26ff68 --- /dev/null +++ b/LICENSE @@ -0,0 +1,18 @@ +All Rights Reserved + +Copyright (c) 2026 Erich Blume + +This software and its associated documentation and source files (the +"Software") are the private, proprietary work of the copyright holder. + +No license, express or implied, is granted to any party. You may not use, +copy, modify, merge, publish, distribute, sublicense, or sell copies of the +Software, in whole or in part, without the prior written permission of the +copyright holder. + +The Software is provided "as is", without warranty of any kind, express or +implied, including but not limited to the warranties of merchantability, +fitness for a particular purpose, and noninfringement. + +Open-source release under a permissive license may be considered in the +future, at the sole discretion of the copyright holder. diff --git a/README.md b/README.md index 9a742a1..8394baf 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,8 @@ -# project-template +# hephaestus -A personal project template with opinionated infrastructure for documentation, CI, and AI-assisted development. +A personal context management system — a unified, self-hosted application that fuses a wiki-style knowledge base (Zettelkasten) with task management. Built in Rust, offline-capable, and syncs to a central instance hosted in blumeops. + +See [the project design document](docs/explanation/design.md) for goals, architecture, and the development roadmap. ## What's Included @@ -11,20 +13,6 @@ A personal project template with opinionated infrastructure for documentation, C - **AI assistance** — `AGENTS.md` + structured docs for Claude Code (C0/C1/C2 change process, Mikado method) - **Task runner** — [mise](https://mise.jdx.dev/) tasks for docs validation, Mikado chain management, release preview, and runner inspection -## Forking This Template - -This is a **Forgejo template repository**. When you create a new repo from this template, Forgejo automatically expands variables like `${REPO_NAME}` and `${REPO_OWNER}` in key files — handling most customization automatically. - -After creating your repo, the remaining manual steps are: - -1. Set `baseUrl` in `docs/quartz.config.ts` to your docs site domain -2. Rename `.dagger/src/project_template_ci/` directory and update class names to match your project -3. Review and tailor the project structure section in `AGENTS.md` -4. Add license information to `README.md` -5. Remove the "First-Time Setup" section from `AGENTS.md` and this section from `README.md` - -If you use Claude Code, it will prompt you to resolve remaining TODOs at the start of your first session. - ## Getting Started ```bash @@ -54,4 +42,4 @@ dagger call build-docs --src=. --version=dev export --path=./docs-dev.tar.gz ## License - +All rights reserved. This is a personal, private project — not licensed for use, copying, modification, or distribution. Open-sourcing may be considered in the future. See [LICENSE](LICENSE). diff --git a/docs/changelog.d/+project-design.doc.md b/docs/changelog.d/+project-design.doc.md new file mode 100644 index 0000000..75cb3fe --- /dev/null +++ b/docs/changelog.d/+project-design.doc.md @@ -0,0 +1 @@ +Add the project design document (`docs/explanation/design.md`, rationale + decision history) and the distilled technical specification (`docs/reference/tech-spec.md`, the build artifact) defining hephaestus as a unified, self-hosted, **client/server + offline-first** task + knowledge-base system: typed node graph, the lived priority discipline ("what is next?"), recurrence with fresh-per-occurrence checklists, op-log/CRDT sync with conflict resolution, OIDC/Authentik auth, the heph.nvim surface, and a test-driven development strategy. diff --git a/docs/changelog.d/+template-setup.infra.md b/docs/changelog.d/+template-setup.infra.md new file mode 100644 index 0000000..f99c513 --- /dev/null +++ b/docs/changelog.d/+template-setup.infra.md @@ -0,0 +1 @@ +Customize the repository from the `project-template`: rename the Dagger module to `hephaestus_ci` (`HephaestusCi`), set the docs `baseUrl`, add an All-Rights-Reserved `LICENSE`, and update `README.md`/`AGENTS.md` for the hephaestus project. diff --git a/docs/explanation/design.md b/docs/explanation/design.md new file mode 100644 index 0000000..9a06a09 --- /dev/null +++ b/docs/explanation/design.md @@ -0,0 +1,419 @@ +--- +title: Design Document +modified: 2026-05-31 +tags: + - explanation + - design +--- + +# Hephaestus — Design Document + +> **Status:** Living design record. This is the **rationale + decision-history** document. For the clean, implementation-facing spec the next session builds from, see **[[tech-spec]]**. Sections marked **❓ OPEN** are unresolved; **🔒 DECIDED** are settled. + +## 1. Purpose & Vision + +**Hephaestus** is a personal context management system that fuses two systems the owner already relies on into a single, purpose-built, self-hosted application: + +1. **The Zettelkasten** (`~/code/personal/zk`) — an Obsidian vault of ~600 markdown files acting as a personal wiki, journal, knowledge base, and "task-context" store for long-running projects. Notes use `[[wiki-links]]`, YAML frontmatter (`id`, `aliases`, `tags`), timestamp-based note IDs (`1722897441-MWFE`), and daily notes (`YYYY-MM-DD`). +2. **Todoist** — the owner's primary task manager, accessed today via a bespoke Python mise-task in blumeops (`mise-tasks/blumeops-tasks`) that polls the Todoist v1 API (projects → tasks; fields: `content`, `description`, `priority`, `due`) using a token from 1Password. + +Today these are loosely coupled by fragile cross-links (`todoist://` in notes, `[[note]]` text in Todoist comments). Hephaestus replaces the fragile coupling with **one unified data model and database** where tasks and knowledge are first-class, linkable entities. + +### Guiding principles + +- **Context-aware:** A task like "Fix the roof leak" is coupled to the relevant knowledge — the home-repair log, the log of contractor calls, etc. Surfacing that context is a core feature, not an afterthought. +- **Workflow-optimized:** Highly optimized, concise answers to recurring questions — above all **"What is next?"** — are the primary UX. Speed and concision beat feature breadth. +- **Owner-built, owner-hosted:** A standalone codebase replacing a loose stack of tools, hosted out of blumeops. + +## 2. Goals & Non-Goals + +### Goals (v1 / prototype) + +- 🔒 Unified data model linking **notes** and **tasks** as first-class, cross-linkable entities. +- 🔒 SQLite backend. +- 🔒 Rust implementation. +- 🔒 **Distributed dual-mode operation:** useful fully offline, auto-syncs to a central self-hosted instance in blumeops (k3s container) when reachable. The central instance may be unavailable for long periods. +- 🔒 Surfaces over a shared core (via the `hephd` daemon): **CLI** (everyday driver), **nvim plugin** (primary editing UX), **web UI** (hub-hosted). +- 🔒 Optimized workflow queries, especially "What is next?", with context surfacing. +- 🔒 **Auth from the ground up** (OIDC/Authentik), with isolated multi-user accounts — sensitive data. + +### Non-Goals (v1) / Later + +- ⏳ iOS app + Apple Watch app for quick voice-dispatched task capture — explicitly a **follow-up goal**, not v1. +- ⏳ Sharing/collaboration between users — accounts are isolated for v1 (multi-user *isolation* is in scope; multi-user *collaboration* is not). +- ❌ Obsidian feature parity (canvas, graph view, plugins) — clean break. +- ❌ **Migration / import / Todoist sync in v1** — the prototype is a **clean break**: heph is the system of record from day one. No ZK import, no Todoist bridge. (Migration may return as a later, optional phase.) +- ❌ Encryption at rest / E2E in v1 — security model is **access restriction (OIDC auth) only**; plain SQLite. May revisit. +- ❌ Inferred/semantic context surfacing in v1 — explicit links + search only. + +## 3. Data Model + +> 🔒 **DECIDED (shape).** SQLite is the canonical store; a node's body is plain markdown text; an `export` mode materializes the whole store as a directory of `.md` files. Clean break from Obsidian-the-app, but markdown stays the portable payload. Tasks are **thin**; rich context lives in documents; everything is connected by typed links. + +### Entities + +- **Node** — the base entity. Every first-class thing is a node with a stable, sync-safe ID (see §3.1). A node has a `kind` discriminator. Kinds (initial set): + - **`doc`** — a rich markdown context document (the bulk of the knowledge base; replaces ZK articles, journals, logs). Body = markdown text. Has `aliases`, timestamps. + - **`task`** — **thin**: ID + state, plus a few scalar attributes (attention-state, do-date, late-on, recurrence) and links to a project/tags/context. **No prose body of its own** — context is supplied by linked `doc` nodes (or a doc *is* the work-log for the task). This is the key unification: ZK-articles-with-todos and Todoist-tasks-with-context both reduce to *typed links between thin tasks and rich docs*. The full task-attribute model is in §6.2 (it encodes the owner's actual lived prioritization system). + - **`project`** — a grouping node (maps to Todoist projects / ZK project dirs like `payrix/`). + - **`tag`** — a label node (so tags are linkable/queryable, not just strings). + - **`journal`** — a daily-note node keyed by date (`YYYY-MM-DD`). +- **Link** — a typed, directional edge between two nodes. Types (initial set): `wiki` (a `[[link]]` found in a doc body), `context-of` (doc ↔ task), `blocks`, `parent`/`child`, `tagged`, `in-project`. Wiki-links in markdown bodies are parsed and materialized as `wiki` links so the graph and the prose stay consistent. + +### 3.1 Identity + +- ❓ **Leaning ULID** (sortable, sync-safe, collision-free across offline devices) as the internal primary key, with a human-facing `slug`/`alias` layer on top so `[[wiki-links]]` can be written by name. The ZK's timestamp-IDs (`1722897441-MWFE`) are ULID-like already; a migration can map them. + +### 3.2 Scalar task attributes vs. links + +**Tags and projects are nodes** (linkable, queryable). **Attention-state, do-date, late-on, recurrence are scalar columns** on the task. See §6.2 for the semantics — these columns are not arbitrary; they encode a specific, battle-tested model. Due dates are *not* first-class nodes (a task has one project/context; "this Friday" doesn't need its own context). + +### 3.3 Recurrence model (recurring tasks & checklists) — IN SCOPE for v1 + +A **recurring task** carries an **RFC-5545 RRULE** and acts as a recurring **definition**. + +> ❌ **The anti-pattern we must avoid (Todoist's mistake):** modeling a recurring task's checklist sub-items as standalone, non-recurring tasks that **carry their completion state forward** when the parent recurs. This is the corner; designing around it now is *why* recurring checklists are in v1 rather than deferred. + +**heph model (proposed — one point to ratify):** + +- The recurring definition holds a **checklist template** (the ordered set of sub-item definitions). +- Each **occurrence** materializes a **fresh checklist instance** — brand-new context items, all `outstanding` — scoped to that occurrence. **Completion state is per-occurrence and never carries forward.** +- Occurrences are retained as history (no hard deletes; ties naturally to the per-task **log**, §6.4). The "what is next?" engine surfaces the **current/active occurrence**; completing it advances to the next per the RRULE. + +**The one fork to confirm:** + +- **(a) Occurrence instances (recommended):** the definition spawns a lightweight **instance** per occurrence (its own do-date + its own fresh checklist items). Full per-occurrence history; unambiguously "a fresh instance per recurrence"; heavier (more rows). +- **(b) Roll-forward in place:** a single task node; on completion, **log the finished occurrence** (to the per-task log) and **reset the checklist to outstanding** + advance the do-date. Lighter; history is narrative (in the log) rather than queryable per-occurrence. + +Both satisfy the correctness requirement (fresh checklist, no carried state). (a) is recommended for clean history and zero ambiguity; (b) is the pragmatic-lite option. + +## 4. Architecture + +> 🔒 **DECIDED (shape).** + +Layers, top to bottom: + +- **Surfaces (thin clients):** the **nvim plugin (`heph.nvim`)** is the **primary surface for v1** — a full "org-mode"-style experience (markdown buffers backed by `doc` nodes; agenda / "what is next" / capture / linking / journaling as plugin commands). It is the explicit **successor to obsidian.nvim**, which the owner currently drives the ZK with (telescope picker, `o*` verbs, `` to follow `[[wiki-links]]`, dailies, multi-state checkboxes) and rarely uses Obsidian-proper alongside. heph.nvim must reach that feature parity (see §6.5) and replace it. The **CLI** (`heph`) is a secondary/utility surface (export, scripting, admin) — explicitly *not* a focus, though it shares the same command surface. The **web UI** is the occasional hub-served surface. A later iOS/Watch client talks to the hub directly. + - ❓ **Dual-mode binary:** a single Rust binary that can run as the **server/daemon** *or* expose the same operations as **CLI subcommands** *or* act as the **backend the nvim plugin drives**. Appealing (one codebase, one command surface) — to confirm as the packaging model. +- **Per-device daemon (`hephd`):** owns the local SQLite handle, the CRDT/op-log state, and background sync. All local surfaces connect to it over a local socket. This is what makes multi-surface access concurrent and safe on one SQLite file, and gives one place to run background sync. +- **Core crate (Rust lib):** data model, query engine, markdown parsing + wiki-link extraction, and sync logic. Linked by both `hephd` and the hub server. +- **Hub:** `hephd` running in "server" mode on blumeops k3s — central SQLite, the sync rendezvous point, and the web UI host. May be offline for long periods; devices keep working and reconcile when it returns. + +Open points: + +- ❓ nvim ↔ daemon transport: JSON-RPC over a unix socket (nvim has a native RPC channel; allows the daemon to *push* updates when sync brings in remote changes) vs. the plugin shelling out to the `heph` CLI. Leaning socket RPC. +- ❓ Web front-end style: server-rendered (Axum + HTMX/Askama) vs. WASM SPA (Leptos/Dioxus). Leaning **SSR + HTMX** since it's the lowest-traffic surface — favor simplicity. + +## 5. Sync & Conflict Resolution + +### Requirements (gathered from the owner) + +- 🔒 ~99% of interaction today is via the **CLI on Gilbert** (dev laptop). Future split: ~90% Gilbert, ~9% phone/watch, ~1% web. +- 🔒 Must work **seamlessly across tailnet workstations** (Gilbert, ringtail, others): dispatch a change on one, walk to another, see it arrive "in short order." +- 🔒 **Genuine offline conflicts will occur** — e.g. Gilbert edits offline, then ringtail edits, then Gilbert reconnects. Must resolve **gracefully**: auto-merge where possible; for the unresolvable remainder, a **conflict queue + alert** ("you have N merge conflicts, run `heph conflicts`"). +- 🔒 **Auth from the ground up** — this is extremely sensitive data. Use **Authentik (OIDC)** as IdP, support **multiple isolated user accounts** (in practice just `eblume`). +- 🔒 A **web UI** is part of the hosted (hub) model. + +### Proposed model — *for confirmation* + +A **hub-and-spoke, op-log + CRDT** design: + +- **Topology:** each device runs `hephd` with a full local SQLite replica; the blumeops **hub** is the rendezvous. Devices push/pull an **append-only operation log** to the hub when reachable. Hub-and-spoke (not P2P mesh) — simpler, matches "central self-hosted app," and the hub is normally up on the tailnet so propagation is prompt. When the hub is down, devices keep working and the op-log drains on reconnect. +- **Merge semantics (so "graceful" is precise):** + - **`doc` bodies (markdown):** a **text CRDT** (e.g. Yrs/Automerge) → concurrent edits *always* merge with no hard conflict. This covers the common case. + - **Scalar task fields (completion, due, priority):** **last-writer-wins via Hybrid Logical Clocks**. Deterministic and cheap. + - **Links / set membership (tags, projects):** add/remove as a CRDT set (OR-Set) → no conflicts. + - **Conflict queue:** because CRDTs auto-merge, true blocking conflicts are rare. We *surface* (not block) the cases that are semantically ambiguous: a discarded LWW value on a meaningful field (e.g. completion flipped both ways, divergent due dates), or concurrent edits to overlapping text regions. These go to `heph conflicts` with an alert; sync never stalls. +- **Identity & ordering:** ULIDs for nodes (§3.1); HLC timestamps stamp every op for causal ordering across offline devices. +- **Auth:** the hub's sync + web endpoints require an **OIDC bearer token from Authentik**. Devices obtain tokens via the **OAuth device-code flow**; refresh token stored in the OS keychain / 1Password. Every node/op is owned by a `user_id`; the hub enforces per-user isolation. Offline devices operate on cached credentials. + +### Open points + +- ❓ **Hub-only vs. P2P fallback:** if the hub is down but Gilbert and ringtail are both on the tailnet, do they sync *directly*, or wait for the hub? Hub-only is simpler; direct P2P is more available. (Leaning hub-only for v1.) +- ❓ **Encryption at rest:** given "extremely sensitive," should each device's SQLite (and the hub's) be encrypted at rest (e.g. SQLCipher), and/or should op-log payloads be end-to-end encrypted so the hub stores ciphertext it can't read? (E2E complicates server-side search/web UI.) +- ❓ **Todoist's role** during/after migration — two-way bridge during transition, import-once, or retire on cutover? +- ❓ How "short order" is short — is a few seconds of propagation (push-based) needed, or is periodic pull (e.g. every 30–60s) fine? + +## 6. Workflow Features + +> Driven by "highly optimized specific workflows." heph is organized around a small set of **workflows**, each a first-class surface in heph.nvim: +> +> - **Task / "What is next?"** (§6.1–§6.3) — the Tactical / Strategic / Organizational task engine. The flagship. +> - **Log** (§6.4) — daily journal + optional append-only per-task work-logs ("narrative > list"). +> - **Wiki / KB** (§6.5) — the whole context corpus as a browsable/queryable personal wiki; the direct **obsidian.nvim replacement**. +> - **Calendar** (§6.6) — entering via the calendar, tied to Strategic mode. *Deferred, but scaffolded carefully.* + +### "What is next?" — the flagship + +> ❓ **ACTIVE DEEP-DIVE.** This is the defining workflow and has ~10 years of the owner's prior art and prior tool attempts behind it. Being designed from that prior art rather than from scratch — see §6.1. The blumeops-tasks ranking (overdue-first, then priority, p3 = backlog) is a known reference point, not the target. + +### Context surfacing — v1 scope + +> 🔒 **DECIDED for v1.** Explicit links only; no inferred/semantic context yet. + +- Every **task** automatically gets **one canonical `doc` (context) node**, created and linked on task creation. That canonical doc is the jumping-off point and can wiki-link onward to other docs/cards. +- nvim commands provide **telescope/fzf/rg-style search** over context docs, optionally with preloaded filters (e.g. scoped to a project). +- Inferred context (shared tags, full-text/semantic similarity) is deferred — large surface area, not the v1 focus. + +### 6.1 "What is next?" — prior art & synthesis + +> Distilled from ~10 years of the owner's thinking — predating the ZK by ~5 years — chiefly the owner's direct account (this design conversation), plus `zk/mole/20240202113646.todo.md` (the most developed written framework), daily logs (`zk/home/2023-09-01.log.md`, `2023-10-12.log.md`), and `zk/payrix/friction_log.md`. Prior tool code may exist at `github.com/eblume/{mole,hermes}` (check detached-head branches — several restarts); the owner expects little reusable there beyond what's captured here. + +#### History: Hermes → Mole → heph (and the lessons) + +- **Hermes — the "time accountant" (≈ earliest).** Goal: a *timeline*-based system fed "blueprints" of what a day should look like, with rules like *"check email twice a day, unless a high-priority email comes in — but even then no more than once an hour,"* continuously re-solving the calendar in real time. Built a working prototype using a **SAT solver (OR-tools)**. **Killer feature deduced: a quick, meaningful answer to "what is next" — here, the next thing on the calendar.** **Why it failed:** the calendar became a *battle zone*, swinging wildly every few minutes as the solver re-resolved the dependency graph → impossible to plan around. **Lesson + salvage:** there may be fruit here as a future, *opt-in* heph **planning mode**, but real-time auto-rescheduling of a visible calendar is anti-useful. +- **Mole — "whack-a-mole" rules engine.** A rules engine that constantly re-evaluated rules and materialized Todoist tasks (canonical first test: *"is there an active task 'whack the mole'? if not, make one"*). **"What is next" = "what Todoist says is available."** Worked reasonably. **Why it stalled:** Todoist accumulated **too many tasks**, so *choosing the actual next thing* became the whole problem again. This pushed the owner to design the Todoist project/priority/filter discipline in §6.2. **The deeper lesson:** Mole's automation **wasn't adding much value over the project/priority discipline itself** — so the owner *turned Mole off and just kept using Todoist*. The manual discipline was the real engine. +- **Today.** No automation beyond blumeops' **read-only** `blumeops-tasks` mise poller. Just Todoist + the §6.2 discipline — but as the direct descendant of all the above. **Implication for heph:** the value is in *embodying and strengthening the discipline* (and finally fixing the weak note↔task link), **not** in clever auto-scheduling or a rules engine. Earn any automation only after the discipline is faithfully supported. + +#### "What's next?" is three questions, not one + +The owner's framework splits the question by **mode**: + +- **Tactical** — blank-slate "what do I do *right now*?" Stay in the editor/terminal; capture interruptions and resume in **milliseconds** ("save state and walk away in milliseconds"). This is the mode a single ranked list serves. +- **Strategic** — "I want to do X — what's the *next step*, or *where was I*?" Exploring a goal, generating the sub-tasks to reach it. +- **Organizational** — "I want to do X — help me *plan* the steps." Project-level enumeration of what needs doing, not how. + +🔒 **REAFFIRMED.** The owner still strongly identifies with this split (the earlier "maybe" was conflating it with the unrelated CLI-first *process* note on the same page). The clean operational test, in the owner's words: **"looking at a project's list of todos = Organizational; deciding which to work on = Strategic; working a todo = Tactical."** Escalation flows upward (project-infra work can become Strategic, then Tactical). + +**How heph embodies it** — as **named modes/views in the nvim plugin**, *not* an inference engine that guesses your mode: + +- **Tactical (execution view):** you are "tactically engaging" a specific committed task. Loads its canonical context + sub-items into a **goal stack**; millisecond interrupt-capture pushes new context items onto the stack; "smallest amount of contextual guidance and next steps to stay productive." Includes **Chore Mode** (synthesize a goal stack from a query — e.g. the `Chores` project — and chug through one by one). +- **Strategic (planning view):** editor present but the goal is to *explore/decompose*, not edit. "To do X I must also do Y and Z." Produces a fleshed-out task with ready sub-items so Tactical can grab and go. **When Strategic completes, no work has been done** — only new/changed tasks, docs, links. +- **Organizational (survey view):** project-level enumeration — *what* needs doing, not *how*. +- **Shared primitive — the goal stack** (see §6.3) spans Tactical and Strategic; transitions between modes are first-class (Strategic *produces* → Tactical *consumes*). + +#### The real cost is resumption, not ranking + +The recurring pain in the logs is **reorienting after a context switch**, not deciding raw priority. Design rules that follow: + +- **Resumption hint invariant:** when work pauses on a task/todo, it should leave at least one concrete next-step so future-you can re-enter quickly. heph should make capturing/“leaving a breadcrumb” frictionless and surface it first on return. +- **Narrative > list:** daily logs proved more motivating and contextual than flat task lists ("a log is the way"). Journaling/log nodes are first-class, and "what is next" should be able to lean on recent log narrative, not just task metadata. + +#### The "weak link" is heph's reason to exist + +The #1 unsolved problem: the **fragile, clunky coupling between Todoist and notes** (manually typing `nb o 45` into task descriptions). heph's thin-task ↔ auto-created canonical-context-doc model is the direct fix: the link is structural, not a hand-typed string. + +#### Mapping prior taxonomy → heph entities + +| Prior (`nb`/Todoist) | Meaning | heph | +|---|---|---| +| nb-project | grouping; completing all members completes it | `project` node | +| nb-todo | planning complex multi-step work; has "a single unified context log" | `task` + its **canonical context `doc`** | +| nb-task | immediate, short-lived "might eventually be done"; checking off = "no longer might be done" (not necessarily *completed*) | thin `task` node | +| Todoist | "deciding what is next and knowing what needs to be done now" | heph's "what is next?" engine | +| Calendar | scheduling / future | due dates (+ later calendar integration) | + +> 🔒 **Preserved.** The nb-task semantic — checking off means "no longer outstanding," *not* "accomplished" — carries into heph. **Context items are `outstanding` / `not-outstanding`** (see §6.3). For **committed tasks**, heph distinguishes end-states: **done** (accomplished) vs. **dropped/dismissed** (let go, e.g. during a Blue review). Both are "not outstanding"; the distinction is kept for honesty and history. + +#### Process steer (now historical) + +The owner's old rule — "avoid ncurses and interactive UIs; write atomic code and call it manually with subcommands… until a strong pattern emerges" — was a *focus device for hand-building tools*. It no longer constrains heph: we go **straight to the nvim org-mode plugin backed by the Rust server** (the patterns are now well understood; see §6.2/§6.3). The CLI remains as a secondary/utility surface. + +#### Distilled requirements for heph's "what is next?" — *to confirm/refine* + +1. **Mode-aware** (Tactical / Strategic / Organizational), not a single global sort. +2. **Sub-millisecond capture** and fast state-save (Tactical interrupt handling). +3. **Resumption-first:** on returning to a task, surface the last breadcrumb/next-step before anything else. +4. **Context-coupled:** every task one keystroke away from its canonical context doc and onward links. +5. **Log-aware:** can factor in / surface recent daily-log narrative. +6. **A sane default ranking** for the Tactical blank-slate case — the concrete model is §6.2, not the blumeops-tasks placeholder. + +### 6.2 The lived priority discipline (the working basis for heph) + +> 🔒 **This is the real model.** The owner has used this Todoist discipline daily for years; Mole was eventually *turned off* because this discipline outvalued the automation. heph should **embody and strengthen** it. Captured verbatim-in-spirit from the owner's account. + +**Projects = contexts (a task has exactly one).** Examples: `Life` (catch-all), `Work`, `Chores`, `Maintenance`, `Outside`, `Allison` (wife), `Errands`, `Cooking`, `Health`. A project *contextualizes* a task; it is not a plan. + +**Attention-state (thought of by color, not number):** + +| State | Meaning | +|---|---| +| **White** (default) | Able to be done **once its do-date arrives**. (See do-date semantics below.) | +| **Orange** | **Top of mind.** Keep **≤ 6**, reassessed *every day*. Goal: you know your top-of-minds *without looking at the tool*. | +| **Red** | Top of mind **+ there will be a consequence if not done ASAP**. Red means **the existence of a consequence, NOT its severity/importance.** Flagging by *importance* is a trap → anxiety (everything due feels important). | +| **Blue** | **On Deck** (backlog). Doable, but deliberately *cooling off*. The pressure-relief valve that keeps the working set light. | + +**Do-date, not due-date (key insight):** "due date" means **DO date** — *don't do it before this date*. Optionally a separate **"Late On"** marker for when it actually becomes a problem. **Due dates are a terrible planning tool** — calendar-style deadlines make everything "yell for attention" constantly. heph must treat the do-date as *earliest actionable*, not a deadline alarm. + +**Working-set discipline (the healthy tensions — a feature, not a bug):** + +- Keep White+Orange+Red ≤ **~30 total**, or the system collapses into noise. Keep Orange ≤ **6**. +- Push freely to Blue to stay light; periodically **review Blue: keep vs. let go** (this is where "drop/dismiss" happens). Target On-Deck **< 100** (owner is at ~149 and hates it). +- These tensions **map the owner's emotional state to the task system** and should be *surfaced honestly*. heph must avoid **two failure modes**: (1) **"fake happy"** — telling you everything's fine when it isn't; (2) **overwhelm** — yelling that you're buried until you freeze and make no progress. Reflect true state; help maintain the tensions. + +**Filters = saved views (queries over the above):** + +- **Top of Mind** — all Red + Orange. +- **Tasks / Work Tasks** — Red/Orange/White that are *actionable now* (do-date arrived), scoped by project. +- **Chores** — like Tasks but from `Chores`, *slightly tuned down* so the (near-universally recurring) chores don't become a nuisance. +- **On Deck** — all Blue. +- **Schedule** — recurring checklists from a few projects (e.g. *Morning Routine* → brush teeth, feed birds, coffee, medicine). The owner uses *this particular filter* sparingly, but the underlying **recurring-checklist mechanism is IN SCOPE for v1** (see §3.3): the only way to be sure we avoid Todoist's mistake is to model it up front. **Each recurrence gets a fresh checklist instance; completion never carries forward.** + +**heph's default "what is next?" (Tactical blank-slate)** therefore ≈ Top of Mind (Red first — by *consequence*, then Orange) **+** White items whose do-date has arrived, scoped to the current project/context, **with Blue hidden**. Concise, honest, light. + +### 6.3 Two kinds of task: commitments vs. context items + +> 🔒 **DECIDED (shape).** A **commitment axis** orthogonal to the §6.2 attention-states. + +- **Committed task** — long-lived, strongly tied to context (e.g. "Complete PLAT-1234", "Build a nursery"). **Participates in §6.2**: has an attention-state, a project, a do-date; is a candidate for "what is next"; may carry sub-structure and a web of context. +- **Context item** — ephemeral, created mid-flow ("six things before this is done", "oh crap, another" pushed on the stack). **Does *not* enter the §6.2 scheduling system** — it is *part of its containing task's context*, with only **outstanding / not-outstanding** state. Never appears in global "what is next." +- **Promotion** is first-class: a context item can be **promoted to a committed task** when you decide to actually schedule it (the old "nb-todo → Todoist-task on activation"). +- **Ephemeral ≠ deleted:** deletes effectively never happen — tombstones + a dedicated cleanup mode only. "Ephemeral" is lifecycle/visibility, not storage. (This also keeps the CRDT sync simple.) +- **The goal stack** is a **session-scoped, synthesized construct**: seeded from a committed task (Tactical) or from a query over contexts (Chore Mode — "grind on some chores"). It holds committed tasks and/or context items for the duration of a work session. + +#### Editing surface for context items — *deliberately deferred to the prototype* + +> 🔑 **Key realization: this is NOT a data-model fork.** In both options below the **stored representation is identical** — context items are outstanding nodes linked to their container; the daemon owns them; markdown `export` renders them as checkboxes regardless. The choice is *purely the nvim editing affordance*, so the backend is shared and we **prototype both and pick by feel** (ergonomics can't be judged on paper). + +- **Option A — checkboxes in the body, derived on save.** Edit the task's canonical context doc as plain markdown; `- [ ]` lines are materialized into context-item state on `:w` (`BufWritePost`), reusing the wiki-link extraction machinery. **No real-time scanning needed** (debounced live updates are optional polish). Remote CRDT edits are *pushed* from the daemon to reconcile an open buffer. Item identity is low-stakes (reword = tombstone-old + add-new) since items are ephemeral; identity is pinned only at **promotion**. Most "personal org-mode"-native; millisecond capture = "type a line." +- **Option B — structured items, rendered.** Items are nodes from the start; the body stays pure prose. Render inline via **virtual text/extmarks** (no pane) or a transient **floating window** (avoids the disliked persistent cutaway); capture via a leader chord + `vim.ui.input`. Cleaner separation and trivial promotion; heavier capture; more UI to build. +- Lean: **A** for capture flow, but **build the shared node backend first** and trial both affordances in the plugin. + +> **Prior-art note (multi-state checkboxes):** the owner's obsidian.nvim config uses checkbox states `" ", "x", ">", "~", "!"` (todo / done / forwarded / ~ / important). heph's task/context-item state model should accommodate richer-than-binary states (at least the done/dropped/outstanding set in §6.1; possibly forwarded/deferred). + +### 6.4 The Log workflow + +> 🔒 **In scope for v1** (daily journal certainly; per-task logs as the model allows). Logs embody the "narrative > list" insight and the resumption-hint invariant from §6.1. + +Two related shapes: + +- **Daily journal.** Today driven by the `zkd` abbrev → `Obsidian dailies`: a telescope picker to create/open dated journal buffers. heph keeps this as a **first-class** feature: `journal` nodes keyed by date, a picker to jump to today / recent / create. The daily journal is the catch-all narrative stream. +- **Per-task log.** Sometimes a task wants an append-only "here's what I was working on" side-commentary. It **half-belongs to the task's context and half doesn't** — so model it as a **separate buffer/node from the task's canonical context doc**, optionally present only for tasks that need longer-duration commentary. Properties: + - **Append-only** (entries accrete; you don't rewrite history). + - **Dispatchable from an isolated context:** you must be able to fire a log entry *without leaving / losing your place in* the context buffer you're navigating — i.e. append to the log from elsewhere (a quick-capture into the task's log), while the context buffer stays where it is. (This is the ergonomic crux; likely a quick-capture command targeting "the current task's log".) + - The per-task log is a strong candidate to *be* the resumption breadcrumb store (§6.1). + +❓ Open: is the per-task log a distinct node `kind` (`log`, append-only), or just a `doc` flagged append-only and linked `log-of` → task? Leaning a dedicated append-only `log` kind to make "dispatch an entry" a cheap, well-defined op. + +### 6.5 The Wiki / KB workflow + +> 🔒 **First-class in v1.** This is the "view all my contexts as one big personal wiki" surface — the direct replacement for obsidian.nvim. + +The owner estimates ~90% of the value comes from a few well-worn primitives, so heph.nvim should reach **obsidian.nvim feature parity** on day one: + +- **Follow `[[wiki-links]]` on ``** to jump between context docs (the core navigation gesture). +- **Telescope-based** search (`rg`), quick-switch between docs, tag browse, **backlinks**, outgoing **links** — mirroring the owner's `o*` verbs (to be rebound under heph's own prefix). +- **Logical layout + tagging** of context docs so the corpus is browsable/queryable as a whole. +- New-doc-from-query and insert-link from the picker (obsidian.nvim's ``/``). + +heph.nvim **replaces** obsidian.nvim outright — the owner already rarely opens Obsidian itself. This workflow is the knowledge-base analogue of the Organizational task view: surveying and traversing the `doc` graph. + +### 6.6 The Calendar workflow + +> ⏳ **Deferred (later version) — but scaffold now, don't paint ourselves into a corner.** Ties closely to **Strategic** mode (planning against time). + +- The owner uses **Calendar.app** bound to a **Gmail calendar** (almost certainly **CalDAV**; Gmail also exposes ICS/WebCal feeds — to confirm when implemented). +- Eventual goal: **integration between calendar views and task views** — enter the system *via* the calendar, see tasks/do-dates in temporal context. +- ⚠️ **Hard-won gotcha:** enabling Todoist's gcal integration **flooded the calendar with a million events** because of recurring rules. heph must **never explode recurrence into individual calendar events**. This echoes the Hermes "battle zone" failure (§6.1) — calendar integration must be *read-mostly, careful, de-duplicated, and opt-in*, not a firehose. +- **Scaffolding guidance (decisions to keep clean now):** keep a crisp separation between *tasks/do-dates* and *calendar events*; model recurrence as RRULEs that are **expanded lazily for display, never materialized as stored events**; design sync so a future **read-only CalDAV ingestion** (calendar → heph) is easy, and treat any heph → calendar export as a later, explicitly bounded, non-exploding feature. + +## 7. Hosting & Deployment + +Reuse the established blumeops patterns (🔒 confirmed by repo conventions): + +- Containerized service deployed to **k3s** (ringtail cluster) via **ArgoCD app-of-apps** + **Kustomize** raw manifests (no Helm/Flux). +- Secrets via **1Password + external-secrets operator** (`ClusterSecretStore onepassword-blumeops`). +- Image built via **Dagger + Forgejo Actions**, pushed to **Zot** registry (`registry.ops.eblu.me`). +- ❓ Rust build approach for the container: Nix (like kingfisher) vs. Dagger-native `cargo` multi-stage build. +- ❓ Persistence for the central instance's SQLite — NFS PVC (sifaka) vs. local PV. (SQLite over NFS has known locking caveats.) +- ❓ How do clients reach the central instance — Tailscale tailnet only, or public via Caddy/Fly proxy? + +## 8. Technology Choices (to decide) + +- ❓ nvim plugin stack: Lua + JSON-RPC over a socket to `hephd` (with daemon→plugin push) vs. CLI shell-out. Telescope as the picker dependency. +- ❓ Web framework (Axum + HTMX/Askama? Leptos? Dioxus?). +- ❓ SQLite access layer (sqlx, rusqlite, SeaORM, Diesel) and migrations. +- ❓ CRDT / sync library (Automerge-rs, Yrs, bespoke op-log). +- ❓ Markdown parser/renderer (pulldown-cmark, comrak) and wiki-link / checkbox extraction. + +## 9. Open Questions Summary + +### Decided + +- ✅ Source of truth: **SQLite-primary**, markdown as the body payload, with a directory `export` mode. Clean break from Obsidian-the-app. +- ✅ Note/task model: **typed node graph** — thin `task` nodes, rich `doc` nodes, connected by typed links. +- ✅ Front-end: **per-device `hephd` daemon** with thin clients; **nvim plugin** as primary editing UX, **CLI** as everyday driver, **web UI** on the hub. +- ✅ Sync (shape): **hub-and-spoke, op-log + CRDT**, HLC ordering, OIDC/Authentik auth, conflict queue for the ambiguous remainder. (Details below still open.) + +### Decided since (B/C/D) + +- ✅ Context surfacing v1: **explicit links only** + auto-created canonical context doc per task + nvim fuzzy search. Inferred/semantic deferred. +- ✅ **v1 is distributed, not local-only:** client/server + **offline-first sync with automatic merge + conflict queue** + **full OIDC/Authentik auth with per-user isolation** are all IN v1 (tech-spec §12/§13). Web UI and k3s deployment are later (hub binary is built deployable). +- ✅ Security v1: **OIDC/Authentik auth + per-user isolation** is the boundary; plain SQLite at rest (no encryption). May revisit. +- ✅ Migration v1: **clean break**, no ZK import / no Todoist bridge. heph is system of record day one. +- ✅ **License: All Rights Reserved / private.** Open-sourcing considered later. + +### Decided (workflows) + +- ✅ heph organized around first-class workflows (§6): **Task** (what-is-next), **Log**, **Wiki/KB**, **Calendar**. +- ✅ **Log** (§6.4) in scope: daily journal (picker over dated `journal` nodes, à la `zkd`/`Obsidian dailies`) + optional **append-only per-task work-logs** (separate buffer from the canonical context doc; dispatchable without leaving the context buffer). +- ✅ **Wiki/KB** (§6.5) first-class: heph.nvim reaches **obsidian.nvim feature parity** (follow `[[links]]` on ``, telescope search/quick-switch/tags/backlinks/links) and **replaces** it. +- ✅ **Calendar** (§6.6) deferred but **scaffolded carefully**: never explode recurrence into stored events (Todoist-gcal-flood / Hermes lesson); keep tasks vs. events separate; lazy RRULE expansion; design for a future read-only CalDAV ingest. + +### Decided (workflow model) + +- ✅ "What is next?" rests on the **lived priority discipline** (§6.2): projects-as-contexts; attention-states White/Orange/Red/Blue (Red = *consequence*, not severity); **do-date not due-date**; working-set tensions surfaced honestly (avoid "fake happy" *and* overwhelm). +- ✅ **Tactical / Strategic / Organizational** reaffirmed as **named modes/views** in the nvim plugin, sharing a **goal-stack** primitive (§6.1). +- ✅ **Commitment axis** (§6.3): committed tasks (in §6.2 system) vs. ephemeral context items (outstanding/not-outstanding, scoped to a container); **promotion** is first-class; **no hard deletes** (tombstones + cleanup mode). +- ✅ Task end-states: **done** vs. **dropped/dismissed** (both "not outstanding"). +- ✅ Prototype goes **straight to the nvim plugin + Rust server**; CLI is secondary. +- ✅ **Rust stack RATIFIED** (§11.2): rusqlite + tokio/JSON-RPC + pulldown-cmark + ulid + rrule + clap. +- ✅ **Recurrence + recurring checklists IN SCOPE for v1** (§3.3): RRULE-driven; **each occurrence gets a fresh checklist instance, completion never carries forward** (avoids Todoist's corner). + +### Still open + +*Prototype-blocking (resolve at kickoff — see §11 / tech-spec):* + +1. **Recurrence model (§3.3): (a) occurrence-instances vs (b) roll-forward-in-place** — both avoid the corner; leaning (a). +2. **CRDT library** for body merge: `yrs` (leaning) vs `automerge`; bespoke op-log/HLC for scalar fields either way. +3. **Hub network transport** (tech-spec §6.1): `axum` HTTP/JSON (leaning) vs gRPC; sync propagation cadence (push vs periodic pull). +4. **Context-item editing surface (§6.3): Option A vs B** — *decided inside the prototype by feel* (shared backend). + +*Not prototype-blocking (later phases):* + +5. Per-task **log** representation: dedicated append-only `log` node kind vs. `doc` flagged append-only (§6.4). +6. Calendar protocol confirmation (CalDAV vs ICS/WebCal) and the careful, non-exploding integration (§6.6). +7. Web front-end style (SSR + HTMX vs. WASM SPA). +8. k3s deployment specifics: Rust container build (Nix vs Dagger-native cargo); central SQLite persistence (local PV vs NFS); client reachability (tailnet-only vs public). +9. Encryption at rest / E2E (currently out; may revisit). + +## 10. Roadmap (provisional) + +- **Phase 0 — Design** (this document + [[tech-spec]]): done enough to build. +- **Phase 1 — v1 prototype (one C1 effort, built in TDD slices):** `heph-core` (model, schema, extraction, recurrence, "what is next", op-log/HLC/CRDT merge) → `hephd` client mode → **hub mode + offline sync + conflict queue** → **OIDC/Authentik auth + per-user isolation** → `heph.nvim` + `heph` CLI. Runnable client/server, offline-capable, on the tailnet. +- **Phase 2 — k3s deployment:** Dagger→Zot image, ArgoCD app + Kustomize manifests, external-secrets; hub on blumeops. +- **Phase 3 — Web UI** on the hub. +- **Phase 4 (later, optional)** — calendar integration (careful CalDAV); migration from ZK / Todoist; iOS / Apple Watch voice capture; Hermes-style planning mode. + +## 11. v1 Prototype — Scope, Stack, and Kickoff + +> The handoff target for the next context session. Items here are **proposals to ratify**; once ratified, a fresh session can build directly from this. + +> **The canonical, implementation-ready scope/stack/schema/API now live in [[tech-spec]].** This section keeps the *intent* and the kickoff process; defer to the spec for details (and update the spec when decisions change). + +### 11.1 Scope — *distributed from day one* (ratified) + +v1 is **not** local-only. It is a **client/server, offline-first** system: + +- **In:** the full data model (§3, §6) incl. **recurrence + recurring checklists** (§3.3); the "what is next?" engine; **client/server architecture** (per-device client `hephd` + a runnable/deployable hub `hephd`); **offline-first op-log + CRDT sync with automatic merge and a conflict queue** (tech-spec §12); **OIDC/Authentik auth with per-user isolation** (tech-spec §13); the `heph.nvim` + `heph` CLI surfaces. +- **Out (later, scaffolded):** the **web UI** (hub serves sync only in v1); **actual k3s deployment** to blumeops (hub binary is built deployable; ArgoCD/Kustomize/Dagger is a fast-follow); calendar; iOS/Watch; inferred context; P2P-over-tailnet fallback; persistent goal stack (an in-plugin session stack suffices). + +**Why this bigger v1:** the distributed/offline/merge behavior and the auth model are *architecturally load-bearing* — bolting them on later would mean reworking the core. The owner explicitly wants them proven from the start. (Recurrence-with-checklists is in for the same reason — see §3.3.) + +### 11.2 Stack — ✅ RATIFIED + +Core: `rusqlite` (bundled) + migrations · `tokio` + JSON-RPC/unix-socket (local) · `ulid` · `rrule` · `pulldown-cmark` · `clap` · `anyhow`/`thiserror` · `tracing`. Distributed/auth additions (some to confirm at kickoff): text CRDT `yrs` (vs `automerge`) for bodies + bespoke op-log/HLC for fields · hub transport `axum`/`reqwest` · OIDC via `openidconnect` + `keyring`. Full detail and the SQLite schema: **tech-spec §4.5, §10, §12, §13**. + +### 11.3 First-session kickoff checklist + +1. `mise run ai-docs`; read **[[tech-spec]]** (the build spec) and this doc's §3/§6 for rationale. +2. **Classify as C1** — greenfield, no existing system to Mikado-untangle. Single long-lived feature branch + early draft PR, docs-first, push as you go ([[agent-change-process]]). +3. Scaffold the cargo workspace + `heph.nvim` skeleton; **fill the AGENTS.md Project Structure section** (last template TODO). +4. Build outward in testable slices (TDD, tech-spec §2/§9): `heph-core` (schema, model, extraction, recurrence, "what is next", **op-log/HLC/CRDT merge**) → `hephd` client mode (local RPC) → **hub mode + sync + conflict queue** → **OIDC auth** → `heph.nvim` + `heph` CLI. +5. Confirm the kickoff-open picks: recurrence model (a vs b, §3.3), CRDT lib (`yrs` vs `automerge`), hub transport, context-item editing surface (A vs B, §6.3). + +## Related + +- [[tech-spec]] — clean implementation-facing technical specification distilled from this document +- [[ai-assistance-guide]] — conventions for AI agents in this repo +- [[agent-change-process]] — C0/C1/C2 change methodology diff --git a/docs/explanation/explanation.md b/docs/explanation/explanation.md index 2eba5e8..b7f4d9d 100644 --- a/docs/explanation/explanation.md +++ b/docs/explanation/explanation.md @@ -10,4 +10,4 @@ tags: Background context and design decisions. - +- [[design]] — Hephaestus design document: vision, data model, architecture, sync, and roadmap diff --git a/docs/quartz.config.ts b/docs/quartz.config.ts index 67d1ae3..a0f1af0 100644 --- a/docs/quartz.config.ts +++ b/docs/quartz.config.ts @@ -13,7 +13,7 @@ const config: QuartzConfig = { enablePopovers: true, analytics: null, locale: "en-US", - baseUrl: "CHANGEME.example.com", // TODO: Update to your docs site URL + baseUrl: "localhost", // TODO: set to hosted docs domain once published (see blumeops docs.eblu.me pattern) ignorePatterns: ["private", "templates", ".obsidian"], defaultDateType: "modified", theme: { diff --git a/docs/reference/reference.md b/docs/reference/reference.md index 7045557..1117159 100644 --- a/docs/reference/reference.md +++ b/docs/reference/reference.md @@ -10,11 +10,15 @@ tags: Technical reference material for the repository tooling that ships with this project. +## Project + +- [[tech-spec]] — Hephaestus technical specification (data model, RPC API, "what is next?" ranking, recurrence, testing strategy, v1 scope) + ## Template Surface Area | Path | Purpose | |------|---------| -| `.dagger/src/project_template_ci/` | Dagger module that builds the Quartz docs tarball used by releases | +| `.dagger/src/hephaestus_ci/` | Dagger module that builds the Quartz docs tarball used by releases | | `.forgejo/workflows/build.yaml` | Generic CI validation workflow | | `.forgejo/workflows/release.yaml` | Manual release workflow that versions, builds docs, and publishes release assets | | `.forgejo/scripts/` | Optional project-specific hooks consumed by the workflows | @@ -64,5 +68,4 @@ Technical reference material for the repository tooling that ships with this pro ## TODO After Templating -- TODO: Set `baseUrl` in `docs/quartz.config.ts` to the hosted docs domain, or `localhost` if the docs are only previewed locally -- TODO: Rename `.dagger/src/project_template_ci/` and the exported Dagger class to match the generated project +- TODO: Set `baseUrl` in `docs/quartz.config.ts` to the hosted docs domain once published (currently `localhost`) diff --git a/docs/reference/tech-spec.md b/docs/reference/tech-spec.md new file mode 100644 index 0000000..194b155 --- /dev/null +++ b/docs/reference/tech-spec.md @@ -0,0 +1,301 @@ +--- +title: Technical Specification +modified: 2026-05-31 +tags: + - reference + - design +--- + +# Hephaestus — Technical Specification + +> Clean, implementation-facing spec for the v1 prototype. For the *why* behind every choice (history, prior art, decision trail), see [[design]]. Where this spec and the design doc disagree, the design doc's latest decision wins — file an update here. + +## 1. Overview + +Hephaestus (heph) is a self-hosted personal context-management system that unifies a markdown knowledge base with task management in one database. **v1 is a distributed client/server system**: each device runs a local replica that works fully **offline**, and syncs to a central **hub** when reachable, with **automatic merge + conflict resolution** for concurrent offline edits. Access is authenticated via **OIDC (Authentik)** with per-user data isolation. The web UI and the actual k3s deployment are later phases; the hub ships in v1 as a runnable/deployable binary. Components: + +- **`heph-core`** — Rust library: data model, SQLite store, query engine, markdown parsing/extraction, recurrence, and the sync engine (op-log, HLC, CRDT merge, conflict detection). +- **`hephd`** — Rust daemon with two modes: + - **client mode** (per device): owns the local SQLite replica; serves a JSON-RPC API over a unix socket to local surfaces; runs background sync to the hub. + - **server/hub mode**: owns the central SQLite; serves the authenticated sync endpoint (and, later, the web UI) over the network. +- **`heph`** — Rust CLI: utility/admin surface (export, scripting, smoke tests, `heph conflicts`). +- **`heph.nvim`** — Lua Neovim plugin: the primary user surface ("org-mode"-style); a thin client of the local `hephd`. + +## 2. Development approach + +**Development is test-driven (TDD).** Write the failing test first; implement to green; refactor. No feature is "done" without tests at the appropriate layer(s) in §9. Core logic must be **deterministic and clock-injected** (no ambient wall-clock reads in `heph-core`; the current time is always passed in) so ranking and recurrence are testable. + +## 3. Architecture + +- Surfaces (`heph.nvim`, `heph` CLI) are thin clients; they never touch SQLite directly. All state operations go through the local `hephd` over the unix socket. Socket path default: `$XDG_RUNTIME_DIR/heph/hephd.sock`. +- Each device's `hephd` (client mode) is the single writer/owner of that device's local SQLite replica (WAL mode), and works **fully offline**. It records every local mutation as an op in an append-only **op-log** and runs **background sync** to the hub. +- The **hub** `hephd` (server mode) is the **hub-and-spoke rendezvous**: devices push/pull ops to it over the network (authenticated, §13). It is *not* required to be online — devices keep working and reconcile when it returns. +- Merge is automatic where possible; the unresolvable remainder goes to a **conflict queue** surfaced to the user (§12). Sync never blocks local work. +- `heph-core` is synchronous and side-effect-light (incl. deterministic merge logic); `hephd` wraps it with async I/O, transport, and auth (`tokio`). DB calls run on a blocking pool. +- See **§12 (Sync & Conflict Resolution)** and **§13 (Authentication)** for the detailed models. + +## 4. Data model + +All first-class entities are **nodes**; relationships are **links**. Markdown bodies are stored in SQLite; files are an export artifact, not the source of truth. + +### 4.1 Node kinds + +| kind | meaning | body | +|---|---|---| +| `doc` | rich context document (knowledge base, work-logs, journals) | markdown | +| `task` | thin task or ephemeral context item (see §4.3) | none (context via links) | +| `project` | grouping/context for tasks | optional | +| `tag` | label | optional | +| `journal` | daily note, titled by ISO date | markdown | + +### 4.2 Link types + +`wiki` (materialized from `[[links]]` in a body), `canonical-context` (task → its auto-created context doc), `context-of`, `log-of` (task → its append-only log), `blocks`, `parent`, `tagged`, `in-project`. + +### 4.3 Task semantics + +- **Attention-state** (required on committed tasks): `white` (do once do-date arrives), `orange` (top of mind), `red` (top of mind + a consequence exists if late — *consequence, not severity*), `blue` (on-deck/backlog). +- **do-date** = *earliest actionable date* ("do date"), **not a deadline**. Optional **late-on** marks when lateness becomes a problem. +- **Commitment axis:** `committed = 1` tasks participate in scheduling/"what is next"; `committed = 0` are **ephemeral context items** scoped to a container (`container_id`), with only `outstanding`/done states, never surfaced globally. Context items may be **promoted** to committed tasks. +- **States:** `outstanding`, `done`, `dropped` (done and dropped are both "not outstanding"; the distinction is retained). +- **No hard deletes:** everything uses `tombstoned`; physical deletion only in an explicit cleanup mode. + +### 4.4 Recurrence (§3.3 of [[design]]) + +A `task` with a non-null `recurrence` (RFC-5545 RRULE) is a **recurring definition**. Sub-items flagged `is_template = 1` form its **checklist template**. **Each occurrence produces a fresh checklist instance** (new `outstanding` context items copied from the template); **completion never carries forward across occurrences** — this is a hard requirement. + +Two candidate implementations (pick at kickoff; (a) is the lean): +- **(a) Occurrence instances:** definition spawns a `task_occurrences` row per occurrence, each with its own do-date and fresh checklist items. Full history. +- **(b) Roll-forward in place:** single node; on completion, log the occurrence, reset the checklist to `outstanding`, advance the do-date. + +### 4.5 SQLite schema (starting point) + +``` +nodes( + id TEXT PRIMARY KEY, -- ULID + owner_id TEXT NOT NULL REFERENCES users(id), -- per-user isolation + kind TEXT NOT NULL, -- doc|task|project|tag|journal + title TEXT NOT NULL, + body TEXT, -- markdown (nullable); materialized view of body_crdt + body_crdt BLOB, -- text-CRDT state for the body (merge), nullable + created_at INTEGER NOT NULL, -- epoch ms + modified_at INTEGER NOT NULL, + hlc TEXT NOT NULL, -- hybrid logical clock of last write (sync ordering) + tombstoned INTEGER NOT NULL DEFAULT 0 +) + +tasks( + node_id TEXT PRIMARY KEY REFERENCES nodes(id), + attention TEXT, -- white|orange|red|blue (committed tasks) + do_date INTEGER, -- epoch ms, nullable + late_on INTEGER, -- epoch ms, nullable + state TEXT NOT NULL, -- outstanding|done|dropped + committed INTEGER NOT NULL, -- 1 committed task, 0 context item + container_id TEXT REFERENCES nodes(id), -- context item → container + recurrence TEXT, -- RRULE; present = recurring definition + is_template INTEGER NOT NULL DEFAULT 0 -- checklist-template item +) + +-- recurrence model (a) only: +task_occurrences( + id TEXT PRIMARY KEY, -- ULID + def_id TEXT NOT NULL REFERENCES nodes(id), + occurrence_date INTEGER NOT NULL, + state TEXT NOT NULL, -- outstanding|done|dropped|skipped + created_at INTEGER NOT NULL, + tombstoned INTEGER NOT NULL DEFAULT 0 +) + +links( + id TEXT PRIMARY KEY, -- ULID + src_id TEXT NOT NULL REFERENCES nodes(id), + dst_id TEXT NOT NULL REFERENCES nodes(id), + type TEXT NOT NULL, + created_at INTEGER NOT NULL, + tombstoned INTEGER NOT NULL DEFAULT 0 +) + +aliases(node_id TEXT REFERENCES nodes(id), alias TEXT) -- wiki-link name resolution +nodes_fts -- FTS5 over title, body + +-- identity & sync -- +users( + id TEXT PRIMARY KEY, -- ULID + oidc_sub TEXT UNIQUE NOT NULL, -- OIDC subject (Authentik) + name TEXT, + created_at INTEGER NOT NULL +) + +oplog( -- append-only operation log (the sync unit) + id TEXT PRIMARY KEY, -- ULID + owner_id TEXT NOT NULL REFERENCES users(id), + hlc TEXT NOT NULL, -- hybrid logical clock (causal order) + origin TEXT NOT NULL, -- originating device id + op_type TEXT NOT NULL, -- node.create|node.body_delta|task.set_field|link.add|link.remove|... + target_id TEXT NOT NULL, + payload TEXT NOT NULL, -- JSON (e.g. CRDT delta, field+value, OR-Set add/remove) + applied INTEGER NOT NULL DEFAULT 0 +) + +sync_state( -- per-peer cursor (device ↔ hub) + peer TEXT PRIMARY KEY, -- 'hub' on a client; device id on the hub + last_pushed_hlc TEXT, + last_pulled_hlc TEXT, + updated_at INTEGER NOT NULL +) + +conflicts( -- ambiguous merges surfaced to the user + id TEXT PRIMARY KEY, + owner_id TEXT NOT NULL REFERENCES users(id), + node_id TEXT NOT NULL REFERENCES nodes(id), + field TEXT NOT NULL, -- which field / 'body-region' + local_val TEXT, remote_val TEXT, + local_hlc TEXT, remote_hlc TEXT, + status TEXT NOT NULL, -- open|resolved + created_at INTEGER NOT NULL +) +``` + +Projects/tags are `nodes`; membership is `links` (`in-project`, `tagged`). All `tasks`/`task_occurrences`/`links` rows inherit ownership via their node(s). + +## 5. Markdown handling + +- Bodies are stored verbatim. On write (node create/update), `heph-core` extracts: + - `[[wiki-links]]` → `wiki` links (resolved via `aliases`/title; unresolved links are allowed and recorded). + - GFM task-list items (`- [ ]` / `- [x]`) → context-item state under the node (Option A editing model — see [[design]] §6.3; the alternative is command-driven items, decided in-prototype). +- Extraction is idempotent and diff-based: re-writing an unchanged body is a no-op; reworded checklist lines tombstone-old + add-new (context items are cheap). +- `export` materializes all non-tombstoned nodes to a directory tree of `.md` files (frontmatter + body), reproducing the corpus portably. + +## 6. Daemon RPC API (JSON-RPC over unix socket) + +Methods (request → response; errors are JSON-RPC errors). Signatures are indicative, not final: + +- `node.get(id) → Node` +- `node.create({kind, title, body?}) → Node` +- `node.update({id, title?, body?}) → Node` (body update re-runs extraction) +- `node.tombstone(id) → ok` +- `task.create({title, project?, attention?, do_date?, late_on?, recurrence?, committed?}) → Task` (auto-creates the canonical context `doc` + `canonical-context` link) +- `task.set_state({id, state}) → Task` (recurring: advances per §4.4) +- `task.set_attention({id, attention}) → Task` +- `task.promote({context_item_id, attention?, project?}) → Task` +- `next({scope?, limit?}) → [RankedTask]` (the "what is next?" query, §7) +- `search({query, filters?}) → [Node]` (FTS) +- `links.outgoing(id) → [Link]` / `links.backlinks(id) → [Link]` +- `journal.open_or_create(date) → Node` +- `log.append({task_id, text}) → ok` (append to the task's `log-of` node) +- `export({path}) → {count}` +- `health() → {orange_count, active_count, on_deck_count, conflict_count, sync_status, ...}` (working-set + sync indicators) +- **auth:** `auth.login() → {device_code_flow...}` / `auth.status() → {user, logged_in}` / `auth.logout()` (§13) +- **sync:** `sync.now() → {pushed, pulled, conflicts}` (force a sync cycle; background sync runs automatically) / `sync.status() → {last_pushed, last_pulled, pending_ops, online}` +- **conflicts:** `conflicts.list() → [Conflict]` / `conflicts.resolve({id, choice}) → ok` + +Every local mutation method records an op in the **op-log** for sync. The local daemon supports **server-push** notifications (e.g. a node changed by an incoming sync) so an open buffer can reconcile in real time. + +### 6.1 Hub (server-mode) sync endpoint + +Separate from the local unix-socket RPC, the hub exposes an **authenticated network endpoint** (HTTP/JSON or gRPC — pick at kickoff) for op exchange: clients present an OIDC bearer token (§13); the hub validates, scopes by `owner_id`, accepts pushed ops, and returns ops the client hasn't seen (by HLC cursor). The hub applies the same `heph-core` merge logic. + +## 7. "What is next?" ranking + +Given an optional `scope` (project/context) and `limit` (default 5): + +1. **Candidates:** committed tasks where `state = outstanding`, not tombstoned, `attention ≠ blue`, and actionable now: `do_date IS NULL OR do_date ≤ now`. For recurring definitions, evaluate the current active occurrence's do-date. Apply `scope` if given. +2. **Order:** + 1. attention: `red` → `orange` → `white`; + 2. urgency: tasks past `late_on` first, then most-overdue (smallest `do_date`) first; + 3. tie-break: earlier `do_date`, then `created_at`. +3. **Output:** concise rows — title, project, attention, do/late, link to canonical context. `red` items always appear regardless of `limit`. + +`blue` (on-deck) is hidden from `next` by design; surfaced only by an explicit on-deck view. `health()` exposes the working-set tensions (orange ≤ 6, active ≤ ~30, on-deck count) honestly — never masking overload nor manufacturing calm. + +## 8. heph.nvim surface (v1) + +Replaces obsidian.nvim. Telescope-backed. Core commands/gestures: + +- Follow `[[wiki-link]]` under cursor on ``. +- Search / quick-switch / tags / backlinks / outgoing links (pickers). +- Daily journal picker (create/open dated `journal` nodes). +- Task capture; show "what is next" (`:Heph next`); set attention; mark done/dropped. +- Open a task's canonical context doc; edit context-item checkboxes (Option A) in the buffer (extracted on `:w`). +- Per-task log quick-append without leaving the current buffer. + +## 9. Testing strategy (TDD, layered) + +All layers are required; CI runs them on every push/PR (extend `.forgejo/scripts/build` to run `cargo test` and the nvim e2e suite; `prek` already runs in `build.yaml`). + +- **Unit (`heph-core`):** model invariants; markdown extraction (wiki-links, checkboxes); **RRULE expansion and the fresh-checklist-per-occurrence rule** (assert completion never carries forward); the "what is next?" ranking (table-driven cases); migration up/down. +- **Property tests (`proptest`):** ranking yields a total order; extraction is idempotent; recurrence never leaks completion state across occurrences; tombstones are never resurrected. +- **Integration (`hephd`, real sockets):** start a daemon against a **temp SQLite file**, connect over a **real unix socket**, and exercise the RPC API end-to-end, asserting resulting DB state. Include **multi-client concurrency** tests on the socket and clock-injection for deterministic time. +- **Sync & offline (multi-replica):** spin up **two client `hephd` replicas + a hub `hephd`**, all over **real network sockets** against temp DBs, and assert convergence: + - online round-trip: edit on A → appears on B via the hub; + - **offline → reconcile:** partition A and B from the hub, make divergent edits on each, reconnect, assert both converge; + - **conflict path:** concurrent conflicting scalar edits (e.g. both set a different do-date) land in the **conflict queue** and `conflicts.resolve` settles them deterministically; + - **body CRDT merge:** concurrent edits to the same `doc` body auto-merge without a hard conflict; + - HLC ordering and op-log idempotency (replaying ops is a no-op). +- **Auth:** OIDC token validation on the hub endpoint (reject missing/invalid/expired); **per-user isolation** (user A cannot read/sync user B's nodes); device-code flow happy path against a mock IdP. +- **End-to-end (headless nvim):** drive `heph.nvim` in `nvim --headless` against a real `hephd` + temp DB, running **scripted example workflows** and asserting outcomes (via RPC/DB state and buffer contents). Minimum workflows: + - capture a task → appears in `:Heph next` → open canonical context → add a checklist item → check it → mark task done; + - create today's journal via the picker; + - follow a `[[link]]` on `` to the target doc; + - **a recurring task with a checklist: complete it, then assert the next occurrence presents a fresh, all-unchecked checklist;** + - **a sync-driven update arrives while a buffer is open and the buffer reconciles.** + - Harness: `plenary.nvim`/busted, or drive nvim via its msgpack-RPC from the test runner. Keep example workflows as reusable fixtures. +- **CLI tests:** invoke `heph` subcommands against a temp DB; snapshot output; assert `export` round-trips the corpus; `heph conflicts` lists/resolves. + +## 10. Technology stack (ratified) + +`rusqlite` (bundled) + migration runner · `tokio` + line-delimited JSON-RPC over unix socket · `ulid` · `rrule` · `pulldown-cmark` · `clap` · `anyhow`/`thiserror` · `tracing`. Neovim plugin in Lua, depending on `telescope.nvim`. Cargo workspace: `crates/heph-core`, `crates/hephd`, `crates/heph`, plus `heph.nvim/`. + +**Added for v1 client/server + auth (some to confirm at kickoff):** + +- **Text CRDT (body merge):** `yrs` (Rust Yjs) — *leaning*; alternative `automerge`. Used for `doc`/`journal`/log bodies. Structured fields use a bespoke op-log + HLC (no library needed). +- **HLC:** small bespoke hybrid-logical-clock (or a crate) — deterministic, clock-injected. +- **Hub network transport:** `axum` (HTTP/JSON) for the sync endpoint — *leaning* (reuses the eventual web-UI server); `reqwest` on the client side. +- **OIDC:** `openidconnect` crate for the Authentik device-code flow; tokens cached in the OS keychain (`keyring`) / 1Password. + +## 11. v1 scope + +**In:** + +- The full data model, markdown handling, "what is next?" ranking, and **recurrence + recurring checklists** (§4–§8). +- **Client/server architecture:** per-device client `hephd` + a central hub `hephd` (runnable/deployable binary). +- **Offline-first** operation with **op-log + CRDT sync** and **automatic merge + a conflict queue** (§12). +- **OIDC/Authentik authentication** with **per-user data isolation** (§13). +- `heph.nvim` + `heph` CLI surfaces (incl. `heph conflicts`). + +**Out (later phases, scaffolded so as not to block):** + +- **Web UI** (the hub serves sync only in v1; reserve `axum` for it later). +- **Actual k3s deployment** to blumeops (Dagger→Zot image, ArgoCD app + Kustomize manifests, external-secrets) — fast-follow once the architecture is proven; the hub binary is built to be deployable. +- **Calendar** integration (read-mostly CalDAV; **never explode recurrence into stored events**), iOS/Watch capture, inferred/semantic context, P2P-over-tailnet sync fallback. + +See [[design]] §5–§7 for the constraints later phases impose on present choices (keep tasks vs. calendar events separate; expand RRULEs lazily). + +## 12. Sync & conflict resolution + +**Topology:** hub-and-spoke. Each device holds a full local replica + op-log; the hub is the rendezvous. Devices **push/pull ops by HLC cursor**; the hub never needs to be online for local work. + +**Merge semantics (the unit of sync is the op):** + +- **`doc`/`journal`/log bodies:** **text CRDT** (`yrs`) → concurrent edits always merge, no hard conflict. +- **Scalar task fields** (attention, do_date, late_on, state, …): **last-writer-wins by HLC**. The losing value, if meaningful, is recorded in `conflicts` (surfaced, not silently dropped). +- **Links / set membership** (tags, project, parent): **OR-Set** add/remove semantics → no conflicts. +- **Tombstones**, never hard deletes → deletion/merge is monotonic and CRDT-friendly. + +**Conflict queue:** the unresolvable/ambiguous remainder (a discarded LWW value on a meaningful field; flagged overlapping body regions) becomes an `open` row in `conflicts`. Surfaced via `health().conflict_count`, `conflicts.list`, `heph conflicts`, and a heph.nvim view: *"you have N conflicts."* `conflicts.resolve({id, choice})` settles each. **Sync never blocks** on conflicts. + +**Determinism:** HLCs are clock-injected; op application is idempotent and order-independent given HLC. These are the core invariants the sync tests assert. + +**Open at kickoff:** CRDT lib confirmation (`yrs` vs `automerge`); hub transport (`axum` HTTP/JSON vs gRPC); propagation cadence (push vs. periodic pull); exactly which fields are "meaningful" enough to enqueue vs. silently LWW. + +## 13. Authentication + +- **OIDC against Authentik.** Clients authenticate via the **OAuth 2.0 device-code flow** (`auth.login`); the resulting tokens are cached in the OS keychain (`keyring`) / 1Password and refreshed automatically. Offline devices operate on cached credentials. +- **Hub enforcement:** the sync endpoint requires a valid **OIDC bearer token**; the hub maps the token's `sub` to a `users` row and **scopes every op by `owner_id`**. No cross-user reads/writes. +- **Per-user isolation:** all nodes (and their dependent rows) carry `owner_id`; queries and sync are always user-scoped. In practice a single user (`eblume`), but the isolation is real from day one. +- **Local trust:** the local unix-socket RPC trusts the OS user (file-permission-scoped socket); app-level auth is for the **network** boundary (device ↔ hub). +- **At-rest:** plain SQLite in v1 (no encryption) — security boundary is auth + (eventually) network restriction. May revisit (see [[design]]). + +## Related + +- [[design]] — full design document with rationale and decision history diff --git a/docs/tutorials/ai-assistance-guide.md b/docs/tutorials/ai-assistance-guide.md index b60a6c2..822d98b 100644 --- a/docs/tutorials/ai-assistance-guide.md +++ b/docs/tutorials/ai-assistance-guide.md @@ -100,7 +100,7 @@ This project ships two Forgejo workflows: - `build.yaml` runs on pushes and pull requests targeting `main`, executes `prek run --all-files`, and then runs an optional project hook at `.forgejo/scripts/build` when present. - `release.yaml` is a manual workflow that computes the next version, optionally builds `CHANGELOG.md` from fragments, packages Quartz docs via Dagger, runs an optional `.forgejo/scripts/release` hook for extra assets, creates the Forgejo release, and pushes changelog updates back to `main` when fragments were consumed. -- TODO: Rename `.dagger/src/project_template_ci/` and the exported Dagger class during first-time setup. +The Dagger module lives at `.dagger/src/hephaestus_ci/` and exports the `HephaestusCi` class. ## Common Pitfalls