eblume/hephaestus
1
0
Fork 0
generated from eblume/project-template
Code Issues Pull requests Projects Releases 10 Packages Wiki Activity Actions

Fold second-pass design review into v1 spec
Some checks failed
Build / validate (push) Failing after 5s
Details

Browse source 
Resolve the open tensions surfaced in the pre-Phase-1 second pass over
design.md and tech-spec.md:

- Context items: Fork A index model — body markdown is the source of
  truth; context items are a locally-derived, non-synced index;
  identity is pinned at promotion. Dissolves the body-CRDT vs.
  extraction convergence problem.
- Recurrence: roll-forward in place; drop task_occurrences and
  is_template; advance to next RRULE instance after now (skip misses).
- Identity: deterministic ids for journal/tag (offline-convergent);
  ULID for content nodes and project.
- Mode/sync: orthogonal hub_url spoke capability; everyday device is
  local + hub_url, not server.
- Auth/owner: nullable oidc_sub, friction-free local user, hub-
  authoritative identity, one-time pre-first-sync adoption rewrite.
- Ranking: do_date is a boolean candidacy filter only; late_on is the
  sole urgency signal (global tier); FIFO tiebreak; order expressed as
  a reorderable named-dimension list.
- Modes are plugin-side compositions; add list() and log.tail().
- Frame v1 as a single deliberate C1; misc cleanups (export, health,
  CI nvim runner, README license).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Erich Blume 2026-05-31 18:08:47 -07:00
commit 2fcc5c0f22
4 changed files with 100 additions and 79 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AGENTS.md
View file

@ -13,7 +13,7 @@ This repository was instantiated from the `project-template` Forgejo template. S
- [x] `baseUrl` in `docs/quartz.config.ts` set to `localhost` (update once docs are hosted)
- [x] Dagger module renamed to `.dagger/src/hephaestus_ci/` (`HephaestusCi` class)
- [ ] Fill in the Project Structure section below once the Rust workspace is scaffolded
- [ ] Fill in license info in `README.md`
- [x] Fill in license info in `README.md` (All Rights Reserved / private)
Delete this section once the remaining items are resolved.

1
docs/changelog.d/+v1-spec-second-pass.doc.md Normal file
View file

@ -0,0 +1 @@
Second-pass review of the v1 design + tech spec before Phase 1: resolved context-item storage (Fork A index model), recurrence (roll-forward in place), key-unique node identity (deterministic ids for journal/tag), mode/sync orthogonality (`hub_url` spoke), the local-only owner / OIDC adoption path, the "what is next?" ranking (do-date as candidacy filter only, late-on as the sole urgency signal), and the plugin-side mode model (added `list` + `log.tail`).

48
docs/explanation/design.md
View file

@ -62,7 +62,8 @@ Today these are loosely coupled by fragile cross-links (`todoist://<task-id>` in
### 3.1 Identity
- ❓ **Leaning ULID** (sortable, sync-safe, collision-free across offline devices) as the internal primary key, with a human-facing `slug`/`alias` layer on top so `[[wiki-links]]` can be written by name. The ZK's timestamp-IDs (`1722897441-MWFE`) are ULID-like already; a migration can map them.
- 🔒 **DECIDED — ULID for content nodes** (`doc`/`task`/`project`), sortable and sync-safe, with a human-facing `slug`/`alias` layer so `[[wiki-links]]` can be written by name. The ZK's timestamp-IDs (`1722897441-MWFE`) are ULID-like already; a future migration can map them.
- 🔒 **DECIDED — deterministic ids for key-unique kinds.** Random ULIDs *diverge* when two offline replicas independently create the *same logical singleton* (today's `journal`, a `tag` by name) — duplicates that never merge. So `journal` ids derive deterministically from `(owner, ISO-date)` and `tag` ids from `(owner, normalized-name)`; independent creation then yields the *same* id and the CRDT/op-log coalesce them (two partial daily notes even merge). `project` stays a random ULID (deliberately created, renameable, tiny stable set). Tag rename = retag (no in-place key change); the normalization function is a fixed, versioned constant. See [[tech-spec]] §4.5.
### 3.2 Scalar task attributes vs. links
@ -74,18 +75,13 @@ A **recurring task** carries an **RFC-5545 RRULE** and acts as a recurring **def
> ❌ **The anti-pattern we must avoid (Todoist's mistake):** modeling a recurring task's checklist sub-items as standalone, non-recurring tasks that **carry their completion state forward** when the parent recurs. This is the corner; designing around it now is *why* recurring checklists are in v1 rather than deferred.
**heph model (proposed — one point to ratify):**
**heph model — 🔒 DECIDED (roll-forward in place):**
- The recurring definition holds a **checklist template** (the ordered set of sub-item definitions).
- Each **occurrence** materializes a **fresh checklist instance** — brand-new context items, all `outstanding` — scoped to that occurrence. **Completion state is per-occurrence and never carries forward.**
- Occurrences are retained as history (no hard deletes; ties naturally to the per-task **log**, §6.4). The "what is next?" engine surfaces the **current/active occurrence**; completing it advances to the next per the RRULE.
- The recurring definition's **checklist template** is just the `- [ ]` lines in its canonical context `doc` body (under Fork A, §6.3, the body *is* the template — no separate template entity).
- A recurring task is a **single node**. On completion: (1) append the finished occurrence to the per-task **log** (§6.4), (2) reset the body's checkboxes to unchecked (a body-CRDT edit), (3) advance the do-date to **the next RRULE instance strictly after `now`**, *skipping* missed occurrences. So a fresh, all-`outstanding` checklist each occurrence; **completion never carries forward**; a missed daily routine is **one** gently-overdue item, not a pile.
- History is narrative (the log), matching "narrative > list" (§6.1). RRULE expansion stays **lazy** — only "next instance after now" is ever computed, never the series (§6.6).
**The one fork to confirm:**
- **(a) Occurrence instances (recommended):** the definition spawns a lightweight **instance** per occurrence (its own do-date + its own fresh checklist items). Full per-occurrence history; unambiguously "a fresh instance per recurrence"; heavier (more rows).
- **(b) Roll-forward in place:** a single task node; on completion, **log the finished occurrence** (to the per-task log) and **reset the checklist to outstanding** + advance the do-date. Lighter; history is narrative (in the log) rather than queryable per-occurrence.
Both satisfy the correctness requirement (fresh checklist, no carried state). (a) is recommended for clean history and zero ambiguity; (b) is the pragmatic-lite option.
**Why not occurrence-instances?** The rejected alternative (a fresh node per occurrence) was attractive for queryable per-occurrence history, but under Fork A each occurrence would need its own body — hence its own node — turning every daily recurrence into a new node-pair: exactly the explosion §6.6 forbids. No v1 query needs occurrence rows (the ranking reads the single node's do-date), and adherence/streak stats can be reconstructed from the log later. See [[tech-spec]] §4.4.
## 4. Architecture
@ -94,7 +90,7 @@ Both satisfy the correctness requirement (fresh checklist, no carried state). (a
Layers, top to bottom:
- **Surfaces (thin clients):** the **nvim plugin (`heph.nvim`)** is the **primary surface for v1** — a full "org-mode"-style experience (markdown buffers backed by `doc` nodes; agenda / "what is next" / capture / linking / journaling as plugin commands). It is the explicit **successor to obsidian.nvim**, which the owner currently drives the ZK with (telescope picker, `<Leader>o*` verbs, `<Enter>` to follow `[[wiki-links]]`, dailies, multi-state checkboxes) and rarely uses Obsidian-proper alongside. heph.nvim must reach that feature parity (see §6.5) and replace it. The **CLI** (`heph`) is a secondary/utility surface (export, scripting, admin) — explicitly *not* a focus, though it shares the same command surface. The **web UI** is the occasional hub-served surface. A later iOS/Watch client talks to the hub directly.
- ❓ **Dual-mode binary:** a single Rust binary that can run as the **server/daemon** *or* expose the same operations as **CLI subcommands** *or* act as the **backend the nvim plugin drives**. Appealing (one codebase, one command surface) — to confirm as the packaging model.
- 🔒 **Single binary, three modes.** One Rust binary runs as `local` / `server` / `client` ([[tech-spec]] §3.1); the `heph` CLI shares the same command surface. Mode is two orthogonal axes (backend + inbound listener) plus an optional `hub_url` that makes any `local` instance a syncing **spoke** — the everyday device is `local` + `hub_url`, the hub is `server`, `client` is the online-only convenience.
- **Per-device daemon (`hephd`):** owns the local SQLite handle, the CRDT/op-log state, and background sync. All local surfaces connect to it over a local socket. This is what makes multi-surface access concurrent and safe on one SQLite file, and gives one place to run background sync.
- **Core crate (Rust lib):** data model, query engine, markdown parsing + wiki-link extraction, and sync logic. Linked by both `hephd` and the hub server.
- **Hub:** `hephd` running in "server" mode on blumeops k3s — central SQLite, the sync rendezvous point, and the web UI host. May be offline for long periods; devices keep working and reconcile when it returns.
@ -251,6 +247,8 @@ The owner's old rule — "avoid ncurses and interactive UIs; write atomic code a
**heph's default "what is next?" (Tactical blank-slate)** therefore ≈ Top of Mind (Red first — by *consequence*, then Orange) **+** White items whose do-date has arrived, scoped to the current project/context, **with Blue hidden**. Concise, honest, light.
> 🔒 **DECIDED ranking mechanics ([[tech-spec]] §7):** `do_date` is a *boolean candidacy filter only* (null ⇒ "now"), **never** an urgency input; **`late_on` is the sole urgency signal** (a global "now a problem" top tier); within a band, FIFO by `created_at` — **age never becomes urgency**. The order is expressed as a reorderable named-dimension list so it can be retuned without touching the engine.
### 6.3 Two kinds of task: commitments vs. context items
> 🔒 **DECIDED (shape).** A **commitment axis** orthogonal to the §6.2 attention-states.
@ -263,10 +261,10 @@ The owner's old rule — "avoid ncurses and interactive UIs; write atomic code a
#### Editing surface for context items — *deliberately deferred to the prototype*
> 🔑 **Key realization: this is NOT a data-model fork.** In both options below the **stored representation is identical** — context items are outstanding nodes linked to their container; the daemon owns them; markdown `export` renders them as checkboxes regardless. The choice is *purely the nvim editing affordance*, so the backend is shared and we **prototype both and pick by feel** (ergonomics can't be judged on paper).
> 🔑 **DECIDED — Fork A "index" backend (the affordance is still trialed both ways).** The stored source of truth is the `doc` **body markdown**: context items are `- [ ]` lines in it, and a context item's checked state *is* the `[ ]`/`[x]`. They are **not** synced nodes — each replica derives a **local index** from its converged CRDT body, so they converge for free (no divergent ULIDs across offline edits — the original sync trap). Identity is pinned only at **promotion**, which mints a real committed `task` node and rewrites the source line into a link to it. This collapses the hardest sync problem into "the body CRDT already converges" and matches this section's "ephemeral, low-stakes identity" intent. **What's still trialed both ways is purely the nvim *affordance*** (Option A checkboxes-in-body vs. Option B command-driven capture) — the backend is shared, so ergonomics can be judged in the prototype. See [[tech-spec]] §4.3, §5.
- **Option A — checkboxes in the body, derived on save.** Edit the task's canonical context doc as plain markdown; `- [ ]` lines are materialized into context-item state on `:w` (`BufWritePost`), reusing the wiki-link extraction machinery. **No real-time scanning needed** (debounced live updates are optional polish). Remote CRDT edits are *pushed* from the daemon to reconcile an open buffer. Item identity is low-stakes (reword = tombstone-old + add-new) since items are ephemeral; identity is pinned only at **promotion**. Most "personal org-mode"-native; millisecond capture = "type a line."
- **Option B — structured items, rendered.** Items are nodes from the start; the body stays pure prose. Render inline via **virtual text/extmarks** (no pane) or a transient **floating window** (avoids the disliked persistent cutaway); capture via a leader chord + `vim.ui.input`. Cleaner separation and trivial promotion; heavier capture; more UI to build.
- **Option B — command-driven capture, rendered.** Capture via a leader chord + `vim.ui.input` (no raw checkbox typing); the new item is still written as a `- [ ]` line into the body-backed store (Fork A), then **rendered** inline via virtual text/extmarks or a transient **floating window** (avoids the disliked persistent cutaway). Heavier capture; more UI to build; promotion identical.
- Lean: **A** for capture flow, but **build the shared node backend first** and trial both affordances in the plugin.
> **Prior-art note (multi-state checkboxes):** the owner's obsidian.nvim config uses checkbox states `" ", "x", ">", "~", "!"` (todo / done / forwarded / ~ / important). heph's task/context-item state model should accommodate richer-than-binary states (at least the done/dropped/outstanding set in §6.1; possibly forwarded/deferred).
@ -364,10 +362,18 @@ Reuse the established blumeops patterns (🔒 confirmed by repo conventions):
*Prototype-blocking (resolve at kickoff — see §11 / tech-spec):*
1. **Recurrence model (§3.3): (a) occurrence-instances vs (b) roll-forward-in-place** — both avoid the corner; leaning (a).
2. **CRDT library** for body merge: `yrs` (leaning) vs `automerge`; bespoke op-log/HLC for scalar fields either way.
3. **Hub network transport** (tech-spec §6.1): `axum` HTTP/JSON (leaning) vs gRPC; sync propagation cadence (push vs periodic pull).
4. **Context-item editing surface (§6.3): Option A vs B** — *decided inside the prototype by feel* (shared backend).
1. **CRDT library** for body merge: `yrs` (leaning) vs `automerge`; bespoke op-log/HLC for scalar fields either way.
2. **Hub network transport** (tech-spec §6.1): `axum` HTTP/JSON (leaning) vs gRPC; sync propagation cadence (push vs periodic pull); **device-id provisioning** (how a spoke mints its `origin` id and registers with the hub).
*Resolved in the second-pass review (2026-05-31), now folded into [[tech-spec]]:*
3. ✅ **Recurrence → roll-forward in place** (§3.3) — occurrence-instances rejected (would explode into a node-per-occurrence under Fork A); advance to the next RRULE instance after `now`, skipping misses.
4. ✅ **Context items → Fork A "index" backend** (§6.3) — body markdown is the source of truth; context items are a local derived index, not synced nodes; identity at promotion. The nvim *affordance* (A vs B) is still trialed in-prototype.
5. ✅ **Identity → deterministic ids for `journal`/`tag`** (§3.1); random ULID for content nodes/`project`.
6. ✅ **Mode/sync orthogonality** (§4): backend + inbound-listener axes plus optional `hub_url`; everyday device = `local` + `hub_url`.
7. ✅ **Auth/owner** (tech-spec §13): nullable `oidc_sub`, local user for local-only, hub-authoritative identity, one-time pre-first-sync adoption rewrite.
8. ✅ **Ranking** (tech-spec §7): `do_date` boolean candidacy filter only; `late_on` sole urgency signal (global tier); FIFO tiebreak; order as a reorderable named-dimension list.
9. ✅ **Modes** (§6.1) are plugin-side compositions of mode-agnostic daemon primitives; added `list` (Organizational) and `log.tail` (resumption breadcrumb).
*Not prototype-blocking (later phases):*
@ -380,7 +386,7 @@ Reuse the established blumeops patterns (🔒 confirmed by repo conventions):
## 10. Roadmap (provisional)
- **Phase 0 — Design** (this document + [[tech-spec]]): done enough to build.
- **Phase 1 — v1 prototype (one C1 effort, built in TDD slices):** `heph-core` (model, schema, extraction, recurrence, "what is next", `Store` trait, op-log/HLC/CRDT merge) → `hephd` **local mode** → **server + client modes (+ lock handoff)** → **offline sync + conflict queue** → **OIDC/Authentik auth + per-user isolation** → `heph.nvim` + `heph` CLI. Local-only works standalone; runnable client/server + offline sync on the tailnet.
- **Phase 1 — v1 prototype (a single C1 effort, deliberately — a one-shot test of delivering a high-complexity prototype from the spec; built in TDD slices):** `heph-core` (model, schema, extraction, recurrence, "what is next", `Store` trait, op-log/HLC/CRDT merge) → `hephd` **local mode** → **server + client modes (+ lock handoff)** → **offline sync + conflict queue** → **OIDC/Authentik auth + per-user isolation** → `heph.nvim` + `heph` CLI. Local-only works standalone; runnable client/server + offline sync on the tailnet. The build order doubles as the cross-session resume tracker (next un-green slice = where to resume). C2/Mikado is *not* used: it sequences prerequisites against existing code under test, and this is greenfield delivery from a complete spec; follow-up C1s or a C2 refactor come later as needed.
- **Phase 2 — k3s deployment:** Dagger→Zot image, ArgoCD app + Kustomize manifests, external-secrets; hub on blumeops.
- **Phase 3 — Web UI** on the hub.
- **Phase 4 (later, optional)** — calendar integration (careful CalDAV); migration from ZK / Todoist; iOS / Apple Watch voice capture; Hermes-style planning mode.
@ -407,10 +413,10 @@ Core: `rusqlite` (bundled) + migrations · `tokio` + JSON-RPC/unix-socket (local
### 11.3 First-session kickoff checklist
1. `mise run ai-docs`; read **[[tech-spec]]** (the build spec) and this doc's §3/§6 for rationale.
2. **Classify as C1** — greenfield, no existing system to Mikado-untangle. Single long-lived feature branch + early draft PR, docs-first, push as you go ([[agent-change-process]]).
2. **Classify as a single C1** (deliberate — a one-shot test of delivering this high-complexity prototype from the spec; C2/Mikado is for sequencing prerequisites against *existing* code, not greenfield delivery). Single long-lived feature branch + early draft PR, docs-first, push as you go ([[agent-change-process]]). The §11.1 build order is the resume tracker; the hairy slices (sync/CRDT) may spin off follow-up C1s rather than blocking the prototype.
3. Scaffold the cargo workspace + `heph.nvim` skeleton; **fill the AGENTS.md Project Structure section** (last template TODO).
4. Build outward in testable slices (TDD, tech-spec §2/§9): `heph-core` (schema, model, extraction, recurrence, "what is next", `Store` trait, **op-log/HLC/CRDT merge**) → `hephd` **local mode** (LocalStore + lock + local RPC) → **server + client modes** (network endpoint, RemoteStore, lock handoff) → **sync + conflict queue** → **OIDC auth** → `heph.nvim` + `heph` CLI.
5. Confirm the kickoff-open picks: recurrence model (a vs b, §3.3), CRDT lib (`yrs` vs `automerge`), hub transport, context-item editing surface (A vs B, §6.3).
5. Confirm the kickoff-open picks: CRDT lib (`yrs` vs `automerge`), hub transport (`axum` vs gRPC) + propagation cadence + device-id provisioning. (Recurrence = roll-forward and the context-item backend = Fork A are now decided; only the nvim *affordance* A vs B is trialed in-prototype.)
## Related

128
docs/reference/tech-spec.md
View file

@ -36,17 +36,23 @@ Hephaestus (heph) is a self-hosted personal context-management system that unifi
- **`LocalStore`** — a SQLite file opened **directly**, acquiring an **exclusive lock** on open (advisory `flock` on the DB file / a sidecar `.lock`). Refuses to start if the file is already locked.
- **`RemoteStore`** — no local file; proxies every operation to a `server` over the network.
`hephd` runs in one of **three modes**:
**Two orthogonal axes, not one knob.** A runtime is configured along two independent axes plus one optional capability:
| Mode | Backend | Network listener | Offline | Notes |
|---|---|---|---|---|
| **local** | `LocalStore` | none | yes | standalone single-host; the simplest deployment |
| **server** | `LocalStore` | yes (authenticated) | yes | fronts the file for remote clients; **the sync hub** |
| **client** | `RemoteStore` → a server | n/a | **no** | thin; online-only convenience |
1. **Backend** — `LocalStore` (own replica + op-log) vs. `RemoteStore` (proxy to a server).
2. **Inbound listener** — does it expose an authenticated network endpoint others connect to (i.e. is it a hub)?
3. **Outbound sync** *(optional, `LocalStore` only)* — an optional **`hub_url`**; when set, the instance is a **spoke** that background-syncs its op-log to that hub. Independent of whether it also listens.
**Lock handoff (the key flexibility):** `local` and `server` both take the file's exclusive lock, so **only one can own a given DB file at a time**. Kill the server → lock releases → a `local` process can open *the exact same file* and just work. Stop it → relaunch in `server` mode → remote clients reconnect. A `client` process never opens the file, so it never contends.
The named **modes** are presets over those axes:
**Where offline + sync live:** offline capability comes from a **local-backed** instance (`local` or `server`) — a full replica with its own op-log. Two *different machines* each run a local-backed instance and **sync** op-logs through a hub (a `server`-mode instance), which is where automatic merge + the **conflict queue** (§12) happen; sync never blocks local work, and the hub need not always be online. A `client`-mode instance is **online-only** (no replica, no offline) — the convenience option (e.g. on the hub host, or an always-connected client). Same-file local↔server is single-host graceful-degradation; device↔device sync uses *separate replicas* that reconcile.
| Mode | Backend | Inbound listener | `hub_url` | Offline | Role |
|---|---|---|---|---|---|
| **local** | `LocalStore` | none | optional | yes | everyday device — standalone if `hub_url` unset, a **syncing spoke** if set |
| **server** | `LocalStore` | yes (authenticated) | n/a (it *is* the hub) | yes | the sync **hub**; fronts the file and serves remote clients |
| **client** | `RemoteStore` → a server | n/a | n/a | **no** | thin online-only convenience (no replica) |
"Inbound listener: none" on `local` means *no inbound endpoint* — it does **not** preclude **outbound** sync. The everyday device (Gilbert, ringtail) runs **`local` + `hub_url`**: a full offline-capable replica that syncs to the blumeops hub. A `local` instance with no `hub_url` is the first-class, fully-offline standalone config. `client` is the escape hatch for "don't keep a replica here" (a borrowed box, CI, the future web-UI backend).
**Lock handoff (the key flexibility):** `local` and `server` both take the file's exclusive lock, so **only one can own a given DB file at a time**. Kill the server → lock releases → a `local` process can open *the exact same file* and just work. Stop it → relaunch in `server` mode → remote clients reconnect. A `client` process never opens the file, so it never contends. Each machine takes the lock on its *own* file; **device↔device sync uses separate replicas that reconcile through the hub** (§12), never a shared file.
## 4. Data model
@ -69,24 +75,28 @@ All first-class entities are **nodes**; relationships are **links**. Markdown bo
### 4.3 Task semantics
- **Attention-state** (required on committed tasks): `white` (do once do-date arrives), `orange` (top of mind), `red` (top of mind + a consequence exists if late — *consequence, not severity*), `blue` (on-deck/backlog).
- **do-date** = *earliest actionable date* ("do date"), **not a deadline**. Optional **late-on** marks when lateness becomes a problem.
- **Commitment axis:** `committed = 1` tasks participate in scheduling/"what is next"; `committed = 0` are **ephemeral context items** scoped to a container (`container_id`), with only `outstanding`/done states, never surfaced globally. Context items may be **promoted** to committed tasks.
- **do-date** = *earliest actionable date* ("do date"), **not a deadline** and **not an urgency signal** — a boolean candidacy gate only (§7). Optional **late-on** marks when lateness becomes a problem; it is the *sole* urgency signal.
- **Commitment axis (Fork A — [[design]] §6.3):** a **committed task** is a `task` node (a row in `tasks`) and participates in "what is next". A **context item** is *not* a synced node — it is a `- [ ]` line in a container `doc` body (its `[ ]`/`[x]` is its only state), a **locally-derived index** over the converged body (§5) that converges for free under sync. Identity is pinned only at **promotion**, which mints a committed `task` node and rewrites the source line into a link to it.
- **States:** `outstanding`, `done`, `dropped` (done and dropped are both "not outstanding"; the distinction is retained).
- **No hard deletes:** everything uses `tombstoned`; physical deletion only in an explicit cleanup mode.
### 4.4 Recurrence (§3.3 of [[design]])
A `task` with a non-null `recurrence` (RFC-5545 RRULE) is a **recurring definition**. Sub-items flagged `is_template = 1` form its **checklist template**. **Each occurrence produces a fresh checklist instance** (new `outstanding` context items copied from the template); **completion never carries forward across occurrences** — this is a hard requirement.
A `task` with a non-null `recurrence` (RFC-5545 RRULE) is a **recurring definition**. Its checklist is the set of `- [ ]` lines in its canonical context `doc` body — under Fork A the body *is* the template (no separate template flag). **Each occurrence presents a fresh, all-unchecked checklist**, and **completion never carries forward across occurrences** — a hard requirement.
Two candidate implementations (pick at kickoff; (a) is the lean):
- **(a) Occurrence instances:** definition spawns a `task_occurrences` row per occurrence, each with its own do-date and fresh checklist items. Full history.
- **(b) Roll-forward in place:** single node; on completion, log the occurrence, reset the checklist to `outstanding`, advance the do-date.
**Model: roll-forward in place** (decided; the occurrence-instances alternative was rejected because, under Fork A, each occurrence would need its own body and thus its own node — the explosion §6.6 forbids). A recurring task is a **single node**. On `task.set_state(done)`:
1. append the completed occurrence to the task's log (`log-of`);
2. reset the context doc's checkboxes to unchecked (a body-CRDT edit);
3. advance `do_date` to **the next RRULE instance strictly after `now`** — *skipping* missed occurrences rather than enqueuing them.
So missing a daily routine for three days leaves **one** gently-overdue item, never a pile; the gap is implied by the hole in the log, not by overdue rows. RRULE expansion is **lazy** — only "the next instance after now" is ever computed, never the series (§6.6). A `skipped` state advances the do-date the same way without logging a completion. History is narrative (the log), per "narrative > list".
### 4.5 SQLite schema (starting point)
```
nodes(
id TEXT PRIMARY KEY, -- ULID
id TEXT PRIMARY KEY, -- ULID (content nodes); deterministic fn(owner,key) for journal/tag (§3.1 of [[design]])
owner_id TEXT NOT NULL REFERENCES users(id), -- per-user isolation
kind TEXT NOT NULL, -- doc|task|project|tag|journal
title TEXT NOT NULL,
@ -98,26 +108,13 @@ nodes(
tombstoned INTEGER NOT NULL DEFAULT 0
)
tasks(
tasks( -- committed tasks only; context items live in doc bodies (§4.3)
node_id TEXT PRIMARY KEY REFERENCES nodes(id),
attention TEXT, -- white|orange|red|blue (committed tasks)
do_date INTEGER, -- epoch ms, nullable
late_on INTEGER, -- epoch ms, nullable
attention TEXT, -- white|orange|red|blue
do_date INTEGER, -- epoch ms, nullable; boolean candidacy gate only (§7)
late_on INTEGER, -- epoch ms, nullable; the sole urgency signal (§7)
state TEXT NOT NULL, -- outstanding|done|dropped
committed INTEGER NOT NULL, -- 1 committed task, 0 context item
container_id TEXT REFERENCES nodes(id), -- context item → container
recurrence TEXT, -- RRULE; present = recurring definition
is_template INTEGER NOT NULL DEFAULT 0 -- checklist-template item
)
-- recurrence model (a) only:
task_occurrences(
id TEXT PRIMARY KEY, -- ULID
def_id TEXT NOT NULL REFERENCES nodes(id),
occurrence_date INTEGER NOT NULL,
state TEXT NOT NULL, -- outstanding|done|dropped|skipped
created_at INTEGER NOT NULL,
tombstoned INTEGER NOT NULL DEFAULT 0
recurrence TEXT -- RRULE; present = recurring definition (roll-forward, §4.4)
)
links(
@ -135,7 +132,7 @@ nodes_fts -- FTS5 over title, bod
-- identity & sync --
users(
id TEXT PRIMARY KEY, -- ULID
oidc_sub TEXT UNIQUE NOT NULL, -- OIDC subject (Authentik)
oidc_sub TEXT UNIQUE, -- OIDC subject (Authentik); NULL = local/unlinked user (§13)
name TEXT,
created_at INTEGER NOT NULL
)
@ -144,7 +141,7 @@ oplog( -- append-only operation log (the sync unit
id TEXT PRIMARY KEY, -- ULID
owner_id TEXT NOT NULL REFERENCES users(id),
hlc TEXT NOT NULL, -- hybrid logical clock (causal order)
origin TEXT NOT NULL, -- originating device id
origin TEXT NOT NULL, -- originating device id (provisioning TBD at sync slice, §12)
op_type TEXT NOT NULL, -- node.create|node.body_delta|task.set_field|link.add|link.remove|...
target_id TEXT NOT NULL,
payload TEXT NOT NULL, -- JSON (e.g. CRDT delta, field+value, OR-Set add/remove)
@ -152,7 +149,7 @@ oplog( -- append-only operation log (the sync unit
)
sync_state( -- per-peer cursor (device ↔ hub)
peer TEXT PRIMARY KEY, -- 'hub' on a client; device id on the hub
peer TEXT PRIMARY KEY, -- 'hub' on a spoke; device id on the hub
last_pushed_hlc TEXT,
last_pulled_hlc TEXT,
updated_at INTEGER NOT NULL
@ -170,15 +167,17 @@ conflicts( -- ambiguous merges surfaced to the user
)
```
Projects/tags are `nodes`; membership is `links` (`in-project`, `tagged`). All `tasks`/`task_occurrences`/`links` rows inherit ownership via their node(s).
Projects/tags are `nodes`; membership is `links` (`in-project`, `tagged`). All `tasks`/`links` rows inherit ownership via their node(s). **Context items are not in this schema** — they are `- [ ]` lines in `doc` bodies (§4.3, §5); a daemon may keep a *local, non-synced* derived index of them for the plugin, rebuilt from bodies and never written to the op-log. **Deterministic ids** for `journal`/`tag` let two offline replicas that independently create the same day's journal or the same-named tag converge automatically; the tag-name normalization function is a fixed, versioned constant so every device derives byte-identical ids (tag rename = retag, not in-place key change).
## 5. Markdown handling
- Bodies are stored verbatim. On write (node create/update), `heph-core` extracts:
- **The body is the source of truth** and merges as a **text CRDT** (`body_crdt`); the `body` TEXT column is its materialized view. A full-body write (`node.update({body})`) is **diffed into the CRDT** (yrs computes the delta), so surfaces can send whole-buffer text without knowing about deltas.
- On write, `heph-core` derives from the body:
- `[[wiki-links]]` → `wiki` links (resolved via `aliases`/title; unresolved links are allowed and recorded).
- GFM task-list items (`- [ ]` / `- [x]`) → context-item state under the node (Option A editing model — see [[design]] §6.3; the alternative is command-driven items, decided in-prototype).
- Extraction is idempotent and diff-based: re-writing an unchanged body is a no-op; reworded checklist lines tombstone-old + add-new (context items are cheap).
- `export` materializes all non-tombstoned nodes to a directory tree of `.md` files (frontmatter + body), reproducing the corpus portably.
- GFM task-list items (`- [ ]` / `- [x]`) → the **local context-item index** (Fork A — see [[design]] §6.3). This index is **derived, not synced**: each replica rebuilds it from its converged CRDT body, so context items converge for free and never get divergent ULIDs. Context-item identity is pinned only at **promotion** (§6).
- Derivation is idempotent and diff-based: re-writing an unchanged body is a no-op.
- **Tombstoned nodes are excluded** from the derived index, `nodes_fts`, `next`/`list`, and `export`.
- `export` materializes all non-tombstoned nodes to a directory tree of `.md` files (**frontmatter — id, kind, task scalars, aliases, links — plus body**), a faithful one-way portable snapshot (no import in v1).
## 6. Daemon RPC API (JSON-RPC over unix socket)
@ -191,51 +190,62 @@ Methods (request → response; errors are JSON-RPC errors). Signatures are indic
- `task.create({title, project?, attention?, do_date?, late_on?, recurrence?, committed?}) → Task` (auto-creates the canonical context `doc` + `canonical-context` link)
- `task.set_state({id, state}) → Task` (recurring: advances per §4.4)
- `task.set_attention({id, attention}) → Task`
- `task.promote({context_item_id, attention?, project?}) → Task`
- `next({scope?, limit?}) → [RankedTask]` (the "what is next?" query, §7)
- `task.promote({container_id, item_ref, attention?, project?}) → Task` (mints a committed task from a context-item line and rewrites that line into a link to it, §4.3)
- `next({scope?, limit?}) → [RankedTask]` (the Tactical blank-slate "what is next?" ranking, §7)
- `list({scope?, attention?, include_blue?, include_future?, group_by?}) → [Task]` (enumeration for the Organizational view — the whole set incl. backlog)
- `search({query, filters?}) → [Node]` (FTS)
- `links.outgoing(id) → [Link]` / `links.backlinks(id) → [Link]`
- `journal.open_or_create(date) → Node`
- `log.append({task_id, text}) → ok` (append to the task's `log-of` node)
- `log.tail({task_id, n?}) → [Entry]` (read the task's latest log entries — the resumption breadcrumb, [[design]] §6.1)
- `export({path}) → {count}`
- `health() → {orange_count, active_count, on_deck_count, conflict_count, sync_status, ...}` (working-set + sync indicators)
- **auth:** `auth.login() → {device_code_flow...}` / `auth.status() → {user, logged_in}` / `auth.logout()` (§13)
- **sync:** `sync.now() → {pushed, pulled, conflicts}` (force a sync cycle; background sync runs automatically) / `sync.status() → {last_pushed, last_pulled, pending_ops, online}`
- **conflicts:** `conflicts.list() → [Conflict]` / `conflicts.resolve({id, choice}) → ok`
Every local mutation method records an op in the **op-log** for sync. The local daemon supports **server-push** notifications (e.g. a node changed by an incoming sync) so an open buffer can reconcile in real time.
The daemon is **mode-agnostic** — Tactical/Strategic/Organizational are plugin-side compositions of these primitives (§8), not daemon concepts. Every local mutation method records an op in the **op-log** for sync. The local daemon supports **server-push** notifications (e.g. a node changed by an incoming sync) so an open buffer can reconcile in real time.
### 6.1 Hub (server-mode) sync endpoint
Separate from the local unix-socket RPC, the hub exposes an **authenticated network endpoint** (HTTP/JSON or gRPC — pick at kickoff) for op exchange: clients present an OIDC bearer token (§13); the hub validates, scopes by `owner_id`, accepts pushed ops, and returns ops the client hasn't seen (by HLC cursor). The hub applies the same `heph-core` merge logic.
## 7. "What is next?" ranking
## 7. "What is next?" ranking (Tactical blank-slate)
Given an optional `scope` (project/context) and `limit` (default 5):
This is the **Tactical** ranking only; Strategic/Organizational are other plugin-side views over different primitives (§6, §8). Given an optional `scope` and `limit` (default 5), the engine is **two stages — a filter, then an order expressed as data** so the sort is trivially reorderable later.
1. **Candidates:** committed tasks where `state = outstanding`, not tombstoned, `attention ≠ blue`, and actionable now: `do_date IS NULL OR do_date ≤ now`. For recurring definitions, evaluate the current active occurrence's do-date. Apply `scope` if given.
2. **Order:**
1. attention: `red` → `orange` → `white`;
2. urgency: tasks past `late_on` first, then most-overdue (smallest `do_date`) first;
3. tie-break: earlier `do_date`, then `created_at`.
1. **Filter (candidacy)** — a pure predicate: `committed` ∧ `state = outstanding` ∧ ¬`tombstoned` ∧ `attention ≠ blue` ∧ **actionable** (`do_date IS NULL OR do_date ≤ now`) ∧ in `scope`. **`do_date` is used *only here*** — a boolean "can this be done now?" gate, never an urgency input. For a recurring task, the single rolled-forward node's `do_date` is the gate (§4.4).
2. **Order** — an **ordered list of named sort dimensions**, applied lexicographically; reordering this list (one place) reshuffles the ranking:
```
RANKING = [ PastLateOn(desc), LateOverdueAmount(desc), Attention(red>orange>white), CreatedAt(asc) ]
```
- **`late_on` is the sole urgency signal** and forms a global top tier: items past `late_on` float above everything (incl. `red`), most-past first. This is the only legitimate "overdue" — `late_on` is a deliberately-set problem marker, not an actionability date. *(Global-tier vs. within-band placement is expected to be revisited; it's just a reordering of `RANKING`.)*
- then **attention band** `red → orange → white`;
- tie-break **`created_at` ascending (FIFO)**. **Age is never urgency** — an item is not hotter for having been actionable longer; raise its attention or set a `late_on` to escalate it deliberately.
3. **Output:** concise rows — title, project, attention, do/late, link to canonical context. `red` items always appear regardless of `limit`.
`blue` (on-deck) is hidden from `next` by design; surfaced only by an explicit on-deck view. `health()` exposes the working-set tensions (orange ≤ 6, active ≤ ~30, on-deck count) honestly — never masking overload nor manufacturing calm.
`blue` (on-deck) is hidden from `next` by design; surfaced only by `list` (§6). `health()` exposes the working-set tensions (orange vs. 6, active vs. ~30, on-deck count) **with over-threshold deltas**, honestly — never masking overload nor manufacturing calm.
## 8. heph.nvim surface (v1)
Replaces obsidian.nvim. Telescope-backed. Core commands/gestures:
Replaces obsidian.nvim. Telescope-backed. **Tactical / Strategic / Organizational are named plugin-side views** composed from the mode-agnostic daemon primitives (§6) — the daemon never infers a mode. The session **goal stack** is plugin state (no persistent stack in v1; the durable re-seed is the per-task breadcrumb, `log.tail`). Core commands/gestures:
- Follow `[[wiki-link]]` under cursor on `<Enter>`.
- Search / quick-switch / tags / backlinks / outgoing links (pickers).
- Daily journal picker (create/open dated `journal` nodes).
- Task capture; show "what is next" (`:Heph next`); set attention; mark done/dropped.
- Open a task's canonical context doc; edit context-item checkboxes (Option A) in the buffer (extracted on `:w`).
- Task capture; show "what is next" (`:Heph next`, Tactical); `list` views (Organizational); set attention; mark done/dropped.
- Open a task's canonical context doc; edit context-item checkboxes (Fork A) in the buffer (derived on `:w`).
- Per-task log quick-append without leaving the current buffer.
**Deferred (fast-follow, scaffolded):** guided working-set rituals — the Blue keep/drop review and Orange daily reconfirm — are pure compositions of `list` + `set_attention` + `set_state(dropped)`; v1 ships `health()` reporting and manual review.
**Known-hard:** reconciling an incoming CRDT body delta into a *dirty* buffer (unsaved local edits, cursor position) — the §9 "update arrives while a buffer is open" case — is genuinely fiddly under Fork A; expect to iterate.
## 9. Testing strategy (TDD, layered)
All layers are required; CI runs them on every push/PR (extend `.forgejo/scripts/build` to run `cargo test` and the nvim e2e suite; `prek` already runs in `build.yaml`).
All layers are required; CI runs them on every push/PR (extend `.forgejo/scripts/build` to run `cargo test` and the nvim e2e suite; `prek` already runs in `build.yaml`). **The Forgejo runner image must provide `neovim` + `plenary.nvim`** for the headless e2e suite.
- **Unit (`heph-core`):** model invariants; markdown extraction (wiki-links, checkboxes); **RRULE expansion and the fresh-checklist-per-occurrence rule** (assert completion never carries forward); the "what is next?" ranking (table-driven cases); migration up/down.
- **Property tests (`proptest`):** ranking yields a total order; extraction is idempotent; recurrence never leaks completion state across occurrences; tombstones are never resurrected.
@ -300,11 +310,15 @@ See [[design]] §5–§7 for the constraints later phases impose on present choi
**Determinism:** HLCs are clock-injected; op application is idempotent and order-independent given HLC. These are the core invariants the sync tests assert.
**Open at kickoff:** CRDT lib confirmation (`yrs` vs `automerge`); hub transport (`axum` HTTP/JSON vs gRPC); propagation cadence (push vs. periodic pull); exactly which fields are "meaningful" enough to enqueue vs. silently LWW.
**Open at kickoff:** CRDT lib confirmation (`yrs` vs `automerge`); hub transport (`axum` HTTP/JSON vs gRPC); propagation cadence (push vs. periodic pull); exactly which fields are "meaningful" enough to enqueue vs. silently LWW; **device-id provisioning** (how a spoke mints its `origin` id and registers with the hub).
## 13. Authentication
- **OIDC against Authentik.** Clients authenticate via the **OAuth 2.0 device-code flow** (`auth.login`); the resulting tokens are cached in the OS keychain (`keyring`) / 1Password and refreshed automatically. Offline devices operate on cached credentials.
- **Auth is a network-boundary concern; ops are authenticated at *exchange*, not creation.** Creating ops needs only local trust, so losing the IdP/hub never blocks local work — ops queue in the op-log and the bearer token gates only the push/pull session. The one degraded case: the refresh token expiring during a long offline stretch → re-run the device-code flow once on reconnect, then the queued op-log drains.
- **Local-only needs no auth.** A `local` instance with no `hub_url` runs against a single **local user** (`users` row, generated `id`, `oidc_sub = NULL`) — friction-free, no IdP. `oidc_sub` is therefore nullable.
- **The hub is authoritative for user identity.** On first authentication it maps `sub` → a canonical `users.id` (creating it if new) and hands the *same* id to every device that authenticates as that human — no device-vs-device id reconciliation.
- **Adoption (local-only → authed)** is a **one-time, pre-first-sync** rewrite: on a previously-local replica's first authenticated sync, fetch the canonical `users.id`, then in one transaction rewrite `owner_id` (and the owner-embedded deterministic ids of §4.5 + the links referencing them) from the local placeholder to the canonical id. Safe precisely because it precedes any sync (no remote refs, no concurrent writers); afterward `owner_id` is immutable. Born-authed replicas stamp the canonical id from the start and never rewrite.
- **Hub enforcement:** the sync endpoint requires a valid **OIDC bearer token**; the hub maps the token's `sub` to a `users` row and **scopes every op by `owner_id`**. No cross-user reads/writes.
- **Per-user isolation:** all nodes (and their dependent rows) carry `owner_id`; queries and sync are always user-scoped. In practice a single user (`eblume`), but the isolation is real from day one.
- **Local trust:** the local unix-socket RPC trusts the OS user (file-permission-scoped socket); app-level auth is for the **network** boundary (device ↔ hub).