From 0aa7e725a550a5d7f406a787e8fd12e6dfed297a Mon Sep 17 00:00:00 2001 From: Erich Blume Date: Tue, 2 Jun 2026 19:21:05 -0700 Subject: [PATCH] docs: revise to a three-surface model (CLI/TUI/nvim) from a Todoist study MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Study of the owner's live Todoist (387 tasks, 34 hierarchical projects) grounds a surface-strategy revision: structured task fields suit CLI flags, large-set triage suits an interactive TUI, and context/KB suits nvim — so v1 adopts three surfaces, each to its strength. Supersedes the earlier "heph.nvim is the primary surface" framing. - design.md: new §6.2.1 (Todoist study) + revised §4 (three-surface model) - tech-spec §1/§8: reframe surface roles; new §8.1 (planned heph-tui agenda) - tech-spec §14: reorder remaining work (CLI-complete task surface in progress, heph-tui next, nvim navigation polish, tags/hierarchy deferred) Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/changelog.d/v1-prototype.doc.md | 1 + docs/explanation/design.md | 26 ++++++++++++++- docs/reference/tech-spec.md | 49 +++++++++++++++++++--------- 3 files changed, 59 insertions(+), 17 deletions(-) create mode 100644 docs/changelog.d/v1-prototype.doc.md diff --git a/docs/changelog.d/v1-prototype.doc.md b/docs/changelog.d/v1-prototype.doc.md new file mode 100644 index 0000000..8867085 --- /dev/null +++ b/docs/changelog.d/v1-prototype.doc.md @@ -0,0 +1 @@ +- Surface strategy revised to a **three-surface model** ([[design]] §4, tech-spec §1/§8), grounded in a study of the owner's live Todoist usage (387 active tasks, 34 hierarchical projects — [[design]] §6.2.1): the **CLI** is task capture/scripting plus the complete daemon API, a planned **`heph-tui`** terminal UI (tech-spec §8.1) is the primary task agenda/triage surface, and **`heph.nvim`** is the primary context/knowledge-base surface. The study also confirms do-date-not-due-date (zero deadlines used), that task descriptions are already full of unresolved `[[wiki-links]]` (the exact task↔KB fusion heph provides), and surfaces new needs (project hierarchy, natural-language recurrence) while showing tags are negligible. diff --git a/docs/explanation/design.md b/docs/explanation/design.md index ff7b711..54d30a3 100644 --- a/docs/explanation/design.md +++ b/docs/explanation/design.md @@ -89,7 +89,11 @@ A **recurring task** carries an **RFC-5545 RRULE** and acts as a recurring **def Layers, top to bottom: -- **Surfaces (thin clients):** the **nvim plugin (`heph.nvim`)** is the **primary surface for v1** — a full "org-mode"-style experience (markdown buffers backed by `doc` nodes; agenda / "what is next" / capture / linking / journaling as plugin commands). It is the explicit **successor to obsidian.nvim**, which the owner currently drives the ZK with (telescope picker, `o*` verbs, `` to follow `[[wiki-links]]`, dailies, multi-state checkboxes) and rarely uses Obsidian-proper alongside. heph.nvim must reach that feature parity (see §6.5) and replace it. The **CLI** (`heph`) is a secondary/utility surface (export, scripting, admin) — explicitly *not* a focus, though it shares the same command surface. The **web UI** is the occasional hub-served surface. A later iOS/Watch client talks to the hub directly. +- **Surfaces (thin clients) — a three-surface model (revised 2026-06 after the §6.2.1 Todoist study; supersedes the earlier "nvim is *the* primary surface" framing).** Tasks and knowledge pull in different interaction directions, so each surface plays to its strength rather than one trying to do everything: + - **`heph.nvim` — the primary *context / knowledge-base* surface.** The full "org-mode"-style experience (markdown buffers backed by `doc` nodes; wiki-links, journaling, the canonical-context doc, per-task log, checklists). The explicit **successor to obsidian.nvim** (telescope picker, follow `[[wiki-links]]` on ``, dailies, multi-state checkboxes); must reach that parity (§6.5). It surfaces tasks for **navigation/reading and context**, not as the primary place to *edit* structured task fields. + - **`heph` CLI — capture/scripting + the complete daemon API.** Every structured task field is a flag (`-a red --do tomorrow --late fri --recur weekly --project Maintenance`), which is exactly what command-line flags are good at and dissolves the "how do I edit N structured fields" problem. The CLI implements the *entire* API (admin, sync, conflicts, export) so it is also the scripting/automation surface. + - **`heph-tui` — the primary *task agenda / triage* surface (planned, [[tech-spec]] §8.1).** The §6.2.1 study shows the dominant task activity is *interactive triage of a large set* (daily orange reconfirm, blue keep/drop review, browse-by-project) — work that is awkward as either CLI flags or nvim buffers. A terminal UI owns that, and **launches into nvim** for a task's context (and nvim launches back). Not yet built. + - The **web UI** is the occasional hub-served surface. A later iOS/Watch client talks to the hub directly. - 🔒 **Single binary, three modes.** One Rust binary runs as `local` / `server` / `client` ([[tech-spec]] §3.1); the `heph` CLI shares the same command surface. Mode is two orthogonal axes (backend + inbound listener) plus an optional `hub_url` that makes any `local` instance a syncing **spoke** — the everyday device is `local` + `hub_url`, the hub is `server`, `client` is the online-only convenience. - **Per-device daemon (`hephd`):** owns the local SQLite handle, the CRDT/op-log state, and background sync. All local surfaces connect to it over a local socket. This is what makes multi-surface access concurrent and safe on one SQLite file, and gives one place to run background sync. - **Core crate (Rust lib):** data model, query engine, markdown parsing + wiki-link extraction, and sync logic. Linked by both `hephd` and the hub server. @@ -249,6 +253,26 @@ The owner's old rule — "avoid ncurses and interactive UIs; write atomic code a > 🔒 **DECIDED ranking mechanics ([[tech-spec]] §7):** `do_date` is a *boolean candidacy filter only* (null ⇒ "now"), **never** an urgency input; **`late_on` is the sole urgency signal** (a global "now a problem" top tier); within a band, FIFO by `created_at` — **age never becomes urgency**. The order is expressed as a reorderable named-dimension list so it can be retuned without touching the engine. +### 6.2.1 Todoist study (2026-06): empirical confirmation + new requirements + +> 📊 **Snapshot of the owner's live Todoist** (read via the blumeops `blumeops-tasks` API pattern), to ground heph against how the discipline of §6.2 *actually* manifests at scale. 387 active tasks, 34 projects. + +**Confirms the model:** + +- **Projects = contexts, used heavily and hierarchically.** 34 projects organized into top-level life-areas (`Life`, `Work`, `Coding`, `Camano`, `Culture`) → sub-projects (`Child`, `Blumeops`, `Daily Routine`, `Maintenance`, `Movies`…). The *hierarchy* is new information (§6.2 listed projects flat). +- **Huge backlog, tiny hot set.** Priority split p1=2, p2=8, p4(default)=229, p3=148 — i.e. the genuinely "hot" set is ~10, the rest is white/blue backlog. The dominant *activity* is therefore **triage of a large set**, not editing single tasks — a direct argument for an interactive agenda surface (§4, [[tech-spec]] §8.1). +- **Do-date, not due-date — validated to the hilt.** **Zero** Todoist *deadlines* are used; only 107/387 carry a due (do) date. `late_on` will be genuinely rare. +- **Recurring checklists are real** ("Prep For Day", every day @ 07:00, 6 children) — exactly the §3.3 reset-each-occurrence mechanism. +- **Daily rituals exist as tasks**: "Excess Tasks to On Deck" (= the blue keep/drop review), "Coordinate with Allison", "Check Personal Email" (= orange reconfirm). The §6.2 working-set rituals are alive in the data; the TUI should make them first-class filters. + +**The strongest validation — descriptions are already heph.** Real task descriptions contain `see [[review-services]]`, `See [[run-1password-backup]]`, `docs/how-to/plans/migrate-forgejo-from-brew`, plus URLs and phone numbers. The owner is **hand-rolling task↔knowledge-base wiki-links inside Todoist descriptions that cannot resolve.** heph's task → canonical-context-`doc` fusion (§6.3) is precisely the thing being worked around. (89/387 tasks carry a description; the task description ≡ heph's canonical context doc body.) + +**New requirements surfaced:** + +- **Project hierarchy** — a project needs an optional parent project. (Model it with the existing `parent` link; deeper hierarchy-aware `scope` is a later refinement.) +- **Natural-language recurrence** — 95/107 dated tasks recur, expressed as `every 3 days`, `every 6 months`, `every workday`, `every April 15`, `every other wed`. heph stores RFC-5545 RRULE; capture should accept the common NL forms and compile them (the easy subset; time-of-day like "at 08:00" deferred — heph's `do_date` is date-grained for ranking). +- **Tags are noise** — 7 labels, **5 task-uses across 387**. Confidently **defer** a tag surface; it is not load-bearing. + ### 6.3 Two kinds of task: commitments vs. context items > 🔒 **DECIDED (shape).** A **commitment axis** orthogonal to the §6.2 attention-states. diff --git a/docs/reference/tech-spec.md b/docs/reference/tech-spec.md index 3d8f94a..8ff1ce9 100644 --- a/docs/reference/tech-spec.md +++ b/docs/reference/tech-spec.md @@ -16,8 +16,11 @@ Hephaestus (heph) is a self-hosted personal context-management system that unifi - **`heph-core`** — Rust library: data model, the `Store` abstraction + local SQLite store, query engine, markdown parsing/extraction, recurrence, and the sync engine (op-log, HLC, CRDT merge, conflict detection). - **`hephd`** — Rust daemon; one binary, three runtime modes (`local` / `server` / `client`, §3.1). Always serves a JSON-RPC API over a local unix socket to local surfaces; in `server` mode it additionally exposes an authenticated network endpoint and runs as the sync hub. -- **`heph`** — Rust CLI: utility/admin surface (export, scripting, smoke tests, `heph conflicts`). -- **`heph.nvim`** — Lua Neovim plugin: the primary user surface ("org-mode"-style); a thin client of the local `hephd`. +- **`heph`** — Rust CLI: **task capture/scripting + the complete daemon API** (every RPC method has a command), plus admin/export/`heph conflicts`. Structured task fields are flags (`-a red --do tomorrow --recur weekly`). +- **`heph.nvim`** — Lua Neovim plugin: the primary **context / knowledge-base** surface ("org-mode"-style — docs, wiki-links, journals, the canonical-context doc, checklists); a thin client of the local `hephd`. Surfaces tasks for navigation/context, not structured-field editing. +- **`heph-tui`** *(planned, §8.1)* — Rust terminal UI: the primary **task agenda / triage** surface (the dominant task activity per [[design]] §6.2.1); launches into `heph.nvim` for a task's context and back. + +> **Surface model (revised 2026-06, [[design]] §4 / §6.2.1).** Tasks and knowledge pull in different interaction directions, so v1 uses **three surfaces**, each to its strength: **CLI** = capture/scripting + complete API; **TUI** = interactive task agenda/triage; **nvim** = context/knowledge base. This supersedes the earlier "heph.nvim is *the* primary surface" framing. ## 2. Development approach @@ -228,21 +231,33 @@ This is the **Tactical** ranking only; Strategic/Organizational are other plugin `blue` (on-deck) is hidden from `next` by design; surfaced only by `list` (§6). `health()` exposes the working-set tensions (orange vs. 6, active vs. ~30, on-deck count) **with over-threshold deltas**, honestly — never masking overload nor manufacturing calm. -## 8. heph.nvim surface (v1) +## 8. heph.nvim surface (v1) — context / knowledge base -Replaces obsidian.nvim. Telescope-backed. **Tactical / Strategic / Organizational are named plugin-side views** composed from the mode-agnostic daemon primitives (§6) — the daemon never infers a mode. The session **goal stack** is plugin state (no persistent stack in v1; the durable re-seed is the per-task breadcrumb, `log.tail`). Core commands/gestures: +Replaces obsidian.nvim. Telescope-backed. **Primarily the context / knowledge-base surface** ([[design]] §4): docs, wiki-links, journals, the canonical-context doc, checklists, per-task log. Task **agenda/triage** is the TUI's job (§8.1) and structured-field *editing* is the CLI's (flags); nvim surfaces tasks for **navigation and context**. **Tactical / Strategic / Organizational are named plugin-side views** composed from the mode-agnostic daemon primitives (§6) — the daemon never infers a mode. The session **goal stack** is plugin state (no persistent stack in v1; the durable re-seed is the per-task breadcrumb, `log.tail`). Core commands/gestures: -- Follow `[[wiki-link]]` under cursor on ``. +- Follow `[[wiki-link]]` under cursor on `` (follow-or-create). - Search / quick-switch / tags / backlinks / outgoing links (pickers). -- Daily journal picker (create/open dated `journal` nodes). -- Task capture; show "what is next" (`:Heph next`, Tactical); `list` views (Organizational); set attention; mark done/dropped. -- Open a task's canonical context doc; edit context-item checkboxes (Fork A) in the buffer (derived on `:w`). +- Daily journal picker (create/open dated `journal` nodes); home/index page. +- Show "what is next" (`:Heph next`, Tactical) and `list` views (Organizational) for **navigation** — `` jumps to a task's canonical context. *(Showing do/late in rows and a clean jump-to-context gesture is a small future polish item.)* +- Open a task's canonical context doc; edit context-item checkboxes (Fork A) in the buffer (derived on `:w`); context-item **promotion**. - Per-task log quick-append without leaving the current buffer. +- Lightweight task mutation (capture, attention, done/drop/skip) remains available, but the **primary** task-mutation surface is the CLI/TUI. The eventual richer nvim task-edit story is **frontmatter-as-edit-surface** (the task buffer presents scalars as editable YAML frontmatter parsed back on `:w` — `export` already serializes this), *not* bespoke form widgets — a later slice. -**Deferred (fast-follow, scaffolded):** guided working-set rituals — the Blue keep/drop review and Orange daily reconfirm — are pure compositions of `list` + `set_attention` + `set_state(dropped)`; v1 ships `health()` reporting and manual review. +**Deferred (fast-follow, scaffolded):** guided working-set rituals — the Blue keep/drop review and Orange daily reconfirm — are pure compositions of `list` + `set_attention` + `set_state(dropped)`; v1 ships `health()` reporting and manual review. (These become first-class **filters in the TUI**, §8.1.) **Known-hard:** reconciling an incoming CRDT body delta into a *dirty* buffer (unsaved local edits, cursor position) — the §9 "update arrives while a buffer is open" case — is genuinely fiddly under Fork A; expect to iterate. +## 8.1 heph-tui surface — task agenda / triage (planned) + +> **Status: planned, not yet built.** The §6.2.1 Todoist study shows the dominant task activity is *interactive triage of a large set* (387 active tasks; daily orange reconfirm, blue keep/drop review, browse-by-project) — work that is awkward as either CLI flags or nvim buffers. A terminal UI owns it; the CLI (capture/scripting) and nvim (context) flank it. + +- **Crate `crates/heph-tui`** — `ratatui` + `crossterm`, a **thin client of the daemon unix socket** (reuse `hephd::Client`); never touches SQLite, same as nvim. +- **Layout** — three panes: **projects/contexts** (the §6.2.1 hierarchy) · **task list** (`next`/`list` rows with attention + human do/late) · **preview** (canonical-context doc body / `log.tail`). +- **Gestures** — `j/k` move · `a` add · `x` done · `space` skip · `A` cycle attention · `e` reschedule (do/late) · `b` push-to-blue · saved filters for the **daily rituals** (Top of Mind, On-Deck review, per-project) — the [[design]] §6.2 "filters = saved views" made interactive. +- **TUI ↔ nvim handoff** — `o`/`` launches `$EDITOR` (nvim) on the task's canonical-context doc (`nvim` with a `+lua` call opening `heph://node/`, or a temp `.md` round-tripped through `node.update`); a nvim command (e.g. `:Heph agenda`) shells back to the TUI. +- **Testing** — TDD against a real daemon; headless smoke via `ratatui`'s `TestBackend`. +- **Prereqs** (surface-agnostic, landing first): the CLI-complete task surface (human dates, `list`/state/`edit`, recurrence) and `task.set_schedule` (reschedule) — both in the current slice. + ## 9. Testing strategy (TDD, layered) All layers are required; CI runs them on every push/PR (extend `.forgejo/scripts/build` to run `cargo test` and the nvim e2e suite; `prek` already runs in `build.yaml`). **The Forgejo runner image must provide `neovim` + `plenary.nvim`** for the headless e2e suite. @@ -357,14 +372,16 @@ See [[design]] §5–§7 for the constraints later phases impose on present choi **Not yet done (resume order)** -> The Rust backend is feature-complete; `heph.nvim` slices 11a–11c are done — the v1 surface (knowledge base + task views + promotion) works end-to-end with CI. The remainder is the deferred reconcile slice plus non-blocking polish + an end-of-v1 sweep (§11). +> The Rust backend is feature-complete; `heph.nvim` slices 11a–11c + a UX iteration are done. **Surface strategy revised 2026-06 to a three-surface model** ([[design]] §4 / §6.2.1, grounded in a study of the owner's live Todoist): **CLI = capture/scripting + complete API**, **TUI = primary task agenda/triage** (to build), **nvim = context/KB**. Remaining work is reordered accordingly. -1. ⏳ **Task-scheduling UX (§7, §8) — the highest-value next gap:** the plugin can only set a task's **title + attention** today. The engine fully supports **do-date / late-on / recurrence / project**, but nothing in the editor sets them — so the ranking has no urgency signals to work with from nvim. Surface these (capture-with-fields and an edit affordance), and show do/late in the `next`/`list` rows. -2. ⏳ **More surfacing of existing engine features (§6, §8):** a **backlinks/links** picker (navigate the graph back), **tags**, a **`health`** working-set view (orange-vs-6, active-vs-~30), and **log read** (`log.tail`) — the resumption breadcrumb on return. -3. ⏳ **`heph.nvim` slice 11d (§6/§8) — DEFERRED, post-parity:** daemon **server-push** notification framing (no-`id` lines on the socket; the client read-loop already demuxes them) + the **dirty-buffer reconcile** (the §8 "known-hard" case) + the "update-arrives-while-open" e2e (§9). -4. ⏳ **Split `heph.nvim` to its own forge repo (§8) — UX polish:** generated from this monorepo (subtree-split in CI) so the lazy spec becomes `{ "eblume/heph.nvim" }` instead of a local-clone `dir` (see [[install-heph]]). -5. ⏳ **Adoption refinement + multi-tenant (§13) — non-blocking:** local→authed **adoption** currently rewrites `owner_id` (`adopt_owner`) but not yet the owner-embedded deterministic ids (journal/tag) + their links; and the hub is single-tenant (one owner per store) — owner-per-token storage is a future extension. -6. ⏳ **Dependency-refresh pass (§11) — before declaring v1 done:** sweep all external deps to latest stable (e.g. `keyring` 3→4, which restructured to `keyring_core`), re-run the full suite. +1. ⏳ **CLI-complete task surface (§1, §6, §7) — IN PROGRESS (this slice):** make `heph` implement the **entire daemon API** with ergonomic task management — human date parse+display (`tomorrow`/`+3d`/`fri`/ISO), recurrence presets + easy NL subset, `list`, `done`/`drop`/`skip`, `attention`, **`edit`** (new backend **`task.set_schedule`** — the missing *reschedule* capability), `promote`, `health`, `log`, project-by-name, links/backlinks, sync, conflicts. (Replaces the old "task-scheduling UX in nvim" item — structured-field entry belongs on the CLI/TUI, not nvim buffers; [[design]] §4.) +2. ⏳ **`heph-tui` — the task agenda/triage surface (§8.1) — the next big build:** ratatui terminal UI over the daemon socket; projects/list/preview panes; daily-ritual filters (orange reconfirm, blue review); launches into nvim for context and back. Planned in §8.1. +3. ⏳ **nvim task-navigation polish (§8) — small:** show do/late in `next`/`list` rows and a clean jump-to-context gesture (read/navigate, not field-edit). +4. ⏳ **Tags + project-hierarchy depth (§4, §6.2.1) — deferred:** tags are barely used (5/387) so low priority; project hierarchy beyond `project add --parent` (hierarchy-aware `scope`, `project list` needing a list-by-kind RPC) is a refinement. +5. ⏳ **`heph.nvim` slice 11d (§6/§8) — DEFERRED, post-parity:** daemon **server-push** notification framing (no-`id` lines on the socket; the client read-loop already demuxes them) + the **dirty-buffer reconcile** (the §8 "known-hard" case) + the "update-arrives-while-open" e2e (§9). +6. ⏳ **Split `heph.nvim` to its own forge repo (§8) — UX polish:** generated from this monorepo (subtree-split in CI) so the lazy spec becomes `{ "eblume/heph.nvim" }` instead of a local-clone `dir` (see [[install-heph]]). +7. ⏳ **Adoption refinement + multi-tenant (§13) — non-blocking:** local→authed **adoption** currently rewrites `owner_id` (`adopt_owner`) but not yet the owner-embedded deterministic ids (journal/tag) + their links; and the hub is single-tenant (one owner per store) — owner-per-token storage is a future extension. +8. ⏳ **Dependency-refresh pass (§11) — before declaring v1 done:** sweep all external deps to latest stable (e.g. `keyring` 3→4, which restructured to `keyring_core`), re-run the full suite. ## Related