hephaestus/docs/reference/tech-spec.md

title	modified	tags
Technical Specification	2026-06-01	reference design

Hephaestus — Technical Specification

Clean, implementation-facing spec for the v1 prototype. For the why behind every choice (history, prior art, decision trail), see design. Where this spec and the design doc disagree, the design doc's latest decision wins — file an update here.

1. Overview

Hephaestus (heph) is a self-hosted personal context-management system that unifies a markdown knowledge base with task management in one database. v1 supports the full spectrum from local-only to distributed, selected by configuration via a targetable storage backend (§3.1): run purely local against a SQLite file, or run a server that fronts that file for remote clients and acts as the sync hub. Multi-device use is offline-first — local-backed replicas reconcile through the hub with automatic merge + conflict resolution — and access is authenticated via OIDC (Authentik) with per-user isolation. The web UI and the actual k3s deployment are later phases; the hub ships in v1 as a runnable/deployable binary. Components:

• heph-core — Rust library: data model, the Store abstraction + local SQLite store, query engine, markdown parsing/extraction, recurrence, and the sync engine (op-log, HLC, CRDT merge, conflict detection).
• hephd — Rust daemon; one binary, three runtime modes (local / server / client, §3.1). Always serves a JSON-RPC API over a local unix socket to local surfaces; in server mode it additionally exposes an authenticated network endpoint and runs as the sync hub.
• heph — Rust CLI: task capture/scripting + the complete daemon API (every RPC method has a command), plus admin/export/heph conflicts. Structured task fields are flags (-a red --do tomorrow --recur weekly).
• heph.nvim — Lua Neovim plugin: the primary context / knowledge-base surface ("org-mode"-style — docs, wiki-links, journals, the canonical-context doc, checklists); a thin client of the local hephd. Surfaces tasks for navigation/context, not structured-field editing.
• heph-tui (planned, §8.1) — Rust terminal UI: the primary task agenda / triage surface (the dominant task activity per design §6.2.1); launches into heph.nvim for a task's context and back.

Surface model (revised 2026-06, design §4 / §6.2.1). Tasks and knowledge pull in different interaction directions, so v1 uses three surfaces, each to its strength: CLI = capture/scripting + complete API; TUI = interactive task agenda/triage; nvim = context/knowledge base. This supersedes the earlier "heph.nvim is the primary surface" framing.

2. Development approach

Development is test-driven (TDD). Write the failing test first; implement to green; refactor. No feature is "done" without tests at the appropriate layer(s) in §9. Core logic must be deterministic and clock-injected (no ambient wall-clock reads in heph-core; the current time is always passed in) so ranking and recurrence are testable.

3. Architecture

• Surfaces (heph.nvim, heph CLI) never touch SQLite directly. They connect to the local hephd over a unix socket (default $XDG_RUNTIME_DIR/heph/hephd.sock) regardless of mode.
• heph-core is synchronous and side-effect-light (incl. deterministic merge logic); hephd wraps it with async I/O, transport, and auth (tokio). DB calls run on a blocking pool.
• See §3.1 (Storage backends & runtime modes), §12 (Sync & Conflict Resolution), §13 (Authentication).

3.1 Storage backends & runtime modes

heph-core exposes a Store trait with two implementations, so what a runtime points at is configuration:

• LocalStore — a SQLite file opened directly, acquiring an exclusive lock on open (advisory flock on the DB file / a sidecar .lock). Refuses to start if the file is already locked.
• RemoteStore — no local file; proxies every operation to a server over the network.

Two orthogonal axes, not one knob. A runtime is configured along two independent axes plus one optional capability:

1. Backend — LocalStore (own replica + op-log) vs. RemoteStore (proxy to a server).
2. Inbound listener — does it expose an authenticated network endpoint others connect to (i.e. is it a hub)?
3. Outbound sync (optional, LocalStore only) — an optional hub_url; when set, the instance is a spoke that background-syncs its op-log to that hub. Independent of whether it also listens.

The named modes are presets over those axes:

Mode	Backend	Inbound listener	hub_url	Offline	Role
local	LocalStore	none	optional	yes	everyday device — standalone if hub_url unset, a syncing spoke if set
server	LocalStore	yes (authenticated)	n/a (it is the hub)	yes	the sync hub; fronts the file and serves remote clients
client	RemoteStore → a server	n/a	n/a	no	thin online-only convenience (no replica)

"Inbound listener: none" on local means no inbound endpoint — it does not preclude outbound sync. The everyday device (Gilbert, ringtail) runs local + hub_url: a full offline-capable replica that syncs to the blumeops hub. A local instance with no hub_url is the first-class, fully-offline standalone config. client is the escape hatch for "don't keep a replica here" (a borrowed box, CI, the future web-UI backend).

Lock handoff (the key flexibility): local and server both take the file's exclusive lock, so only one can own a given DB file at a time. Kill the server → lock releases → a local process can open the exact same file and just work. Stop it → relaunch in server mode → remote clients reconnect. A client process never opens the file, so it never contends. Each machine takes the lock on its own file; device↔device sync uses separate replicas that reconcile through the hub (§12), never a shared file.

4. Data model

All first-class entities are nodes; relationships are links. Markdown bodies are stored in SQLite; files are an export artifact, not the source of truth.

4.1 Node kinds

kind	meaning	body
doc	rich context document (knowledge base, work-logs, journals)	markdown
task	thin task or ephemeral context item (see §4.3)	none (context via links)
project	grouping/context for tasks	optional
tag	label	optional
journal	daily note, titled by ISO date	markdown

4.2 Link types

wiki (materialized from [[links]] in a body), canonical-context (task → its auto-created context doc), context-of, log-of (task → its append-only log), blocks, parent, tagged, in-project.

4.3 Task semantics

• Attention-state (required on committed tasks): white (do once do-date arrives), orange (top of mind), red (top of mind + a consequence exists if late — consequence, not severity), blue (on-deck/backlog).
• do-date = earliest actionable date ("do date"), not a deadline and not an urgency signal — a boolean candidacy gate only (§7). Optional late-on marks when lateness becomes a problem; it is the sole urgency signal.
• Commitment axis (Fork A — design §6.3): a committed task is a task node (a row in tasks) and participates in "what is next". A context item is not a synced node — it is a - [ ] line in a container doc body (its [ ]/[x] is its only state), a locally-derived index over the converged body (§5) that converges for free under sync. Identity is pinned only at promotion, which mints a committed task node and rewrites the source line into a link to it.
• States: outstanding, done, dropped (done and dropped are both "not outstanding"; the distinction is retained).
• No hard deletes: everything uses tombstoned; physical deletion only in an explicit cleanup mode.

4.4 Recurrence (§3.3 of design)

A task with a non-null recurrence (RFC-5545 RRULE) is a recurring definition. Its checklist is the set of - [ ] lines in its canonical context doc body — under Fork A the body is the template (no separate template flag). Each occurrence presents a fresh, all-unchecked checklist, and completion never carries forward across occurrences — a hard requirement.

Model: roll-forward in place (decided; the occurrence-instances alternative was rejected because, under Fork A, each occurrence would need its own body and thus its own node — the explosion §6.6 forbids). A recurring task is a single node. On task.set_state(done):

1. append the completed occurrence to the task's log (log-of);
2. reset the context doc's checkboxes to unchecked (a body-CRDT edit);
3. advance do_date to the next RRULE instance strictly after now — skipping missed occurrences rather than enqueuing them.

So missing a daily routine for three days leaves one gently-overdue item, never a pile; the gap is implied by the hole in the log, not by overdue rows. RRULE expansion is lazy — only "the next instance after now" is ever computed, never the series (§6.6). A skipped state advances the do-date the same way without logging a completion. History is narrative (the log), per "narrative > list".

4.5 SQLite schema (starting point)

nodes(
  id           TEXT PRIMARY KEY,     -- ULID (content nodes); deterministic fn(owner,key) for journal/tag (§3.1 of [[design]])
  owner_id     TEXT NOT NULL REFERENCES users(id),  -- per-user isolation
  kind         TEXT NOT NULL,        -- doc|task|project|tag|journal
  title        TEXT NOT NULL,
  body         TEXT,                 -- markdown (nullable); materialized view of body_crdt
  body_crdt    BLOB,                 -- text-CRDT state for the body (merge), nullable
  created_at   INTEGER NOT NULL,     -- epoch ms
  modified_at  INTEGER NOT NULL,
  hlc          TEXT NOT NULL,        -- hybrid logical clock of last write (sync ordering)
  tombstoned   INTEGER NOT NULL DEFAULT 0
)

tasks(                               -- committed tasks only; context items live in doc bodies (§4.3)
  node_id      TEXT PRIMARY KEY REFERENCES nodes(id),
  attention    TEXT,                 -- white|orange|red|blue
  do_date      INTEGER,              -- epoch ms, nullable; boolean candidacy gate only (§7)
  late_on      INTEGER,              -- epoch ms, nullable; the sole urgency signal (§7)
  state        TEXT NOT NULL,        -- outstanding|done|dropped
  recurrence   TEXT                  -- RRULE; present = recurring definition (roll-forward, §4.4)
)

links(
  id          TEXT PRIMARY KEY,      -- ULID
  src_id      TEXT NOT NULL REFERENCES nodes(id),
  dst_id      TEXT NOT NULL REFERENCES nodes(id),
  type        TEXT NOT NULL,
  created_at  INTEGER NOT NULL,
  tombstoned  INTEGER NOT NULL DEFAULT 0
)

aliases(node_id TEXT REFERENCES nodes(id), alias TEXT)   -- wiki-link name resolution
nodes_fts                                                -- FTS5 over title, body

-- identity & sync --
users(
  id          TEXT PRIMARY KEY,      -- ULID
  oidc_sub    TEXT UNIQUE,           -- OIDC subject (Authentik); NULL = local/unlinked user (§13)
  name        TEXT,
  created_at  INTEGER NOT NULL
)

oplog(                               -- append-only operation log (the sync unit)
  id          TEXT PRIMARY KEY,      -- ULID
  owner_id    TEXT NOT NULL REFERENCES users(id),
  hlc         TEXT NOT NULL,         -- hybrid logical clock (causal order)
  origin      TEXT NOT NULL,         -- originating device id (provisioning TBD at sync slice, §12)
  op_type     TEXT NOT NULL,         -- node.create|node.body_delta|task.set_field|link.add|link.remove|...
  target_id   TEXT NOT NULL,
  payload     TEXT NOT NULL,         -- JSON (e.g. CRDT delta, field+value, OR-Set add/remove)
  applied     INTEGER NOT NULL DEFAULT 0
)

sync_state(                          -- per-peer cursor (device ↔ hub)
  peer        TEXT PRIMARY KEY,      -- 'hub' on a spoke; device id on the hub
  last_pushed_hlc TEXT,
  last_pulled_hlc TEXT,
  updated_at  INTEGER NOT NULL
)

conflicts(                           -- ambiguous merges surfaced to the user
  id          TEXT PRIMARY KEY,
  owner_id    TEXT NOT NULL REFERENCES users(id),
  node_id     TEXT NOT NULL REFERENCES nodes(id),
  field       TEXT NOT NULL,         -- which field / 'body-region'
  local_val   TEXT, remote_val TEXT,
  local_hlc   TEXT, remote_hlc TEXT,
  status      TEXT NOT NULL,         -- open|resolved
  created_at  INTEGER NOT NULL
)

Projects/tags are nodes; membership is links (in-project, tagged). All tasks/links rows inherit ownership via their node(s). Context items are not in this schema — they are - [ ] lines in doc bodies (§4.3, §5); a daemon may keep a local, non-synced derived index of them for the plugin, rebuilt from bodies and never written to the op-log. Deterministic ids for journal/tag let two offline replicas that independently create the same day's journal or the same-named tag converge automatically; the tag-name normalization function is a fixed, versioned constant so every device derives byte-identical ids (tag rename = retag, not in-place key change).

5. Markdown handling

• The body is the source of truth and merges as a text CRDT (body_crdt); the body TEXT column is its materialized view. A full-body write (node.update({body})) is diffed into the CRDT (yrs computes the delta), so surfaces can send whole-buffer text without knowing about deltas.
• On write, heph-core derives from the body:
  • [[wiki-links]] → wiki links (resolved via aliases/title; unresolved links are allowed and recorded).
  • GFM task-list items (- [ ] / - [x]) → the local context-item index (Fork A — see design §6.3). This index is derived, not synced: each replica rebuilds it from its converged CRDT body, so context items converge for free and never get divergent ULIDs. Context-item identity is pinned only at promotion (§6).
• Derivation is idempotent and diff-based: re-writing an unchanged body is a no-op.
• Tombstoned nodes are excluded from the derived index, nodes_fts, next/list, and export.
• export materializes all non-tombstoned nodes to a directory tree of .md files (frontmatter — id, kind, task scalars, aliases, links — plus body), a faithful one-way portable snapshot (no import in v1).

6. Daemon RPC API (JSON-RPC over unix socket)

Methods (request → response; errors are JSON-RPC errors). Signatures are indicative, not final:

• node.get(id) → Node
• node.create({kind, title, body?}) → Node
• node.update({id, title?, body?}) → Node (body update re-runs extraction)
• node.tombstone(id) → ok
• task.create({title, project?, attention?, do_date?, late_on?, recurrence?, committed?}) → Task (auto-creates the canonical context doc + canonical-context link)
• task.set_state({id, state}) → Task (recurring: advances per §4.4)
• task.set_attention({id, attention}) → Task
• task.promote({container_id, item_ref, attention?, project?}) → Task (mints a committed task from a context-item line and rewrites that line into a link to it, §4.3)
• next({scope?, limit?}) → [RankedTask] (the Tactical blank-slate "what is next?" ranking, §7)
• list({scope?, attention?, include_blue?, include_future?, group_by?}) → [Task] (enumeration for the Organizational view — the whole set incl. backlog)
• search({query, filters?}) → [Node] (FTS)
• links.outgoing(id) → [Link] / links.backlinks(id) → [Link]
• journal.open_or_create(date) → Node
• log.append({task_id, text}) → ok (append to the task's log-of node)
• log.tail({task_id, n?}) → [Entry] (read the task's latest log entries — the resumption breadcrumb, design §6.1)
• export({path}) → {count}
• health() → {orange_count, active_count, on_deck_count, conflict_count, sync_status, ...} (working-set + sync indicators)
• auth: auth.login() → {device_code_flow...} / auth.status() → {user, logged_in} / auth.logout() (§13)
• sync: sync.now() → {pushed, pulled, conflicts} (force a sync cycle; background sync runs automatically) / sync.status() → {last_pushed, last_pulled, pending_ops, online}
• conflicts: conflicts.list() → [Conflict] / conflicts.resolve({id, choice}) → ok

The daemon is mode-agnostic — Tactical/Strategic/Organizational are plugin-side compositions of these primitives (§8), not daemon concepts. Every local mutation method records an op in the op-log for sync. The local daemon supports server-push notifications (e.g. a node changed by an incoming sync) so an open buffer can reconcile in real time.

6.1 Hub (server-mode) sync endpoint

Separate from the local unix-socket RPC, the hub exposes an authenticated network endpoint (HTTP/JSON or gRPC — pick at kickoff) for op exchange: clients present an OIDC bearer token (§13); the hub validates, scopes by owner_id, accepts pushed ops, and returns ops the client hasn't seen (by HLC cursor). The hub applies the same heph-core merge logic.

7. "What is next?" ranking (Tactical blank-slate)

This is the Tactical ranking only; Strategic/Organizational are other plugin-side views over different primitives (§6, §8). Given an optional scope and limit (default 5), the engine is two stages — a filter, then an order expressed as data so the sort is trivially reorderable later.

1. Filter (candidacy) — a pure predicate: committed ∧ state = outstanding ∧ ¬tombstoned ∧ attention ≠ blue ∧ actionable (do_date IS NULL OR do_date ≤ now) ∧ in scope. do_date is used only here — a boolean "can this be done now?" gate, never an urgency input. For a recurring task, the single rolled-forward node's do_date is the gate (§4.4).

2. Order — an ordered list of named sort dimensions, applied lexicographically; reordering this list (one place) reshuffles the ranking:

   RANKING = [ PastLateOn(desc), LateOverdueAmount(desc), Attention(red>orange>white), CreatedAt(asc) ]
   

   • late_on is the sole urgency signal and forms a global top tier: items past late_on float above everything (incl. red), most-past first. This is the only legitimate "overdue" — late_on is a deliberately-set problem marker, not an actionability date. (Global-tier vs. within-band placement is expected to be revisited; it's just a reordering of RANKING.)
   • then attention band red → orange → white;
   • tie-break created_at ascending (FIFO). Age is never urgency — an item is not hotter for having been actionable longer; raise its attention or set a late_on to escalate it deliberately.

3. Output: concise rows — title, project, attention, do/late, link to canonical context. red items always appear regardless of limit.

blue (on-deck) is hidden from next by design; surfaced only by list (§6). health() exposes the working-set tensions (orange vs. 6, active vs. ~30, on-deck count) with over-threshold deltas, honestly — never masking overload nor manufacturing calm.

8. heph.nvim surface (v1) — context / knowledge base

Replaces obsidian.nvim. Telescope-backed. Primarily the context / knowledge-base surface (design §4): docs, wiki-links, journals, the canonical-context doc, checklists, per-task log. Task agenda/triage is the TUI's job (§8.1) and structured-field editing is the CLI's (flags); nvim surfaces tasks for navigation and context. Tactical / Strategic / Organizational are named plugin-side views composed from the mode-agnostic daemon primitives (§6) — the daemon never infers a mode. The session goal stack is plugin state (no persistent stack in v1; the durable re-seed is the per-task breadcrumb, log.tail). Core commands/gestures:

• Follow [[wiki-link]] under cursor on <Enter> (follow-or-create).
• Search / quick-switch / tags / backlinks / outgoing links (pickers).
• Daily journal picker (create/open dated journal nodes); home/index page.
• Show "what is next" (:Heph next, Tactical) and list views (Organizational) for navigation — <CR> jumps to a task's canonical context. (Showing do/late in rows and a clean jump-to-context gesture is a small future polish item.)
• Open a task's canonical context doc; edit context-item checkboxes (Fork A) in the buffer (derived on :w); context-item promotion.
• Per-task log quick-append without leaving the current buffer.
• Lightweight task mutation (capture, attention, done/drop/skip) remains available, but the primary task-mutation surface is the CLI/TUI. The eventual richer nvim task-edit story is frontmatter-as-edit-surface (the task buffer presents scalars as editable YAML frontmatter parsed back on :w — export already serializes this), not bespoke form widgets — a later slice.

Deferred (fast-follow, scaffolded): guided working-set rituals — the Blue keep/drop review and Orange daily reconfirm — are pure compositions of list + set_attention + set_state(dropped); v1 ships health() reporting and manual review. (These become first-class filters in the TUI, §8.1.)

Known-hard: reconciling an incoming CRDT body delta into a dirty buffer (unsaved local edits, cursor position) — the §9 "update arrives while a buffer is open" case — is genuinely fiddly under Fork A; expect to iterate.

8.1 heph-tui surface — task agenda / triage (planned)

Status: planned, not yet built. The §6.2.1 Todoist study shows the dominant task activity is interactive triage of a large set (387 active tasks; daily orange reconfirm, blue keep/drop review, browse-by-project) — work that is awkward as either CLI flags or nvim buffers. A terminal UI owns it; the CLI (capture/scripting) and nvim (context) flank it.

• Crate crates/heph-tui — ratatui + crossterm, a thin client of the daemon unix socket (reuse hephd::Client); never touches SQLite, same as nvim.
• Layout — three panes: projects/contexts (the §6.2.1 hierarchy) · task list (next/list rows with attention + human do/late) · preview (canonical-context doc body / log.tail).
• Gestures — j/k move · a add · x done · space skip · A cycle attention · e reschedule (do/late) · b push-to-blue · the left pane lists the §8.2 named filter views (Top of Mind, Tasks, Work Tasks, Chores, On Deck) — the design §6.2 "filters = saved views" made interactive.
• TUI ↔ nvim handoff — o/<CR> launches $EDITOR (nvim) on the task's canonical-context doc (nvim with a +lua call opening heph://node/<ctx-id>, or a temp .md round-tripped through node.update); a nvim command (e.g. :Heph agenda) shells back to the TUI.
• Testing — TDD against a real daemon; headless smoke via ratatui's TestBackend.
• Prereqs (land first): §8.2 filter views (the TUI's saved-filter pane is just those views); the CLI-complete task surface and task.set_schedule (done).

8.2 Filter views (saved agenda slices) — planned, the next slice

Status: planned, the next slice. design §6.2 / §6.2.1 establish that the owner navigates work through a fixed set of saved filters, not one flat list — so next alone is too coarse. This slice makes those filters first-class, shared by the CLI now and the TUI (§8.1) next. (The reference-context noise those filters excluded — ##Culture, #Camano Info — has already been reclassified out of tasks into wiki docs, design §6.2.1.)

The five built-in views (the owner's sixth Todoist filter, Schedule, is intentionally dropped — see below), each derived from the verbatim Todoist query (design §6.2.1) and realized in heph terms (attention: p1→red, p2→orange, p4→white, p3→blue):

View	Todoist query (origin)	heph realization
Top of Mind	(p1|p2) & (no date|today|overdue)	attention ∈ {red,orange} ∧ actionable
On Deck	p3 & (no date|overdue|today)	attention = blue ∧ actionable
Chores	(today|overdue|no date) & (#Chores|#Camano Chores)	scope ∈ {Chores, Camano Chores} ∧ actionable
Work Tasks	#Work & !p3 & (…) & !subtask	scope = Work subtree ∧ attention ≠ blue ∧ actionable
Tasks	!p3 & (…) & !(#Daily Routine|#Work Routine|#Chores|#Camano Chores|#Work|##Culture|#Camano Info) & !subtask	attention ≠ blue ∧ actionable ∧ not in the routine/work/chore projects

Engine work — extend list (§6) so a view is a predicate expressed as data (mirroring §7's "order as data"):

• attention: a set of states (was a single value) — e.g. {red,orange}.
• scope: a project including its descendant projects (subtree, for ##Culture / Work-tree), and/or multiple projects (Chores + Camano Chores).
• exclude_projects: a list subtracted from the result (the "Tasks" leftover view).
• actionable: a bool toggle applying the §7 do-date candidacy gate inside list (today the gate is next-only).
• (recurring: optional filter for the Schedule approximation.)

Project-subtree resolution needs the parent-project links (design §6.2.1) — a small links/query addition. !subtask is largely implicit (heph list/next return committed tasks, not context items).

Surface:

• CLI: heph view <name> (tom|tasks|work|chores|ondeck) prints the slice using the same row format as next/list. A heph view with no name lists the available views.
• TUI (§8.1): the same views power the left filter pane.
• nvim: may later expose them (:Heph view <name>) — navigation, not required for this slice.

Defaults vs. custom: the five ship as built-in named views seeded from the queries above. User-defined filters (a small config-driven view list, eventually a query DSL) are a later extension — deliberately deferred to avoid building a Todoist-query-language parser now.

Schedule view dropped (decided 2026-06). Schedule's !no time selects items with a time-of-day, which heph's date-grained do_date can't represent; rather than ship a misleading approximation it is omitted for now. It is entangled with the chores rework below (timed routines), so revisit both together if/when time-of-day lands on tasks.

Future — chores as a first-class feature (noted, not scheduled). The Chores view here is an interim project-scoped filter. The intent is to make chores a first-class concept with their own do-date / recurrence semantics (distinct from regular tasks), retiring the #Chores / #Camano Chores projects (and the Camano split) entirely — chores would be a task flag/kind, not a project you scope to. When that lands, the Chores view becomes "tasks where is_chore," and Schedule (timed routines) is reconsidered alongside it. See design §6.2.1.

9. Testing strategy (TDD, layered)

All layers are required; CI runs them on every push/PR (extend .forgejo/scripts/build to run cargo test and the nvim e2e suite; prek already runs in build.yaml). The Forgejo runner image must provide neovim + plenary.nvim for the headless e2e suite.

• Unit (heph-core): model invariants; markdown extraction (wiki-links, checkboxes); RRULE expansion and the fresh-checklist-per-occurrence rule (assert completion never carries forward); the "what is next?" ranking (table-driven cases); migration up/down.
• Property tests (proptest): ranking yields a total order; extraction is idempotent; recurrence never leaks completion state across occurrences; tombstones are never resurrected.
• Integration (hephd, real sockets): start a daemon against a temp SQLite file, connect over a real unix socket, and exercise the RPC API end-to-end, asserting resulting DB state. Include multi-client concurrency tests on the socket and clock-injection for deterministic time.
• Sync & offline (multi-replica): spin up two client hephd replicas + a hub hephd, all over real network sockets against temp DBs, and assert convergence:
  • online round-trip: edit on A → appears on B via the hub;
  • offline → reconcile: partition A and B from the hub, make divergent edits on each, reconnect, assert both converge;
  • conflict path: concurrent conflicting scalar edits (e.g. both set a different do-date) land in the conflict queue and conflicts.resolve settles them deterministically;
  • body CRDT merge: concurrent edits to the same doc body auto-merge without a hard conflict;
  • HLC ordering and op-log idempotency (replaying ops is a no-op).
• Auth: OIDC token validation on the hub endpoint (reject missing/invalid/expired); per-user isolation (user A cannot read/sync user B's nodes); device-code flow happy path against a mock IdP.
• End-to-end (headless nvim): drive heph.nvim in nvim --headless against a real hephd + temp DB, running scripted example workflows and asserting outcomes (via RPC/DB state and buffer contents). Minimum workflows:
  • capture a task → appears in :Heph next → open canonical context → add a checklist item → check it → mark task done;
  • create today's journal via the picker;
  • follow a [[link]] on <Enter> to the target doc;
  • a recurring task with a checklist: complete it, then assert the next occurrence presents a fresh, all-unchecked checklist;
  • a sync-driven update arrives while a buffer is open and the buffer reconciles.
  • Harness: plenary.nvim/busted, or drive nvim via its msgpack-RPC from the test runner. Keep example workflows as reusable fixtures.
• CLI tests: invoke heph subcommands against a temp DB; snapshot output; assert export round-trips the corpus; heph conflicts lists/resolves.

10. Technology stack (ratified)

rusqlite (bundled) + migration runner · tokio + line-delimited JSON-RPC over unix socket · ulid · rrule · pulldown-cmark · clap · anyhow/thiserror · tracing. Neovim plugin in Lua, depending on telescope.nvim. Cargo workspace: crates/heph-core, crates/hephd, crates/heph, plus heph.nvim/.

Added for v1 client/server + auth (some to confirm at kickoff):

• Text CRDT (body merge): ✅ yrs (Rust Yjs) (ratified at the Phase 1 sync kickoff, 2026-05-31; automerge was the alternative). Used for doc/journal/log bodies. Structured fields use a bespoke op-log + HLC (no library needed).
• HLC: small bespoke hybrid-logical-clock (or a crate) — deterministic, clock-injected.
• Hub network transport: axum (HTTP/JSON) for the sync endpoint — leaning (reuses the eventual web-UI server); reqwest on the client side.
• OIDC: openidconnect crate for the Authentik device-code flow; tokens cached in the OS keychain (keyring) / 1Password.

11. v1 scope

In:

• The full data model, markdown handling, "what is next?" ranking, and recurrence + recurring checklists (§4–§8).
• Targetable storage backend + all three runtime modes — local, server, client — with the exclusive-lock handoff over a shared SQLite file (§3.1). Local-only is a first-class, fully-supported configuration.
• Offline-first operation for local-backed instances, with op-log + CRDT sync and automatic merge + a conflict queue (§12).
• OIDC/Authentik authentication with per-user data isolation (§13), enforced on the server network endpoint.
• heph.nvim + heph CLI surfaces (incl. heph conflicts).

Out (later phases, scaffolded so as not to block):

• Web UI (the hub serves sync only in v1; reserve axum for it later).
• Actual k3s deployment to blumeops (Dagger→Zot image, ArgoCD app + Kustomize manifests, external-secrets) — fast-follow once the architecture is proven; the hub binary is built to be deployable.
• Calendar integration (read-mostly CalDAV; never explode recurrence into stored events), iOS/Watch capture, inferred/semantic context, P2P-over-tailnet sync fallback.
• Dependency-refresh pass: before declaring v1 done, sweep every external dependency (Cargo crates, the Neovim plugin's deps, CI/tooling) up to its latest stable release, re-running the full suite + clippy/fmt to catch breakage. Slices add deps at the version current when written; this reconciles them once at the end rather than churning mid-build.

See design §5–§7 for the constraints later phases impose on present choices (keep tasks vs. calendar events separate; expand RRULEs lazily).

12. Sync & conflict resolution

Topology: hub-and-spoke. Each device holds a full local replica + op-log; the hub is the rendezvous. Devices push/pull ops by HLC cursor; the hub never needs to be online for local work.

Merge semantics (the unit of sync is the op):

• doc/journal/log bodies: text CRDT (yrs) → concurrent edits always merge, no hard conflict.
• Scalar task fields (attention, do_date, late_on, state, …): last-writer-wins by HLC. The losing value, if meaningful, is recorded in conflicts (surfaced, not silently dropped).
• Links / set membership (tags, project, parent): OR-Set add/remove semantics → no conflicts.
• Tombstones, never hard deletes → deletion/merge is monotonic and CRDT-friendly.

Conflict queue: the unresolvable/ambiguous remainder (a discarded LWW value on a meaningful field; flagged overlapping body regions) becomes an open row in conflicts. Surfaced via health().conflict_count, conflicts.list, heph conflicts, and a heph.nvim view: "you have N conflicts." conflicts.resolve({id, choice}) settles each. Sync never blocks on conflicts.

Determinism: HLCs are clock-injected; op application is idempotent and order-independent given HLC. These are the core invariants the sync tests assert.

Open at kickoff: CRDT lib confirmation (yrs vs automerge); hub transport (axum HTTP/JSON vs gRPC); propagation cadence (push vs. periodic pull); exactly which fields are "meaningful" enough to enqueue vs. silently LWW; device-id provisioning (how a spoke mints its origin id and registers with the hub).

13. Authentication

• OIDC against Authentik. Clients authenticate via the OAuth 2.0 device-code flow (auth.login); the resulting tokens are cached in the OS keychain (keyring) / 1Password and refreshed automatically. Offline devices operate on cached credentials.
• Auth is a network-boundary concern; ops are authenticated at exchange, not creation. Creating ops needs only local trust, so losing the IdP/hub never blocks local work — ops queue in the op-log and the bearer token gates only the push/pull session. The one degraded case: the refresh token expiring during a long offline stretch → re-run the device-code flow once on reconnect, then the queued op-log drains.
• Local-only needs no auth. A local instance with no hub_url runs against a single local user (users row, generated id, oidc_sub = NULL) — friction-free, no IdP. oidc_sub is therefore nullable.
• The hub is authoritative for user identity. On first authentication it maps sub → a canonical users.id (creating it if new) and hands the same id to every device that authenticates as that human — no device-vs-device id reconciliation.
• Adoption (local-only → authed) is a one-time, pre-first-sync rewrite: on a previously-local replica's first authenticated sync, fetch the canonical users.id, then in one transaction rewrite owner_id (and the owner-embedded deterministic ids of §4.5 + the links referencing them) from the local placeholder to the canonical id. Safe precisely because it precedes any sync (no remote refs, no concurrent writers); afterward owner_id is immutable. Born-authed replicas stamp the canonical id from the start and never rewrite.
• Hub enforcement: the sync endpoint requires a valid OIDC bearer token; the hub maps the token's sub to a users row and scopes every op by owner_id. No cross-user reads/writes.
• Per-user isolation: all nodes (and their dependent rows) carry owner_id; queries and sync are always user-scoped. In practice a single user (eblume), but the isolation is real from day one.
• Local trust: the local unix-socket RPC trusts the OS user (file-permission-scoped socket); app-level auth is for the network boundary (device ↔ hub).
• At-rest: plain SQLite in v1 (no encryption) — security boundary is auth + (eventually) network restriction. May revisit (see design).

14. Implementation status (Phase 1 tracker)

Cross-session resume tracker for the Phase 1 C1 (branch feature/v1-prototype, PR #1). Updated 2026-06-02 — 117 Rust tests (cargo test --all) + 17 heph.nvim headless e2e specs (mise run test-nvim; also runs in CI via dagger call test-nvim), clippy -D warnings + fmt + prek clean. Workspace: crates/heph-core, crates/hephd, crates/heph, plus heph.nvim/ (slices 11a–11c + a UX iteration, below). The plugin is installed and running on the dev machine (built from the forge; see install-heph).

Done

• ✅ Data model + schema (§4): migrations v1–v3 (base graph, FTS5, meta). Node kinds, tasks, links, aliases, users, oplog, sync_state, conflicts, meta. Store trait + LocalStore.
• ✅ Markdown handling (§5): wiki-link + checkbox extraction (pure, idempotent, code-aware); update_node materializes/reconciles wiki links; export to a <kind>/<id>.md tree.
• ✅ Recurrence (§4.4): roll-forward in place — fresh checklist, logged occurrence, advance-skipping-misses; completion never carries forward (proptest). Per-task logs; skip.
• ✅ Ranking (§7): pure two-stage filter + reorderable named dimensions; proptest total order.
• ✅ Daemon RPC (§6): node.get/create/update/tombstone, task.create/set_state/set_attention/skip, next, list, health, journal.open_or_create, search, links.outgoing/backlinks, log.append/tail, export, conflicts.list/resolve, sync.now/sync.status. Line-delimited JSON-RPC over a unix socket; sync Client. (ops_since/apply_op are Store methods exchanged over the hub HTTP endpoint, not the unix socket.)
• ✅ Runtime modes (§3.1) — local + server + client: exclusive file-lock handoff via LockGuard (local/server only); --mode local|server|client, --hub-url, --http-addr, --server-url.
• ✅ Sync engine (§12) minus network: HLC (clock-injected, monotonic) + persistent device origin; op-log per mutation; apply_op merge — LWW task scalars + titles with a conflict queue, OR-set links, monotonic tombstones, idempotent; two-replica convergence proven. adopt_owner = basic §13 canonical-owner adoption.
• ✅ Body text CRDT (§5, §12, slice 8d): node bodies merge through the yrs text CRDT (body_crdt BLOB) instead of LWW. A device authors under a stable client_id derived from its origin; whole-buffer writes are diffed (common prefix/suffix, char-boundary safe) into the doc; the yrs delta rides the node.create/node.set op (body_crdt field) and apply merges it — concurrent disjoint edits both survive and never enqueue a conflict.
• ✅ Network sync (§6.1, §12, slice 9a): transport ratified = axum HTTP/JSON. The hub (server mode) exposes POST /sync/push + GET /sync/pull?after=<hlc> over the same store; a spoke (local + hub_url) runs sync::sync_once (pull→merge, then push) and background-syncs on a 30s interval. Incremental by HLC cursor (sync_state/SyncCursors); idempotent re-push is a no-op. Two spokes converge through a real-HTTP hub (incl. scalar conflict) in tests/sync_http.rs. Unauthenticated for now, single-owner (auth + per-user scoping is slice 10).
• ✅ client mode + RemoteStore (§3.1, slice 9b): a no-replica backend that proxies every Store call to a server's POST /rpc (the full dispatch, over HTTP) via a blocking reqwest client — the online-only escape hatch. Daemon is now generic over dyn Store + Send, so the same unix-socket surface fronts either a LocalStore or a RemoteStore. Sync primitives are stubbed (a client has no op-log). Proven in tests/client_mode.rs. dispatch gained task.get + links.add.
• ✅ Hub auth — verification side (§13, slice 10a): the hub validates an OIDC bearer token (jsonwebtoken, RS256-pinned, exact iss+aud, exp/nbf, required sub; JWKS discovered + cached, refetched on unknown kid) on /sync/* + /rpc. A [TokenVerifier] trait seam keeps it mockable; single-tenant owner gate (authorize_owner_sub: claim-on-first, then require-match → 403 for any other identity). --oidc-issuer/--oidc-audience enable it (open when unset, for local dev). Tested fully offline: stub-verifier middleware tests + an adversarial battery against an in-process mock IdP (expired/wrong-iss/wrong-aud/unknown-kid/tampered/alg-confusion/missing-sub all rejected).
• ✅ Auth — client side (§13, slice 10b): OAuth2 device-code flow (hephd::oauth::DeviceFlow: discover → start → poll handling authorization_pending/slow_down, + refresh). TokenStore (OS keyring via keyring, in-memory for tests); current_bearer refreshes on expiry. heph auth login runs the flow + caches the token; spokes (sync_once) and client mode (RemoteStore) attach the bearer, refreshing as needed (--oidc-issuer/--oidc-client-id). All auth/proxy HTTP uses ureq (runtime-free blocking) — reqwest::blocking panics inside the daemon's spawn_blocking; async reqwest remains only for sync_once. Tested offline against a mock OAuth server (device flow, refresh, store) + a full spoke⇄authed-hub loop.
• ✅ CLI (§1) — the complete daemon API + task driver: heph covers every RPC (next/list/task/done/drop/skip/attention/edit/promote/show/log/health/doc/node/get/resolve/search/journal/links/backlinks/link/project[+list]/sync/conflicts/export/auth) with human dates (--do-date tomorrow|+3d|fri|ISO) and recurrence (presets + NL + raw --rrule; datespec). Reschedule is the new task.set_schedule RPC. Process-tested over a real socket.
• ✅ Daemon as an OS service (§3, design §4): heph daemon start/stop/restart/status/uninstall idempotently manages a launchd agent (macOS) / systemd user service (Linux) running hephd on the default store; render fns unit-tested, verified live on macOS. The default socket is now a stable <data-dir>/heph/hephd.sock (was $TMPDIR-based) so a persistent service and its clients always agree. Surfaces are connect-only (no auto-spawn).
• ✅ Todoist importer (tooling): mise run import-todoist seeds a store from Todoist (dry-run default, -- --commit to write) — see import-todoist.
• ✅ CI (§9): Forgejo build.yaml runs entirely through Dagger (the k8s job image is a thin Alpine + Dagger orchestrator with a DinD sidecar — no native Rust/nvim toolchain): dagger call check (cargo fmt/clippy/test on rust:1-bookworm) + dagger call test-nvim (build hephd + headless e2e). Cargo parallelism capped (CARGO_BUILD_JOBS) to avoid OOMing the build engine; cargo caches shared across runs. prek runs locally via git hooks, not in CI.
• ✅ heph.nvim slice 11a (§8) — the primary surface begins: the Lua plugin (heph.nvim/) as a thin client of the hephd unix socket. RPC client over a vim.uv pipe (blocking call via vim.wait; id-demuxed; partial-line buffered; luanil so JSON null→nil; isolated Sessions for tests). Buffer-backed nodes — heph://node/<id> buffers (buftype=acwrite), BufReadCmd→node.get / BufWriteCmd→node.update (whole-buffer body, CRDT-diffed; exact round-trip). [[wiki-link]] follow on <CR> via a new node.resolve {title} RPC (exact alias-then-title match, the same mapping that materializes wiki links — never fuzzy search; unresolved links allowed). Daily journal (:Heph today/journal <date>, idempotent). :Heph command surface + completion. Headless e2e (§9): drives the plugin in nvim --headless against a real daemon over a temp socket via a self-contained busted-style runner (tests/e2e/runner.lua — no external plugins/network, deterministic exit codes); specs cover journal round-trip, follow-link (+ unresolved no-op), and link-two-docs/backlink. mise run test-nvim builds the daemon and runs the suite (dev: system nvim/rustc; CI: a Dagger container provides them — slice 11c).
• ✅ heph.nvim slice 11b (§8) — task views: list enriched to titled [RankedTask] rows (title + canonical_context_id, shared ranked_from_row with next) so the Organizational view needs no N+1 node.get. Plugin: Tactical next + Organizational list views (rendered scratch buffers, <CR> opens a row's canonical-context doc — the node autocmd narrowed to heph://node/* so view buffers don't trip it); task capture, set-attention, done/drop, skip, per-task log append — all resolving "the current task" from the buffer (a task node, or a context doc via its canonical-context backlink); vim.ui.select picker (picker.lua) with Telescope auto-upgrade; :Heph next/list/capture/attention/done/drop/skip/log/search subcommands. e2e specs: capture→next→open context→add/check checklist→done, and recurring fresh-checklist (complete rolls forward in place; the next occurrence is all-unchecked — the §4.4 hard requirement).
• ✅ heph.nvim slice 11c (§8) — promotion + Dagger CI: backend task.promote {container_id, item_ref, attention?, project?} — mints a committed task from the item_ref-th - [ ] context item (1-based, document order via a new extract::context_item_lines) and rewrites that source line into a [[link]] to it. Wiki-link resolution now excludes canonical-context docs (resolve_id), so [[Task Title]] deterministically resolves to the task, not its identically-titled context doc — a general fix surfaced by promotion. Plugin: :Heph promote, promote_under_cursor (save-if-dirty → util.context_item_index_at_cursor mirrors extract's fence rules → task.promote → reload). e2e spec (f). CI via Dagger: a test_nvim function in .dagger/ bakes a pinned, arch-detected Neovim (v0.11.2; Debian's is too old for vim.uv) onto rust:1-bookworm, builds hephd, and runs the shim suite (cargo + cargo-target cache volumes); build.yaml calls dagger call test-nvim. run.lua fails on zero-specs (no false-green) — validated end-to-end (failing spec → Dagger exit 1).
• ✅ heph.nvim UX iteration + install (§8) — post-11c, makes the plugin a daily driver:
  • Managed daemon (plug-and-play autostart — SUPERSEDED 2026-06 by heph daemon, below): the plugin used to spawn + supervise its own hephd and kill it on exit. Removed once the CLI became a first-class surface — a surface-owned daemon can't be shared. Lifecycle is now an explicit OS service (design §4); all surfaces are connect-only.
  • Knowledge-base UX: follow-or-create (<CR> on an unresolved [[link]] mints the doc + materializes the source backlink), :Heph doc, :Heph home (an open-or-create index/landing page), :Heph journals (recent-days dailies picker with Telescope preview + @create).
  • Interactive task views: :Heph next/list buffers gained a add / d done / r refresh (+ <CR> open), with a dimmed key-hint header line (virt-lines-above the first row render off-screen — use a real header).
  • Dev/installed isolation: installed heph/hephd own the default paths; mise run dev runs the working-tree daemon on .dev/ paths; $HEPH_SOCKET/$HEPH_DB point a dev Neovim at it.
  • CI is now fully Dagger (build.yaml: dagger call check + test-nvim; prek dropped from CI — the Alpine job image has no Rust/nvim/prek, only Dagger + DinD). First-ever green CI.

Not yet done (resume order)

The Rust backend is feature-complete; the CLI is the complete API + task driver, the daemon runs as an OS service (heph daemon; all surfaces connect-only), and the live store has been seeded from Todoist with reference contexts reclassified to wiki docs (design §6.2.1). Surface strategy = three-surface model (design §4): CLI = capture/scripting + complete API (done), TUI = primary task agenda/triage (to build), nvim = context/KB. Remaining work, in order:

1. ⏳ Filter views (§8.2) — the next slice: make the owner's saved filters (Top of Mind / Tasks / Work Tasks / Chores / On Deck — Schedule dropped, §8.2) first-class so the agenda isn't one flat list. Extend list (§6) to a data-expressed predicate — attention set, project-subtree / multi scope, exclude-projects, actionable toggle (+ parent-project link resolution) — and surface five built-in views via heph view <name> (the TUI reuses them). Seeded from the verbatim Todoist queries (design §6.2.1). (Future, noted: chores become a first-class kind with their own do-date/recurrence semantics, retiring the Chores/Camano-Chores projects — §8.2.)
2. ⏳ heph-tui — the task agenda/triage surface (§8.1) — the next big build: ratatui terminal UI over the daemon socket; projects/list/preview panes; the §8.2 filter views as the saved-filter pane; launches into nvim for context and back. Depends on the filter-views slice.
3. ⏳ nvim task-navigation polish (§8) — small: show do/late in next/list rows and a clean jump-to-context gesture (read/navigate, not field-edit).
4. ⏳ Tags + project-hierarchy depth (§4, §6.2.1) — deferred: tags are barely used (5/387) so low priority; project hierarchy beyond project add --parent (and the subtree scope the filter-views slice introduces) is a refinement.
5. ⏳ heph.nvim slice 11d (§6/§8) — DEFERRED, post-parity: daemon server-push notification framing (no-id lines on the socket; the client read-loop already demuxes them) + the dirty-buffer reconcile (the §8 "known-hard" case) + the "update-arrives-while-open" e2e (§9).
6. ⏳ Split heph.nvim to its own forge repo (§8) — UX polish: generated from this monorepo (subtree-split in CI) so the lazy spec becomes { "eblume/heph.nvim" } instead of a local-clone dir (see install-heph).
7. ⏳ Adoption refinement + multi-tenant (§13) — non-blocking: local→authed adoption currently rewrites owner_id (adopt_owner) but not yet the owner-embedded deterministic ids (journal/tag) + their links; and the hub is single-tenant (one owner per store) — owner-per-token storage is a future extension.
8. ⏳ Dependency-refresh pass (§11) — before declaring v1 done: sweep all external deps to latest stable (e.g. keyring 3→4, which restructured to keyring_core), re-run the full suite.

Related

• design — full design document with rationale and decision history