generated from eblume/project-template
Set up hephaestus from template and add design + tech spec
Some checks failed
Build / validate (push) Failing after 2s
Some checks failed
Build / validate (push) Failing after 2s
Customize the generated repo (rename Dagger module to hephaestus_ci /
HephaestusCi, set docs baseUrl, add All-Rights-Reserved LICENSE, update
README/AGENTS), and add the project's foundational design documentation:
- docs/explanation/design.md — rationale + decision-history record
- docs/reference/tech-spec.md — implementation-ready technical spec
These define hephaestus as a self-hosted, client/server + offline-first
system unifying a markdown knowledge base with task management: typed node
graph, the lived priority discipline ("what is next?"), recurrence with
fresh-per-occurrence checklists, op-log/CRDT sync with conflict resolution,
OIDC/Authentik auth, the heph.nvim surface, and a TDD strategy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
28c1f7886a
commit
cbf859b2d7
14 changed files with 768 additions and 36 deletions
3
.dagger/src/hephaestus_ci/__init__.py
Normal file
3
.dagger/src/hephaestus_ci/__init__.py
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
"""Hephaestus CI — Dagger build functions."""
|
||||||
|
|
||||||
|
from .main import HephaestusCi as HephaestusCi
|
||||||
|
|
@ -2,9 +2,8 @@ import dagger
|
||||||
from dagger import dag, function, object_type
|
from dagger import dag, function, object_type
|
||||||
|
|
||||||
|
|
||||||
# TODO: Rename class to match your project (also rename the src/ directory)
|
|
||||||
@object_type
|
@object_type
|
||||||
class ProjectTemplateCi:
|
class HephaestusCi:
|
||||||
@function
|
@function
|
||||||
async def build_docs(self, src: dagger.Directory, version: str) -> dagger.File:
|
async def build_docs(self, src: dagger.Directory, version: str) -> dagger.File:
|
||||||
"""Build Quartz docs site. Returns docs tarball."""
|
"""Build Quartz docs site. Returns docs tarball."""
|
||||||
|
|
@ -1,4 +0,0 @@
|
||||||
"""Hephaestus CI — Dagger build functions."""
|
|
||||||
|
|
||||||
# TODO: Rename class to match your project (also rename the src/ directory)
|
|
||||||
from .main import ProjectTemplateCi as ProjectTemplateCi
|
|
||||||
17
AGENTS.md
17
AGENTS.md
|
|
@ -8,13 +8,16 @@ Guidance for Claude Code working in this repository. See also [[ai-assistance-gu
|
||||||
|
|
||||||
## First-Time Setup
|
## First-Time Setup
|
||||||
|
|
||||||
This repository is a **Forgejo template**. Most customization points are auto-resolved by Forgejo's template variable expansion when a new repo is created. The remaining manual steps are:
|
This repository was instantiated from the `project-template` Forgejo template. Setup status:
|
||||||
|
|
||||||
1. **Set `baseUrl`** in `docs/quartz.config.ts` — this is the hosted docs domain, not the repo URL. localhost if not hosted.
|
- [x] `baseUrl` in `docs/quartz.config.ts` set to `localhost` (update once docs are hosted)
|
||||||
2. **Rename `.dagger/src/project_template_ci/`** directory to match your project, and update the class names inside
|
- [x] Dagger module renamed to `.dagger/src/hephaestus_ci/` (`HephaestusCi` class)
|
||||||
3. **Fill in project structure** in the `AGENTS.md` Project Structure section
|
- [ ] Fill in the Project Structure section below once the Rust workspace is scaffolded
|
||||||
4. **Fill in license info** in `README.md`
|
- [ ] Fill in license info in `README.md`
|
||||||
5. **When all TODOs are resolved:** delete this "First-Time Setup" section entirely from `AGENTS.md` — it is only needed once
|
|
||||||
|
Delete this section once the remaining items are resolved.
|
||||||
|
|
||||||
|
> **This is now a generated repo, not the template source.** C1/C2 changes use feature branches + PRs (`tea pr create`); noteworthy changes get changelog fragments in `docs/changelog.d/`.
|
||||||
|
|
||||||
## Rules
|
## Rules
|
||||||
|
|
||||||
|
|
@ -52,7 +55,7 @@ See [[agent-change-process]] for the full methodology.
|
||||||
```
|
```
|
||||||
./docs/ # Diataxis docs, Quartz config, and release content
|
./docs/ # Diataxis docs, Quartz config, and release content
|
||||||
./docs/changelog.d/ # keep only .gitkeep in the template; generated repos add towncrier fragments here
|
./docs/changelog.d/ # keep only .gitkeep in the template; generated repos add towncrier fragments here
|
||||||
./.dagger/ # Dagger module; rename src/project_template_ci/ when forking
|
./.dagger/ # Dagger module (src/hephaestus_ci/) backing docs builds and releases
|
||||||
./.forgejo/workflows/ # generic build and release workflows for generated repos
|
./.forgejo/workflows/ # generic build and release workflows for generated repos
|
||||||
./.forgejo/scripts/ # optional per-project build/release hooks consumed by the workflows
|
./.forgejo/scripts/ # optional per-project build/release hooks consumed by the workflows
|
||||||
./mise-tasks/ # repo automation via `mise run`
|
./mise-tasks/ # repo automation via `mise run`
|
||||||
|
|
|
||||||
18
LICENSE
Normal file
18
LICENSE
Normal file
|
|
@ -0,0 +1,18 @@
|
||||||
|
All Rights Reserved
|
||||||
|
|
||||||
|
Copyright (c) 2026 Erich Blume
|
||||||
|
|
||||||
|
This software and its associated documentation and source files (the
|
||||||
|
"Software") are the private, proprietary work of the copyright holder.
|
||||||
|
|
||||||
|
No license, express or implied, is granted to any party. You may not use,
|
||||||
|
copy, modify, merge, publish, distribute, sublicense, or sell copies of the
|
||||||
|
Software, in whole or in part, without the prior written permission of the
|
||||||
|
copyright holder.
|
||||||
|
|
||||||
|
The Software is provided "as is", without warranty of any kind, express or
|
||||||
|
implied, including but not limited to the warranties of merchantability,
|
||||||
|
fitness for a particular purpose, and noninfringement.
|
||||||
|
|
||||||
|
Open-source release under a permissive license may be considered in the
|
||||||
|
future, at the sole discretion of the copyright holder.
|
||||||
22
README.md
22
README.md
|
|
@ -1,6 +1,8 @@
|
||||||
# project-template
|
# hephaestus
|
||||||
|
|
||||||
A personal project template with opinionated infrastructure for documentation, CI, and AI-assisted development.
|
A personal context management system — a unified, self-hosted application that fuses a wiki-style knowledge base (Zettelkasten) with task management. Built in Rust, offline-capable, and syncs to a central instance hosted in blumeops.
|
||||||
|
|
||||||
|
See [the project design document](docs/explanation/design.md) for goals, architecture, and the development roadmap.
|
||||||
|
|
||||||
## What's Included
|
## What's Included
|
||||||
|
|
||||||
|
|
@ -11,20 +13,6 @@ A personal project template with opinionated infrastructure for documentation, C
|
||||||
- **AI assistance** — `AGENTS.md` + structured docs for Claude Code (C0/C1/C2 change process, Mikado method)
|
- **AI assistance** — `AGENTS.md` + structured docs for Claude Code (C0/C1/C2 change process, Mikado method)
|
||||||
- **Task runner** — [mise](https://mise.jdx.dev/) tasks for docs validation, Mikado chain management, release preview, and runner inspection
|
- **Task runner** — [mise](https://mise.jdx.dev/) tasks for docs validation, Mikado chain management, release preview, and runner inspection
|
||||||
|
|
||||||
## Forking This Template
|
|
||||||
|
|
||||||
This is a **Forgejo template repository**. When you create a new repo from this template, Forgejo automatically expands variables like `${REPO_NAME}` and `${REPO_OWNER}` in key files — handling most customization automatically.
|
|
||||||
|
|
||||||
After creating your repo, the remaining manual steps are:
|
|
||||||
|
|
||||||
1. Set `baseUrl` in `docs/quartz.config.ts` to your docs site domain
|
|
||||||
2. Rename `.dagger/src/project_template_ci/` directory and update class names to match your project
|
|
||||||
3. Review and tailor the project structure section in `AGENTS.md`
|
|
||||||
4. Add license information to `README.md`
|
|
||||||
5. Remove the "First-Time Setup" section from `AGENTS.md` and this section from `README.md`
|
|
||||||
|
|
||||||
If you use Claude Code, it will prompt you to resolve remaining TODOs at the start of your first session.
|
|
||||||
|
|
||||||
## Getting Started
|
## Getting Started
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -54,4 +42,4 @@ dagger call build-docs --src=. --version=dev export --path=./docs-dev.tar.gz
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
<!-- TODO: Add license information -->
|
All rights reserved. This is a personal, private project — not licensed for use, copying, modification, or distribution. Open-sourcing may be considered in the future. See [LICENSE](LICENSE).
|
||||||
|
|
|
||||||
1
docs/changelog.d/+project-design.doc.md
Normal file
1
docs/changelog.d/+project-design.doc.md
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
Add the project design document (`docs/explanation/design.md`, rationale + decision history) and the distilled technical specification (`docs/reference/tech-spec.md`, the build artifact) defining hephaestus as a unified, self-hosted, **client/server + offline-first** task + knowledge-base system: typed node graph, the lived priority discipline ("what is next?"), recurrence with fresh-per-occurrence checklists, op-log/CRDT sync with conflict resolution, OIDC/Authentik auth, the heph.nvim surface, and a test-driven development strategy.
|
||||||
1
docs/changelog.d/+template-setup.infra.md
Normal file
1
docs/changelog.d/+template-setup.infra.md
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
Customize the repository from the `project-template`: rename the Dagger module to `hephaestus_ci` (`HephaestusCi`), set the docs `baseUrl`, add an All-Rights-Reserved `LICENSE`, and update `README.md`/`AGENTS.md` for the hephaestus project.
|
||||||
419
docs/explanation/design.md
Normal file
419
docs/explanation/design.md
Normal file
|
|
@ -0,0 +1,419 @@
|
||||||
|
---
|
||||||
|
title: Design Document
|
||||||
|
modified: 2026-05-31
|
||||||
|
tags:
|
||||||
|
- explanation
|
||||||
|
- design
|
||||||
|
---
|
||||||
|
|
||||||
|
# Hephaestus — Design Document
|
||||||
|
|
||||||
|
> **Status:** Living design record. This is the **rationale + decision-history** document. For the clean, implementation-facing spec the next session builds from, see **[[tech-spec]]**. Sections marked **❓ OPEN** are unresolved; **🔒 DECIDED** are settled.
|
||||||
|
|
||||||
|
## 1. Purpose & Vision
|
||||||
|
|
||||||
|
**Hephaestus** is a personal context management system that fuses two systems the owner already relies on into a single, purpose-built, self-hosted application:
|
||||||
|
|
||||||
|
1. **The Zettelkasten** (`~/code/personal/zk`) — an Obsidian vault of ~600 markdown files acting as a personal wiki, journal, knowledge base, and "task-context" store for long-running projects. Notes use `[[wiki-links]]`, YAML frontmatter (`id`, `aliases`, `tags`), timestamp-based note IDs (`1722897441-MWFE`), and daily notes (`YYYY-MM-DD`).
|
||||||
|
2. **Todoist** — the owner's primary task manager, accessed today via a bespoke Python mise-task in blumeops (`mise-tasks/blumeops-tasks`) that polls the Todoist v1 API (projects → tasks; fields: `content`, `description`, `priority`, `due`) using a token from 1Password.
|
||||||
|
|
||||||
|
Today these are loosely coupled by fragile cross-links (`todoist://<task-id>` in notes, `[[note]]` text in Todoist comments). Hephaestus replaces the fragile coupling with **one unified data model and database** where tasks and knowledge are first-class, linkable entities.
|
||||||
|
|
||||||
|
### Guiding principles
|
||||||
|
|
||||||
|
- **Context-aware:** A task like "Fix the roof leak" is coupled to the relevant knowledge — the home-repair log, the log of contractor calls, etc. Surfacing that context is a core feature, not an afterthought.
|
||||||
|
- **Workflow-optimized:** Highly optimized, concise answers to recurring questions — above all **"What is next?"** — are the primary UX. Speed and concision beat feature breadth.
|
||||||
|
- **Owner-built, owner-hosted:** A standalone codebase replacing a loose stack of tools, hosted out of blumeops.
|
||||||
|
|
||||||
|
## 2. Goals & Non-Goals
|
||||||
|
|
||||||
|
### Goals (v1 / prototype)
|
||||||
|
|
||||||
|
- 🔒 Unified data model linking **notes** and **tasks** as first-class, cross-linkable entities.
|
||||||
|
- 🔒 SQLite backend.
|
||||||
|
- 🔒 Rust implementation.
|
||||||
|
- 🔒 **Distributed dual-mode operation:** useful fully offline, auto-syncs to a central self-hosted instance in blumeops (k3s container) when reachable. The central instance may be unavailable for long periods.
|
||||||
|
- 🔒 Surfaces over a shared core (via the `hephd` daemon): **CLI** (everyday driver), **nvim plugin** (primary editing UX), **web UI** (hub-hosted).
|
||||||
|
- 🔒 Optimized workflow queries, especially "What is next?", with context surfacing.
|
||||||
|
- 🔒 **Auth from the ground up** (OIDC/Authentik), with isolated multi-user accounts — sensitive data.
|
||||||
|
|
||||||
|
### Non-Goals (v1) / Later
|
||||||
|
|
||||||
|
- ⏳ iOS app + Apple Watch app for quick voice-dispatched task capture — explicitly a **follow-up goal**, not v1.
|
||||||
|
- ⏳ Sharing/collaboration between users — accounts are isolated for v1 (multi-user *isolation* is in scope; multi-user *collaboration* is not).
|
||||||
|
- ❌ Obsidian feature parity (canvas, graph view, plugins) — clean break.
|
||||||
|
- ❌ **Migration / import / Todoist sync in v1** — the prototype is a **clean break**: heph is the system of record from day one. No ZK import, no Todoist bridge. (Migration may return as a later, optional phase.)
|
||||||
|
- ❌ Encryption at rest / E2E in v1 — security model is **access restriction (OIDC auth) only**; plain SQLite. May revisit.
|
||||||
|
- ❌ Inferred/semantic context surfacing in v1 — explicit links + search only.
|
||||||
|
|
||||||
|
## 3. Data Model
|
||||||
|
|
||||||
|
> 🔒 **DECIDED (shape).** SQLite is the canonical store; a node's body is plain markdown text; an `export` mode materializes the whole store as a directory of `.md` files. Clean break from Obsidian-the-app, but markdown stays the portable payload. Tasks are **thin**; rich context lives in documents; everything is connected by typed links.
|
||||||
|
|
||||||
|
### Entities
|
||||||
|
|
||||||
|
- **Node** — the base entity. Every first-class thing is a node with a stable, sync-safe ID (see §3.1). A node has a `kind` discriminator. Kinds (initial set):
|
||||||
|
- **`doc`** — a rich markdown context document (the bulk of the knowledge base; replaces ZK articles, journals, logs). Body = markdown text. Has `aliases`, timestamps.
|
||||||
|
- **`task`** — **thin**: ID + state, plus a few scalar attributes (attention-state, do-date, late-on, recurrence) and links to a project/tags/context. **No prose body of its own** — context is supplied by linked `doc` nodes (or a doc *is* the work-log for the task). This is the key unification: ZK-articles-with-todos and Todoist-tasks-with-context both reduce to *typed links between thin tasks and rich docs*. The full task-attribute model is in §6.2 (it encodes the owner's actual lived prioritization system).
|
||||||
|
- **`project`** — a grouping node (maps to Todoist projects / ZK project dirs like `payrix/`).
|
||||||
|
- **`tag`** — a label node (so tags are linkable/queryable, not just strings).
|
||||||
|
- **`journal`** — a daily-note node keyed by date (`YYYY-MM-DD`).
|
||||||
|
- **Link** — a typed, directional edge between two nodes. Types (initial set): `wiki` (a `[[link]]` found in a doc body), `context-of` (doc ↔ task), `blocks`, `parent`/`child`, `tagged`, `in-project`. Wiki-links in markdown bodies are parsed and materialized as `wiki` links so the graph and the prose stay consistent.
|
||||||
|
|
||||||
|
### 3.1 Identity
|
||||||
|
|
||||||
|
- ❓ **Leaning ULID** (sortable, sync-safe, collision-free across offline devices) as the internal primary key, with a human-facing `slug`/`alias` layer on top so `[[wiki-links]]` can be written by name. The ZK's timestamp-IDs (`1722897441-MWFE`) are ULID-like already; a migration can map them.
|
||||||
|
|
||||||
|
### 3.2 Scalar task attributes vs. links
|
||||||
|
|
||||||
|
**Tags and projects are nodes** (linkable, queryable). **Attention-state, do-date, late-on, recurrence are scalar columns** on the task. See §6.2 for the semantics — these columns are not arbitrary; they encode a specific, battle-tested model. Due dates are *not* first-class nodes (a task has one project/context; "this Friday" doesn't need its own context).
|
||||||
|
|
||||||
|
### 3.3 Recurrence model (recurring tasks & checklists) — IN SCOPE for v1
|
||||||
|
|
||||||
|
A **recurring task** carries an **RFC-5545 RRULE** and acts as a recurring **definition**.
|
||||||
|
|
||||||
|
> ❌ **The anti-pattern we must avoid (Todoist's mistake):** modeling a recurring task's checklist sub-items as standalone, non-recurring tasks that **carry their completion state forward** when the parent recurs. This is the corner; designing around it now is *why* recurring checklists are in v1 rather than deferred.
|
||||||
|
|
||||||
|
**heph model (proposed — one point to ratify):**
|
||||||
|
|
||||||
|
- The recurring definition holds a **checklist template** (the ordered set of sub-item definitions).
|
||||||
|
- Each **occurrence** materializes a **fresh checklist instance** — brand-new context items, all `outstanding` — scoped to that occurrence. **Completion state is per-occurrence and never carries forward.**
|
||||||
|
- Occurrences are retained as history (no hard deletes; ties naturally to the per-task **log**, §6.4). The "what is next?" engine surfaces the **current/active occurrence**; completing it advances to the next per the RRULE.
|
||||||
|
|
||||||
|
**The one fork to confirm:**
|
||||||
|
|
||||||
|
- **(a) Occurrence instances (recommended):** the definition spawns a lightweight **instance** per occurrence (its own do-date + its own fresh checklist items). Full per-occurrence history; unambiguously "a fresh instance per recurrence"; heavier (more rows).
|
||||||
|
- **(b) Roll-forward in place:** a single task node; on completion, **log the finished occurrence** (to the per-task log) and **reset the checklist to outstanding** + advance the do-date. Lighter; history is narrative (in the log) rather than queryable per-occurrence.
|
||||||
|
|
||||||
|
Both satisfy the correctness requirement (fresh checklist, no carried state). (a) is recommended for clean history and zero ambiguity; (b) is the pragmatic-lite option.
|
||||||
|
|
||||||
|
## 4. Architecture
|
||||||
|
|
||||||
|
> 🔒 **DECIDED (shape).**
|
||||||
|
|
||||||
|
Layers, top to bottom:
|
||||||
|
|
||||||
|
- **Surfaces (thin clients):** the **nvim plugin (`heph.nvim`)** is the **primary surface for v1** — a full "org-mode"-style experience (markdown buffers backed by `doc` nodes; agenda / "what is next" / capture / linking / journaling as plugin commands). It is the explicit **successor to obsidian.nvim**, which the owner currently drives the ZK with (telescope picker, `<Leader>o*` verbs, `<Enter>` to follow `[[wiki-links]]`, dailies, multi-state checkboxes) and rarely uses Obsidian-proper alongside. heph.nvim must reach that feature parity (see §6.5) and replace it. The **CLI** (`heph`) is a secondary/utility surface (export, scripting, admin) — explicitly *not* a focus, though it shares the same command surface. The **web UI** is the occasional hub-served surface. A later iOS/Watch client talks to the hub directly.
|
||||||
|
- ❓ **Dual-mode binary:** a single Rust binary that can run as the **server/daemon** *or* expose the same operations as **CLI subcommands** *or* act as the **backend the nvim plugin drives**. Appealing (one codebase, one command surface) — to confirm as the packaging model.
|
||||||
|
- **Per-device daemon (`hephd`):** owns the local SQLite handle, the CRDT/op-log state, and background sync. All local surfaces connect to it over a local socket. This is what makes multi-surface access concurrent and safe on one SQLite file, and gives one place to run background sync.
|
||||||
|
- **Core crate (Rust lib):** data model, query engine, markdown parsing + wiki-link extraction, and sync logic. Linked by both `hephd` and the hub server.
|
||||||
|
- **Hub:** `hephd` running in "server" mode on blumeops k3s — central SQLite, the sync rendezvous point, and the web UI host. May be offline for long periods; devices keep working and reconcile when it returns.
|
||||||
|
|
||||||
|
Open points:
|
||||||
|
|
||||||
|
- ❓ nvim ↔ daemon transport: JSON-RPC over a unix socket (nvim has a native RPC channel; allows the daemon to *push* updates when sync brings in remote changes) vs. the plugin shelling out to the `heph` CLI. Leaning socket RPC.
|
||||||
|
- ❓ Web front-end style: server-rendered (Axum + HTMX/Askama) vs. WASM SPA (Leptos/Dioxus). Leaning **SSR + HTMX** since it's the lowest-traffic surface — favor simplicity.
|
||||||
|
|
||||||
|
## 5. Sync & Conflict Resolution
|
||||||
|
|
||||||
|
### Requirements (gathered from the owner)
|
||||||
|
|
||||||
|
- 🔒 ~99% of interaction today is via the **CLI on Gilbert** (dev laptop). Future split: ~90% Gilbert, ~9% phone/watch, ~1% web.
|
||||||
|
- 🔒 Must work **seamlessly across tailnet workstations** (Gilbert, ringtail, others): dispatch a change on one, walk to another, see it arrive "in short order."
|
||||||
|
- 🔒 **Genuine offline conflicts will occur** — e.g. Gilbert edits offline, then ringtail edits, then Gilbert reconnects. Must resolve **gracefully**: auto-merge where possible; for the unresolvable remainder, a **conflict queue + alert** ("you have N merge conflicts, run `heph conflicts`").
|
||||||
|
- 🔒 **Auth from the ground up** — this is extremely sensitive data. Use **Authentik (OIDC)** as IdP, support **multiple isolated user accounts** (in practice just `eblume`).
|
||||||
|
- 🔒 A **web UI** is part of the hosted (hub) model.
|
||||||
|
|
||||||
|
### Proposed model — *for confirmation*
|
||||||
|
|
||||||
|
A **hub-and-spoke, op-log + CRDT** design:
|
||||||
|
|
||||||
|
- **Topology:** each device runs `hephd` with a full local SQLite replica; the blumeops **hub** is the rendezvous. Devices push/pull an **append-only operation log** to the hub when reachable. Hub-and-spoke (not P2P mesh) — simpler, matches "central self-hosted app," and the hub is normally up on the tailnet so propagation is prompt. When the hub is down, devices keep working and the op-log drains on reconnect.
|
||||||
|
- **Merge semantics (so "graceful" is precise):**
|
||||||
|
- **`doc` bodies (markdown):** a **text CRDT** (e.g. Yrs/Automerge) → concurrent edits *always* merge with no hard conflict. This covers the common case.
|
||||||
|
- **Scalar task fields (completion, due, priority):** **last-writer-wins via Hybrid Logical Clocks**. Deterministic and cheap.
|
||||||
|
- **Links / set membership (tags, projects):** add/remove as a CRDT set (OR-Set) → no conflicts.
|
||||||
|
- **Conflict queue:** because CRDTs auto-merge, true blocking conflicts are rare. We *surface* (not block) the cases that are semantically ambiguous: a discarded LWW value on a meaningful field (e.g. completion flipped both ways, divergent due dates), or concurrent edits to overlapping text regions. These go to `heph conflicts` with an alert; sync never stalls.
|
||||||
|
- **Identity & ordering:** ULIDs for nodes (§3.1); HLC timestamps stamp every op for causal ordering across offline devices.
|
||||||
|
- **Auth:** the hub's sync + web endpoints require an **OIDC bearer token from Authentik**. Devices obtain tokens via the **OAuth device-code flow**; refresh token stored in the OS keychain / 1Password. Every node/op is owned by a `user_id`; the hub enforces per-user isolation. Offline devices operate on cached credentials.
|
||||||
|
|
||||||
|
### Open points
|
||||||
|
|
||||||
|
- ❓ **Hub-only vs. P2P fallback:** if the hub is down but Gilbert and ringtail are both on the tailnet, do they sync *directly*, or wait for the hub? Hub-only is simpler; direct P2P is more available. (Leaning hub-only for v1.)
|
||||||
|
- ❓ **Encryption at rest:** given "extremely sensitive," should each device's SQLite (and the hub's) be encrypted at rest (e.g. SQLCipher), and/or should op-log payloads be end-to-end encrypted so the hub stores ciphertext it can't read? (E2E complicates server-side search/web UI.)
|
||||||
|
- ❓ **Todoist's role** during/after migration — two-way bridge during transition, import-once, or retire on cutover?
|
||||||
|
- ❓ How "short order" is short — is a few seconds of propagation (push-based) needed, or is periodic pull (e.g. every 30–60s) fine?
|
||||||
|
|
||||||
|
## 6. Workflow Features
|
||||||
|
|
||||||
|
> Driven by "highly optimized specific workflows." heph is organized around a small set of **workflows**, each a first-class surface in heph.nvim:
|
||||||
|
>
|
||||||
|
> - **Task / "What is next?"** (§6.1–§6.3) — the Tactical / Strategic / Organizational task engine. The flagship.
|
||||||
|
> - **Log** (§6.4) — daily journal + optional append-only per-task work-logs ("narrative > list").
|
||||||
|
> - **Wiki / KB** (§6.5) — the whole context corpus as a browsable/queryable personal wiki; the direct **obsidian.nvim replacement**.
|
||||||
|
> - **Calendar** (§6.6) — entering via the calendar, tied to Strategic mode. *Deferred, but scaffolded carefully.*
|
||||||
|
|
||||||
|
### "What is next?" — the flagship
|
||||||
|
|
||||||
|
> ❓ **ACTIVE DEEP-DIVE.** This is the defining workflow and has ~10 years of the owner's prior art and prior tool attempts behind it. Being designed from that prior art rather than from scratch — see §6.1. The blumeops-tasks ranking (overdue-first, then priority, p3 = backlog) is a known reference point, not the target.
|
||||||
|
|
||||||
|
### Context surfacing — v1 scope
|
||||||
|
|
||||||
|
> 🔒 **DECIDED for v1.** Explicit links only; no inferred/semantic context yet.
|
||||||
|
|
||||||
|
- Every **task** automatically gets **one canonical `doc` (context) node**, created and linked on task creation. That canonical doc is the jumping-off point and can wiki-link onward to other docs/cards.
|
||||||
|
- nvim commands provide **telescope/fzf/rg-style search** over context docs, optionally with preloaded filters (e.g. scoped to a project).
|
||||||
|
- Inferred context (shared tags, full-text/semantic similarity) is deferred — large surface area, not the v1 focus.
|
||||||
|
|
||||||
|
### 6.1 "What is next?" — prior art & synthesis
|
||||||
|
|
||||||
|
> Distilled from ~10 years of the owner's thinking — predating the ZK by ~5 years — chiefly the owner's direct account (this design conversation), plus `zk/mole/20240202113646.todo.md` (the most developed written framework), daily logs (`zk/home/2023-09-01.log.md`, `2023-10-12.log.md`), and `zk/payrix/friction_log.md`. Prior tool code may exist at `github.com/eblume/{mole,hermes}` (check detached-head branches — several restarts); the owner expects little reusable there beyond what's captured here.
|
||||||
|
|
||||||
|
#### History: Hermes → Mole → heph (and the lessons)
|
||||||
|
|
||||||
|
- **Hermes — the "time accountant" (≈ earliest).** Goal: a *timeline*-based system fed "blueprints" of what a day should look like, with rules like *"check email twice a day, unless a high-priority email comes in — but even then no more than once an hour,"* continuously re-solving the calendar in real time. Built a working prototype using a **SAT solver (OR-tools)**. **Killer feature deduced: a quick, meaningful answer to "what is next" — here, the next thing on the calendar.** **Why it failed:** the calendar became a *battle zone*, swinging wildly every few minutes as the solver re-resolved the dependency graph → impossible to plan around. **Lesson + salvage:** there may be fruit here as a future, *opt-in* heph **planning mode**, but real-time auto-rescheduling of a visible calendar is anti-useful.
|
||||||
|
- **Mole — "whack-a-mole" rules engine.** A rules engine that constantly re-evaluated rules and materialized Todoist tasks (canonical first test: *"is there an active task 'whack the mole'? if not, make one"*). **"What is next" = "what Todoist says is available."** Worked reasonably. **Why it stalled:** Todoist accumulated **too many tasks**, so *choosing the actual next thing* became the whole problem again. This pushed the owner to design the Todoist project/priority/filter discipline in §6.2. **The deeper lesson:** Mole's automation **wasn't adding much value over the project/priority discipline itself** — so the owner *turned Mole off and just kept using Todoist*. The manual discipline was the real engine.
|
||||||
|
- **Today.** No automation beyond blumeops' **read-only** `blumeops-tasks` mise poller. Just Todoist + the §6.2 discipline — but as the direct descendant of all the above. **Implication for heph:** the value is in *embodying and strengthening the discipline* (and finally fixing the weak note↔task link), **not** in clever auto-scheduling or a rules engine. Earn any automation only after the discipline is faithfully supported.
|
||||||
|
|
||||||
|
#### "What's next?" is three questions, not one
|
||||||
|
|
||||||
|
The owner's framework splits the question by **mode**:
|
||||||
|
|
||||||
|
- **Tactical** — blank-slate "what do I do *right now*?" Stay in the editor/terminal; capture interruptions and resume in **milliseconds** ("save state and walk away in milliseconds"). This is the mode a single ranked list serves.
|
||||||
|
- **Strategic** — "I want to do X — what's the *next step*, or *where was I*?" Exploring a goal, generating the sub-tasks to reach it.
|
||||||
|
- **Organizational** — "I want to do X — help me *plan* the steps." Project-level enumeration of what needs doing, not how.
|
||||||
|
|
||||||
|
🔒 **REAFFIRMED.** The owner still strongly identifies with this split (the earlier "maybe" was conflating it with the unrelated CLI-first *process* note on the same page). The clean operational test, in the owner's words: **"looking at a project's list of todos = Organizational; deciding which to work on = Strategic; working a todo = Tactical."** Escalation flows upward (project-infra work can become Strategic, then Tactical).
|
||||||
|
|
||||||
|
**How heph embodies it** — as **named modes/views in the nvim plugin**, *not* an inference engine that guesses your mode:
|
||||||
|
|
||||||
|
- **Tactical (execution view):** you are "tactically engaging" a specific committed task. Loads its canonical context + sub-items into a **goal stack**; millisecond interrupt-capture pushes new context items onto the stack; "smallest amount of contextual guidance and next steps to stay productive." Includes **Chore Mode** (synthesize a goal stack from a query — e.g. the `Chores` project — and chug through one by one).
|
||||||
|
- **Strategic (planning view):** editor present but the goal is to *explore/decompose*, not edit. "To do X I must also do Y and Z." Produces a fleshed-out task with ready sub-items so Tactical can grab and go. **When Strategic completes, no work has been done** — only new/changed tasks, docs, links.
|
||||||
|
- **Organizational (survey view):** project-level enumeration — *what* needs doing, not *how*.
|
||||||
|
- **Shared primitive — the goal stack** (see §6.3) spans Tactical and Strategic; transitions between modes are first-class (Strategic *produces* → Tactical *consumes*).
|
||||||
|
|
||||||
|
#### The real cost is resumption, not ranking
|
||||||
|
|
||||||
|
The recurring pain in the logs is **reorienting after a context switch**, not deciding raw priority. Design rules that follow:
|
||||||
|
|
||||||
|
- **Resumption hint invariant:** when work pauses on a task/todo, it should leave at least one concrete next-step so future-you can re-enter quickly. heph should make capturing/“leaving a breadcrumb” frictionless and surface it first on return.
|
||||||
|
- **Narrative > list:** daily logs proved more motivating and contextual than flat task lists ("a log is the way"). Journaling/log nodes are first-class, and "what is next" should be able to lean on recent log narrative, not just task metadata.
|
||||||
|
|
||||||
|
#### The "weak link" is heph's reason to exist
|
||||||
|
|
||||||
|
The #1 unsolved problem: the **fragile, clunky coupling between Todoist and notes** (manually typing `nb o 45` into task descriptions). heph's thin-task ↔ auto-created canonical-context-doc model is the direct fix: the link is structural, not a hand-typed string.
|
||||||
|
|
||||||
|
#### Mapping prior taxonomy → heph entities
|
||||||
|
|
||||||
|
| Prior (`nb`/Todoist) | Meaning | heph |
|
||||||
|
|---|---|---|
|
||||||
|
| nb-project | grouping; completing all members completes it | `project` node |
|
||||||
|
| nb-todo | planning complex multi-step work; has "a single unified context log" | `task` + its **canonical context `doc`** |
|
||||||
|
| nb-task | immediate, short-lived "might eventually be done"; checking off = "no longer might be done" (not necessarily *completed*) | thin `task` node |
|
||||||
|
| Todoist | "deciding what is next and knowing what needs to be done now" | heph's "what is next?" engine |
|
||||||
|
| Calendar | scheduling / future | due dates (+ later calendar integration) |
|
||||||
|
|
||||||
|
> 🔒 **Preserved.** The nb-task semantic — checking off means "no longer outstanding," *not* "accomplished" — carries into heph. **Context items are `outstanding` / `not-outstanding`** (see §6.3). For **committed tasks**, heph distinguishes end-states: **done** (accomplished) vs. **dropped/dismissed** (let go, e.g. during a Blue review). Both are "not outstanding"; the distinction is kept for honesty and history.
|
||||||
|
|
||||||
|
#### Process steer (now historical)
|
||||||
|
|
||||||
|
The owner's old rule — "avoid ncurses and interactive UIs; write atomic code and call it manually with subcommands… until a strong pattern emerges" — was a *focus device for hand-building tools*. It no longer constrains heph: we go **straight to the nvim org-mode plugin backed by the Rust server** (the patterns are now well understood; see §6.2/§6.3). The CLI remains as a secondary/utility surface.
|
||||||
|
|
||||||
|
#### Distilled requirements for heph's "what is next?" — *to confirm/refine*
|
||||||
|
|
||||||
|
1. **Mode-aware** (Tactical / Strategic / Organizational), not a single global sort.
|
||||||
|
2. **Sub-millisecond capture** and fast state-save (Tactical interrupt handling).
|
||||||
|
3. **Resumption-first:** on returning to a task, surface the last breadcrumb/next-step before anything else.
|
||||||
|
4. **Context-coupled:** every task one keystroke away from its canonical context doc and onward links.
|
||||||
|
5. **Log-aware:** can factor in / surface recent daily-log narrative.
|
||||||
|
6. **A sane default ranking** for the Tactical blank-slate case — the concrete model is §6.2, not the blumeops-tasks placeholder.
|
||||||
|
|
||||||
|
### 6.2 The lived priority discipline (the working basis for heph)
|
||||||
|
|
||||||
|
> 🔒 **This is the real model.** The owner has used this Todoist discipline daily for years; Mole was eventually *turned off* because this discipline outvalued the automation. heph should **embody and strengthen** it. Captured verbatim-in-spirit from the owner's account.
|
||||||
|
|
||||||
|
**Projects = contexts (a task has exactly one).** Examples: `Life` (catch-all), `Work`, `Chores`, `Maintenance`, `Outside`, `Allison` (wife), `Errands`, `Cooking`, `Health`. A project *contextualizes* a task; it is not a plan.
|
||||||
|
|
||||||
|
**Attention-state (thought of by color, not number):**
|
||||||
|
|
||||||
|
| State | Meaning |
|
||||||
|
|---|---|
|
||||||
|
| **White** (default) | Able to be done **once its do-date arrives**. (See do-date semantics below.) |
|
||||||
|
| **Orange** | **Top of mind.** Keep **≤ 6**, reassessed *every day*. Goal: you know your top-of-minds *without looking at the tool*. |
|
||||||
|
| **Red** | Top of mind **+ there will be a consequence if not done ASAP**. Red means **the existence of a consequence, NOT its severity/importance.** Flagging by *importance* is a trap → anxiety (everything due feels important). |
|
||||||
|
| **Blue** | **On Deck** (backlog). Doable, but deliberately *cooling off*. The pressure-relief valve that keeps the working set light. |
|
||||||
|
|
||||||
|
**Do-date, not due-date (key insight):** "due date" means **DO date** — *don't do it before this date*. Optionally a separate **"Late On"** marker for when it actually becomes a problem. **Due dates are a terrible planning tool** — calendar-style deadlines make everything "yell for attention" constantly. heph must treat the do-date as *earliest actionable*, not a deadline alarm.
|
||||||
|
|
||||||
|
**Working-set discipline (the healthy tensions — a feature, not a bug):**
|
||||||
|
|
||||||
|
- Keep White+Orange+Red ≤ **~30 total**, or the system collapses into noise. Keep Orange ≤ **6**.
|
||||||
|
- Push freely to Blue to stay light; periodically **review Blue: keep vs. let go** (this is where "drop/dismiss" happens). Target On-Deck **< 100** (owner is at ~149 and hates it).
|
||||||
|
- These tensions **map the owner's emotional state to the task system** and should be *surfaced honestly*. heph must avoid **two failure modes**: (1) **"fake happy"** — telling you everything's fine when it isn't; (2) **overwhelm** — yelling that you're buried until you freeze and make no progress. Reflect true state; help maintain the tensions.
|
||||||
|
|
||||||
|
**Filters = saved views (queries over the above):**
|
||||||
|
|
||||||
|
- **Top of Mind** — all Red + Orange.
|
||||||
|
- **Tasks / Work Tasks** — Red/Orange/White that are *actionable now* (do-date arrived), scoped by project.
|
||||||
|
- **Chores** — like Tasks but from `Chores`, *slightly tuned down* so the (near-universally recurring) chores don't become a nuisance.
|
||||||
|
- **On Deck** — all Blue.
|
||||||
|
- **Schedule** — recurring checklists from a few projects (e.g. *Morning Routine* → brush teeth, feed birds, coffee, medicine). The owner uses *this particular filter* sparingly, but the underlying **recurring-checklist mechanism is IN SCOPE for v1** (see §3.3): the only way to be sure we avoid Todoist's mistake is to model it up front. **Each recurrence gets a fresh checklist instance; completion never carries forward.**
|
||||||
|
|
||||||
|
**heph's default "what is next?" (Tactical blank-slate)** therefore ≈ Top of Mind (Red first — by *consequence*, then Orange) **+** White items whose do-date has arrived, scoped to the current project/context, **with Blue hidden**. Concise, honest, light.
|
||||||
|
|
||||||
|
### 6.3 Two kinds of task: commitments vs. context items
|
||||||
|
|
||||||
|
> 🔒 **DECIDED (shape).** A **commitment axis** orthogonal to the §6.2 attention-states.
|
||||||
|
|
||||||
|
- **Committed task** — long-lived, strongly tied to context (e.g. "Complete PLAT-1234", "Build a nursery"). **Participates in §6.2**: has an attention-state, a project, a do-date; is a candidate for "what is next"; may carry sub-structure and a web of context.
|
||||||
|
- **Context item** — ephemeral, created mid-flow ("six things before this is done", "oh crap, another" pushed on the stack). **Does *not* enter the §6.2 scheduling system** — it is *part of its containing task's context*, with only **outstanding / not-outstanding** state. Never appears in global "what is next."
|
||||||
|
- **Promotion** is first-class: a context item can be **promoted to a committed task** when you decide to actually schedule it (the old "nb-todo → Todoist-task on activation").
|
||||||
|
- **Ephemeral ≠ deleted:** deletes effectively never happen — tombstones + a dedicated cleanup mode only. "Ephemeral" is lifecycle/visibility, not storage. (This also keeps the CRDT sync simple.)
|
||||||
|
- **The goal stack** is a **session-scoped, synthesized construct**: seeded from a committed task (Tactical) or from a query over contexts (Chore Mode — "grind on some chores"). It holds committed tasks and/or context items for the duration of a work session.
|
||||||
|
|
||||||
|
#### Editing surface for context items — *deliberately deferred to the prototype*
|
||||||
|
|
||||||
|
> 🔑 **Key realization: this is NOT a data-model fork.** In both options below the **stored representation is identical** — context items are outstanding nodes linked to their container; the daemon owns them; markdown `export` renders them as checkboxes regardless. The choice is *purely the nvim editing affordance*, so the backend is shared and we **prototype both and pick by feel** (ergonomics can't be judged on paper).
|
||||||
|
|
||||||
|
- **Option A — checkboxes in the body, derived on save.** Edit the task's canonical context doc as plain markdown; `- [ ]` lines are materialized into context-item state on `:w` (`BufWritePost`), reusing the wiki-link extraction machinery. **No real-time scanning needed** (debounced live updates are optional polish). Remote CRDT edits are *pushed* from the daemon to reconcile an open buffer. Item identity is low-stakes (reword = tombstone-old + add-new) since items are ephemeral; identity is pinned only at **promotion**. Most "personal org-mode"-native; millisecond capture = "type a line."
|
||||||
|
- **Option B — structured items, rendered.** Items are nodes from the start; the body stays pure prose. Render inline via **virtual text/extmarks** (no pane) or a transient **floating window** (avoids the disliked persistent cutaway); capture via a leader chord + `vim.ui.input`. Cleaner separation and trivial promotion; heavier capture; more UI to build.
|
||||||
|
- Lean: **A** for capture flow, but **build the shared node backend first** and trial both affordances in the plugin.
|
||||||
|
|
||||||
|
> **Prior-art note (multi-state checkboxes):** the owner's obsidian.nvim config uses checkbox states `" ", "x", ">", "~", "!"` (todo / done / forwarded / ~ / important). heph's task/context-item state model should accommodate richer-than-binary states (at least the done/dropped/outstanding set in §6.1; possibly forwarded/deferred).
|
||||||
|
|
||||||
|
### 6.4 The Log workflow
|
||||||
|
|
||||||
|
> 🔒 **In scope for v1** (daily journal certainly; per-task logs as the model allows). Logs embody the "narrative > list" insight and the resumption-hint invariant from §6.1.
|
||||||
|
|
||||||
|
Two related shapes:
|
||||||
|
|
||||||
|
- **Daily journal.** Today driven by the `zkd` abbrev → `Obsidian dailies`: a telescope picker to create/open dated journal buffers. heph keeps this as a **first-class** feature: `journal` nodes keyed by date, a picker to jump to today / recent / create. The daily journal is the catch-all narrative stream.
|
||||||
|
- **Per-task log.** Sometimes a task wants an append-only "here's what I was working on" side-commentary. It **half-belongs to the task's context and half doesn't** — so model it as a **separate buffer/node from the task's canonical context doc**, optionally present only for tasks that need longer-duration commentary. Properties:
|
||||||
|
- **Append-only** (entries accrete; you don't rewrite history).
|
||||||
|
- **Dispatchable from an isolated context:** you must be able to fire a log entry *without leaving / losing your place in* the context buffer you're navigating — i.e. append to the log from elsewhere (a quick-capture into the task's log), while the context buffer stays where it is. (This is the ergonomic crux; likely a quick-capture command targeting "the current task's log".)
|
||||||
|
- The per-task log is a strong candidate to *be* the resumption breadcrumb store (§6.1).
|
||||||
|
|
||||||
|
❓ Open: is the per-task log a distinct node `kind` (`log`, append-only), or just a `doc` flagged append-only and linked `log-of` → task? Leaning a dedicated append-only `log` kind to make "dispatch an entry" a cheap, well-defined op.
|
||||||
|
|
||||||
|
### 6.5 The Wiki / KB workflow
|
||||||
|
|
||||||
|
> 🔒 **First-class in v1.** This is the "view all my contexts as one big personal wiki" surface — the direct replacement for obsidian.nvim.
|
||||||
|
|
||||||
|
The owner estimates ~90% of the value comes from a few well-worn primitives, so heph.nvim should reach **obsidian.nvim feature parity** on day one:
|
||||||
|
|
||||||
|
- **Follow `[[wiki-links]]` on `<Enter>`** to jump between context docs (the core navigation gesture).
|
||||||
|
- **Telescope-based** search (`rg`), quick-switch between docs, tag browse, **backlinks**, outgoing **links** — mirroring the owner's `<Leader>o*` verbs (to be rebound under heph's own prefix).
|
||||||
|
- **Logical layout + tagging** of context docs so the corpus is browsable/queryable as a whole.
|
||||||
|
- New-doc-from-query and insert-link from the picker (obsidian.nvim's `<C-x>`/`<C-l>`).
|
||||||
|
|
||||||
|
heph.nvim **replaces** obsidian.nvim outright — the owner already rarely opens Obsidian itself. This workflow is the knowledge-base analogue of the Organizational task view: surveying and traversing the `doc` graph.
|
||||||
|
|
||||||
|
### 6.6 The Calendar workflow
|
||||||
|
|
||||||
|
> ⏳ **Deferred (later version) — but scaffold now, don't paint ourselves into a corner.** Ties closely to **Strategic** mode (planning against time).
|
||||||
|
|
||||||
|
- The owner uses **Calendar.app** bound to a **Gmail calendar** (almost certainly **CalDAV**; Gmail also exposes ICS/WebCal feeds — to confirm when implemented).
|
||||||
|
- Eventual goal: **integration between calendar views and task views** — enter the system *via* the calendar, see tasks/do-dates in temporal context.
|
||||||
|
- ⚠️ **Hard-won gotcha:** enabling Todoist's gcal integration **flooded the calendar with a million events** because of recurring rules. heph must **never explode recurrence into individual calendar events**. This echoes the Hermes "battle zone" failure (§6.1) — calendar integration must be *read-mostly, careful, de-duplicated, and opt-in*, not a firehose.
|
||||||
|
- **Scaffolding guidance (decisions to keep clean now):** keep a crisp separation between *tasks/do-dates* and *calendar events*; model recurrence as RRULEs that are **expanded lazily for display, never materialized as stored events**; design sync so a future **read-only CalDAV ingestion** (calendar → heph) is easy, and treat any heph → calendar export as a later, explicitly bounded, non-exploding feature.
|
||||||
|
|
||||||
|
## 7. Hosting & Deployment
|
||||||
|
|
||||||
|
Reuse the established blumeops patterns (🔒 confirmed by repo conventions):
|
||||||
|
|
||||||
|
- Containerized service deployed to **k3s** (ringtail cluster) via **ArgoCD app-of-apps** + **Kustomize** raw manifests (no Helm/Flux).
|
||||||
|
- Secrets via **1Password + external-secrets operator** (`ClusterSecretStore onepassword-blumeops`).
|
||||||
|
- Image built via **Dagger + Forgejo Actions**, pushed to **Zot** registry (`registry.ops.eblu.me`).
|
||||||
|
- ❓ Rust build approach for the container: Nix (like kingfisher) vs. Dagger-native `cargo` multi-stage build.
|
||||||
|
- ❓ Persistence for the central instance's SQLite — NFS PVC (sifaka) vs. local PV. (SQLite over NFS has known locking caveats.)
|
||||||
|
- ❓ How do clients reach the central instance — Tailscale tailnet only, or public via Caddy/Fly proxy?
|
||||||
|
|
||||||
|
## 8. Technology Choices (to decide)
|
||||||
|
|
||||||
|
- ❓ nvim plugin stack: Lua + JSON-RPC over a socket to `hephd` (with daemon→plugin push) vs. CLI shell-out. Telescope as the picker dependency.
|
||||||
|
- ❓ Web framework (Axum + HTMX/Askama? Leptos? Dioxus?).
|
||||||
|
- ❓ SQLite access layer (sqlx, rusqlite, SeaORM, Diesel) and migrations.
|
||||||
|
- ❓ CRDT / sync library (Automerge-rs, Yrs, bespoke op-log).
|
||||||
|
- ❓ Markdown parser/renderer (pulldown-cmark, comrak) and wiki-link / checkbox extraction.
|
||||||
|
|
||||||
|
## 9. Open Questions Summary
|
||||||
|
|
||||||
|
### Decided
|
||||||
|
|
||||||
|
- ✅ Source of truth: **SQLite-primary**, markdown as the body payload, with a directory `export` mode. Clean break from Obsidian-the-app.
|
||||||
|
- ✅ Note/task model: **typed node graph** — thin `task` nodes, rich `doc` nodes, connected by typed links.
|
||||||
|
- ✅ Front-end: **per-device `hephd` daemon** with thin clients; **nvim plugin** as primary editing UX, **CLI** as everyday driver, **web UI** on the hub.
|
||||||
|
- ✅ Sync (shape): **hub-and-spoke, op-log + CRDT**, HLC ordering, OIDC/Authentik auth, conflict queue for the ambiguous remainder. (Details below still open.)
|
||||||
|
|
||||||
|
### Decided since (B/C/D)
|
||||||
|
|
||||||
|
- ✅ Context surfacing v1: **explicit links only** + auto-created canonical context doc per task + nvim fuzzy search. Inferred/semantic deferred.
|
||||||
|
- ✅ **v1 is distributed, not local-only:** client/server + **offline-first sync with automatic merge + conflict queue** + **full OIDC/Authentik auth with per-user isolation** are all IN v1 (tech-spec §12/§13). Web UI and k3s deployment are later (hub binary is built deployable).
|
||||||
|
- ✅ Security v1: **OIDC/Authentik auth + per-user isolation** is the boundary; plain SQLite at rest (no encryption). May revisit.
|
||||||
|
- ✅ Migration v1: **clean break**, no ZK import / no Todoist bridge. heph is system of record day one.
|
||||||
|
- ✅ **License: All Rights Reserved / private.** Open-sourcing considered later.
|
||||||
|
|
||||||
|
### Decided (workflows)
|
||||||
|
|
||||||
|
- ✅ heph organized around first-class workflows (§6): **Task** (what-is-next), **Log**, **Wiki/KB**, **Calendar**.
|
||||||
|
- ✅ **Log** (§6.4) in scope: daily journal (picker over dated `journal` nodes, à la `zkd`/`Obsidian dailies`) + optional **append-only per-task work-logs** (separate buffer from the canonical context doc; dispatchable without leaving the context buffer).
|
||||||
|
- ✅ **Wiki/KB** (§6.5) first-class: heph.nvim reaches **obsidian.nvim feature parity** (follow `[[links]]` on `<Enter>`, telescope search/quick-switch/tags/backlinks/links) and **replaces** it.
|
||||||
|
- ✅ **Calendar** (§6.6) deferred but **scaffolded carefully**: never explode recurrence into stored events (Todoist-gcal-flood / Hermes lesson); keep tasks vs. events separate; lazy RRULE expansion; design for a future read-only CalDAV ingest.
|
||||||
|
|
||||||
|
### Decided (workflow model)
|
||||||
|
|
||||||
|
- ✅ "What is next?" rests on the **lived priority discipline** (§6.2): projects-as-contexts; attention-states White/Orange/Red/Blue (Red = *consequence*, not severity); **do-date not due-date**; working-set tensions surfaced honestly (avoid "fake happy" *and* overwhelm).
|
||||||
|
- ✅ **Tactical / Strategic / Organizational** reaffirmed as **named modes/views** in the nvim plugin, sharing a **goal-stack** primitive (§6.1).
|
||||||
|
- ✅ **Commitment axis** (§6.3): committed tasks (in §6.2 system) vs. ephemeral context items (outstanding/not-outstanding, scoped to a container); **promotion** is first-class; **no hard deletes** (tombstones + cleanup mode).
|
||||||
|
- ✅ Task end-states: **done** vs. **dropped/dismissed** (both "not outstanding").
|
||||||
|
- ✅ Prototype goes **straight to the nvim plugin + Rust server**; CLI is secondary.
|
||||||
|
- ✅ **Rust stack RATIFIED** (§11.2): rusqlite + tokio/JSON-RPC + pulldown-cmark + ulid + rrule + clap.
|
||||||
|
- ✅ **Recurrence + recurring checklists IN SCOPE for v1** (§3.3): RRULE-driven; **each occurrence gets a fresh checklist instance, completion never carries forward** (avoids Todoist's corner).
|
||||||
|
|
||||||
|
### Still open
|
||||||
|
|
||||||
|
*Prototype-blocking (resolve at kickoff — see §11 / tech-spec):*
|
||||||
|
|
||||||
|
1. **Recurrence model (§3.3): (a) occurrence-instances vs (b) roll-forward-in-place** — both avoid the corner; leaning (a).
|
||||||
|
2. **CRDT library** for body merge: `yrs` (leaning) vs `automerge`; bespoke op-log/HLC for scalar fields either way.
|
||||||
|
3. **Hub network transport** (tech-spec §6.1): `axum` HTTP/JSON (leaning) vs gRPC; sync propagation cadence (push vs periodic pull).
|
||||||
|
4. **Context-item editing surface (§6.3): Option A vs B** — *decided inside the prototype by feel* (shared backend).
|
||||||
|
|
||||||
|
*Not prototype-blocking (later phases):*
|
||||||
|
|
||||||
|
5. Per-task **log** representation: dedicated append-only `log` node kind vs. `doc` flagged append-only (§6.4).
|
||||||
|
6. Calendar protocol confirmation (CalDAV vs ICS/WebCal) and the careful, non-exploding integration (§6.6).
|
||||||
|
7. Web front-end style (SSR + HTMX vs. WASM SPA).
|
||||||
|
8. k3s deployment specifics: Rust container build (Nix vs Dagger-native cargo); central SQLite persistence (local PV vs NFS); client reachability (tailnet-only vs public).
|
||||||
|
9. Encryption at rest / E2E (currently out; may revisit).
|
||||||
|
|
||||||
|
## 10. Roadmap (provisional)
|
||||||
|
|
||||||
|
- **Phase 0 — Design** (this document + [[tech-spec]]): done enough to build.
|
||||||
|
- **Phase 1 — v1 prototype (one C1 effort, built in TDD slices):** `heph-core` (model, schema, extraction, recurrence, "what is next", op-log/HLC/CRDT merge) → `hephd` client mode → **hub mode + offline sync + conflict queue** → **OIDC/Authentik auth + per-user isolation** → `heph.nvim` + `heph` CLI. Runnable client/server, offline-capable, on the tailnet.
|
||||||
|
- **Phase 2 — k3s deployment:** Dagger→Zot image, ArgoCD app + Kustomize manifests, external-secrets; hub on blumeops.
|
||||||
|
- **Phase 3 — Web UI** on the hub.
|
||||||
|
- **Phase 4 (later, optional)** — calendar integration (careful CalDAV); migration from ZK / Todoist; iOS / Apple Watch voice capture; Hermes-style planning mode.
|
||||||
|
|
||||||
|
## 11. v1 Prototype — Scope, Stack, and Kickoff
|
||||||
|
|
||||||
|
> The handoff target for the next context session. Items here are **proposals to ratify**; once ratified, a fresh session can build directly from this.
|
||||||
|
|
||||||
|
> **The canonical, implementation-ready scope/stack/schema/API now live in [[tech-spec]].** This section keeps the *intent* and the kickoff process; defer to the spec for details (and update the spec when decisions change).
|
||||||
|
|
||||||
|
### 11.1 Scope — *distributed from day one* (ratified)
|
||||||
|
|
||||||
|
v1 is **not** local-only. It is a **client/server, offline-first** system:
|
||||||
|
|
||||||
|
- **In:** the full data model (§3, §6) incl. **recurrence + recurring checklists** (§3.3); the "what is next?" engine; **client/server architecture** (per-device client `hephd` + a runnable/deployable hub `hephd`); **offline-first op-log + CRDT sync with automatic merge and a conflict queue** (tech-spec §12); **OIDC/Authentik auth with per-user isolation** (tech-spec §13); the `heph.nvim` + `heph` CLI surfaces.
|
||||||
|
- **Out (later, scaffolded):** the **web UI** (hub serves sync only in v1); **actual k3s deployment** to blumeops (hub binary is built deployable; ArgoCD/Kustomize/Dagger is a fast-follow); calendar; iOS/Watch; inferred context; P2P-over-tailnet fallback; persistent goal stack (an in-plugin session stack suffices).
|
||||||
|
|
||||||
|
**Why this bigger v1:** the distributed/offline/merge behavior and the auth model are *architecturally load-bearing* — bolting them on later would mean reworking the core. The owner explicitly wants them proven from the start. (Recurrence-with-checklists is in for the same reason — see §3.3.)
|
||||||
|
|
||||||
|
### 11.2 Stack — ✅ RATIFIED
|
||||||
|
|
||||||
|
Core: `rusqlite` (bundled) + migrations · `tokio` + JSON-RPC/unix-socket (local) · `ulid` · `rrule` · `pulldown-cmark` · `clap` · `anyhow`/`thiserror` · `tracing`. Distributed/auth additions (some to confirm at kickoff): text CRDT `yrs` (vs `automerge`) for bodies + bespoke op-log/HLC for fields · hub transport `axum`/`reqwest` · OIDC via `openidconnect` + `keyring`. Full detail and the SQLite schema: **tech-spec §4.5, §10, §12, §13**.
|
||||||
|
|
||||||
|
### 11.3 First-session kickoff checklist
|
||||||
|
|
||||||
|
1. `mise run ai-docs`; read **[[tech-spec]]** (the build spec) and this doc's §3/§6 for rationale.
|
||||||
|
2. **Classify as C1** — greenfield, no existing system to Mikado-untangle. Single long-lived feature branch + early draft PR, docs-first, push as you go ([[agent-change-process]]).
|
||||||
|
3. Scaffold the cargo workspace + `heph.nvim` skeleton; **fill the AGENTS.md Project Structure section** (last template TODO).
|
||||||
|
4. Build outward in testable slices (TDD, tech-spec §2/§9): `heph-core` (schema, model, extraction, recurrence, "what is next", **op-log/HLC/CRDT merge**) → `hephd` client mode (local RPC) → **hub mode + sync + conflict queue** → **OIDC auth** → `heph.nvim` + `heph` CLI.
|
||||||
|
5. Confirm the kickoff-open picks: recurrence model (a vs b, §3.3), CRDT lib (`yrs` vs `automerge`), hub transport, context-item editing surface (A vs B, §6.3).
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- [[tech-spec]] — clean implementation-facing technical specification distilled from this document
|
||||||
|
- [[ai-assistance-guide]] — conventions for AI agents in this repo
|
||||||
|
- [[agent-change-process]] — C0/C1/C2 change methodology
|
||||||
|
|
@ -10,4 +10,4 @@ tags:
|
||||||
|
|
||||||
Background context and design decisions.
|
Background context and design decisions.
|
||||||
|
|
||||||
<!-- TODO: Add explanation entries as the project grows -->
|
- [[design]] — Hephaestus design document: vision, data model, architecture, sync, and roadmap
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@ const config: QuartzConfig = {
|
||||||
enablePopovers: true,
|
enablePopovers: true,
|
||||||
analytics: null,
|
analytics: null,
|
||||||
locale: "en-US",
|
locale: "en-US",
|
||||||
baseUrl: "CHANGEME.example.com", // TODO: Update to your docs site URL
|
baseUrl: "localhost", // TODO: set to hosted docs domain once published (see blumeops docs.eblu.me pattern)
|
||||||
ignorePatterns: ["private", "templates", ".obsidian"],
|
ignorePatterns: ["private", "templates", ".obsidian"],
|
||||||
defaultDateType: "modified",
|
defaultDateType: "modified",
|
||||||
theme: {
|
theme: {
|
||||||
|
|
|
||||||
|
|
@ -10,11 +10,15 @@ tags:
|
||||||
|
|
||||||
Technical reference material for the repository tooling that ships with this project.
|
Technical reference material for the repository tooling that ships with this project.
|
||||||
|
|
||||||
|
## Project
|
||||||
|
|
||||||
|
- [[tech-spec]] — Hephaestus technical specification (data model, RPC API, "what is next?" ranking, recurrence, testing strategy, v1 scope)
|
||||||
|
|
||||||
## Template Surface Area
|
## Template Surface Area
|
||||||
|
|
||||||
| Path | Purpose |
|
| Path | Purpose |
|
||||||
|------|---------|
|
|------|---------|
|
||||||
| `.dagger/src/project_template_ci/` | Dagger module that builds the Quartz docs tarball used by releases |
|
| `.dagger/src/hephaestus_ci/` | Dagger module that builds the Quartz docs tarball used by releases |
|
||||||
| `.forgejo/workflows/build.yaml` | Generic CI validation workflow |
|
| `.forgejo/workflows/build.yaml` | Generic CI validation workflow |
|
||||||
| `.forgejo/workflows/release.yaml` | Manual release workflow that versions, builds docs, and publishes release assets |
|
| `.forgejo/workflows/release.yaml` | Manual release workflow that versions, builds docs, and publishes release assets |
|
||||||
| `.forgejo/scripts/` | Optional project-specific hooks consumed by the workflows |
|
| `.forgejo/scripts/` | Optional project-specific hooks consumed by the workflows |
|
||||||
|
|
@ -64,5 +68,4 @@ Technical reference material for the repository tooling that ships with this pro
|
||||||
|
|
||||||
## TODO After Templating
|
## TODO After Templating
|
||||||
|
|
||||||
- TODO: Set `baseUrl` in `docs/quartz.config.ts` to the hosted docs domain, or `localhost` if the docs are only previewed locally
|
- TODO: Set `baseUrl` in `docs/quartz.config.ts` to the hosted docs domain once published (currently `localhost`)
|
||||||
- TODO: Rename `.dagger/src/project_template_ci/` and the exported Dagger class to match the generated project
|
|
||||||
|
|
|
||||||
301
docs/reference/tech-spec.md
Normal file
301
docs/reference/tech-spec.md
Normal file
|
|
@ -0,0 +1,301 @@
|
||||||
|
---
|
||||||
|
title: Technical Specification
|
||||||
|
modified: 2026-05-31
|
||||||
|
tags:
|
||||||
|
- reference
|
||||||
|
- design
|
||||||
|
---
|
||||||
|
|
||||||
|
# Hephaestus — Technical Specification
|
||||||
|
|
||||||
|
> Clean, implementation-facing spec for the v1 prototype. For the *why* behind every choice (history, prior art, decision trail), see [[design]]. Where this spec and the design doc disagree, the design doc's latest decision wins — file an update here.
|
||||||
|
|
||||||
|
## 1. Overview
|
||||||
|
|
||||||
|
Hephaestus (heph) is a self-hosted personal context-management system that unifies a markdown knowledge base with task management in one database. **v1 is a distributed client/server system**: each device runs a local replica that works fully **offline**, and syncs to a central **hub** when reachable, with **automatic merge + conflict resolution** for concurrent offline edits. Access is authenticated via **OIDC (Authentik)** with per-user data isolation. The web UI and the actual k3s deployment are later phases; the hub ships in v1 as a runnable/deployable binary. Components:
|
||||||
|
|
||||||
|
- **`heph-core`** — Rust library: data model, SQLite store, query engine, markdown parsing/extraction, recurrence, and the sync engine (op-log, HLC, CRDT merge, conflict detection).
|
||||||
|
- **`hephd`** — Rust daemon with two modes:
|
||||||
|
- **client mode** (per device): owns the local SQLite replica; serves a JSON-RPC API over a unix socket to local surfaces; runs background sync to the hub.
|
||||||
|
- **server/hub mode**: owns the central SQLite; serves the authenticated sync endpoint (and, later, the web UI) over the network.
|
||||||
|
- **`heph`** — Rust CLI: utility/admin surface (export, scripting, smoke tests, `heph conflicts`).
|
||||||
|
- **`heph.nvim`** — Lua Neovim plugin: the primary user surface ("org-mode"-style); a thin client of the local `hephd`.
|
||||||
|
|
||||||
|
## 2. Development approach
|
||||||
|
|
||||||
|
**Development is test-driven (TDD).** Write the failing test first; implement to green; refactor. No feature is "done" without tests at the appropriate layer(s) in §9. Core logic must be **deterministic and clock-injected** (no ambient wall-clock reads in `heph-core`; the current time is always passed in) so ranking and recurrence are testable.
|
||||||
|
|
||||||
|
## 3. Architecture
|
||||||
|
|
||||||
|
- Surfaces (`heph.nvim`, `heph` CLI) are thin clients; they never touch SQLite directly. All state operations go through the local `hephd` over the unix socket. Socket path default: `$XDG_RUNTIME_DIR/heph/hephd.sock`.
|
||||||
|
- Each device's `hephd` (client mode) is the single writer/owner of that device's local SQLite replica (WAL mode), and works **fully offline**. It records every local mutation as an op in an append-only **op-log** and runs **background sync** to the hub.
|
||||||
|
- The **hub** `hephd` (server mode) is the **hub-and-spoke rendezvous**: devices push/pull ops to it over the network (authenticated, §13). It is *not* required to be online — devices keep working and reconcile when it returns.
|
||||||
|
- Merge is automatic where possible; the unresolvable remainder goes to a **conflict queue** surfaced to the user (§12). Sync never blocks local work.
|
||||||
|
- `heph-core` is synchronous and side-effect-light (incl. deterministic merge logic); `hephd` wraps it with async I/O, transport, and auth (`tokio`). DB calls run on a blocking pool.
|
||||||
|
- See **§12 (Sync & Conflict Resolution)** and **§13 (Authentication)** for the detailed models.
|
||||||
|
|
||||||
|
## 4. Data model
|
||||||
|
|
||||||
|
All first-class entities are **nodes**; relationships are **links**. Markdown bodies are stored in SQLite; files are an export artifact, not the source of truth.
|
||||||
|
|
||||||
|
### 4.1 Node kinds
|
||||||
|
|
||||||
|
| kind | meaning | body |
|
||||||
|
|---|---|---|
|
||||||
|
| `doc` | rich context document (knowledge base, work-logs, journals) | markdown |
|
||||||
|
| `task` | thin task or ephemeral context item (see §4.3) | none (context via links) |
|
||||||
|
| `project` | grouping/context for tasks | optional |
|
||||||
|
| `tag` | label | optional |
|
||||||
|
| `journal` | daily note, titled by ISO date | markdown |
|
||||||
|
|
||||||
|
### 4.2 Link types
|
||||||
|
|
||||||
|
`wiki` (materialized from `[[links]]` in a body), `canonical-context` (task → its auto-created context doc), `context-of`, `log-of` (task → its append-only log), `blocks`, `parent`, `tagged`, `in-project`.
|
||||||
|
|
||||||
|
### 4.3 Task semantics
|
||||||
|
|
||||||
|
- **Attention-state** (required on committed tasks): `white` (do once do-date arrives), `orange` (top of mind), `red` (top of mind + a consequence exists if late — *consequence, not severity*), `blue` (on-deck/backlog).
|
||||||
|
- **do-date** = *earliest actionable date* ("do date"), **not a deadline**. Optional **late-on** marks when lateness becomes a problem.
|
||||||
|
- **Commitment axis:** `committed = 1` tasks participate in scheduling/"what is next"; `committed = 0` are **ephemeral context items** scoped to a container (`container_id`), with only `outstanding`/done states, never surfaced globally. Context items may be **promoted** to committed tasks.
|
||||||
|
- **States:** `outstanding`, `done`, `dropped` (done and dropped are both "not outstanding"; the distinction is retained).
|
||||||
|
- **No hard deletes:** everything uses `tombstoned`; physical deletion only in an explicit cleanup mode.
|
||||||
|
|
||||||
|
### 4.4 Recurrence (§3.3 of [[design]])
|
||||||
|
|
||||||
|
A `task` with a non-null `recurrence` (RFC-5545 RRULE) is a **recurring definition**. Sub-items flagged `is_template = 1` form its **checklist template**. **Each occurrence produces a fresh checklist instance** (new `outstanding` context items copied from the template); **completion never carries forward across occurrences** — this is a hard requirement.
|
||||||
|
|
||||||
|
Two candidate implementations (pick at kickoff; (a) is the lean):
|
||||||
|
- **(a) Occurrence instances:** definition spawns a `task_occurrences` row per occurrence, each with its own do-date and fresh checklist items. Full history.
|
||||||
|
- **(b) Roll-forward in place:** single node; on completion, log the occurrence, reset the checklist to `outstanding`, advance the do-date.
|
||||||
|
|
||||||
|
### 4.5 SQLite schema (starting point)
|
||||||
|
|
||||||
|
```
|
||||||
|
nodes(
|
||||||
|
id TEXT PRIMARY KEY, -- ULID
|
||||||
|
owner_id TEXT NOT NULL REFERENCES users(id), -- per-user isolation
|
||||||
|
kind TEXT NOT NULL, -- doc|task|project|tag|journal
|
||||||
|
title TEXT NOT NULL,
|
||||||
|
body TEXT, -- markdown (nullable); materialized view of body_crdt
|
||||||
|
body_crdt BLOB, -- text-CRDT state for the body (merge), nullable
|
||||||
|
created_at INTEGER NOT NULL, -- epoch ms
|
||||||
|
modified_at INTEGER NOT NULL,
|
||||||
|
hlc TEXT NOT NULL, -- hybrid logical clock of last write (sync ordering)
|
||||||
|
tombstoned INTEGER NOT NULL DEFAULT 0
|
||||||
|
)
|
||||||
|
|
||||||
|
tasks(
|
||||||
|
node_id TEXT PRIMARY KEY REFERENCES nodes(id),
|
||||||
|
attention TEXT, -- white|orange|red|blue (committed tasks)
|
||||||
|
do_date INTEGER, -- epoch ms, nullable
|
||||||
|
late_on INTEGER, -- epoch ms, nullable
|
||||||
|
state TEXT NOT NULL, -- outstanding|done|dropped
|
||||||
|
committed INTEGER NOT NULL, -- 1 committed task, 0 context item
|
||||||
|
container_id TEXT REFERENCES nodes(id), -- context item → container
|
||||||
|
recurrence TEXT, -- RRULE; present = recurring definition
|
||||||
|
is_template INTEGER NOT NULL DEFAULT 0 -- checklist-template item
|
||||||
|
)
|
||||||
|
|
||||||
|
-- recurrence model (a) only:
|
||||||
|
task_occurrences(
|
||||||
|
id TEXT PRIMARY KEY, -- ULID
|
||||||
|
def_id TEXT NOT NULL REFERENCES nodes(id),
|
||||||
|
occurrence_date INTEGER NOT NULL,
|
||||||
|
state TEXT NOT NULL, -- outstanding|done|dropped|skipped
|
||||||
|
created_at INTEGER NOT NULL,
|
||||||
|
tombstoned INTEGER NOT NULL DEFAULT 0
|
||||||
|
)
|
||||||
|
|
||||||
|
links(
|
||||||
|
id TEXT PRIMARY KEY, -- ULID
|
||||||
|
src_id TEXT NOT NULL REFERENCES nodes(id),
|
||||||
|
dst_id TEXT NOT NULL REFERENCES nodes(id),
|
||||||
|
type TEXT NOT NULL,
|
||||||
|
created_at INTEGER NOT NULL,
|
||||||
|
tombstoned INTEGER NOT NULL DEFAULT 0
|
||||||
|
)
|
||||||
|
|
||||||
|
aliases(node_id TEXT REFERENCES nodes(id), alias TEXT) -- wiki-link name resolution
|
||||||
|
nodes_fts -- FTS5 over title, body
|
||||||
|
|
||||||
|
-- identity & sync --
|
||||||
|
users(
|
||||||
|
id TEXT PRIMARY KEY, -- ULID
|
||||||
|
oidc_sub TEXT UNIQUE NOT NULL, -- OIDC subject (Authentik)
|
||||||
|
name TEXT,
|
||||||
|
created_at INTEGER NOT NULL
|
||||||
|
)
|
||||||
|
|
||||||
|
oplog( -- append-only operation log (the sync unit)
|
||||||
|
id TEXT PRIMARY KEY, -- ULID
|
||||||
|
owner_id TEXT NOT NULL REFERENCES users(id),
|
||||||
|
hlc TEXT NOT NULL, -- hybrid logical clock (causal order)
|
||||||
|
origin TEXT NOT NULL, -- originating device id
|
||||||
|
op_type TEXT NOT NULL, -- node.create|node.body_delta|task.set_field|link.add|link.remove|...
|
||||||
|
target_id TEXT NOT NULL,
|
||||||
|
payload TEXT NOT NULL, -- JSON (e.g. CRDT delta, field+value, OR-Set add/remove)
|
||||||
|
applied INTEGER NOT NULL DEFAULT 0
|
||||||
|
)
|
||||||
|
|
||||||
|
sync_state( -- per-peer cursor (device ↔ hub)
|
||||||
|
peer TEXT PRIMARY KEY, -- 'hub' on a client; device id on the hub
|
||||||
|
last_pushed_hlc TEXT,
|
||||||
|
last_pulled_hlc TEXT,
|
||||||
|
updated_at INTEGER NOT NULL
|
||||||
|
)
|
||||||
|
|
||||||
|
conflicts( -- ambiguous merges surfaced to the user
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
owner_id TEXT NOT NULL REFERENCES users(id),
|
||||||
|
node_id TEXT NOT NULL REFERENCES nodes(id),
|
||||||
|
field TEXT NOT NULL, -- which field / 'body-region'
|
||||||
|
local_val TEXT, remote_val TEXT,
|
||||||
|
local_hlc TEXT, remote_hlc TEXT,
|
||||||
|
status TEXT NOT NULL, -- open|resolved
|
||||||
|
created_at INTEGER NOT NULL
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Projects/tags are `nodes`; membership is `links` (`in-project`, `tagged`). All `tasks`/`task_occurrences`/`links` rows inherit ownership via their node(s).
|
||||||
|
|
||||||
|
## 5. Markdown handling
|
||||||
|
|
||||||
|
- Bodies are stored verbatim. On write (node create/update), `heph-core` extracts:
|
||||||
|
- `[[wiki-links]]` → `wiki` links (resolved via `aliases`/title; unresolved links are allowed and recorded).
|
||||||
|
- GFM task-list items (`- [ ]` / `- [x]`) → context-item state under the node (Option A editing model — see [[design]] §6.3; the alternative is command-driven items, decided in-prototype).
|
||||||
|
- Extraction is idempotent and diff-based: re-writing an unchanged body is a no-op; reworded checklist lines tombstone-old + add-new (context items are cheap).
|
||||||
|
- `export` materializes all non-tombstoned nodes to a directory tree of `.md` files (frontmatter + body), reproducing the corpus portably.
|
||||||
|
|
||||||
|
## 6. Daemon RPC API (JSON-RPC over unix socket)
|
||||||
|
|
||||||
|
Methods (request → response; errors are JSON-RPC errors). Signatures are indicative, not final:
|
||||||
|
|
||||||
|
- `node.get(id) → Node`
|
||||||
|
- `node.create({kind, title, body?}) → Node`
|
||||||
|
- `node.update({id, title?, body?}) → Node` (body update re-runs extraction)
|
||||||
|
- `node.tombstone(id) → ok`
|
||||||
|
- `task.create({title, project?, attention?, do_date?, late_on?, recurrence?, committed?}) → Task` (auto-creates the canonical context `doc` + `canonical-context` link)
|
||||||
|
- `task.set_state({id, state}) → Task` (recurring: advances per §4.4)
|
||||||
|
- `task.set_attention({id, attention}) → Task`
|
||||||
|
- `task.promote({context_item_id, attention?, project?}) → Task`
|
||||||
|
- `next({scope?, limit?}) → [RankedTask]` (the "what is next?" query, §7)
|
||||||
|
- `search({query, filters?}) → [Node]` (FTS)
|
||||||
|
- `links.outgoing(id) → [Link]` / `links.backlinks(id) → [Link]`
|
||||||
|
- `journal.open_or_create(date) → Node`
|
||||||
|
- `log.append({task_id, text}) → ok` (append to the task's `log-of` node)
|
||||||
|
- `export({path}) → {count}`
|
||||||
|
- `health() → {orange_count, active_count, on_deck_count, conflict_count, sync_status, ...}` (working-set + sync indicators)
|
||||||
|
- **auth:** `auth.login() → {device_code_flow...}` / `auth.status() → {user, logged_in}` / `auth.logout()` (§13)
|
||||||
|
- **sync:** `sync.now() → {pushed, pulled, conflicts}` (force a sync cycle; background sync runs automatically) / `sync.status() → {last_pushed, last_pulled, pending_ops, online}`
|
||||||
|
- **conflicts:** `conflicts.list() → [Conflict]` / `conflicts.resolve({id, choice}) → ok`
|
||||||
|
|
||||||
|
Every local mutation method records an op in the **op-log** for sync. The local daemon supports **server-push** notifications (e.g. a node changed by an incoming sync) so an open buffer can reconcile in real time.
|
||||||
|
|
||||||
|
### 6.1 Hub (server-mode) sync endpoint
|
||||||
|
|
||||||
|
Separate from the local unix-socket RPC, the hub exposes an **authenticated network endpoint** (HTTP/JSON or gRPC — pick at kickoff) for op exchange: clients present an OIDC bearer token (§13); the hub validates, scopes by `owner_id`, accepts pushed ops, and returns ops the client hasn't seen (by HLC cursor). The hub applies the same `heph-core` merge logic.
|
||||||
|
|
||||||
|
## 7. "What is next?" ranking
|
||||||
|
|
||||||
|
Given an optional `scope` (project/context) and `limit` (default 5):
|
||||||
|
|
||||||
|
1. **Candidates:** committed tasks where `state = outstanding`, not tombstoned, `attention ≠ blue`, and actionable now: `do_date IS NULL OR do_date ≤ now`. For recurring definitions, evaluate the current active occurrence's do-date. Apply `scope` if given.
|
||||||
|
2. **Order:**
|
||||||
|
1. attention: `red` → `orange` → `white`;
|
||||||
|
2. urgency: tasks past `late_on` first, then most-overdue (smallest `do_date`) first;
|
||||||
|
3. tie-break: earlier `do_date`, then `created_at`.
|
||||||
|
3. **Output:** concise rows — title, project, attention, do/late, link to canonical context. `red` items always appear regardless of `limit`.
|
||||||
|
|
||||||
|
`blue` (on-deck) is hidden from `next` by design; surfaced only by an explicit on-deck view. `health()` exposes the working-set tensions (orange ≤ 6, active ≤ ~30, on-deck count) honestly — never masking overload nor manufacturing calm.
|
||||||
|
|
||||||
|
## 8. heph.nvim surface (v1)
|
||||||
|
|
||||||
|
Replaces obsidian.nvim. Telescope-backed. Core commands/gestures:
|
||||||
|
|
||||||
|
- Follow `[[wiki-link]]` under cursor on `<Enter>`.
|
||||||
|
- Search / quick-switch / tags / backlinks / outgoing links (pickers).
|
||||||
|
- Daily journal picker (create/open dated `journal` nodes).
|
||||||
|
- Task capture; show "what is next" (`:Heph next`); set attention; mark done/dropped.
|
||||||
|
- Open a task's canonical context doc; edit context-item checkboxes (Option A) in the buffer (extracted on `:w`).
|
||||||
|
- Per-task log quick-append without leaving the current buffer.
|
||||||
|
|
||||||
|
## 9. Testing strategy (TDD, layered)
|
||||||
|
|
||||||
|
All layers are required; CI runs them on every push/PR (extend `.forgejo/scripts/build` to run `cargo test` and the nvim e2e suite; `prek` already runs in `build.yaml`).
|
||||||
|
|
||||||
|
- **Unit (`heph-core`):** model invariants; markdown extraction (wiki-links, checkboxes); **RRULE expansion and the fresh-checklist-per-occurrence rule** (assert completion never carries forward); the "what is next?" ranking (table-driven cases); migration up/down.
|
||||||
|
- **Property tests (`proptest`):** ranking yields a total order; extraction is idempotent; recurrence never leaks completion state across occurrences; tombstones are never resurrected.
|
||||||
|
- **Integration (`hephd`, real sockets):** start a daemon against a **temp SQLite file**, connect over a **real unix socket**, and exercise the RPC API end-to-end, asserting resulting DB state. Include **multi-client concurrency** tests on the socket and clock-injection for deterministic time.
|
||||||
|
- **Sync & offline (multi-replica):** spin up **two client `hephd` replicas + a hub `hephd`**, all over **real network sockets** against temp DBs, and assert convergence:
|
||||||
|
- online round-trip: edit on A → appears on B via the hub;
|
||||||
|
- **offline → reconcile:** partition A and B from the hub, make divergent edits on each, reconnect, assert both converge;
|
||||||
|
- **conflict path:** concurrent conflicting scalar edits (e.g. both set a different do-date) land in the **conflict queue** and `conflicts.resolve` settles them deterministically;
|
||||||
|
- **body CRDT merge:** concurrent edits to the same `doc` body auto-merge without a hard conflict;
|
||||||
|
- HLC ordering and op-log idempotency (replaying ops is a no-op).
|
||||||
|
- **Auth:** OIDC token validation on the hub endpoint (reject missing/invalid/expired); **per-user isolation** (user A cannot read/sync user B's nodes); device-code flow happy path against a mock IdP.
|
||||||
|
- **End-to-end (headless nvim):** drive `heph.nvim` in `nvim --headless` against a real `hephd` + temp DB, running **scripted example workflows** and asserting outcomes (via RPC/DB state and buffer contents). Minimum workflows:
|
||||||
|
- capture a task → appears in `:Heph next` → open canonical context → add a checklist item → check it → mark task done;
|
||||||
|
- create today's journal via the picker;
|
||||||
|
- follow a `[[link]]` on `<Enter>` to the target doc;
|
||||||
|
- **a recurring task with a checklist: complete it, then assert the next occurrence presents a fresh, all-unchecked checklist;**
|
||||||
|
- **a sync-driven update arrives while a buffer is open and the buffer reconciles.**
|
||||||
|
- Harness: `plenary.nvim`/busted, or drive nvim via its msgpack-RPC from the test runner. Keep example workflows as reusable fixtures.
|
||||||
|
- **CLI tests:** invoke `heph` subcommands against a temp DB; snapshot output; assert `export` round-trips the corpus; `heph conflicts` lists/resolves.
|
||||||
|
|
||||||
|
## 10. Technology stack (ratified)
|
||||||
|
|
||||||
|
`rusqlite` (bundled) + migration runner · `tokio` + line-delimited JSON-RPC over unix socket · `ulid` · `rrule` · `pulldown-cmark` · `clap` · `anyhow`/`thiserror` · `tracing`. Neovim plugin in Lua, depending on `telescope.nvim`. Cargo workspace: `crates/heph-core`, `crates/hephd`, `crates/heph`, plus `heph.nvim/`.
|
||||||
|
|
||||||
|
**Added for v1 client/server + auth (some to confirm at kickoff):**
|
||||||
|
|
||||||
|
- **Text CRDT (body merge):** `yrs` (Rust Yjs) — *leaning*; alternative `automerge`. Used for `doc`/`journal`/log bodies. Structured fields use a bespoke op-log + HLC (no library needed).
|
||||||
|
- **HLC:** small bespoke hybrid-logical-clock (or a crate) — deterministic, clock-injected.
|
||||||
|
- **Hub network transport:** `axum` (HTTP/JSON) for the sync endpoint — *leaning* (reuses the eventual web-UI server); `reqwest` on the client side.
|
||||||
|
- **OIDC:** `openidconnect` crate for the Authentik device-code flow; tokens cached in the OS keychain (`keyring`) / 1Password.
|
||||||
|
|
||||||
|
## 11. v1 scope
|
||||||
|
|
||||||
|
**In:**
|
||||||
|
|
||||||
|
- The full data model, markdown handling, "what is next?" ranking, and **recurrence + recurring checklists** (§4–§8).
|
||||||
|
- **Client/server architecture:** per-device client `hephd` + a central hub `hephd` (runnable/deployable binary).
|
||||||
|
- **Offline-first** operation with **op-log + CRDT sync** and **automatic merge + a conflict queue** (§12).
|
||||||
|
- **OIDC/Authentik authentication** with **per-user data isolation** (§13).
|
||||||
|
- `heph.nvim` + `heph` CLI surfaces (incl. `heph conflicts`).
|
||||||
|
|
||||||
|
**Out (later phases, scaffolded so as not to block):**
|
||||||
|
|
||||||
|
- **Web UI** (the hub serves sync only in v1; reserve `axum` for it later).
|
||||||
|
- **Actual k3s deployment** to blumeops (Dagger→Zot image, ArgoCD app + Kustomize manifests, external-secrets) — fast-follow once the architecture is proven; the hub binary is built to be deployable.
|
||||||
|
- **Calendar** integration (read-mostly CalDAV; **never explode recurrence into stored events**), iOS/Watch capture, inferred/semantic context, P2P-over-tailnet sync fallback.
|
||||||
|
|
||||||
|
See [[design]] §5–§7 for the constraints later phases impose on present choices (keep tasks vs. calendar events separate; expand RRULEs lazily).
|
||||||
|
|
||||||
|
## 12. Sync & conflict resolution
|
||||||
|
|
||||||
|
**Topology:** hub-and-spoke. Each device holds a full local replica + op-log; the hub is the rendezvous. Devices **push/pull ops by HLC cursor**; the hub never needs to be online for local work.
|
||||||
|
|
||||||
|
**Merge semantics (the unit of sync is the op):**
|
||||||
|
|
||||||
|
- **`doc`/`journal`/log bodies:** **text CRDT** (`yrs`) → concurrent edits always merge, no hard conflict.
|
||||||
|
- **Scalar task fields** (attention, do_date, late_on, state, …): **last-writer-wins by HLC**. The losing value, if meaningful, is recorded in `conflicts` (surfaced, not silently dropped).
|
||||||
|
- **Links / set membership** (tags, project, parent): **OR-Set** add/remove semantics → no conflicts.
|
||||||
|
- **Tombstones**, never hard deletes → deletion/merge is monotonic and CRDT-friendly.
|
||||||
|
|
||||||
|
**Conflict queue:** the unresolvable/ambiguous remainder (a discarded LWW value on a meaningful field; flagged overlapping body regions) becomes an `open` row in `conflicts`. Surfaced via `health().conflict_count`, `conflicts.list`, `heph conflicts`, and a heph.nvim view: *"you have N conflicts."* `conflicts.resolve({id, choice})` settles each. **Sync never blocks** on conflicts.
|
||||||
|
|
||||||
|
**Determinism:** HLCs are clock-injected; op application is idempotent and order-independent given HLC. These are the core invariants the sync tests assert.
|
||||||
|
|
||||||
|
**Open at kickoff:** CRDT lib confirmation (`yrs` vs `automerge`); hub transport (`axum` HTTP/JSON vs gRPC); propagation cadence (push vs. periodic pull); exactly which fields are "meaningful" enough to enqueue vs. silently LWW.
|
||||||
|
|
||||||
|
## 13. Authentication
|
||||||
|
|
||||||
|
- **OIDC against Authentik.** Clients authenticate via the **OAuth 2.0 device-code flow** (`auth.login`); the resulting tokens are cached in the OS keychain (`keyring`) / 1Password and refreshed automatically. Offline devices operate on cached credentials.
|
||||||
|
- **Hub enforcement:** the sync endpoint requires a valid **OIDC bearer token**; the hub maps the token's `sub` to a `users` row and **scopes every op by `owner_id`**. No cross-user reads/writes.
|
||||||
|
- **Per-user isolation:** all nodes (and their dependent rows) carry `owner_id`; queries and sync are always user-scoped. In practice a single user (`eblume`), but the isolation is real from day one.
|
||||||
|
- **Local trust:** the local unix-socket RPC trusts the OS user (file-permission-scoped socket); app-level auth is for the **network** boundary (device ↔ hub).
|
||||||
|
- **At-rest:** plain SQLite in v1 (no encryption) — security boundary is auth + (eventually) network restriction. May revisit (see [[design]]).
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- [[design]] — full design document with rationale and decision history
|
||||||
|
|
@ -100,7 +100,7 @@ This project ships two Forgejo workflows:
|
||||||
- `build.yaml` runs on pushes and pull requests targeting `main`, executes `prek run --all-files`, and then runs an optional project hook at `.forgejo/scripts/build` when present.
|
- `build.yaml` runs on pushes and pull requests targeting `main`, executes `prek run --all-files`, and then runs an optional project hook at `.forgejo/scripts/build` when present.
|
||||||
- `release.yaml` is a manual workflow that computes the next version, optionally builds `CHANGELOG.md` from fragments, packages Quartz docs via Dagger, runs an optional `.forgejo/scripts/release` hook for extra assets, creates the Forgejo release, and pushes changelog updates back to `main` when fragments were consumed.
|
- `release.yaml` is a manual workflow that computes the next version, optionally builds `CHANGELOG.md` from fragments, packages Quartz docs via Dagger, runs an optional `.forgejo/scripts/release` hook for extra assets, creates the Forgejo release, and pushes changelog updates back to `main` when fragments were consumed.
|
||||||
|
|
||||||
- TODO: Rename `.dagger/src/project_template_ci/` and the exported Dagger class during first-time setup.
|
The Dagger module lives at `.dagger/src/hephaestus_ci/` and exports the `HephaestusCi` class.
|
||||||
|
|
||||||
## Common Pitfalls
|
## Common Pitfalls
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue