heph-tui + PWA cosmetic polish: humanized recurrence, scrolling/indented/counted project sidebar #10

Merged
eblume merged 2 commits from feature/tui-polish-project-tree into main 2026-06-06 09:29:11 -07:00
2 changed files with 88 additions and 0 deletions
Showing only changes of commit 00da36c637 - Show all commits

doc(explanation): hub+spoke data-evolution / migration rules
All checks were successful
Build / validate (pull_request) Successful in 6m18s

Document why heph's op-based sync lets most new features (new link types,
read-side queries, optional payload fields) ship without a coordinated
migration across the hub and spokes, and the narrow case — a new required
SQLite column the apply path writes — that does need a hub-first rollout.

Groundwork for the indented/counted project sidebar, which is pure read-side
(existing parent links + a GROUP BY) and needs no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Erich Blume 2026-06-05 17:31:11 -07:00

View file

@ -12,3 +12,4 @@ Background context and design decisions.
- [[design]] — Hephaestus design document: vision, data model, architecture, sync, and roadmap
- [[task-lifecycle]] — the two-axis task model (lifecycle state × attention), drop vs delete, and where each task shows up
- [[hub-spoke-data-evolution]] — why op-based sync lets most new features skip migrations, and when a coordinated SQLite migration is actually required

View file

@ -0,0 +1,87 @@
---
title: Hub + Spoke Data Evolution
modified: 2026-06-05
tags:
- explanation
- sync
---
# Hub + Spoke Data Evolution
How the data model evolves safely when nodes run different versions across the
hub/spoke deployment (indri is the hub; see [[set-up-sync-hub]] and
[[host-heph-pwa]]). The short version: **sync is op-based, not schema-based**, so
most new features need no coordinated migration — but adding a SQLite *column*
does.
## Two independent layers
heph keeps two layers that evolve on different clocks:
1. **The op-log (synced).** Every change is an operation — `node.create`,
`node.set`, `task.set`, `link.add`, `link.remove`, … — carrying an HLC, an
origin device, and a JSON payload. Spokes push/pull ops to/from the hub; both
sides run the **same** merge logic from `heph-core` (`sqlite/apply.rs`). This
is the only thing that crosses the wire.
2. **The SQLite schema (local, per node).** Each node materializes ops into local
tables. The schema version is tracked by SQLite's `PRAGMA user_version` and
advanced by the ordered, append-only migration list in
`heph-core/src/sqlite/migrations.rs`. **No schema or migration state is ever
synced.** A spoke can sit on an older schema than the hub indefinitely.
Because the wire format is ops — not rows — a node only has to understand the
*ops* its peers emit, not their table layout.
## What forward/backward compatibility already buys you
The merge engine is deliberately lenient:
- **Unknown op types are stored but not applied** (`apply.rs`) — a spoke that
receives a newer op type keeps it in the log (so a later upgrade can replay it)
but doesn't choke on it.
- **Unknown payload fields are ignored.** Field extraction is by name
(`str_field` / `i64_field`), so a payload with extra keys an older node doesn't
recognize just drops the extras.
- **Links are schema-free.** A link's `type` is a string column. A brand-new link
kind (a new `LinkType`) needs no migration — every version reads it as text and
applies OR-set add/remove identically.
## The rule of thumb
| Change | Needs coordinated migration? |
|--------|------------------------------|
| New `LinkType` (e.g. a new relationship between nodes) | **No** — just emit `link.add` with the new `type` string |
| New optional/nullable scalar carried in an op payload | **No, if** every node's `apply` reads it defensively and tolerates its absence |
| New *read-side* feature over existing data (counts, hierarchy from existing `parent` links) | **No** — pure local queries, no op or schema change |
| New **required** SQLite column that `apply` must write on every relevant op | **Yes** — old spokes lack the column and the `UPDATE` fails |
| Renaming/removing a column other nodes' `apply` paths reference | **Yes** |
## When a migration *is* required, do it hub-first
If a change genuinely needs a new column that the apply path writes:
1. Ship the migration to **every** node (hub and all spokes) **before** any node
emits an op that depends on the new column. The migration list is
append-only and ordered, so rolling the new `hephd` out everywhere is the
gate.
2. Keep new columns **nullable / defaulted** so an op that predates the column
still applies, and so a node that hasn't yet upgraded degrades to "field
absent" rather than erroring.
3. Prefer encoding the new fact as a **link or an op-payload field** over a new
column whenever you can — that keeps the change in the no-migration column of
the table above.
## Worked example: indented, counted projects
The sidebar's subproject indentation and per-project task counts (see
[[install-heph]] and the agenda surface in [[design]] §8.1) are a pure read-side
feature:
- **Nesting** is read from `parent` links that already exist — created by
`heph project add <name> --parent <parent>` — via the existing
`project_subtree` traversal.
- **Counts** are a read-only `SELECT … GROUP BY` over the `tasks`/`links` tables.
No new column, no new op type, no migration — it works against a hub and a spoke
on any schema version that already understands `parent` links. That is the case
the rule of thumb is meant to make obvious.