generated from eblume/project-template
heph-tui + PWA cosmetic polish: humanized recurrence, scrolling/indented/counted project sidebar #10
2 changed files with 88 additions and 0 deletions
doc(explanation): hub+spoke data-evolution / migration rules
All checks were successful
Build / validate (pull_request) Successful in 6m18s
All checks were successful
Build / validate (pull_request) Successful in 6m18s
Document why heph's op-based sync lets most new features (new link types, read-side queries, optional payload fields) ship without a coordinated migration across the hub and spokes, and the narrow case — a new required SQLite column the apply path writes — that does need a hub-first rollout. Groundwork for the indented/counted project sidebar, which is pure read-side (existing parent links + a GROUP BY) and needs no migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
commit
00da36c637
|
|
@ -12,3 +12,4 @@ Background context and design decisions.
|
|||
|
||||
- [[design]] — Hephaestus design document: vision, data model, architecture, sync, and roadmap
|
||||
- [[task-lifecycle]] — the two-axis task model (lifecycle state × attention), drop vs delete, and where each task shows up
|
||||
- [[hub-spoke-data-evolution]] — why op-based sync lets most new features skip migrations, and when a coordinated SQLite migration is actually required
|
||||
|
|
|
|||
87
docs/explanation/hub-spoke-data-evolution.md
Normal file
87
docs/explanation/hub-spoke-data-evolution.md
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
---
|
||||
title: Hub + Spoke Data Evolution
|
||||
modified: 2026-06-05
|
||||
tags:
|
||||
- explanation
|
||||
- sync
|
||||
---
|
||||
|
||||
# Hub + Spoke Data Evolution
|
||||
|
||||
How the data model evolves safely when nodes run different versions across the
|
||||
hub/spoke deployment (indri is the hub; see [[set-up-sync-hub]] and
|
||||
[[host-heph-pwa]]). The short version: **sync is op-based, not schema-based**, so
|
||||
most new features need no coordinated migration — but adding a SQLite *column*
|
||||
does.
|
||||
|
||||
## Two independent layers
|
||||
|
||||
heph keeps two layers that evolve on different clocks:
|
||||
|
||||
1. **The op-log (synced).** Every change is an operation — `node.create`,
|
||||
`node.set`, `task.set`, `link.add`, `link.remove`, … — carrying an HLC, an
|
||||
origin device, and a JSON payload. Spokes push/pull ops to/from the hub; both
|
||||
sides run the **same** merge logic from `heph-core` (`sqlite/apply.rs`). This
|
||||
is the only thing that crosses the wire.
|
||||
2. **The SQLite schema (local, per node).** Each node materializes ops into local
|
||||
tables. The schema version is tracked by SQLite's `PRAGMA user_version` and
|
||||
advanced by the ordered, append-only migration list in
|
||||
`heph-core/src/sqlite/migrations.rs`. **No schema or migration state is ever
|
||||
synced.** A spoke can sit on an older schema than the hub indefinitely.
|
||||
|
||||
Because the wire format is ops — not rows — a node only has to understand the
|
||||
*ops* its peers emit, not their table layout.
|
||||
|
||||
## What forward/backward compatibility already buys you
|
||||
|
||||
The merge engine is deliberately lenient:
|
||||
|
||||
- **Unknown op types are stored but not applied** (`apply.rs`) — a spoke that
|
||||
receives a newer op type keeps it in the log (so a later upgrade can replay it)
|
||||
but doesn't choke on it.
|
||||
- **Unknown payload fields are ignored.** Field extraction is by name
|
||||
(`str_field` / `i64_field`), so a payload with extra keys an older node doesn't
|
||||
recognize just drops the extras.
|
||||
- **Links are schema-free.** A link's `type` is a string column. A brand-new link
|
||||
kind (a new `LinkType`) needs no migration — every version reads it as text and
|
||||
applies OR-set add/remove identically.
|
||||
|
||||
## The rule of thumb
|
||||
|
||||
| Change | Needs coordinated migration? |
|
||||
|--------|------------------------------|
|
||||
| New `LinkType` (e.g. a new relationship between nodes) | **No** — just emit `link.add` with the new `type` string |
|
||||
| New optional/nullable scalar carried in an op payload | **No, if** every node's `apply` reads it defensively and tolerates its absence |
|
||||
| New *read-side* feature over existing data (counts, hierarchy from existing `parent` links) | **No** — pure local queries, no op or schema change |
|
||||
| New **required** SQLite column that `apply` must write on every relevant op | **Yes** — old spokes lack the column and the `UPDATE` fails |
|
||||
| Renaming/removing a column other nodes' `apply` paths reference | **Yes** |
|
||||
|
||||
## When a migration *is* required, do it hub-first
|
||||
|
||||
If a change genuinely needs a new column that the apply path writes:
|
||||
|
||||
1. Ship the migration to **every** node (hub and all spokes) **before** any node
|
||||
emits an op that depends on the new column. The migration list is
|
||||
append-only and ordered, so rolling the new `hephd` out everywhere is the
|
||||
gate.
|
||||
2. Keep new columns **nullable / defaulted** so an op that predates the column
|
||||
still applies, and so a node that hasn't yet upgraded degrades to "field
|
||||
absent" rather than erroring.
|
||||
3. Prefer encoding the new fact as a **link or an op-payload field** over a new
|
||||
column whenever you can — that keeps the change in the no-migration column of
|
||||
the table above.
|
||||
|
||||
## Worked example: indented, counted projects
|
||||
|
||||
The sidebar's subproject indentation and per-project task counts (see
|
||||
[[install-heph]] and the agenda surface in [[design]] §8.1) are a pure read-side
|
||||
feature:
|
||||
|
||||
- **Nesting** is read from `parent` links that already exist — created by
|
||||
`heph project add <name> --parent <parent>` — via the existing
|
||||
`project_subtree` traversal.
|
||||
- **Counts** are a read-only `SELECT … GROUP BY` over the `tasks`/`links` tables.
|
||||
|
||||
No new column, no new op type, no migration — it works against a hub and a spoke
|
||||
on any schema version that already understands `parent` links. That is the case
|
||||
the rule of thumb is meant to make obvious.
|
||||
Loading…
Add table
Add a link
Reference in a new issue