@niscorp/tide 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/DESIGN.md ADDED
@@ -0,0 +1,1061 @@
1
+ # Tide — Design Document
2
+
3
+ > **Status: built, and once reviewed hard.** The engine, the store contract and the
4
+ > grammar are implemented and tested; see [DOCS.md](./DOCS.md) for the API. This is the
5
+ > design answer to [`automation-requirements.md`](../../docs/archive/automation-requirements.md),
6
+ > which stays authoritative on *what must be true*; this says *how*. Where the two
7
+ > disagree, one of them has rotted — report it, don't guess.
8
+ >
9
+ > A section marked **⟲** records something this package got wrong and what replaced it.
10
+ > They are kept because the wrong version was plausible, and a reader who does not know
11
+ > why it was wrong will propose it again.
12
+
13
+ ## Purpose
14
+
15
+ Every nisc package answers one question: vex answers *what*, nova answers *how it
16
+ looks*, charter answers *who may*, prism answers *into what shape*, cortex *thinks*.
17
+ Nobody answers **when**. Tide is the answer: a host-blind automation engine in which
18
+ work initiated by the clock, by a change, by an external event, or by a person is
19
+ declared as data, executed as one named effect, and accounted for in a durable ledger.
20
+
21
+ **One sentence:** a reflex turns the clock and the fact into one named effect, through
22
+ a durable ledger — and everything in between is rows.
23
+
24
+ The thesis, inherited from the stack and load-bearing here: **an automation is an
25
+ artifact.** It parses a schema, it is stored as a row, it diffs in review, it can be
26
+ shown to a non-technical operator, and it can be previewed against real data before it
27
+ is armed. A scheduled TypeScript function that queries a database and charges a card
28
+ would be the most consequential logic in an application and the only piece with none
29
+ of those properties. Tide exists so that piece never has to be written.
30
+
31
+ ---
32
+
33
+ ## Core principles
34
+
35
+ 1. **An automation is an artifact.** The unit of authorship is the **reflex** — a
36
+ trigger, an optional selection, and exactly one effect, as validated JSON. If a
37
+ proposed feature cannot be expressed as data, it goes in an effect handler (code,
38
+ at the seam built for code) or it doesn't go in.
39
+
40
+ 2. **There is no run body.** Tide has no step language, no workflow interpreter, and
41
+ no in-memory execution state that matters. A reflex is a straight arc from stimulus
42
+ to one response — no deliberation between. A multi-step flow is a *chain*: an
43
+ effect completes by producing a fact (directly, or — under a host like moss — by
44
+ writing a row that becomes one), and other reflexes fire on that fact. The database
45
+ is the interpreter. Every joint between steps is a committed row, which is why a
46
+ crash between steps loses nothing and why every intermediate state is named,
47
+ durable, and visible. Checkpointing is not a feature; it is the shape.
48
+
49
+ 3. **Two sources of change, and only two: the clock and the fact.** A calendar trigger
50
+ is the clock. A data change is a fact, PUSHED by the host that saw its own write —
51
+ under moss, the vex bridge: every committed statement arrives as a fact with the
52
+ rows it returned, stamped with the identity of the tenant whose write it was. A
53
+ webhook is somebody else's write, arriving as a fact over HTTP; an external source
54
+ with no write choke point enters through an importer that ingests write facts at
55
+ the door. A person pressing "run now" fires a reflex by hand. There is no poll —
56
+ see ⟲ under **The machine**.
57
+
58
+ 4. **Deterministic core, injected everything.** Tide contains no SQL dialect, no
59
+ expression evaluator, no HTTP client, no identity model, and no timer. Storage,
60
+ selection, transformation, effects, and identity are five seams the host fills.
61
+ With stubs, the entire engine runs in memory under a fake clock — which is not a
62
+ testing convenience but the design: a headless check advances time and asserts on
63
+ rows, with no sleeping and no mocks of tide itself.
64
+
65
+ 5. **Identity-blind.** Tide never learns what a principal is. Every reflex names an
66
+ `as`; the host resolves it to whatever authority means there (under moss: a
67
+ charter-resolved, engine-enforced `ActorContext`) and tide threads it through to
68
+ effects untouched. It is also *written down* on every run, so a host can scope its
69
+ own ledger by it without tide knowing what it scoped. The same blindness charter
70
+ has toward universes, tide has toward identity — and for the same reason:
71
+ enforcement belongs to the governed target, never to the middleman.
72
+
73
+ 6. **The wall clock is never read.** The only time tide knows is the `now` handed to
74
+ `advance({ now })` and the `at` stamped on ingested facts. The DRIVER owns waking:
75
+ it advances to quiescence on every ingest (a chain advances one committed hop per
76
+ call, so the driver loops until nothing moves), and between ingests it sleeps until
77
+ the instant `nextDue(now)` names — a clock occurrence, a retry backoff, a lease
78
+ lapse, a delayed fact. Tide tells the driver when it wants waking; the driver owns
79
+ the timer. A slow janitor beat recovers what a crashed process left behind — it is
80
+ how work is FOUND, never how work moves. A dev check is its own driver, handing
81
+ `advance` a fake `now` and asserting on rows.
82
+
83
+ 7. **One effect per reflex, and effects are opaque named calls.** The effect
84
+ vocabulary is deliberately open — "send an email" cannot be a closed grammar,
85
+ because it reaches an outside service — but the *shape* of the seam is closed: a
86
+ registered handler with a name, an input the reflex shapes as data, an optional
87
+ preview, and `writes` metadata for the flow graph — under moss DERIVED from the
88
+ vex mutation behind the effect, never declared. The definition stays declarative;
89
+ the doing stays code; the boundary stays one function wide.
90
+
91
+ 8. **A table needs a guarantee, not a name.** Seven design nouns each got a table
92
+ once; six of them were vocabulary. The test a new table has to pass is not "is this
93
+ a real concept" — fact, delivery, firing, task, attempt, window and watermark were
94
+ all real concepts — but "what does the schema *promise* that nothing else can".
95
+ Four survive it. See **The ledger**.
96
+
97
+ ---
98
+
99
+ ## Vocabulary
100
+
101
+ The working set, one line each. Terms are defined in depth where they operate.
102
+
103
+ | Term | Meaning |
104
+ |---|---|
105
+ | **reflex** | the artifact: trigger + optional selection + one effect + policy — a tenant's reflex is the tenant's own row, with its own id |
106
+ | **trigger** | the reflex's `on` clause: `clock` \| `fact` \| `manual` |
107
+ | **occurrence** | one slot of a clock trigger, keyed by local calendar fields (`2026-03`) |
108
+ | **fact** | the intake contract — "something happened": a write, a signal, a manual poke, a settled run |
109
+ | **run** | one activation of one reflex: cause, version, identity, selection, fan-out, outcome |
110
+ | **task** | one unit of effect work inside a run; the idempotency grain |
111
+ | **attempt** | one execution try of a task, fenced by a token — a *count and a last error*, not a table |
112
+ | **lease** | how long a claim is good for; expiry is the reclaim |
113
+ | **unit** | what a task acts on: one selected row (`each`), the whole result (`batch`), or the trigger itself |
114
+ | **effect** | the named call a reflex makes; reference in data, handler in the registry |
115
+ | **template** | a transform-config slot inside a reflex (`effect.input`, `when`, selection query) |
116
+ | **`$`** | the closed environment templates evaluate against |
117
+ | **cause** | the provenance pointer on every run and emitted fact; composes into causality chains |
118
+ | **settle** | reach a final state (`done` or `failed`) — deliberately not "complete": a run with failures has settled |
119
+ | **drain** | announce a settled run as a fact, exactly once — the fan-in step |
120
+ | **chain** | reflexes linked through facts — informal, derived, deliberately **not** an artifact |
121
+ | **graph** | the static structure (watches × writes × run subscriptions), verified at load |
122
+ | **ledger** | the collective rows: facts, runs, tasks — host-readable data |
123
+ | **fire / retry / preview** | the human verbs: activate now, re-attempt a failed task, dry-run |
124
+ | **advance / nextDue / ingest / load** | the driver-facing verbs: one committed increment, the next instant worth waking for, accept a fact, accept artifacts |
125
+
126
+ **⟲ Naming.** This package called a run a **firing** for its first version, to avoid
127
+ colliding with moss's `RunRecord`. The collision it created instead was worse and
128
+ internal: `firing` was simultaneously a row type, a trigger kind (`fact: { firing }`)
129
+ and a fact kind, so `on: { fact: { firing: 'x' } }` read as a puzzle and every sentence
130
+ about one had to say which. One word, three jobs. It is **run** now, and the trigger is
131
+ `fact: { run: 'x' }`. The fact kind for external events stays **`signal`** —
132
+ conventional, and colliding with `@niscorp/signal` only in prose, never in an API.
133
+ There is deliberately no word for "workflow"; the closest thing, *chain*, is defined so
134
+ it can't become one.
135
+
136
+ ---
137
+
138
+ ## The grammar
139
+
140
+ Everything below is a Zod schema; everything a schema validates is plain JSON.
141
+
142
+ ### Reflex
143
+
144
+ ```jsonc
145
+ {
146
+ "id": "billing.charge-due",
147
+ "intent": "Charge every subscription due this month.", // one factual sentence — mandatory, like a vex entry
148
+ "on": { "clock": { "every": "month", "on": 1, "at": "03:00", "tz": "Europe/Vienna" } }, // THIS tenant's timezone — the row is theirs
149
+ "as": "automation.billing@studio_42", // opaque to tide; the host resolves it, and tide records it
150
+ "params": { "graceDays": 3 }, // authored knobs, visible to templates as $.params
151
+
152
+ "select": { // OPTIONAL — omitted = the trigger itself is the unit
153
+ "query": { /* opaque — handed to the `select` seam verbatim, after template evaluation */ },
154
+ "mode": "each", // 'each' → one task per row; 'batch' → one task, all rows
155
+ "unitKey": "subscription_id" // the row field that keys a unit task ('each' mode only)
156
+ },
157
+
158
+ "when": { /* transform predicate over $ — FACT triggers only (schema-enforced); prefer `select` where a query can filter */ },
159
+
160
+ "effect": {
161
+ "name": "payments.charge", // a registered effect
162
+ "input": { /* template — evaluated per unit */ }
163
+ },
164
+
165
+ "policy": {
166
+ "retry": { "max": 4, "backoff": "exponential", "baseMs": 60000 },
167
+ "timeoutMs": 30000,
168
+ "overlap": "skip", // 'skip' | 'allow' — may a run start while the last is unsettled?
169
+ "catchUp": "run", // 'run' | 'skip' | 'latest' — after downtime
170
+ "order": "any" // 'any' | 'serial' — serial = one in-flight task at a time
171
+ },
172
+
173
+ "enabled": true // the HOST's switch, read here; tide keeps no copy
174
+ }
175
+ ```
176
+
177
+ The template environment `$` is small and closed:
178
+
179
+ ```
180
+ $ = { params, // the reflex's own knobs
181
+ occurrence?, // clock runs: { key, at }
182
+ fact?, facts?, // fact runs: the fact, or the batch in `at` order
183
+ row?, rows?, // per-unit: the selected row (each) or all rows (batch)
184
+ now } // the advance's LOGICAL now — never a wall-clock read
185
+ ```
186
+
187
+ A manual run (via `fire()`) gets `$.fact = { kind: 'manual', payload: <input> }` —
188
+ templates see a fact like any other, and a template that requires `$.occurrence` on a
189
+ manually-fired clock reflex fails loudly at preview, which is where authoring errors
190
+ belong.
191
+
192
+ **Truthiness is one predicate.** `false`, `null`, `undefined`, `0`, `''` and `[]` are
193
+ falsy, in the matcher and in `preview` alike, because they import the same function.
194
+ **⟲** They did not, once: preview rejected only `false/null/undefined`, so a `when`
195
+ returning an empty list previewed as *this will fire* and then didn't. A dry run that
196
+ disagrees with the engine is worse than no dry run, because it is trusted.
197
+
198
+ ### Tenancy — a reflex is a tenant's row
199
+
200
+ Tide has **no tenant concept**, because the stack already solved multi-tenancy one
201
+ layer down and everything above inherits it. A reflex is tenant data, like a theme:
202
+ studio 42's billing reflex is studio 42's row — its own id, its tz, its params, its
203
+ `as`. Tide requires ids to be unique and runs the rows it is given; how a host mints
204
+ them is the host's business.
205
+
206
+ The boundary needs nothing from tide, because it is **vex's, engine-side, already
207
+ built**: a reflex's selections and writes execute under its own tenant principal's
208
+ compiled scope policy — the same wall that holds for humans. A reflex physically
209
+ cannot see or touch another tenant's rows, so tenants cannot influence each other
210
+ through tide any more than through a screen. Even the stray case fails safe: a reflex
211
+ woken by another tenant's write runs its selection under its *own* policy, sees zero
212
+ rows, and does nothing — the ordinary outcome, again.
213
+
214
+ **The ledger is scopeable for the same reason, and it needs one column to be.** A run
215
+ records the `as` its reflex declared. Tide does not know what it means; a host scopes
216
+ on it — under moss, an ordinary scope rule matching `tide_run.as_who` to a value
217
+ derived from the principal. **⟲** Before that column existed, the host's alternative
218
+ was to read the whole ledger and filter it in application code on a prefix convention
219
+ in the reflex id: the one tenant boundary in the reference app *not* enforced by the
220
+ engine, and one rename away from being silently wrong.
221
+
222
+ What tide deliberately does **not** parse: the `select.query` blob and the template
223
+ configs. The first belongs to the `select` seam (under moss: a vex
224
+ `{ fingerprint, context }` replay; under plain Node: perhaps `{ sql, params }`); the
225
+ second to the `transform` seam (under any nisc host: Prism). Tide stores them, diffs
226
+ them, hashes them into the reflex version — and hands them over verbatim. Nova does
227
+ exactly this with Prism endpoint configs; tide does not get to be smarter than nova
228
+ about other packages' languages. The honest cost: tide cannot validate a template at
229
+ load — a typo'd `$ref` surfaces at preview or execution, recorded on the task.
230
+
231
+ ### Trigger
232
+
233
+ Three kinds (and two more fact flavours). Structured on purpose — a reflex is shown to the operator it affects, and
234
+ `{ every: 'month', on: 1, at: '03:00' }` is reviewable by the person it bills where
235
+ `0 3 1 * *` is a shibboleth.
236
+
237
+ ```jsonc
238
+ { "clock": { "every": "day" | "week" | "month" | "year", "on"?: 1..31 | "mon".."sun" | "MM-DD",
239
+ "at": "HH:MM", "tz": "IANA name" } } // the owning tenant's tz — the row is theirs
240
+ { "clock": { "at": "2026-09-14T09:00", "tz": "IANA name" } } // one-shot
241
+ { "fact": { "entity": "charge_attempts", "op"?: "insert" } } // pushed changes, optionally one op
242
+ { "fact": { "signal": "stripe" } } // pushed external events
243
+ { "fact": { "run": "billing.charge-due" } } // a settled run — fan-in and dependencies
244
+ { "manual": {} } // fired only by hand
245
+ ```
246
+
247
+ Any reflex — whatever its trigger — may additionally be fired by hand through
248
+ `fire(reflexId, input?)`, **including a disarmed one**: arming gates *triggers*, not
249
+ people, and testing before arming is half of what `fire` is for; the ledger records
250
+ `manual:<who>` either way. A `manual` trigger just means *only* by hand. And `fire`
251
+ is sugar over `ingest`: it mints a `manual` fact aimed at one reflex — `at` supplied
252
+ by the caller, like every fact — so even the human verb enters through the one intake
253
+ and reads no clock.
254
+
255
+ ### Fact — the public intake contract
256
+
257
+ The fact is tide's whole ingestion surface. Anything that can produce this shape can
258
+ drive tide; nothing else about the producer is tide's business.
259
+
260
+ ```jsonc
261
+ {
262
+ "kind": "write" | "signal" | "manual" | "run",
263
+ "entity"?: "charge_attempts", // write facts: what changed (REQUIRED for a write)
264
+ "op"?: "insert" | "update" | "delete",
265
+ "row"?: { /* the row, as returned by the write */ },
266
+ "name"?: "stripe", // signal facts: which intake (REQUIRED for a signal)
267
+ "payload"?: { /* validated at the host's boundary before it gets here */ },
268
+ "reflex"?: "billing.charge-due", // run facts: whose run settled (minted by tide)
269
+ "runId"?: "…",
270
+ "occurrence"?: "2026-03", // run facts: the calendar key, when the clock was the cause
271
+ "stats"?: { "total": 500, "done": 488, "failed": 12 },
272
+ "target"?: "billing.charge-due", // manual facts: the one reflex it is aimed at
273
+ "at": 1755600000000, // supplied by the caller — tide reads no clocks
274
+ "notBefore"?: 1755859200000, // a delayed fact: timers as data
275
+ "dedupeKey"?: "evt_92xk…", // provider event ids; a duplicate is refused, and announced
276
+ "cause"?: "task:…" // set by tide when an effect emits — the causality chain
277
+ }
278
+ ```
279
+
280
+ **The union is enforced, and ⟲ it was not.** Each kind requires the fields that make it
281
+ addressable and refuses the fields belonging to the others. `{ kind: 'write' }` with no
282
+ entity used to parse, store, match nothing, get marked delivered, and vanish — a
283
+ producer with no way to learn its fact was never going to wake anything. An
284
+ unenforced union is worse than none, because it is documented.
285
+
286
+ `notBefore` earns its place twice: "retry the decline in three days" and "remind them
287
+ in an hour" are both delayed facts sitting in the ledger — visible, queryable rows
288
+ surviving restarts, not `setTimeout`s hiding in a process. And a delayed fact needs no
289
+ cancel API: the reflex it wakes re-checks reality through its selection, and **zero
290
+ rows is an ordinary outcome** — the guard is a query at fire time, not state to clean
291
+ up. Reality is the cancellation token.
292
+
293
+ Because the fact is durable *before* it is interpreted, a delayed fact meets the
294
+ reflexes loaded on the day it comes due, not the ones loaded when it arrived. That is
295
+ the point of the intake being a row: see **The ledger**.
296
+
297
+ **Ops are distinct stimuli, and intent is a row.** `op` rides every write fact —
298
+ "only on create" is `on: { fact: { entity: '…', op: 'insert' } }`, and things that
299
+ happen exactly once ride inserts naturally when the domain models intent as rows: a
300
+ receipt doesn't follow "invoices where status = paid" (a *state*, which every later
301
+ address edit still satisfies) — it follows a **payment**, inserted once, the moment
302
+ it happens. Where a domain does hang state on a wide row, the guard is the
303
+ **selection**: don't ask "is the row in state X" — ask "is there work left to do"
304
+ (*paid invoices without a receipt*). The chain records its own progress because
305
+ effects write outcomes, and zero rows is the ordinary outcome doing its job.
306
+
307
+ **Run facts are the fan-in mechanism, and they cost nothing.** Tide is the
308
+ bookkeeper of its own fan-out: the run knows it minted 500 tasks, tasks settle one
309
+ by one, and when the last settles, tide mints a `run` fact with the stats. "Send
310
+ one digest when the batch is done" is an ordinary reflex on that fact — no barrier
311
+ primitive, no "am I last?" logic in handlers. The same fact is the dependency
312
+ mechanism ("run B after A succeeds" = `on: { fact: { run: 'A' } }` with a `when` on
313
+ `$.fact.stats.failed`). A zero-task run settles immediately and mints its fact with
314
+ `total: 0`; a digest that shouldn't fire on empty runs says so in `when`.
315
+
316
+ ---
317
+
318
+ ## The machine
319
+
320
+ ```
321
+ advance({ now, limit }) — ONE committed increment; the driver's verb
322
+ │
323
+ ├─ 1 MATERIALIZE clock triggers → due runs, keyed by LOCAL calendar fields
324
+ │ (idempotent on the key; catch-up applied: run late / skip / latest;
325
+ │ skipped occurrences leave run rows saying so. A DISARMED reflex
326
+ │ materializes nothing AND keeps its watermark moving — see ⟲ below)
327
+ ├─ 2 MATCH undelivered, due facts × fact reflexes → `when` filters (a `when`
328
+ │ that THROWS is a recorded non-match, never a crashed advance and
329
+ │ never a silent one) → runs. Matching is per (fact, reflex); one
330
+ │ fact can wake five — but never across identities (see the leak
331
+ │ doctrine). A reflex matches only facts at or after its arming.
332
+ │ With ZERO reflexes loaded the pass does nothing at all, rather
333
+ │ than marking the backlog delivered against nobody.
334
+ ├─ 3 FAN OUT pending runs → unit tasks, committed IN ONE TRANSACTION with the
335
+ │ run's move to `fanned`. A write fact CARRIES its row: with no
336
+ │ selection declared, that row IS the unit (re-selecting would ask
337
+ │ a question the fact already answers); a declared selection is
338
+ │ ENRICHMENT, re-run under the reflex's own principal — the guard
339
+ │ that re-checks reality. Duplicate unitKey refuses the run
340
+ │ loudly; a transient seam failure DEFERS it.
341
+ ├─ 4 CLAIM due tasks and lapsed leases, in one statement — exactly-once,
342
+ │ `order: 'serial'` honoured inside the claim, `attempt`
343
+ │ incremented as part of it, one fencing token per claim
344
+ ├─ 5 EXECUTE transform(effect.input, $) → effects[name].run(input, ctx),
345
+ │ raced against timeoutMs
346
+ └─ 6 RECORD done / retry-scheduled / failed, token-checked; buffered emits
347
+ committed in the same transaction; the run's counters updated;
348
+ settled runs drained into their run fact, exactly once
349
+ ```
350
+
351
+ Every stage is a pure function over `(reflexes, ledger, now)` plus the injected seams.
352
+ A chain advances one hop per call — which is why the driver drains to quiescence on
353
+ every wake, bounded by the chain-depth ceiling rather than by anybody's patience. Two
354
+ instances advancing concurrently are safe by construction: run inserts are idempotent
355
+ on `(reflexId, cause)`, task claims are exactly-once through the store contract, and a
356
+ non-monotonic `now` (clock skew between hosts) merely delays work — it cannot
357
+ duplicate it.
358
+
359
+ **⟲ There was a poll trigger, and there was a beat.** Polls existed for hosts with no
360
+ write choke point: run a selection on an interval, diff a cursor, mint write facts for
361
+ the delta. In this stack the DAL is the choke point — every application write becomes
362
+ a fact at the vex bridge the instant it commits — so a poll could only re-discover
363
+ what was already pushed, one interval late; and the interval itself (`setInterval`
364
+ calling the old `tick`) was both a latency floor and a throughput ceiling: a chain
365
+ advanced one hop per minute, and a 100-task limit on a 60-second beat capped the
366
+ whole system at 144k tasks a day with morning-long backlogs. Both are gone. The
367
+ committed step survives as `advance` — it is what makes every check deterministic —
368
+ and the pacing moved to the host's driver: wake on ingest, drain to quiescence, sleep
369
+ until `nextDue`. The poll's hardest-won lesson (a non-unique cursor loses ties; a
370
+ minted delta must not cross tenants) lives on in the identity fence, which outlived
371
+ the trigger that taught it.
372
+
373
+ ### The ledger
374
+
375
+ **Four tables, and only two of them grow with events.** Each is here because it carries
376
+ a guarantee nothing else can:
377
+
378
+ | | the guarantee |
379
+ |---|---|
380
+ | **`fact`** | durable *before* anything interprets it — so `ingest` is one atomic write, matching is retryable in the advance, and a delayed fact meets the reflexes of the day it fires |
381
+ | **`run`** | `UNIQUE(reflexId, cause)` — *the* idempotency, one constraint serving all four trigger kinds; plus atomic fan-out, the completion counting fan-in reads, and `overlap: 'skip'` |
382
+ | **`task`** | `UNIQUE(runId, unit)`, written before the effect; the lease; the retry counter |
383
+ | **`state`** | where a reflex has got to: arming baseline, clock through-line. One row per reflex — bounded by how many exist, never by what happened |
384
+
385
+ - **`fact`** — everything that arrived: kind, payload, `dedupeKey`, depth, delivery,
386
+ any park. The intake log *is* the audit trail of change.
387
+ - **`run`** — one activation of one reflex: the cause (occurrence key, fact id, run id,
388
+ or `manual:<who>`), the version hash, the identity it ran `as`, what the selection
389
+ returned, the counts, the outcome.
390
+ - **`task`** — one unit of effect work: unit key, pinned `env`, state, attempt count,
391
+ last error, and the handler's returned **output** — stored, which is what makes
392
+ non-write effects (an external call, an agent's answer) selectable downstream
393
+ without inventing a domain table.
394
+
395
+ **⟲ There were seven, and three of them cost real correctness.** The removed ones and
396
+ why each was vocabulary rather than schema:
397
+
398
+ - **`delivery`** — a join table for "which reflexes did this fact wake", which is now
399
+ the `cause` column: `WHERE cause = 'fact:<id>'`. Its other job, recording
400
+ non-matches, grew one row per (fact × loaded reflex) — faster than the work itself —
401
+ and is an event now, which a host logs at whatever grain it wants.
402
+ - **`attempt`** — correctness needs the count and the last error, both of which belong
403
+ on the task. Per-attempt history is the host's log, which is where it was already
404
+ being read from; nothing in this repo ever called `listAttempts`.
405
+ - **`window`** — deleted with coalesce (below).
406
+ - **`watermark`** — an unbounded key/value table with a growing set of string keys,
407
+ for a couple of facts about each reflex. It is columns on `state` now. The clock
408
+ through-line *is* derivable from `MAX(cause)` over the runs, but only while those
409
+ runs exist, and retention is allowed to delete them.
410
+
411
+ Collapsing `fact` into `run` was considered and refused, and the reasoning is the
412
+ sharper half of the principle. Two tables would mean matching moves onto `ingest`:
413
+ evaluating every loaded reflex's `when` and opening N runs, synchronously, on the
414
+ host's request path — where a throw at reflex three of five leaves two runs and no
415
+ record that the fact ever arrived. A delayed fact would resolve against the reflexes of
416
+ its *arrival*, and a fact that matched nothing would leave no trace to dedupe or
417
+ explain. **Durability precedes interpretation** is the same argument as *idempotency
418
+ precedes the effect*, one layer earlier, and the intake table is what makes it true.
419
+
420
+ Causality is first-class: every run knows its cause and every emitted fact knows its
421
+ task, so *"why did this member get this email"* is a walk up a chain of rows —
422
+ `email ← draft approved ← draft written ← charge.failed ← evt_92xk` — not an
423
+ archaeology project.
424
+
425
+ **Retention is a policy, not an accident.** The ledger grows without bound unless told
426
+ otherwise; `sweep(now, { facts?, runs?, tasks? })` deletes settled rows past a horizon,
427
+ and the default is *keep forever*, which is the only honest default when the rows are
428
+ billing history. Deleting a run takes its tasks with it — **⟲** independent horizons
429
+ per table meant sweeping tasks could destroy the `UNIQUE(runId, unit)` row that *is*
430
+ the "this unit already ran" record, and a restore would then re-charge the invoice.
431
+
432
+ ### Execution semantics
433
+
434
+ The requirements doc calls this "the part that must not be hand-waved." In order:
435
+
436
+ - **Idempotency.** The task row is written *before* the effect runs, keyed
437
+ `UNIQUE(runId, unit)`. A duplicate trigger, a second instance, a crashed-and-restarted
438
+ advance — all collide with the existing row and are refused. There is no path to the
439
+ effect that does not pass through the insert. Handlers additionally receive the task
440
+ key in `ctx` for downstream idempotency — a payment capture passes it to the provider.
441
+ - **Fan-out is transactional.** A run's tasks commit together with its move to
442
+ `fanned`. This exists because the alternative is quietly catastrophic: a fan-out that
443
+ crashes at row 200 of 500 and resumes would *re-select against moved data* — a member
444
+ who paid in the gap keeps a task minted from the stale pass. Atomic fan-out means a
445
+ crash leaves nothing to resume from; the re-run selects fresh and mints clean. The
446
+ run's `total` is what was **written**, not what was offered — **⟲** counting the
447
+ input array meant any refusal by the unique key left `total` permanently
448
+ unreachable, so the run never settled, never drained, and blocked its reflex for good.
449
+ - **A bad minute is not a bad reflex.** A fan-out failure is two different things. A
450
+ `TideError` is tide's own refusal — a duplicate unit key, a missing `select` seam —
451
+ so the reflex is wrong and will be wrong again next time: the run is `skipped`, with
452
+ a note. Anything else came out of a host seam, and the run stays `pending` to be
453
+ retried, which is a row past its due time that a query can see. **⟲** Every throw
454
+ used to skip; because runs are idempotent on `(reflexId, cause)`, the occurrence
455
+ could never re-materialize, so one unreachable database destroyed that night's
456
+ billing run permanently — and since `skipped` is not `settled`, everything waiting on
457
+ it waited forever.
458
+ - **Retry is a type distinction, not metadata.** An effect that **returns** is done —
459
+ a card decline is not an error but a *domain outcome*, which the handler records and
460
+ the flow branches on via reflexes. An effect that **throws** is transient — tide
461
+ retries on the bounded backoff, then parks the task `failed`: terminal, visible, and
462
+ exit-able only through the human verb `retry()`.
463
+ - **`retry()` rewinds the run, and ⟲ it did not.** A failed task had already settled
464
+ its run, and a claim only reaches tasks that are claimable; reopening the task alone
465
+ left it `pending` forever, so the only documented exit from `failed` did nothing at
466
+ all. Reopening now moves the run back to `fanned` and decrements `failed`, in one
467
+ transaction. The old objection was real — a digest already went out saying twelve
468
+ failed, and re-settling must not send it again — and it is answered by `drained`, a
469
+ flag on the run recording that its settlement has been announced. The run rewinds;
470
+ the announcement does not repeat.
471
+ - **Attempts are fenced.** A timeout marks the attempt failed and schedules a retry —
472
+ but the timed-out effect may still be running. Each claim mints a token; recording a
473
+ result requires the token to still be current, so a zombie completion is discarded
474
+ instead of overwriting the live attempt. (Its external side effect is exactly what
475
+ the downstream idempotency key defends against.) One token per *claim*, not per task:
476
+ the fence's job is to tell this claim apart from an earlier one, and a task appears
477
+ in at most one claim.
478
+ - **A lease is the whole recovery story.** A claim writes `claimedUntil`; a task still
479
+ claimed past it is claimable again. **⟲** There was no lease, no expiry and no
480
+ reaper, so a process that died between the effect and the record left the task
481
+ `claimed` forever: the run never settled, never drained, fan-in stalled, and an
482
+ `overlap: 'skip'` reflex was blocked permanently. The alternative designs — a reaper
483
+ process, a heartbeat table, a liveness service — all add an organ; expiry adds a
484
+ column, and the fencing token is what makes it safe.
485
+ - **Emits are buffered, validated, and narrowed.** `ctx.emit` facts commit with the
486
+ successful attempt, in the same transaction; a throwing attempt discards its buffer,
487
+ or every retry of an emit-then-throw handler would double-mint facts and fire the
488
+ chain twice. Emit accepts `write` and `signal` only: **⟲** it accepted anything and
489
+ validated nothing, so a handler could emit `kind: 'manual'` and fire a **disarmed**
490
+ reflex — manual facts are checked before enablement on purpose, because arming gates
491
+ triggers rather than people, and a handler is not a person.
492
+ - **Concurrency has three dials, each where its limit lives.** Per-task claims are
493
+ exactly-once (in SQL: `FOR UPDATE SKIP LOCKED`). Per-reflex: `overlap: 'skip'`
494
+ refuses to start a run while the last is unsettled, and `order: 'serial'` allows one
495
+ in-flight task at a time — enforced *inside* the claim, because a caller that counts
496
+ what is busy and then claims loses the race to its own second instance. Provider rate
497
+ limits need no tide vocabulary at all: a throttled provider answers 429, the handler
498
+ throws, and the bounded backoff *is* the backpressure.
499
+ - **Overlap governs repeats, not distinct events.** *"May this start while the last is
500
+ unsettled"* is the right question about an occurrence — next month's billing run
501
+ arriving while this month's is still going is the same work coming round again — and
502
+ about a `manual` fact, which is a human pressing the button twice. It is the wrong
503
+ question about a payment. **⟲** It was asked of every cause, and `'skip'` is the
504
+ default, so every fact-triggered reflex silently dropped its second fact per pass:
505
+ three members joining in one minute produced one welcome email and two skipped runs.
506
+ The failure scaled with traffic, which is the wrong direction, and it was invisible
507
+ in the reference app because nothing there ever exercised the watched path with more
508
+ than one row. Discarding an external event that exists exactly once is data loss;
509
+ `order: 'serial'` is the dial that bounds concurrency by queueing instead.
510
+ - **Partial failure.** One task per unit, one transaction per task. If number 237 of
511
+ 500 throws, the first 236 stay done, 237 retries or parks, 238 onward run. There is
512
+ no all-or-nothing batch to get wrong.
513
+ - **Catch-up is authored, not guessed.** After downtime, `'run'` fires missed
514
+ occurrences late, `'skip'` drops them, `'latest'` fires only the most recent of
515
+ several missed. Each leaves a run row saying which happened — a skipped run is a
516
+ recorded decision. When a single advance would exceed the per-call cap, the watermark
517
+ stops at the last occurrence actually materialized: **⟲** it used to jump to `now`
518
+ regardless, so everything past the cap was unreachable forever with no row saying it
519
+ had existed.
520
+ - **Timeouts.** Every effect races `timeoutMs`; a hung external call marks the attempt
521
+ failed and frees the schedule.
522
+ - **Silence is visible.** Occurrences are materialized ahead, so a run that should have
523
+ happened and didn't is a `pending` row past its due time — a *query*, not a vanished
524
+ event. "Alert on silence" is a tide reflex watching tide's own ledger.
525
+
526
+ ### The tenant boundary inside the engine
527
+
528
+ Every database access a reflex makes already goes through the host: a
529
+ selection and an effect's write both run as that reflex's own principal, under
530
+ a compiled scope policy, and neither can see another tenant's rows. That is
531
+ vex's job and it does it.
532
+
533
+ **Tide is the one place a row travels without being read.** A bridge-minted
534
+ fact carries the row its write committed; `ctx.emit` carries a row a handler
535
+ built; a settled run carries its counts. Those are payloads, not queries — so
536
+ no policy is consulted when they move, and nothing downstream can catch it if
537
+ they move to the wrong place.
538
+
539
+ **⟲ They did, and this is the most serious defect the package has had.** The
540
+ matcher paired facts to reflexes by ENTITY alone. Two studios each running
541
+ "welcome somebody who joins" both watched `memberships`; one studio's fact
542
+ matched the other's reflex, which fanned out over a person it had never
543
+ selected. The competitor's automation then emailed that member — under its own
544
+ studio id, correctly stamped, with somebody else's name and address inside.
545
+ The scope engine was not wrong to allow it: the row written was legitimately
546
+ the sender's. Only the data in it had crossed.
547
+
548
+ So a fact carries **`as`** — the identity of the reflex that minted it — and a
549
+ reflex matches only facts bearing its own. Stamped by the engine and never by
550
+ a handler, for the same reason `depth` is: a handler that could choose its own
551
+ identity could choose somebody else's. The string stays opaque; equality is
552
+ the whole rule, and tide still never learns what a principal is.
553
+
554
+ The rule is **strict**, including at the intake. `ingest(fact, { as })` is how
555
+ a host says whose webhook it just received; a fact with no identity reaches
556
+ only reflexes with no identity. An earlier version exempted unlabelled facts
557
+ on the reasoning that the host knows what it ingested — which is a hole with a
558
+ rationalisation on top, and it failed open.
559
+
560
+ **Identity equality** stops a fact crossing between tenants at every door —
561
+ ingest (the bridge stamps `as` from the WRITE's own scope, mapped by the
562
+ app's naming rule), fire, emit, and run-settlement. And within one tenant, a
563
+ write fact is a STREAM: every reflex watching the entity hears the same fact,
564
+ which is what makes "welcome them" and "tell the desk" two reflexes instead
565
+ of one with two jobs.
566
+
567
+ This is defence in depth, not a replacement for scoping. Vex still enforces
568
+ every read and every write; this closes the gap vex structurally cannot see.
569
+
570
+ ### Occurrences and time
571
+
572
+ Occurrence identity is **(reflex, local calendar fields)**, never instants: the
573
+ March monthly run of a studio's billing reflex is `<its id>:2026-03`, the daily is
574
+ `…:2026-03-07` — and since a reflex id names one tenant's row, no two tenants' runs can
575
+ meet on a key. A DST boundary can move the *instant* a key fires at; it cannot mint a
576
+ second key or lose one — the double-fire and the skip are structurally impossible
577
+ rather than carefully avoided. The calendar edges are decided, not discovered:
578
+
579
+ - `every: 'month', on: 31` **clamps to the last day of the month** — February bills on
580
+ the 28th (29th), because "skip February" is never what a billing rule means.
581
+ - A local time erased by spring-forward resolves to the **first valid instant after**;
582
+ a local time that fall-back makes ambiguous takes the **first occurrence**. The key
583
+ is identical either way.
584
+ - Timezone resolution uses the platform's IANA data (`Intl`), keeping the dependency
585
+ count at zero. "03:00 in Vienna" and "03:00 in Denver" are both honest.
586
+
587
+ Occurrences being enumerable is also what makes **backfill** (deferred) natural:
588
+ "run the report reflex over the last 12 months" is materializing twelve past keys.
589
+
590
+ ### Arming, and what a pause means
591
+
592
+ Matching starts at a reflex's arming, and a reflex never retro-fires. Two faces of
593
+ one rule, each needing its own sentence because each has its own way of reaching for
594
+ the past: a **clock** reflex with no arming time establishes its baseline on the
595
+ first advance and materializes nothing; and a **fact** older than the arming belongs
596
+ to a world in which the reflex did not exist.
597
+
598
+ A **paused** reflex materializes nothing and keeps its watermark moving. A reflex
599
+ *entering* the loaded set re-baselines to the `at` passed to `load`. Both say the same
600
+ thing: coming back is not the same as never having left.
601
+
602
+ **⟲ This is where the worst defect in the package lived.** A disarmed reflex was
603
+ skipped outright, which froze its watermark at the moment of the pause. Eight days off
604
+ meant eight occurrences materialized the instant it came back — and eight real effects
605
+ executed. Verified by running the built engine, not by reading it.
606
+
607
+ **⟲ And there is no `arm`/`disarm`.** There was a pair, and they mutated an in-memory
608
+ map: a second copy of a fact the host already owned as a column, lost on restart, and
609
+ disagreeing with the host's own screen in between — so re-reading the rows silently
610
+ undid a pause somebody had just made. Enablement is a property of the reflex the host
611
+ hands over. To pause one: write your own row, `load` again. One source of truth, it
612
+ survives a restart, and it leaves something to audit where a method call left nothing.
613
+
614
+ ### Preview — dry run as a verb
615
+
616
+ `preview(reflexId, { now, fact? })` runs the real pipeline — materialize or match,
617
+ select against real data, evaluate every template — and stubs **one function**: the
618
+ effect executor. What comes back is the run that would happen: the occurrence or
619
+ fact, the selected rows (the eleven members, by name), each task's resolved input,
620
+ plus whatever the effect's optional `preview` hook renders. Because every effect passes
621
+ through the same choke point, there is no per-reflex dry-run flag and no `if (dryRun)`
622
+ to forget — a reflex *cannot* opt out of being previewable. Preview is also where
623
+ template typos surface, which makes it the authoring loop's inner verb.
624
+
625
+ ### The graph, verified at load
626
+
627
+ `load(reflexes)` validates every artifact, hashes versions, and derives the graph:
628
+ what reflexes WATCH (their triggers) × what their effects WRITE × their **run-fact
629
+ subscriptions** — that last edge class matters, because fan-in creates cycles the
630
+ write edges alone can't see (B fires on A's run; B's run feeds C; C writes what
631
+ triggers A).
632
+
633
+ Cycles are **classified, not banned** — because tide's own best patterns are cycles
634
+ on purpose: a drip campaign is a reflex whose delayed fact fires itself; "retry the
635
+ decline in 3 days" loops through `notBefore`; a drain loop subscribes to its own run.
636
+ All converge because every loop passes through a **guard**: a selection that re-checks
637
+ reality, a `when`, or a `notBefore` delay. Static analysis cannot tell a convergent
638
+ cycle from a divergent one, so it does not pretend to:
639
+
640
+ - An **unguarded** cycle — every hop unconditional, no selection, no `when`, no
641
+ `notBefore` anywhere on the loop — is refused at load: it diverges by construction.
642
+ - A **guarded** cycle is legal and reported as a finding, so the author sees the loop
643
+ they built.
644
+ - The runtime backstop is nearly free because causality already exists: a fact whose
645
+ cause chain exceeds `maxChainDepth` is **parked for review instead of fired**. Depth
646
+ is stamped on the task at fan-out rather than read back from the run at emit time —
647
+ **⟲** a run swept out from under a long-running chain answered `undefined`, resetting
648
+ the ceiling to zero in exactly the swept, long-running case the backstop exists for.
649
+ Releasing a parked fact **records** the override; **⟲** clearing the park alone was a
650
+ ping-pong, because the depth that parked it had not changed and never would.
651
+
652
+ Load **refuses** outright: an unguarded cycle, a reflex naming an unregistered
653
+ effect, a `when` on a non-fact trigger. The verification is only as good as the
654
+ effects' `writes`: under moss they are *derived* (from `mutationEffect` on the vex
655
+ mutation behind each effect), so an edge is never guessed; a plain host declares
656
+ them or forgoes the check, and an effect with none is reported as a **blind** edge,
657
+ never silently trusted — under moss, blind means something bypassed vex, which is
658
+ exactly when a loud word is wanted. If it loads, it's coherent — the moss tradition,
659
+ one layer down.
660
+
661
+ ---
662
+
663
+ ## Flows — chains, not bodies
664
+
665
+ The requirements' hardest sentence: *"charge the card, and if that succeeds mark the
666
+ invoice paid and send the receipt; if it fails, record the failure and schedule a
667
+ retry — without dropping into imperative code."* As a chain:
668
+
669
+ ```
670
+ clock ─→ [billing.charge-due] ── payments.charge ──→ charge_attempts row
671
+ │
672
+ fact: charge_attempts (succeeded) ─→ [billing.mark-paid] ─→ invoices row (paid)
673
+ fact: charge_attempts (declined) ─→ [billing.record-decline] ─→ invoice failed row
674
+ fact: invoices (paid) ─→ [billing.send-receipt] ─→ mail
675
+ fact: run billing.charge-due ─→ [billing.digest] ─→ owner's summary mail
676
+ clock (daily) + select: declines ≥ 3 days old, still open ─→ [billing.dunning-notice]
677
+ ```
678
+
679
+ Six reflexes, co-located in one `billing.reflexes.ts`. Every arrow is a committed row;
680
+ a crash anywhere resumes from the last row; every intermediate state has a name an
681
+ operator can query. Note the dunning line: it needs no cancel API when the member pays
682
+ — its selection asks "still open", and zero rows is an ordinary outcome.
683
+
684
+ **Data passes through three lanes, and the principle is one sentence:** *facts carry
685
+ the datum, the ledger carries the history, the domain rows carry the state — and
686
+ selection reaches all three.* The fact brings `$.fact.row`; task `output` makes any
687
+ effect's result selectable; and a selection joins whatever it needs (under moss, tide's
688
+ tables are vex entities — the digest's selection reads the tasks of `$.fact.runId`
689
+ joined to members). Nothing is "passed" in memory; everything is written where the next
690
+ reflex can read it.
691
+
692
+ **The granularity rule** — the one authored judgment this design asks for: *a joint
693
+ the business cares about (retry it independently, observe it, resume from it) is a
694
+ fact + reflex; a joint it doesn't care about lives inside one effect handler.* Two
695
+ external API calls that form one atomic business step are one handler — an endpoint is
696
+ a sanctioned home for imperative code, and the artifact thesis governs the automation,
697
+ not every HTTP call inside an effect.
698
+
699
+ **Mode guidance:** `batch` pins the whole result into one task and is for bounded sets
700
+ (a digest of dozens); a large set is `each` + a run-fact digest — 500 tasks and one
701
+ summary reflex, not one task carrying 500 rows.
702
+
703
+ **⟲ And there is no `coalesce`.** A policy field held matching facts in a window and
704
+ fired once with the batch. It cost two port methods, a table, an exactly-once promise
705
+ and a `DELETE … RETURNING` on every advance — and nothing ever set it; its only driver was
706
+ one test. It also had a quiet correctness bug: a throwing group key fell back to the
707
+ shared `''` window, so a per-customer digest could carry another tenant's rows. A
708
+ digest that genuinely needs batching is a **delayed run**: the first fact of a group
709
+ mints one at `now + window` under `cause: 'coalesce:<key>:<start>'`, the rest collide on
710
+ `UNIQUE(reflexId, cause)` and are refused, and when it comes due the reflex selects
711
+ what changed. Mechanisms that already exist, consistent with *the database is the
712
+ interpreter*, and no table.
713
+
714
+ ---
715
+
716
+ ## The seams
717
+
718
+ Five injection points. Every one is a move the stack has already made once — which is
719
+ the argument that tide is a *nisc* library and not a scheduler with JSON config:
720
+
721
+ | Seam | Contract | The host injects | Precedent |
722
+ |---|---|---|---|
723
+ | `store` | `TideStore` — six capabilities over four tables (below) | `createMemoryStore()` shipped; `createTideStore(pool)` from **moss** for persistence | vex's `CacheBackend` — structural, driver-free |
724
+ | `transform` | `(config, source) => value` | Prism's `evaluate`, or anything | **nova's exact socket** — nova doesn't know Prism; neither does tide |
725
+ | `select` | `(query, ctx) => AsyncIterable<row>` — the query blob is opaque | vex replay under the actor's policy; or raw SQL; or an array | vex's `generateDsl` / `mapToShape` |
726
+ | `effects` | `name → { run(input, ctx), writes?, preview? }` | vex mutation replays, fns, cortex agents; or plain functions | moss's `functions` seam |
727
+ | `ctx` | opaque, threaded through to `select` and `effects`; carries `taskKey`, `emit`, `now` | moss: `ActorContext` (principal, policy, wire) | charter's universe-blindness, applied to identity |
728
+
729
+ Plus the driver-facing edges, contracts rather than injections: **`ingest(fact)`**
730
+ in, **`advance({ now, limit })`** to run one committed increment, and
731
+ **`nextDue(now)`** so the driver knows when next to wake.
732
+
733
+ Dependencies: **`zod`**. Nothing else — the same bar charter clears.
734
+
735
+ ### The store contract
736
+
737
+ ```ts
738
+ type TideStore = {
739
+ transact: <T>(fn: (tx: TideStore) => Promise<T>) => Promise<T>;
740
+ appendIfAbsent: (table, row) => Promise<Row | undefined>;
741
+ claim: (spec: ClaimSpec) => Promise<readonly Row[]>;
742
+ cas: (table, id, expect, set) => Promise<boolean>;
743
+ query: (spec: QuerySpec) => Promise<readonly Row[]>;
744
+ remove: (spec: RemoveSpec) => Promise<number>;
745
+ };
746
+ ```
747
+
748
+ **⟲ It was twenty-seven methods**, shaped by nouns × verbs, and everything wrong with
749
+ the storage layer was downstream of that. `claimTasks`, `drainSettled` and
750
+ `claimClosedWindows` were three hand-written implementations of *one* operation, and
751
+ they diverged three ways; nine of the methods were reporting reads the engine never
752
+ called, so a reporting surface lived inside an execution contract. Shaped by what the
753
+ guarantees need instead, it is six — and the engine holds every opinion about retry,
754
+ arming, occurrences and coalescing, while the store holds none.
755
+
756
+ `UNIQUE_BY` is exported alongside it: the four unique keys **are** the exactly-once
757
+ promises, written down once so no store can drift from another's reading of a comment.
758
+
759
+ **Ledger reads left the port entirely.** Under moss they are vex entries over the same
760
+ tables — authored, scoped, cached, inspectable; standalone they are the host's own
761
+ queries. `tide.ledger.*` is a convenience over `query`, not a contract a store
762
+ implements.
763
+
764
+ ### Holding two stores to one definition
765
+
766
+ `STORE_CONTRACT`, exported from `@niscorp/tide/testing`, is the executable definition
767
+ of everything above: a list of `{ name, run(store) }` checks that throw, with no test
768
+ framework, so any runner can drive them. The memory store runs it; moss's SQL store
769
+ runs the same list against a real database.
770
+
771
+ **⟲ This exists because of the store that was deleted.** `store/postgres.ts` was 700
772
+ lines, had never been instantiated anywhere in this repo, and claimed in its own header
773
+ to be "held to the same tests" — of which there were none. It diverged from the memory
774
+ store in eleven ways: `LIMIT` applied before the serial filter, so one serial reflex
775
+ with 5,000 due tasks starved every other reflex; list reads answering opposite ends of
776
+ the same list; a dedupe index freed by the sweep, so a replayed webhook re-fired after
777
+ the retention horizon; three transactional methods that skipped the ready-guard and
778
+ raised `undefined_table` *inside* a transaction. Every one was invisible. The lesson is
779
+ not "write more tests" — it is that a contract nobody can execute is a comment.
780
+
781
+ ---
782
+
783
+ ## Hosts
784
+
785
+ ### Moss — the reference host
786
+
787
+ Moss fills every seam, and now the last one:
788
+
789
+ | Tide seam | Moss fills it with |
790
+ |---|---|
791
+ | `store` | **`createTideStore(pool)`** — four tables beside `vex_cache`, `ON CONFLICT DO NOTHING` for `appendIfAbsent`, `FOR UPDATE SKIP LOCKED` for `claim`, CHECK constraints on every state column, and a foreign key that cascades work with its run |
792
+ | `select` | vex replay **through the actor's own wire** — selection inherits replay-only, scoped, cached, inspectable; the tenant filter is in the SQL, engine-injected, unforgeable |
793
+ | `effects` | every seeded vex **mutation** auto-registered under its fingerprint (with `writes` *derived* from `mutationEffect`) + the app's `effects(actor)` |
794
+ | `transform` | prism — the same wiring the shell's transform socket already gets |
795
+ | `ctx` | `ActorContext`: the charter-resolved, engine-enforced, no-shell principal |
796
+ | the ledger | ordinary vex entries over `tide_run`, scoped by the host on the identity the run declared |
797
+
798
+ **The engine's writes do not go through vex's mutation pipeline, and the reason is
799
+ worth stating because the opposite looks tidier.** A vex mutation is an authored,
800
+ scope-compiled statement replayed on behalf of a principal — the right shape for an
801
+ application write, and the wrong shape for an engine claiming its own task: there is no
802
+ principal, no tenant to scope to, and no user input to police. Routing bookkeeping
803
+ through a policy compiler would add a scope rule permitting everything, which is a way
804
+ of saying the rule is not doing anything. The **reads** are the half that belongs to
805
+ vex, and they are authored entries like any other.
806
+
807
+ **Tide's tables are ordinary tables.** The first instinct was to hide them from
808
+ introspection, since they carry no tenancy of their own — but that is true of any table
809
+ with no scope rule, and it is the scope rule that has always been what makes
810
+ `memberships.read` safe. Hiding them would have closed the door as well as the hole: a
811
+ scoped entry over the run ledger cannot compile against a table vex has never heard of.
812
+
813
+ **Two live vex bugs surfaced on the way here**, both fixed rather than worked around:
814
+ no adapter implemented `transaction`, so *every batch mutation in every app* threw
815
+ "Batch mutations require a transactional client" — a message about a client nothing in
816
+ the repo could construct; and the PGlite shim's method needed its receiver, so
817
+ destructuring it produced a transaction that failed inside the library.
818
+
819
+ ### Plain Node — the floor
820
+
821
+ ```ts
822
+ const tide = createTide({
823
+ store: createMemoryStore(), // or your own, held to STORE_CONTRACT
824
+ transform: (config, source) => evaluate(config, source), // prism, or your own
825
+ select: async function* (q) { yield* (await pg.query(q.sql, q.params)).rows; },
826
+ effects: {
827
+ 'mail.send': { run: (input) => mailer.send(input) },
828
+ 'orders.close': { run: async (input, ctx) => { await pg.query(/* … */);
829
+ ctx.emit({ kind: 'write', entity: 'orders', row, at: ctx.now }); },
830
+ writes: ['orders'] },
831
+ },
832
+ });
833
+ tide.load(reflexes, { at: Date.now() });
834
+ app.post('/webhook/stripe', (req) => tide.ingest(toFact(req))); // validate at the boundary first
835
+
836
+ // The driver: wake on ingest, drain to quiescence, sleep until nextDue.
837
+ // (Under moss this is `createTideDriver`; standalone it is these lines.)
838
+ const drain = async () => { for (;;) { const r = await tide.advance({ now: Date.now() }); if (isQuiet(r)) break; } };
839
+ ```
840
+
841
+ No vex, no moss, no nova — and no magic either: facts must be ingested (a host with
842
+ its own write choke point pushes; one without builds an importer that pushes),
843
+ `writes` must be declared, the driver is theirs to run, and the outbox guarantee is
844
+ theirs to keep. `ctx.emit` is how a chain continues without a write choke point.
845
+
846
+ ---
847
+
848
+ ## File structure
849
+
850
+ ```
851
+ src/
852
+ index.ts Public API barrel — createTide, schemas, the store contract
853
+ testing.ts STORE_CONTRACT — the executable definition of a store
854
+ tide.ts createTide: load, ingest, advance, nextDue, fire, retry, preview, ledger, sweep
855
+ types.ts ledger rows, seam types, TideStore, UNIQUE_BY, reports
856
+ errors.ts TideError (code + details)
857
+
858
+ schemas/
859
+ reflex.schema.ts ReflexSchema — trigger, select, effect, policy, enabled
860
+ trigger.schema.ts clock | fact | manual, with narrowing helpers
861
+ fact.schema.ts FactInputSchema — the public intake contract, union enforced
862
+ policy.schema.ts retry, backoff, overlap, order, catchUp, lateMs
863
+ index.ts
864
+
865
+ engine/
866
+ advance.ts the pipeline, in order, over (reflexes, store, now)
867
+ due.ts nextDue — the instant the driver should wake for
868
+ runtime.ts EngineDeps, the `$` environment, isTruthy, versionOf
869
+ occurrence.ts calendar math, local-field keys, clamping, DST edges
870
+ materialize.ts clock → runs (catch-up, watermark)
871
+ match.ts fact × reflex, `when`, and openRun — the one place a run is born
872
+ fanout.ts selection → transactional unit-task commit; refuse vs defer
873
+ execute.ts claim → fence → transform → effect → record; the ONE door
874
+ preview.ts the same pipeline with the executor stubbed
875
+ graph.ts watches × writes × run subscriptions; refusal at load
876
+
877
+ store/
878
+ memory.ts createMemoryStore — the reference implementation + snapshot()
879
+
880
+ test/
881
+ occurrence.test.ts DST both directions, clamping, key stability
882
+ engine.test.ts the execution semantics, end to end on a fake clock
883
+ durability.test.ts one test per defect that reached the built engine
884
+ load.test.ts the load gate: validation, cycles, versioning
885
+ preview.test.ts dry run writes nothing and shows everything
886
+ store.test.ts STORE_CONTRACT × every store
887
+ ```
888
+
889
+ `materialize.ts` is split out of `advance.ts` because the clock is where the engine
890
+ *creates* work from nothing, and it reads better alone than buried in the
891
+ orchestrator. There is no `verbs.ts`: `fire` is four lines of sugar over `ingest`,
892
+ and `retry` belongs beside the executor whose invariants it rewinds.
893
+
894
+ There is no `store/postgres.ts`. **The moss integration lives in moss, not here — tide
895
+ must never import a host**, and persistence is where that rule earns its keep.
896
+
897
+ ---
898
+
899
+ ## Key design decisions
900
+
901
+ 1. **No run body — the database is the interpreter.** Three earlier shapes died to
902
+ get here: a bespoke step language (new grammar the stack didn't need), nova's
903
+ trigger steps executed headlessly (the right grammar, but it hauled in shells and
904
+ render machinery), and a moss subsystem (which handed moss a vocabulary, the one
905
+ thing moss refuses to own). What survived: an in-memory step chain must journal each
906
+ step to survive a crash — rebuilding a workflow engine's checkpointing — while a
907
+ chain of facts and reflexes gets durability, resumability, per-step retry policy and
908
+ per-step observability *from its shape*. The cost is real and named: flows span
909
+ artifacts. The answer is co-location in authoring and the derived graph at load.
910
+
911
+ 2. **A table earns its place with a guarantee, not a noun.** Seven became four. The
912
+ surviving test is short: *what does this schema promise that nothing else can?* An
913
+ intake that is durable before interpretation, one uniqueness constraint that is the
914
+ idempotency for every trigger kind, one that is the idempotency for a unit of work,
915
+ and a bounded row saying where each reflex has got to. Everything else was a
916
+ vocabulary word that had been handed a table and, with it, accessors, retention,
917
+ divergence and bugs.
918
+
919
+ 3. **Fan-in is bookkeeping, not a primitive.** The run fact exists because tide
920
+ already knows when the last task settles — emitting that knowledge as an ordinary
921
+ fact turns fan-in, dependencies and batch digests into ordinary reflexes. Exactly
922
+ once through `drained`, a flag on the row rather than an in-memory queue: a queue
923
+ cannot survive a restart and cannot be rewound, which is what made fan-in and the
924
+ recovery verb mutually exclusive.
925
+
926
+ 4. **Fan-out is transactional; unit keys collide loudly.** A resumable partial
927
+ fan-out re-selects against moved data — for a billing run, that is charging someone
928
+ who already paid. Atomicity makes the crash case the clean case. And a duplicate
929
+ `unitKey` inside one run refuses the run rather than silently dropping a row —
930
+ authoring errors are loud or they are invisible.
931
+
932
+ 5. **Attempts are fenced; emits are buffered; claims are leased.** All three exist for
933
+ one reason: a retry must not let a previous try's ghost act, and a dead process must
934
+ not hold work forever. The token check, the buffer-until-success rule, and lease
935
+ expiry close the three leaks a naive executor ships with — and none of them needs a
936
+ background process.
937
+
938
+ 6. **Retry classification is a calling convention.** Return = domain outcome, recorded
939
+ as data, branched on by reflexes. Throw = transient, retried on bounded backoff to
940
+ a terminal, human-visible state. Per-effect metadata declaring which errors are
941
+ retryable is a registry someone forgets to update; a type distinction at the seam
942
+ cannot be forgotten, and it puts the decision where the knowledge lives.
943
+
944
+ 7. **The effect executor is the only door.** Everything that leaves tide — a write, an
945
+ email, an agent — passes through one function. That single choke point is what
946
+ makes preview a *verb*, timeout uniform, the task key uniformly available, and the
947
+ ledger complete. A second path outward would quietly break all four.
948
+
949
+ 8. **Limits live where they bind.** Task uniqueness at the store; run overlap and
950
+ task order on the reflex; provider ceilings in the handler that owns the provider
951
+ relationship. Tide ships no rate vocabulary of its own.
952
+
953
+ 9. **Occurrence identity is local calendar fields, with decided edges.** `2026-03`,
954
+ not an epoch millisecond — DST cannot mint or lose a key. Day-31 clamps to
955
+ month-end; erased local times take the next valid instant; ambiguous ones take the
956
+ first. Decisions in the schema's documentation, not discoveries in production.
957
+
958
+ 10. **A reflex never retro-fires, and a pause is a pause.** Matching starts at arming;
959
+ history before a reflex existed is reachable only through deliberate backfill. The
960
+ two faces — the clock's first advance, a fact older than the
961
+ arming — are one principle: *a new watcher starts watching now.* The fourth face,
962
+ learned the hard way, is that a paused watcher's clock keeps moving.
963
+
964
+ 11. **Enablement is the host's column, not the engine's method.** Two copies of one
965
+ fact is the bug; which copy wins after a restart is only the symptom. `load` again
966
+ is the whole mechanism, and it is auditable where a method call was not.
967
+
968
+ 12. **Selection and shaping are opaque on purpose.** Tide stores, diffs and hashes the
969
+ query and template blobs but never interprets them. The moment tide validates a
970
+ query it owns a query language; the moment it evaluates an expression it owns an
971
+ evaluator — both exist in the stack already, better, behind seams nova proved out.
972
+ The honest cost — template typos surface at preview, not load — is why preview is
973
+ the authoring loop's inner verb, and why preview must use the engine's own
974
+ truthiness rather than a second reading of it.
975
+
976
+ 13. **Identity is the host's, entirely — and written down.** Tide threads an opaque
977
+ `ctx`; moss's `ActorContext` supplies principal, compiled policy and tenancy. An
978
+ automation that "runs as root" is unbuildable through this seam, because tide has
979
+ no root to offer. Recording `as` on the run is what lets a host scope the ledger
980
+ without tide learning what it scoped.
981
+
982
+ 14. **Versioning is a content hash, and the ledger pins it.** A reflex's version is the
983
+ hash of its definition, so the ledger can always say "March ran the old version,
984
+ April the new one." `enabled` is excluded — flipping it isn't an edit. `params` are
985
+ included — changing `graceDays` from 3 to 7 is a behavioural change the ledger must
986
+ explain.
987
+
988
+ 15. **A contract nobody can execute is a comment.** `STORE_CONTRACT` ships from the
989
+ package because the alternative was tried: a store whose header said it was held to
990
+ the same tests, and eleven silent divergences.
991
+
992
+ ---
993
+
994
+ ## Against the requirements
995
+
996
+ The wishlist, answered — including its own open questions:
997
+
998
+ | Requirement | Mechanism |
999
+ |---|---|
1000
+ | Clock / change / external / manual triggers | clock, write facts (pushed at the choke point), signal facts, `fire()` — one ledger under all of them |
1001
+ | Timezone honesty, no DST double-fire | tz is a field on the tenant's own reflex row + local-calendar occurrence keys + decided clamping (§ Tenancy, § decision 9) |
1002
+ | Selection = an ordinary vex read | the `select` seam; under moss, literally a fingerprint replay under the actor's policy |
1003
+ | Per-row and per-batch | `select.mode`; large sets = `each` + a run-fact digest |
1004
+ | Streaming large sets | `select` returns an `AsyncIterable`; fan-out commits in one transaction |
1005
+ | Effects: write / call out / agent / sequenced with explicit failure | the effects registry; sequences are chains; failure branches are reflexes on outcome rows |
1006
+ | Idempotency before the effect | task insert refused by `UNIQUE(runId, unit)`; task key in `ctx` for downstream |
1007
+ | Retry vs terminal | throw vs return (§ decision 6); `retry()` for the parked, and it rewinds the run |
1008
+ | Concurrency, overlap, ordering, partial failure, timeout, catch-up | leased claims + fencing, `overlap`, `order: 'serial'` inside the claim, task-per-unit, `timeoutMs`, `catchUp` |
1009
+ | Scoped no-session identity, tenant-bounded | `ctx` seam ← moss's `ActorContext`; engine-side enforcement, unchanged |
1010
+ | Run ledger, dry run, preview with names, silence alerting, ledger-as-data | facts/runs/tasks; `preview()`; materialized-ahead pending rows; host-readable tables with a retention policy |
1011
+ | Per-tenant config, instant disarm, versioning across runs | reflexes are tenant rows; disarming is the host's row write + `load` (§ decision 11); content-hash versions (§ decision 14) |
1012
+ | Host-agnostic wake-up; serverless and local both | `advance({ now })` + `nextDue(now)`: the driver wakes on ingest and sleeps until the next due instant; its janitor beat is recovery, not pacing |
1013
+ | Deterministic headless checks | memory store + fake clock; no timers exist to wait on |
1014
+ | Durable across restarts | `createTideStore(pool)` in moss; the memory store remains the reference and the check harness |
1015
+ | Wishlist: dependency between automations | run facts — `on: { fact: { run: 'A' } }` with `when` on the stats |
1016
+ | **Open: one library or two?** | one, standalone — "react to a change" is a fact reflex; same machine |
1017
+ | **Open: does the identity belong here?** | no — moss's `ActorContext`, consumed through an opaque seam |
1018
+ | **Open: how much effect vocabulary is closed?** | the seam is closed (one shape), the vocabulary is open (registered handlers) |
1019
+ | **Open: what does versioning mean?** | content hash pinned by the ledger; `enabled` is the host's field, so pausing isn't a version |
1020
+ | **Open: do notifications belong here?** | no — an automation writes rows; whatever notifies is another consumer of rows |
1021
+
1022
+ ---
1023
+
1024
+ ## Non-goals
1025
+
1026
+ - **A workflow engine's machinery, refused — the capability is the point.** A click
1027
+ cascading into writes, calls, and follow-ups *is* a workflow by any honest name,
1028
+ and it is exactly what tide is for. What tide refuses is the machinery tradition:
1029
+ a central orchestrator holding in-memory state, parked step interpreters,
1030
+ approval-step primitives, barriers. Here the workflow exists as rows — which is
1031
+ why it survives a crash, shows up in a ledger, and needs no engine to resume.
1032
+ Human-in-the-loop is the same shape: an automation writes a draft, a person's
1033
+ ordinary action approves it (a write), a reflex carries on from that row.
1034
+ - **Not a message bus, not an event-sourcing substrate.** The fact ledger is an intake
1035
+ log, not a topology; nothing subscribes to tide but tide.
1036
+ - **Not real-time.** Minute-level cadence is the floor and the promise. Stream-shaped
1037
+ work belongs in a stream processor; tide is for work a diligent clerk could have
1038
+ done on a schedule.
1039
+ - **Not a replacement for provider retries.** Where Stripe already does it well, the
1040
+ effect handler defers to Stripe and records the outcome.
1041
+ - **Not a lock service.** `order: 'serial'` is as exact as the store's claim can make
1042
+ it in one statement, and `overlap: 'skip'` — which is a unique constraint — is the
1043
+ guarantee to reach for when it must be exact.
1044
+
1045
+ ## Deferred
1046
+
1047
+ - **Backfill** — materializing past occurrence keys deliberately. Natural because keys
1048
+ are enumerable; deferred because catch-up covers outages.
1049
+ - **A true multi-way barrier** ("when A *and* B *and* C") — run facts cover sequential
1050
+ dependency and single-run fan-in; a real conjunction barrier waits for a real case,
1051
+ because it is the first organ of a workflow engine.
1052
+ - **Coalesce, if a digest case becomes real** — as a delayed run on a
1053
+ `coalesce:<key>:<window>` cause, never as a table.
1054
+ - **Cron-string sugar** in tooling — the stored artifact stays structured.
1055
+ - **Ephemeral fact observers** (live UI invalidation under moss) — a second,
1056
+ best-effort consumer of the host's fact stream; deliberately not tide's.
1057
+ - **Engine-level throttling** — backpressure already exists, and proactive throttling
1058
+ lives in the handler that owns the provider relationship.
1059
+ - **Distributed drive leadership** — unnecessary while idempotent materialization and
1060
+ fenced, leased, exactly-once claims make concurrent advances safe; revisit only if
1061
+ advance *cost* (not correctness) becomes a problem at scale.