@niscorp/tide 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DESIGN.md +1061 -0
- package/DOCS.md +506 -0
- package/LICENSE +202 -0
- package/README.md +145 -2
- package/dist/agent.cjs +207 -0
- package/dist/agent.cjs.map +1 -0
- package/dist/agent.d.cts +43 -0
- package/dist/agent.d.ts +43 -0
- package/dist/agent.js +68 -0
- package/dist/agent.js.map +1 -0
- package/dist/chunk-PJNVL4OD.js +196 -0
- package/dist/chunk-PJNVL4OD.js.map +1 -0
- package/dist/index.cjs +1566 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +298 -0
- package/dist/index.d.ts +298 -0
- package/dist/index.js +1337 -0
- package/dist/index.js.map +1 -0
- package/dist/reflex.schema-DeiyN5eN.d.cts +345 -0
- package/dist/reflex.schema-DeiyN5eN.d.ts +345 -0
- package/dist/testing.cjs +305 -0
- package/dist/testing.cjs.map +1 -0
- package/dist/testing.d.cts +10 -0
- package/dist/testing.d.ts +10 -0
- package/dist/testing.js +303 -0
- package/dist/testing.js.map +1 -0
- package/dist/types-lAPfp7JD.d.cts +314 -0
- package/dist/types-lAPfp7JD.d.ts +314 -0
- package/package.json +76 -3
package/DESIGN.md
ADDED
|
@@ -0,0 +1,1061 @@
|
|
|
1
|
+
# Tide — Design Document
|
|
2
|
+
|
|
3
|
+
> **Status: built, and once reviewed hard.** The engine, the store contract and the
|
|
4
|
+
> grammar are implemented and tested; see [DOCS.md](./DOCS.md) for the API. This is the
|
|
5
|
+
> design answer to [`automation-requirements.md`](../../docs/archive/automation-requirements.md),
|
|
6
|
+
> which stays authoritative on *what must be true*; this says *how*. Where the two
|
|
7
|
+
> disagree, one of them has rotted — report it, don't guess.
|
|
8
|
+
>
|
|
9
|
+
> A section marked **⟲** records something this package got wrong and what replaced it.
|
|
10
|
+
> They are kept because the wrong version was plausible, and a reader who does not know
|
|
11
|
+
> why it was wrong will propose it again.
|
|
12
|
+
|
|
13
|
+
## Purpose
|
|
14
|
+
|
|
15
|
+
Every nisc package answers one question: vex answers *what*, nova answers *how it
|
|
16
|
+
looks*, charter answers *who may*, prism answers *into what shape*, cortex *thinks*.
|
|
17
|
+
Nobody answers **when**. Tide is the answer: a host-blind automation engine in which
|
|
18
|
+
work initiated by the clock, by a change, by an external event, or by a person is
|
|
19
|
+
declared as data, executed as one named effect, and accounted for in a durable ledger.
|
|
20
|
+
|
|
21
|
+
**One sentence:** a reflex turns the clock and the fact into one named effect, through
|
|
22
|
+
a durable ledger — and everything in between is rows.
|
|
23
|
+
|
|
24
|
+
The thesis, inherited from the stack and load-bearing here: **an automation is an
|
|
25
|
+
artifact.** It parses a schema, it is stored as a row, it diffs in review, it can be
|
|
26
|
+
shown to a non-technical operator, and it can be previewed against real data before it
|
|
27
|
+
is armed. A scheduled TypeScript function that queries a database and charges a card
|
|
28
|
+
would be the most consequential logic in an application and the only piece with none
|
|
29
|
+
of those properties. Tide exists so that piece never has to be written.
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## Core principles
|
|
34
|
+
|
|
35
|
+
1. **An automation is an artifact.** The unit of authorship is the **reflex** — a
|
|
36
|
+
trigger, an optional selection, and exactly one effect, as validated JSON. If a
|
|
37
|
+
proposed feature cannot be expressed as data, it goes in an effect handler (code,
|
|
38
|
+
at the seam built for code) or it doesn't go in.
|
|
39
|
+
|
|
40
|
+
2. **There is no run body.** Tide has no step language, no workflow interpreter, and
|
|
41
|
+
no in-memory execution state that matters. A reflex is a straight arc from stimulus
|
|
42
|
+
to one response — no deliberation between. A multi-step flow is a *chain*: an
|
|
43
|
+
effect completes by producing a fact (directly, or — under a host like moss — by
|
|
44
|
+
writing a row that becomes one), and other reflexes fire on that fact. The database
|
|
45
|
+
is the interpreter. Every joint between steps is a committed row, which is why a
|
|
46
|
+
crash between steps loses nothing and why every intermediate state is named,
|
|
47
|
+
durable, and visible. Checkpointing is not a feature; it is the shape.
|
|
48
|
+
|
|
49
|
+
3. **Two sources of change, and only two: the clock and the fact.** A calendar trigger
|
|
50
|
+
is the clock. A data change is a fact, PUSHED by the host that saw its own write —
|
|
51
|
+
under moss, the vex bridge: every committed statement arrives as a fact with the
|
|
52
|
+
rows it returned, stamped with the identity of the tenant whose write it was. A
|
|
53
|
+
webhook is somebody else's write, arriving as a fact over HTTP; an external source
|
|
54
|
+
with no write choke point enters through an importer that ingests write facts at
|
|
55
|
+
the door. A person pressing "run now" fires a reflex by hand. There is no poll —
|
|
56
|
+
see ⟲ under **The machine**.
|
|
57
|
+
|
|
58
|
+
4. **Deterministic core, injected everything.** Tide contains no SQL dialect, no
|
|
59
|
+
expression evaluator, no HTTP client, no identity model, and no timer. Storage,
|
|
60
|
+
selection, transformation, effects, and identity are five seams the host fills.
|
|
61
|
+
With stubs, the entire engine runs in memory under a fake clock — which is not a
|
|
62
|
+
testing convenience but the design: a headless check advances time and asserts on
|
|
63
|
+
rows, with no sleeping and no mocks of tide itself.
|
|
64
|
+
|
|
65
|
+
5. **Identity-blind.** Tide never learns what a principal is. Every reflex names an
|
|
66
|
+
`as`; the host resolves it to whatever authority means there (under moss: a
|
|
67
|
+
charter-resolved, engine-enforced `ActorContext`) and tide threads it through to
|
|
68
|
+
effects untouched. It is also *written down* on every run, so a host can scope its
|
|
69
|
+
own ledger by it without tide knowing what it scoped. The same blindness charter
|
|
70
|
+
has toward universes, tide has toward identity — and for the same reason:
|
|
71
|
+
enforcement belongs to the governed target, never to the middleman.
|
|
72
|
+
|
|
73
|
+
6. **The wall clock is never read.** The only time tide knows is the `now` handed to
|
|
74
|
+
`advance({ now })` and the `at` stamped on ingested facts. The DRIVER owns waking:
|
|
75
|
+
it advances to quiescence on every ingest (a chain advances one committed hop per
|
|
76
|
+
call, so the driver loops until nothing moves), and between ingests it sleeps until
|
|
77
|
+
the instant `nextDue(now)` names — a clock occurrence, a retry backoff, a lease
|
|
78
|
+
lapse, a delayed fact. Tide tells the driver when it wants waking; the driver owns
|
|
79
|
+
the timer. A slow janitor beat recovers what a crashed process left behind — it is
|
|
80
|
+
how work is FOUND, never how work moves. A dev check is its own driver, handing
|
|
81
|
+
`advance` a fake `now` and asserting on rows.
|
|
82
|
+
|
|
83
|
+
7. **One effect per reflex, and effects are opaque named calls.** The effect
|
|
84
|
+
vocabulary is deliberately open — "send an email" cannot be a closed grammar,
|
|
85
|
+
because it reaches an outside service — but the *shape* of the seam is closed: a
|
|
86
|
+
registered handler with a name, an input the reflex shapes as data, an optional
|
|
87
|
+
preview, and `writes` metadata for the flow graph — under moss DERIVED from the
|
|
88
|
+
vex mutation behind the effect, never declared. The definition stays declarative;
|
|
89
|
+
the doing stays code; the boundary stays one function wide.
|
|
90
|
+
|
|
91
|
+
8. **A table needs a guarantee, not a name.** Seven design nouns each got a table
|
|
92
|
+
once; six of them were vocabulary. The test a new table has to pass is not "is this
|
|
93
|
+
a real concept" — fact, delivery, firing, task, attempt, window and watermark were
|
|
94
|
+
all real concepts — but "what does the schema *promise* that nothing else can".
|
|
95
|
+
Four survive it. See **The ledger**.
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Vocabulary
|
|
100
|
+
|
|
101
|
+
The working set, one line each. Terms are defined in depth where they operate.
|
|
102
|
+
|
|
103
|
+
| Term | Meaning |
|
|
104
|
+
|---|---|
|
|
105
|
+
| **reflex** | the artifact: trigger + optional selection + one effect + policy — a tenant's reflex is the tenant's own row, with its own id |
|
|
106
|
+
| **trigger** | the reflex's `on` clause: `clock` \| `fact` \| `manual` |
|
|
107
|
+
| **occurrence** | one slot of a clock trigger, keyed by local calendar fields (`2026-03`) |
|
|
108
|
+
| **fact** | the intake contract — "something happened": a write, a signal, a manual poke, a settled run |
|
|
109
|
+
| **run** | one activation of one reflex: cause, version, identity, selection, fan-out, outcome |
|
|
110
|
+
| **task** | one unit of effect work inside a run; the idempotency grain |
|
|
111
|
+
| **attempt** | one execution try of a task, fenced by a token — a *count and a last error*, not a table |
|
|
112
|
+
| **lease** | how long a claim is good for; expiry is the reclaim |
|
|
113
|
+
| **unit** | what a task acts on: one selected row (`each`), the whole result (`batch`), or the trigger itself |
|
|
114
|
+
| **effect** | the named call a reflex makes; reference in data, handler in the registry |
|
|
115
|
+
| **template** | a transform-config slot inside a reflex (`effect.input`, `when`, selection query) |
|
|
116
|
+
| **`$`** | the closed environment templates evaluate against |
|
|
117
|
+
| **cause** | the provenance pointer on every run and emitted fact; composes into causality chains |
|
|
118
|
+
| **settle** | reach a final state (`done` or `failed`) — deliberately not "complete": a run with failures has settled |
|
|
119
|
+
| **drain** | announce a settled run as a fact, exactly once — the fan-in step |
|
|
120
|
+
| **chain** | reflexes linked through facts — informal, derived, deliberately **not** an artifact |
|
|
121
|
+
| **graph** | the static structure (watches × writes × run subscriptions), verified at load |
|
|
122
|
+
| **ledger** | the collective rows: facts, runs, tasks — host-readable data |
|
|
123
|
+
| **fire / retry / preview** | the human verbs: activate now, re-attempt a failed task, dry-run |
|
|
124
|
+
| **advance / nextDue / ingest / load** | the driver-facing verbs: one committed increment, the next instant worth waking for, accept a fact, accept artifacts |
|
|
125
|
+
|
|
126
|
+
**⟲ Naming.** This package called a run a **firing** for its first version, to avoid
|
|
127
|
+
colliding with moss's `RunRecord`. The collision it created instead was worse and
|
|
128
|
+
internal: `firing` was simultaneously a row type, a trigger kind (`fact: { firing }`)
|
|
129
|
+
and a fact kind, so `on: { fact: { firing: 'x' } }` read as a puzzle and every sentence
|
|
130
|
+
about one had to say which. One word, three jobs. It is **run** now, and the trigger is
|
|
131
|
+
`fact: { run: 'x' }`. The fact kind for external events stays **`signal`** —
|
|
132
|
+
conventional, and colliding with `@niscorp/signal` only in prose, never in an API.
|
|
133
|
+
There is deliberately no word for "workflow"; the closest thing, *chain*, is defined so
|
|
134
|
+
it can't become one.
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
138
|
+
## The grammar
|
|
139
|
+
|
|
140
|
+
Everything below is a Zod schema; everything a schema validates is plain JSON.
|
|
141
|
+
|
|
142
|
+
### Reflex
|
|
143
|
+
|
|
144
|
+
```jsonc
|
|
145
|
+
{
|
|
146
|
+
"id": "billing.charge-due",
|
|
147
|
+
"intent": "Charge every subscription due this month.", // one factual sentence — mandatory, like a vex entry
|
|
148
|
+
"on": { "clock": { "every": "month", "on": 1, "at": "03:00", "tz": "Europe/Vienna" } }, // THIS tenant's timezone — the row is theirs
|
|
149
|
+
"as": "automation.billing@studio_42", // opaque to tide; the host resolves it, and tide records it
|
|
150
|
+
"params": { "graceDays": 3 }, // authored knobs, visible to templates as $.params
|
|
151
|
+
|
|
152
|
+
"select": { // OPTIONAL — omitted = the trigger itself is the unit
|
|
153
|
+
"query": { /* opaque — handed to the `select` seam verbatim, after template evaluation */ },
|
|
154
|
+
"mode": "each", // 'each' → one task per row; 'batch' → one task, all rows
|
|
155
|
+
"unitKey": "subscription_id" // the row field that keys a unit task ('each' mode only)
|
|
156
|
+
},
|
|
157
|
+
|
|
158
|
+
"when": { /* transform predicate over $ — FACT triggers only (schema-enforced); prefer `select` where a query can filter */ },
|
|
159
|
+
|
|
160
|
+
"effect": {
|
|
161
|
+
"name": "payments.charge", // a registered effect
|
|
162
|
+
"input": { /* template — evaluated per unit */ }
|
|
163
|
+
},
|
|
164
|
+
|
|
165
|
+
"policy": {
|
|
166
|
+
"retry": { "max": 4, "backoff": "exponential", "baseMs": 60000 },
|
|
167
|
+
"timeoutMs": 30000,
|
|
168
|
+
"overlap": "skip", // 'skip' | 'allow' — may a run start while the last is unsettled?
|
|
169
|
+
"catchUp": "run", // 'run' | 'skip' | 'latest' — after downtime
|
|
170
|
+
"order": "any" // 'any' | 'serial' — serial = one in-flight task at a time
|
|
171
|
+
},
|
|
172
|
+
|
|
173
|
+
"enabled": true // the HOST's switch, read here; tide keeps no copy
|
|
174
|
+
}
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
The template environment `$` is small and closed:
|
|
178
|
+
|
|
179
|
+
```
|
|
180
|
+
$ = { params, // the reflex's own knobs
|
|
181
|
+
occurrence?, // clock runs: { key, at }
|
|
182
|
+
fact?, facts?, // fact runs: the fact, or the batch in `at` order
|
|
183
|
+
row?, rows?, // per-unit: the selected row (each) or all rows (batch)
|
|
184
|
+
now } // the advance's LOGICAL now — never a wall-clock read
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
A manual run (via `fire()`) gets `$.fact = { kind: 'manual', payload: <input> }` —
|
|
188
|
+
templates see a fact like any other, and a template that requires `$.occurrence` on a
|
|
189
|
+
manually-fired clock reflex fails loudly at preview, which is where authoring errors
|
|
190
|
+
belong.
|
|
191
|
+
|
|
192
|
+
**Truthiness is one predicate.** `false`, `null`, `undefined`, `0`, `''` and `[]` are
|
|
193
|
+
falsy, in the matcher and in `preview` alike, because they import the same function.
|
|
194
|
+
**⟲** They did not, once: preview rejected only `false/null/undefined`, so a `when`
|
|
195
|
+
returning an empty list previewed as *this will fire* and then didn't. A dry run that
|
|
196
|
+
disagrees with the engine is worse than no dry run, because it is trusted.
|
|
197
|
+
|
|
198
|
+
### Tenancy — a reflex is a tenant's row
|
|
199
|
+
|
|
200
|
+
Tide has **no tenant concept**, because the stack already solved multi-tenancy one
|
|
201
|
+
layer down and everything above inherits it. A reflex is tenant data, like a theme:
|
|
202
|
+
studio 42's billing reflex is studio 42's row — its own id, its tz, its params, its
|
|
203
|
+
`as`. Tide requires ids to be unique and runs the rows it is given; how a host mints
|
|
204
|
+
them is the host's business.
|
|
205
|
+
|
|
206
|
+
The boundary needs nothing from tide, because it is **vex's, engine-side, already
|
|
207
|
+
built**: a reflex's selections and writes execute under its own tenant principal's
|
|
208
|
+
compiled scope policy — the same wall that holds for humans. A reflex physically
|
|
209
|
+
cannot see or touch another tenant's rows, so tenants cannot influence each other
|
|
210
|
+
through tide any more than through a screen. Even the stray case fails safe: a reflex
|
|
211
|
+
woken by another tenant's write runs its selection under its *own* policy, sees zero
|
|
212
|
+
rows, and does nothing — the ordinary outcome, again.
|
|
213
|
+
|
|
214
|
+
**The ledger is scopeable for the same reason, and it needs one column to be.** A run
|
|
215
|
+
records the `as` its reflex declared. Tide does not know what it means; a host scopes
|
|
216
|
+
on it — under moss, an ordinary scope rule matching `tide_run.as_who` to a value
|
|
217
|
+
derived from the principal. **⟲** Before that column existed, the host's alternative
|
|
218
|
+
was to read the whole ledger and filter it in application code on a prefix convention
|
|
219
|
+
in the reflex id: the one tenant boundary in the reference app *not* enforced by the
|
|
220
|
+
engine, and one rename away from being silently wrong.
|
|
221
|
+
|
|
222
|
+
What tide deliberately does **not** parse: the `select.query` blob and the template
|
|
223
|
+
configs. The first belongs to the `select` seam (under moss: a vex
|
|
224
|
+
`{ fingerprint, context }` replay; under plain Node: perhaps `{ sql, params }`); the
|
|
225
|
+
second to the `transform` seam (under any nisc host: Prism). Tide stores them, diffs
|
|
226
|
+
them, hashes them into the reflex version — and hands them over verbatim. Nova does
|
|
227
|
+
exactly this with Prism endpoint configs; tide does not get to be smarter than nova
|
|
228
|
+
about other packages' languages. The honest cost: tide cannot validate a template at
|
|
229
|
+
load — a typo'd `$ref` surfaces at preview or execution, recorded on the task.
|
|
230
|
+
|
|
231
|
+
### Trigger
|
|
232
|
+
|
|
233
|
+
Three kinds (and two more fact flavours). Structured on purpose — a reflex is shown to the operator it affects, and
|
|
234
|
+
`{ every: 'month', on: 1, at: '03:00' }` is reviewable by the person it bills where
|
|
235
|
+
`0 3 1 * *` is a shibboleth.
|
|
236
|
+
|
|
237
|
+
```jsonc
|
|
238
|
+
{ "clock": { "every": "day" | "week" | "month" | "year", "on"?: 1..31 | "mon".."sun" | "MM-DD",
|
|
239
|
+
"at": "HH:MM", "tz": "IANA name" } } // the owning tenant's tz — the row is theirs
|
|
240
|
+
{ "clock": { "at": "2026-09-14T09:00", "tz": "IANA name" } } // one-shot
|
|
241
|
+
{ "fact": { "entity": "charge_attempts", "op"?: "insert" } } // pushed changes, optionally one op
|
|
242
|
+
{ "fact": { "signal": "stripe" } } // pushed external events
|
|
243
|
+
{ "fact": { "run": "billing.charge-due" } } // a settled run — fan-in and dependencies
|
|
244
|
+
{ "manual": {} } // fired only by hand
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Any reflex — whatever its trigger — may additionally be fired by hand through
|
|
248
|
+
`fire(reflexId, input?)`, **including a disarmed one**: arming gates *triggers*, not
|
|
249
|
+
people, and testing before arming is half of what `fire` is for; the ledger records
|
|
250
|
+
`manual:<who>` either way. A `manual` trigger just means *only* by hand. And `fire`
|
|
251
|
+
is sugar over `ingest`: it mints a `manual` fact aimed at one reflex — `at` supplied
|
|
252
|
+
by the caller, like every fact — so even the human verb enters through the one intake
|
|
253
|
+
and reads no clock.
|
|
254
|
+
|
|
255
|
+
### Fact — the public intake contract
|
|
256
|
+
|
|
257
|
+
The fact is tide's whole ingestion surface. Anything that can produce this shape can
|
|
258
|
+
drive tide; nothing else about the producer is tide's business.
|
|
259
|
+
|
|
260
|
+
```jsonc
|
|
261
|
+
{
|
|
262
|
+
"kind": "write" | "signal" | "manual" | "run",
|
|
263
|
+
"entity"?: "charge_attempts", // write facts: what changed (REQUIRED for a write)
|
|
264
|
+
"op"?: "insert" | "update" | "delete",
|
|
265
|
+
"row"?: { /* the row, as returned by the write */ },
|
|
266
|
+
"name"?: "stripe", // signal facts: which intake (REQUIRED for a signal)
|
|
267
|
+
"payload"?: { /* validated at the host's boundary before it gets here */ },
|
|
268
|
+
"reflex"?: "billing.charge-due", // run facts: whose run settled (minted by tide)
|
|
269
|
+
"runId"?: "…",
|
|
270
|
+
"occurrence"?: "2026-03", // run facts: the calendar key, when the clock was the cause
|
|
271
|
+
"stats"?: { "total": 500, "done": 488, "failed": 12 },
|
|
272
|
+
"target"?: "billing.charge-due", // manual facts: the one reflex it is aimed at
|
|
273
|
+
"at": 1755600000000, // supplied by the caller — tide reads no clocks
|
|
274
|
+
"notBefore"?: 1755859200000, // a delayed fact: timers as data
|
|
275
|
+
"dedupeKey"?: "evt_92xk…", // provider event ids; a duplicate is refused, and announced
|
|
276
|
+
"cause"?: "task:…" // set by tide when an effect emits — the causality chain
|
|
277
|
+
}
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
**The union is enforced, and ⟲ it was not.** Each kind requires the fields that make it
|
|
281
|
+
addressable and refuses the fields belonging to the others. `{ kind: 'write' }` with no
|
|
282
|
+
entity used to parse, store, match nothing, get marked delivered, and vanish — a
|
|
283
|
+
producer with no way to learn its fact was never going to wake anything. An
|
|
284
|
+
unenforced union is worse than none, because it is documented.
|
|
285
|
+
|
|
286
|
+
`notBefore` earns its place twice: "retry the decline in three days" and "remind them
|
|
287
|
+
in an hour" are both delayed facts sitting in the ledger — visible, queryable rows
|
|
288
|
+
surviving restarts, not `setTimeout`s hiding in a process. And a delayed fact needs no
|
|
289
|
+
cancel API: the reflex it wakes re-checks reality through its selection, and **zero
|
|
290
|
+
rows is an ordinary outcome** — the guard is a query at fire time, not state to clean
|
|
291
|
+
up. Reality is the cancellation token.
|
|
292
|
+
|
|
293
|
+
Because the fact is durable *before* it is interpreted, a delayed fact meets the
|
|
294
|
+
reflexes loaded on the day it comes due, not the ones loaded when it arrived. That is
|
|
295
|
+
the point of the intake being a row: see **The ledger**.
|
|
296
|
+
|
|
297
|
+
**Ops are distinct stimuli, and intent is a row.** `op` rides every write fact —
|
|
298
|
+
"only on create" is `on: { fact: { entity: '…', op: 'insert' } }`, and things that
|
|
299
|
+
happen exactly once ride inserts naturally when the domain models intent as rows: a
|
|
300
|
+
receipt doesn't follow "invoices where status = paid" (a *state*, which every later
|
|
301
|
+
address edit still satisfies) — it follows a **payment**, inserted once, the moment
|
|
302
|
+
it happens. Where a domain does hang state on a wide row, the guard is the
|
|
303
|
+
**selection**: don't ask "is the row in state X" — ask "is there work left to do"
|
|
304
|
+
(*paid invoices without a receipt*). The chain records its own progress because
|
|
305
|
+
effects write outcomes, and zero rows is the ordinary outcome doing its job.
|
|
306
|
+
|
|
307
|
+
**Run facts are the fan-in mechanism, and they cost nothing.** Tide is the
|
|
308
|
+
bookkeeper of its own fan-out: the run knows it minted 500 tasks, tasks settle one
|
|
309
|
+
by one, and when the last settles, tide mints a `run` fact with the stats. "Send
|
|
310
|
+
one digest when the batch is done" is an ordinary reflex on that fact — no barrier
|
|
311
|
+
primitive, no "am I last?" logic in handlers. The same fact is the dependency
|
|
312
|
+
mechanism ("run B after A succeeds" = `on: { fact: { run: 'A' } }` with a `when` on
|
|
313
|
+
`$.fact.stats.failed`). A zero-task run settles immediately and mints its fact with
|
|
314
|
+
`total: 0`; a digest that shouldn't fire on empty runs says so in `when`.
|
|
315
|
+
|
|
316
|
+
---
|
|
317
|
+
|
|
318
|
+
## The machine
|
|
319
|
+
|
|
320
|
+
```
|
|
321
|
+
advance({ now, limit }) — ONE committed increment; the driver's verb
|
|
322
|
+
│
|
|
323
|
+
├─ 1 MATERIALIZE clock triggers → due runs, keyed by LOCAL calendar fields
|
|
324
|
+
│ (idempotent on the key; catch-up applied: run late / skip / latest;
|
|
325
|
+
│ skipped occurrences leave run rows saying so. A DISARMED reflex
|
|
326
|
+
│ materializes nothing AND keeps its watermark moving — see ⟲ below)
|
|
327
|
+
├─ 2 MATCH undelivered, due facts × fact reflexes → `when` filters (a `when`
|
|
328
|
+
│ that THROWS is a recorded non-match, never a crashed advance and
|
|
329
|
+
│ never a silent one) → runs. Matching is per (fact, reflex); one
|
|
330
|
+
│ fact can wake five — but never across identities (see the leak
|
|
331
|
+
│ doctrine). A reflex matches only facts at or after its arming.
|
|
332
|
+
│ With ZERO reflexes loaded the pass does nothing at all, rather
|
|
333
|
+
│ than marking the backlog delivered against nobody.
|
|
334
|
+
├─ 3 FAN OUT pending runs → unit tasks, committed IN ONE TRANSACTION with the
|
|
335
|
+
│ run's move to `fanned`. A write fact CARRIES its row: with no
|
|
336
|
+
│ selection declared, that row IS the unit (re-selecting would ask
|
|
337
|
+
│ a question the fact already answers); a declared selection is
|
|
338
|
+
│ ENRICHMENT, re-run under the reflex's own principal — the guard
|
|
339
|
+
│ that re-checks reality. Duplicate unitKey refuses the run
|
|
340
|
+
│ loudly; a transient seam failure DEFERS it.
|
|
341
|
+
├─ 4 CLAIM due tasks and lapsed leases, in one statement — exactly-once,
|
|
342
|
+
│ `order: 'serial'` honoured inside the claim, `attempt`
|
|
343
|
+
│ incremented as part of it, one fencing token per claim
|
|
344
|
+
├─ 5 EXECUTE transform(effect.input, $) → effects[name].run(input, ctx),
|
|
345
|
+
│ raced against timeoutMs
|
|
346
|
+
└─ 6 RECORD done / retry-scheduled / failed, token-checked; buffered emits
|
|
347
|
+
committed in the same transaction; the run's counters updated;
|
|
348
|
+
settled runs drained into their run fact, exactly once
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
Every stage is a pure function over `(reflexes, ledger, now)` plus the injected seams.
|
|
352
|
+
A chain advances one hop per call — which is why the driver drains to quiescence on
|
|
353
|
+
every wake, bounded by the chain-depth ceiling rather than by anybody's patience. Two
|
|
354
|
+
instances advancing concurrently are safe by construction: run inserts are idempotent
|
|
355
|
+
on `(reflexId, cause)`, task claims are exactly-once through the store contract, and a
|
|
356
|
+
non-monotonic `now` (clock skew between hosts) merely delays work — it cannot
|
|
357
|
+
duplicate it.
|
|
358
|
+
|
|
359
|
+
**⟲ There was a poll trigger, and there was a beat.** Polls existed for hosts with no
|
|
360
|
+
write choke point: run a selection on an interval, diff a cursor, mint write facts for
|
|
361
|
+
the delta. In this stack the DAL is the choke point — every application write becomes
|
|
362
|
+
a fact at the vex bridge the instant it commits — so a poll could only re-discover
|
|
363
|
+
what was already pushed, one interval late; and the interval itself (`setInterval`
|
|
364
|
+
calling the old `tick`) was both a latency floor and a throughput ceiling: a chain
|
|
365
|
+
advanced one hop per minute, and a 100-task limit on a 60-second beat capped the
|
|
366
|
+
whole system at 144k tasks a day with morning-long backlogs. Both are gone. The
|
|
367
|
+
committed step survives as `advance` — it is what makes every check deterministic —
|
|
368
|
+
and the pacing moved to the host's driver: wake on ingest, drain to quiescence, sleep
|
|
369
|
+
until `nextDue`. The poll's hardest-won lesson (a non-unique cursor loses ties; a
|
|
370
|
+
minted delta must not cross tenants) lives on in the identity fence, which outlived
|
|
371
|
+
the trigger that taught it.
|
|
372
|
+
|
|
373
|
+
### The ledger
|
|
374
|
+
|
|
375
|
+
**Four tables, and only two of them grow with events.** Each is here because it carries
|
|
376
|
+
a guarantee nothing else can:
|
|
377
|
+
|
|
378
|
+
| | the guarantee |
|
|
379
|
+
|---|---|
|
|
380
|
+
| **`fact`** | durable *before* anything interprets it — so `ingest` is one atomic write, matching is retryable in the advance, and a delayed fact meets the reflexes of the day it fires |
|
|
381
|
+
| **`run`** | `UNIQUE(reflexId, cause)` — *the* idempotency, one constraint serving all four trigger kinds; plus atomic fan-out, the completion counting fan-in reads, and `overlap: 'skip'` |
|
|
382
|
+
| **`task`** | `UNIQUE(runId, unit)`, written before the effect; the lease; the retry counter |
|
|
383
|
+
| **`state`** | where a reflex has got to: arming baseline, clock through-line. One row per reflex — bounded by how many exist, never by what happened |
|
|
384
|
+
|
|
385
|
+
- **`fact`** — everything that arrived: kind, payload, `dedupeKey`, depth, delivery,
|
|
386
|
+
any park. The intake log *is* the audit trail of change.
|
|
387
|
+
- **`run`** — one activation of one reflex: the cause (occurrence key, fact id, run id,
|
|
388
|
+
or `manual:<who>`), the version hash, the identity it ran `as`, what the selection
|
|
389
|
+
returned, the counts, the outcome.
|
|
390
|
+
- **`task`** — one unit of effect work: unit key, pinned `env`, state, attempt count,
|
|
391
|
+
last error, and the handler's returned **output** — stored, which is what makes
|
|
392
|
+
non-write effects (an external call, an agent's answer) selectable downstream
|
|
393
|
+
without inventing a domain table.
|
|
394
|
+
|
|
395
|
+
**⟲ There were seven, and three of them cost real correctness.** The removed ones and
|
|
396
|
+
why each was vocabulary rather than schema:
|
|
397
|
+
|
|
398
|
+
- **`delivery`** — a join table for "which reflexes did this fact wake", which is now
|
|
399
|
+
the `cause` column: `WHERE cause = 'fact:<id>'`. Its other job, recording
|
|
400
|
+
non-matches, grew one row per (fact × loaded reflex) — faster than the work itself —
|
|
401
|
+
and is an event now, which a host logs at whatever grain it wants.
|
|
402
|
+
- **`attempt`** — correctness needs the count and the last error, both of which belong
|
|
403
|
+
on the task. Per-attempt history is the host's log, which is where it was already
|
|
404
|
+
being read from; nothing in this repo ever called `listAttempts`.
|
|
405
|
+
- **`window`** — deleted with coalesce (below).
|
|
406
|
+
- **`watermark`** — an unbounded key/value table with a growing set of string keys,
|
|
407
|
+
for a couple of facts about each reflex. It is columns on `state` now. The clock
|
|
408
|
+
through-line *is* derivable from `MAX(cause)` over the runs, but only while those
|
|
409
|
+
runs exist, and retention is allowed to delete them.
|
|
410
|
+
|
|
411
|
+
Collapsing `fact` into `run` was considered and refused, and the reasoning is the
|
|
412
|
+
sharper half of the principle. Two tables would mean matching moves onto `ingest`:
|
|
413
|
+
evaluating every loaded reflex's `when` and opening N runs, synchronously, on the
|
|
414
|
+
host's request path — where a throw at reflex three of five leaves two runs and no
|
|
415
|
+
record that the fact ever arrived. A delayed fact would resolve against the reflexes of
|
|
416
|
+
its *arrival*, and a fact that matched nothing would leave no trace to dedupe or
|
|
417
|
+
explain. **Durability precedes interpretation** is the same argument as *idempotency
|
|
418
|
+
precedes the effect*, one layer earlier, and the intake table is what makes it true.
|
|
419
|
+
|
|
420
|
+
Causality is first-class: every run knows its cause and every emitted fact knows its
|
|
421
|
+
task, so *"why did this member get this email"* is a walk up a chain of rows —
|
|
422
|
+
`email ← draft approved ← draft written ← charge.failed ← evt_92xk` — not an
|
|
423
|
+
archaeology project.
|
|
424
|
+
|
|
425
|
+
**Retention is a policy, not an accident.** The ledger grows without bound unless told
|
|
426
|
+
otherwise; `sweep(now, { facts?, runs?, tasks? })` deletes settled rows past a horizon,
|
|
427
|
+
and the default is *keep forever*, which is the only honest default when the rows are
|
|
428
|
+
billing history. Deleting a run takes its tasks with it — **⟲** independent horizons
|
|
429
|
+
per table meant sweeping tasks could destroy the `UNIQUE(runId, unit)` row that *is*
|
|
430
|
+
the "this unit already ran" record, and a restore would then re-charge the invoice.
|
|
431
|
+
|
|
432
|
+
### Execution semantics
|
|
433
|
+
|
|
434
|
+
The requirements doc calls this "the part that must not be hand-waved." In order:
|
|
435
|
+
|
|
436
|
+
- **Idempotency.** The task row is written *before* the effect runs, keyed
|
|
437
|
+
`UNIQUE(runId, unit)`. A duplicate trigger, a second instance, a crashed-and-restarted
|
|
438
|
+
advance — all collide with the existing row and are refused. There is no path to the
|
|
439
|
+
effect that does not pass through the insert. Handlers additionally receive the task
|
|
440
|
+
key in `ctx` for downstream idempotency — a payment capture passes it to the provider.
|
|
441
|
+
- **Fan-out is transactional.** A run's tasks commit together with its move to
|
|
442
|
+
`fanned`. This exists because the alternative is quietly catastrophic: a fan-out that
|
|
443
|
+
crashes at row 200 of 500 and resumes would *re-select against moved data* — a member
|
|
444
|
+
who paid in the gap keeps a task minted from the stale pass. Atomic fan-out means a
|
|
445
|
+
crash leaves nothing to resume from; the re-run selects fresh and mints clean. The
|
|
446
|
+
run's `total` is what was **written**, not what was offered — **⟲** counting the
|
|
447
|
+
input array meant any refusal by the unique key left `total` permanently
|
|
448
|
+
unreachable, so the run never settled, never drained, and blocked its reflex for good.
|
|
449
|
+
- **A bad minute is not a bad reflex.** A fan-out failure is two different things. A
|
|
450
|
+
`TideError` is tide's own refusal — a duplicate unit key, a missing `select` seam —
|
|
451
|
+
so the reflex is wrong and will be wrong again next time: the run is `skipped`, with
|
|
452
|
+
a note. Anything else came out of a host seam, and the run stays `pending` to be
|
|
453
|
+
retried, which is a row past its due time that a query can see. **⟲** Every throw
|
|
454
|
+
used to skip; because runs are idempotent on `(reflexId, cause)`, the occurrence
|
|
455
|
+
could never re-materialize, so one unreachable database destroyed that night's
|
|
456
|
+
billing run permanently — and since `skipped` is not `settled`, everything waiting on
|
|
457
|
+
it waited forever.
|
|
458
|
+
- **Retry is a type distinction, not metadata.** An effect that **returns** is done —
|
|
459
|
+
a card decline is not an error but a *domain outcome*, which the handler records and
|
|
460
|
+
the flow branches on via reflexes. An effect that **throws** is transient — tide
|
|
461
|
+
retries on the bounded backoff, then parks the task `failed`: terminal, visible, and
|
|
462
|
+
exit-able only through the human verb `retry()`.
|
|
463
|
+
- **`retry()` rewinds the run, and ⟲ it did not.** A failed task had already settled
|
|
464
|
+
its run, and a claim only reaches tasks that are claimable; reopening the task alone
|
|
465
|
+
left it `pending` forever, so the only documented exit from `failed` did nothing at
|
|
466
|
+
all. Reopening now moves the run back to `fanned` and decrements `failed`, in one
|
|
467
|
+
transaction. The old objection was real — a digest already went out saying twelve
|
|
468
|
+
failed, and re-settling must not send it again — and it is answered by `drained`, a
|
|
469
|
+
flag on the run recording that its settlement has been announced. The run rewinds;
|
|
470
|
+
the announcement does not repeat.
|
|
471
|
+
- **Attempts are fenced.** A timeout marks the attempt failed and schedules a retry —
|
|
472
|
+
but the timed-out effect may still be running. Each claim mints a token; recording a
|
|
473
|
+
result requires the token to still be current, so a zombie completion is discarded
|
|
474
|
+
instead of overwriting the live attempt. (Its external side effect is exactly what
|
|
475
|
+
the downstream idempotency key defends against.) One token per *claim*, not per task:
|
|
476
|
+
the fence's job is to tell this claim apart from an earlier one, and a task appears
|
|
477
|
+
in at most one claim.
|
|
478
|
+
- **A lease is the whole recovery story.** A claim writes `claimedUntil`; a task still
|
|
479
|
+
claimed past it is claimable again. **⟲** There was no lease, no expiry and no
|
|
480
|
+
reaper, so a process that died between the effect and the record left the task
|
|
481
|
+
`claimed` forever: the run never settled, never drained, fan-in stalled, and an
|
|
482
|
+
`overlap: 'skip'` reflex was blocked permanently. The alternative designs — a reaper
|
|
483
|
+
process, a heartbeat table, a liveness service — all add an organ; expiry adds a
|
|
484
|
+
column, and the fencing token is what makes it safe.
|
|
485
|
+
- **Emits are buffered, validated, and narrowed.** `ctx.emit` facts commit with the
|
|
486
|
+
successful attempt, in the same transaction; a throwing attempt discards its buffer,
|
|
487
|
+
or every retry of an emit-then-throw handler would double-mint facts and fire the
|
|
488
|
+
chain twice. Emit accepts `write` and `signal` only: **⟲** it accepted anything and
|
|
489
|
+
validated nothing, so a handler could emit `kind: 'manual'` and fire a **disarmed**
|
|
490
|
+
reflex — manual facts are checked before enablement on purpose, because arming gates
|
|
491
|
+
triggers rather than people, and a handler is not a person.
|
|
492
|
+
- **Concurrency has three dials, each where its limit lives.** Per-task claims are
|
|
493
|
+
exactly-once (in SQL: `FOR UPDATE SKIP LOCKED`). Per-reflex: `overlap: 'skip'`
|
|
494
|
+
refuses to start a run while the last is unsettled, and `order: 'serial'` allows one
|
|
495
|
+
in-flight task at a time — enforced *inside* the claim, because a caller that counts
|
|
496
|
+
what is busy and then claims loses the race to its own second instance. Provider rate
|
|
497
|
+
limits need no tide vocabulary at all: a throttled provider answers 429, the handler
|
|
498
|
+
throws, and the bounded backoff *is* the backpressure.
|
|
499
|
+
- **Overlap governs repeats, not distinct events.** *"May this start while the last is
|
|
500
|
+
unsettled"* is the right question about an occurrence — next month's billing run
|
|
501
|
+
arriving while this month's is still going is the same work coming round again — and
|
|
502
|
+
about a `manual` fact, which is a human pressing the button twice. It is the wrong
|
|
503
|
+
question about a payment. **⟲** It was asked of every cause, and `'skip'` is the
|
|
504
|
+
default, so every fact-triggered reflex silently dropped its second fact per pass:
|
|
505
|
+
three members joining in one minute produced one welcome email and two skipped runs.
|
|
506
|
+
The failure scaled with traffic, which is the wrong direction, and it was invisible
|
|
507
|
+
in the reference app because nothing there ever exercised the watched path with more
|
|
508
|
+
than one row. Discarding an external event that exists exactly once is data loss;
|
|
509
|
+
`order: 'serial'` is the dial that bounds concurrency by queueing instead.
|
|
510
|
+
- **Partial failure.** One task per unit, one transaction per task. If number 237 of
|
|
511
|
+
500 throws, the first 236 stay done, 237 retries or parks, 238 onward run. There is
|
|
512
|
+
no all-or-nothing batch to get wrong.
|
|
513
|
+
- **Catch-up is authored, not guessed.** After downtime, `'run'` fires missed
|
|
514
|
+
occurrences late, `'skip'` drops them, `'latest'` fires only the most recent of
|
|
515
|
+
several missed. Each leaves a run row saying which happened — a skipped run is a
|
|
516
|
+
recorded decision. When a single advance would exceed the per-call cap, the watermark
|
|
517
|
+
stops at the last occurrence actually materialized: **⟲** it used to jump to `now`
|
|
518
|
+
regardless, so everything past the cap was unreachable forever with no row saying it
|
|
519
|
+
had existed.
|
|
520
|
+
- **Timeouts.** Every effect races `timeoutMs`; a hung external call marks the attempt
|
|
521
|
+
failed and frees the schedule.
|
|
522
|
+
- **Silence is visible.** Occurrences are materialized ahead, so a run that should have
|
|
523
|
+
happened and didn't is a `pending` row past its due time — a *query*, not a vanished
|
|
524
|
+
event. "Alert on silence" is a tide reflex watching tide's own ledger.
|
|
525
|
+
|
|
526
|
+
### The tenant boundary inside the engine
|
|
527
|
+
|
|
528
|
+
Every database access a reflex makes already goes through the host: a
|
|
529
|
+
selection and an effect's write both run as that reflex's own principal, under
|
|
530
|
+
a compiled scope policy, and neither can see another tenant's rows. That is
|
|
531
|
+
vex's job and it does it.
|
|
532
|
+
|
|
533
|
+
**Tide is the one place a row travels without being read.** A bridge-minted
|
|
534
|
+
fact carries the row its write committed; `ctx.emit` carries a row a handler
|
|
535
|
+
built; a settled run carries its counts. Those are payloads, not queries — so
|
|
536
|
+
no policy is consulted when they move, and nothing downstream can catch it if
|
|
537
|
+
they move to the wrong place.
|
|
538
|
+
|
|
539
|
+
**⟲ They did, and this is the most serious defect the package has had.** The
|
|
540
|
+
matcher paired facts to reflexes by ENTITY alone. Two studios each running
|
|
541
|
+
"welcome somebody who joins" both watched `memberships`; one studio's fact
|
|
542
|
+
matched the other's reflex, which fanned out over a person it had never
|
|
543
|
+
selected. The competitor's automation then emailed that member — under its own
|
|
544
|
+
studio id, correctly stamped, with somebody else's name and address inside.
|
|
545
|
+
The scope engine was not wrong to allow it: the row written was legitimately
|
|
546
|
+
the sender's. Only the data in it had crossed.
|
|
547
|
+
|
|
548
|
+
So a fact carries **`as`** — the identity of the reflex that minted it — and a
|
|
549
|
+
reflex matches only facts bearing its own. Stamped by the engine and never by
|
|
550
|
+
a handler, for the same reason `depth` is: a handler that could choose its own
|
|
551
|
+
identity could choose somebody else's. The string stays opaque; equality is
|
|
552
|
+
the whole rule, and tide still never learns what a principal is.
|
|
553
|
+
|
|
554
|
+
The rule is **strict**, including at the intake. `ingest(fact, { as })` is how
|
|
555
|
+
a host says whose webhook it just received; a fact with no identity reaches
|
|
556
|
+
only reflexes with no identity. An earlier version exempted unlabelled facts
|
|
557
|
+
on the reasoning that the host knows what it ingested — which is a hole with a
|
|
558
|
+
rationalisation on top, and it failed open.
|
|
559
|
+
|
|
560
|
+
**Identity equality** stops a fact crossing between tenants at every door —
|
|
561
|
+
ingest (the bridge stamps `as` from the WRITE's own scope, mapped by the
|
|
562
|
+
app's naming rule), fire, emit, and run-settlement. And within one tenant, a
|
|
563
|
+
write fact is a STREAM: every reflex watching the entity hears the same fact,
|
|
564
|
+
which is what makes "welcome them" and "tell the desk" two reflexes instead
|
|
565
|
+
of one with two jobs.
|
|
566
|
+
|
|
567
|
+
This is defence in depth, not a replacement for scoping. Vex still enforces
|
|
568
|
+
every read and every write; this closes the gap vex structurally cannot see.
|
|
569
|
+
|
|
570
|
+
### Occurrences and time
|
|
571
|
+
|
|
572
|
+
Occurrence identity is **(reflex, local calendar fields)**, never instants: the
|
|
573
|
+
March monthly run of a studio's billing reflex is `<its id>:2026-03`, the daily is
|
|
574
|
+
`…:2026-03-07` — and since a reflex id names one tenant's row, no two tenants' runs can
|
|
575
|
+
meet on a key. A DST boundary can move the *instant* a key fires at; it cannot mint a
|
|
576
|
+
second key or lose one — the double-fire and the skip are structurally impossible
|
|
577
|
+
rather than carefully avoided. The calendar edges are decided, not discovered:
|
|
578
|
+
|
|
579
|
+
- `every: 'month', on: 31` **clamps to the last day of the month** — February bills on
|
|
580
|
+
the 28th (29th), because "skip February" is never what a billing rule means.
|
|
581
|
+
- A local time erased by spring-forward resolves to the **first valid instant after**;
|
|
582
|
+
a local time that fall-back makes ambiguous takes the **first occurrence**. The key
|
|
583
|
+
is identical either way.
|
|
584
|
+
- Timezone resolution uses the platform's IANA data (`Intl`), keeping the dependency
|
|
585
|
+
count at zero. "03:00 in Vienna" and "03:00 in Denver" are both honest.
|
|
586
|
+
|
|
587
|
+
Occurrences being enumerable is also what makes **backfill** (deferred) natural:
|
|
588
|
+
"run the report reflex over the last 12 months" is materializing twelve past keys.
|
|
589
|
+
|
|
590
|
+
### Arming, and what a pause means
|
|
591
|
+
|
|
592
|
+
Matching starts at a reflex's arming, and a reflex never retro-fires. Two faces of
|
|
593
|
+
one rule, each needing its own sentence because each has its own way of reaching for
|
|
594
|
+
the past: a **clock** reflex with no arming time establishes its baseline on the
|
|
595
|
+
first advance and materializes nothing; and a **fact** older than the arming belongs
|
|
596
|
+
to a world in which the reflex did not exist.
|
|
597
|
+
|
|
598
|
+
A **paused** reflex materializes nothing and keeps its watermark moving. A reflex
|
|
599
|
+
*entering* the loaded set re-baselines to the `at` passed to `load`. Both say the same
|
|
600
|
+
thing: coming back is not the same as never having left.
|
|
601
|
+
|
|
602
|
+
**⟲ This is where the worst defect in the package lived.** A disarmed reflex was
|
|
603
|
+
skipped outright, which froze its watermark at the moment of the pause. Eight days off
|
|
604
|
+
meant eight occurrences materialized the instant it came back — and eight real effects
|
|
605
|
+
executed. Verified by running the built engine, not by reading it.
|
|
606
|
+
|
|
607
|
+
**⟲ And there is no `arm`/`disarm`.** There was a pair, and they mutated an in-memory
|
|
608
|
+
map: a second copy of a fact the host already owned as a column, lost on restart, and
|
|
609
|
+
disagreeing with the host's own screen in between — so re-reading the rows silently
|
|
610
|
+
undid a pause somebody had just made. Enablement is a property of the reflex the host
|
|
611
|
+
hands over. To pause one: write your own row, `load` again. One source of truth, it
|
|
612
|
+
survives a restart, and it leaves something to audit where a method call left nothing.
|
|
613
|
+
|
|
614
|
+
### Preview — dry run as a verb
|
|
615
|
+
|
|
616
|
+
`preview(reflexId, { now, fact? })` runs the real pipeline — materialize or match,
|
|
617
|
+
select against real data, evaluate every template — and stubs **one function**: the
|
|
618
|
+
effect executor. What comes back is the run that would happen: the occurrence or
|
|
619
|
+
fact, the selected rows (the eleven members, by name), each task's resolved input,
|
|
620
|
+
plus whatever the effect's optional `preview` hook renders. Because every effect passes
|
|
621
|
+
through the same choke point, there is no per-reflex dry-run flag and no `if (dryRun)`
|
|
622
|
+
to forget — a reflex *cannot* opt out of being previewable. Preview is also where
|
|
623
|
+
template typos surface, which makes it the authoring loop's inner verb.
|
|
624
|
+
|
|
625
|
+
### The graph, verified at load
|
|
626
|
+
|
|
627
|
+
`load(reflexes)` validates every artifact, hashes versions, and derives the graph:
|
|
628
|
+
what reflexes WATCH (their triggers) × what their effects WRITE × their **run-fact
|
|
629
|
+
subscriptions** — that last edge class matters, because fan-in creates cycles the
|
|
630
|
+
write edges alone can't see (B fires on A's run; B's run feeds C; C writes what
|
|
631
|
+
triggers A).
|
|
632
|
+
|
|
633
|
+
Cycles are **classified, not banned** — because tide's own best patterns are cycles
|
|
634
|
+
on purpose: a drip campaign is a reflex whose delayed fact fires itself; "retry the
|
|
635
|
+
decline in 3 days" loops through `notBefore`; a drain loop subscribes to its own run.
|
|
636
|
+
All converge because every loop passes through a **guard**: a selection that re-checks
|
|
637
|
+
reality, a `when`, or a `notBefore` delay. Static analysis cannot tell a convergent
|
|
638
|
+
cycle from a divergent one, so it does not pretend to:
|
|
639
|
+
|
|
640
|
+
- An **unguarded** cycle — every hop unconditional, no selection, no `when`, no
|
|
641
|
+
`notBefore` anywhere on the loop — is refused at load: it diverges by construction.
|
|
642
|
+
- A **guarded** cycle is legal and reported as a finding, so the author sees the loop
|
|
643
|
+
they built.
|
|
644
|
+
- The runtime backstop is nearly free because causality already exists: a fact whose
|
|
645
|
+
cause chain exceeds `maxChainDepth` is **parked for review instead of fired**. Depth
|
|
646
|
+
is stamped on the task at fan-out rather than read back from the run at emit time —
|
|
647
|
+
**⟲** a run swept out from under a long-running chain answered `undefined`, resetting
|
|
648
|
+
the ceiling to zero in exactly the swept, long-running case the backstop exists for.
|
|
649
|
+
Releasing a parked fact **records** the override; **⟲** clearing the park alone was a
|
|
650
|
+
ping-pong, because the depth that parked it had not changed and never would.
|
|
651
|
+
|
|
652
|
+
Load **refuses** outright: an unguarded cycle, a reflex naming an unregistered
|
|
653
|
+
effect, a `when` on a non-fact trigger. The verification is only as good as the
|
|
654
|
+
effects' `writes`: under moss they are *derived* (from `mutationEffect` on the vex
|
|
655
|
+
mutation behind each effect), so an edge is never guessed; a plain host declares
|
|
656
|
+
them or forgoes the check, and an effect with none is reported as a **blind** edge,
|
|
657
|
+
never silently trusted — under moss, blind means something bypassed vex, which is
|
|
658
|
+
exactly when a loud word is wanted. If it loads, it's coherent — the moss tradition,
|
|
659
|
+
one layer down.
|
|
660
|
+
|
|
661
|
+
---
|
|
662
|
+
|
|
663
|
+
## Flows — chains, not bodies
|
|
664
|
+
|
|
665
|
+
The requirements' hardest sentence: *"charge the card, and if that succeeds mark the
|
|
666
|
+
invoice paid and send the receipt; if it fails, record the failure and schedule a
|
|
667
|
+
retry — without dropping into imperative code."* As a chain:
|
|
668
|
+
|
|
669
|
+
```
|
|
670
|
+
clock ─→ [billing.charge-due] ── payments.charge ──→ charge_attempts row
|
|
671
|
+
│
|
|
672
|
+
fact: charge_attempts (succeeded) ─→ [billing.mark-paid] ─→ invoices row (paid)
|
|
673
|
+
fact: charge_attempts (declined) ─→ [billing.record-decline] ─→ invoice failed row
|
|
674
|
+
fact: invoices (paid) ─→ [billing.send-receipt] ─→ mail
|
|
675
|
+
fact: run billing.charge-due ─→ [billing.digest] ─→ owner's summary mail
|
|
676
|
+
clock (daily) + select: declines ≥ 3 days old, still open ─→ [billing.dunning-notice]
|
|
677
|
+
```
|
|
678
|
+
|
|
679
|
+
Six reflexes, co-located in one `billing.reflexes.ts`. Every arrow is a committed row;
|
|
680
|
+
a crash anywhere resumes from the last row; every intermediate state has a name an
|
|
681
|
+
operator can query. Note the dunning line: it needs no cancel API when the member pays
|
|
682
|
+
— its selection asks "still open", and zero rows is an ordinary outcome.
|
|
683
|
+
|
|
684
|
+
**Data passes through three lanes, and the principle is one sentence:** *facts carry
|
|
685
|
+
the datum, the ledger carries the history, the domain rows carry the state — and
|
|
686
|
+
selection reaches all three.* The fact brings `$.fact.row`; task `output` makes any
|
|
687
|
+
effect's result selectable; and a selection joins whatever it needs (under moss, tide's
|
|
688
|
+
tables are vex entities — the digest's selection reads the tasks of `$.fact.runId`
|
|
689
|
+
joined to members). Nothing is "passed" in memory; everything is written where the next
|
|
690
|
+
reflex can read it.
|
|
691
|
+
|
|
692
|
+
**The granularity rule** — the one authored judgment this design asks for: *a joint
|
|
693
|
+
the business cares about (retry it independently, observe it, resume from it) is a
|
|
694
|
+
fact + reflex; a joint it doesn't care about lives inside one effect handler.* Two
|
|
695
|
+
external API calls that form one atomic business step are one handler — an endpoint is
|
|
696
|
+
a sanctioned home for imperative code, and the artifact thesis governs the automation,
|
|
697
|
+
not every HTTP call inside an effect.
|
|
698
|
+
|
|
699
|
+
**Mode guidance:** `batch` pins the whole result into one task and is for bounded sets
|
|
700
|
+
(a digest of dozens); a large set is `each` + a run-fact digest — 500 tasks and one
|
|
701
|
+
summary reflex, not one task carrying 500 rows.
|
|
702
|
+
|
|
703
|
+
**⟲ And there is no `coalesce`.** A policy field held matching facts in a window and
|
|
704
|
+
fired once with the batch. It cost two port methods, a table, an exactly-once promise
|
|
705
|
+
and a `DELETE … RETURNING` on every advance — and nothing ever set it; its only driver was
|
|
706
|
+
one test. It also had a quiet correctness bug: a throwing group key fell back to the
|
|
707
|
+
shared `''` window, so a per-customer digest could carry another tenant's rows. A
|
|
708
|
+
digest that genuinely needs batching is a **delayed run**: the first fact of a group
|
|
709
|
+
mints one at `now + window` under `cause: 'coalesce:<key>:<start>'`, the rest collide on
|
|
710
|
+
`UNIQUE(reflexId, cause)` and are refused, and when it comes due the reflex selects
|
|
711
|
+
what changed. Mechanisms that already exist, consistent with *the database is the
|
|
712
|
+
interpreter*, and no table.
|
|
713
|
+
|
|
714
|
+
---
|
|
715
|
+
|
|
716
|
+
## The seams
|
|
717
|
+
|
|
718
|
+
Five injection points. Every one is a move the stack has already made once — which is
|
|
719
|
+
the argument that tide is a *nisc* library and not a scheduler with JSON config:
|
|
720
|
+
|
|
721
|
+
| Seam | Contract | The host injects | Precedent |
|
|
722
|
+
|---|---|---|---|
|
|
723
|
+
| `store` | `TideStore` — six capabilities over four tables (below) | `createMemoryStore()` shipped; `createTideStore(pool)` from **moss** for persistence | vex's `CacheBackend` — structural, driver-free |
|
|
724
|
+
| `transform` | `(config, source) => value` | Prism's `evaluate`, or anything | **nova's exact socket** — nova doesn't know Prism; neither does tide |
|
|
725
|
+
| `select` | `(query, ctx) => AsyncIterable<row>` — the query blob is opaque | vex replay under the actor's policy; or raw SQL; or an array | vex's `generateDsl` / `mapToShape` |
|
|
726
|
+
| `effects` | `name → { run(input, ctx), writes?, preview? }` | vex mutation replays, fns, cortex agents; or plain functions | moss's `functions` seam |
|
|
727
|
+
| `ctx` | opaque, threaded through to `select` and `effects`; carries `taskKey`, `emit`, `now` | moss: `ActorContext` (principal, policy, wire) | charter's universe-blindness, applied to identity |
|
|
728
|
+
|
|
729
|
+
Plus the driver-facing edges, contracts rather than injections: **`ingest(fact)`**
|
|
730
|
+
in, **`advance({ now, limit })`** to run one committed increment, and
|
|
731
|
+
**`nextDue(now)`** so the driver knows when next to wake.
|
|
732
|
+
|
|
733
|
+
Dependencies: **`zod`**. Nothing else — the same bar charter clears.
|
|
734
|
+
|
|
735
|
+
### The store contract
|
|
736
|
+
|
|
737
|
+
```ts
|
|
738
|
+
type TideStore = {
|
|
739
|
+
transact: <T>(fn: (tx: TideStore) => Promise<T>) => Promise<T>;
|
|
740
|
+
appendIfAbsent: (table, row) => Promise<Row | undefined>;
|
|
741
|
+
claim: (spec: ClaimSpec) => Promise<readonly Row[]>;
|
|
742
|
+
cas: (table, id, expect, set) => Promise<boolean>;
|
|
743
|
+
query: (spec: QuerySpec) => Promise<readonly Row[]>;
|
|
744
|
+
remove: (spec: RemoveSpec) => Promise<number>;
|
|
745
|
+
};
|
|
746
|
+
```
|
|
747
|
+
|
|
748
|
+
**⟲ It was twenty-seven methods**, shaped by nouns × verbs, and everything wrong with
|
|
749
|
+
the storage layer was downstream of that. `claimTasks`, `drainSettled` and
|
|
750
|
+
`claimClosedWindows` were three hand-written implementations of *one* operation, and
|
|
751
|
+
they diverged three ways; nine of the methods were reporting reads the engine never
|
|
752
|
+
called, so a reporting surface lived inside an execution contract. Shaped by what the
|
|
753
|
+
guarantees need instead, it is six — and the engine holds every opinion about retry,
|
|
754
|
+
arming, occurrences and coalescing, while the store holds none.
|
|
755
|
+
|
|
756
|
+
`UNIQUE_BY` is exported alongside it: the four unique keys **are** the exactly-once
|
|
757
|
+
promises, written down once so no store can drift from another's reading of a comment.
|
|
758
|
+
|
|
759
|
+
**Ledger reads left the port entirely.** Under moss they are vex entries over the same
|
|
760
|
+
tables — authored, scoped, cached, inspectable; standalone they are the host's own
|
|
761
|
+
queries. `tide.ledger.*` is a convenience over `query`, not a contract a store
|
|
762
|
+
implements.
|
|
763
|
+
|
|
764
|
+
### Holding two stores to one definition
|
|
765
|
+
|
|
766
|
+
`STORE_CONTRACT`, exported from `@niscorp/tide/testing`, is the executable definition
|
|
767
|
+
of everything above: a list of `{ name, run(store) }` checks that throw, with no test
|
|
768
|
+
framework, so any runner can drive them. The memory store runs it; moss's SQL store
|
|
769
|
+
runs the same list against a real database.
|
|
770
|
+
|
|
771
|
+
**⟲ This exists because of the store that was deleted.** `store/postgres.ts` was 700
|
|
772
|
+
lines, had never been instantiated anywhere in this repo, and claimed in its own header
|
|
773
|
+
to be "held to the same tests" — of which there were none. It diverged from the memory
|
|
774
|
+
store in eleven ways: `LIMIT` applied before the serial filter, so one serial reflex
|
|
775
|
+
with 5,000 due tasks starved every other reflex; list reads answering opposite ends of
|
|
776
|
+
the same list; a dedupe index freed by the sweep, so a replayed webhook re-fired after
|
|
777
|
+
the retention horizon; three transactional methods that skipped the ready-guard and
|
|
778
|
+
raised `undefined_table` *inside* a transaction. Every one was invisible. The lesson is
|
|
779
|
+
not "write more tests" — it is that a contract nobody can execute is a comment.
|
|
780
|
+
|
|
781
|
+
---
|
|
782
|
+
|
|
783
|
+
## Hosts
|
|
784
|
+
|
|
785
|
+
### Moss — the reference host
|
|
786
|
+
|
|
787
|
+
Moss fills every seam, and now the last one:
|
|
788
|
+
|
|
789
|
+
| Tide seam | Moss fills it with |
|
|
790
|
+
|---|---|
|
|
791
|
+
| `store` | **`createTideStore(pool)`** — four tables beside `vex_cache`, `ON CONFLICT DO NOTHING` for `appendIfAbsent`, `FOR UPDATE SKIP LOCKED` for `claim`, CHECK constraints on every state column, and a foreign key that cascades work with its run |
|
|
792
|
+
| `select` | vex replay **through the actor's own wire** — selection inherits replay-only, scoped, cached, inspectable; the tenant filter is in the SQL, engine-injected, unforgeable |
|
|
793
|
+
| `effects` | every seeded vex **mutation** auto-registered under its fingerprint (with `writes` *derived* from `mutationEffect`) + the app's `effects(actor)` |
|
|
794
|
+
| `transform` | prism — the same wiring the shell's transform socket already gets |
|
|
795
|
+
| `ctx` | `ActorContext`: the charter-resolved, engine-enforced, no-shell principal |
|
|
796
|
+
| the ledger | ordinary vex entries over `tide_run`, scoped by the host on the identity the run declared |
|
|
797
|
+
|
|
798
|
+
**The engine's writes do not go through vex's mutation pipeline, and the reason is
|
|
799
|
+
worth stating because the opposite looks tidier.** A vex mutation is an authored,
|
|
800
|
+
scope-compiled statement replayed on behalf of a principal — the right shape for an
|
|
801
|
+
application write, and the wrong shape for an engine claiming its own task: there is no
|
|
802
|
+
principal, no tenant to scope to, and no user input to police. Routing bookkeeping
|
|
803
|
+
through a policy compiler would add a scope rule permitting everything, which is a way
|
|
804
|
+
of saying the rule is not doing anything. The **reads** are the half that belongs to
|
|
805
|
+
vex, and they are authored entries like any other.
|
|
806
|
+
|
|
807
|
+
**Tide's tables are ordinary tables.** The first instinct was to hide them from
|
|
808
|
+
introspection, since they carry no tenancy of their own — but that is true of any table
|
|
809
|
+
with no scope rule, and it is the scope rule that has always been what makes
|
|
810
|
+
`memberships.read` safe. Hiding them would have closed the door as well as the hole: a
|
|
811
|
+
scoped entry over the run ledger cannot compile against a table vex has never heard of.
|
|
812
|
+
|
|
813
|
+
**Two live vex bugs surfaced on the way here**, both fixed rather than worked around:
|
|
814
|
+
no adapter implemented `transaction`, so *every batch mutation in every app* threw
|
|
815
|
+
"Batch mutations require a transactional client" — a message about a client nothing in
|
|
816
|
+
the repo could construct; and the PGlite shim's method needed its receiver, so
|
|
817
|
+
destructuring it produced a transaction that failed inside the library.
|
|
818
|
+
|
|
819
|
+
### Plain Node — the floor
|
|
820
|
+
|
|
821
|
+
```ts
|
|
822
|
+
const tide = createTide({
|
|
823
|
+
store: createMemoryStore(), // or your own, held to STORE_CONTRACT
|
|
824
|
+
transform: (config, source) => evaluate(config, source), // prism, or your own
|
|
825
|
+
select: async function* (q) { yield* (await pg.query(q.sql, q.params)).rows; },
|
|
826
|
+
effects: {
|
|
827
|
+
'mail.send': { run: (input) => mailer.send(input) },
|
|
828
|
+
'orders.close': { run: async (input, ctx) => { await pg.query(/* … */);
|
|
829
|
+
ctx.emit({ kind: 'write', entity: 'orders', row, at: ctx.now }); },
|
|
830
|
+
writes: ['orders'] },
|
|
831
|
+
},
|
|
832
|
+
});
|
|
833
|
+
tide.load(reflexes, { at: Date.now() });
|
|
834
|
+
app.post('/webhook/stripe', (req) => tide.ingest(toFact(req))); // validate at the boundary first
|
|
835
|
+
|
|
836
|
+
// The driver: wake on ingest, drain to quiescence, sleep until nextDue.
|
|
837
|
+
// (Under moss this is `createTideDriver`; standalone it is these lines.)
|
|
838
|
+
const drain = async () => { for (;;) { const r = await tide.advance({ now: Date.now() }); if (isQuiet(r)) break; } };
|
|
839
|
+
```
|
|
840
|
+
|
|
841
|
+
No vex, no moss, no nova — and no magic either: facts must be ingested (a host with
|
|
842
|
+
its own write choke point pushes; one without builds an importer that pushes),
|
|
843
|
+
`writes` must be declared, the driver is theirs to run, and the outbox guarantee is
|
|
844
|
+
theirs to keep. `ctx.emit` is how a chain continues without a write choke point.
|
|
845
|
+
|
|
846
|
+
---
|
|
847
|
+
|
|
848
|
+
## File structure
|
|
849
|
+
|
|
850
|
+
```
|
|
851
|
+
src/
|
|
852
|
+
index.ts Public API barrel — createTide, schemas, the store contract
|
|
853
|
+
testing.ts STORE_CONTRACT — the executable definition of a store
|
|
854
|
+
tide.ts createTide: load, ingest, advance, nextDue, fire, retry, preview, ledger, sweep
|
|
855
|
+
types.ts ledger rows, seam types, TideStore, UNIQUE_BY, reports
|
|
856
|
+
errors.ts TideError (code + details)
|
|
857
|
+
|
|
858
|
+
schemas/
|
|
859
|
+
reflex.schema.ts ReflexSchema — trigger, select, effect, policy, enabled
|
|
860
|
+
trigger.schema.ts clock | fact | manual, with narrowing helpers
|
|
861
|
+
fact.schema.ts FactInputSchema — the public intake contract, union enforced
|
|
862
|
+
policy.schema.ts retry, backoff, overlap, order, catchUp, lateMs
|
|
863
|
+
index.ts
|
|
864
|
+
|
|
865
|
+
engine/
|
|
866
|
+
advance.ts the pipeline, in order, over (reflexes, store, now)
|
|
867
|
+
due.ts nextDue — the instant the driver should wake for
|
|
868
|
+
runtime.ts EngineDeps, the `$` environment, isTruthy, versionOf
|
|
869
|
+
occurrence.ts calendar math, local-field keys, clamping, DST edges
|
|
870
|
+
materialize.ts clock → runs (catch-up, watermark)
|
|
871
|
+
match.ts fact × reflex, `when`, and openRun — the one place a run is born
|
|
872
|
+
fanout.ts selection → transactional unit-task commit; refuse vs defer
|
|
873
|
+
execute.ts claim → fence → transform → effect → record; the ONE door
|
|
874
|
+
preview.ts the same pipeline with the executor stubbed
|
|
875
|
+
graph.ts watches × writes × run subscriptions; refusal at load
|
|
876
|
+
|
|
877
|
+
store/
|
|
878
|
+
memory.ts createMemoryStore — the reference implementation + snapshot()
|
|
879
|
+
|
|
880
|
+
test/
|
|
881
|
+
occurrence.test.ts DST both directions, clamping, key stability
|
|
882
|
+
engine.test.ts the execution semantics, end to end on a fake clock
|
|
883
|
+
durability.test.ts one test per defect that reached the built engine
|
|
884
|
+
load.test.ts the load gate: validation, cycles, versioning
|
|
885
|
+
preview.test.ts dry run writes nothing and shows everything
|
|
886
|
+
store.test.ts STORE_CONTRACT × every store
|
|
887
|
+
```
|
|
888
|
+
|
|
889
|
+
`materialize.ts` is split out of `advance.ts` because the clock is where the engine
|
|
890
|
+
*creates* work from nothing, and it reads better alone than buried in the
|
|
891
|
+
orchestrator. There is no `verbs.ts`: `fire` is four lines of sugar over `ingest`,
|
|
892
|
+
and `retry` belongs beside the executor whose invariants it rewinds.
|
|
893
|
+
|
|
894
|
+
There is no `store/postgres.ts`. **The moss integration lives in moss, not here — tide
|
|
895
|
+
must never import a host**, and persistence is where that rule earns its keep.
|
|
896
|
+
|
|
897
|
+
---
|
|
898
|
+
|
|
899
|
+
## Key design decisions
|
|
900
|
+
|
|
901
|
+
1. **No run body — the database is the interpreter.** Three earlier shapes died to
|
|
902
|
+
get here: a bespoke step language (new grammar the stack didn't need), nova's
|
|
903
|
+
trigger steps executed headlessly (the right grammar, but it hauled in shells and
|
|
904
|
+
render machinery), and a moss subsystem (which handed moss a vocabulary, the one
|
|
905
|
+
thing moss refuses to own). What survived: an in-memory step chain must journal each
|
|
906
|
+
step to survive a crash — rebuilding a workflow engine's checkpointing — while a
|
|
907
|
+
chain of facts and reflexes gets durability, resumability, per-step retry policy and
|
|
908
|
+
per-step observability *from its shape*. The cost is real and named: flows span
|
|
909
|
+
artifacts. The answer is co-location in authoring and the derived graph at load.
|
|
910
|
+
|
|
911
|
+
2. **A table earns its place with a guarantee, not a noun.** Seven became four. The
|
|
912
|
+
surviving test is short: *what does this schema promise that nothing else can?* An
|
|
913
|
+
intake that is durable before interpretation, one uniqueness constraint that is the
|
|
914
|
+
idempotency for every trigger kind, one that is the idempotency for a unit of work,
|
|
915
|
+
and a bounded row saying where each reflex has got to. Everything else was a
|
|
916
|
+
vocabulary word that had been handed a table and, with it, accessors, retention,
|
|
917
|
+
divergence and bugs.
|
|
918
|
+
|
|
919
|
+
3. **Fan-in is bookkeeping, not a primitive.** The run fact exists because tide
|
|
920
|
+
already knows when the last task settles — emitting that knowledge as an ordinary
|
|
921
|
+
fact turns fan-in, dependencies and batch digests into ordinary reflexes. Exactly
|
|
922
|
+
once through `drained`, a flag on the row rather than an in-memory queue: a queue
|
|
923
|
+
cannot survive a restart and cannot be rewound, which is what made fan-in and the
|
|
924
|
+
recovery verb mutually exclusive.
|
|
925
|
+
|
|
926
|
+
4. **Fan-out is transactional; unit keys collide loudly.** A resumable partial
|
|
927
|
+
fan-out re-selects against moved data — for a billing run, that is charging someone
|
|
928
|
+
who already paid. Atomicity makes the crash case the clean case. And a duplicate
|
|
929
|
+
`unitKey` inside one run refuses the run rather than silently dropping a row —
|
|
930
|
+
authoring errors are loud or they are invisible.
|
|
931
|
+
|
|
932
|
+
5. **Attempts are fenced; emits are buffered; claims are leased.** All three exist for
|
|
933
|
+
one reason: a retry must not let a previous try's ghost act, and a dead process must
|
|
934
|
+
not hold work forever. The token check, the buffer-until-success rule, and lease
|
|
935
|
+
expiry close the three leaks a naive executor ships with — and none of them needs a
|
|
936
|
+
background process.
|
|
937
|
+
|
|
938
|
+
6. **Retry classification is a calling convention.** Return = domain outcome, recorded
|
|
939
|
+
as data, branched on by reflexes. Throw = transient, retried on bounded backoff to
|
|
940
|
+
a terminal, human-visible state. Per-effect metadata declaring which errors are
|
|
941
|
+
retryable is a registry someone forgets to update; a type distinction at the seam
|
|
942
|
+
cannot be forgotten, and it puts the decision where the knowledge lives.
|
|
943
|
+
|
|
944
|
+
7. **The effect executor is the only door.** Everything that leaves tide — a write, an
|
|
945
|
+
email, an agent — passes through one function. That single choke point is what
|
|
946
|
+
makes preview a *verb*, timeout uniform, the task key uniformly available, and the
|
|
947
|
+
ledger complete. A second path outward would quietly break all four.
|
|
948
|
+
|
|
949
|
+
8. **Limits live where they bind.** Task uniqueness at the store; run overlap and
|
|
950
|
+
task order on the reflex; provider ceilings in the handler that owns the provider
|
|
951
|
+
relationship. Tide ships no rate vocabulary of its own.
|
|
952
|
+
|
|
953
|
+
9. **Occurrence identity is local calendar fields, with decided edges.** `2026-03`,
|
|
954
|
+
not an epoch millisecond — DST cannot mint or lose a key. Day-31 clamps to
|
|
955
|
+
month-end; erased local times take the next valid instant; ambiguous ones take the
|
|
956
|
+
first. Decisions in the schema's documentation, not discoveries in production.
|
|
957
|
+
|
|
958
|
+
10. **A reflex never retro-fires, and a pause is a pause.** Matching starts at arming;
|
|
959
|
+
history before a reflex existed is reachable only through deliberate backfill. The
|
|
960
|
+
two faces — the clock's first advance, a fact older than the
|
|
961
|
+
arming — are one principle: *a new watcher starts watching now.* The fourth face,
|
|
962
|
+
learned the hard way, is that a paused watcher's clock keeps moving.
|
|
963
|
+
|
|
964
|
+
11. **Enablement is the host's column, not the engine's method.** Two copies of one
|
|
965
|
+
fact is the bug; which copy wins after a restart is only the symptom. `load` again
|
|
966
|
+
is the whole mechanism, and it is auditable where a method call was not.
|
|
967
|
+
|
|
968
|
+
12. **Selection and shaping are opaque on purpose.** Tide stores, diffs and hashes the
|
|
969
|
+
query and template blobs but never interprets them. The moment tide validates a
|
|
970
|
+
query it owns a query language; the moment it evaluates an expression it owns an
|
|
971
|
+
evaluator — both exist in the stack already, better, behind seams nova proved out.
|
|
972
|
+
The honest cost — template typos surface at preview, not load — is why preview is
|
|
973
|
+
the authoring loop's inner verb, and why preview must use the engine's own
|
|
974
|
+
truthiness rather than a second reading of it.
|
|
975
|
+
|
|
976
|
+
13. **Identity is the host's, entirely — and written down.** Tide threads an opaque
|
|
977
|
+
`ctx`; moss's `ActorContext` supplies principal, compiled policy and tenancy. An
|
|
978
|
+
automation that "runs as root" is unbuildable through this seam, because tide has
|
|
979
|
+
no root to offer. Recording `as` on the run is what lets a host scope the ledger
|
|
980
|
+
without tide learning what it scoped.
|
|
981
|
+
|
|
982
|
+
14. **Versioning is a content hash, and the ledger pins it.** A reflex's version is the
|
|
983
|
+
hash of its definition, so the ledger can always say "March ran the old version,
|
|
984
|
+
April the new one." `enabled` is excluded — flipping it isn't an edit. `params` are
|
|
985
|
+
included — changing `graceDays` from 3 to 7 is a behavioural change the ledger must
|
|
986
|
+
explain.
|
|
987
|
+
|
|
988
|
+
15. **A contract nobody can execute is a comment.** `STORE_CONTRACT` ships from the
|
|
989
|
+
package because the alternative was tried: a store whose header said it was held to
|
|
990
|
+
the same tests, and eleven silent divergences.
|
|
991
|
+
|
|
992
|
+
---
|
|
993
|
+
|
|
994
|
+
## Against the requirements
|
|
995
|
+
|
|
996
|
+
The wishlist, answered — including its own open questions:
|
|
997
|
+
|
|
998
|
+
| Requirement | Mechanism |
|
|
999
|
+
|---|---|
|
|
1000
|
+
| Clock / change / external / manual triggers | clock, write facts (pushed at the choke point), signal facts, `fire()` — one ledger under all of them |
|
|
1001
|
+
| Timezone honesty, no DST double-fire | tz is a field on the tenant's own reflex row + local-calendar occurrence keys + decided clamping (§ Tenancy, § decision 9) |
|
|
1002
|
+
| Selection = an ordinary vex read | the `select` seam; under moss, literally a fingerprint replay under the actor's policy |
|
|
1003
|
+
| Per-row and per-batch | `select.mode`; large sets = `each` + a run-fact digest |
|
|
1004
|
+
| Streaming large sets | `select` returns an `AsyncIterable`; fan-out commits in one transaction |
|
|
1005
|
+
| Effects: write / call out / agent / sequenced with explicit failure | the effects registry; sequences are chains; failure branches are reflexes on outcome rows |
|
|
1006
|
+
| Idempotency before the effect | task insert refused by `UNIQUE(runId, unit)`; task key in `ctx` for downstream |
|
|
1007
|
+
| Retry vs terminal | throw vs return (§ decision 6); `retry()` for the parked, and it rewinds the run |
|
|
1008
|
+
| Concurrency, overlap, ordering, partial failure, timeout, catch-up | leased claims + fencing, `overlap`, `order: 'serial'` inside the claim, task-per-unit, `timeoutMs`, `catchUp` |
|
|
1009
|
+
| Scoped no-session identity, tenant-bounded | `ctx` seam ← moss's `ActorContext`; engine-side enforcement, unchanged |
|
|
1010
|
+
| Run ledger, dry run, preview with names, silence alerting, ledger-as-data | facts/runs/tasks; `preview()`; materialized-ahead pending rows; host-readable tables with a retention policy |
|
|
1011
|
+
| Per-tenant config, instant disarm, versioning across runs | reflexes are tenant rows; disarming is the host's row write + `load` (§ decision 11); content-hash versions (§ decision 14) |
|
|
1012
|
+
| Host-agnostic wake-up; serverless and local both | `advance({ now })` + `nextDue(now)`: the driver wakes on ingest and sleeps until the next due instant; its janitor beat is recovery, not pacing |
|
|
1013
|
+
| Deterministic headless checks | memory store + fake clock; no timers exist to wait on |
|
|
1014
|
+
| Durable across restarts | `createTideStore(pool)` in moss; the memory store remains the reference and the check harness |
|
|
1015
|
+
| Wishlist: dependency between automations | run facts — `on: { fact: { run: 'A' } }` with `when` on the stats |
|
|
1016
|
+
| **Open: one library or two?** | one, standalone — "react to a change" is a fact reflex; same machine |
|
|
1017
|
+
| **Open: does the identity belong here?** | no — moss's `ActorContext`, consumed through an opaque seam |
|
|
1018
|
+
| **Open: how much effect vocabulary is closed?** | the seam is closed (one shape), the vocabulary is open (registered handlers) |
|
|
1019
|
+
| **Open: what does versioning mean?** | content hash pinned by the ledger; `enabled` is the host's field, so pausing isn't a version |
|
|
1020
|
+
| **Open: do notifications belong here?** | no — an automation writes rows; whatever notifies is another consumer of rows |
|
|
1021
|
+
|
|
1022
|
+
---
|
|
1023
|
+
|
|
1024
|
+
## Non-goals
|
|
1025
|
+
|
|
1026
|
+
- **A workflow engine's machinery, refused — the capability is the point.** A click
|
|
1027
|
+
cascading into writes, calls, and follow-ups *is* a workflow by any honest name,
|
|
1028
|
+
and it is exactly what tide is for. What tide refuses is the machinery tradition:
|
|
1029
|
+
a central orchestrator holding in-memory state, parked step interpreters,
|
|
1030
|
+
approval-step primitives, barriers. Here the workflow exists as rows — which is
|
|
1031
|
+
why it survives a crash, shows up in a ledger, and needs no engine to resume.
|
|
1032
|
+
Human-in-the-loop is the same shape: an automation writes a draft, a person's
|
|
1033
|
+
ordinary action approves it (a write), a reflex carries on from that row.
|
|
1034
|
+
- **Not a message bus, not an event-sourcing substrate.** The fact ledger is an intake
|
|
1035
|
+
log, not a topology; nothing subscribes to tide but tide.
|
|
1036
|
+
- **Not real-time.** Minute-level cadence is the floor and the promise. Stream-shaped
|
|
1037
|
+
work belongs in a stream processor; tide is for work a diligent clerk could have
|
|
1038
|
+
done on a schedule.
|
|
1039
|
+
- **Not a replacement for provider retries.** Where Stripe already does it well, the
|
|
1040
|
+
effect handler defers to Stripe and records the outcome.
|
|
1041
|
+
- **Not a lock service.** `order: 'serial'` is as exact as the store's claim can make
|
|
1042
|
+
it in one statement, and `overlap: 'skip'` — which is a unique constraint — is the
|
|
1043
|
+
guarantee to reach for when it must be exact.
|
|
1044
|
+
|
|
1045
|
+
## Deferred
|
|
1046
|
+
|
|
1047
|
+
- **Backfill** — materializing past occurrence keys deliberately. Natural because keys
|
|
1048
|
+
are enumerable; deferred because catch-up covers outages.
|
|
1049
|
+
- **A true multi-way barrier** ("when A *and* B *and* C") — run facts cover sequential
|
|
1050
|
+
dependency and single-run fan-in; a real conjunction barrier waits for a real case,
|
|
1051
|
+
because it is the first organ of a workflow engine.
|
|
1052
|
+
- **Coalesce, if a digest case becomes real** — as a delayed run on a
|
|
1053
|
+
`coalesce:<key>:<window>` cause, never as a table.
|
|
1054
|
+
- **Cron-string sugar** in tooling — the stored artifact stays structured.
|
|
1055
|
+
- **Ephemeral fact observers** (live UI invalidation under moss) — a second,
|
|
1056
|
+
best-effort consumer of the host's fact stream; deliberately not tide's.
|
|
1057
|
+
- **Engine-level throttling** — backpressure already exists, and proactive throttling
|
|
1058
|
+
lives in the handler that owns the provider relationship.
|
|
1059
|
+
- **Distributed drive leadership** — unnecessary while idempotent materialization and
|
|
1060
|
+
fenced, leased, exactly-once claims make concurrent advances safe; revisit only if
|
|
1061
|
+
advance *cost* (not correctness) becomes a problem at scale.
|