@niscorp/tide 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +66 -0
- package/DESIGN.md +40 -14
- package/DOCS.md +93 -6
- package/README.md +16 -4
- package/dist/chunk-4CUQXWVY.js +20 -0
- package/dist/chunk-4CUQXWVY.js.map +1 -0
- package/dist/index.cjs +95 -0
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +19 -1
- package/dist/index.d.ts +19 -1
- package/dist/index.js +97 -18
- package/dist/index.js.map +1 -1
- package/dist/postgres.cjs +596 -0
- package/dist/postgres.cjs.map +1 -0
- package/dist/postgres.d.cts +26 -0
- package/dist/postgres.d.ts +26 -0
- package/dist/postgres.js +473 -0
- package/dist/postgres.js.map +1 -0
- package/package.json +20 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,71 @@
|
|
|
1
1
|
# @niscorp/tide
|
|
2
2
|
|
|
3
|
+
## 0.1.6
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 7fb8322: Tide's Postgres store and its driver ship from tide. They were in `@niscorp/moss`.
|
|
8
|
+
|
|
9
|
+
**A new entry point, `@niscorp/tide/postgres`: the store.**
|
|
10
|
+
- `createTideStore(pool, options?)` answers a `TideStore` plus `ready: Promise<void>`, over four tables it migrates itself through a strata ledger. With it come `TIDE_SEQUENCE` (the tables, as a strata sequence, for an app that migrates them in its own run), `TIDE_TABLES`, `TIDE_DDL`, and the types `TideStoreOptions`, `TidePool` and `TidePoolQuery`.
|
|
11
|
+
- The pool is a type of the store's own, `TidePool`: `query(text, values?)`, and `transaction(fn)`. Vex's `PgPool`, a PGlite behind a shim and a `pg` pool that checks a client out for each transaction are each one. Tide imports no driver and no host.
|
|
12
|
+
- `@niscorp/strata` is a new peer, and it is OPTIONAL: only this entry point imports it. An app that uses `@niscorp/tide/postgres` installs strata; an app on the memory store or a store of its own does not.
|
|
13
|
+
- The sequence is the one every database that kept tide's tables under moss has run: its id is still `nisc.moss.tide`, and its migrations, their order and their steps are unchanged. A database migrated by an earlier release finds nothing pending.
|
|
14
|
+
- A row the store reads back is read by a schema, one for each table. A row that does not fit (a state the engine has no name for, a number where text belongs) throws where it is read, naming the table and the field. It used to be handed to the engine as it was.
|
|
15
|
+
- What a `where` compares a field with (`{ lte: 5 }`, `{ in: [...] }`) is read by a schema too. One that does not fit (`in` that is not a list, `isNull` that is not a boolean) throws, naming the field. The types of `Where` already refuse such a call, so nothing written against them changes.
|
|
16
|
+
|
|
17
|
+
**New in the main entry: the driver.**
|
|
18
|
+
- `createTideDriver({ tide, now?, janitorMs?, retention? })` answers a `TideDriver`: `ingest` and `fire` that wake the engine, `wake()`, and `stop()`. It advances to quiescence on every ingest, sleeps until the instant `nextDue` names, and sweeps on a slow beat. The types `TideDriver` and `TideDriverConfig` come with it.
|
|
19
|
+
- The engine still reads no clock and sets no timer. The driver is what does, for a host that does not want to write those twenty lines.
|
|
20
|
+
- **What the driver logs begins `[tide:driver]`.** It began `[moss:tide-driver]` while the driver was moss's. There are four such lines, all to `console.error`: a drain that exceeded its 10,000 steps, a next-due read that failed, a wake that threw, and a sweep that failed. A deployment that searches its logs for the old prefix, or alerts on it, searches for the new one.
|
|
21
|
+
|
|
22
|
+
**Two instances cannot both claim in one serial group.**
|
|
23
|
+
- A reflex with `order: 'serial'` runs one task at a time: the store claims in its group only when no task of it is held. The Postgres store asked whether the group was held and then claimed, as two statements. With two processes on one database, both could ask before either claimed, and each took a task: two tasks of a serial reflex ran at once.
|
|
24
|
+
- The two statements now run in one transaction, behind an advisory lock on the group (`pg_try_advisory_xact_lock`) that ends with the transaction. An instance that does not get the lock leaves the group to the one that has it; nothing waits.
|
|
25
|
+
- Shown on PostgreSQL 18.4 with two pools claiming at once (300 rounds, and 150 rounds of four claims each): exactly one held every round, and two held in 299 of 300 with the lock taken out. No test in the repository holds this: PGlite has one connection, so two claimers never meet there.
|
|
26
|
+
- `createTideStore` over a pool with no `transaction` now refuses a serial claim with the sentence `transact` already refused with. Such a pool could not run tide before: fan-out is a transaction.
|
|
27
|
+
|
|
28
|
+
**A wait longer than a timer can hold no longer makes the driver a loop.**
|
|
29
|
+
- The driver sleeps until the instant `nextDue` names, with one timer. Node keeps a timer's delay in 32 bits: one over 2^31 − 1 ms (24.8 days) is not refused, it is set to 1 ms, with a `TimeoutOverflowWarning`. A reflex whose next occurrence was further off than that had the driver wake every millisecond, advance, find nothing and ask again. Every wait is cut to 2^31 − 1 ms now: the driver wakes then, finds nothing, and waits out the rest.
|
|
30
|
+
- The same for a `janitorMs` set that long, which beat every millisecond.
|
|
31
|
+
|
|
32
|
+
**What to change:** nothing, for an app that did not take these from moss. One that did imports them from here: see the moss changeset.
|
|
33
|
+
|
|
34
|
+
## 0.1.5
|
|
35
|
+
|
|
36
|
+
### Patch Changes
|
|
37
|
+
|
|
38
|
+
- f834f3f: `evaluate` checks a config object once and only evaluates it after that; `prismTransform` is gone. **Breaking.**
|
|
39
|
+
|
|
40
|
+
BREAKING — approved by moccadroid, 2026-10-09: replace `prismTransform` with `evaluate` (`transform: evaluate`), and pass limits as `{ limits: … }`.
|
|
41
|
+
|
|
42
|
+
Prism had three ways to run a config, and the one a host was told to hand to its seam (`prismTransform`) was the one that reported worst, while the hosts on a hot path each wrapped `evaluate`, which checked the config on every call. There are two now, with one job each.
|
|
43
|
+
|
|
44
|
+
**`evaluate(config, source, options?)`** — for a config a program holds.
|
|
45
|
+
- The first time it is handed a config object it checks it and keeps its tree; the second time it prepares the tree as `compile` would; from then on a call only evaluates. Four fields picked and renamed, the same config object again: 3.5 µs before, 0.4 µs now. A nested choice on two fields: 18 µs before, 0.7 µs now. A config that is a new object on every call costs what it did.
|
|
46
|
+
- It is kept **by the object**: a config changed in place after its first call is not read again. Hand over a new object.
|
|
47
|
+
- Both arguments are `unknown`, so it is what a host's transform seam takes: `createTide({ transform: evaluate })`, `createUpgrader(grammars, { transform: evaluate })`.
|
|
48
|
+
- The third argument is `{ limits?, check? }`. Limits were the third argument themselves: `evaluate(config, source, { maxSteps: 100 })` is now `evaluate(config, source, { limits: { maxSteps: 100 } })`.
|
|
49
|
+
- `check: 'always'` is for writing a config, for tests and for tools: nothing is kept, the config is checked on every call, and the source must be plain JSON (a source holding `undefined`, a function, a `Date` or a non-finite number is refused with `E_TYPE`).
|
|
50
|
+
- An invalid config is a `PrismError` (`E_SCHEMA`) naming the part that is wrong, however deep the config. `prismTransform` threw zod's own error, and a `RangeError` for a config nested far past the limit.
|
|
51
|
+
|
|
52
|
+
`evaluateSafe` takes the same arguments.
|
|
53
|
+
|
|
54
|
+
**`compile` and `execute`** — for a config that is stored. Unchanged.
|
|
55
|
+
|
|
56
|
+
**`prismTransform` is removed**, from `@niscorp/prism` and from `@niscorp/prism/migrations`. What it added over `evaluate` was a check that the source is plain JSON; that check is now `check: 'always'`, and a seam's source is taken as it is.
|
|
57
|
+
|
|
58
|
+
**moss, nova, vex, loom, nisc, create-nisc and the cli move with it.** Each has Prism as a peer or a dependency, or has one of those that do, and a Prism at 0.3 is outside the range their last versions accept: an app takes them together. Nothing in vex, loom or the cli changed beyond that range.
|
|
59
|
+
|
|
60
|
+
moss's shell and nova's `$prism` binding called `evaluate` on every use and so paid the check each time: an endpoint's request and response under moss, and a `$prism` value on each draw and for each row of a list, are 8 to 45 times faster for the configs measured. tide's and strata's documentation name `evaluate` where they named `prismTransform`.
|
|
61
|
+
|
|
62
|
+
**What to change**
|
|
63
|
+
- `import { prismTransform } from '@niscorp/prism'` or `'@niscorp/prism/migrations'` → `import { evaluate } from '@niscorp/prism'`, and `transform: prismTransform` → `transform: evaluate`. Every app made by `create-nisc` has this in `src/dev/strata.ts`; the app's typecheck and `npm run strata` both stop on that line until it is changed.
|
|
64
|
+
- A third argument of limits becomes `{ limits: … }`.
|
|
65
|
+
- A wrapper that parsed the config before calling `evaluate` (`evaluate(ConfigSchema.parse(config), …)`) makes a new object on every call and so is checked on every call: hand `evaluate` the config itself.
|
|
66
|
+
- Code that changes a config object in place and evaluates it again must hand over a new object, or pass `{ check: 'always' }`.
|
|
67
|
+
- A host that relied on `prismTransform` refusing a source that is not plain JSON checks it itself, or passes `{ check: 'always' }`.
|
|
68
|
+
|
|
3
69
|
## 0.1.4
|
|
4
70
|
|
|
5
71
|
### Patch Changes
|
package/DESIGN.md
CHANGED
|
@@ -77,7 +77,9 @@ of those properties. Tide exists so that piece never has to be written.
|
|
|
77
77
|
the instant `nextDue(now)` names — a clock occurrence, a retry backoff, a lease
|
|
78
78
|
lapse, a delayed fact. Tide tells the driver when it wants waking; the driver owns
|
|
79
79
|
the timer. A slow janitor beat recovers what a crashed process left behind — it is
|
|
80
|
-
how work is FOUND, never how work moves.
|
|
80
|
+
how work is FOUND, never how work moves. `createTideDriver` is that driver,
|
|
81
|
+
shipped beside the engine for a host that does not write its own: the one piece
|
|
82
|
+
of the package that reads a clock to decide when the engine runs. A dev check is its own driver, handing
|
|
81
83
|
`advance` a fake `now` and asserting on rows.
|
|
82
84
|
|
|
83
85
|
7. **One effect per reflex, and effects are opaque named calls.** The effect
|
|
@@ -739,7 +741,7 @@ the argument that tide is a *nisc* library and not a scheduler with JSON config:
|
|
|
739
741
|
|
|
740
742
|
| Seam | Contract | The host injects | Precedent |
|
|
741
743
|
|---|---|---|---|
|
|
742
|
-
| `store` | `TideStore` — six capabilities over four tables (below) | `createMemoryStore()
|
|
744
|
+
| `store` | `TideStore` — six capabilities over four tables (below) | `createMemoryStore()`, or `createTideStore(pool)` from `@niscorp/tide/postgres` for persistence; or a store of the host's own | vex's `CacheBackend` — structural, driver-free |
|
|
743
745
|
| `transform` | `(config, source) => value` | Prism's `evaluate`, or anything | **nova's exact socket** — nova doesn't know Prism; neither does tide |
|
|
744
746
|
| `select` | `(query, ctx) => AsyncIterable<row>` — the query blob is opaque | vex replay under the actor's policy; or raw SQL; or an array | vex's `generateDsl` / `mapToShape` |
|
|
745
747
|
| `effects` | `name → { run(input, ctx), writes?, preview? }` | vex mutation replays, fns, cortex agents; or plain functions | moss's `functions` seam |
|
|
@@ -750,7 +752,8 @@ in, **`advance({ now, limit })`** to run one committed increment, and
|
|
|
750
752
|
**`nextDue(now)`** so the driver knows when next to wake.
|
|
751
753
|
|
|
752
754
|
Dependencies: **`zod`**. Nothing else — the same bar charter clears. (The
|
|
753
|
-
`/agent` entry imports `@niscorp/cortex`,
|
|
755
|
+
`/agent` entry imports `@niscorp/cortex`, and the `/postgres` entry
|
|
756
|
+
`@niscorp/strata`: optional peers both, and the engine imports neither.)
|
|
754
757
|
|
|
755
758
|
### The store contract
|
|
756
759
|
|
|
@@ -785,8 +788,8 @@ implements.
|
|
|
785
788
|
|
|
786
789
|
`STORE_CONTRACT`, exported from `@niscorp/tide/testing`, is the executable definition
|
|
787
790
|
of everything above: a list of `{ name, run(store) }` checks that throw, with no test
|
|
788
|
-
framework, so any runner can drive them. The memory store runs it;
|
|
789
|
-
runs the same list against a real database.
|
|
791
|
+
framework, so any runner can drive them. The memory store runs it; the Postgres
|
|
792
|
+
store runs the same list against a real database (`test/postgres-store.test.ts`).
|
|
790
793
|
|
|
791
794
|
**⟲ This exists because of the store that was deleted.** `store/postgres.ts` was 700
|
|
792
795
|
lines, had never been instantiated anywhere in this repo, and claimed in its own header
|
|
@@ -798,17 +801,23 @@ the retention horizon; three transactional methods that skipped the ready-guard
|
|
|
798
801
|
raised `undefined_table` *inside* a transaction. Every one was invisible. The lesson is
|
|
799
802
|
not "write more tests" — it is that a contract nobody can execute is a comment.
|
|
800
803
|
|
|
804
|
+
The store at that path today is not that one. It was written in moss against the
|
|
805
|
+
contract, held to `STORE_CONTRACT` and to the engine's own scenarios on a real
|
|
806
|
+
database, and moved here with those tests when it was plain that it is tide's storage
|
|
807
|
+
and not a host's.
|
|
808
|
+
|
|
801
809
|
---
|
|
802
810
|
|
|
803
811
|
## Hosts
|
|
804
812
|
|
|
805
813
|
### Moss — the reference host
|
|
806
814
|
|
|
807
|
-
Moss fills every seam,
|
|
815
|
+
Moss fills every seam but the store, which is tide's own:
|
|
808
816
|
|
|
809
817
|
| Tide seam | Moss fills it with |
|
|
810
818
|
|---|---|
|
|
811
|
-
| `store` | **`createTideStore(pool)`** — four tables beside `vex_cache`, `ON CONFLICT DO NOTHING` for `appendIfAbsent`, `FOR UPDATE SKIP LOCKED` for `claim`, CHECK constraints on every state column, and a foreign key that cascades work with its run |
|
|
819
|
+
| `store` | **`createTideStore(pool)`** from `@niscorp/tide/postgres`, on the pool the app's runtime holds — four tables beside `vex_cache`, `ON CONFLICT DO NOTHING` for `appendIfAbsent`, `FOR UPDATE SKIP LOCKED` for `claim`, CHECK constraints on every state column, and a foreign key that cascades work with its run |
|
|
820
|
+
| facts in | `createTideFacts` (`@niscorp/moss/tide`), a feature of moss: every write vex commits is minted as a `write` fact, stamped with the identity the write's own scope names |
|
|
812
821
|
| `select` | vex replay **through the actor's own wire** — selection inherits replay-only, scoped, cached, inspectable; the tenant filter is in the SQL, engine-injected, unforgeable |
|
|
813
822
|
| `effects` | every seeded vex **mutation** auto-registered under its fingerprint (with `writes` *derived* from `mutationEffect`) + the app's `effects(actor)` |
|
|
814
823
|
| `transform` | prism — the same wiring the shell's transform socket already gets |
|
|
@@ -841,7 +850,7 @@ destructuring it produced a transaction that failed inside the library.
|
|
|
841
850
|
```ts
|
|
842
851
|
const tide = createTide({
|
|
843
852
|
store: createMemoryStore(), // or your own, held to STORE_CONTRACT
|
|
844
|
-
transform:
|
|
853
|
+
transform: evaluate, // prism, or your own
|
|
845
854
|
select: async function* (q) { yield* (await pg.query(q.sql, q.params)).rows; },
|
|
846
855
|
effects: {
|
|
847
856
|
'mail.send': { run: (input) => mailer.send(input) },
|
|
@@ -854,7 +863,7 @@ tide.load(reflexes, { at: Date.now() });
|
|
|
854
863
|
app.post('/webhook/stripe', (req) => tide.ingest(toFact(req))); // validate at the boundary first
|
|
855
864
|
|
|
856
865
|
// The driver: wake on ingest, drain to quiescence, sleep until nextDue.
|
|
857
|
-
// (
|
|
866
|
+
// (`createTideDriver` is these lines, written once, with the sleep and a janitor.)
|
|
858
867
|
const drain = async () => { for (;;) { const r = await tide.advance({ now: Date.now() }); if (isQuiet(r)) break; } };
|
|
859
868
|
```
|
|
860
869
|
|
|
@@ -869,7 +878,8 @@ theirs to keep. `ctx.emit` is how a chain continues without a write choke point.
|
|
|
869
878
|
|
|
870
879
|
```
|
|
871
880
|
src/
|
|
872
|
-
index.ts Public API barrel — createTide, schemas, the store contract
|
|
881
|
+
index.ts Public API barrel — createTide, schemas, the store contract, the driver
|
|
882
|
+
driver.ts createTideDriver — wake on ingest, sleep until nextDue, a janitor
|
|
873
883
|
testing.ts STORE_CONTRACT — the executable definition of a store (the ./testing entry)
|
|
874
884
|
tide.ts createTide: load, ingest, advance, nextDue, fire, retry, preview, ledger, sweep
|
|
875
885
|
types.ts ledger rows, seam types, TideStore, UNIQUE_BY, reports
|
|
@@ -898,6 +908,14 @@ src/
|
|
|
898
908
|
|
|
899
909
|
store/
|
|
900
910
|
memory.ts createMemoryStore — the reference implementation + snapshot()
|
|
911
|
+
postgres/ the Postgres store — the ./postgres entry; the only directory that imports strata
|
|
912
|
+
index.ts the door: createTideStore, TIDE_SEQUENCE, TIDE_TABLES, TIDE_DDL, TidePool
|
|
913
|
+
pool.ts the pool the store runs on, as much of one as it uses
|
|
914
|
+
tables.ts the four tables, and the strata sequence that makes them
|
|
915
|
+
rows.ts a row and an entity, both directions, read by a schema
|
|
916
|
+
compile.ts a where, an order and a set, compiled to SQL
|
|
917
|
+
claim.ts the claim: one statement, and one instance at a time in a serial group
|
|
918
|
+
store.ts createTideStore: the six capabilities over a pool
|
|
901
919
|
|
|
902
920
|
agent/
|
|
903
921
|
reflex-agent.ts createReflexAgent — the ./agent entry; the only file that imports cortex
|
|
@@ -909,7 +927,11 @@ test/
|
|
|
909
927
|
durability.test.ts one test per defect that reached the built engine
|
|
910
928
|
load.test.ts the load gate: validation, cycles, versioning
|
|
911
929
|
preview.test.ts dry run writes nothing and shows everything
|
|
912
|
-
store.test.ts STORE_CONTRACT ×
|
|
930
|
+
store.test.ts STORE_CONTRACT × the memory store
|
|
931
|
+
postgres-store.test.ts STORE_CONTRACT × the Postgres store, in two pool shapes; a serial group and two instances
|
|
932
|
+
postgres-engine.test.ts the engine's scenarios on a real database
|
|
933
|
+
baselines.test.ts the checksum of each migration of TIDE_SEQUENCE, pinned
|
|
934
|
+
driver.test.ts the driver: stopping, and a wait longer than a timer holds
|
|
913
935
|
anchor.test.ts a draft's timer, anchored when it is saved
|
|
914
936
|
agent.test.ts the reflex agent — its answer schemas, effect check and conversation turns
|
|
915
937
|
support.ts shared fixtures
|
|
@@ -920,8 +942,12 @@ test/
|
|
|
920
942
|
orchestrator. There is no `verbs.ts`: `fire` is four lines of sugar over `ingest`,
|
|
921
943
|
and `retry` belongs beside the executor whose invariants it rewinds.
|
|
922
944
|
|
|
923
|
-
|
|
924
|
-
|
|
945
|
+
`store/postgres/` is here, and **tide still imports no host**. Persistence is where
|
|
946
|
+
that rule earns its keep, and the store keeps it: the pool it runs on is a type it
|
|
947
|
+
writes out itself (`TidePool`), the one package it imports is strata, for the ledger
|
|
948
|
+
its tables are migrated through, and it sits on an entry point of its own so that
|
|
949
|
+
nothing the engine's entry reaches imports even that. What stays in moss is what only
|
|
950
|
+
a host can do: turning its own writes into facts.
|
|
925
951
|
|
|
926
952
|
---
|
|
927
953
|
|
|
@@ -1040,7 +1066,7 @@ The wishlist, answered — including its own open questions:
|
|
|
1040
1066
|
| Per-tenant config, instant disarm, versioning across runs | reflexes are tenant rows; disarming is the host's row write + `load` (§ decision 11); content-hash versions (§ decision 14) |
|
|
1041
1067
|
| Host-agnostic wake-up; serverless and local both | `advance({ now })` + `nextDue(now)`: the driver wakes on ingest and sleeps until the next due instant; its janitor beat is recovery, not pacing |
|
|
1042
1068
|
| Deterministic headless checks | memory store + fake clock; no timers exist to wait on |
|
|
1043
|
-
| Durable across restarts | `createTideStore(pool)
|
|
1069
|
+
| Durable across restarts | `createTideStore(pool)`, `@niscorp/tide/postgres`; the memory store remains the reference and the check harness |
|
|
1044
1070
|
| Wishlist: dependency between automations | run facts — `on: { fact: { run: 'A' } }` with `when` on the stats |
|
|
1045
1071
|
| **Open: one library or two?** | one, standalone — "react to a change" is a fact reflex; same machine |
|
|
1046
1072
|
| **Open: does the identity belong here?** | no — moss's `ActorContext`, consumed through an opaque seam |
|
package/DOCS.md
CHANGED
|
@@ -16,13 +16,16 @@ Every symbol the package exports, and what it promises. See [DESIGN.md](./DESIGN
|
|
|
16
16
|
- [Events](#events)
|
|
17
17
|
- [The seams](#the-seams)
|
|
18
18
|
- [The store contract](#the-store-contract)
|
|
19
|
+
- [The Postgres store](#the-postgres-store)
|
|
20
|
+
- [The driver](#the-driver)
|
|
19
21
|
- [Occurrence math](#occurrence-math)
|
|
20
22
|
- [The reflex agent](#the-reflex-agent)
|
|
21
23
|
- [Errors](#errors)
|
|
22
24
|
|
|
23
|
-
|
|
24
|
-
`@niscorp/tide/agent` and `@niscorp/tide/
|
|
25
|
-
|
|
25
|
+
Four entry points: `@niscorp/tide` (everything below unless it says otherwise),
|
|
26
|
+
`@niscorp/tide/agent`, `@niscorp/tide/testing` and `@niscorp/tide/postgres`.
|
|
27
|
+
`zod` is the one required peer. Two peers are optional, each imported by one
|
|
28
|
+
entry only: `@niscorp/cortex` by `/agent`, `@niscorp/strata` by `/postgres`.
|
|
26
29
|
|
|
27
30
|
---
|
|
28
31
|
|
|
@@ -251,7 +254,7 @@ the reflex selects what changed. Mechanisms that already exist, and no table.
|
|
|
251
254
|
A template is a transform-config slot inside a reflex — `effect.input`, `when`
|
|
252
255
|
and `select.query`. Tide stores, diffs and hashes them but never interprets
|
|
253
256
|
them: they go to the `transform` seam verbatim (under moss, Prism —
|
|
254
|
-
`
|
|
257
|
+
`evaluate` from `@niscorp/prism` fits the seam as it is).
|
|
255
258
|
|
|
256
259
|
What tide puts in `$` itself is plain JSON: a fact carries no key for a value
|
|
257
260
|
it was never given, in the memory store as in a database. What the host handed
|
|
@@ -605,8 +608,92 @@ for (const check of STORE_CONTRACT) it(check.name, () => check.run(makeStore()))
|
|
|
605
608
|
It is exported rather than kept in `test/` because the contract is not tide's
|
|
606
609
|
private business. The last SQL store in this package said in its own header that
|
|
607
610
|
it was "held to the same tests" and had none; it diverged in eleven ways and
|
|
608
|
-
every one of them was invisible.
|
|
609
|
-
|
|
611
|
+
every one of them was invisible. Tide's own Postgres store (below) runs this
|
|
612
|
+
same list against a real database, in two pool shapes.
|
|
613
|
+
|
|
614
|
+
---
|
|
615
|
+
|
|
616
|
+
## The Postgres store
|
|
617
|
+
|
|
618
|
+
```typescript
|
|
619
|
+
import { createTideStore, TIDE_SEQUENCE, TIDE_TABLES, TIDE_DDL } from '@niscorp/tide/postgres';
|
|
620
|
+
import type { TidePool, TidePoolQuery, TideStoreOptions } from '@niscorp/tide/postgres';
|
|
621
|
+
```
|
|
622
|
+
|
|
623
|
+
An entry point of its own, so that the engine's entry imports nothing but zod.
|
|
624
|
+
It imports `@niscorp/strata`, an optional peer: install it to use this entry.
|
|
625
|
+
|
|
626
|
+
```typescript
|
|
627
|
+
createTideStore(pool: TidePool, options?: TideStoreOptions): TideStore & { ready: Promise<void> }
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
A `TideStore` over four tables: `tide_fact`, `tide_run`, `tide_work`,
|
|
631
|
+
`tide_reflex_state` (`TIDE_TABLES`). `appendIfAbsent` is `ON CONFLICT DO
|
|
632
|
+
NOTHING` on the key `UNIQUE_BY` names; `claim` is one `UPDATE … FOR UPDATE SKIP
|
|
633
|
+
LOCKED`; the state columns carry CHECK constraints; deleting a run cascades to
|
|
634
|
+
its work.
|
|
635
|
+
|
|
636
|
+
- **`pool`** is a `TidePool`: `{ query(text, values?), transaction?(fn) }`. It
|
|
637
|
+
is a type of the store's own, and structural: vex's `PgPool`, a PGlite behind
|
|
638
|
+
a shim, a `pg` pool that checks a client out for each transaction. `query`
|
|
639
|
+
and `transaction` are called on the object that has them. A pool with no
|
|
640
|
+
`transaction` cannot run tide: fan-out, attempt recording and a serial claim
|
|
641
|
+
are transactions, and each refuses with a sentence.
|
|
642
|
+
- **`ready`** resolves when the tables are there. Every method awaits it. The
|
|
643
|
+
store migrates them itself, through a strata ledger, unless
|
|
644
|
+
`options.migrate` is `false`: then whoever owns the database's migrations has
|
|
645
|
+
run `TIDE_SEQUENCE` in a run of its own (under moss, `runtime.tables`).
|
|
646
|
+
- **`TIDE_SEQUENCE`** is the tables as a strata sequence. Its id is
|
|
647
|
+
`nisc.moss.tide`: the store was in `@niscorp/moss` when the sequence was
|
|
648
|
+
named, and every ledger that has run it holds it under that name.
|
|
649
|
+
`TIDE_DDL` is the text of its first migration. Both are history: the tables
|
|
650
|
+
change by appending a migration, never by editing one.
|
|
651
|
+
- **A row is read by a schema.** What comes back from the database is parsed,
|
|
652
|
+
one schema for each table: a row that does not fit throws where it is read,
|
|
653
|
+
naming the table and the field.
|
|
654
|
+
- **A serial group is claimed by one instance at a time.** Asking whether a
|
|
655
|
+
group is held and claiming in it run in one transaction, behind an advisory
|
|
656
|
+
lock on the group (`pg_try_advisory_xact_lock`). An instance that does not
|
|
657
|
+
get the lock leaves the group to the one that has it.
|
|
658
|
+
|
|
659
|
+
The engine's own writes go through the pool, not through a host's write path:
|
|
660
|
+
an engine claiming its own task has no principal to run a policy for. The
|
|
661
|
+
ledger READS are the host's, as ordinary queries over these tables.
|
|
662
|
+
|
|
663
|
+
---
|
|
664
|
+
|
|
665
|
+
## The driver
|
|
666
|
+
|
|
667
|
+
```typescript
|
|
668
|
+
import { createTideDriver } from '@niscorp/tide';
|
|
669
|
+
import type { TideDriver, TideDriverConfig } from '@niscorp/tide';
|
|
670
|
+
|
|
671
|
+
createTideDriver({ tide, now?, janitorMs?, retention? }): TideDriver
|
|
672
|
+
```
|
|
673
|
+
|
|
674
|
+
The engine reads no clock and paces nothing; this is the thing that does, for
|
|
675
|
+
a host that does not write its own.
|
|
676
|
+
|
|
677
|
+
- **Wake** — `ingest` and `fire` advance the engine immediately, to
|
|
678
|
+
quiescence: a chain advances one committed hop per step, so the driver
|
|
679
|
+
loops until a step reports nothing moved. `wake()` returns the promise of
|
|
680
|
+
that drain, so a "run it now" button can await a settled world.
|
|
681
|
+
- **Sleep** — after quiescence, `tide.nextDue(now)` names the next instant
|
|
682
|
+
worth waking for and one timer sleeps until then, preempted by any ingest.
|
|
683
|
+
No beat, no polling for work. A wait longer than a timer can hold (2^31 − 1
|
|
684
|
+
ms, 24.8 days) is cut to that: the driver wakes, finds nothing, and waits
|
|
685
|
+
out the rest.
|
|
686
|
+
- **Janitor** — a slow fallback wake plus the retention sweep. It finds
|
|
687
|
+
nothing when nothing is broken; it exists because a durable multi-worker
|
|
688
|
+
design needs the scan that notices work committed by a process that died
|
|
689
|
+
before its own wake ran. It is how work is *recovered*, never how it moves.
|
|
690
|
+
|
|
691
|
+
A `TideDriver` is `{ ingest, fire, wake, stop }`. Hand a host's intake the
|
|
692
|
+
driver, not the bare engine, so a fact wakes it instead of waiting for
|
|
693
|
+
somebody's beat. `stop()` resolves once nothing the driver started still
|
|
694
|
+
touches the store: await it before closing the database. `now` replaces the
|
|
695
|
+
clock (`Date.now`); `janitorMs` defaults to 5 minutes; absent `retention`,
|
|
696
|
+
nothing is swept. What it logs, it logs as `[tide:driver]`.
|
|
610
697
|
|
|
611
698
|
---
|
|
612
699
|
|
package/README.md
CHANGED
|
@@ -18,11 +18,11 @@ pnpm add @niscorp/prism @niscorp/strata
|
|
|
18
18
|
|
|
19
19
|
```typescript
|
|
20
20
|
import { createTide, createMemoryStore } from '@niscorp/tide';
|
|
21
|
-
import {
|
|
21
|
+
import { evaluate } from '@niscorp/prism';
|
|
22
22
|
|
|
23
23
|
const tide = createTide({
|
|
24
24
|
store: createMemoryStore(),
|
|
25
|
-
transform:
|
|
25
|
+
transform: evaluate,
|
|
26
26
|
select: async (query) => db.rows(query),
|
|
27
27
|
effects: {
|
|
28
28
|
'mail.send': {
|
|
@@ -48,7 +48,8 @@ await tide.load([
|
|
|
48
48
|
|
|
49
49
|
// The HOST owns waking. Tide reads no clock — it advances one committed step
|
|
50
50
|
// at a time and says when it next wants waking, and a driver does the rest:
|
|
51
|
-
// drain on every ingest, then sleep until that instant.
|
|
51
|
+
// drain on every ingest, then sleep until that instant. `createTideDriver`
|
|
52
|
+
// is that driver, written once; these are the lines it is made of.
|
|
52
53
|
const drain = async () => { for (;;) { const r = await tide.advance({ now: Date.now() }); if (quiet(r)) break; } };
|
|
53
54
|
await drain();
|
|
54
55
|
const at = await tide.nextDue(Date.now()); // undefined = nothing scheduled
|
|
@@ -104,6 +105,17 @@ Four tables, and only two of them grow with events:
|
|
|
104
105
|
|
|
105
106
|
The port is six capabilities, not twenty-seven nouns: `transact`, `appendIfAbsent`, `claim`, `cas`, `query`, `remove`. `createMemoryStore()` is the reference implementation. `STORE_CONTRACT`, exported from `@niscorp/tide/testing`, is the same set of checks both it and any other store must pass — importable so a host's store is held to one definition rather than to two readings of a comment.
|
|
106
107
|
|
|
108
|
+
**Durable: `@niscorp/tide/postgres`.** `createTideStore(pool)` is the same contract over Postgres, held to the same checks against a real database. It takes a pool described by a type of its own (`query`, `transaction`), so it imports no driver and no host, and it migrates its four tables through a [strata](../strata/README.md) ledger (`TIDE_SEQUENCE`). Strata is an optional peer, needed by this entry only.
|
|
109
|
+
|
|
110
|
+
```typescript
|
|
111
|
+
import { createTide, createTideDriver } from '@niscorp/tide';
|
|
112
|
+
import { createTideStore } from '@niscorp/tide/postgres';
|
|
113
|
+
|
|
114
|
+
const tide = createTide({ store: createTideStore(pool), transform, effects });
|
|
115
|
+
const driver = createTideDriver({ tide }); // wakes on ingest, sleeps until nextDue
|
|
116
|
+
await driver.ingest({ kind: 'signal', name: 'stripe', at: Date.now() });
|
|
117
|
+
```
|
|
118
|
+
|
|
107
119
|
## The reflex agent — `@niscorp/tide/agent`
|
|
108
120
|
|
|
109
121
|
A model can WRITE a reflex; it never runs one. `createReflexAgent({ effects, triggers, fields })` is a cortex agent handed what somebody wants automated, the local `now` and timezone, and what the host offers:
|
|
@@ -132,7 +144,7 @@ if (result.ok && isDraft(result.output.data)) {
|
|
|
132
144
|
|
|
133
145
|
## Hosts
|
|
134
146
|
|
|
135
|
-
Tide imports no host. Under **moss** the seams are filled with vex (selection under the actor's scope policy, mutations auto-registered as effects), prism (transform), cortex agents (effects)
|
|
147
|
+
Tide imports no host. Under **moss** the seams are filled with vex (selection under the actor's scope policy, mutations auto-registered as effects), prism (transform), cortex agents (effects) and moss's `ActorContext` (identity); persistence is tide's own `createTideStore(pool)` on the pool the app's runtime holds, and every committed write reaches tide as a fact through moss's `createTideFacts` feature (`@niscorp/moss/tide`). In plain Node the seams are filled with raw SQL and plain functions. Nothing above the seam can tell the difference.
|
|
136
148
|
|
|
137
149
|
## Documentation
|
|
138
150
|
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
// src/types.ts
|
|
2
|
+
var PRIMARY_KEY = {
|
|
3
|
+
fact: "id",
|
|
4
|
+
run: "id",
|
|
5
|
+
task: "id",
|
|
6
|
+
state: "reflexId"
|
|
7
|
+
};
|
|
8
|
+
var UNIQUE_BY = {
|
|
9
|
+
// `entity` is part of the key. Without it two polls over different tables
|
|
10
|
+
// that happened to agree on a cursor value silently ate each other's rows.
|
|
11
|
+
fact: { by: ["kind", "entity", "name", "dedupeKey"], when: "dedupeKey" },
|
|
12
|
+
run: { by: ["reflexId", "cause"] },
|
|
13
|
+
task: { by: ["runId", "unit"] },
|
|
14
|
+
state: { by: ["reflexId"] }
|
|
15
|
+
};
|
|
16
|
+
var COMPARISON_OPS = ["eq", "ne", "lt", "lte", "gt", "gte", "in", "notIn", "isNull"];
|
|
17
|
+
|
|
18
|
+
export { COMPARISON_OPS, PRIMARY_KEY, UNIQUE_BY };
|
|
19
|
+
//# sourceMappingURL=chunk-4CUQXWVY.js.map
|
|
20
|
+
//# sourceMappingURL=chunk-4CUQXWVY.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"sources":["../src/types.ts"],"names":[],"mappings":";AAuWO,IAAM,WAAA,GAAkE;AAAA,EAC7E,IAAA,EAAM,IAAA;AAAA,EACN,GAAA,EAAK,IAAA;AAAA,EACL,IAAA,EAAM,IAAA;AAAA,EACN,KAAA,EAAO;AACT;AAaO,IAAM,SAAA,GAAgD;AAAA;AAAA;AAAA,EAG3D,IAAA,EAAM,EAAE,EAAA,EAAI,CAAC,MAAA,EAAQ,UAAU,MAAA,EAAQ,WAAW,CAAA,EAAG,IAAA,EAAM,WAAA,EAAY;AAAA,EACvE,KAAK,EAAE,EAAA,EAAI,CAAC,UAAA,EAAY,OAAO,CAAA,EAAE;AAAA,EACjC,MAAM,EAAE,EAAA,EAAI,CAAC,OAAA,EAAS,MAAM,CAAA,EAAE;AAAA,EAC9B,KAAA,EAAO,EAAE,EAAA,EAAI,CAAC,UAAU,CAAA;AAC1B;AAcO,IAAM,cAAA,GAAiB,CAAC,IAAA,EAAM,IAAA,EAAM,IAAA,EAAM,OAAO,IAAA,EAAM,KAAA,EAAO,IAAA,EAAM,OAAA,EAAS,QAAQ","file":"chunk-4CUQXWVY.js","sourcesContent":["import type { FactInput, Reflex, ReflexInput } from './schemas';\n\nexport type Row = Record<string, unknown>;\n\n// ═══════════════════════════════════════════════════════════════\n// The ledger — four tables, and only two of them grow with events\n//\n// There were seven: fact, delivery, firing, task, attempt, window,\n// watermark. Six of those were vocabulary. A design noun is not a\n// reason for a table; a GUARANTEE is, and each of these carries one:\n//\n// fact durable before anything interprets it, so `ingest` is one\n// write and matching is retryable in the advance\n// run UNIQUE(reflexId, cause) — THE idempotency, one constraint\n// for every trigger kind; atomic fan-out; fan-in counting\n// task UNIQUE(runId, unit) written BEFORE the effect; the lease;\n// the retry counter\n// state where each reflex has got to. One row per reflex, bounded\n// by how many reflexes exist rather than by what happened\n//\n// What went, and where it went:\n// delivery → the run's `cause`; a non-match is an event, not a row\n// attempt → `attempt` and `error` on the task; per-attempt history\n// is the host's log, and nothing read it here\n// window → deleted with coalesce, which nothing used\n// watermark → the clock through-line and the arming baseline are two\n// fields on `state`, not an unbounded key/value table with\n// a growing set of string keys\n// ═══════════════════════════════════════════════════════════════\n\nexport type Fact = FactInput & {\n id: string;\n // WHOSE FACT THIS IS — the `as` of the reflex that minted it, and absent on\n // anything the host ingested itself.\n //\n // Stamped by the engine, never by a handler, for the same reason `depth` is:\n // a handler that could choose its own identity could choose somebody else's.\n // Tide does not know what the string means. It only knows that a fact minted\n // under one identity must not wake a reflex running under another — which is\n // what stops a row selected for one tenant being handed to another tenant's\n // effect. See `reflexMatchesFact`.\n as?: string;\n // Distance from the root stimulus. An effect's emit inherits its\n // task's depth + 1, which is what lets a divergent loop hit a\n // ceiling instead of melting the ledger.\n depth: number;\n // A fact is done once it has been offered to every LOADED reflex —\n // matching is per (fact, reflex), and one fact can wake five.\n deliveredAt?: number;\n parked?: string;\n // A human looked at a parked fact and said \"run it anyway\". Without\n // this the release is a ping-pong: the matcher re-parks on the next\n // tick, because the depth that parked it has not changed and never\n // will. The override is recorded rather than faked by rewriting depth,\n // which would make the ledger lie about causality.\n released?: boolean;\n // THE CHAIN THIS FACT BELONGS TO — the id of the fact that started it,\n // absent on a root (a fact the host ingested, or the first facts of a run\n // nothing caused). Stamped at the door (engine/facts.ts), never by a handler.\n root?: string;\n // On a root only: how many facts its chain has minted beneath it. Depth\n // caps how LONG a chain runs; this caps how WIDE — a reflex that writes two\n // facts per fact it sees doubles each hop, and under a depth ceiling of 24\n // that is 2^24 runs before anything parks.\n descendants?: number;\n};\n\nexport type RunState = 'pending' | 'fanned' | 'settled' | 'skipped';\n\n// One execution of one reflex. Named `run` rather than `firing` because\n// `firing` was also a trigger kind and a fact kind — one word doing three\n// jobs, and the reason `fact: { firing: … }` read as a puzzle.\nexport type Run = {\n id: string;\n reflexId: string;\n version: string;\n // The host identity this ran under, copied from the reflex. Opaque to\n // tide and written down so the ledger is scopeable by an ordinary\n // host-side rule instead of by parsing reflex ids.\n as?: string;\n // 'occurrence:<key>' | 'fact:<id>' | 'run:<id>' | 'manual:<who>' —\n // provenance, and half of the task's idempotency key.\n cause: string;\n occurrence?: string;\n factIds?: readonly string[];\n state: RunState;\n depth: number;\n selected?: number;\n total: number;\n done: number;\n failed: number;\n dueAt: number;\n createdAt: number;\n settledAt?: number;\n // Whether this run's settlement has already been announced as a fact.\n // Separate from `settledAt` because a reopened task RE-settles its run\n // and must not mint a second fan-in fact: the digest watching it already\n // went out. The flag is what lets the recovery verb rewind safely.\n drained?: boolean;\n note?: string;\n};\n\n// How one attempt ended. There is no attempt TABLE — the count and the last\n// error live on the task, and per-attempt history is the host's log, which\n// is where it was already being read from. The distinction still matters to\n// the executor, so it still has a name.\nexport type AttemptOutcome = 'ok' | 'error' | 'timeout';\n\nexport type TaskState = 'pending' | 'claimed' | 'retrying' | 'done' | 'failed';\n\nexport type Task = {\n id: string;\n runId: string;\n reflexId: string;\n // The unit's grain: a row key in `each`, '' for batch and unit-less runs.\n unit: string;\n cause: string;\n // The pinned environment slice. Frozen at fan-out so a retry evaluates\n // against the same data the selection saw, never against moved data.\n env: Row;\n // Stamped from the run at fan-out, not read back from it at execution\n // time. A run swept out from under a long-running chain used to reset\n // the ceiling to 0 — defeating the backstop in exactly the long-running\n // case it exists for. The task carries what the task needs.\n depth: number;\n state: TaskState;\n attempt: number;\n token?: string;\n // THE LEASE, and expiry IS the reclaim. Without one, a process that died\n // between the effect and the record left the task `claimed` forever: its\n // run never settled, never drained, and an `overlap: 'skip'` reflex was\n // blocked for good. No reaper, no heartbeat — a claim whose lease has\n // lapsed is simply claimable again, which the fencing token makes safe.\n //\n // ZERO, not null, when unclaimed. \"Never claimed or lapsed\" is then one\n // comparison rather than a disjunction, which keeps the claim expressible\n // in a query grammar that has no OR — and a grammar with no OR is a\n // grammar every store can implement.\n claimedUntil: number;\n notBefore: number;\n input?: unknown;\n output?: unknown;\n error?: string;\n createdAt: number;\n settledAt?: number;\n};\n\n// Where a reflex has got to. ONE ROW PER REFLEX — bounded by how many\n// reflexes exist, never by how much has happened, which is what separates\n// it from the ledger proper and is why it survived the collapse.\nexport type ReflexState = {\n reflexId: string;\n // Matching starts here. A reflex never retro-fires: history from before\n // it existed is reachable by deliberate backfill, never by accident.\n armedAt: number;\n // The clock's through-line. Derivable from MAX(cause) over the runs, but\n // only while those runs exist — and retention is allowed to delete them.\n materializedThrough?: number;\n};\n\n// ═══════════════════════════════════════════════════════════════\n// The seams — five injection points\n// ═══════════════════════════════════════════════════════════════\n\n// What an effect handler and the select seam receive. `actor` is opaque:\n// tide never learns what a principal is (under moss it is an ActorContext,\n// charter-resolved and engine-enforced).\nexport type TideCtx = {\n reflexId: string;\n runId: string;\n taskId: string;\n // The downstream idempotency key. A payment capture passes this to the\n // provider, which is what makes a fenced-out zombie attempt harmless.\n taskKey: string;\n attempt: number;\n // How deep in a cause chain this task sits. A handler whose side effects\n // re-enter as facts (a write through the host's own DAL, minted back by\n // the bridge) forwards this with the write, so the chain ceiling survives\n // the trip through the database instead of resetting at every hop.\n depth: number;\n now: number;\n actor: unknown;\n // Continue a chain from inside a handler. Buffered — the fact commits\n // with a SUCCESSFUL attempt, so a retry cannot double-mint it — and\n // narrowed to `write` and `signal`, because a handler that could emit\n // `manual` would reach past the arming switch.\n emit: (fact: Omit<FactInput, 'cause'>) => void;\n};\n\nexport type EffectHandler = {\n run: (input: unknown, ctx: TideCtx) => Promise<unknown> | unknown;\n // What this effect writes. Feeds the flow graph and its cycle refusal —\n // an edge is a write meeting a watch. Under moss it is DERIVED from the\n // vex mutation behind the effect, never declared; a handler with no\n // `writes` is a BLIND edge the load report names.\n writes?: readonly string[];\n // Rendered by `preview` in place of running. Nothing else is needed to\n // make a reflex previewable — the executor is the only door out.\n preview?: (input: unknown, ctx: PreviewCtx) => unknown;\n};\n\nexport type PreviewCtx = { reflexId: string; unit: string; now: number; actor: unknown };\n\nexport type EffectRegistry = Record<string, EffectHandler>;\n\nexport type TransformFn = (config: unknown, source: Row) => unknown;\n\nexport type SelectFn = (query: unknown, ctx: SelectCtx) => AsyncIterable<Row> | Iterable<Row> | Promise<Iterable<Row>>;\n\nexport type SelectCtx = { reflexId: string; now: number; actor: unknown; env: Row };\n\nexport type ActorFn = (as: string | undefined) => unknown;\n\n// ═══════════════════════════════════════════════════════════════\n// Config and reports\n// ═══════════════════════════════════════════════════════════════\n\nexport type Retention = Partial<Record<'facts' | 'runs' | 'tasks', number>>;\n\nexport type TideConfig = {\n store: TideStore;\n transform: TransformFn;\n select?: SelectFn;\n effects?: EffectRegistry | ((as: string | undefined) => EffectRegistry);\n actor?: ActorFn;\n // A fact whose cause chain runs deeper than this is PARKED for review\n // rather than fired — the runtime backstop behind the load-time cycle\n // rules, and nearly free because causality is already recorded.\n maxChainDepth?: number;\n // A chain that has minted more facts than this beneath its root has the\n // rest PARKED — the breadth twin of maxChainDepth. Default 10,000.\n maxChainFacts?: number;\n // KEEP A WRITE FACT NOTHING WATCHES? Default true, which is what the ledger\n // means today: `ledger.facts()` is every write the host committed, and\n // `causeChain` walks it — an audit trail that answers \"why did this person\n // get this\" without anybody having predicted the question.\n //\n // A host that mints a fact per committed ROW pays for that trail on the hot\n // path of every write: an INSERT here, awaited, for a fact no loaded reflex\n // could ever match. A host whose ledger is not its audit log sets this false\n // and pays only for the writes something is actually listening to.\n //\n // NOT retroactive and not clever: it drops nothing while no reflexes are\n // loaded (a fact arriving in that window still waits, exactly as matchFacts\n // has always let it), and it ignores enablement, arming and identity — the\n // question it asks is only \"could any loaded reflex ever watch this entity\n // and op\", which is strictly weaker than the matcher.\n storeUnwatchedWrites?: boolean;\n // How many units one fan-out may mint before the run is refused.\n maxFanOut?: number;\n // How long a claim is good for. A task still `claimed` past this is\n // claimable again — the only recovery a died-mid-effect process needs.\n leaseMs?: number;\n onEvent?: (event: TideEvent) => void;\n};\n\nexport type TideEvent =\n | { type: 'run.created'; run: Run }\n | { type: 'run.settled'; run: Run }\n | { type: 'run.skipped'; reflexId: string; reason: string }\n // A fan-out that will be tried again next tick. Not a skip: a transient\n // selection failure must never consume an occurrence.\n | { type: 'run.deferred'; reflexId: string; runId: string; reason: string }\n | { type: 'task.done'; task: Task }\n | { type: 'task.failed'; task: Task }\n | { type: 'task.retrying'; task: Task; nextAt: number }\n // A lease lapsed and the work was taken back. Rare and worth seeing:\n // it means something died holding a task.\n | { type: 'task.reclaimed'; task: Task }\n | { type: 'fact.ingested'; fact: Fact }\n // A fact refused by its dedupeKey. Not a row — a provider retrying a\n // webhook forever would grow the table without bound, which is what the\n // key exists to stop — but not nothing either: without it, a repeat and\n // a chain that silently stopped read the same.\n | { type: 'fact.deduped'; fact: NewFact }\n | { type: 'fact.parked'; fact: Fact; reason: string }\n // The `when` said no, or threw. Not a row — one per (fact, reflex) pair\n // is a table that grows faster than the work does — but not nothing\n // either, because \"why didn't it fire\" deserves an answer.\n | { type: 'fact.unmatched'; fact: Fact; reflexId: string; reason: string };\n\nexport type AdvanceReport = {\n now: number;\n materialized: number;\n skippedOccurrences: number;\n factsMatched: number;\n runsCreated: number;\n tasksCreated: number;\n executed: number;\n succeeded: number;\n failed: number;\n retrying: number;\n reclaimed: number;\n runsSettled: number;\n parked: number;\n};\n\nexport type LoadReport = {\n loaded: number;\n // Guarded cycles are legal — a drip campaign IS a cycle. They are\n // reported so the author sees the loop they built.\n cycles: readonly { reflexIds: readonly string[]; guarded: boolean }[];\n blind: readonly { reflexId: string; effect: string }[];\n warnings: readonly string[];\n};\n\nexport type PreviewUnit = {\n unit: string;\n env: Row;\n input?: unknown;\n render?: unknown;\n error?: string;\n};\n\nexport type PreviewReport = {\n reflexId: string;\n version: string;\n fired: boolean;\n reason?: string;\n cause?: string;\n occurrence?: string;\n effect: string;\n selected: number;\n units: readonly PreviewUnit[];\n};\n\nexport type NewFact = FactInput & { depth: number; as?: string };\n\n// ═══════════════════════════════════════════════════════════════\n// The store contract — six capabilities, not twenty-seven nouns\n//\n// The old port had a method per noun per verb, so `claimTasks`,\n// `drainSettled` and `claimClosedWindows` were three implementations\n// of ONE operation and diverged three ways. Shaped by what the\n// guarantees need instead:\n//\n// transact several writes land together or not at all\n// appendIfAbsent a row exists once, decided by the table's unique\n// key rather than by a read the caller races\n// claim take work exactly once, under a fence\n// cas move a row only from the state you expected\n// query read\n// remove retention; the only thing that shrinks the ledger\n//\n// Structural and driver-free, the way vex's CacheBackend is. Nothing\n// here knows about retry, arming, coalescing or occurrences — the\n// engine holds every opinion, and the store holds none.\n// ═══════════════════════════════════════════════════════════════\n\nexport type TideTables = {\n fact: Fact;\n run: Run;\n task: Task;\n state: ReflexState;\n};\n\nexport type TableName = keyof TideTables;\n\n// The row identity `cas` and `remove` address a single row by.\nexport const PRIMARY_KEY: { [T in TableName]: keyof TideTables[T] & string } = {\n fact: 'id',\n run: 'id',\n task: 'id',\n state: 'reflexId',\n};\n\nexport type UniqueKey<T extends TableName> = {\n by: readonly (keyof TideTables[T] & string)[];\n // The constraint applies only when this column is present — in Postgres,\n // a partial unique index. A fact with no `dedupeKey` is not claiming to\n // be unique and is always appended.\n when?: keyof TideTables[T] & string;\n};\n\n// WHERE THE IDEMPOTENCY LIVES, written down once so every store agrees.\n// These four constraints are the engine's exactly-once promises; they are\n// not an optimisation and a store that omits one is not a tide store.\nexport const UNIQUE_BY: { [T in TableName]: UniqueKey<T> } = {\n // `entity` is part of the key. Without it two polls over different tables\n // that happened to agree on a cursor value silently ate each other's rows.\n fact: { by: ['kind', 'entity', 'name', 'dedupeKey'], when: 'dedupeKey' },\n run: { by: ['reflexId', 'cause'] },\n task: { by: ['runId', 'unit'] },\n state: { by: ['reflexId'] },\n};\n\nexport type Comparison<V> = {\n eq?: V;\n ne?: V;\n lt?: V;\n lte?: V;\n gt?: V;\n gte?: V;\n in?: readonly V[];\n notIn?: readonly V[];\n isNull?: boolean;\n};\n\nexport const COMPARISON_OPS = ['eq', 'ne', 'lt', 'lte', 'gt', 'gte', 'in', 'notIn', 'isNull'] as const;\n\nexport type Where<R> = { [K in keyof R]?: R[K] | Comparison<R[K]> };\n\nexport type Order<R> = readonly { by: keyof R & string; dir?: 'asc' | 'desc' }[];\n\nexport type QuerySpec<T extends TableName = TableName> = {\n table: T;\n where?: Where<TideTables[T]>;\n order?: Order<TideTables[T]>;\n limit?: number;\n};\n\n// A patch, plus the one thing a caller cannot compute without a read it\n// would then race: an increment. `attempt` has to rise AT CLAIM TIME, or a\n// handler that reliably kills the process retries forever.\nexport type Mutation<R> = { [K in keyof R]?: R[K] | { inc: number } };\n\nexport type ClaimSpec<T extends TableName = TableName> = {\n table: T;\n where?: Where<TideTables[T]>;\n order?: Order<TideTables[T]>;\n limit: number;\n // Applied to every claimed row, atomically with the selection.\n set: Mutation<TideTables[T]>;\n // `order: 'serial'`. At most one row per distinct value of `column` may\n // be held at once, counting rows some earlier claim is already holding —\n // so it lives INSIDE the claim rather than as a filter the caller applies\n // first and loses the race on.\n //\n // `only` names exactly the values this applies to, and it is REQUIRED\n // rather than defaulting to \"everything\": a store implementing \"all\n // groups\" has to enumerate the groups first, and an omitted list that\n // silently means \"all of them\" is the kind of default two implementations\n // read two ways. Everything outside it claims freely. Omit `onePer`\n // entirely when nothing is serial.\n onePer?: { column: keyof TideTables[T] & string; held: Where<TideTables[T]>; only: readonly string[] };\n};\n\nexport type RemoveSpec<T extends TableName = TableName> = {\n table: T;\n where?: Where<TideTables[T]>;\n};\n\nexport type TideStore = {\n // Several writes land together or not at all. The engine's two atomic\n // moments — fan-out committing with its run's transition, and an attempt\n // recording with the facts it emitted — are transactions, not conventions.\n transact: <T>(fn: (tx: TideStore) => Promise<T>) => Promise<T>;\n\n // Idempotent on the table's unique key. A duplicate materialization, a\n // replayed webhook, or a second instance's tick collides and gets\n // `undefined` — a refusal, not an error. `id` is minted by the store.\n appendIfAbsent: <T extends TableName>(\n table: T,\n row: Omit<TideTables[T], 'id'> & { id?: string },\n ) => Promise<TideTables[T] | undefined>;\n\n // Take rows exactly once. Selection and mutation are one step, which is\n // the whole promise: two instances ticking at the same moment contend\n // rather than duplicate.\n claim: <T extends TableName>(spec: ClaimSpec<T>) => Promise<readonly TideTables[T][]>;\n\n // Move a row only from the state you expected. The fence: a timed-out\n // attempt finishing late finds its token superseded and is discarded\n // rather than overwriting the live one.\n cas: <T extends TableName>(\n table: T,\n id: string,\n expect: Where<TideTables[T]>,\n set: Mutation<TideTables[T]>,\n ) => Promise<boolean>;\n\n query: <T extends TableName>(spec: QuerySpec<T>) => Promise<readonly TideTables[T][]>;\n\n // Retention, and the only thing that shrinks the ledger. Deleting a run\n // cascades to its tasks: the run and its members are one fact about the\n // world, and half of it is worse than neither.\n remove: <T extends TableName>(spec: RemoveSpec<T>) => Promise<number>;\n};\n\nexport type { Reflex, ReflexInput, FactInput };\n"]}
|
package/dist/index.cjs
CHANGED
|
@@ -1510,6 +1510,100 @@ var createMemoryStore = () => {
|
|
|
1510
1510
|
return store;
|
|
1511
1511
|
};
|
|
1512
1512
|
|
|
1513
|
+
// src/driver.ts
|
|
1514
|
+
var DEFAULT_JANITOR_MS = 3e5;
|
|
1515
|
+
var MAX_STEPS_PER_DRAIN = 1e4;
|
|
1516
|
+
var LONGEST_TIMER_MS = 2 ** 31 - 1;
|
|
1517
|
+
var waitable = (ms) => Math.min(ms, LONGEST_TIMER_MS);
|
|
1518
|
+
var letProcessEnd = (timer) => {
|
|
1519
|
+
if (typeof timer !== "object" || timer === null || !("unref" in timer)) return;
|
|
1520
|
+
const unref = timer.unref;
|
|
1521
|
+
if (typeof unref === "function") Reflect.apply(unref, timer, []);
|
|
1522
|
+
};
|
|
1523
|
+
var progressOf = (report) => report.materialized + report.skippedOccurrences + report.factsMatched + report.runsCreated + report.tasksCreated + report.executed + report.reclaimed + report.runsSettled + report.parked;
|
|
1524
|
+
var createTideDriver = (config) => {
|
|
1525
|
+
const { tide } = config;
|
|
1526
|
+
const now = config.now ?? (() => Date.now());
|
|
1527
|
+
const janitorMs = waitable(config.janitorMs ?? DEFAULT_JANITOR_MS);
|
|
1528
|
+
let running = false;
|
|
1529
|
+
let queued = false;
|
|
1530
|
+
let stopped = false;
|
|
1531
|
+
let cycle;
|
|
1532
|
+
let scheduling;
|
|
1533
|
+
let timer;
|
|
1534
|
+
const drain = async () => {
|
|
1535
|
+
for (let steps = 0; steps < MAX_STEPS_PER_DRAIN; steps += 1) {
|
|
1536
|
+
const report = await tide.advance({ now: now() });
|
|
1537
|
+
if (progressOf(report) === 0) return;
|
|
1538
|
+
}
|
|
1539
|
+
console.error(`[tide:driver] a drain exceeded ${MAX_STEPS_PER_DRAIN} steps \u2014 stopping this pass; the janitor will resume it`);
|
|
1540
|
+
};
|
|
1541
|
+
const schedule = async () => {
|
|
1542
|
+
if (stopped) return;
|
|
1543
|
+
let due;
|
|
1544
|
+
try {
|
|
1545
|
+
due = await tide.nextDue(now());
|
|
1546
|
+
} catch (err) {
|
|
1547
|
+
if (!stopped) console.error("[tide:driver] next due", err);
|
|
1548
|
+
return;
|
|
1549
|
+
}
|
|
1550
|
+
if (stopped) return;
|
|
1551
|
+
if (timer !== void 0) clearTimeout(timer);
|
|
1552
|
+
if (due === void 0) return;
|
|
1553
|
+
const delay = due - now();
|
|
1554
|
+
timer = setTimeout(() => void wake(), delay > 0 ? waitable(delay) : janitorMs);
|
|
1555
|
+
letProcessEnd(timer);
|
|
1556
|
+
};
|
|
1557
|
+
const wake = () => {
|
|
1558
|
+
if (stopped) return Promise.resolve();
|
|
1559
|
+
queued = true;
|
|
1560
|
+
if (running) return cycle ?? Promise.resolve();
|
|
1561
|
+
running = true;
|
|
1562
|
+
cycle = (async () => {
|
|
1563
|
+
try {
|
|
1564
|
+
while (queued) {
|
|
1565
|
+
queued = false;
|
|
1566
|
+
await drain();
|
|
1567
|
+
}
|
|
1568
|
+
} catch (err) {
|
|
1569
|
+
console.error("[tide:driver]", err);
|
|
1570
|
+
} finally {
|
|
1571
|
+
running = false;
|
|
1572
|
+
cycle = void 0;
|
|
1573
|
+
scheduling = schedule();
|
|
1574
|
+
}
|
|
1575
|
+
})();
|
|
1576
|
+
return cycle;
|
|
1577
|
+
};
|
|
1578
|
+
const janitor = setInterval(() => {
|
|
1579
|
+
if (config.retention !== void 0) {
|
|
1580
|
+
void tide.sweep(now(), config.retention).catch((err) => console.error("[tide:driver] sweep", err));
|
|
1581
|
+
}
|
|
1582
|
+
void wake();
|
|
1583
|
+
}, janitorMs);
|
|
1584
|
+
letProcessEnd(janitor);
|
|
1585
|
+
return {
|
|
1586
|
+
ingest: async (fact, options) => {
|
|
1587
|
+
const stored = await tide.ingest(fact, options);
|
|
1588
|
+
if (stored !== void 0) void wake();
|
|
1589
|
+
return stored;
|
|
1590
|
+
},
|
|
1591
|
+
fire: async (reflexId, options) => {
|
|
1592
|
+
const fact = await tide.fire(reflexId, options);
|
|
1593
|
+
void wake();
|
|
1594
|
+
return fact;
|
|
1595
|
+
},
|
|
1596
|
+
wake,
|
|
1597
|
+
stop: async () => {
|
|
1598
|
+
stopped = true;
|
|
1599
|
+
clearInterval(janitor);
|
|
1600
|
+
if (timer !== void 0) clearTimeout(timer);
|
|
1601
|
+
await cycle;
|
|
1602
|
+
await scheduling;
|
|
1603
|
+
}
|
|
1604
|
+
};
|
|
1605
|
+
};
|
|
1606
|
+
|
|
1513
1607
|
// src/engine/anchor.ts
|
|
1514
1608
|
var timerMs = (timer) => (((timer.hours ?? 0) * 60 + (timer.minutes ?? 0)) * 60 + (timer.seconds ?? 0)) * 1e3;
|
|
1515
1609
|
var pad22 = (value) => String(value).padStart(2, "0");
|
|
@@ -1552,6 +1646,7 @@ exports.WriteTriggerSchema = WriteTriggerSchema;
|
|
|
1552
1646
|
exports.anchorDraft = anchorDraft;
|
|
1553
1647
|
exports.createMemoryStore = createMemoryStore;
|
|
1554
1648
|
exports.createTide = createTide;
|
|
1649
|
+
exports.createTideDriver = createTideDriver;
|
|
1555
1650
|
exports.daysInMonth = daysInMonth;
|
|
1556
1651
|
exports.draftSchemaOf = draftSchemaOf;
|
|
1557
1652
|
exports.isTideError = isTideError;
|