@earendil-works/chord 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +77 -194
- package/dist/cell.d.ts +101 -0
- package/dist/cell.d.ts.map +1 -0
- package/dist/cell.js +316 -0
- package/dist/cell.js.map +1 -0
- package/dist/cluster.d.ts +26 -0
- package/dist/cluster.d.ts.map +1 -0
- package/dist/cluster.js +214 -0
- package/dist/cluster.js.map +1 -0
- package/dist/connection.d.ts +157 -0
- package/dist/connection.d.ts.map +1 -0
- package/dist/connection.js +757 -0
- package/dist/connection.js.map +1 -0
- package/dist/context/index.d.ts +12 -1
- package/dist/context/index.d.ts.map +1 -1
- package/dist/context/index.js.map +1 -1
- package/dist/contract.d.ts +133 -0
- package/dist/contract.d.ts.map +1 -0
- package/dist/contract.js +157 -0
- package/dist/contract.js.map +1 -0
- package/dist/declare.d.ts +70 -0
- package/dist/declare.d.ts.map +1 -0
- package/dist/declare.js +22 -0
- package/dist/declare.js.map +1 -0
- package/dist/delta/apply-immutable-trusted.js.map +1 -1
- package/dist/delta/diff.d.ts +1 -1
- package/dist/delta/diff.d.ts.map +1 -1
- package/dist/delta/diff.js.map +1 -1
- package/dist/delta/index.d.ts +2 -2
- package/dist/delta/index.d.ts.map +1 -1
- package/dist/delta/index.js +41 -0
- package/dist/delta/index.js.map +1 -1
- package/dist/delta/revision-validator.d.ts +1 -1
- package/dist/delta/revision-validator.d.ts.map +1 -1
- package/dist/delta/revision-validator.js.map +1 -1
- package/dist/delta/tracker.d.ts +1 -1
- package/dist/delta/tracker.d.ts.map +1 -1
- package/dist/delta/tracker.js +40 -4
- package/dist/delta/tracker.js.map +1 -1
- package/dist/errors.d.ts +27 -0
- package/dist/errors.d.ts.map +1 -0
- package/dist/errors.js +44 -0
- package/dist/errors.js.map +1 -0
- package/dist/host.d.ts +254 -0
- package/dist/host.d.ts.map +1 -0
- package/dist/host.js +795 -0
- package/dist/host.js.map +1 -0
- package/dist/index.d.ts +17 -10
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +8 -8
- package/dist/index.js.map +1 -1
- package/dist/json.d.ts +9 -1
- package/dist/json.d.ts.map +1 -1
- package/dist/json.js.map +1 -1
- package/dist/link.d.ts +72 -0
- package/dist/link.d.ts.map +1 -0
- package/dist/link.js +169 -0
- package/dist/link.js.map +1 -0
- package/dist/loader.d.ts +4 -0
- package/dist/loader.d.ts.map +1 -0
- package/dist/loader.js +68 -0
- package/dist/loader.js.map +1 -0
- package/dist/manager.d.ts +115 -0
- package/dist/manager.d.ts.map +1 -0
- package/dist/manager.js +319 -0
- package/dist/manager.js.map +1 -0
- package/dist/node.d.ts +3 -4
- package/dist/node.d.ts.map +1 -1
- package/dist/node.js +3 -2
- package/dist/node.js.map +1 -1
- package/dist/publish.d.ts +5 -0
- package/dist/publish.d.ts.map +1 -0
- package/dist/publish.js +40 -0
- package/dist/publish.js.map +1 -0
- package/dist/record.d.ts +320 -0
- package/dist/record.d.ts.map +1 -0
- package/dist/record.js +1292 -0
- package/dist/record.js.map +1 -0
- package/dist/registry.d.ts +25 -0
- package/dist/registry.d.ts.map +1 -0
- package/dist/registry.js +6 -0
- package/dist/registry.js.map +1 -0
- package/dist/reload.d.ts +118 -0
- package/dist/reload.d.ts.map +1 -0
- package/dist/reload.js +589 -0
- package/dist/reload.js.map +1 -0
- package/dist/remote.d.ts +70 -0
- package/dist/remote.d.ts.map +1 -0
- package/dist/remote.js +308 -0
- package/dist/remote.js.map +1 -0
- package/dist/schema.d.ts +21 -0
- package/dist/schema.d.ts.map +1 -0
- package/dist/schema.js +228 -0
- package/dist/schema.js.map +1 -0
- package/dist/state.d.ts +92 -0
- package/dist/state.d.ts.map +1 -0
- package/dist/{services/state.js → state.js} +28 -138
- package/dist/state.js.map +1 -0
- package/dist/types.d.ts +62 -262
- package/dist/types.d.ts.map +1 -1
- package/dist/types.js.map +1 -1
- package/dist/websocket.d.ts +16 -0
- package/dist/websocket.d.ts.map +1 -0
- package/dist/websocket.js +15 -0
- package/dist/websocket.js.map +1 -0
- package/dist/wire.d.ts +135 -0
- package/dist/wire.d.ts.map +1 -0
- package/dist/wire.js +112 -0
- package/dist/wire.js.map +1 -0
- package/docs/guide.md +750 -0
- package/docs/spec.md +261 -0
- package/docs/tutorial.md +909 -0
- package/package.json +9 -10
- package/src/delta/README.md +5 -3
- package/dist/api.d.ts +0 -20
- package/dist/api.d.ts.map +0 -1
- package/dist/api.js +0 -77
- package/dist/api.js.map +0 -1
- package/dist/bundler.d.ts +0 -6
- package/dist/bundler.d.ts.map +0 -1
- package/dist/bundler.js +0 -3
- package/dist/bundler.js.map +0 -1
- package/dist/facets/host.d.ts +0 -12
- package/dist/facets/host.d.ts.map +0 -1
- package/dist/facets/host.js +0 -763
- package/dist/facets/host.js.map +0 -1
- package/dist/facets/loader.d.ts +0 -3
- package/dist/facets/loader.d.ts.map +0 -1
- package/dist/facets/loader.js +0 -5
- package/dist/facets/loader.js.map +0 -1
- package/dist/node/bundle-loader.d.ts +0 -30
- package/dist/node/bundle-loader.d.ts.map +0 -1
- package/dist/node/bundle-loader.js +0 -377
- package/dist/node/bundle-loader.js.map +0 -1
- package/dist/node/bundle.d.ts +0 -26
- package/dist/node/bundle.d.ts.map +0 -1
- package/dist/node/bundle.js +0 -183
- package/dist/node/bundle.js.map +0 -1
- package/dist/node/manifest.d.ts +0 -36
- package/dist/node/manifest.d.ts.map +0 -1
- package/dist/node/manifest.js +0 -6
- package/dist/node/manifest.js.map +0 -1
- package/dist/node/package.d.ts +0 -15
- package/dist/node/package.d.ts.map +0 -1
- package/dist/node/package.js +0 -199
- package/dist/node/package.js.map +0 -1
- package/dist/services/consumer.d.ts +0 -11
- package/dist/services/consumer.d.ts.map +0 -1
- package/dist/services/consumer.js +0 -585
- package/dist/services/consumer.js.map +0 -1
- package/dist/services/errors.d.ts +0 -8
- package/dist/services/errors.d.ts.map +0 -1
- package/dist/services/errors.js +0 -22
- package/dist/services/errors.js.map +0 -1
- package/dist/services/handle.d.ts +0 -15
- package/dist/services/handle.d.ts.map +0 -1
- package/dist/services/handle.js +0 -101
- package/dist/services/handle.js.map +0 -1
- package/dist/services/instances.d.ts +0 -26
- package/dist/services/instances.d.ts.map +0 -1
- package/dist/services/instances.js +0 -140
- package/dist/services/instances.js.map +0 -1
- package/dist/services/loopback.d.ts +0 -5
- package/dist/services/loopback.d.ts.map +0 -1
- package/dist/services/loopback.js +0 -15
- package/dist/services/loopback.js.map +0 -1
- package/dist/services/provider.d.ts +0 -37
- package/dist/services/provider.d.ts.map +0 -1
- package/dist/services/provider.js +0 -521
- package/dist/services/provider.js.map +0 -1
- package/dist/services/state-codec.d.ts +0 -15
- package/dist/services/state-codec.d.ts.map +0 -1
- package/dist/services/state-codec.js +0 -133
- package/dist/services/state-codec.js.map +0 -1
- package/dist/services/state-internals.d.ts +0 -13
- package/dist/services/state-internals.d.ts.map +0 -1
- package/dist/services/state-internals.js +0 -10
- package/dist/services/state-internals.js.map +0 -1
- package/dist/services/state.d.ts +0 -28
- package/dist/services/state.d.ts.map +0 -1
- package/dist/services/state.js.map +0 -1
- package/dist/services/wire.d.ts +0 -63
- package/dist/services/wire.d.ts.map +0 -1
- package/dist/services/wire.js +0 -186
- package/dist/services/wire.js.map +0 -1
package/docs/spec.md
ADDED
|
@@ -0,0 +1,261 @@
|
|
|
1
|
+
# Chord specification
|
|
2
|
+
|
|
3
|
+
This document specifies the runtime in `src`. Where it and the code disagree, the disagreement is a bug. [Known limitations and decisions](#12-known-limitations-and-decisions) lists what the runtime does not do yet and the choices the sections below leave open.
|
|
4
|
+
|
|
5
|
+
Hard constraints:
|
|
6
|
+
- Chord's runtime depends only on its own `delta` module (tracker, applyImmutable, encoder/decoder, diff) and Node/browser built-ins plus `typebox` for schemas. No other Pi packages.
|
|
7
|
+
- Built-in plugins are trusted, loaded from source in-process. The plugin loader uses dynamic `import()` with a cache-busting query (owner-approved exception, `TODO(loader)`: replace with an isolating, evicting loader; old module copies leak; TUI plugins need isolation).
|
|
8
|
+
- Hot reload is required. Host restart only for Chord/runtime code changes or host-level crash recovery.
|
|
9
|
+
|
|
10
|
+
## 1. Concepts
|
|
11
|
+
|
|
12
|
+
| Term | Meaning |
|
|
13
|
+
| --- | --- |
|
|
14
|
+
| type | Name + contract. `remote` (addressable from other hosts; schemas for input, variant state members, method args/results) or `local` (host-only; members may be anything, including functions and native objects, but must be listed in the contract: `method`, `opaque` or a state schema). |
|
|
15
|
+
| factory | `(input, principal, self) => behavior`. Synchronous. Registered with `host.serve(type, factory, options)` from a plugin facet. |
|
|
16
|
+
| record | Stable identity per `(type, canonicalInput, principalKey)`. Owns cells, resources, holds, listeners, tasks, registrations. Survives reloads. |
|
|
17
|
+
| behavior | What a factory run returns (methods + state members; `kind` for remote variants). Immutable, per epoch. |
|
|
18
|
+
| handle | What `get` returns. Late-bound. Never the record or behavior. |
|
|
19
|
+
| hold | One handle's claim. A record lives while held. |
|
|
20
|
+
| principal | App-defined. Stamped by links. In-host code acts as the `host` principal. |
|
|
21
|
+
| root | One short Chord call tree, admitted at trusted ingress (link request without root, UI event, timer, `host.root`). Carries `{ rootId, epoch }`. |
|
|
22
|
+
| epoch | One committed set of behaviors/registrations per host (server-issued id). |
|
|
23
|
+
| code holder | Long-lived execution that is NOT a root: Durable phases/tool runs, browsers, `self.task` loops. Every Chord call they make starts a fresh short root. |
|
|
24
|
+
|
|
25
|
+
## 2. Contracts
|
|
26
|
+
|
|
27
|
+
Contracts are runtime values for every type (shipped bundles have no TypeScript; third-party plugins are built separately). TS types are inferred from them.
|
|
28
|
+
|
|
29
|
+
```ts
|
|
30
|
+
export const Chat = remote("pi.chat", {
|
|
31
|
+
version: 1, // contract version: a plain number, bumped only on breaking changes
|
|
32
|
+
input: Type.Object({ session: Type.String(), conversation: Type.Number() }),
|
|
33
|
+
variants: {
|
|
34
|
+
viewer: { state: { view: ViewSchema } },
|
|
35
|
+
participant: {
|
|
36
|
+
state: { view: ViewSchema },
|
|
37
|
+
methods: { send: method(SendSchema, Type.Object({ submissionId: Type.Number() })) },
|
|
38
|
+
},
|
|
39
|
+
},
|
|
40
|
+
});
|
|
41
|
+
|
|
42
|
+
export const Index = local("pi.index", {
|
|
43
|
+
version: 1,
|
|
44
|
+
input: Type.Object({ session: Type.String() }),
|
|
45
|
+
members: {
|
|
46
|
+
search: method(Type.Object({ query: Type.String() }), Type.Array(Hit)),
|
|
47
|
+
onChange: method(Type.Function([Change], Type.Void()), Unsubscribe), // non-JSON allowed for locals
|
|
48
|
+
harness: opaque("pi.durable-harness@1"), // native object, by name+version
|
|
49
|
+
},
|
|
50
|
+
});
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
- Input identity: normalize (apply defaults, strip unknown), validate, then canonical JSON with fields equal to their schema default omitted. Changing defaults or normalization requires a new type name.
|
|
54
|
+
- Remote calls validate args/results and state values at the provider boundary. Local calls are not validated per call.
|
|
55
|
+
- Local contracts list every member (`members: { search: method(...), harness: opaque(...), status: Schema }`). A handle that is still `loading` must know whether `h.x` is a method (a caller that waits until live), a state member (a readable) or an opaque value, before any behavior exists. Members missing from the contract are found only once the handle is live.
|
|
56
|
+
- Every bundle embeds the contract tokens it was built against (its own copy). Fingerprint = hash of the canonical description.
|
|
57
|
+
|
|
58
|
+
### Compatibility rule
|
|
59
|
+
|
|
60
|
+
A consumer built against version M binds to a provider at version N iff N = M and the provider's description covers the consumer's; otherwise only through `accepts[M]` adapters declared by the provider (`serve(type, factory, { accepts: { 1: (current) => v1View } })`); else `contract_mismatch`.
|
|
61
|
+
|
|
62
|
+
Covers: every consumer member exists with the same kind; methods: provider accepts every argument the consumer can send (new fields optional, nothing newly required or retyped) and the provider's result fits the consumer's expected result (extra fields fine); state: provider value fits consumer schema (extra fields fine); opaque: same name@version.
|
|
63
|
+
|
|
64
|
+
Additive changes need no bump. Breaking changes (remove/rename/retype, newly required, or a change of meaning with the same shape) bump the number. Same version + non-covering description = forgotten bump, refused. Meaning changes without a bump are undetectable (author's responsibility, as with Durable document versions).
|
|
65
|
+
|
|
66
|
+
Checked at: `get` binding, reload prepare (new provider token vs every surviving consumer holding the type and every loaded plugin importing it), remote holds (consumer token in the hold frame).
|
|
67
|
+
|
|
68
|
+
The exact host-owned eager bootstrap hold is a lifecycle pin, not a surviving old-contract consumer. When its
|
|
69
|
+
provider is replaced, commit replaces that hold with one bound to the new provider contract, acquiring it before
|
|
70
|
+
releasing the old hold so the same record, state and unchanged resources survive. It needs no `accepts` adapter
|
|
71
|
+
for its own previous contract. Prepare/abort do not change that hold. All other consumers retain their own tokens
|
|
72
|
+
and the normal coverage/adapter checks; state-version and migration checks are unchanged.
|
|
73
|
+
|
|
74
|
+
## 3. Factory and declarations
|
|
75
|
+
|
|
76
|
+
Factories are synchronous; returning a promise throws. Declarations are identified across factory reruns by **property names** in object literals (not call order, not separate string keys). Names are unique per record per declaration kind; a declaration method may be called more than once if names don't repeat. Holds are identified by their address.
|
|
77
|
+
|
|
78
|
+
```ts
|
|
79
|
+
host.serve(StatusStore, ({ session }, _host, self) => {
|
|
80
|
+
const state = self.state({
|
|
81
|
+
status: cell(StatusSchema, { running: false, tasks: 0 }), // schema required, initial value
|
|
82
|
+
errors: cell(Type.Array(Type.String()), []),
|
|
83
|
+
});
|
|
84
|
+
const catalog = self.get(Catalog, {}); // hold; identity = canonical address (+ at)
|
|
85
|
+
const res = self.resources({
|
|
86
|
+
watch: resource({
|
|
87
|
+
rev: 1, deps: [catalog], exclusive: false,
|
|
88
|
+
open: async ({ signal }) => openWatch(session, signal),
|
|
89
|
+
close: async (w) => w.close(),
|
|
90
|
+
}),
|
|
91
|
+
});
|
|
92
|
+
self.listen({
|
|
93
|
+
changes: listener(res.watch, (w) => w.events, (event, ctx) => state.status.change(ctx, (d) => { d.tasks = event.tasks; })),
|
|
94
|
+
});
|
|
95
|
+
self.tasks({ poll: async (ctx) => { /* loop until ctx.signal aborts */ } });
|
|
96
|
+
const derived = self.derive({ visible: derivation([state.status], (s) => project(s)) }); // synchronous
|
|
97
|
+
self.register(Routes, { "GET /status": handler }); // registry keys are the registry's own keys; eager records only
|
|
98
|
+
return { status: derived.visible, report: (args, ctx) => { state.status.replace(ctx, args); return null; } };
|
|
99
|
+
}, {
|
|
100
|
+
state: { version: 2, migrate: { status: (old, from) => upgrade(old, from) } }, // migrate gets the old value and its version
|
|
101
|
+
linger: 30_000,
|
|
102
|
+
eager: false,
|
|
103
|
+
});
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
| Declaration | Survives reload | Notes |
|
|
107
|
+
| --- | --- | --- |
|
|
108
|
+
| `state({ name: cell(schema, initial) })` | same cell and value; migrated on state-version bump | Schema required (no TypeScript at runtime). `change(ctx, draft => void)`, `replace(ctx, value)`, `value`, `subscribe`. Writes through `change` touch only mutated paths. |
|
|
109
|
+
| `get(type, input, { at? })` | kept if declared again (same address) | returns a handle |
|
|
110
|
+
| `resources({ name: resource({ rev, deps?, exclusive?, open, close }) })` | kept unless `rev`/`deps` change | `open` runs at prepare (non-exclusive) or at the gate (exclusive); handle exposes `.value` |
|
|
111
|
+
| `listen({ name: listener(source, select?, fn) })` | same subscription while source (resource) and `select` (identity or source text) are unchanged; `fn` swapped | each event starts a fresh root in the current epoch; `select` is not re-run for a kept subscription |
|
|
112
|
+
| `tasks({ name: fn })` | restarted with new code at commit (old one aborted via ctx.signal) | a code holder; calls inside start fresh roots |
|
|
113
|
+
| `derive({ name: derivation(sources, fn) })` | kept; fn swapped | synchronous fn |
|
|
114
|
+
| `register(registry, { key: value })` | value swapped at commit | eager records only (v1) |
|
|
115
|
+
| `expose(resource)` | follows the resource | a local member (usually `opaque`) whose handle value is the resource's value once live |
|
|
116
|
+
|
|
117
|
+
State members of a behavior may be cells, derivations, or state members of handles the record holds (they are defined once the record is live). `cell.replace(ctx, value, { diff? })` publishes a structural diff when the new value shares containers with the old one (or `diff: true`), else `r`. Facet `setup` receives `{ serve, name, root, registry }` of its host.
|
|
118
|
+
|
|
119
|
+
`migrate` per cell is `(old, fromVersion) => new`. A record may be several versions behind (author bumped more than once between reloads).
|
|
120
|
+
|
|
121
|
+
Removed vs guide: `self.own`, `holdWhile`, async factories, inferred dependencies.
|
|
122
|
+
|
|
123
|
+
### Synchronous factory, asynchronous activation
|
|
124
|
+
|
|
125
|
+
`self.get` and `self.resource` only declare and return placeholders. Activation of a record:
|
|
126
|
+
|
|
127
|
+
1. run the factory synchronously (declarations);
|
|
128
|
+
2. await declared resources' `open()` (e.g. connect a DB);
|
|
129
|
+
3. await declared holds becoming live (dependencies activate first, coalesced);
|
|
130
|
+
4. record is live: handles `live`, `ready` resolves.
|
|
131
|
+
|
|
132
|
+
Inside the factory, placeholders must not be used (`resource.value` throws `not_ready`; handle calls throw). Methods/listeners/tasks run only on live records, so their dependencies are live. External consumers (UI, roots) holding a `loading` handle: calls wait until live (ctx cancellation, deadline); state members read `undefined` until hydrated. Activation failures propagate to dependents and all waiting gets; the next get retries.
|
|
133
|
+
|
|
134
|
+
### Host eager readiness
|
|
135
|
+
|
|
136
|
+
- `host.install(facet, { awaitEager? })` installs declarations synchronously and returns a promise for this facet's
|
|
137
|
+
eager activations. `host.load(facets, { awaitEager? })` installs the entire batch before awaiting its eager
|
|
138
|
+
activations. `awaitEager` defaults to true. Unheld lazy objects are not activated.
|
|
139
|
+
- Factory, resource-open, dependency and activation-finalization failures reject the returned promise with the
|
|
140
|
+
activation error (ChordError codes preserved; ordinary errors wrapped as `unavailable`). Eager failures also
|
|
141
|
+
reach `onError`. Installation is not a rollback transaction; setup/registration errors from `install` throw
|
|
142
|
+
synchronously. Abandoned startup should close the host.
|
|
143
|
+
- `{ awaitEager: false }` starts the same eager objects but resolves after installation, without waiting for their
|
|
144
|
+
activation. `host.ready()` awaits a snapshot of the host's eager holds at call time (including direct eager
|
|
145
|
+
`serve` and deferred loads), rejects their activation failures, and does not acquire additional holds. It is not
|
|
146
|
+
an ongoing health check and does not await task completion or remote state hydration.
|
|
147
|
+
- Local bootstrap readiness and upstream admission are separate. A worker connects its upstream link, loads with
|
|
148
|
+
`{ awaitEager: false }`, awaits its local bootstrap handle (e.g. Harness, with no upstream dependency), then
|
|
149
|
+
announces control readiness so its parent can accept the link. Only after admission does it await `host.ready()`.
|
|
150
|
+
Otherwise an eager reporter depending on server SessionStatus deadlocks with the parent waiting for control
|
|
151
|
+
readiness. Deferred loads skip the whole barrier, not selected dependencies; callers explicitly choose local
|
|
152
|
+
bootstrap handles, keep bootstrap busy until full readiness, and propagate later startup errors.
|
|
153
|
+
|
|
154
|
+
## 4. Principals, variants, handles
|
|
155
|
+
|
|
156
|
+
- `createHost({ principalKey })` maps principal to a key that includes connection provenance and grant revision. Records are per principal key. `host` is a built-in principal for in-host calls.
|
|
157
|
+
- Remote factories return `{ kind, ...members }`; `kind` must be a declared variant; members must match it. Local factories return any object.
|
|
158
|
+
- Handle: `status` (`loading | live | released | gone`), `ready` promise, `kind`, `shape` (`{ kind, members, contract }`), `onStatus(listener)` (every status change, with the error for `gone`), late-bound members, `release()`. A remote handle's state member holds a value only while subscribed (`undefined` after its last subscriber leaves). Shape changes (reload, variant) are pushed; members missing from the current variant reject `not_in_variant`; released handles reject `released`; forced disposal makes handles `gone`.
|
|
159
|
+
- Methods are `(args, ctx) => Promise<result>`. Factories still check arguments; variants are not complete authorization.
|
|
160
|
+
- Revocation: `host.invalidatePrincipal(match)` force-disposes matching records on this host and propagates over links; links re-stamp or close.
|
|
161
|
+
|
|
162
|
+
## 5. Lifetime
|
|
163
|
+
|
|
164
|
+
- Hold owners: record (declared `get`), call (`ctx.get`, released at settle), connection (remote holds; relay maps per origin connection), host (eager), UI.
|
|
165
|
+
- Activation coalesced per address; if all requesters leave before it completes, the result is disposed. Factory throw: unwind declarations, reject gets, next get retries.
|
|
166
|
+
- Disposal after last hold + `linger`: tasks aborted, listeners removed, resources closed (reverse order; all run; errors aggregated), holds released (cascade).
|
|
167
|
+
- Bulk forced disposal (`host.close`, `invalidatePrincipal`) goes in waves, holders before what they hold, so a resource's `close` can still use its record's holds; `host.close` closes links after the records.
|
|
168
|
+
- Forced disposal: handles `gone` immediately, new calls rejected, in-flight call contexts aborted, then cooperative cleanup.
|
|
169
|
+
- Hold graph acyclic in-host (checked at get); across hosts by convention.
|
|
170
|
+
- Links have ping/timeout; a dead link releases everything held through it.
|
|
171
|
+
|
|
172
|
+
## 6. Roots and epochs
|
|
173
|
+
|
|
174
|
+
- `Ctx = { signal, rootId, epoch, get(...) , principal? }`. Explicit; no AsyncLocalStorage dependency (may be used in dev to warn).
|
|
175
|
+
- A link creates a root for a request that carries none; nested remote calls carry `{rootId, epoch}`.
|
|
176
|
+
- Dispatch: a call with epoch E runs the record's behavior for E. A host retains at most `current` and `previous` epochs (two-version rule). A call for an epoch the host doesn't have (older than previous, or host started later) is rejected `superseded` (retryable by starting a new root). Never substitute another epoch.
|
|
177
|
+
- Browsers never carry epochs into the server: browser calls start roots at the server. Browsers learn of changes via shape/reset pushes and a server `Manifest` object; incompatible contract fingerprint -> `upgrade_required`.
|
|
178
|
+
|
|
179
|
+
## 7. Reload
|
|
180
|
+
|
|
181
|
+
Explicit trigger (`reload_plugins` tool, button, API). Serialized; no nesting.
|
|
182
|
+
|
|
183
|
+
1. Build/type-check gate (application tooling, outside Chord): dependency-closed set; refuse breaking remote contract changes without a contract version bump.
|
|
184
|
+
2. Load: on every affected host, import new facet modules (cache-busted).
|
|
185
|
+
3. Prepare: for each live record of each affected type, run the new factory in recording mode: existing cells readable, writes throw; listeners/tasks/registrations recorded, not started; holds recorded; non-exclusive resources whose key is new or `rev`/`deps` changed are opened as candidate-owned (closed on abort). Validate keys, variants, registry collisions. New records activated during prepare use the current epoch (old code). Any failure: abort, discard candidates. Vote.
|
|
186
|
+
4. Gate (only records whose state version was bumped or which replace an exclusive resource, plus records depending on those resources):
|
|
187
|
+
- Admission cut: from now, an old-epoch root that has not touched the record gets `upgrade_conflict` (retryable), not queued. Roots that already hold an activation lease on it are grandfathered; their re-entry is admitted.
|
|
188
|
+
- Wait for grandfathered activations to finish, deadline (default 5 s). Missed deadline: abort the reload, reopen, nothing changed (report blocking roots).
|
|
189
|
+
- Exclusive resources: close old, open new; on failure reopen old and abort.
|
|
190
|
+
- Run `migrate` into private copies; validate.
|
|
191
|
+
5. Commit (server commits itself, then sends commit to children; the decision is not persisted, see section 12; FIFO links guarantee no host receives a new-epoch root before committing): each host in one synchronous step installs migrated cell values and state versions, installs behaviors as epoch E', swaps registry tables, restarts tasks with new code, rebinds listeners, pushes shape/reset to subscribers, opens gates. E' current, E previous.
|
|
192
|
+
The coordinator's result carries `children`: each participating child's prepared report (its vote).
|
|
193
|
+
6. Retire: when no roots of `previous` remain (or on the next reload, which first waits for retirement with a deadline), drop its behaviors and release declarations only it declared.
|
|
194
|
+
|
|
195
|
+
Compatible changes (no state bump, no exclusive resource change) skip the gate: no waiting.
|
|
196
|
+
|
|
197
|
+
State compatibility rule: reload prepare compares each cell's old and new schema. Unchanged: compatible (no gate). Changed with the state version bumped: gate + `migrate`, output validated against the new schema. Changed without a bump: reload refused. A change of meaning with an identical schema needs a bump too (undetectable; author's responsibility). Initial values and migrate outputs are validated; per-write validation in dev mode.
|
|
198
|
+
|
|
199
|
+
## 8. Registries
|
|
200
|
+
|
|
201
|
+
```ts
|
|
202
|
+
export const Routes = defineRegistry<RouteHandler>("http.routes", { conflict: "error" });
|
|
203
|
+
self.register(Routes, { "GET /download": handler }); // in an eager record; keys are the registry's keys
|
|
204
|
+
host.registry(Routes).get("GET /download", ctx); // table of ctx.epoch (or current without ctx)
|
|
205
|
+
host.registry(Routes).subscribe((table, epoch) => ...); // bridges for imperative consumers (Durable)
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Per-epoch immutable tables; order = first registration order, kept on replace. An application can implement Durable's `RegistryReader` from the table (snapshot per epoch) so phases pin a complete snapshot; Chord does not ship that bridge.
|
|
209
|
+
|
|
210
|
+
## 9. Hosts and links
|
|
211
|
+
|
|
212
|
+
- `createHost({ name, principalKey, upstream? })`; `host.accept(link)` for children; `host.route(prefix, { authorize(principal, hostName), resolve(hostName) => Promise<Link> })`.
|
|
213
|
+
- Links: `memoryLinkPair()` (tests), Node IPC (child_process), WebSocket (server `ws`, browser `WebSocket`).
|
|
214
|
+
- Principal stamping: the accepting side stamps the principal of what connects (from its authenticator); forwarded principals accepted only from the parent link; payload principal fields ignored.
|
|
215
|
+
- A child that connects behind its parent's epoch is fenced (nothing routed to it) and reconciled by a reload of its modules to the parent's epoch. Calls its roots of the old epoch make upstream are admitted in the parent's current epoch (the parent never ran that epoch), not rejected `superseded`.
|
|
216
|
+
- Relay: server forwards frames for `at` hosts to the resolved child link, mapping holds per origin connection; authorizes before resolving/spawning and re-checks after.
|
|
217
|
+
- Wire frames (JSON): `hello`/`bye`, `hold`/`release` (a hold returns `shape` or `gone`; there is no separate `get`), `shape`, `call`/`result`/`cancel`, `sub`/`unsub` and `state` (`snap`/`ops`/`reset` with a per-subscription delta encoder), `gone`, `root_end`, `retire`, `invalidate`, `ping`/`pong`, reload control (`prepare`/`vote`/`commit`/`abort`).
|
|
218
|
+
|
|
219
|
+
## 10. Host manager (library)
|
|
220
|
+
|
|
221
|
+
`createHostManager({ driver, keepAlive })`: fork driver (entry, init, lock, env), serialized start/stop/replace, exit-before-replace, process-tree kill after grace, control channel (ready/fatal/shutdown), liveness callback (booting, activeCalls, viewers with heartbeats, durable tasks, idleFor).
|
|
222
|
+
|
|
223
|
+
## 11. Test coverage
|
|
224
|
+
|
|
225
|
+
The tests cover at least: lifetime/holds/linger/forced disposal/gone; activation coalescing and abandonment; principals and variants; remote calls over memory links incl. relay and principal forgery attempts; state snap/ops/reset over a link with reconnect; registries per epoch; reload: compatible (no wait, pinned roots), prepare isolation (writes throw, failing factory leaves everything unchanged), state bump with migrate and gate, `upgrade_conflict` for late old roots, deadline abort, exclusive resource replacement, cross-host commit ordering, two-version retirement and `superseded`.
|
|
226
|
+
|
|
227
|
+
## 12. Known limitations and decisions
|
|
228
|
+
|
|
229
|
+
### Still open
|
|
230
|
+
|
|
231
|
+
- **Retirement across hosts.** A child retires an epoch only after its parent retired it (`retire` frame) and its own roots of that epoch ended; the server decides from its own roots. A root started on a worker that calls up to a server which has already retired the epoch both had still gets `superseded` (retryable).
|
|
232
|
+
- **The commit decision is not persisted** (section 7, step 5). A server crash between its own commit and a child's commit leaves that child on the old epoch until it reconnects and is reconciled.
|
|
233
|
+
- **One level of reload participants.** Only the coordinator's direct child hosts take part; a child does not forward `prepare` to its own children.
|
|
234
|
+
- **Reconciliation needs the child's module list.** It is remembered from cluster reloads (`host.remoteModules`), so a server restart forgets it, and a child behind a restarted server then reloads no modules: it gets the epoch id with its own code. The reconciliation vote deadline is fixed at 30 s; a failed reconciliation closes the link, so the host manager restarts the worker.
|
|
235
|
+
- **Links do not resume.** The provider releases everything held through a closed link at once, so re-held records are new ones. Keeping holds for a reconnect window would need session ids on links. Calls in flight when a link drops reject `unavailable`; whether they ran is unknown.
|
|
236
|
+
- **Refused clients cannot tell.** The WebSocket adapter closes unauthenticated sockets; the client sees a dropped upstream and its calls wait for a reconnect until their ctx is aborted.
|
|
237
|
+
- **The publication queue is process-wide and synchronous.** A slow observer delays every later notification in the process, including those of other hosts in the same process.
|
|
238
|
+
- **No automatic restart.** A terminal host failure is picked up by the host manager's `keepAlive` or `fatal` path, not inside Chord.
|
|
239
|
+
- **The Node loader leaks.** Every reload imports fresh module copies that are never evicted, and it does not isolate plugins (`TODO(loader)`).
|
|
240
|
+
- **Stale socket locks.** On Unix systems other than Linux, two processes that probe the same stale socket file at the same moment could both take the lock. Managers serialize starts per lock name.
|
|
241
|
+
- **Listener `select` identity.** A listener's subscription is kept across factory runs when its origin and `select` are unchanged; `select` counts as unchanged when identical or with the same source text, so a `select` whose behavior depends on captured values that change between runs is not detected.
|
|
242
|
+
- **Shutdown and dead links.** `host.close()` disposes records in waves (holders before what they hold) and closes links last. A `close` that awaits a remote call over a link that is already down waits for a reconnect; bound such calls.
|
|
243
|
+
|
|
244
|
+
### Decisions
|
|
245
|
+
|
|
246
|
+
- **Errors.** The guide's codes; a `reason` field carries the detail, for example `invalid/args`, `invalid/cycle`, `invalid/duplicate`, `invalid/read_only`, `invalid/not_eager`, `invalid/registry_conflict`, or `unavailable/unknown_type`. A `ChordError` thrown by a factory or a dependency keeps its code through every dependent; other errors become `unavailable`, with the original as `cause`.
|
|
247
|
+
- **Reload errors.** There is no `reload_aborted` code. A failed reload throws the cause's code and reason with the message `reload aborted: …` (the cause is attached). Reasons of its own are `aborted/gate_deadline` (with the blocking roots in `details`), `aborted/deadline`, `aborted/vote_deadline` and `unavailable/reload_running`. Across hosts, the message names the host that failed. A child cannot reload on its own (`unavailable/not_coordinator`).
|
|
248
|
+
- **New records during prepare.** A reload pre-allocates the next epoch ("pending"). Types the same reload adds are served from it; their records install behavior only for the pending epoch and are force-disposed if the reload aborts.
|
|
249
|
+
- **Gate admission.** From the gate until commit or abort, creating a new record of a changed type fails with `upgrade_conflict` (retryable, reason `gate`); nothing is paused, so nothing can deadlock. Gated records get `minEpoch = E'`: calls and writes carrying an older epoch are rejected `upgrade_conflict` permanently, and a gated record drops its previous-epoch behavior at commit. Only records whose state version or exclusive resources change are gated.
|
|
250
|
+
- **Tasks and listeners during the gate.** Tasks of gated records are aborted at the admission cut and restart with new code at commit, or with the old code on abort. Listener events for gated records are queued and replayed through the new listener at commit, or the old one at abort.
|
|
251
|
+
- **Grandfathered roots** are the roots that called into the record and are still running at the cut. Every host remembers on which links a root's calls went out and sends `root_end` on them when the root ends, so leases outlive calls across hosts.
|
|
252
|
+
- **Removed types and missed retirement.** Records of removed types are force-disposed at commit. When a retirement deadline is missed, retirement is forced, so those roots get `superseded` on their next call.
|
|
253
|
+
- **Two-version rule.** A record of a changed type created after a commit has only new code; a previous-epoch root calling it gets `superseded`.
|
|
254
|
+
- **Validation.** Remote calls are validated at handle dispatch, in-host too: a direct binding against the provider's schema, a binding through an adapter against the consumer's. A provider validates each snapshot and reset of a remote state member once per (re)subscription, against the consumer's contract through an `accepts` adapter and the provider's otherwise; ops are validated only in dev mode. At reload, the new provider's description must cover the old provider's when the version is unchanged, in addition to the check against live consumers and facets' `uses`.
|
|
255
|
+
- **Contracts.** Local contracts list their members, so a loading handle knows whether a member is a method, state or opaque. Canonical identity omits input fields equal to their default; with TypeBox, use `Type.Optional(Type.X({ default }))` to let callers omit them. Provider variants must cover consumer variants; members the consumer's contract does not list are hidden. Schema change detection compares canonical JSON, so reordering `anyOf` branches counts as a change. Fingerprints use a non-cryptographic 64-bit hash: synchronous and browser-safe; contracts are trusted.
|
|
256
|
+
- **Principals.** A root ctx at ingress acts as its principal (`host.root(fn, { principal })`); contexts inside objects act as `host`. The acceptor decides a link's kind and name (`accept(link, { principal, kind, name })`); only configured host links are addressable, carry roots and may vote, and control frames are accepted only from the parent. Clients never carry roots or epochs. Relaying needs a matching `route`; `authorize(principal, hostName)` runs before `resolve` and again after, and revoking or re-stamping a principal closes its relays at once. `invalidatePrincipal(match)` re-stamps or closes matching links and propagates pattern matches to child hosts.
|
|
257
|
+
- **Cells and readables.** A value returned from `change` replaces the value and is always validated; `replace` transfers ownership without copying; by default (`{ diff }` overrides) it publishes a diff when the new value reuses a container of the old one at the same key, else `r`. `subscribe(fn)` delivers values, starting with the current one; `events(fn)` delivers `snap`, then `ops`, or `reset`. Handle state members are stable readables, `undefined` while loading or outside the variant, and reset when their source moves. Remote state members hydrate only while subscribed and forget their value when the last listener leaves.
|
|
258
|
+
- **Holds and calls.** A hold's identity is its address; a second `self.get` of the same address in one run returns the same handle. A handle call without ctx, or with a task ctx, starts a fresh root in the current epoch; a ctx whose call or root already settled is refused (`aborted/expired`), one from another host `invalid/foreign_ctx`. Calling a handle inside a factory throws `not_ready`.
|
|
259
|
+
- **Resources and disposal.** New and replaced exclusive resources open at the gate, and so do non-exclusive resources of the same record that depend on them. A new record for an address that is still being disposed waits for that disposal. A resource open that completes after its signal aborted closes its value at once.
|
|
260
|
+
- **Links and the host manager.** Every received frame is validated against its schema; malformed frames, duplicate ids and control frames from the wrong side close the link. Each connection pings every `pingInterval` and closes after `pingTimeout` without a frame. The child holds a kernel lock (an abstract socket on Linux and Android, a named pipe on Windows, a probed socket file elsewhere), acquired by `childControl().lock()` before storage opens. The manager kills the child's process group on every exit path and derives `booting` and `idleFor` from the child's liveness reports. WebSocket support is structural, so Chord does not depend on `ws`.
|
|
261
|
+
- **The state API stays.** Objects keep named cells and derivations; giving each object exactly one state value was rejected. Do not reopen this without a new, concrete problem.
|