@earendil-works/chord 1.4.0 → 1.5.0

package/docs/next/slice-3b.md

@@ -0,0 +1,197 @@
+# Slice 3b: surviving drops (spec, for the owner's approval)
+
+Status: **approved (owner, 2026-10-22)**, after Astra's and Fable's spec reviews. Guiding rule (owner): simple, fast,
+correct. The single source for building 3b, together with the feature map in `slices.md` (entries assigned to 3b),
+`design.md` read in full (most relevant: §3 links, recovery and timeouts, §4, §6, §7 references, §8, §19, §22), the
+decision log, the carry-over (L4, L5, L6) and the root `AGENTS.md` style section. Builds on L1 and slices 1, 2 and 3a
+(`76b32ed2d`).
+
+## What an app can do after this slice
+
+```ts
+const notes = host.hold(Notes, { session: "S1" }, ctx);        // a reference at once, not awaited
+notes.state.value;                                              // { kind: "loading" }
+const w = await notes.state.watch(ctx);                         // the current state, then every change
+w.start(async (state) => render(state));                        // loading → live → loading (link dropped) → live …
+
+await notes.add({ text: "milk" }, ctx);                         // parks while loading, sent once live
+if (notes.state.value.kind === "failed") notes.retry();         // a Retry button
+
+client.connect(newLink, { principal: { kind: "server" } }, ctx);  // after a drop: every reference re-holds
+
+host.serve(Notes, factory, { linger: 5 * 60_000 });             // a just-released object stays warm for 5 minutes
+```
+
+## 1. Reference state (§7)
+
+- Every reference has `ref.state`, a `NativeWatchable<RefState>` (in-process: it holds errors; open point 1):
+
+  ```ts
+  type RefState =
+    | { kind: "loading" }
+    | { kind: "live" }
+    | { kind: "failed"; cause: FailureCause; retryAt: number | undefined }   // undefined: broken, no automatic retry
+    | { kind: "released" };
+  ```
+
+  `ref.state.watch(ctx)` delivers the current state, then every change. On release the state becomes `released` and
+  the watch ends `ended` (the end is the notification; a native watch drops undelivered frames when it ends).
+- **One rule for both sides:** a reference is `loading` while its object is activating or retrying, while it waits for
+  `load` or a parent link, and while its link is down; `failed` while its object is failed and no attempt runs; `live`
+  when bound; `released` once released. `variant` is `undefined` before the first binding and keeps the last bound
+  variant while `loading` or `failed`.
+
+  | State | Calls | State members |
+  | --- | --- | --- |
+  | `loading` | park (section 3) | readable: the last value; before the first binding a read throws `not_ready` and a `watch` waits for the first value |
+  | `live` | run | readable |
+  | `failed` | reject at once with the cause's error | reads throw the cause's error; existing watches stay open |
+  | `released` | throw `released` | reads throw `released`; watches end `ended` |
+
+- Member watches survive `loading` and `failed` (reference states): they get the next frame when the reference is live
+  again. They end only when the reference is released, the member itself fails (W1, `failed`), or the member is not in a
+  newly bound variant (`ended`).
+- **Retry** (open point 2): `ref.retry()` asks the object for an attempt now, skipping the backoff window and renewing a
+  broken object's readiness (carry-over L5); one attempt runs at a time, so a retry during one is ignored. Local: the
+  object's `attempt()`. Remote: a new message `retry { hold }`; the provider does the same for the held object, for all
+  its holders. On `live` and `released` references `retry()` does nothing; without an open parent link the next
+  `connect` re-holds anyway.
+- **Attempt notice** (open point 2): when a failed object starts a new attempt (backoff, retry, or a dependency going
+  live), its holds get `loading { hold }`, so remote references move `failed → loading` and park calls exactly like local
+  ones. `failed { hold, cause, retryIn? }` carries the remaining delay of the provider's window (`until - now`, from a
+  new `until` on slice 2's backoff entry, carry-over L4) as advice for `retryAt`; the attempt itself is announced by
+  `loading`.
+
+## 2. Acquiring: `hold` and `get` (§7)
+
+| Situation | `host.hold(Type, params, ctx, options?)` | `host.get(…)` |
+| --- | --- | --- |
+| invalid params, a local type not served here, no principal for a same-host remote type | throws at once (made before `load`: `failed` broken once `load` resolved, since the types are known only then) | rejects at once |
+| before `load` resolved, or no parent link yet (remote) | `loading` | waits |
+| the object activates | `loading` | waits |
+| a transient failure | `failed` (with `retryAt`), then `loading` on the next attempt | waits; at its deadline it releases the reference and rejects with the last cause |
+| a broken failure | `failed` | releases the reference and rejects at once with the cause |
+| the link drops before the first binding | `loading`, re-held on the next `connect` | keeps waiting within its one deadline |
+| live | `live` | resolves |
+| the caller's context is aborted | (no effect on a `hold`) | releases the reference and rejects |
+
+A `hold` follows its object until released and never rejects. A `get` has one deadline (`readyTimeout`, per get) from
+the moment it is called, across reconnects.
+
+## 3. Parking (§3, §7)
+
+- A call made on a `loading` reference is queued with its own deadline, measured from the moment it was called (a
+  reconnect never renews it). When the reference goes `live`, queued calls are **sent** in the order they were made,
+  before any later call (a new call cannot overtake queued ones); their completions may arrive in any order. When the
+  reference goes `failed` they reject with the cause; at its deadline a call rejects `unavailable/park_timeout`; the
+  caller's cancellation rejects it at once; release rejects them `released`.
+- A queued call is resolved against the binding it is sent on: after a reconnect that bound another variant, a method
+  the new variant lacks rejects `invalid/member`; indexes are taken from the new binding.
+- Calls already **sent** when the link drops fail `unavailable` (never resent, §7).
+- **The park deadline is `readyTimeout`** (owner, 2026-10-22; C13's two host timeouts stay): a queued call waits at most
+  the reference's `readyTimeout` (host default 60 s, per `get`/`hold` override), then rejects `unavailable/park_timeout`.
+
+## 4. Link drops and re-holds (§3 recovery, §4, §22)
+
+- **One owner of remote objects across links:** the host keeps its remote references (their objects) independent of any
+  one link; each `connect` creates a link with fresh id counters (ids are link-local) and re-holds every non-released
+  remote reference on it; a closed link's tables are dropped, and nothing of an old link (a late callback) touches the
+  new one.
+- **On a drop:** every non-released remote reference goes `loading` (also a `failed` one), sent calls fail
+  `unavailable`, new calls queue. `host.connect(link, …)` may be called again once the previous parent link has closed
+  (open point 6); while one is open it throws.
+- **The re-hold:** a new `hold` (new id); on its `bound`, in **one batch** of this host and before any queued call is
+  sent: each member of the bound variant gets the snapshot value with one frame `[["r", value]]` and a fresh Delta
+  decoder (open point 4); members not in the bound variant end (`ended`, derives over them fail); new members get their
+  first value; failed members listed in `bound` fail (W1); `variant` is updated. A different variant is the normal result
+  of a changed grant (design §4: reconnect, authenticate anew, re-hold). Then queued calls are sent.
+- **The provider side** is unchanged: a closed link releases its holds (objects with params end with their last
+  reference unless they linger, section 5); a reconnecting child is a new `accept` with new holds.
+- **References a host's objects hold** (`self.get` of a remote type) go `loading` the same way; a **live** dependent
+  stays live (R18: a dependency leaving `live` does not change it) and its calls through them queue. An **activating**
+  dependent keeps waiting for the dependency; its own `readyTimeout` bounds it (slice 2), and R18 retries it on `bound`.
+- Who reconnects is the app (its link adapter calls `connect` again); Chord only re-holds.
+
+## 5. `linger` (§3, §6, §19, D8, C7)
+
+`host.serve(Type, factory, { linger })` (milliseconds, default 0): a **live** object with params whose last reference
+is released stays live for `linger`, keeping its own dependencies; a `get`, `hold` or arriving `hold` message within
+that time reuses it; otherwise it ends. An object that is waiting, activating or failed when its last reference goes
+ends at once (slice 2). A failed state member is not a failed object and does not end a lingering one. `host.close`
+ends lingering objects at once. Host-lifetime objects (`local`, empty params) never end by references; a remote type
+with empty params lingers like any other.
+
+## Races (listed before sizing)
+
+1. A sent call when the link drops: fails `unavailable`, never resent.
+2. A call made while `loading`: queued with its deadline; sent once live, in order; `park_timeout`; cancellation;
+   release.
+3. A reference released while `loading` (no link): nothing sent; not re-held; queued calls reject `released`.
+4. `connect` while the previous parent link has not finished closing: throws.
+5. A drop during a re-hold's handshake or before its `bound`: stays `loading`; the next `connect` re-holds again.
+6. The re-hold finds the object failed: `failed` with `retryIn`; the provider's next attempt sends `loading`, then
+   `bound` when live.
+7. A reconnect creates a fresh object (the old one ended without linger): new snapshot; the old in-memory documents are
+   gone (documented).
+8. A get or an arriving `hold` within `linger` versus the linger timer: a reuse cancels the timer; the holder count
+   decides, so an expiry never ends an object that has a holder again; no grace period.
+9. A failing object without references ends at once, never lingers; the last release during activation ends it.
+10. `host.close` with queued calls and loading references: queued calls reject `released`, references go `released`.
+11. `ref.retry()` during an attempt or a pending re-hold: ignored.
+12. Shared remote retry: two consumers hold one failed object; one retries; both get `loading`, then `bound` or
+    `failed`.
+13. A broken object held by another consumer: a `retry` renews it for everyone.
+14. A reconnect binds another variant (a changed grant): replicas rebuilt in one batch; vanished members' watches end;
+    queued calls checked against the new variant.
+15. A release or a call's deadline while a snapshot is being adopted or queued calls are being sent: the reference's
+    state decides at each step; nothing is sent for a released reference.
+16. A member `watch` before the first binding: waits; rejects on `failed` (cause) or release.
+17. A re-hold arrives while the previous instance (released by the drop, no linger) still tears down: waits for that
+    ending (slice 2), then a fresh object.
+18. An activating dependent whose remote dependency's link drops: it fails transient at the drop (the dependency's
+    pending outcome rejects `unavailable`); R18 retries it when the dependency is bound again.
+19. A drop while `failed`: `loading`, re-held on the next `connect`.
+
+## Cost per hop
+
+Nothing changes for live references (calls, commits, frames as 3a). A drop costs one `hold` and one snapshot per
+reference on reconnect; a queued call holds its arguments until sent; `ref.state` is created lazily on first access.
+
+## Benchmarks
+
+Extend `test/next/bench/links.ts` (both codecs): reconnect with 1 and 100 references (time to all live, bytes), 1,000
+queued calls flushed on reconnect (time, order), and the live paths of 3a unchanged (regression rows).
+
+## Carry-over taken by this slice
+
+L4: `retryAt` (backoff `until`). L5: `ref.retry()` renewing a broken object's readiness; calls on a failed reference
+reject with its cause; C8. L6: `linger`, `ref.state`, `ref.retry()`, `host.hold`. Moved out: a second `bound` as a reset
+(slice 4 for `reset` after overflow, slice 7 for restarted objects); the park deadline's per-type override is not
+built.
+
+## Tests
+
+- `ref.state`: every transition for local and remote references, watched (first value, each change, the end on
+  release); `variant` before, during and after bindings.
+- `hold` and `get`: every row of the table in section 2.
+- Parking: queue order, one deadline across reconnects, cancellation, release, failure, a variant change.
+- Drops and re-holds: references go loading; sent calls fail; reconnect re-holds with fresh ids; one batch per bound
+  with `[["r", value]]` frames; another variant on reconnect; old-link callbacks never touch the new link.
+- Retry: local and remote; shared across consumers; broken renewed; ignored during an attempt; `loading` notices.
+- `linger`: reuse within the window, end after it, only live objects linger, failed members do not end it,
+  `host.close`, reconnect within linger keeps in-memory documents.
+- Every race above, both codecs where a link is involved.
+- GC: a released `hold` reference, a reference across a drop, a closed link's tables, a lingering object after expiry.
+- Examples: `10-drops.ts` (drop, park, reconnect, re-hold, `ref.state`), `11-linger-and-retry.ts`.
+
+## Decided with the owner (2026-10-22)
+
+`ref.state` is a `NativeWatchable<RefState>` (the `NativeWatchable` interface joins `Watchable` in `src/watch.ts`,
+upstream first); remote retries use two new messages, `retry { hold }` and `loading { hold }`; parked calls wait at most
+`readyTimeout` (no third host timeout); a re-hold replaces each member with `[["r", value]]`; C8 (`state: false`) is
+cut; `connect` may be called again once the previous parent link has closed; `host.root` moves to slice 8.
+
+## Size
+
+About 1,100-1,400 source lines (reference state, acquisition, parking with its queue, re-holds with variant changes,
+retry and loading notices, `linger`).

package/docs/next/slice-4.md

@@ -0,0 +1,146 @@
+# Slice 4: the topology (draft: scope, structural changes, races)
+
+Status: **overview** (owner, 2026-10-23: slice 4 is split into 4a and 4b). The 4a spec proper is `slice-4a.md`, which
+wins where the two differ; 4b is sketched at the end of this file. Reads with `design.md` (§3 hosts, links, routes, flow control; §4 principals and
+revocation; §6 cycles; §7; §19 presence; §22 wire), the decision log (R1, R2, Q1-Q5, Q13, R20, "Flow control: an
+ack-clocked window", "One pending queue for slow consumers of state", "Compression at the origin", "Wire protocol"),
+the carry-over (L6, L7) and slices 3a/3b. Builds on `9066b343c`.
+
+## The split
+
+| Slice | Contents |
+| --- | --- |
+| **4a: links and relays** | IPC and WebSocket links; trusted links; parts as opaque bytes; flow control (acks, pending queue, call window); host kinds, addresses, `placement`; `route` (`authorize`, `resolve`, re-resolve with backoff); relayed holds and calls (ids mapped, principal and origin incarnation stamped, per-hop acks, `reset` and a second `bound`) |
+| **4b: authority and housekeeping** | revocation (`revokedBy`, `host.revoke`, propagation to children); cycle checks across hosts (activation chain in hold headers); `host.references` (origin ids); `host.activity`; size limits per part (the admitting link's decoded limit stamped on relayed work); compression at the origin |
+
+4a is what the playground needs (server, workers over IPC, tabs over WebSocket). 4b adds header fields on top of 4a's.
+
+## 4a: structural changes the current code needs
+
+1. **A link carries both directions.** Slice 3 has `ProviderLink` (`accept`: serve only) and `ConsumerLink` (`connect`:
+   hold only). With routes, the server holds worker objects over a resolved link *and* serves the worker's holds of
+   server types over the same link; the worker serves the server's holds over its `connect` link *and* holds server
+   types over it. One per-link object with both sides (two id spaces: holds this side sent, holds it received; message
+   kinds are directional, so dispatch stays by kind).
+2. **Parts are opaque bytes.** Today the codecs parse every value (`params`, `args`, `value`, `ops`, snapshot members)
+   at decode. A relay must forward them unparsed (R1), so values become byte parts that only the end that owns them
+   decodes (the consumer's replica, the provider's `checkArgs`). Exception: a `hold`'s params, which a relay parses to
+   compute placement and authorize (small; checked by `checkParams` anyway). Sizes then come for free (`byteLength`).
+3. **Stamped header fields.** `hold` and `call` gain the origin's principal and incarnation (Q13), set by the relay
+   from the link the work came in on; honored only on a link from a parent; overwritten on every other link. 4b adds
+   the activation chain, the origin id and the decoded limit the same way.
+4. **`ack`** (consumer to provider, after applying each `bound`/`state` message) and the call window in `hello`
+   (`calls: N`).
+5. **Resets as `r` ops** (decided below): a `state` entry flagged `reset: true` carries `[["r", value]]` per member;
+   the consumer applies it like any ops (the decoder restarts at `r`). A second `bound` on a live hold stays refused
+   until slice 7.
+6. **`loading` on a live hold**: a relay whose upstream link dropped tells its downstream holds; 3b's consumer moves
+   only `failed -> loading` on it today.
+7. **The pending queue** (fold, leaf index, cap 10,000) per hold member on the provider side, and in Chord's shared
+   watch handle (pi first; carry-over L6).
+
+## 4a: races
+
+Routes and resolve:
+
+1. Two holds (or a hold and a call's re-hold) for one address while its `resolve` runs: one `resolve` per address,
+   shared; both holds go out on the link it returns.
+2. `resolve` rejects or its signal aborts: the holds stay `loading`; re-resolve with the per-address backoff (1 s
+   doubling to 60 s) while holds for the address exist.
+3. Every hold for an address is released while `resolve` runs: the resolve finishes (a worker start that produced a
+   link finishes even if its first requester left, §3); the link is kept for the next hold. Who closes an idle worker
+   link is the app's supervisor (`host.activity`, 4b).
+4. `authorize` refuses: the hold fails broken `forbidden` before anything starts; a later hold with another principal
+   for the same address still resolves.
+5. Placement: the caller computes `placement(params)` and routes by it; the provider computes it again and refuses an
+   object that is not its own (`invalid/placement`, broken). A placement rule changed by a reload between the two is
+   that refusal (rollout handling is slice 7).
+6. A worker restarts (crash, the supervisor stopped it): its link drops; the new link's `hello` carries a new
+   incarnation; re-holds get fresh objects (in-memory documents gone, as 3b race 7).
+
+Relays (server between a tab and a worker):
+
+7. The worker link drops with relayed holds: the server sends `loading { hold }` downstream (tabs go `loading`,
+   values readable, new calls park on the tab), fails relayed calls in flight with `unavailable` (never resent),
+   re-resolves, re-holds upstream, and forwards the new `bound` downstream (accepted while `loading`, as 3b's
+   re-hold; another variant is allowed). Superseded by `slice-4a.md` section 4.
+8. A tab call that crossed the `loading` notice reaches the server while the upstream link is down: answered
+   `unavailable` (as a call sent before a drop, §7), not parked at the relay.
+9. The tab releases a hold while its upstream re-hold or `resolve` is pending: the relay releases upstream once the
+   upstream hold exists; nothing is sent downstream after the release (Q1 covers a `bound` crossing it).
+10. The tab link closes: the relay releases every upstream hold it made for that tab and cancels its relayed calls.
+11. Two tabs of one principal hold the same object: two upstream holds, each with its own encoder (R1, as today).
+12. A worker sends `gone` for a relayed hold: forwarded downstream; the relay forgets the mapping.
+13. A worker holds a server type (or another worker's): it goes over its `connect` link; the server serves it or
+    routes it on (worker to worker through the server, §3).
+
+Flow control:
+
+14. An `ack` crossing a `reset`, a `release` or a link close: acks count messages per link, not holds; an ack for a
+    message whose hold was released still frees its slot.
+15. A `reset { hold }` reaching the provider while the hold is pending, failed or released: ignored unless bound
+    (Q1).
+16. A relay with a full window towards a tab: it drops new parts for that tab's holds and marks them; when the tab
+    acks, it sends `reset { hold }` upstream for each marked hold and drops further parts for a hold until its new
+    `bound` arrives.
+17. **Torn resets:** one commit touched holds A and B of a tab; both were dropped and reset; the two new `bound`s
+    arrive as two messages, so a derive over A and B could see A new and B old in between. Proposal: `reset` carries
+    a list of holds and the provider answers with one message holding all their snapshots.
+18. The pending queue reaches its cap while the window is full: only "replace with the current value" stays; when a
+    slot frees, the provider sends a snapshot of the member (a second `bound` part or an `r` op; see open point 3).
+19. Calls: more than N calls from one link are parked by the caller (one FIFO per link, across references); a
+    cancel frees the slot at the caller when sent and at the provider when received (per-direction order keeps the
+    counts equal).
+20. **Call fairness at the relay:** each tab has N calls towards the server, but the server has one window of N
+    towards a worker for all tabs and its own calls. Proposal: the relay parks relayed calls per origin and sends
+    round-robin, so one flooding tab cannot occupy the worker's window; the relay holds at most tabs x N parked calls.
+
+Links and transports:
+
+21. IPC: `send` while the child is exiting, `disconnect` racing a message in flight: the link closes once; messages
+    after close are dropped; `process.send` errors close the link.
+22. WebSocket: a frame above the transport's `maxPayload` closes the connection (code 1009); a client that stops
+    reading is bounded by the window (2 messages) plus the pending queue cap, not by the socket buffer.
+23. Trusted links skip the message schema check (IPC, carry-over L6) but never grant stamping: stamped fields are
+    honored only from the link to the parent.
+
+## Decided (owner, 2026-10-23)
+
+1. **Trusted links are an explicit option:** `accept(link, { principal, trusted: true })`, and `resolve` may return
+   `{ link, principal, trusted }`; default untrusted. Trusted skips the message schema check; stamping is still
+   honored only from the parent link.
+2. **Resets and overflow are `r` ops in an ordinary `state` message** (races 16-18): a relay's `reset { holds }`
+   (a list) is answered with one `state` message whose entries carry `[["r", value]]` for every member of each reset
+   hold that is still bound, all snapshots taken in one synchronous step, applied by the consumer in one batch (no
+   torn commit across the reset holds). An overflowed pending queue sends `[["r", value]]` the same way. Delta's
+   encoder and decoder restart at an `r`. A state entry gets the header flag `reset: true`, so a relay (which never
+   reads parts) knows when to stop dropping a reset hold's parts. A second `bound` stays only for slice 7 (a restarted
+   object may choose another variant).
+3. **Call fairness: round-robin per origin at the relay** (race 20): each origin's calls keep their order; while the
+   worker's window is full, origins take turns, so one flooding tab cannot starve the others.
+4. **The Durable conversation view:** known cost until Durable's windowed view (pi #11327, waiting for its reporter)
+   lands: a re-hold sends the whole active transcript.
+
+## 4b: scope and races (sketch)
+
+- **Revocation:** `createHost({ revokedBy })`; `host.revoke(r)` ends every object and closes every link whose
+  principal has `r` in `revokedBy` (L1's `revocationKeys`), then propagates `r` to children (a `revoke` message on
+  trusted child links). Races: a hold stamped with a revoked principal already on its way to a worker (it binds, then
+  its link closes: R10's window); revoke during an activation; revoke while a relayed call runs (its handler is
+  aborted; R11 jobs later).
+- **Cycles across hosts (Q5):** a hold made during an activation carries the chain of activating instance ids
+  (bounded); the provider refuses a hold whose chain contains the instance being activated (`invalid/cycle`); chains
+  from untrusted links are ignored. Races: the chain crossing a relay (copied, never extended by the relay); a cycle
+  through a host that restarts.
+- **`host.references(Type)`** (presence, §19): who holds objects of a type, with an origin id per client (stamped by
+  the relay, so two tabs are two clients); a link drop is a departure.
+- **`host.activity`**: `{ references, calls, jobs, lingering }` as one readable for the app's keep-alive.
+- **Size limits per part (R2, Q2)**: checked from the header before decoding; an oversized part fails only its hold or
+  call (`too_large`); the admitting link's decoded limit is stamped on relayed work and enforced where it is decoded.
+- **Compression at the origin**: a part above a threshold is compressed once by its origin (codec field in the
+  header); relays forward it compressed; only the end that decodes inflates it, with the decoded limit.
+
+## Size
+
+Reviewers' estimate for slice 4 as a whole: 2,500-3,000 source lines; 4a carries most of it (the two-sided link, the
+relay, transports, flow control).