@procedures/client 0.3.0 → 0.5.0

package/CHANGELOG.md

@@ -1,5 +1,73 @@
 # @procedures/client
 
+## 0.5.0
+
+### Minor Changes
+
+- 09e5566: Remote events (`documentation/EVENTS.md`): a peer subscribes to the topics of the event bus of another peer. A subscription is a streaming call of the procedure `events.subscribe`, so it uses the frames of each call: credit is the backpressure, and a cancel is the unsubscribe. It works over WebSocket in the two directions, HTTP with NDJSON, a message port and the local transport. The protocol stays v1.
+  
+  - `WebSocketTransport.subscribe(topics, handler)` gives the unsubscribe function, with the `ready` promise of the first confirmation. `WebSocketTransport.events(topics)` gives an async iterator. After a drop, the transport resubscribes on the new connection.
+  - `subscribeRemote()` and `remoteEvents()` do the same over each transport. After a failure that a retry can repair, they resubscribe with a backoff.
+  - `ProcedureServer` has the option `events`: a bus, a topic rule (`topics`), an `allow` function and a `buffer`. The server pins `events.subscribe` outside its registry, so it is not an MCP tool. `ProcedureServer.publish()` emits on the bus. `stop()` ends each subscription with `UNAVAILABLE`.
+  - `WebSocketServerTransport.subscribe(connectionId, topics, handler)` and `events(connectionId, topics)` subscribe to the bus of a client. `LocalTransport` takes an `EventPublisher` with its option `events`.
+  - The delivery is at most once, in the order of `emit()`. When the queue of a slow subscriber is full, the publisher drops the oldest event, and the next event tells the loss (`dropped`).
+  - The new error code `TOPIC_NOT_FOUND` (404) is for a topic outside the topic rule. A topic that `allow` refuses gives `FORBIDDEN` with the reason `topic-not-allowed`.
+  - The `onEvent` option of `WebSocketTransport` is now a wrapper: it gets each event of each subscription of the transport. A raw `event` frame still goes to it.
+  - The WebSocket client does not warn about a `stream` frame of a call that it cancelled. The WebSocket server ignores a late `stream`, `response` or `error` frame of a call that it stopped. Before, it answered such a frame with an `error` frame.
+- ab77f75: Sequence numbers of the stream frames (`documentation/WIRE-PROTOCOL.md`). Before, a duplicate `stream` frame gave a duplicate item. The change is additive, and the protocol stays v1.
+  
+  - Each `stream` frame has the field `seq`: 1 in the first frame of a call, and one more in each next frame. The end frame has a number too. Each NDJSON line `item` or `done` has the same field.
+  - The receiver drops a frame whose `seq` is not above the last one. Thus a duplicate frame or a late frame gives no item.
+  - A gap does not end the stream, because the delivery is at most once. The next item has the number of lost frames in the metadata key `stream-gap` (`STREAM_GAP_METADATA_KEY`), and the console gets a warning. The receiver gives back the credit of the lost frames.
+  - A frame without `seq` comes from an older peer. The receiver takes it as before.
+  - The rules apply to WebSocket in the two directions, the message port and HTTP with NDJSON. The local transport has no frames.
+  - `WebSocketMessage` has the optional field `seq`.
+- a47bee6: The hosts serve the remote events (`documentation/EVENTS.md`).
+  
+  - `hostPolicy()` takes the option `events`: the topic rule, with an optional bus, `allow` function and buffer. `procedureServerOptions()` gives it to the `ProcedureServer`, which pins `events.subscribe`. `HostPolicy` has the new field `events`.
+  - `procedure-server` and `mark serve` have the option `--events <patterns>`. The option is repeatable, and it takes comma-separated patterns: exact names, prefixes (`orders.*`) or `*`. `parseTopicPatterns()` checks them.
+  - `mark --server` has the same option `--events`.
+  - These hosts always serve `events.subscribe`. Without `--events`, they serve no topic, and each subscription gets `TOPIC_NOT_FOUND`. The bus is the global bus, which is also the `ctx.bus` of their procedures.
+  - The MCP server serves no events, because its transport carries only MCP.
+  - `mark --port`, `--host`, `--transport` and `--events` without `--server` are an error now, also before `serve` and `mcp`. Before, `mark --port 4000 serve` ignored `--port`.
+- bc74091: Two changes for the contract of `CollectionStorage`:
+  
+  - The collection procedure `collections.<name>.get` takes the input `found: true`. Then its output is `{ found, value }`, so a stored `null` and a missing id give different outputs in one call. Without `found`, the output is the item, as before.
+  - `ProcedureServer.stop()` and `closeStorages()` close each storage one time, also a storage of two collections, and remove the storages from the server. Before, a server that started again used the closed storages.
+
+### Patch Changes
+
+- 0f75644: The dependencies go to `@mark1russell7/algebra` `^0.7.0` and `@mark1russell7/service` `^0.4.1`. A host that uses algebra 0.7.0 gets one copy of it. Algebra 0.7.0 only adds to 0.6.0.
+
+## 0.4.0
+
+### Minor Changes
+
+- d919678: `ERROR_REGISTRY` agrees with `ERROR_CODES` of the wire protocol. `ERROR_CODES` is the source: the registry has an entry for each code of the protocol, and that entry takes its HTTP status and its retry flag from `ERROR_CODES`. Changes that a caller sees:
+  
+  - `TIMEOUT` has the status 504 and is not retryable (before: 408, retryable).
+  - `INTERNAL_ERROR` is not retryable (before: retryable).
+  - `ABORTED` has the status 499 (before: none).
+  - The codes of the protocol that the registry did not have (`HANDLER_ERROR`, `VALIDATION_ERROR`, `UNAVAILABLE` and the others) have entries, with the meaning of the protocol as their texts.
+  - `errorCodeToHTTPStatus` gives the status of `ERROR_CODES` for a code of the protocol, for example 504 for `TIMEOUT`.
+- b2a1a1e: A handler error keeps its own code on each face, as `WIRE-PROTOCOL.md` says. `errorResponse` (the server, and through it HTTP, WebSocket and MCP) and `LocalTransport` give the code of the thrown error when it is a string. Before, they changed it to `HANDLER_ERROR` for a value, and `LocalTransport` also for a stream. An error without a code is still `HANDLER_ERROR`.
+  
+  The HTTP status follows the code: a code of `ERROR_CODES` gets its status, another code of `ERROR_REGISTRY` gets the status of the registry, and each other code gets 500. For example, a handler error with the code `UNAVAILABLE` now gives 503.
+- 057d69a: `client.import` gives `success: true` only for a procedure reference: an object whose `$proc` is a list of texts. Other JSON, for example `42`, an object without `$proc` or a list of calls, gives `success: false` and an error. Before, each JSON value gave `success: true`.
+- 057d69a: `client.isObject` gives true only for a plain object: an object literal, or an object without a prototype. A `Date`, a `Map` or another class instance gives false. Before, each object that is not an array gave true.
+- 057d69a: `client.stringifyJson` and `client.export` refuse a missing value (undefined) with `VALIDATION_ERROR`, because undefined has no JSON text. Before, the call failed with `OUTPUT_VALIDATION_ERROR`. The `value` field of `client.stringifyJson` and the `procedure` field of `client.export` are now required. A value without JSON text, for example a function, gives an error from `client.stringifyJson` and an error result from `client.export`.
+- 057d69a: `client.mod` refuses `b = 0` with the error `Division by zero`, as `client.divide` does. Before, it gave `NaN`.
+- afd6a19: The `client-registry` command is removed. `mark` finds the packages, and the `/register` entries register them.
+- 057d69a: `client.unique` compares the items by value: two equal JSON values are duplicates. An array compares by its items in order, and a plain object by its fields in any order. With `key`, the values of the field compare by value too. The result keeps the first item of each value, in the order of the input. Before, objects and arrays compared by reference, so two equal objects both stayed.
+- 4e751da: The new entry `@procedures/client/node` exports a `WebSocketTransport` whose sockets are of the `WebSocket` class of the `ws` package. A Node application gets `terminate()` without the option `webSocket`: a drop by the heartbeat or by the time limit of the handshake ends the socket at once. The option `webSocket` still gives another class. The other entries keep the global `WebSocket`, and the browser entry does not import `ws`.
+  
+  A socket of the `ws` package that closes during its handshake emits an error. Before, the transport removed its error handler first, so a handshake timeout or a `close()` during the handshake ended the process with an uncaught error. The transport now keeps a handler that ignores that error.
+
+### Patch Changes
+
+- 057d69a: The description of the `match` field of `client.filter` lists all 11 modes. Before, it did not name `gt`, `gte`, `lt` and `lte`.
+- 057d69a: The input schemas of `client.not`, `client.all`, `client.any` and `client.none` say "values", not "values or calls". These procedures do not run a call in their operands: a call is an object, thus truthy. In a program, the calls of their input run first, as for each procedure that is not control flow. `client.and` and `client.or` run calls. The behavior does not change.
+
 ## 0.3.0
 
 ### Minor Changes

package/README.md

@@ -27,7 +27,7 @@ The core depends on no other package of the repository. The procedure components
 - **Type-Safe Procedures**: Schema-validated procedures with auto-discovery and registry management
 - **Middleware System**: Composable middleware for retry, cache, timeout, circuit breaker, rate limiting, auth, tracing, and validation
 - **Procedure-as-Data**: JSON programs of calls, references and control flow ([PROGRAM-FORMAT.md](../documentation/PROGRAM-FORMAT.md))
-- **Event Bus**: Pub/sub messaging with async iteration support
+- **Event Bus**: Pub/sub messaging with async iteration support. A peer subscribes to the bus of another peer over each transport ([EVENTS.md](../documentation/EVENTS.md))
 - **Error System**: 70+ error codes with rich metadata and recovery suggestions
 
 ## Installation
@@ -267,8 +267,14 @@ interface WebSocketTransportOptions {
     timeout?: number;    // default: 5000ms
     misses?: number;     // default: 1; pings without a pong that drop the connection
   };
+  webSocket?: WebSocketClass;  // the class of the sockets; default: the global WebSocket
 }
 
+// In Node, import the transport of the Node entry. Its sockets are of the `ws` package, so a
+// drop ends a socket at once with terminate(). The global WebSocket has only close(): with a
+// dead peer, its socket stays open until the kernel stops it (924.6 s in the cells of R36).
+import { WebSocketTransport } from "@procedures/client/node";
+
 // The server pings each connection, and closes one that misses 2 pongs (option heartbeat)
 new WebSocketServerTransport(server, { server: http, heartbeat: { interval: seconds(30), misses: 2 } });
 
@@ -278,6 +284,17 @@ new WebSocketServerTransport(server, { server: http, heartbeat: { interval: seco
 const transport = new WebSocketTransport({ url, serve: new ProcedureServer({ registry: uiRegistry, autoRegister: true }) });
 const toClient = new Client({ transport: wsServer.transportFor(connectionId) });   // or peer.clientFor(id)
 await toClient.call({ service: "ui", operation: "notify" }, { text: "Build done" });
+
+// Remote events (documentation/EVENTS.md): subscribe to topics of the bus of the server. The
+// function gives the unsubscribe. After a drop, the transport resubscribes on the new connection.
+const unsubscribe = transport.subscribe("orders.created", (order, event) => console.log(event.seq, order), {
+  onSubscribed: (topics) => console.log("subscribed", topics),
+  onError: (error) => console.error(error.code),   // TOPIC_NOT_FOUND, FORBIDDEN, NOT_FOUND
+});
+await unsubscribe.ready;
+unsubscribe();
+// The same events as an async iterator: a break or the signal unsubscribes
+for await (const event of transport.events(["orders.created", "orders.paid"], { signal })) console.log(event.topic, event.data);
 ```
 
 #### `LocalTransport`
@@ -494,6 +511,39 @@ const unsubscribe = bus.on("event", handler)
 unsubscribe()
 ```
 
+#### Remote events
+
+A peer subscribes to the topics of the bus of another peer ([EVENTS.md](../documentation/EVENTS.md)). A subscription is a streaming call of the procedure `events.subscribe`, so it works over WebSocket (in the two directions), HTTP with NDJSON, a message port and the local transport.
+
+```typescript
+// The publisher: a ProcedureServer with the option events
+const bus = createEventBus()
+const server = new ProcedureServer({
+  registry,
+  autoRegister: true,
+  events: {
+    bus,                                  // the procedures of the server get it as ctx.bus
+    topics: ["orders.*", "system:*"],     // the topic rule: an exact name, a prefix with *, or *
+    allow: (topic, call) => !topic.startsWith("system:") || call.capability.clearance === "admin",
+    buffer: 1024,                         // the events that one subscription holds for a slow subscriber
+  },
+})
+server.publish("orders.created", { order: 1 })   // the same as bus.emit()
+
+// The subscriber, over each transport
+const unsubscribe = subscribeRemote(transport, "orders.created", (order, event) => console.log(order, event.seq))
+for await (const event of remoteEvents(transport, ["orders.created"], { signal })) console.log(event.data)
+
+// The server subscribes to the bus of a client: the client serves it with `serve`
+new WebSocketTransport({ url, serve: new ProcedureServer({ events: { bus: clientBus, topics: ["ui.*"] } }) })
+wsServer.subscribe(connectionId, "ui.clicked", (click) => console.log(click))
+```
+
+- The delivery is at most once, in the order of `emit()`. `seq` numbers the events of a subscription, and `dropped` tells the events that the publisher dropped for a slow subscriber.
+- A topic outside the topic rule gives `TOPIC_NOT_FOUND`. A topic that `allow` refuses gives `FORBIDDEN` with the reason `topic-not-allowed`. The procedure reads the resource `events`, so the capability of the host applies to it.
+- `ProcedureServer.stop()` ends each subscription with `UNAVAILABLE`, and the subscriber resubscribes later.
+- `events.subscribe` is not in a registry: it is not an MCP tool.
+
 ### Server
 
 ```typescript
@@ -544,6 +594,7 @@ interface ProcedureServerOptions {
   registry?: ProcedureRegistry;
   storage?: StorageConfig;
   transports?: ServerTransport[];
+  events?: EventPublisherOptions | EventPublisher;   // the remote events: refer to "Remote events"
 }
 ```
 
@@ -587,22 +638,26 @@ import { ... } from "@procedures/client/procedures";
 import { ... } from "@procedures/client/events";
 ```
 
-## Procedure Auto-Discovery
+| Entry | Use |
+|---|---|
+| `@procedures/client` | Node. The browser entry and the server transports. Its `WebSocketTransport` uses the global `WebSocket`. |
+| `@procedures/client/browser` | A browser bundle. It does not import `http` or `ws`. |
+| `@procedures/client/node` | Node. A `WebSocketTransport` whose sockets are of the `ws` package: a drop ends a socket at once. |
+| `@procedures/client/register` | It registers the core procedures in the global registry. |
 
-Packages can declare procedures in `package.json`:
+## Procedure Discovery
+
+A package declares its procedures in `package.json`:
 
 ```json
 {
   "client": {
     "procedures": "./dist/register.js"
-  },
-  "scripts": {
-    "postinstall": "client-registry announce"
   }
 }
 ```
 
-When installed, the package announces its procedures in `.client-registry.json`. Run `client-registry discover` to generate a file that imports the register entry of each announced package.
+The `mark` CLI finds the packages by the `client.procedures` field. An app imports the `/register` entry of each package that it uses, for example `import "@procedures/fs/register"`.
 
 ## Type-Safe Middleware