@procedures/client 0.6.0 → 0.8.0

package/CHANGELOG.md

@@ -1,5 +1,71 @@
 # @procedures/client
 
+## 0.8.0
+
+### Minor Changes
+
+- 4b09a03: `debounceChannel()` and `throttleChannel()` take a `Duration`, as every other time of the client does since 0.3.0. A plain number of milliseconds throws a `TypeError`. A new option `clock` gives the clock of their timers, for example a fake clock in a test. MINOR: a breaking change of a signature (the API is before 1.0.0).
+  
+  Migration: `debounceChannel(bus, "in", "out", 300)` becomes `debounceChannel(bus, "in", "out", ms(300))`.
+- 698e24d: Three gaps of the remote events (`documentation/EVENTS.md`) are closed. The option `backoff` of a subscription changes, so this release is breaking for a caller that gives it.
+  
+  - **Shared subscriptions.** The subscribers of the same topics with the same options on one transport share one call: one `request` on the wire, with a count of its subscribers. Only the last unsubscribe sends the `cancel`. Before, each `subscribeRemote()`, `remoteEvents()` or `transport.subscribe()` made its own call. Each subscriber keeps its own id, its own `ready` and its own queue of at most 1024 events. A late subscriber gets no replay: its `ready` resolves at once, and its `seq` starts at 1. An error or an end of the shared call goes to each subscriber. `WebSocketServerTransport` shares the subscriptions of one connection.
+  - **The end at `clear()`.** `clear()` and `clearAll()` of the bus of the publisher end each subscription of a cleared topic. The subscriber first gets the events of the queue, then the error `TOPIC_CLEARED` (new protocol code, HTTP 410, not retryable) through `onError`, `ready` or the iterator. Before, the listeners went, and the subscriber saw no end. The new option `onClear` of `EventBus.on(channel, handler, { onClear })` (type `EventListenOptions`) gives the same notice to a local handler.
+  - **The limit of the resubscribe.** The new option `maxAttempts` (default 10) is the number of resubscribe tries without a confirmation. When the tries run out, the subscription ends with a `SubscriptionError` that has the code of the last failure. Before, a subscription over HTTP resubscribed forever. The WebSocket transport uses `reconnect.maxAttempts` (default `Infinity`), so its behavior stays. The new option `clock` is the clock of the backoff, for a fake clock in a test. `SubscriptionHooks` has the new fields `maxAttempts` and `clock`.
+  
+  Migration: the option `backoff` is an object of durations (type `ResubscribeBackoff`), with the names of the `reconnect` options of `WebSocketTransport`: `initialDelay` (default `ms(250)`), `maxDelay` (default `seconds(30)`), `backoffMultiplier` (default 2) and `jitter` (default 1). Before, it was a function of milliseconds. A function or a plain number throws a `TypeError` at the call. For the old behavior over HTTP, give `maxAttempts: Infinity`.
+  
+  ```ts
+  // Before
+  subscribeRemote(transport, "orders.created", handler, { backoff: (attempt) => Math.min(30_000, 500 * 2 ** (attempt - 1)) });
+  // Now
+  subscribeRemote(transport, "orders.created", handler, { backoff: { initialDelay: ms(500), maxDelay: seconds(30), jitter: 0 }, maxAttempts: Infinity });
+  ```
+- d25805a: `ProcedureServer.useStorage` with `close` waits for the calls on the old storage before it closes it. Before, it closed the old storage at once, and a call that started on it could fail as closed.
+  
+  MINOR: the call of `useStorage` with `close` can now take up to 30 seconds to resolve. That is a change of behaviour that a caller can see. Give `grace: ms(0)` for the old behaviour.
+  
+  - The new calls use the new storage at once. The server waits only for the calls that took the old storage. A stream holds its storage until its end.
+  - The new option `grace` of `UseStorageOptions` is the longest wait, a `Duration`. A plain number throws a `TypeError`. The default is 30 seconds, `TIMER_DEFAULTS.server.storageGraceMs`. 0 closes at once.
+  - When the time ends, the server closes the old storage. A call that still uses it then fails with the error of a closed storage.
+  - The new option `clock` of `ProcedureServerOptions` is the clock of this wait. A test gives a fake clock.
+
+### Patch Changes
+
+- 4b09a03: New `callerOf(...modules)` in the root and the browser entry. It gives a function that runs the procedures of the modules and gives one value. It makes the registry of the modules at the first call and does not use the global registry. An app wrote the three steps itself before: `createRegistry`, `invokePath` and `outputValue`. PATCH: a new export (the API is before 1.0.0).
+- aa6e3aa: `ApiStorage` has `getIds()`. A full pull through it does not read each entry for the ids.
+  
+  - The collections have the procedure `collections.<name>.ids`. It gives the ids in the order of the entries, without the values. Its effects read the collection and write nothing. A storage without `getIds()` gives the ids of its entries.
+  - The `CollectionStorage` of the core has the optional method `getIds()`, as the `CollectionStorage` of `@procedures/collections` has.
+  - `ApiStorage.getIds()` uses the procedure `ids`. An older server without it answers `NOT_FOUND`. Then the storage reads the ids from the entries, and it does not ask for `ids` again.
+  - `ApiStorageOptions.operations` takes `ids`, the name of the operation. The default is `ids`.
+- 4b09a03: `bus.once()` rejects with an `AbortError` when `clear()` or `clearAll()` removes its channel. Before, its promise never settled.
+- 600c497: A server on port 0 tells its real port.
+  
+  - New `HttpServerTransport.port()` returns the port that the server listens on. With the option `port` 0, the system gives a free port, and the function returns that port.
+  - `peer.getEndpoints()` gives the real port of each HTTP and WebSocket endpoint. Before, a peer on port 0 gave the address `:0`.
+  - The tests of the body and the stop of the peer listen on port 0. Before, a helper found a free port and closed it before the peer took it, so one CI run failed with `EADDRINUSE`.
+- 7c2e7f7: The `onAfterRetry` hook of `createRetryMiddleware` hears of a failure when the reader stops at the error item. A `Client` with `throwOnError` (the default) stops there. Before, the hook ran after the yield of that item, so it ran only for a success. Now it runs when the call ends: after the last item, or when the reader stops. A thrown error and an abort still do not call it. PATCH: a bug fix.
+- 90ea741: A procedure can declare its rules under a root of the files in `metadata.fsRoot` (the new type `FsRootRules`): the inputs that name a path, or the reason that a root cannot confine it. A handler gets `ctx.caller`, the procedure whose handler made the nested call. The framework sets it from the invocation, so an input cannot change it (roadmap R33).
+- c4cfa34: `ProcedureServer.useStorage` with `close` keeps an old storage open while another collection still uses it. The server closes it when the last collection leaves it. Before, the close broke the calls of the other collection.
+
+## 0.7.0
+
+### Minor Changes
+
+- 841d083: An NDJSON stream over HTTP gets keep-alive lines. Before, a proxy or a client with an idle timeout (for example 300 s) closed a long procedure that was quiet for that time.
+  
+  - `HttpServerTransport` writes a keep-alive line, a blank line, after each quiet time of the new option `keepAlive` (default `seconds(15)`, `TIMER_DEFAULTS.httpServer.keepAliveMs`). Each line starts the time again. No keep-alive line comes after the `done` line, an `error` line, an abort or a closed connection. `keepAlive: false` turns it off.
+  - The new option `clock` of `HttpServerTransport` is the clock of the keep-alive timers. A test gives the `fakeClock` of `@mark1russell7/service/testing`.
+  - A keep-alive line has no `seq`, so the sequence check of the reader sees no gap. `WIRE-PROTOCOL.md` specifies the line: it is an additive change of protocol v1, because each reader skipped a blank line before.
+  - New `writeNdjsonStream(res, items, options?)` in `@procedures/client/node`. It writes the NDJSON stream response of protocol v1 for a `ServerResponse` of `node:http`, without Express. It writes the headers, the `item`, `done` and `error` lines with `seq`, and the keep-alive lines. It waits for `drain` when the buffer is full, and it stops when the connection closes or `signal` aborts. Its options are `signal`, `keepAlive`, `errorOf` and `clock`. `HttpServerTransport` uses it, so the two write the same bytes.
+  - `HttpTransport` gives each item of an NDJSON stream the metadata key `stream` with the value `true` (the new constant `STREAM_ITEM_METADATA_KEY`). The error item that ends the stream has it too. The key does not cross the wire, and the client removes a key `stream` that the peer sent.
+  - When the client goes away while the handler is quiet, the writer of the stream stops at once. Before, it waited for the next item of a handler that ignores its signal.
+
+### Patch Changes
+
+- 389b054: A token check no longer tells the length of the token through its time. The peer compares the SHA-256 digests of the two tokens with `timingSafeEqual`. The `tokenMatches` of the client, which runs in a browser too, compares each character of the expected token for each presented token. Before, a token of another length returned at once. The re-audit of jarvis found it.
+
 ## 0.6.0
 
 No changes in this release.

package/README.md

@@ -179,6 +179,18 @@ const getUserProcedure = defineProcedure({
 registerProcedures([getUserProcedure]);
 ```
 
+### Procedures of a package in the same process
+
+`callerOf(...modules)` gives a function that runs the procedures of the modules. It makes the registry of the modules at the first call, and it does not use the global registry. Each call gives one value: the output, or the last item of a stream.
+
+```typescript
+import { callerOf } from "@procedures/client";
+import { ghModule } from "@procedures/gh";
+
+const gh = callerOf(ghModule);
+const runs = await gh<RunList>(["gh", "run", "list"], { repo: "owner/name" });
+```
+
 ## API Reference
 
 ### Client
@@ -289,7 +301,7 @@ await toClient.call({ service: "ui", operation: "notify" }, { text: "Build done"
 // function gives the unsubscribe. After a drop, the transport resubscribes on the new connection.
 const unsubscribe = transport.subscribe("orders.created", (order, event) => console.log(event.seq, order), {
   onSubscribed: (topics) => console.log("subscribed", topics),
-  onError: (error) => console.error(error.code),   // TOPIC_NOT_FOUND, FORBIDDEN, NOT_FOUND
+  onError: (error) => console.error(error.code),   // TOPIC_NOT_FOUND, FORBIDDEN, NOT_FOUND, TOPIC_CLEARED
 });
 await unsubscribe.ready;
 unsubscribe();
@@ -531,7 +543,10 @@ const server = new ProcedureServer({
 server.publish("orders.created", { order: 1 })   // the same as bus.emit()
 
 // The subscriber, over each transport
-const unsubscribe = subscribeRemote(transport, "orders.created", (order, event) => console.log(order, event.seq))
+const unsubscribe = subscribeRemote(transport, "orders.created", (order, event) => console.log(order, event.seq), {
+  maxAttempts: 10,                                                   // the resubscribe tries without a confirmation (default 10; WebSocket: reconnect.maxAttempts)
+  backoff: { initialDelay: ms(250), maxDelay: seconds(30), backoffMultiplier: 2, jitter: 1 },   // Durations: a plain number throws a TypeError
+})
 for await (const event of remoteEvents(transport, ["orders.created"], { signal })) console.log(event.data)
 
 // The server subscribes to the bus of a client: the client serves it with `serve`
@@ -541,7 +556,9 @@ wsServer.subscribe(connectionId, "ui.clicked", (click) => console.log(click))
 
 - The delivery is at most once, in the order of `emit()`. `seq` numbers the events of a subscription, and `dropped` tells the events that the publisher dropped for a slow subscriber.
 - A topic outside the topic rule gives `TOPIC_NOT_FOUND`. A topic that `allow` refuses gives `FORBIDDEN` with the reason `topic-not-allowed`. The procedure reads the resource `events`, so the capability of the host applies to it.
-- `ProcedureServer.stop()` ends each subscription with `UNAVAILABLE`, and the subscriber resubscribes later.
+- `ProcedureServer.stop()` ends each subscription with `UNAVAILABLE`, and the subscriber resubscribes later, at most `maxAttempts` times without a confirmation.
+- `clear()` or `clearAll()` of the bus ends each subscription of a cleared topic with `TOPIC_CLEARED`. The subscriber does not resubscribe. `bus.on(channel, handler, { onClear })` gives the same notice to a local handler.
+- The subscribers of the same topics with the same options on one transport share one call. The last unsubscribe sends the cancel. A late subscriber gets no replay: its `ready` resolves at once, and its `seq` starts at 1.
 - `events.subscribe` is not in a registry: it is not an MCP tool.
 
 ### Server
@@ -567,13 +584,52 @@ await server.stop()
 new HttpServerTransport(server: Server, options: HttpServerTransportOptions)
 
 interface HttpServerTransportOptions {
-  port: number;
-  hostname?: string;
+  app: Express;                  // required
+  httpServer?: HttpServer;       // an existing server: the transport does not listen
+  port?: number;                 // default: 3000
+  host?: string;                 // default: "127.0.0.1"
+  basePath?: string;             // default: "/api"
   urlStrategy?: HttpUrlStrategy;
-  cors?: boolean | CorsOptions;
+  cors?: boolean;
+  corsOptions?: { origin?: string | string[]; methods?: string[]; allowedHeaders?: string[]; credentials?: boolean };
+  bodyLimit?: number;            // default: 1 MiB
+  keepAlive?: Duration | false;  // default: seconds(15)
+  clock?: Clock;                 // the clock of the keep-alive timers
 }
 ```
 
+A client that accepts `application/x-ndjson` gets a stream as NDJSON lines ([WIRE-PROTOCOL.md](../documentation/WIRE-PROTOCOL.md)).
+
+- After each quiet time of `keepAlive`, the server writes a keep-alive line: a blank line. Thus a proxy or a client with an idle timeout does not close a long, quiet stream.
+- Each line starts the quiet time again. No keep-alive line comes after the end, an error, an abort or a closed connection.
+- `keepAlive: false` turns the keep-alive lines off.
+- The HTTP client skips each blank line. Each item that it reads from an NDJSON stream has the metadata key `stream: true` (`STREAM_ITEM_METADATA_KEY`).
+
+A server without Express writes the same lines with `writeNdjsonStream` of `@procedures/client/node`:
+
+```typescript
+import { createServer } from "node:http";
+import { seconds } from "@procedures/client";
+import { writeNdjsonStream } from "@procedures/client/node";
+
+async function* progress() {
+  for (let step = 1; step <= 3; step++) {
+    await new Promise((resolve) => setTimeout(resolve, 60_000)); // a quiet minute: 4 keep-alive lines
+    yield { step };
+  }
+}
+
+createServer((req, res) => {
+  // The headers, the item, done and error lines with seq, the keep-alive lines and the backpressure
+  void writeNdjsonStream(res, progress(), {
+    keepAlive: seconds(15),
+    errorOf: (error) => ({ code: "UNAVAILABLE", message: String(error), retryable: true }),
+  });
+}).listen(3000);
+```
+
+The promise of `writeNdjsonStream` resolves when the response ends. A closed connection or an abort of `signal` stops the stream, and the iterator of the items gets `return()`.
+
 #### WebSocket Server Transport
 ```typescript
 new WebSocketServerTransport(server: Server, options: WebSocketServerTransportOptions)
@@ -642,7 +698,7 @@ import { ... } from "@procedures/client/events";
 |---|---|
 | `@procedures/client` | Node. The browser entry and the server transports. Its `WebSocketTransport` uses the global `WebSocket`. |
 | `@procedures/client/browser` | A browser bundle. It does not import `http` or `ws`. |
-| `@procedures/client/node` | Node. A `WebSocketTransport` whose sockets are of the `ws` package: a drop ends a socket at once. |
+| `@procedures/client/node` | Node. A `WebSocketTransport` whose sockets are of the `ws` package: a drop ends a socket at once. `writeNdjsonStream`: the NDJSON stream response of `HttpServerTransport`, for a `ServerResponse` of `node:http`. |
 | `@procedures/client/register` | It registers the core procedures in the global registry. |
 
 ## Procedure Discovery