@procedures/client 0.35.0 → 0.36.0

package/CHANGELOG.md

@@ -1,5 +1,72 @@
 # @procedures/client
 
+## 0.36.0
+
+### Minor Changes
+
+- f1fae86: The publisher of the remote events filters the change events of a collection row by row, with the rule of its rows (roadmap CLI-78). Thus a principal with a rule of the rows can subscribe to a topic of a collection. It gets only the ids of the rows that it can read.
+  
+  MINOR (breaking, pre-1.0): this change has a new default. Before, each subscriber got each id of an event, and `withheld` named the rows that the rule refused. With this change, a row that the rule refuses is not in the event of that subscriber. Before 1.0.0, a changed default raises the MINOR.
+  
+  - The rule is `admission.admit` of `createCollectionProcedures()`. The reads apply the same rule. The publisher finds it in the registry of the call, so it also filters the events of other publishers on the topic.
+  - A changed row that the rule admits stays in `changed`. A changed row that the rule refuses, and a deleted row, go to `deleted` only when the rule admitted the row before the change. The subscriber does not get the id of a row that it could not read.
+  - The rule examines the rows of the event: the row after the change, from the change log, and the row before the change. While a subscriber has a filter, each write of the procedures reads the rows before it with one `getBatch()`. The publisher does not read a storage.
+  - When the event has no image of a row, or the rule fails, the subscriber gets a hint without ids and without `since`. It then reads the changes through the rule of the reads.
+  - An event with `reset: true` and an event without ids pass, without ids.
+  - When the subscriber can read no row of an event, it gets no event. Its next event gets the `since` of the last event that it got, so it applies the next delta without a read. The `seq` of the events counts only the events that the subscriber gets, so the subscriber sees no gap.
+  - With values, an event has only the values of the rows that the rule admits. `withheld` names only the rows that the output filter of the host (`admitItem`) refused.
+  - The new optional `admission.admitsAll(invocation, partition)` tells that a subscriber reads each row of its partition. The events of that subscriber then pass as before, with no filter and no read before a write.
+  - At a re-check after a change of the rights (`rightsChanged()`, roadmap CLI-44), each subscriber with a filter gets a `reset` hint, so it reads its rows again. A subscriber whose `admitsAll` changes at a re-check also gets a `reset` hint.
+  - A delta of `changes` gives only ids that the caller can read. Before, it put the id of each refused row in `deleted`. A delta with a refused row or a deleted row becomes a full answer (`full: true`) of the admitted rows. The copy of the caller then drops each other row. When `admitsAll` admits the caller, the delta stays as before.
+  - `admittedStorage()` has a new optional parameter `partition`. `admitsAll` gets it.
+  - The rules get an invocation of the collection, not the call of `events.subscribe`. It has the principal, the capability, the metadata and the signal of the subscription. Its path is the path of `changes`, and its input has the partition of the topic. Thus a rule that reads `invocation.input.partition` works for the events too.
+  - `rightsChanged(change, scope)` of `EventPublisher` has a new optional parameter `scope`: `{ collection?, partition? }` or a list of them. Each field is a name or a test of the name, for example a test of each partition of a tenant. A test that throws matches.
+  - With a scope, only the topics of the scope with a filter get a `reset` hint and a new `admitsAll`. Without it, the change touches each topic, as before. The source `recheck.rightsChanges` can give a scope to its listener too. The new types are `RightsScope` and `RightsScopes`.
+  - New exports: `withHintImages(event, images)`, `hintImagesOf(event)` and the types `HintImages` and `HintRowState`. A publisher of change events, for example a watcher of a change stream, gives the rows before and after the change. Without them, a subscriber with a filter gets a hint without ids.
+  
+  Migration:
+  
+  - A subscriber that read `withheld` to find the rows that the rule refused: these rows are not in the event. A row that the subscriber read before the change and that the rule refuses after it comes as a delete.
+  - A reader of `changes` with a rule of the rows: apply a full answer as a replacement of the copy. The sync of `@procedures/client-store` does this. A full answer comes after each delete, so give `admitsAll` to the principals with full rights.
+  - A host whose `allow` of `events.subscribe` refused a principal with a rule of the rows can admit it.
+  - A host with full rights for some principals: give `admission.admitsAll`. Without it, the publisher examines each row of each event for each subscriber. Also, each write reads its rows before the change while such a subscriber is open.
+  - Graph: give `admitsAll` for the principals that read a whole partition. Then admit the principals with a `when` condition in `allow`.
+  - The site, MiniMongo and jarvis: no change, if their collections have no `admission.admit`. Then their events do not change.
+
+## 0.35.1
+
+### Patch Changes
+
+- 9f0405d: `watchStorageChanges()` publishes the change stream of a `MongoDocumentStorage` collection as collection hints, and `versionOfStored()` gives the version of a raw stored document (roadmap CLI-77).
+  
+  PATCH: the change only adds exports and new optional options, and keeps each behavior of 0.35.0 (the watcher is not released yet). Before 1.0.0, a new feature raises the PATCH. The fixed group shares one version, so this rides the next release (0.35.0 when it is not out yet).
+  
+  - `versionOfStored(doc, form?, partition?)` gives the version that `storage.versionOf(id)` gives for a stored document: `_seq` as text, `0` for a document without `_seq`, and `undefined` for a tombstone or a document without an id. It gives the value that `versionOf()` reads.
+  - `watchStorageChanges(storage, publish, options?)` watches the change stream of the collection. It publishes each change on the topic of the collection procedures, `collectionChangeTopic(name, partition)`, with the partition from `storage.form.partitionOf()`. The event is a `CollectionChangeEvent` without `since` (a hint), with the id in `changed` or `deleted` and the version of the item in `versions`. A tombstone and a delete are deletes. It returns `{ ready, stop() }`.
+  - With `ownWrites: "skip"` (the default), the watcher skips the events of the writes of this process through the storage, because the collection procedures publish them already. Each write of the storage notes its sequence numbers while a watcher is open. `ownWrites: "publish"` publishes each change.
+  - With `rawWrites: "adopt"` (the default), the watcher gives a raw write a new `_seq` through the write path of the storage, so `changes(since)` has it. It reads from the event alone whether a write is raw. It gives back the storage fields and the partition fields that a raw replace took, and never changes a field of the item. The write matches only the state that the event saw, so one of two racing watchers wins and the other does nothing. The event of that write publishes the hint with the new version; a raw write does not carry its old version. The adoptions of a watcher wait in one bounded queue, one at a time, in batches that take one block of numbers. The watcher keeps the ids of the raw writes that do not fit in a full queue or whose last try failed (up to 10,000), and adopts them again when the queue is empty, with the documents then. A failure that can go away tries again after the backoff. A failed last try, a second failure, or an id over that limit gives the partition a reset hint. The adoption is exact only with `changeStreamPreAndPostImages`. With the images before the change (`changeStreamPreAndPostImages`), a raw delete becomes a tombstone; without them, it is a hint without a version. A prune of tombstones publishes nothing. `rawWrites: "ignore"` writes nothing and publishes a hint without a version. A read-only user gets the same hint, with one log.
+  - With `sweep` (on by default with `rawWrites: "adopt"`), the watcher adopts each document without `_seq` after each gap, in pages of the index of `_id`. `sweepAtOpen: true` (off by default) also sweeps when the stream opens: the first watcher on a legacy collection adopts every legacy document, and the next delta of every subscriber is the whole collection. A document that the sweep cannot place goes to the log once, and the next sweeps leave it.
+  - `ready` rejects when the first open fails with an error that does not go away (a single server, a refused access, a wrong option). The watcher asks the server with `hello` first, so on a single server `ready` rejects at once. The watcher tries again after other errors.
+  - After an error of the stream, the watcher opens it again after a backoff (`backoff`, on the injected `clock`), from the resume token of the last event. When the server does not know the token any more, or the collection was dropped or renamed, it publishes a reset hint (no ids, `reset: true`) on the plain topic, on each partition of the counters, on each partition that it saw, and on each channel of the bus of the collection. The option `gapPartitions` adds more.
+  - `@procedures/client`: `CollectionChangeEvent` has the new optional field `versions`, `[{ id, version }]`: the version of each changed item, as `versionOf(id)` of the storage gives it. A publisher that knows the versions sets it. The new optional field `reset`: `true` tells the subscriber to read the collection in full, with `changes` and no `since`. PATCH: new optional fields of an output type, and before 1.0.0 a new feature raises the PATCH.
+  - `@procedures/store`: `EventsChangeFeed` reads `versions` and `reset` of an event, and `HybridStorage` does a full pull for a hint with `reset: true`. PATCH: a fix, so that a store does not miss the changes of a reset.
+  - `storageChangeOf(change, form?)` reads the change of an item from a change event, for an app that watches the collection itself.
+  - Fix: `ViewMaterializer` leaves `_seq` and `_tomb` of a source out of the content of a row, before the hash. A row is no write of a storage, so the watcher of the target does not read it as a raw write. The other fields of the source stay in the row.
+- f5c2407: PATCH: a new optional option whose default keeps the old behavior. The error policy of a server (roadmap CLI-80).
+  
+  - The new option `errors` of `Server` and `ProcedureServer` tells what an error tells the caller. A host gives it with `serverOptions`. `Server.handle()` applies it, so each face sends the same error: HTTP, WebSocket, MCP and a message port.
+  - `expose: "all"` is the default. Each error goes to the caller as before.
+  - `expose: "coded"` sends an error with a known code as it is. Each other error goes to the caller as `INTERNAL_ERROR` with the message `internal error` and its real retryable flag. It keeps only the metadata keys of the new option `keepMetadata`: by default the trace keys `traceparent` and `tracestate`. The server gives the real error to `onError`, or to `console.error` without `onError`.
+  - A known code is a code of `ERROR_CODES` or of `ERROR_REGISTRY`, for example `NOT_FOUND` or `CONFLICT`. The new function `isKnownErrorCode(code)` does this check. `ECONNREFUSED` and a numeric code of MongoDB are not known codes.
+  - The option `codes` adds the codes of the application that go out as they are, for example `QUOTA_EXCEEDED`.
+  - The codes `OUTPUT_VALIDATION_ERROR` and `NO_OUTPUT` tell of a failure of the server, so `"coded"` hides them, with `INTERNAL_ERROR`, `HANDLER_ERROR` and `UNKNOWN`.
+  - The server applies `map(error, { path, transport })` first, with the real error of its own class. Its result goes to the caller as it is, with its `retryAfterMs` and its conflicts. `undefined` gives the error to `expose`.
+  - The policy applies to an error before a stream and to the error that ends a stream. A nested call of `ctx.client` keeps the real error.
+  - The new optional field `transport` of `ServerRequest` names the face of a request. The HTTP, WebSocket, message port and MCP faces set it.
+  - The last-resort catch of each face applies the same policy when `handle()` throws. This is true for the HTTP, WebSocket, message port and MCP faces, and for the tracked call of a host (`@procedures/peer`). The new function `lastResortError(handler, error, request)` and the new optional method `RequestHandler.exposeError()` do this. The logger gets each hidden error one time.
+  - The new types are `ServerErrorPolicy`, `ExposedError` and `ErrorCall`. The new exports are `exposeError`, `lastResortError`, `HIDDEN_ERROR_MESSAGE` and `DEFAULT_KEPT_METADATA`.
+  - An MCP result for an unknown tool has `_meta` with the code `NOT_FOUND`.
+
 ## 0.35.0
 
 No changes in this release.