@procedures/collections 0.14.1 → 0.15.1

package/CHANGELOG.md

@@ -1,5 +1,53 @@
 # @procedures/collections
 
+## 0.15.1
+
+### Patch Changes
+
+- Updated dependencies [c78da2d]
+- Updated dependencies [d99b2d5]
+- Updated dependencies [534e4ea]
+- Updated dependencies [44fdd60]
+  - @procedures/client@0.15.1
+
+## 0.15.0
+
+### Minor Changes
+
+- c7198ca: Breaking (roadmap CLI-16, decision D11): the collection writes take an optional expected version. This is its own MINOR of the client group (D13).
+  
+  New:
+  
+  - `collections.<name>.set` and `delete` take `expectedVersion`, and each item of `setBatch` takes `expectedVersion`. The value is the version of the current item that the write expects. `null` expects no item.
+  - A conflict changes nothing. The call fails with the new protocol code `VERSION_CONFLICT` (HTTP 409, not retryable). The metadata key `conflicts` (and the field `conflicts` of the HTTP body) has the current item and its version for each id.
+  - `StorageItemWriteOptions.expectedVersion` (`set`, `delete`) and `StorageBatchWriteOptions.expectedVersions` (`setBatch`) in both copies of `CollectionStorage`. `VersionConflictError`, `versionConflictsOf()` and the other helpers come from `@procedures/client/query`, `@procedures/client` and `@procedures/collections`.
+  - The option `versionOf` of `InMemoryStorage`, `SqliteStorage` and `MongoDocumentStorage` gives the version of an item. The test and the write are one step: synchronous in memory, one transaction in SQLite, the transaction or a compare-and-set on `_seq` in MongoDB. `MongoStorage` refuses a precondition.
+  - `ApiStorage` sends the expected versions to the collection procedures, and gives a `VERSION_CONFLICT` back as a `VersionConflictError`.
+  - `HybridStorage` sends the base version of each queued write as its expected version. A conflict of the remote goes to `conflictResolution`, as a conflict of a pull. The new option `sendBaseVersions: false` stops it.
+  - `ClientError.metadata` gives the metadata of the error response.
+  - The contract suite `@procedures/collections/testing` has the cases of the preconditions, and the option `createWithVersions`.
+  
+  The changes of behavior:
+  
+  - A storage must test a precondition, or refuse the write. A storage without `versionOf` refuses a write with an expected version with a `TypeError` whose code is `PRECONDITION_NOT_SUPPORTED`. The contract suite tests it for each storage.
+  - A `HybridStorage` with `versionOf` now sends the base version of each queued write. A queued write over a change of another writer got through before. Now the remote refuses it with `VERSION_CONFLICT`, and `conflictResolution` decides: `remote` drops the queued write, `merge` sends the merged item, `local` sends the write again over the current version, and `error` throws `ConflictError` and keeps the queue. The remote must test the precondition: a server storage without `versionOf` refuses each push.
+  - `ApiStorage`: a write without an expected version writes as before, and gets no `VERSION_CONFLICT`. Only a caller that gives an expected version, or a `HybridStorage` with `versionOf` over it, can get one.
+  - `HybridStorage` with `versionOf`: a queued write that won before can now get the conflict path. The opt-out is `new HybridStorage(remote, { versionOf, sendBaseVersions: false })`: the push sends no expected version, and the last write wins, as before. The default is the new behavior.
+  - A write without an expected version writes as before: the last write wins.
+  
+  Migration:
+  
+  - **Site.** No change. The site does not write collections.
+  - **MiniMongo.** No change for a storage or a `HybridStorage` without `versionOf`. A `HybridStorage` with `versionOf` over `ApiStorage` needs a server storage with the same `versionOf` (for example `new InMemoryStorage({ versionOf })`), or the option `sendBaseVersions: false`.
+  - **Graph.** Graph keeps last-write-wins first (D11): it gives no expected version, so nothing changes. `GraphDocsStorage` must refuse a write with an expected version until it tests one. Add the guard to its `set`, `delete` and `setBatch`, and run the contract suite: its first case tests the refusal. To adopt the version later, stamp a version field on the server, give the storage a `versionOf` that tests it in the write, and give the same `versionOf` to the `HybridStorage` of the client. Then add `createWithVersions` to the contract run.
+  - **jarvis.** No change: jarvis gives no expected version. A storage of jarvis that implements `CollectionStorage` must refuse a write with an expected version, as for Graph.
+  - **Other storages.** Each class that implements `CollectionStorage` reads the new options of `set`, `delete` and `setBatch`. It tests them in the write, or it rejects the write with `preconditionNotSupported()`. It never ignores them.
+
+### Patch Changes
+
+- Updated dependencies [c7198ca]
+  - @procedures/client@0.15.0
+
 ## 0.14.1
 
 ### Patch Changes

package/README.md

@@ -94,14 +94,15 @@ interface CollectionStorage<T> {
   has(id: string): Promise<boolean>;
   size(): Promise<number>;
 
-  // Write Operations (the options give the origin of the write to the observers)
-  set(id: string, value: T, options?: StorageWriteOptions): Promise<void>;
-  delete(id: string, options?: StorageWriteOptions): Promise<boolean>;
+  // Write Operations (the options give the origin of the write to the observers, and the
+  // expected version of the current item: refer to "The precondition of a write")
+  set(id: string, value: T, options?: StorageItemWriteOptions): Promise<void>;
+  delete(id: string, options?: StorageItemWriteOptions): Promise<boolean>;
   clear(options?: StorageWriteOptions): Promise<void>;
   patch?(id: string, fields: StoragePatch, options?: StorageWriteOptions): Promise<T | undefined>;
 
   // Bulk Operations
-  setBatch(items: Array<[string, T]>, options?: StorageWriteOptions): Promise<void>;
+  setBatch(items: Array<[string, T]>, options?: StorageBatchWriteOptions): Promise<void>;
   deleteBatch(ids: string[], options?: StorageWriteOptions): Promise<number>;
   getBatch(ids: string[]): Promise<Map<string, T>>;
 
@@ -122,8 +123,40 @@ Each storage agrees on these points:
 - `query()` gives the result of `queryEntries()` for the entries of the storage. Refer to "The query language".
 - `observe()` gives one batch of changes for each write that changes the storage, before the write resolves. Each change has the `origin` of its write.
 - `patch()` merges fields into one item in one step, as a JSON Merge Patch (RFC 7386). Two patches of one item at the same time keep each field.
+- A write with an expected version tests the version of the current item and writes in one step. A storage that cannot test it rejects the write. Refer to "The precondition of a write".
 - The optional methods are features. A caller tests for each one before it uses it.
 
+### The precondition of a write
+
+`set`, `delete` and `setBatch` take an expected version of the current item (roadmap CLI-16, decision D11). It is optimistic concurrency: two writers cannot write over the changes of each other without a notice.
+
+```typescript
+const notes = new InMemoryStorage<Note>({ versionOf: (note) => note.rev });
+
+await notes.set("n1", { text: "a", rev: "1" }, { expectedVersion: null }); // only when "n1" has no item
+await notes.set("n1", { text: "b", rev: "2" }, { expectedVersion: "1" }); // only when "n1" has the version "1"
+await notes.setBatch([["n1", { text: "c", rev: "3" }]], { expectedVersions: [["n1", "2"]] });
+await notes.delete("n1", { expectedVersion: "3" });
+```
+
+- The version of an item is the value of the function `versionOf` of the storage, for example a revision field. `HybridStorage` reads the same versions with its own `versionOf`.
+- `null` expects no item.
+- When the current item has another version, the write changes nothing. It rejects with `VersionConflictError`, whose `conflicts` have the current item and its version for each id.
+- A `setBatch` with one conflict changes no item. An id without an expected version has no precondition.
+- The test and the write are one step. Two writes with the same expected version at the same time: one writes, and the other gets the conflict.
+- A storage without `versionOf` cannot test a precondition. It rejects such a write with a `TypeError` whose code is `PRECONDITION_NOT_SUPPORTED`, and it changes nothing. A storage never ignores a precondition.
+- A write without an expected version writes as before: the last write wins.
+- On the wire, the conflict is the error code `VERSION_CONFLICT` (HTTP 409). The metadata key `conflicts` has the current items. `ApiStorage` gives it back as a `VersionConflictError`.
+
+| Storage | How it tests the precondition |
+|---|---|
+| `InMemoryStorage` | The test and the write are one synchronous step. |
+| `SqliteStorage` | The test and the write are one transaction, also for two processes on one file. |
+| `MongoDocumentStorage` | On a replica set, the test reads in the transaction of the write. On a single server, the write has the `_seq` of the test in its filter. A batch of more than one item is all or nothing only in a transaction. |
+| `MongoStorage` | It cannot test it: its documents have no version. It rejects a write with an expected version. |
+| `ApiStorage` | The server tests it: the collection procedures give the expected version to the storage of the server. |
+| `HybridStorage` | The local copy tests it. In `write-through` mode, the remote tests it too. In `remote-only` mode, only the remote tests it. |
+
 ### The query language
 
 A selector is JSON: a subset of the query selectors of MongoDB. The same selector goes to `query()` of a storage and to the procedure `collections.<name>.find` of `@procedures/client`.
@@ -273,6 +306,7 @@ describeCollectionStorageContract(
   {
     create: () => new MyStorage<ContractItem>(),
     createWithMaxTombstones: (max) => new MyStorage<ContractItem>({ maxTombstones: max }),
+    createWithVersions: (versionOf) => new MyStorage<ContractItem>({ versionOf }),
   },
   { describe, it },
 );
@@ -282,6 +316,7 @@ describeCollectionStorageContract(
 - It tests the copies, the JSON values with `null`, and the writes at the same time.
 - It tests `getChangesSince` with its versions and its order, `flush`, `getMetadata`, `close` and the errors.
 - It tests `query()` with fixed and random selectors against `queryEntries()`, `patch()` with patches at the same time, and the batches and origins of `observe()`.
+- It tests the preconditions of the writes. A storage of `create()` must reject a write with an expected version. With `createWithVersions`, the suite tests the conflicts, the current items, `null`, the batches, and two writes at the same time.
 - A case for an optional method skips a storage without the method.
 - A random sequence of writes runs against a `Map`. Each saved version must give the right changes, in the order of the last change of each id.
 - The suite imports no test framework. The caller gives `describe` and `it`, and a failed check throws `ContractViolation`. The runtime entries do not import this entry.
@@ -313,6 +348,7 @@ const user = await storage.get("user-1");
 - `getIds()` gives the ids without a copy of the values.
 - `observe()` gives each batch before the write resolves. The error of an observer goes to the option `onObserverError`. The default throws it again in a microtask, as an `EventTarget` does.
 - `close()` drops the items. After it, each operation rejects, and the methods of the `Map` interface (`keys()`, `values()`, `entries()`, `forEach()`) throw.
+- The option `versionOf` gives the version of an item. Then `set`, `delete` and `setBatch` take an expected version. Refer to "The precondition of a write".
 
 ### ApiStorage and HybridStorage