@procedures/store 0.2.2 → 0.3.0

package/CHANGELOG.md

@@ -1,5 +1,68 @@
 # @procedures/store
 
+## 0.3.0
+
+### Minor Changes
+
+- 7873940: The collection mode switch (roadmap R28). `HybridStorage` has four modes: `local-only` (no remote call, each write waits in the queue), `write-through`, `write-back` and `remote-only` (no local copy). The option `mode` gives the first mode, and `writeStrategy` stays as a deprecated name of it. `setMode()` changes the mode while the storage operates. The switch waits for the operations that run, sends the queue to the remote first, and syncs the local copy when it leaves `local-only` or `remote-only`. A switch to `remote-only` drops the local copy only after the flush. When the remote fails, the mode stays and `setMode()` throws `ModeSwitchError`. A switch to `local-only` makes no remote call.
+  
+  The queue is durable: the option `queueStore` takes a `QueueStore` (`load`, `append`, `replace`, `clear`). `MemoryQueueStore` is the default, and `FileQueueStore` of the new entry `@procedures/store/file-queue` keeps the queue in a JSON Lines file. A write ends only after the store kept it (`QueueStoreError` when it cannot). A new storage with the same store loads the queue, shows its writes and sends them. The modes `write-back` and `local-only` always use the queue: before, `write-back` with `offlineQueue: false` never sent a write.
+  
+  A sync asks for the changes since the version of the last sync. Storages get the optional `getChangesSince(version)` (`InMemoryStorage` keeps a change log with tombstones, `ApiStorage` calls the new collection procedure `collections.<name>.changes`), and a remote without it gives all its entries. The option `versionOf` gives the version of an item: with versions, a conflict is a queued local write of an id whose remote version changed, and `conflictResolution` decides (`error` throws `ConflictError`). The option `pullInterval` starts a periodic pull. Storages get the optional `flush()`, and `ProcedureServer.useStorage(name, storage, { drain, close })` flushes the old storage before the change. `createSyncedRegistry({ type: "hybrid", mode })` takes the mode, and the synced registry has `setMode()` and `getMode()`.
+- c799cdd: `HybridStorage` cannot lose data (roadmap R2). A sync gets the ids of the remote items from the new `getEntries()` of a storage (the collection procedure `collections.<name>.entries`), or else from `keyOf`. A remote item without a key stops the sync with `MissingKeysError` before any change: before, such a list emptied the local cache. A write-through that the remote cannot take waits in the queue and throws `PendingWriteError`, and a timer sends the queue. A write that the remote refuses is undone in the local cache. A full queue refuses a write with `QueueFullError`: before, it dropped the oldest operation. A second try of a write replaces its queued operation. A write-through sends the older queued operations first. The reads wait for the first sync.
+- 4ef0d44: The new entry `@procedures/store/indexeddb` has the stores of a browser (roadmap R45). `IndexedDbProgramStore` keeps programs in IndexedDB, and it follows the contract of the other program stores. `IndexedDbQueueStore` keeps the queue of a `HybridStorage` in IndexedDB (the open point of R28). `hasIndexedDb()` tells if the browser has IndexedDB, and the entry also exports `MemoryProgramStore` and `MemoryQueueStore` as fallbacks. The entry does not import the client or zod, and the package root does not import it.
+  
+  `HybridStorage` has the new option `local`: the storage of the local copy (the type `LocalCopyStorage`). The default is a new `InMemoryStorage`, as before. A durable local copy, for example `SqliteStorage` of `@procedures/sqlite`, keeps the items after a restart. Then a new storage in `local-only` mode reads the items of the last run (the open point "the local copy is not durable" of R28). `close()` now closes the local copy too. A local copy that cannot take the loaded queue makes the storage refuse each operation, as a queue store that cannot load does.
+- fb30718: Typed durations (roadmap R11). This change breaks the API. Each public option that is a time takes a `Duration` of `@mark1russell7/quantity`, made with `ms()`, `seconds()`, `minutes()` or `hours()` of `@procedures/client`. A plain number is a type error, and at run time the option throws a `TypeError` that names it. This covers the timeouts and `resilience()`, the retry delays, the cache, the circuit breaker, the rate limit, the HTTP, WebSocket and mock transports, the WebSocket server heartbeat, `timeoutInterceptor({ limit })` (was `ms`), `hostPolicy({ timeout })` (was `timeoutMs`), the idempotency cache (`ttl`, was `ttlMs`), `ApiStorage`, `HybridStorage`, the synced registry and `connect()` of the peer. The per-call overrides in a context (`timeout`, `retry.delay`, `cache.ttl`) and the `route()` overrides (`timeout.overall`, was `timeout.ms`) take durations too. The wire and the inputs of procedures keep milliseconds.
+  
+  Migration: `createOverallTimeoutMiddleware({ overall: 5000 })` becomes `createOverallTimeoutMiddleware({ overall: seconds(5) })`.
+
+### Patch Changes
+
+- 43f7de5: Retry and the circuit breaker (roadmap R12). The retry middleware has a retry budget (`budget`, default: 0.2 of the calls and at least 10 retries a second), so a failing service gets a fraction more requests and not `maxRetries` times more. The delay has full jitter by default (`jitter: 1`: a random time from 0 to the backoff, before ±10 percent). A server that asks for a wait gets it: `retryAfterMs` of a thrown error becomes the `Retry-After` header and the `retry-after` metadata, and the retry waits at least that long. `maxAttempts` in a context counts the tries, as its name says (before, it counted the retries). The circuit breaker counts only the failures of availability (`isAvailabilityFailure`: UNAVAILABLE, TIMEOUT, a stopped server, a gateway error, network errors): an abort or a validation error no longer opens it. Its windows use a monotonic clock. `ApiStorage` asks a retry middleware of its client for one try, so the two loops cannot multiply.
+- Updated dependencies [ddea529]
+- Updated dependencies [b2a3e3a]
+- Updated dependencies [a8c47f1]
+- Updated dependencies [381ce3a]
+- Updated dependencies [0c627f8]
+- Updated dependencies [ae7f05f]
+- Updated dependencies [7873940]
+- Updated dependencies [a70d445]
+- Updated dependencies [bc70357]
+- Updated dependencies [07cc998]
+- Updated dependencies [a6b01a9]
+- Updated dependencies [c799cdd]
+- Updated dependencies [540f8d9]
+- Updated dependencies [b6daaec]
+- Updated dependencies [5b9264e]
+- Updated dependencies [956e88b]
+- Updated dependencies [ff62281]
+- Updated dependencies [d81d0ed]
+- Updated dependencies [aa67cc7]
+- Updated dependencies [061fa22]
+- Updated dependencies [cfb9d69]
+- Updated dependencies [43f7de5]
+- Updated dependencies [b07bb50]
+- Updated dependencies [4a3bfa1]
+- Updated dependencies [bd0b626]
+- Updated dependencies [a6be873]
+- Updated dependencies [fb30718]
+- Updated dependencies [c5528e4]
+- Updated dependencies [fee5d9c]
+- Updated dependencies [21f7743]
+- Updated dependencies [31a0b46]
+  - @procedures/client@0.3.0
+  - @procedures/collections@0.3.0
+
+## 0.2.3
+
+### Patch Changes
+
+- 15e48cf: Three fixes of the architecture review (2026-10-16). A `timeoutMs` that is not a finite number of zero or more asks for nothing: before, `NaN` removed the deadline. A procedure of `procedure.define` declares no effects of its own, so a read-only capability runs it and examines each nested call. `mcp.list-tools`, `mcp.serve`, `_discovery.announce`, `manifest.generate`, `server.create`, `procedure.export` and `lib.audit` read the registry of the call, not the global registry. `getProcedureInfos` and `watchProcedures` take a registry.
+- Updated dependencies [15e48cf]
+  - @procedures/client@0.2.3
+  - @procedures/collections@0.2.3
+
 ## 0.2.2
 
 ### Patch Changes

package/README.md

@@ -37,6 +37,26 @@ console.log(await client.exec(["math", "addTen"], { x: 5 })); // 15
 | `MemoryProgramStore` | In the memory of the process |
 | `FileProgramStore` | In one JSON file. A lock file and an atomic rename keep the programs of two processes. |
 | `CollectionProgramStore` | In a collection storage: `InMemoryStorage` of `@procedures/collections`, `ApiStorage` (the collection procedures of a server) or `HybridStorage` (a local cache with a remote copy) |
+| `IndexedDbProgramStore` | In the IndexedDB of a browser. It is in `@procedures/store/indexeddb`. |
+
+### The stores of a browser
+
+The entry `@procedures/store/indexeddb` has the stores of a browser. The package root does not import it, and it does not import the client or zod.
+
+- `IndexedDbProgramStore` keeps the programs in one database, `procedures-programs` by default.
+- `IndexedDbQueueStore` keeps the queue of a `HybridStorage`. Give each storage its own database.
+- `hasIndexedDb()` tells if the browser has IndexedDB. `MemoryProgramStore` and `MemoryQueueStore` are the fallbacks.
+- A browser can refuse to open a database, for example in a private window. Then the first operation of a store fails.
+- The store checks each record that it reads. A record of an old version or of another application is an error.
+
+```ts
+import { IndexedDbProgramStore, MemoryProgramStore, hasIndexedDb } from "@procedures/store/indexeddb";
+
+const programs = hasIndexedDb() ? new IndexedDbProgramStore("my-app-programs") : new MemoryProgramStore();
+await programs.put({ key: "reports.weekly", path: ["reports", "weekly"], program, storedAt: Date.now() });
+```
+
+The Composer of the site saves programs with `IndexedDbProgramStore`. It loads the store only when the reader opens the saved programs.
 
 A host gives a registry its store with `attachProgramStore(registry, store)`. The procedures use the store of the caller's registry:
 
@@ -49,7 +69,7 @@ A host gives a registry its store with `attachProgramStore(registry, store)`. Th
 
 Without a store, `procedure.store`, `load` and `sync` fail with the code `NOT_CONFIGURED`.
 
-A loaded program is a runtime-defined procedure, so it is data-driven: a server with an expose rule lets it call only the exposed procedures. Load programs only from a store that you trust. A host with no expose rule runs each call of a program.
+A loaded program is a runtime-defined procedure, so it is data-driven: a server with an expose rule lets it call only the exposed procedures. Load programs only from a store that you trust. A host with no expose rule does not limit the calls of a program.
 
 ## The storage layer
 
@@ -59,7 +79,94 @@ These parts moved here from `@procedures/client` (roadmap 3.2):
 - The synced registry (`createSyncedRegistry()`, `SyncedProcedureRegistry`), the serialization of procedures, and the dynamic handler loader with an allowlist (`createDynamicHandlerLoader`).
 - `procedure.register`, which registers a procedure from a handler reference.
 
-Importing `@procedures/store/register` registers its procedures. An import of the package root registers nothing.
+An import of `@procedures/store/register` registers its procedures. An import of the package root registers nothing.
+
+## The modes of `HybridStorage`
+
+`HybridStorage` keeps a local copy of a remote collection storage. It has four modes, and `setMode()` changes the mode while the storage operates.
+
+| Mode | Reads | Writes | Remote calls |
+|---|---|---|---|
+| `local-only` | The local copy | The local copy and the queue | None |
+| `write-through` | The local copy, and the remote on a miss | The local copy, then the remote | Yes |
+| `write-back` | The local copy, and the remote on a miss | The local copy and the queue. A timer sends the queue. | Yes |
+| `remote-only` | The remote | The remote | Yes. The storage keeps no local copy. |
+
+```ts
+import { minutes } from "@procedures/client";
+import { ApiStorage, HybridStorage } from "@procedures/store";
+import { FileQueueStore } from "@procedures/store/file-queue";
+
+const users = new HybridStorage(new ApiStorage(client, { collection: "users" }), {
+  mode: "write-back",
+  queueStore: new FileQueueStore("./users.queue.jsonl"),
+  versionOf: (user) => user.etag,
+  conflictResolution: "remote",
+  pullInterval: minutes(1),
+});
+
+await users.setMode("local-only"); // No remote call. The writes wait in the queue.
+await users.setMode("write-through"); // The switch sends the queue, then syncs the local copy.
+```
+
+### The switch
+
+- A switch waits for the operations that run. An operation that starts during the switch waits for the switch.
+- A switch to a remote mode sends the queue first. When the remote fails, the mode stays, and `setMode()` throws `ModeSwitchError`.
+- A switch to `local-only` makes no remote call. The queue stays until the next switch to a remote mode.
+- A switch to `remote-only` drops the local copy after the queue arrives at the remote.
+- A switch from `local-only` or `remote-only` to a mode with a local copy syncs the local copy. The reads wait for the sync.
+- The switches and the syncs run one at a time.
+
+### The queue
+
+- In the modes `local-only` and `write-back`, each write goes in the queue.
+- In the mode `write-through`, a write that the remote cannot take goes in the queue, and the write throws `PendingWriteError`.
+- A full queue refuses the next write with `QueueFullError`. The limit is `maxOfflineOps`, 1000 by default.
+- The option `queueStore` keeps the queue. The storage writes each change to the store before the write ends.
+- A new storage with the same store loads the queue, shows its writes, and sends them.
+- `MemoryQueueStore` is the default. `FileQueueStore` keeps the queue in a JSON Lines file, for Node.
+- `FileQueueStore` is in `@procedures/store/file-queue`, because it uses `node:fs`. The package root does not import it.
+- `IndexedDbQueueStore` keeps the queue in IndexedDB, for a browser. It is in `@procedures/store/indexeddb`.
+
+### The local copy
+
+- The option `local` gives the storage of the local copy. The default is a new `InMemoryStorage`.
+- A durable local copy keeps the items after a restart. `SqliteStorage` of `@procedures/sqlite` keeps them in a SQLite file.
+- With a durable local copy, a new storage in `local-only` mode reads the items of the last run. It makes no remote call.
+- Give a durable `queueStore` with a durable local copy. A restart applies the loaded queue to the local copy again, and the result does not change.
+- The first sync after a restart reads all remote items. It removes each local item that the remote deleted and that no queued write changes.
+- `close()` closes the local copy. A durable local copy keeps its items.
+
+```ts
+import { ApiStorage, HybridStorage } from "@procedures/store";
+import { FileQueueStore } from "@procedures/store/file-queue";
+import { SqliteStorage } from "@procedures/sqlite/storage";
+
+const users = new HybridStorage(new ApiStorage(client, { collection: "users" }), {
+  mode: "write-back",
+  local: new SqliteStorage({ path: "./users.db", table: "users" }),
+  queueStore: new FileQueueStore("./users.queue.jsonl"),
+});
+```
+
+### The sync, the versions and the conflicts
+
+- A sync asks the remote for the changes since the version of the last sync. `ApiStorage` uses the procedure `collections.<name>.changes`.
+- A remote without a change log gives all its entries.
+- The option `versionOf` gives the version of an item, for example its ETag.
+- With versions, a conflict is a queued local write of an id whose remote version changed. The option `conflictResolution` gives the result.
+- The values of `conflictResolution` are `local`, `remote`, `merge` with `mergeFn`, and `error`. With `error`, the sync throws `ConflictError`.
+- Without versions, a queued write always wins. A local value that differs from the remote value is a conflict.
+- The option `pullInterval` starts a periodic pull in the modes `write-through` and `write-back`. `close()` stops it.
+
+### The change of a storage in a server
+
+`ProcedureServer.useStorage(name, storage, { drain: true, close: true })` first starts `flush()` of the old storage. Then it uses the new storage and closes the old one. When the flush fails, the old storage stays.
+
+### The mode of a synced registry
+
+`createSyncedRegistry({ type: "hybrid", client, mode })` takes the first mode. `setMode()` of the registry changes the mode of its storage.
 
 ## License