@procedures/client 0.24.1 → 0.26.0

package/CHANGELOG.md

@@ -1,5 +1,98 @@
 # @procedures/client
 
+## 0.26.0
+
+### Minor Changes
+
+- a16af64: Changed: a program runs on the kernel of coeval (`@mark1russell7/coeval` `^0.5.0`, roadmap item M14, coeval decision D13). The program format and its rules do not change.
+  
+  - The range `^0.5.0` of `@mark1russell7/coeval` has the kernel fixes of 0.4.1 to 0.4.3 (the narrowing of capabilities in `check()`, the place of a compile throw and of a waiting runner, the step parts after a failed `let`, `#DEPTH!` for an endless recursion, unknown `ops.*`/`ext.*` names in `check()`, `docEquals()` on dates and maps). The types of the client that name coeval types stay the same, with one addition: `Machine`, the type that `programMachine()` gives, has the new optional member `known` (coeval 0.4.3). `World`, `Key`, `Json` and `EvalError` do not change.
+  
+  - `desugarProgram()` changes the `$proc` form into a run document of coeval, and one run of the kernel runs it (`runProgram()`). `client.exec()`, `client.execStream()`, `hydrateInput()`, `prepareRootCall()`, `executeRef()`, the control-flow procedures and the body of a defined procedure use this run.
+  - Each call of a program is an effect. The client does it with `client.call()`, so `invokeProcedure()` stays the authority for the admission, the validation, the interceptors and the exposure rules of each call. A failed call throws the same error as before, with its message and its code.
+  - The logic of `client.chain`, `client.parallel`, `client.conditional`, `client.and`, `client.or`, `client.map`, `client.reduce` and `client.tryCatch` is a runner of the kernel. The procedures keep their registrations and their schemas.
+  - The envelope form reads its envelopes with the helpers of coeval. Its literal stays `{ "type": "literal", "value": x }`, as in 0.23.0, and `toEnvelopeProgram()` writes this form. The type `"lit"` (the kind of the kernel, coeval D12) is an alias: `{ "type": "lit", "value": x }` is also a literal.
+  - New exports: `runProgram`, `runDocument`, `runControl`, `runOperand`, `programMachine`, `PROGRAM_WORLD`, `desugarProgram`, `desugarInput`, `desugarControlValue`, `originOf`, `documentAt`, `namesOfScope`, `scopeOfNames`, `nameKey`, `nameOfKey` and their types.
+  - `@procedures/compiler`: the name rules (`unknown-name`, `use-before-definition`) run through `check()` of the static kit of coeval on the run document. New exports `nameDiagnostics()` and `bindScopes()`. `resolveScopes()` gives the same diagnostics, with the same codes and messages, in a different order (see below).
+  - PROGRAM-FORMAT.md revision 2: section 13 "Programs on the kernel of coeval".
+  
+  MINOR (D26): before 1.0.0, a breaking change raises MINOR. These behaviors change:
+  
+  1. `client.parallel` with `failFast`: the first error comes at once, as before. The other tasks now stop at their next wait, so their later calls do not start. A call that started goes on in its procedure. Before, the other tasks went on in the background to their end.
+  2. `HydrateOptions.parallel`: the refs that start at the same time now always operate with failFast. After the first error, the other refs stop at their next wait. Before, they went on in the background (`Promise.all`).
+  3. A call or an operand that throws `undefined`: the caller now gets `new Error(<reason>)`, with the reason of the kernel. The error of a task of `client.parallel` and the name `error` of `client.tryCatch` get that reason. Before, the caller got `undefined`, and the error text was `"undefined"`. An `Error`, a text and the other values stay as they are.
+  4. `resolveScopes()` of `@procedures/compiler` gives the warnings for the raw input first, and then the diagnostics of the names. Before, the diagnostics came in the order of the program tree.
+  5. In the envelope form, the types `["ref"]` and `["lit"]` (arrays of one key) are the kinds `ref` and `lit` of the kernel. Before, they were calls of the procedures with the paths `["ref"]` and `["lit"]`. A ref into the world, `{ "type": "ref", "value": ["world", "ref"] }`, stays a call of the procedure, and `toEnvelopeProgram()` now writes this type for these two procedures.
+  
+  Migration:
+  
+  - For 1 and 2: when the later calls of the other tasks must run, wait for those tasks yourself (for example, start them in a separate program and join their results), or do not use `failFast` (for 1) or `parallel` (for 2).
+  - For 3: throw an `Error`.
+  - For 4: sort the diagnostics by `pointer` when the order matters.
+  - For 5: for a call of a procedure with the path `["ref"]` or `["lit"]`, write a ref into the world as the type: `{ "type": { "type": "ref", "value": ["world", "ref"] }, "value": input }`. Or keep the program in the `$proc` form: `{ "$proc": ["ref"], "input": input }` is a call, as before.
+  
+  Speed: the property tests that run each program three ways (`equivalence.test.ts` of the compiler) are about 1.4 times slower on the CI runners: 29.4 s before and 40.0 s after, the sum of the Node 22, 24 and 26 jobs, the mean of 3 runs of main and 4 runs of this change. The program tests of the client (`src/procedures/`) take about the same time: 8.5 s and 8.8 s.
+
+## 0.25.0
+
+### Minor Changes
+
+- 75bff3b: Breaking: an in-band end of a stream before its first item gets the default flag of its code (roadmap CLI-68, decision D25). The release before this one marked the old flag `false` as deprecated (D13).
+  
+  The new behavior:
+  
+  - A handler error without a `retryable` flag that ends a stream before its first item gets the flag of its code in `ERROR_CODES`. Thus `UNAVAILABLE`, `CONNECTION_CLOSED` and `RATE_LIMITED` are retryable there. The other codes stay `false`.
+  - This applies on each face: the `error` line of NDJSON, the `error` frame of WebSocket and the `_meta.retryable` of the MCP stream result. The HTTP status of an NDJSON stream stays 200.
+  - An error before the stream starts already got this default (CLI-48). Thus a streaming handler and a unary handler that throw the same error before their first item now give the same flag.
+  - After the first item, an error without a flag still gets `false` (CLI-58). An error with its own flag keeps it, before and after the first item.
+  - The retry middleware of the client retries while no item reached the reader. Thus it now retries a stream that fails with `UNAVAILABLE` before its first item. The caller gets the items of the retry, not the error.
+  - A keyed call to a procedure that is not idempotent: a retryable end of its stream lets the next try with the same idempotency key run again. Before, the next try got `DUPLICATE`.
+  - Without NDJSON (the last item as JSON), a stream error before the first item gets the flag and the HTTP status of its code, for example 503 for `UNAVAILABLE`. After an item, the flag is `false`, thus 500.
+  
+  What a caller sees:
+  
+  | Stream error without a `retryable` flag | Before | From this release |
+  |---|---|---|
+  | Before the first item, code `UNAVAILABLE` | `retryable: false`, no retry | `retryable: true`, the retry middleware sends the call again |
+  | Before the first item, a code that is not retryable | `retryable: false` | `retryable: false` |
+  | After the first item | `retryable: false` | `retryable: false` |
+  | A handler gives `retryable: true` or `false` | that flag | that flag |
+  
+  The retry costs time and calls: each try waits for the backoff, and the server gets one more call for each try. A handler with an effect before its error does that effect again on each try.
+  
+  New exports and options:
+  
+  - `retryableOf(error, items)` of `@procedures/client` gives the flag of an error that ends a call, after `items` items. Each face uses it.
+  - The `errorOf` option of `writeNdjsonStream` also gets the count of the item lines before the error. A function with one parameter still works.
+  - `IdempotencyCache.forget(key, result)` removes the result of a key.
+  
+  The opt-out:
+  
+  - A handler that is not safe to run again gives `retryable: false` in its error. For example, `throw Object.assign(new Error("busy"), { code: "UNAVAILABLE", retryable: false })`.
+  - Or the caller does not use the retry middleware for that call. For example, give `shouldRetry` that gives `false` for that method, or set `retry: { maxAttempts: 1 }` in the context of the call.
+  
+  Migration steps:
+  
+  - A streaming handler that throws `UNAVAILABLE`, `CONNECTION_CLOSED` or `RATE_LIMITED` without a flag before its first item: is a second run safe? If not, give `retryable: false`.
+  - A test that expects `retryable: false` for such an end before the first item: expect the flag of the code.
+  - A custom `errorOf` of `writeNdjsonStream`: read its second parameter to give the same rule.
+  - The site: no change is necessary. The lab simulates its server with explicit flags. The streams of the journey do not fail.
+  - MiniMongo: no change is necessary. Its browser client has the retry middleware, but its procedures throw no `UNAVAILABLE`, `CONNECTION_CLOSED` or `RATE_LIMITED` without a flag.
+  - Graph: no change is necessary. Its handlers give `UNAVAILABLE` with an explicit flag, and it serves no stream procedure of its own.
+  - jarvis: no change is necessary. The cuts of its host have their own flag, and it uses no retry middleware.
+
+### Patch Changes
+
+- 16562ee: Deprecated: five behaviors of the programs (decision D13). The next MINOR (D26: the programs on the kernel of coeval, roadmap item M14) changes them. This release changes no behavior and no API: it adds the notes "Deprecated behavior (D26)" to the doc comments, and the section "Deprecated behaviors (D26)" of PROGRAM-FORMAT.md. The functions and the options stay, so the notes are not `@deprecated` tags.
+  
+  - `failFast` of `client.parallel`: at this time, the other tasks go on in the background after the first error. From the next MINOR, they stop at their next wait, and their later calls do not start. When those calls must start, wait for the tasks yourself, or do not use `failFast`.
+  - `HydrateOptions.parallel`: at this time, the other refs go on in the background after the first error (`Promise.all`). From the next MINOR, the refs always operate with failFast. When their calls must start, give `parallel: false`.
+  - A call that throws `undefined`: at this time, the program fails with `undefined`, and the error text of `client.parallel` and `client.tryCatch` is `"undefined"`. From the next MINOR, the program fails with an `Error` that has the reason of the kernel. Throw an `Error`.
+  - `resolveScopes()` of `@procedures/compiler`: at this time, the diagnostics come in the order of the program tree. From the next MINOR, the warnings for the raw input come first, and then the diagnostics of the names. Sort the diagnostics by `pointer` when the order matters.
+  - The types `["ref"]` and `["lit"]` of the envelope form (arrays of one key): at this time, they are calls of the procedures with these paths. From the next MINOR, they are the kinds `ref` and `lit` of the kernel. For a call of these procedures, write a ref into the world as the type: `{ "type": { "type": "ref", "value": ["world", "ref"] }, "value": input }`. A ref into the world stays a call, and the `$proc` form does not change.
+  
+  PATCH: documentation only. No export, no type and no behavior changes.
+
 ## 0.24.1
 
 ### Patch Changes

package/README.md

@@ -364,6 +364,9 @@ interface RetryOptions {
 }
 
 // A Retry-After of the server makes the delay at least that long.
+// A stream goes again only while no item reached the reader: its first item is a retryable error.
+// A stream that fails before its first item with an error without a flag gets the flag of its code,
+// so UNAVAILABLE goes again (CLI-68). A handler gives retryable: false to stop that.
 // The overrides of one call, in its context or as route() overrides (maxAttempts counts the tries)
 client.withContext({ retry: { maxAttempts: 5, delay: ms(200), backoffMultiplier: 3, maxDelay: seconds(2), idempotent: true } });
 ```
@@ -510,6 +513,16 @@ const result = await client.exec(procedureRef)
 const hydrated = await hydrateInput(input, executor)
 ```
 
+A program runs on the kernel [coeval](https://github.com/mark1russell7/coeval) (`@mark1russell7/coeval`). `desugarProgram()` changes the `$proc` form into a run document, and one run of the kernel runs it. Each call of the program is an effect that the client does with `client.call()`, so each call keeps its admission, its validation and its interceptors. The static kit of coeval (`check()`) reads the same document. The section "Programs on the kernel of coeval" of [PROGRAM-FORMAT.md](../documentation/PROGRAM-FORMAT.md) gives the run document.
+
+```typescript
+import { check } from "@mark1russell7/coeval";
+import { desugarProgram, programMachine, rawInputRule } from "@procedures/client";
+
+const doc = desugarProgram(program, { expand: true });
+const issues = check(programMachine({ rawInput: rawInputRule(registry) }), doc); // for example, a name that nothing binds
+```
+
 ### Components, Collections and the Program Store
 
 They are not in the core. See the README of `@procedures/splay` (components), `@procedures/collections` (data structures) and `@procedures/store` (the program store and the storage layer).
@@ -645,6 +658,8 @@ createServer((req, res) => {
 }).listen(3000);
 ```
 
+`errorOf` also gets the count of the item lines before the error. The default gives an error without a flag the flag of its code before the first item, and `false` after it (CLI-68).
+
 The promise of `writeNdjsonStream` resolves when the response ends. A closed connection or an abort of `signal` stops the stream, and the iterator of the items gets `return()`.
 
 #### Procedure handler without a framework