@byokit/decide 0.1.1 → 0.3.0

package/CHANGELOG.md

@@ -0,0 +1,30 @@
+# Changelog
+
+## Unreleased
+
+## 0.3.0 (2026-09-29)
+
+- FIX: Jev answers no longer drop the backend's token counts: every answer now carries `usage`
+  (`input_tokens`/`output_tokens` when the backend sends them) and the raw backend response (`raw`), even
+  when abstaining on a malformed answer. Any backend can report the same pair through its `Raw`.
+- `decide()` takes an optional pluggable answer cache (`get`/`set`, sync or async): it computes a stable
+  key (sha256 of the canonical request body, exported as `cacheKey`), reports `source: 'cache' | 'api'` on
+  every answer, and serves a cached answer with the same `usage`/`raw` it was stored with. Ships the
+  in-memory `MemoryCache` reference; no on-disk cache in the library.
+- `jev()` retries 429s with backoff, honouring `Retry-After` when present (options `maxRetries` default 2,
+  `retryBaseMs`, `retryMaxMs`; the total wait stays under `maxRetries` x `retryMaxMs`), respects the caller's
+  timeout/`AbortSignal` including aborts mid-backoff, and never retries other statuses. Each retry is
+  API key (billed per use) like the first call.
+
+## 0.2.0
+
+- `answerer({ name, leaves, ask })` makes a decision backend of any `(prompt, signal) => text`, such as the ChatGPT the person signed in to; any reply that is not the requested JSON is an abstain.
+- The main entry is plain TypeScript with `fetch`, so it bundles for React Native and the web.
+
+## 0.1.1
+
+- The Apache-2.0 LICENSE ships in the tarball.
+
+## 0.1.0
+
+- Typed decisions with rules and Jev backends, abstaining below a confidence floor, and the `byokit-eval` runner.

package/README.md

@@ -1,8 +1,69 @@
-# @byokit/decide
+<h1 align="center">@byokit/decide</h1>
 
-Typed questions in, a typed answer with confidence out, and an abstain below a floor, so your app takes its safe
-default (ask the person) instead of guessing. Backends: your own `rules`, and [Jev](https://openrouter.ai/docs/guides/community/jev)
-over TypeSafe's API or OpenRouter.
+<p align="center">
+  <a href="https://www.npmjs.com/package/@byokit/decide"><img alt="npm" src="https://img.shields.io/npm/v/@byokit/decide?style=flat&label=npm" /></a>
+  <a href="https://github.com/umeranjum17/byokit/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/umeranjum17/byokit/ci.yml?style=flat&branch=main" /></a>
+  <a href="LICENSE"><img alt="Apache 2.0" src="https://img.shields.io/badge/license-Apache--2.0-666?style=flat" /></a>
+  <img alt="Node | browsers | React Native" src="https://img.shields.io/badge/platform-Node%20%7C%20browsers%20%7C%20React%20Native-666?style=flat" />
+</p>
+
+<p align="center"><strong>Typed questions in, a typed answer with confidence out.</strong><br/>
+Below a floor it abstains, so your app takes its safe default (ask the person) instead of guessing. Backends are your
+own <code>rules</code>, any model you can send a prompt to (on a phone, the person's own ChatGPT), and
+<a href="https://openrouter.ai/docs/guides/community/jev">Jev</a> (API-billed) over TypeSafe's API or OpenRouter. Labelled eval files
+set the floors.</p>
+
+## Install
+
+```sh
+npm install @byokit/decide
+```
+
+[![npm](https://img.shields.io/npm/v/@byokit/decide?style=flat&label=)](https://www.npmjs.com/package/@byokit/decide) · [Latest release](https://github.com/umeranjum17/byokit/releases?q=decide-v) · [All releases](https://github.com/umeranjum17/byokit/releases)
+
+## Quickstart
+
+```sh
+npm install @byokit/decide
+```
+
+A rules-only backend: nothing leaves the device and no key is needed. The obvious case gets an answer; the rest abstains.
+
+```ts
+import { decide, rules, type Question } from '@byokit/decide';
+
+const questions: Record<string, Question> = {
+  intent: { kind: 'choice', options: { task: 'Something new to do', followup: 'About an earlier job', chat: 'Just talking' } },
+};
+const backends = [rules((s) => (/^(thanks|thank you)\b/i.test(s.text) ? 'chat' : undefined))];
+
+for (const text of ['thanks, that worked!', 'can you check if the plumber replied?']) {
+  const { intent } = await decide({ text }, questions, { privacy: 'stays-here', backends });
+  console.log(text, '->', intent);
+}
+```
+
+```text
+thanks, that worked! -> {
+  answer: 'chat',
+  confidence: 1,
+  probabilities: { task: 0, followup: 0, chat: 1 },
+  abstained: false,
+  by: 'rules',
+  ms: 0
+}
+can you check if the plumber replied? -> {
+  answer: null,
+  confidence: 0,
+  abstained: true,
+  reason: 'no answer',
+  by: 'rules',
+  ms: 0
+}
+```
+
+Add a model after the rules for the cases they leave open. Jev is API-billed: it needs a TypeSafe or OpenRouter key
+that your host holds.
 
 ```ts
 import { decide, jev, rules } from '@byokit/decide';
@@ -16,28 +77,97 @@ const { intent } = await decide({ text: 'can you check if the plumber replied?'
 }, { privacy: 'may-leave', backends });
 
 if (intent.abstained) askThePerson(); else route(intent.answer);
-// { answer: 'followup', confidence: 0.91, probabilities: {...}, abstained: false, by: 'jev', ms: 214 }
 ```
 
+## API at a glance
+
+| Export | What it does |
+|---|---|
+| `decide(state, questions, { privacy, backends, timeoutMs?, cache? })` | Asks each backend in order for the questions still unanswered; returns an `Answer` per question |
+| `rules(fn)` | Your own function as a backend: return the answer for an obvious case, `undefined` otherwise. Stays on the device |
+| `answerer({ name, leaves, ask })` | Any `(prompt, signal) => text` model as a backend |
+| `jev({ key, via?, fetch?, maxRetries?, retryBaseMs?, retryMaxMs? })` | Jev as a backend, over TypeSafe's API (default) or OpenRouter (`via: 'openrouter'`). API-billed; retries 429s with backoff |
+| `MemoryCache`, `cacheKey(state, questions)` | In-memory reference cache for `decide({ cache })`, and the stable request key it uses |
+| `resolve(question, raw)` | The floors on one raw answer, for an app that holds a recorded answer |
+| `FLOOR` | The default floor, 0.6 |
+| `Question`, `Answer`, `Raw`, `Usage`, `Backend`, `DecideCache`, `Options` | The types |
+| `@byokit/decide/eval`: `evaluate`, `replay`, `parse`, `format`, `summary` | Run and print an eval report over any backends |
+| `byokit-eval` (bin) | Replay or refresh an eval file from the command line |
+
+## Questions and floors
+
 - **Questions**: `choice` (options with a one-line description each), `yesno`, and `score` (an ordered rubric, lowest
   first; the answer is the level's index).
 - **The floors are code, not a prompt** (ported from firstmate's dispatch resolver): a 0.6 floor on the answer's
-  confidence by default (`floor` per question); a choice option can declare its own floor (`floors`), checked against its
-  own probability, and a pick under it falls to the most probable other option that clears its own; a tie abstains.
-  An answer whose probabilities are missing an option, out of range or don't sum to 1 is an abstain, never an error.
-- **Backends** are tried in order for the questions still unanswered. A backend that fails or takes longer than
-  `timeoutMs` (default 5 s) answers nothing. `privacy: 'stays-here'` skips every backend the state would leave the
-  device for (Jev), so private text never goes to one.
-- **Keys**: `jev()` takes the key your host read from its own environment or config. The kit never reads an environment
-  variable, and the key goes only into the one request header. Never ship a key inside an app: keep it on the home
-  computer and let paired devices ask it.
+  confidence by default (`floor` per question). A choice option can declare its own floor (`floors`), checked against
+  its own probability; a pick under it falls to the most probable other option that clears its own. A tie abstains.
+- An answer whose probabilities are missing an option, out of range or don't sum to 1 is an abstain, never an error.
+
+## Backends
+
+- Backends are tried in order for the questions still unanswered. A backend that fails or takes longer than
+  `timeoutMs` (default 5 s) answers nothing.
+- `privacy: 'stays-here'` skips every backend the state would leave the device for (Jev, or any answerer with
+  `leaves: true`), so private text never goes to one.
+- **Any model**: `answerer({ name, leaves, ask })` makes a backend of any `(prompt, signal) => text`. On a phone, that
+  is the ChatGPT the person signed in to with [`@byokit/accounts`](../accounts), on their own plan:
+
+  ```ts
+  import { answerer } from '@byokit/decide';
+
+  const chatgpt = answerer({
+    name: 'chatgpt',
+    leaves: true,
+    ask: (p, signal) => accounts.respond(me, { instructions: 'Reply with JSON only.', input: p, signal }),
+  });
+  ```
+
+  It asks for each answer's probability as JSON; any other reply is an abstain.
+  - **Billing**: `rules` costs nothing. `answerer` with the person's ChatGPT uses their subscription. `jev()` is billed
+  to the TypeSafe or OpenRouter key you pass.
+
+## Usage, cache and retries
+
+Every `Answer` carries what its backend reported: `usage` (`input_tokens`/`output_tokens` when sent) and the raw
+backend response (`raw`), on answered and abstained answers alike, so cost accounting never loses a count. `source`
+tells whether the answer was decided live (`'api'`) or served from cache (`'cache'`).
+
+```ts
+import { decide, jev, MemoryCache } from '@byokit/decide';
+
+const cache = new MemoryCache(); // reference implementation; bring your own get/set (sync or async) to persist
+const backend = jev({ key: hostConfig.jevKey, maxRetries: 2, retryMaxMs: 2000 });
+
+const first = await decide({ text }, questions, { privacy: 'may-leave', backends: [backend], cache });
+console.log(first.intent.source, first.intent.usage); // 'api' { input_tokens: 42, output_tokens: 7 }
+const second = await decide({ text }, questions, { privacy: 'may-leave', backends: [backend], cache });
+console.log(second.intent.source); // 'cache': same usage/raw, no backend call
+```
+
+- **Cache**: the key is the sha256 of the canonical `{ state, questions }` body (`cacheKey(state, questions)`), stable
+  across key order. The library ships no on-disk cache; a `get`/`set` pair over your own store is enough.
+- **Retries**: only 429s retry, never other statuses. A 429 waits for `Retry-After` when present (seconds or HTTP
+  date), else an exponential backoff from `retryBaseMs`; each wait is capped at `retryMaxMs`, so the total stays
+  under `maxRetries` x `retryMaxMs`. Backoff respects the caller's `timeoutMs`/`AbortSignal`, including an abort
+  mid-wait, and each retry is API-billed like the first call.
+
+## Phones and browsers
+
+The main entry is plain TypeScript with `fetch` (the eval CLI is its own entry), so it bundles for React Native and the
+web; `test/react-native.test.ts` runs it where there is no Node.
+
+## Keys
+
+`jev()` takes the key your host read from its own environment or config. The kit never reads an environment variable,
+and the key goes only into the one request header. Never ship a key inside an app: keep it on the home computer and let
+paired devices ask it.
 
 ## Evals
 
 Each decision gets a labelled file, `evals/<decision>.jsonl`: a header `{ decision, question, note }`, then one case per
 line, `{ state, expect, jev, ms }`. `expect` is the right answer, a list of right answers, or `null` when only an abstain
-is right; `jev` is a Jev-shaped answer, replayed offline so CI never calls a model. The included example is hand-made,
-not a live recording.
+is right; `jev` is a Jev-shaped answer, replayed offline so CI never calls a model. The included example,
+`evals/example-urgent.jsonl`, shows the format; it is hand-made, not a live recording.
 
 ```sh
 npx --package=@byokit/decide byokit-eval evals/intent.jsonl                    # replay stored answers
@@ -45,8 +175,50 @@ npx --package=@byokit/decide byokit-eval evals/intent.jsonl --floor 0.7        #
 TYPESAFE_API_KEY=… npx --package=@byokit/decide byokit-eval evals/intent.jsonl --live typesafe --record
 ```
 
-Agreement counts right answers and correctly expected abstentions; abstentions are also reported separately. Clear-but-wrong
-(answered, and wrong) is the number that must stay near 0; the command exits 1 when its rate is above
-`--max-clear-wrong` (default 0). Set floors from the eval, not by guessing. `evaluate()` in `@byokit/decide/eval` runs the
-same report over any backends, including your rules. `--record` keeps previous answers when a live refresh fails and
-marks a partially refreshed file as such. `evals/example-urgent.jsonl` shows the format.
+Replaying the included example from this repo, at the default floor and then at 0.95:
+
+```text
+$ npx byokit-eval packages/decide/evals/example-urgent.jsonl
+packages/decide/evals/example-urgent.jsonl: urgent (jev, recorded): 5 cases
+  agree 4/5   clear-but-wrong 0 (0%)   abstained 1 (20%)   ms min/median/max 165/180/201
+$ npx byokit-eval packages/decide/evals/example-urgent.jsonl --floor 0.95
+packages/decide/evals/example-urgent.jsonl: urgent (jev, recorded): 5 cases
+  agree 2/5   clear-but-wrong 0 (0%)   abstained 3 (60%)   ms min/median/max 165/180/201
+```
+
+- Agreement counts right answers and correctly expected abstentions; abstentions are also reported separately.
+- Clear-but-wrong (answered, and wrong) is the number that must stay near 0; the command exits 1 when its rate is above
+  `--max-clear-wrong` (default 0).
+- Set floors from the eval, not by guessing.
+- Only `--live` reads a key (`TYPESAFE_API_KEY`, or `OPENROUTER_API_KEY` with `--live openrouter`), and each live call
+  is API-billed. `--record` keeps previous answers when a live refresh fails and marks a partially refreshed file as
+  such.
+
+`evaluate()` in `@byokit/decide/eval` runs the same report over any backends, including your rules. Run
+from a byokit checkout, it replays the included example:
+
+```ts
+import { readFileSync } from 'node:fs';
+import { decide, rules } from '@byokit/decide';
+import { evaluate, parse, summary } from '@byokit/decide/eval';
+
+const f = parse(readFileSync('packages/decide/evals/example-urgent.jsonl', 'utf8'));
+const backends = [rules((s: string) => (/\b(now|today|before \d)/i.test(s) ? true : undefined))];
+const report = await evaluate(f.cases, async (c) => (await decide(c.state, { urgent: f.question }, { privacy: 'stays-here', backends })).urgent);
+console.log(summary(f.decision, 'rules', report));
+```
+
+```text
+urgent (rules): 5 cases
+  agree 2/5   clear-but-wrong 0 (0%)   abstained 3 (60%)   ms min/median/max 0/0/1
+```
+
+## Links
+
+- [byokit](../../README.md): the other packages
+- [`examples/expo`](../../examples/expo): uses `@byokit/decide` in a React Native app
+- [CHANGELOG.md](CHANGELOG.md)
+
+## License
+
+Apache-2.0. See [LICENSE](LICENSE) and [NOTICE](https://github.com/umeranjum17/byokit/blob/main/NOTICE).

package/dist/index.d.ts

@@ -31,13 +31,27 @@ export type Answer = {
     reason?: string;
     by: string;
     ms: number;
+    /** Token counts the backend reported for this answer, when it did. Never dropped when present. */
+    usage?: Usage;
+    /** The backend's response behind this answer, when there was one (even a malformed one). */
+    raw?: unknown;
+    /** Whether this answer was decided live or served from the `cache` in `Options`. Always set by `decide()`. */
+    source?: 'api' | 'cache';
+};
+/** Token counts a backend reports for its answer. Each count is present only when the backend sent a valid one. */
+export type Usage = {
+    input_tokens?: number;
+    output_tokens?: number;
 };
 /** A backend's answer before the floors: every option's probability, keyed as options (choice), 'true'/'false'
- * (yesno) or level indexes (score). A missing or malformed one is an abstain. */
+ * (yesno) or level indexes (score). A missing or malformed one is an abstain. `usage`/`raw` ride along through
+ * the floors onto the `Answer`, so any backend can report cost accounting and the raw response, not only Jev. */
 export type Raw = {
     probabilities: Record<string, number>;
     confidence?: number;
     pick?: string;
+    usage?: Usage;
+    raw?: unknown;
 };
 export type Backend = {
     name: string;
@@ -49,11 +63,44 @@ export type Options = {
     privacy: 'stays-here' | 'may-leave';
     backends: Backend[];
     timeoutMs?: number;
+    /** Optional pluggable answer cache. `decide()` computes a stable key (sha256 of the canonical
+     * `{ state, questions }` body, see `cacheKey`) and reports `source: 'cache' | 'api'` on every
+     * answer. A cached answer returns the same `usage`/`raw` it was stored with. No default on-disk
+     * cache ships with the kit; `MemoryCache` is the in-memory reference. Cache errors never fail a decision. */
+    cache?: DecideCache;
+};
+/** Pluggable answer cache for `decide()`: `get` returns the stored answers for a key, `set` stores them.
+ * Either may be sync or async. */
+export type DecideCache = {
+    get(key: string): Record<string, Answer> | undefined | Promise<Record<string, Answer> | undefined>;
+    set(key: string, value: Record<string, Answer>): void | Promise<void>;
 };
+/** In-memory reference cache: the shape a `DecideCache` takes. Copies answers on the way in and out. */
+export declare class MemoryCache implements DecideCache {
+    private map;
+    get(key: string): Record<string, Answer> | undefined;
+    set(key: string, value: Record<string, Answer>): void;
+    get size(): number;
+}
+/** Stable cache key for a decision: the sha256 of the canonical `{ state, questions }` body, so the same
+ * question about the same state hits whatever the key order. Pure TypeScript: no Node imports, safe on phones. */
+export declare function cacheKey(state: unknown, questions: Record<string, Question>): string;
 export declare const FLOOR = 0.6;
-/** Asks each backend in order for the questions still unanswered; a failed or slow backend answers nothing. */
+/** Asks each backend in order for the questions still unanswered; a failed or slow backend answers nothing.
+ * With `opts.cache`, a stored answer is served as `source: 'cache'` without calling any backend; fresh answers
+ * are stored as `source: 'api'` with the same `usage`/`raw` they carry. */
 export declare function decide(state: unknown, questions: Record<string, Question>, opts: Options): Promise<Record<string, Answer>>;
-/** The floors on one raw answer. Exported for apps that hold a recorded answer. */
+/** The floors on one raw answer. Exported for apps that hold a recorded answer.
+ * `usage`/`raw` on the raw ride through onto the answer, answered or abstained. */
 export declare function resolve(q: Question, raw: Raw | undefined): Omit<Answer, 'by' | 'ms'>;
 /** The app's own function as a backend: return the answer when the case is obvious, undefined otherwise. Stays here. */
 export declare function rules(fn: (state: any, name: string, q: Question) => string | boolean | number | undefined): Backend;
+/** Any model as a backend: `ask` gets one prompt and returns the model's text (on a phone, the signed-in ChatGPT:
+ *  `(p, signal) => accounts.respond(me, { instructions: '', input: p, signal })`). The model is asked for each option's
+ *  probability as JSON; an answer that isn't that JSON is no answer, so the question abstains. `leaves`: whether the
+ *  state goes off this device (true for any hosted model). */
+export declare function answerer(o: {
+    name: string;
+    leaves: boolean;
+    ask: (prompt: string, signal: AbortSignal) => Promise<string>;
+}): Backend;

package/dist/index.js

@@ -1,9 +1,121 @@
 // Typed questions in, a typed answer with confidence out, abstaining below a floor. The floor, the per-option floors,
 // the runner-up and the tie are code, never a prompt: ported from firstmate's bin/fm-dispatch-resolve.sh.
 export { jev } from "./jev.js";
+/** In-memory reference cache: the shape a `DecideCache` takes. Copies answers on the way in and out. */
+export class MemoryCache {
+    map = new Map();
+    get(key) {
+        const hit = this.map.get(key);
+        if (!hit)
+            return undefined;
+        return Object.fromEntries(Object.entries(hit).map(([k, a]) => [k, { ...a }]));
+    }
+    set(key, value) {
+        this.map.set(key, Object.fromEntries(Object.entries(value).map(([k, a]) => [k, { ...a }])));
+    }
+    get size() {
+        return this.map.size;
+    }
+}
+/** Stable cache key for a decision: the sha256 of the canonical `{ state, questions }` body, so the same
+ * question about the same state hits whatever the key order. Pure TypeScript: no Node imports, safe on phones. */
+export function cacheKey(state, questions) {
+    return sha256Hex(stableStringify({ state, questions }));
+}
+function stableStringify(v) {
+    if (v === null || typeof v !== 'object')
+        return JSON.stringify(v) ?? 'null';
+    if (Array.isArray(v))
+        return `[${v.map((e) => stableStringify(e)).join(',')}]`;
+    const o = v;
+    return `{${Object.keys(o).sort().map((k) => `${JSON.stringify(k)}:${stableStringify(o[k])}`).join(',')}}`;
+}
+function utf8Bytes(s) {
+    const out = [];
+    for (let i = 0; i < s.length; i++) {
+        let c = s.charCodeAt(i);
+        if (c >= 0xd800 && c <= 0xdbff && i + 1 < s.length) {
+            const lo = s.charCodeAt(i + 1);
+            if (lo >= 0xdc00 && lo <= 0xdfff) {
+                c = 0x10000 + ((c - 0xd800) << 10) + (lo - 0xdc00);
+                i++;
+            }
+        }
+        if (c < 0x80)
+            out.push(c);
+        else if (c < 0x800)
+            out.push(0xc0 | (c >> 6), 0x80 | (c & 0x3f));
+        else if (c < 0x10000)
+            out.push(0xe0 | (c >> 12), 0x80 | ((c >> 6) & 0x3f), 0x80 | (c & 0x3f));
+        else
+            out.push(0xf0 | (c >> 18), 0x80 | ((c >> 12) & 0x3f), 0x80 | ((c >> 6) & 0x3f), 0x80 | (c & 0x3f));
+    }
+    return out;
+}
+/** SHA-256 as hex, self-contained so the main entry stays free of Node imports on phones and browsers. */
+function sha256Hex(s) {
+    const k = [
+        0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5,
+        0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174,
+        0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da,
+        0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967,
+        0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85,
+        0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070,
+        0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
+        0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2,
+    ];
+    let h0 = 0x6a09e667, h1 = 0xbb67ae85, h2 = 0x3c6ef372, h3 = 0xa54ff53a;
+    let h4 = 0x510e527f, h5 = 0x9b05688c, h6 = 0x1f83d9ab, h7 = 0x5be0cd19;
+    const bytes = utf8Bytes(s);
+    const bitLen = bytes.length * 8;
+    bytes.push(0x80);
+    while (bytes.length % 64 !== 56)
+        bytes.push(0);
+    bytes.push(0, 0, 0, 0, (bitLen >>> 24) & 0xff, (bitLen >>> 16) & 0xff, (bitLen >>> 8) & 0xff, bitLen & 0xff);
+    const w = new Array(64);
+    const rotr = (x, n) => (x >>> n) | (x << (32 - n));
+    for (let off = 0; off < bytes.length; off += 64) {
+        for (let i = 0; i < 16; i++)
+            w[i] = ((bytes[off + i * 4] << 24) | (bytes[off + i * 4 + 1] << 16) | (bytes[off + i * 4 + 2] << 8) | bytes[off + i * 4 + 3]) | 0;
+        for (let i = 16; i < 64; i++) {
+            const s0 = rotr(w[i - 15], 7) ^ rotr(w[i - 15], 18) ^ (w[i - 15] >>> 3);
+            const s1 = rotr(w[i - 2], 17) ^ rotr(w[i - 2], 19) ^ (w[i - 2] >>> 10);
+            w[i] = (w[i - 16] + s0 + w[i - 7] + s1) | 0;
+        }
+        let [a, b, c, d, e, f, g, h] = [h0, h1, h2, h3, h4, h5, h6, h7];
+        for (let i = 0; i < 64; i++) {
+            const s1 = rotr(e, 6) ^ rotr(e, 11) ^ rotr(e, 25);
+            const ch = (e & f) ^ (~e & g);
+            const t1 = (h + s1 + ch + k[i] + w[i]) | 0;
+            const s0 = rotr(a, 2) ^ rotr(a, 13) ^ rotr(a, 22);
+            const maj = (a & b) ^ (a & c) ^ (b & c);
+            const t2 = (s0 + maj) | 0;
+            [h, g, f, e, d, c, b, a] = [g, f, e, (d + t1) | 0, c, b, a, (t1 + t2) | 0];
+        }
+        [h0, h1, h2, h3, h4, h5, h6, h7] = [(h0 + a) | 0, (h1 + b) | 0, (h2 + c) | 0, (h3 + d) | 0, (h4 + e) | 0, (h5 + f) | 0, (h6 + g) | 0, (h7 + h) | 0];
+    }
+    return [h0, h1, h2, h3, h4, h5, h6, h7].map((x) => (x >>> 0).toString(16).padStart(8, '0')).join('');
+}
 export const FLOOR = 0.6;
-/** Asks each backend in order for the questions still unanswered; a failed or slow backend answers nothing. */
+/** Asks each backend in order for the questions still unanswered; a failed or slow backend answers nothing.
+ * With `opts.cache`, a stored answer is served as `source: 'cache'` without calling any backend; fresh answers
+ * are stored as `source: 'api'` with the same `usage`/`raw` they carry. */
 export async function decide(state, questions, opts) {
+    const key = opts.cache ? cacheKey(state, questions) : undefined;
+    if (opts.cache && key) {
+        try {
+            const hit = await opts.cache.get(key);
+            if (hit && typeof hit === 'object' && Object.keys(questions).every((k) => Object.hasOwn(hit, k) && hit[k])) {
+                const out = Object.create(null);
+                for (const k of Object.keys(questions))
+                    out[k] = { ...hit[k], source: 'cache' };
+                return out;
+            }
+        }
+        catch {
+            // A broken cache never fails a decision; fall through and ask live.
+        }
+    }
     const out = Object.create(null);
     const open = () => Object.fromEntries(Object.entries(questions).filter(([k]) => !Object.hasOwn(out, k) || out[k].abstained));
     for (const b of opts.backends) {
@@ -42,10 +154,22 @@ export async function decide(state, questions, opts) {
     for (const k of Object.keys(questions))
         if (!Object.hasOwn(out, k))
             out[k] = { answer: null, confidence: 0, abstained: true, reason: 'no backend answered', by: 'none', ms: 0 };
+    for (const k of Object.keys(out))
+        out[k].source = 'api';
+    if (opts.cache && key) {
+        try {
+            await opts.cache.set(key, Object.fromEntries(Object.entries(out).map(([k, a]) => [k, { ...a }])));
+        }
+        catch {
+            // Storing must not fail the answer just decided.
+        }
+    }
     return out;
 }
-/** The floors on one raw answer. Exported for apps that hold a recorded answer. */
+/** The floors on one raw answer. Exported for apps that hold a recorded answer.
+ * `usage`/`raw` on the raw ride through onto the answer, answered or abstained. */
 export function resolve(q, raw) {
+    const carried = { ...(raw?.usage !== undefined && { usage: raw.usage }), ...(raw?.raw !== undefined && { raw: raw.raw }) };
     const keys = q.kind === 'choice' ? Object.keys(q.options) : q.kind === 'yesno' ? ['true', 'false'] : q.levels.map((_, i) => String(i));
     const p = raw?.probabilities;
     const ok = p && Object.keys(p).length === keys.length && keys.every((k) => Object.hasOwn(p, k) && typeof p[k] === 'number' && p[k] >= 0 && p[k] <= 1)
@@ -53,15 +177,15 @@ export function resolve(q, raw) {
         && (raw.confidence === undefined || (raw.confidence >= 0 && raw.confidence <= 1))
         && (raw.pick === undefined || keys.includes(raw.pick));
     if (!ok)
-        return { answer: null, confidence: 0, abstained: true, reason: raw ? 'malformed answer' : 'no answer' };
+        return { answer: null, confidence: 0, abstained: true, reason: raw ? 'malformed answer' : 'no answer', ...carried };
     const ranked = [...keys].sort((a, b) => p[b] - p[a]);
     const picked = raw.pick ?? ranked[0];
     const confidence = raw.confidence ?? p[picked];
     const floor = q.floor ?? FLOOR;
     const own = (k) => (q.kind === 'choice' && q.floors && Object.hasOwn(q.floors, k) ? q.floors[k] : undefined) ?? floor;
     const typed = (k) => (q.kind === 'choice' ? k : q.kind === 'yesno' ? k === 'true' : Number(k));
-    const done = (k, reason) => ({ answer: typed(k), confidence: k === picked ? confidence : p[k], probabilities: p, abstained: false, ...(reason && { reason }) });
-    const abstain = (reason) => ({ answer: null, confidence, probabilities: p, abstained: true, reason });
+    const done = (k, reason) => ({ answer: typed(k), confidence: k === picked ? confidence : p[k], probabilities: p, abstained: false, ...(reason && { reason }), ...carried });
+    const abstain = (reason) => ({ answer: null, confidence, probabilities: p, abstained: true, reason, ...carried });
     if (p[ranked[0]] === p[ranked[1]])
         return abstain('tie');
     // Without a declared floor on the pick, the one floor applies to the answer's confidence, exactly as firstmate's.
@@ -95,3 +219,35 @@ export function rules(fn) {
         },
     };
 }
+/** Any model as a backend: `ask` gets one prompt and returns the model's text (on a phone, the signed-in ChatGPT:
+ *  `(p, signal) => accounts.respond(me, { instructions: '', input: p, signal })`). The model is asked for each option's
+ *  probability as JSON; an answer that isn't that JSON is no answer, so the question abstains. `leaves`: whether the
+ *  state goes off this device (true for any hosted model). */
+export function answerer(o) {
+    return {
+        name: o.name,
+        leaves: o.leaves,
+        async ask(state, questions, signal) {
+            const described = Object.fromEntries(Object.entries(questions).map(([k, q]) => [k,
+                q.kind === 'choice' ? { pick_one_of: q.options, instructions: q.instructions }
+                    : q.kind === 'yesno' ? { yes_or_no: q.question, yes: q.yes, no: q.no, answer_keys: ['true', 'false'] }
+                        : { rate_on: Object.fromEntries(q.levels.map((l, i) => [String(i), l])), instructions: q.instructions }]));
+            const prompt = 'Answer each question about the state below. For each question give every answer key a probability ' +
+                'between 0 and 1, summing to 1. Reply with JSON only, shaped {"<question>": {"<answer key>": <probability>}}.\n\n' +
+                `State: ${JSON.stringify(state)}\n\nQuestions: ${JSON.stringify(described)}`;
+            const text = await o.ask(prompt, signal);
+            let parsed;
+            try {
+                parsed = JSON.parse(text.trim());
+            }
+            catch {
+                return {};
+            }
+            const out = Object.create(null);
+            for (const k of Object.keys(questions))
+                if (parsed?.[k] && typeof parsed[k] === 'object')
+                    out[k] = { probabilities: parsed[k] };
+            return out;
+        },
+    };
+}

package/dist/jev.d.ts

@@ -3,6 +3,12 @@ export declare function jev(opts: {
     key: string;
     via?: 'typesafe' | 'openrouter';
     fetch?: typeof fetch;
+    /** 429 retries after the first attempt (default 2). Only 429 retries, never other 4xx. */
+    maxRetries?: number;
+    /** Wait when a 429 carries no usable Retry-After (default 1000 ms); attempt n waits baseMs * 2^n. */
+    retryBaseMs?: number;
+    /** Each backoff wait is capped here (default 2000 ms), so the total wait stays under maxRetries * retryMaxMs. */
+    retryMaxMs?: number;
 }): Backend;
 /** Jev's answer as a Raw; anything off-shape is undefined, which the floors treat as an abstain. */
 export declare function raw(q: Question, a: any): Raw | undefined;

package/dist/jev.js

@@ -1,25 +1,98 @@
 const BASE = { typesafe: 'https://api.typesafe.ai', openrouter: 'https://openrouter.ai/api' };
 export function jev(opts) {
-    const { key, via = 'typesafe', fetch: f = globalThis.fetch } = opts;
+    const { key, via = 'typesafe', fetch: f = globalThis.fetch, maxRetries = 2, retryBaseMs = 1000, retryMaxMs = 2000 } = opts;
     if (!key)
         throw new Error('jev needs a key');
+    if (!Number.isSafeInteger(maxRetries) || maxRetries < 0)
+        throw new Error('jev maxRetries must be a safe non-negative integer');
+    for (const [name, ms] of [['retryBaseMs', retryBaseMs], ['retryMaxMs', retryMaxMs]]) {
+        if (!Number.isFinite(ms) || ms < 0)
+            throw new Error(`jev ${name} must be a finite number >= 0`);
+    }
     return {
         name: 'jev',
         leaves: true,
         async ask(state, questions, signal) {
-            const res = await f(`${BASE[via]}/v1/systemone`, {
-                method: 'POST',
-                signal,
-                headers: { 'content-type': 'application/json', authorization: `Bearer ${key}` },
-                body: JSON.stringify({ model: 'jev-latest', state, questions: Object.fromEntries(Object.entries(questions).map(([k, q]) => [k, wire(q)])) }),
-            });
-            if (!res.ok)
-                throw new Error(`http ${res.status}`);
-            const answers = (await res.json())?.answers ?? {};
-            return Object.fromEntries(Object.entries(questions).map(([k, q]) => [k, raw(q, Object.hasOwn(answers, k) ? answers[k] : undefined)]));
+            const body = JSON.stringify({ model: 'jev-latest', state, questions: Object.fromEntries(Object.entries(questions).map(([k, q]) => [k, wire(q)])) });
+            for (let attempt = 0;; attempt++) {
+                if (signal.aborted)
+                    throw abortError(signal);
+                const res = await f(`${BASE[via]}/v1/systemone`, {
+                    method: 'POST',
+                    signal,
+                    headers: { 'content-type': 'application/json', authorization: `Bearer ${key}` },
+                    body,
+                });
+                if (res.status === 429 && attempt < maxRetries) {
+                    const wait = Math.min(Math.max(0, parseRetryAfter(res.headers.get('retry-after')) ?? retryBaseMs * 2 ** attempt), retryMaxMs);
+                    await pause(wait, signal);
+                    continue;
+                }
+                if (!res.ok)
+                    throw new Error(`http ${res.status}`);
+                const json = await res.json();
+                const answers = json !== null && typeof json === 'object' ? json.answers : undefined;
+                const usage = parseUsage(json !== null && typeof json === 'object' ? json.usage : undefined);
+                const map = answers !== null && typeof answers === 'object' ? answers : {};
+                return Object.fromEntries(Object.entries(questions).map(([k, q]) => {
+                    if (!Object.hasOwn(map, k))
+                        return [k, undefined];
+                    const r = raw(q, map[k]);
+                    // A per-question response that is off-shape still carries usage/raw: the floors abstain on it.
+                    if (r)
+                        return [k, { ...r, ...(usage && { usage }), raw: json }];
+                    return [k, { probabilities: {}, ...(usage && { usage }), raw: json }];
+                }));
+            }
         },
     };
 }
+/** Seconds ("120", "1.5") or an HTTP date; undefined when absent or unparsable. */
+function parseRetryAfter(header) {
+    const h = header?.trim();
+    if (!h)
+        return undefined;
+    if (/^\d+(?:\.\d+)?$/.test(h))
+        return Number(h) * 1000;
+    const date = Date.parse(h);
+    if (!Number.isNaN(date))
+        return date - Date.now();
+    return undefined;
+}
+/** The counts the backend reported, when it sent valid ones; anything else is absent, never an error. */
+function parseUsage(u) {
+    if (u === null || typeof u !== 'object')
+        return undefined;
+    const out = {};
+    for (const k of ['input_tokens', 'output_tokens']) {
+        const v = u[k];
+        if (v === undefined)
+            continue;
+        if (Number.isSafeInteger(v) && v >= 0)
+            out[k] = v;
+    }
+    return out.input_tokens !== undefined || out.output_tokens !== undefined ? out : undefined;
+}
+/** Sleep that settles promptly when the caller's signal fires, so a decide() timeout is never outwaited. */
+function pause(ms, signal) {
+    if (!(ms > 0))
+        return Promise.resolve();
+    return new Promise((resolve, reject) => {
+        const timer = setTimeout(() => {
+            signal.removeEventListener('abort', onAbort);
+            resolve();
+        }, ms);
+        const onAbort = () => {
+            clearTimeout(timer);
+            reject(abortError(signal));
+        };
+        signal.addEventListener('abort', onAbort, { once: true });
+    });
+}
+/** The caller's abort reason, or a plain error when it carries none. */
+function abortError(signal) {
+    return signal.reason instanceof Error ? signal.reason : new Error(typeof signal.reason === 'string' && signal.reason ? signal.reason : 'aborted');
+}
 function wire(q) {
     if (q.kind === 'choice')
         return { type: 'choice', instructions: q.instructions ?? 'Which option fits the state?', criteria: q.options };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@byokit/decide",
-  "version": "0.1.1",
+  "version": "0.3.0",
   "description": "Typed questions in, a typed answer with confidence out, abstaining below a floor. Rules and Jev backends, and an eval runner.",
   "license": "Apache-2.0",
   "type": "module",
@@ -11,7 +11,7 @@
     "./eval": { "types": "./dist/eval.d.ts", "default": "./dist/eval.js" }
   },
   "bin": { "byokit-eval": "dist/cli.js" },
-  "files": ["dist", "evals"],
+  "files": ["dist", "evals", "CHANGELOG.md"],
   "scripts": { "prepack": "tsc -b" },
   "publishConfig": { "access": "public" }
 }