tinydecide 0.0.0-stage → 0.1.2

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Christian Yun
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,256 @@
-# Temporary Holding Version
+# TinyDecide
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+[![npm](https://img.shields.io/npm/v/tinydecide)](https://www.npmjs.com/package/tinydecide) [![test](https://github.com/Christian1011/tinydecide/actions/workflows/test.yml/badge.svg)](https://github.com/Christian1011/tinydecide/actions/workflows/test.yml) [![bundle size](https://img.shields.io/badge/browser-12%20KB%20gzipped-blue)](#how-it-works) [![license](https://img.shields.io/npm/l/tinydecide)](LICENSE)
+
+Wrap your decision API. TinyDecide learns from its answers and takes over the calls it can answer locally, in microseconds, with agreement certified.
+
+```sh
+npm install tinydecide      # or: bun add tinydecide
+```
+
+```js
+import { shadow } from 'tinydecide';
+
+const classify = shadow('support', (text) => api.classify(text));
+
+await classify('my card was charged twice'); // the same answer your API gives
+```
+
+On day one every call still goes to your API. After 2,000 answers TinyDecide compiles a small local model and certifies how far to trust it. From then on, calls it is sure about are answered in about 10 µs, and the rest still go to your API.
+
+```sh
+npx tinydecide try    # 15 seconds, no API key: TinyDecide learns a simulated API, which then changes
+```
+
+Wrapping an LLM classifier is the common case. [`examples/llm-classifier.js`](examples/llm-classifier.js) does it for any OpenAI-compatible endpoint, defaulting to an open-weights model.
+
+## What you can count on
+
+1. **Your API stays the reference.** Across all calls, the answers you get agree with your API at least 98% of the time, counting your API's own answers as agreeing. This is certified at 95% confidence on traffic like the traffic it learned from. You set the target. With `compile: { guarantee: 'local' }`, the local answers alone meet it too.
+2. **A call never fails because of TinyDecide.** If anything inside it throws (the model, the disk, your custom hooks), the call goes to your API.
+3. **API changes are caught.** A small random sample of local answers is re-checked against your API, and calls your API answers anyway are checked for free. When your API's answers move, every call goes back to it until a model trained on the new answers certifies. Replaying 221,636 real recorded calls produced no false alarms.
+4. **Restarts resume.** In Node, a named decision keeps its answers, model and state in `.tinydecide/<name>/`.
+5. **Training runs off the main thread.** Compiles train in a worker thread. Two steps still pause the main thread: reading the answer log at the start of each compile, and trimming the log once every 25,000 answers. On an M3 Pro each takes about 45 ms with 100,000 short answers, and about 0.6 s when inputs are 2,000 characters long.
+6. **It says when it is not worth it.** Some decisions are beyond a word-level model. TinyDecide tells you and keeps calling your API.
+
+No dependencies. Node 22+, Bun, browsers, workers and edge runtimes; 12 KB gzipped in a browser bundle.
+
+## How a decision graduates
+
+| Mode | Who answers | Until |
+|---|---|---|
+| **learning** | Your API, every call | 2,000 answers are logged and a compile certifies |
+| **local** | The model when its margin clears the certified cutoff; your API otherwise | A live check fails, or your API returns a label the model has never seen |
+| **guarded** | Your API, every call | A compile on answers given after the change certifies |
+
+It recompiles each time the answers double, and at least every 20,000 answers. It logs one line when something changes, never per call:
+
+```
+[tinydecide] support: learning from your API. First compile after 2,000 answers.
+[tinydecide] support: compiled from 2,000 answers: answers 60% of calls locally, with ≥ 98% agreement with your API certified at 95% confidence. 0.16 MB, 8.1 µs per call.
+[tinydecide] support: your API's answers changed for "billing": on calls the model leaned toward "billing" but left to your API, they now agree 9% of the time where the model expected 47%. Every call goes to your API until a model trained on its new answers certifies.
+```
+
+`npx tinydecide status` shows every decision in `.tinydecide/` at a glance.
+
+## Your API's responses
+
+`shadow(name, api)` takes any function of one input. It reads the label from:
+
+- a string: `'billing'`
+- an object with `label`, `choice`, `decision` or `answer`, plus optional `probs` (a label → probability map, or an array in `labels` order). Probabilities help: the model learns from how sure your API was.
+
+Local answers come back in the same shape: a string for a string, or an object with the label under the same key, plus `probs` and `confidence` if your API returns them. Other fields of your API's response are not reproduced. For anything else, pass `choice: (response) => label` (or `{ choice, probs }`) to read it and `respond: (decision) => …` to build local answers.
+
+The input can be a string, `{ a, b }` for a pair, or an object with a string `state`, `text`, `input`, `content` or `message`. The model reads the first of those it finds, or every field when there is none. The whole input is logged with the answer.
+
+## Options
+
+```js
+shadow('support', api, {
+  target: 0.98,         // agreement with your API to certify, overall across all calls
+  group: (input) => input.conversationId, // keep a conversation's calls together when certifying
+  fallback: true,       // answer locally if your API throws and a model exists
+  log: false,           // or (line, event) => logger.info(line)
+});
+```
+
+| Option | Default | |
+|---|---|---|
+| `target` | `0.98` | Agreement to certify. Overall: your API's own answers count as agreeing |
+| `delta` | `0.05` | 1 − confidence of the certificate |
+| `audit` | `0.02` | Share of calls your API also answers in the background, to check agreement live |
+| `group` | none | Keeps related calls in one split, so the certificate covers new conversations, not just new messages |
+| `labels` | inferred | Fixes the label set and order. Needed when your API returns probability arrays; also names index labels |
+| `fallback` | `false` | Answer locally when your API throws |
+| `log` | `true` | One line per lifecycle event. `false`, or a function to route it |
+| `onEvent` | none | Every event as an object: `start`, `compile:start`, `compile`, `install`, `guard`, `discard`, `audit`, `unreadable`, `error` |
+| `dir` | `.tinydecide` | Node: where named decisions live. Also `$TINYDECIDE_DIR` |
+| `store` | file in Node, memory elsewhere | Where answers, the model and its state live (see below) |
+| `model` | none | Start from a compiled model |
+| `autoCompile` | `{ min: 2000, growth: 2, maxEvery: 20000, window: 50000 }` | `false` compiles only on `decide.compile()` |
+| `choice`, `respond` | built in | Read custom responses, build custom local answers |
+
+Misspelled options, and out-of-range `target`, `delta` or `audit`, throw at startup with a message that says what to write instead.
+
+The function you get back also has:
+
+- `decide.detailed(input)` returns `{ value, source, choice, confidence, ms }`; `source` is `'local'`, `'api'` or `'fallback'`.
+- `decide.stats()` returns the mode, counts, calls saved, and live agreement with its interval.
+- `decide.ready` resolves once a saved model is loaded. Calls before that go to your API.
+- `decide.compile()` compiles now. `decide.close()` stops compiling and closes the store.
+- `decide.mode` and `decide.model` are read-only.
+
+## Where things live
+
+In Node, `shadow('support', api)` keeps one folder:
+
+```
+.tinydecide/
+  .gitignore        written once: the folder holds raw inputs, so git ignores it
+  support/
+    answers.jsonl   your API's recent answers, trimmed to the newest 100,000
+    model.tdm       the installed model, and report.json, its compile report
+    state.json      only while guarded, so a restart stays guarded
+    lock            the pid of the process using the folder
+```
+
+Model and state writes are atomic, and a line cut off by a crash is skipped. The lock keeps each folder to one process. If the folder can't be opened (it is read-only, or another process holds it), answers stay in memory and one line is logged.
+
+To keep answers elsewhere, pass a `store`: an object with `append(record)` and `load()`, plus optional `size`, `loadModel`, `saveModel`, `loadState`, `saveState` and `close` (any of them may be async). `memoryStore()` and `fileStore(dir)` are the built-in ones.
+
+## Deploying
+
+**One long-running server process.** Nothing to do. Answers and models persist in `.tinydecide/`. If a process manager runs several copies (`pm2 -i`, Node cluster), the first one keeps the folder and the others keep their answers in memory.
+
+**Several instances, or serverless.** Each instance can learn on its own, but they learn faster together. Send answers to one place with a custom `store`, compile on a schedule, and ship the model:
+
+```sh
+npx tinydecide compile answers.jsonl --out support.tdm
+```
+
+```js
+import { loadModel, shadow } from 'tinydecide';
+
+const classify = shadow('support', api, { model: loadModel('support.tdm'), autoCompile: false, store });
+```
+
+The live checks still run on every instance and fall back to your API on their own.
+
+**Browsers, workers and edge.** The same import works; answers stay in memory. Load a compiled model with `Model.fromBytes(await (await fetch('/support.tdm')).arrayBuffer())` and pass it as `model`.
+
+## The guarantee
+
+Each compile splits the logged answers into train, calibration and test, by `group` or else by normalized text so repeats never leak. It trains, quantizes to int8, and then runs Learn-Then-Test with fixed-sequence binomial testing over a margin grid fixed in advance. It picks the loosest cutoff for which
+
+> with probability at least 1 − `delta` over the calibration draw, agreement with your API on traffic like the calibration rows is at least `target`.
+
+Only the uniform random sample calibrates. Calls logged because the model deferred them are biased toward hard cases, so they train but never certify. Every number in a report is measured on the int8 model that ships. Pass `compile: { guarantee: 'local' }` to certify agreement on local answers alone; it answers fewer calls.
+
+The certificate covers traffic like the calibration sample. The live checks cover change:
+
+| Check | Watches | Trips when |
+|---|---|---|
+| Audit | The random sample of local answers, re-asked of your API | A CUSUM test finds overall agreement below target. About 2% false alarms over a model's life when agreement sits exactly at target, fewer above it |
+| Deferred | Calls your API answers anyway, against the model's guess | Agreement falls far below calibration overall, or for one label far below what the model's own calibrated confidence predicts |
+| Schema | Your API's labels | A label appears that the model has never seen |
+
+A compile that is running when a check trips is cancelled and never installed, because its answers straddle the change.
+
+## Results
+
+### Replay on real decision-API traffic
+
+Fifteen decisions, with answers recorded from a production decision model. For each one: compile on the first N calls, then answer later calls the model never saw. Target 98%, delta 5%.
+
+| Decision | Calls compiled | Answered locally | Agreement [95% CI] | Verdict |
+|---|---:|---:|---|---|
+| Toxic chat prompts | 2,000 | 87.2% | 98.3% [97.1–99.1] | ship |
+| Support intents (CLINC, 151 labels) | 10,000 | 73.1% | 98.6% [98.0–98.9] | ship |
+| PubMed sentence role † | 50,000 | 67.6% | 98.3% [97.7–98.7] | ship |
+| Economic news relevance | 2,000 | 67.1% | 99.2% [98.5–99.6] | ship |
+| Dialog emotion † | 20,000 | 62.4% | 98.2% [97.7–98.6] | ship |
+| Voice-assistant intents (SLURP, 60 labels) | 5,000 | 44.1% | 99.1% [98.6–99.4] | ship |
+| Adverse drug events | 10,000 | 38.1% | 98.3% [97.7–98.7] | ship |
+| Airline sentiment | 5,000 | 36.7% | 99.0% [98.4–99.3] | ship |
+| Hate / offensive speech | 10,000 | 31.6% | 98.5% [98.0–98.9] | ship |
+| Sarcastic headlines | 20,000 | 19.8% | 98.6% [98.1–99.0] | marginal |
+| Relation types (SemEval, 19 labels) | 5,000 | 19.5% | 98.5% [97.7–99.0] | marginal |
+| Sexism detection (EDOS) | 5,000 | 11.9% | 98.6% [98.0–99.0] | marginal |
+| Adversarial hate (DynaHate) | 5,000 | 7.5% | 99.4% [98.8–99.7] | marginal |
+| Scientific inference pairs (SciNLI) † | 20,000 | 4.1% | 99.1% [98.7–99.4] | not worth it |
+| Paraphrase pairs (PAWS) | 10,000 | 3.6% | 98.4% [97.8–98.8] | not worth it |
+
+† The recorded answers mix the decision model with other labellers.
+
+Across all 56 (decision, N) cells from 1,000 to 50,000 calls, 3 measured below 98%, close to the 5% that `delta` allows; 1 was clearly below. Pooled agreement was 98.8%. On a deliberately shifted future set built from clusters the compile never saw, 9 of 56 measured below and 2 clearly. The recorded traffic is research data and is not distributed.
+
+### Live, against a real API
+
+TinyDecide wrapped an open-weights model on OpenRouter (Mistral Nemo) labelling 3,000 public airline tweets. Total cost $0.0034. After the first compile at 2,000 answers, the next 1,000 calls were 24% local with overall agreement 98.9% [98.0–99.4]. API p50 541 ms; local 16.8 µs.
+
+### The guard
+
+- **False alarms:** none in 221,636 recorded calls across all 15 decisions, replayed with no API change.
+- **Detection:** when a simulated 10-label API starts filing one label under another, the guard trips after a median of 1,226 calls (754–3,657 over 10 runs), with 25–309 wrong local answers in the meantime.
+
+Both are in [`bench/results/guard-2026-10-24.md`](bench/results/guard-2026-10-24.md); `npm run bench:guard` reruns the detection runs.
+
+## Command line
+
+```
+tinydecide try                  a 15 s simulation: learn a simulated API, which then changes
+tinydecide status [dir]         what each decision is doing
+tinydecide compile <answers>    compile from a .jsonl log or a decision folder
+tinydecide inspect <model.tdm>  labels and certificate
+tinydecide decide <model.tdm>   answer inputs from arguments or stdin
+```
+
+`compile` prints the verdict, a held-out check and what other targets would allow:
+
+```
+  tinydecide compile  traffic.jsonl → traffic.tdm
+
+  10,000 calls · 151 labels · target 98.0% overall agreement (95% confidence)
+
+  ✓ ship   Answers 69.1% of calls locally while keeping overall agreement with your API ≥ 98% (95% confidence).
+
+  held-out check   1,015 calls   69.0% local   98.4% agreement [97.5%–99.0%]
+  local answers    97.7% agree with your API
+  what if          90% → 96%   95% → 83%   97% → 76%   98% → 69%   99.0% → 56%   99.5% → 36%
+  model            0.66 MB · 15,948 features · 11.1 µs per call · trained in 3.5 s
+```
+
+Answer logs are one JSON object per line: `{"input": "…", "choice": "billing"}`, optionally with `probs` and `group`. The label may also be under `label`, `decision` or `answer`.
+
+## How it works
+
+- **Features.** Text is normalized (NFKC, lowercase). Features are word unigrams and bigrams, number shapes, and character 2–4 grams within words, hashed to 32 bits. Han and kana characters are their own tokens. Pairs get side-aware and overlap features.
+- **Model.** Words and characters are pooled separately and summed into a linear head, fastText-style, trained with SGD on your API's soft labels and stored as int8 with one scale per row.
+- **Cutoff.** The score is the margin between the top two logits. A fitted temperature makes the reported confidence match how often your API agrees. It does not change the cutoff.
+- **Size and speed.** 0.2–6 MB per decision, most under 3 MB. On an M3 Pro, a call takes 8–30 µs for short texts, 45–70 µs for pairs, and about 200 µs for news articles a few thousand characters long.
+
+## Limits
+
+1. **Word-level models.** Decisions that turn on meaning across a pair, or on world knowledge, mostly stay with your API. The verdict tells you which.
+2. **Detection takes calls.** A change confined to one label of ten was caught after a median of about 1,200 calls. A higher `audit` rate catches changes sooner and saves less.
+3. **The answer log holds raw inputs.** Keep `.tinydecide/` private, redact inputs before they reach `shadow`, or use your own `store`.
+4. **One input, one label.** Multi-label and free-text outputs are out of scope.
+5. **Long inputs need memory to compile.** Features take about 22 KB per training row at 2,000 characters, about 0.7 GB at the default 50,000-answer window. Lower `autoCompile.window` for long texts on small machines.
+
+## Development
+
+```sh
+npm test               # 49 tests, about 15 s
+npm run test:bun       # the same tests under Bun
+npm run typecheck      # the public types, with tsc
+npm run bench:guard    # how fast the guard catches an API change, over 10 runs
+```
+
+Issues and pull requests are welcome; please run the tests first. Security reports go through [SECURITY.md](SECURITY.md).
+
+To release: bump `version` in package.json, add a CHANGELOG entry, and push a `vX.Y.Z` tag. CI runs the tests and stages the version on npm, and it goes public once a maintainer approves it with 2FA.
+
+## License
+
+MIT

package/bin/tinydecide.js

@@ -0,0 +1,253 @@
+#!/usr/bin/env node
+import { existsSync, readdirSync, readFileSync, statSync, writeFileSync } from 'node:fs';
+import { basename, join } from 'node:path';
+import { createInterface } from 'node:readline';
+import { supportApi } from '../examples/support-api.js';
+import { compile, loadModel, memoryStore, readJsonl, saveModel, shadow } from '../src/node.js';
+
+const HELP = `tinydecide: wrap your decision API; answer the calls it can locally, with agreement certified
+
+usage
+  tinydecide try                 a 15 s simulation, no keys: learn a simulated API, which then changes
+  tinydecide status [dir]        what each decision in .tinydecide/ is doing
+  tinydecide compile <answers>   compile a model from a .jsonl log, or a decision folder
+                     [--out model.tdm] [--target 0.98] [--delta 0.05] [--guarantee overall|local]
+                     [--labels a,b,c] [--audit 0.02] [--report report.json]
+  tinydecide inspect <model.tdm> print a model's labels and certificate
+  tinydecide decide  <model.tdm> [text …]   answer inputs; reads one per line from stdin when none are given
+
+in code
+  import { shadow } from 'tinydecide';
+  const classify = shadow('support', (text) => yourApi.classify(text));`;
+
+const tty = process.stdout.isTTY && !process.env.NO_COLOR;
+const paint = (code) => (s) => (tty ? `\x1b[${code}m${s}\x1b[0m` : String(s));
+const dim = paint('2'), bold = paint('1'), green = paint('32'), yellow = paint('33'), red = paint('31'), blue = paint('34');
+const pct = (x, digits = 1) => (x == null ? '—' : `${(x * 100).toFixed(digits)}%`);
+const int = (x) => Math.round(x).toLocaleString('en-US');
+
+function parse(argv) {
+  const pos = [], flags = {};
+  for (let i = 0; i < argv.length; i++) {
+    const a = argv[i];
+    if (a.startsWith('--')) {
+      const [k, v] = a.slice(2).split('=', 2);
+      flags[k] = v ?? (argv[i + 1] && !argv[i + 1].startsWith('--') ? argv[++i] : true);
+    } else pos.push(a);
+  }
+  return { pos, flags };
+}
+
+const VERDICT = {
+  ship: () => green('✓ ship'),
+  marginal: () => yellow('~ marginal'),
+  'not-worth-it': () => red('× not worth it'),
+  'need-more-data': () => yellow('… need more data'),
+};
+
+// ── try ──────────────────────────────────────────────────────────────────────────────────────────────
+
+async function cmdTry(f) {
+  const CALLS = +(f.calls ?? 14_000), CHANGE_AT = Math.round(CALLS * 0.55), RATE = +(f.rate ?? 1000);
+  const API_MS = 400, PRICE = 0.4 / 1000;
+  const sim = supportApi({ seed: +(f.seed ?? 1) });
+  const out = process.stdout;
+  const width = Math.min(out.columns || 100, 110);
+  // before the change · between the change and the guard · after the guard
+  const phase = [{ local: 0, right: 0, calls: 0 }, { local: 0, right: 0, calls: 0 }, { local: 0, right: 0, calls: 0 }];
+  let i = 0, p = 0, status = '', last = null;
+  const recent = new Uint8Array(500), agree = new Uint8Array(500);
+  let agreeN = 0;
+
+  const wrap = (text, indent) => {
+    const lines = [];
+    let line = '';
+    for (const word of text.split(' ')) {
+      if (line && indent + visible(line).length + 1 + visible(word).length > width - 2) { lines.push(line); line = word; } else line = line ? `${line} ${word}` : word;
+    }
+    lines.push(line);
+    return lines.join(`\n${' '.repeat(indent)}`);
+  };
+  const clear = () => { if (tty && status) out.write('\r\x1b[2K'); };
+  const draw = () => {
+    if (!tty) return;
+    const share = recent.reduce((a, x) => a + x, 0) / Math.min(Math.max(i, 1), recent.length);
+    const n = Math.min(agreeN, agree.length), ok = agree.reduce((a, x) => a + x, 0);
+    status = `  ${dim('calls')} ${int(i).padStart(6)}   ${dim('answered locally')} ${blue(pct(share, 0).padStart(4))}   ${dim('agreement on recent local answers')} ${n ? pct(ok / n) : '—'}`;
+    out.write(`\r\x1b[2K${status}`);
+  };
+  const say = (line) => { clear(); out.write(`${dim(int(i).padStart(8))}  ${wrap(line, 10)}\n`); draw(); };
+
+  out.write(`\n  ${bold('tinydecide try')}\n\n`);
+  out.write(`  ${wrap(`A simulated support-ticket classifier stands in for your API: ${sim.labels.length} labels, ${API_MS} ms and $0.40 per 1,000 calls (assumed). TinyDecide wraps it. Partway through, the API changes how it labels some messages.`, 2)}\n\n`);
+
+  const decide = shadow('support', sim.api, {
+    store: memoryStore(),
+    log: (line, e) => {
+      if (e.type === 'compile') last = e.report;
+      if (e.type === 'guard' && p === 1) p = 2; // only after the scripted change
+      say(line.replace(/^\[tinydecide\] support: /, ''));
+    },
+  });
+
+  const t0 = Date.now();
+  for (; i < CALLS; i++) {
+    if (i === CHANGE_AT) { p = 1; say(yellow(`▸ ${sim.change()}. Nothing tells TinyDecide.`)); }
+    const text = sim.next();
+    const d = await decide.detailed(text);
+    const isLocal = d.source === 'local';
+    recent[i % recent.length] = isLocal ? 1 : 0;
+    phase[p].calls++;
+    if (isLocal) {
+      const right = d.choice === sim.truth(text);
+      phase[p].local++;
+      if (right) phase[p].right++;
+      agree[agreeN++ % agree.length] = right ? 1 : 0;
+    }
+    if (i % 25 === 0) {
+      draw();
+      const ahead = ((i + 1) / RATE) * 1000 - (Date.now() - t0);
+      await new Promise((r) => setTimeout(r, Math.max(0, ahead)));
+    }
+  }
+  const s = decide.stats();
+  clear();
+  await decide.close();
+
+  const local = phase.reduce((a, x) => a + x.local, 0);
+  const row = (k, v) => out.write(`  ${dim(k.padEnd(18))}${wrap(v, 20)}\n`);
+  // Overall agreement, as certified: your API's own answers count as agreeing.
+  const rate = (x) => (x.calls ? pct(1 - (x.local - x.right) / x.calls) : '—');
+  out.write(`\n  ${bold('Done.')} ${int(CALLS)} calls, ${int(local)} answered locally ${dim(`(${pct(local / CALLS, 0)})`)}.\n\n`);
+  row('agreement', `${rate(phase[0])} before the change, ${rate(phase[2])} after the guard ${dim('(overall, as certified; checked on every call, which only a simulation can do)')}`);
+  if (p === 2) row('the change', `caught after ${int(phase[1].calls)} calls; ${int(phase[1].local - phase[1].right)} local answers were wrong in that window`);
+  row('API calls saved', `${int(s.apiCallsSaved)} ${dim(`$${(s.apiCallsSaved * PRICE).toFixed(2)} and ${(s.apiCallsSaved * API_MS / 60_000).toFixed(0)} min of waiting, at the assumed price and latency`)}`);
+  if (last) row('time per answer', `${last.latencyUs} µs locally, vs ${API_MS} ms from your API`);
+  out.write(`\n  ${dim('Next: wrap your own API')}\n\n    import { shadow } from 'tinydecide';\n    const classify = shadow('support', (text) => yourApi.classify(text));\n\n`);
+}
+
+const visible = (s) => s.replace(/\x1b\[[0-9;]*m/g, '');
+
+// ── status ───────────────────────────────────────────────────────────────────────────────────────────
+
+function cmdStatus(dir = process.env.TINYDECIDE_DIR || '.tinydecide') {
+  if (!existsSync(dir)) {
+    console.log(`\n  No decisions in ${dir}/ yet. Wrap your API with shadow('name', api) and run your app.\n`);
+    return;
+  }
+  const names = readdirSync(dir).filter((n) => statSync(join(dir, n)).isDirectory()).sort();
+  console.log(`\n  ${bold(dir + '/')}\n`);
+  if (!names.length) console.log(dim('  (empty)'));
+  const width = Math.max(...names.map((n) => n.length), 8);
+  for (const name of names) {
+    const at = (f) => join(dir, name, f);
+    const answers = existsSync(at('answers.jsonl')) ? countLines(at('answers.jsonl')) : 0;
+    const report = readJson(at('report.json'));
+    const state = readJson(at('state.json'));
+    const mode = state?.guardedSince ? 'guarded' : existsSync(at('model.tdm')) ? 'local' : 'learning';
+    const color = { local: blue, learning: dim, guarded: yellow }[mode];
+    let detail;
+    if (mode === 'guarded') detail = `back to your API ${ago(state.guardedSince)}; relearning from its new answers`;
+    else if (mode === 'local' && report) detail = `answers about ${pct(report.calib.share, 0)} locally at ≥ ${pct(report.target, 0)} agreement · compiled ${ago(Date.parse(statSync(at('model.tdm')).mtime))}`;
+    else detail = report ? `last compile: ${report.verdict}` : 'every call goes to your API until the first compile';
+    console.log(`  ${name.padEnd(width)}  ${color(mode.padEnd(8))}  ${int(answers).padStart(8)} answers   ${dim(detail)}`);
+  }
+  console.log('');
+}
+
+function ago(t) {
+  const s = (Date.now() - t) / 1000;
+  if (s < 90) return 'just now';
+  if (s < 5400) return `${Math.round(s / 60)} min ago`;
+  if (s < 129_600) return `${Math.round(s / 3600)} h ago`;
+  return `${Math.round(s / 86_400)} days ago`;
+}
+
+function readJson(path) {
+  try { return JSON.parse(readFileSync(path, 'utf8')); } catch { return null; }
+}
+
+function countLines(path) {
+  const buf = readFileSync(path);
+  let n = 0;
+  for (let i = 0; i < buf.length; i++) if (buf[i] === 10) n++;
+  return n;
+}
+
+// ── compile, inspect, decide ─────────────────────────────────────────────────────────────────────────
+
+async function cmdCompile(file, f) {
+  if (!file) throw new Error('compile needs a .jsonl file of answers, or a decision folder like .tinydecide/support');
+  if (existsSync(file) && statSync(file).isDirectory()) file = join(file, 'answers.jsonl');
+  if (!existsSync(file)) throw new Error(`no such file: ${file}`);
+  let skippedLines = 0;
+  const records = readJsonl(file, { onSkip: () => skippedLines++ });
+  const out = f.out ?? file.replace(/\.jsonl(\.gz)?$/, '') + '.tdm';
+  const opts = {
+    target: f.target ? +f.target : undefined, delta: f.delta ? +f.delta : undefined,
+    guarantee: f.guarantee, audit: f.audit ? +f.audit : undefined, seed: f.seed ? +f.seed : undefined,
+    labels: f.labels ? String(f.labels).split(',') : undefined,
+  };
+  for (const k of Object.keys(opts)) if (opts[k] === undefined) delete opts[k];
+  let shown = -1;
+  const { model, report } = await compile(records, {
+    ...opts,
+    onProgress: ({ phase, done }) => {
+      if (!process.stderr.isTTY || phase !== 'train') return;
+      const p = Math.floor(done * 20);
+      if (p !== shown) { shown = p; process.stderr.write(`\r  training ${'▮'.repeat(p)}${dim('▯'.repeat(20 - p))}`); }
+    },
+  });
+  if (process.stderr.isTTY) process.stderr.write('\r\x1b[2K');
+  saveModel(out, model);
+  if (f.report) writeFileSync(f.report, JSON.stringify(report, null, 2) + '\n');
+
+  const r = report, t = r.test;
+  const scope = r.guarantee === 'local' ? 'agreement on local answers' : 'overall agreement';
+  console.log(`\n  ${bold('tinydecide compile')}  ${basename(file)} ${dim('→')} ${basename(out)}\n`);
+  console.log(dim(`  ${int(r.rows.total)} calls · ${r.labels.length} labels · target ${pct(r.target, 1)} ${scope} (${pct(1 - r.delta, 0)} confidence)${skippedLines ? ` · ${skippedLines} unreadable lines skipped` : ''}\n`));
+  console.log(`  ${VERDICT[r.verdict]()}   ${r.summary}\n`);
+  const row = (k, v) => console.log(`  ${dim(k.padEnd(15))}  ${v}`);
+  if (t) row('held-out check', `${int(t.n)} calls   ${pct(t.share)} local   ${pct(t.overallAgreement)} agreement ${dim(`[${pct(t.overallCI[0])}–${pct(t.overallCI[1])}]`)}`);
+  if (t?.localAgreement != null) row('local answers', `${pct(t.localAgreement)} agree with your API`);
+  row('what if', r.whatIf.map((w) => `${pct(w.target, w.target >= 0.99 ? 1 : 0)} ${dim('→')} ${pct(w.share, 0)}`).join('   '));
+  row('model', `${(r.bytes / 1e6).toFixed(2)} MB · ${int(r.features)} features · ${r.latencyUs} µs per call · trained in ${(r.ms.train / 1000).toFixed(1)} s`);
+  console.log('');
+}
+
+function cmdInspect(file) {
+  if (!file) throw new Error('inspect needs a .tdm model file');
+  const m = loadModel(file);
+  console.log(JSON.stringify({
+    labels: m.labels, features: m.keys.length, d: m.d, bytes: m.bytes, temperature: m.T,
+    cutoff: Number.isFinite(m.cutoff) ? m.cutoff : null, certificate: m.meta?.certificate ?? {}, expected: m.meta?.expected, created: m.meta?.created,
+  }, null, 2));
+}
+
+async function cmdDecide(file, texts) {
+  if (!file) throw new Error('decide needs a .tdm model file');
+  const m = loadModel(file);
+  const one = (s) => {
+    let input = s;
+    try { const j = JSON.parse(s); if (j && typeof j === 'object') input = j; } catch {}
+    const d = m.decide(input);
+    console.log(JSON.stringify({ choice: d.choice, confidence: +d.confidence.toFixed(4), local: d.local }));
+  };
+  if (texts.length) return texts.forEach(one);
+  for await (const line of createInterface({ input: process.stdin })) if (line.trim()) one(line);
+}
+
+const { pos, flags } = parse(process.argv.slice(2));
+const [cmd, ...rest] = pos;
+try {
+  if (cmd === 'try') await cmdTry(flags);
+  else if (cmd === 'status') cmdStatus(rest[0]);
+  else if (cmd === 'compile') await cmdCompile(rest[0], flags);
+  else if (cmd === 'inspect') cmdInspect(rest[0]);
+  else if (cmd === 'decide') await cmdDecide(rest[0], rest.slice(1));
+  else if (!cmd || cmd === 'help' || cmd === '-h' || flags.help) console.log(HELP);
+  else throw new Error(`unknown command "${cmd}". Run tinydecide help.`);
+} catch (err) {
+  console.error(`tinydecide: ${err.message}`);
+  process.exit(1);
+}

package/examples/support-api.js

@@ -0,0 +1,118 @@
+// A simulated support-ticket classifier, used by `tinydecide try` in place of a real API.
+//
+// It labels customer messages the way an LLM classifier does at temperature 0: the same text always
+// gets the same label, clear messages get the obvious label (with a little noise), and vague or
+// mixed messages get a label no word-level model could predict. No network, no keys.
+
+import { hashString, rng } from '../src/hash.js';
+
+export const LABELS = ['refund', 'billing', 'login', 'order_status', 'change_address', 'cancel', 'upgrade', 'bug', 'feature_request', 'human'];
+
+const SLOTS = {
+  item: ['my headphones', 'the blender', 'the jacket', 'the charger', 'the desk lamp', 'my shoes', 'the coffee maker', 'the backpack', 'the keyboard', 'my phone case', 'the monitor', 'the rug', 'the tent', 'the kettle', 'my sunglasses'],
+  plan: ['pro', 'the team plan', 'premium', 'the annual plan', 'business', 'the family plan'],
+  card: ['visa', 'credit card', 'bank', 'amex', 'paypal', 'debit card'],
+  device: ['iphone', 'android', 'my laptop', 'chrome', 'safari', 'the ipad app', 'windows'],
+  city: ['denver', 'austin', 'seattle', 'boston', 'chicago', 'portland', 'atlanta', 'miami', 'phoenix', 'toronto'],
+  action: ['upload a photo', 'check out', 'open settings', 'export a report', 'log out', 'search', 'save my profile', 'add a card'],
+  feature: ['search', 'checkout', 'the dashboard', 'notifications', 'sync', 'the mobile app', 'file upload', 'the calendar'],
+  wish: ['dark mode', 'bulk export', 'a public api', 'offline mode', 'calendar sync', 'sms login', 'more themes', 'csv import'],
+  integration: ['slack', 'zapier', 'shopify', 'google sheets', 'salesforce', 'notion', 'hubspot'],
+  code: ['500', '403', 'E1029', '404', 'TIMEOUT', 'E-77', '502'],
+};
+
+const TEMPLATES = {
+  refund: ['i want a refund for {item}', 'can i get my money back for {item}', 'please refund order {order}', '{item} arrived broken, i want a refund', 'how do i return {item}', 'requesting a refund, {item} is not what i ordered', 'refund please, {item} stopped working after a week', 'i returned {item} two weeks ago and still no refund'],
+  billing: ['i was charged twice for {plan}', 'there are two charges of ${amount} on my {card}', 'why was i billed ${amount} this month', 'unexpected charge on my {card} statement', 'my invoice says ${amount} but it should be less', 'duplicate charge for order {order}', 'what is this ${amount} charge'],
+  login: ["i can't log in", 'my password reset email never arrives', 'locked out of my account after too many attempts', "the 2fa code isn't working", 'it says invalid password but i know it is right', "can't sign in on {device}", 'login page keeps spinning on {device}', 'forgot my password and the reset link expired'],
+  order_status: ['where is my order {order}', "my package hasn't arrived yet", 'tracking says delivered but i got nothing', 'when will {item} ship', 'order {order} is stuck in processing', 'any update on {item}? it has been ten days', 'has order {order} shipped'],
+  change_address: ['i need to change my shipping address', 'please send order {order} to {city} instead', 'i moved to {city}, update my address', 'wrong address on order {order}', 'can i change where {item} gets delivered'],
+  cancel: ['cancel my subscription', 'how do i cancel {plan}', "please don't renew my plan", 'i want to close my account', 'cancel {plan} at the end of the month', 'end my membership please', 'how do i stop my subscription'],
+  upgrade: ['how do i upgrade to {plan}', 'what does {plan} cost', 'i need more seats for my team', 'switch me to {plan}', 'is there a discount for {plan}', 'can i try {plan} before paying', 'upgrade my account to {plan}'],
+  bug: ['the app crashes when i {action}', '{feature} is broken on {device}', 'getting error {code} when i {action}', "the page won't load on {device}", '{feature} has been down since this morning', 'error {code} every time i {action}', '{feature} shows a blank screen'],
+  feature_request: ['can you add {wish}', 'it would be great if you supported {integration}', 'please add {wish}', 'any plans for a {integration} integration', 'feature request: {wish}', 'would love {wish} in the app'],
+  human: ['talk to a human', 'can i speak to an agent', 'get me a real person', 'is there a phone number i can call', 'i need to talk to someone', 'transfer me to support staff'],
+};
+
+const VAGUE = ["it's not working", 'help', 'i have a problem with my account', 'something is wrong', 'please fix this', 'i need help with my order', 'this is ridiculous', 'hello?', 'are you there', 'why is this happening', 'ok', 'same problem as before'];
+const PREFIX = ['', '', '', 'hi, ', 'hello, ', 'hey team, ', 'urgent: ', 'quick question: ', 'good morning, '];
+const SUFFIX = ['', '', '', ' thanks', ' asap', ' please', '!!', ' thx', ' thank you'];
+
+/**
+ * @param {{ seed?: number }} [options]
+ * @returns {{ labels: string[], next(): string, api(text: string): Promise<string>, truth(text: string): string, change(): string }}
+ */
+export function supportApi({ seed = 1 } = {}) {
+  const rand = rng(seed);
+  const pick = (a) => a[Math.floor(rand() * a.length)];
+  const answers = new Map();
+  let changed = false;
+
+  function fill(template) {
+    return template.replace(/\{(\w+)\}/g, (_, k) => {
+      if (k === 'order') return `#${10000 + Math.floor(rand() * 90000)}`;
+      if (k === 'amount') return (5 + rand() * 195).toFixed(2);
+      return pick(SLOTS[k]);
+    });
+  }
+
+  function typo(text) {
+    const words = text.split(' ');
+    const i = Math.floor(rand() * words.length), w = words[i];
+    if (w.length < 4) return text;
+    const j = 1 + Math.floor(rand() * (w.length - 2));
+    words[i] = w.slice(0, j) + w[j + 1] + w[j] + w.slice(j + 2);
+    return words.join(' ');
+  }
+
+  // The label is fixed per text, as with a deterministic API: a hash picks among the candidates.
+  const coin = (text, salt) => (hashString(text, salt) % 10_000) / 10_000;
+
+  function next() {
+    const u = rand();
+    let text, label;
+    if (u < 0.12) {
+      text = pick(VAGUE);
+      const options = ['human', 'bug', 'login', 'order_status', 'billing'];
+      label = options[Math.floor(coin(text + rand().toFixed(1), 7) * options.length)];
+    } else if (u < 0.22) {
+      const a = pick(LABELS), b = pick(LABELS);
+      text = `${fill(pick(TEMPLATES[a]))}${pick([' and ', '. also ', ', plus '])}${fill(pick(TEMPLATES[b]))}`;
+      label = coin(text, 11) < 0.55 ? a : b;
+    } else {
+      label = pick(LABELS);
+      text = fill(pick(TEMPLATES[label]));
+      if (coin(text, 13) < 0.03) label = pick(LABELS); // an LLM is not perfectly consistent either
+    }
+    text = pick(PREFIX) + text + pick(SUFFIX);
+    if (rand() < 0.08) text = typo(text);
+    if (rand() < 0.2) text = text[0].toUpperCase() + text.slice(1);
+    if (!answers.has(text)) answers.set(text, label);
+    return text;
+  }
+
+  function truth(text) {
+    const label = answers.get(text) ?? route(text);
+    return changed && label === 'billing' ? 'refund' : label;
+  }
+
+  return {
+    labels: LABELS,
+    next,
+    truth,
+    api: async (text) => truth(text),
+    change() { changed = true; return 'your API now files duplicate and unexpected charges under refund'; },
+  };
+}
+
+/** A keyword router for texts the generator never produced, such as ones typed by hand. */
+function route(text) {
+  const t = text.toLowerCase();
+  for (const [label, words] of [
+    ['refund', ['refund', 'money back', 'return']], ['billing', ['charge', 'billed', 'invoice']],
+    ['login', ['log in', 'password', 'sign in', '2fa', 'locked out']], ['order_status', ['where is', 'tracking', 'shipped', 'arrived']],
+    ['change_address', ['address']], ['cancel', ['cancel', 'renew', 'close my account']], ['upgrade', ['upgrade', 'seats', 'plan']],
+    ['bug', ['crash', 'error', 'broken', "won't load"]], ['feature_request', ['add ', 'support', 'integration']],
+  ]) if (words.some((w) => t.includes(w))) return label;
+  return 'human';
+}