@matchbox-ai/train 0.1.0 → 0.2.1

package/docs/examples/time.md

@@ -0,0 +1,43 @@
+# Date, time, and duration parsing
+
+This repository example trains a token model for durations, relative offsets, and clock times on today or tomorrow. The output stays relative to avoid choosing an application timezone or reference date.
+
+| Input                 | Output                                                      |
+| --------------------- | ----------------------------------------------------------- |
+| `for 90 minutes`      | `{ kind: "duration", seconds: 5400 }`.                      |
+| `in two hours`        | `{ kind: "relative", seconds: 7200 }`.                      |
+| `tomorrow at 3:30 pm` | `{ kind: "datetime", dayOffset: 1, hour: 15, minute: 30 }`. |
+
+## Run the example
+
+From the Matchbox repository:
+
+```sh
+bun install
+bun run build:packages
+bun run matchbox train examples/time
+bun run matchbox eval examples/time
+bun run matchbox dev examples/time
+```
+
+The website's [examples page](/training) runs the exported model. This is a repository example, not a CLI template in 0.1.0.
+
+## What learns, what is code
+
+The model learns quantity, unit, duration/offset cue, day, clock, and meridiem labels from annotated examples. Training-only annotations live in `data/train-spans.json`.
+
+The browser decoder in `decode/` reads those labels, converts digits or a small explicit number-word vocabulary, multiplies units, and validates clock ranges. It can produce numeric amounts absent from training because it copies recognized input spans instead of classifying finite output values.
+
+A duration is elapsed seconds. A relative result is an offset in seconds from a reference instant your app chooses. A datetime result is a local clock time on a relative calendar day. The application must resolve timezone, reference date, and daylight-saving ambiguity. Do not treat a calendar day as a fixed 24-hour offset.
+
+## Limits
+
+This first model covers English seconds, minutes, hours, and elapsed days, up to seven days. The authored quantity helper supports digits, decimals, one through twenty, `a`, `an`, and `half`. Calendar output supports today and tomorrow, 24-hour clocks, and AM/PM.
+
+Weekdays, named months, absolute dates, timezones, recurrence, and fuzzy phrases such as “sometime soon” are outside the example. Unsupported interpretations should return uncertainty. Confidence is uncalibrated, and familiar words in a novel arrangement can still produce a wrong answer.
+
+## Evaluation
+
+The generator writes only training data. Validation, test, and negative challenge fixtures are maintained separately. The test split checks held-out quantities and clock values; all twelve original test inputs have token sequences present in training after numeric normalization. They are regression checks, not evidence of compositional generalization. Training reports include accuracy, loss and export validation. Browser speed is measured by the demo on your device.
+
+The frozen `evals/generalization.json` suite tests new contexts, unit combinations, and rejection cases. Run `bun run eval:examples` after training for accuracy, coverage, and rejection measurements. See [the example audit](../example-evaluation.md).

package/docs/getting-started.md

@@ -1,88 +1,82 @@
 # Getting started
 
-Matchbox is a TypeScript framework for building small models that run in the browser. Start with the money example inside your existing application, train it locally, and import the result.
+Train the included money parser, try it locally, and call it from your app. You need an existing JavaScript project with a `package.json`, Bun 1.4.2+, and Node.js 24+ for native Burn training. No API key is required.
 
-The CLI requires Bun 1.4.2 or newer and Node 24 or newer for the local training toolchain.
+## 1. Add a task
 
-## Add Matchbox to your app
-
-Once the first release is published, run this inside your existing React or Next.js app:
+Run from your application's root:
 
 ```sh
 bunx matchbox-ai init money --template money
-bunx matchbox-ai dev money
 ```
 
-The CLI installs `@matchbox-ai/core` as an application dependency and `@matchbox-ai/train` and `matchbox-ai` as development dependencies, using your package manager. It preserves existing scripts and framework configuration. Run your app's dev server separately. Use `--skip-install` to defer installation.
-
-When developing Matchbox itself, run `bun install` and `bun run build:packages` in this repository. Run `bun run matchbox init money --template money --directory /path/to/app --skip-install` to inspect a scaffold before the packages are published. It writes registry versions; it does not vendor a copy of the checkout.
+You can also launch the CLI with `npx matchbox-ai` or `pnpm dlx matchbox-ai`. Bun is still required by the training toolchain. Initialization installs dependencies using your app's package manager and adds these files:
 
-## Train in the workbench
+```text
+matchbox/money/
+  parser.ts                # Valid input and output.
+  pipeline.ts              # Explicit learning strategy.
+  recipe.ts            # Training token labels.
+  decode/decode.ts            # Convert recognized spans to output.
+  data/train.jsonl
+  data/train-spans.json
+  evals/validation.jsonl    # Gate model export.
+  evals/test.jsonl          # Measure the selected model.
+```
 
-Choose **Train model**. Once training finishes, try `$15` or `around twenty six grand in euros`. Predictions run in your browser. Choose **Evaluate** to check independent test examples, or **Measure browser speed** to time the current input.
+Your framework config and dev server stay as they are.
 
-The same operations are available in the terminal:
+## 2. Train and try it
 
 ```sh
 bunx matchbox-ai train money
+bunx matchbox-ai parse money '$15' --json
 bunx matchbox-ai eval money
 ```
 
-## Understand the authored task
+Training writes `.matchbox/money/model.ts`, `model.matchbox`, and `report.json`. On a successful parse, `result.value` contains:
 
-```text
-matchbox/money/
-  parser.ts
-  pipeline.ts
-  lib/recipe.ts
-  lib/decode.ts
-  data/train.jsonl
-  data/train-spans.json
-  evals/validation.jsonl
-  evals/test.jsonl
-.matchbox/money/
-  model.matchbox
-  model.ts
-  report.json
+```json
+{ "amount": 15, "currency": "USD", "approximate": false }
 ```
 
-The parser defines valid output. The pipeline explicitly selects a token model. The recipe supplies supervised token labels; the decoder normalizes recognized spans. New training examples may need new token annotations. Read the generated task README before extending it.
+The full result also includes `status` and an uncalibrated `confidence` score.
 
-Use `matchbox-ai init my-task --template blank` to author another task. The blank starter explicitly uses word features and finite field classification; replace that pipeline if it does not suit your task. Schemas do not automatically select numeric encodings or transformations.
+For a browser workbench, run `bunx matchbox-ai dev money`. It opens at `http://localhost:4190`. Try inputs, inspect recognized tokens, and measure inference on your device. Training runs locally; predictions run in the browser.
 
-## Import into React or Next.js
+## 3. Use it in your app
 
-The generated TypeScript wrapper needs no bundler plugin. This example assumes a component at your application root; adjust the relative path for your component's location. In Next.js, use a client component:
+The generated TypeScript module works without a bundler plugin. From a file in your application root:
 
-```tsx
-"use client";
+```ts
+import money from "./.matchbox/money/model";
 
-import { useMatchbox } from "@matchbox-ai/core/react";
+const result = await money.parse("twenty dollars");
+if (result.status === "ok") {
+  console.log(result.value.amount); // Typed as number.
+} else {
+  console.log(result.reason); // Ask for clarification or use a fallback.
+}
+```
 
-const loadMoney = () => import("./.matchbox/money/model");
+Adjust the relative import for your file's location. In Next.js, import the model from a client component. See [React integration](react.md) for loading state and `useMatchbox`.
 
-export function MoneyButton() {
-  const { parse, status } = useMatchbox(loadMoney);
+## 4. Change the behavior
 
-  return (
-    <button
-      disabled={status !== "ready"}
-      onClick={async () => {
-        const result = await parse("twenty dollars");
-        if (result.status === "ok") console.log(result.value.amount);
-        else console.log(result.reason);
-      }}
-    >
-      Parse an amount
-    </button>
-  );
-}
+Add input/output rows to `data/train.jsonl`. In this token-based example, add corresponding token labels to `data/train-spans.json`. Run `train` again, then `eval`. Keep validation and test examples separate from training.
+
+The model learns which spans represent an amount, currency, or multiplier. `decode/decode.ts` does the arithmetic. Read [the money example](examples/money.md) before extending its supported number formats.
+
+To start your own task:
+
+```sh
+bunx matchbox-ai init intent --template blank
 ```
 
-Train before your application build. Generated artifacts remain ignored, so CI needs to train or restore a previously evaluated artifact. Import training helpers only in the authored pipeline and training scripts, never in client components. For `.matchbox` imports with Vite, the optional [Vite integration](react.md) remains available.
+The blank template uses a finite field classifier and empty datasets. Define your labels, add training and independent eval examples, then train. [Choose a pipeline](pipelines.md) before using it for numeric or compositional output.
 
-## Run the repository showcase
+## 5. Build and deploy
 
-From the Matchbox checkout, `bun run dev` trains the examples and starts the showcase. The home page demonstrates customer filtering; `/training` demonstrates money parsing and the parity training check.
+Generated models are ignored by Git. Train before your app build, for example `bunx matchbox-ai train money && npm run build`, or restore a previously evaluated artifact together with its matching schema and decoder.
 
-Continue with [CLI](cli.md), [Evaluation](evaluation.md), [Training pipelines](pipelines.md), or [Project structure](project-structure.md).
+Continue with [Training](training.md), [Evaluating](evaluation.md), [CLI commands](cli.md), [API reference](reference/README.md), or [project configuration](reference/configuration.md).

package/docs/native-packages.md

@@ -0,0 +1,31 @@
+# Native packages
+
+The training package bundles prebuilt Node-API binaries. Installing Matchbox does not download executables from GitHub, run an install script, or compile Rust. The loader selects the binary for the operating system, CPU architecture and Linux libc. Browser inference continues to use the separate WASM asset in the core package.
+
+## Platform coverage
+
+The build and installed-package checks run on these targets:
+
+| Platform    | Architectures | Verification environment  |
+| ----------- | ------------- | ------------------------- |
+| macOS       | x64 and ARM64 | macOS 15.                 |
+| Linux glibc | x64 and ARM64 | Ubuntu 22.04.             |
+| Linux musl  | x64 and ARM64 | Node 24 Alpine container. |
+| Windows     | x64           | Windows Server 2025.      |
+| Windows     | ARM64         | Windows 11 ARM.           |
+
+Node 24 runs the installed-package checks on every target. Bun checks additionally run on macOS, glibc Linux and Windows x64. This matrix does not claim Bun support on Windows ARM64 or Alpine. Older operating systems and other architectures are outside the verified support matrix.
+
+All binaries ship inside @matchbox-ai/train. This increases the development dependency's download size, but keeps installation independent of optional dependencies and install-script permissions. These binaries do not enter the browser bundle.
+
+## Building and verifying
+
+`scripts/native-targets.json` defines the supported targets and artifact names. The native workflow compiles each target, then exercises sequence and record training through its loader. Musl builds use a dynamic C runtime and run in Alpine rather than being tested against glibc.
+
+The packed-package workflow assembles all binaries, packs the three npm packages, and installs those archives in isolated directories with `--ignore-scripts`. It trains through @matchbox-ai/train and runs the generated model, including WASM inference. These directories contain no Rust sources or build output. Every target must pass before the release pack job runs.
+
+The release workflow also checks that all eight binaries are present before Changesets packs. Model binaries and generated WASM remain ignored by Git. Contributors still need Rust to build from source; package consumers do not.
+
+Run `bun run build:rust` for the local platform. A missing platform binary produces a loader error identifying the operating system, architecture and libc. There is no automatic compilation fallback.
+
+The packaging convention follows [prebuild's bundled distribution](https://github.com/prebuild/prebuildify) and uses [node-gyp-build](https://github.com/prebuild/node-gyp-build) only as a loader.

package/docs/neural-training.md

@@ -1,3 +1,3 @@
 # Neural training
 
-See [pipelines](pipelines.md) for the supported native TensorFlow trainers and [evaluation](evaluation.md) for dataset separation and export checks. Examples use explicit pipeline.ts declarations; recipes and decoders are application-owned.
+See [pipelines](pipelines.md) for the supported native Burn trainers and [evaluation](evaluation.md) for dataset separation and export checks. Examples use explicit pipeline.ts declarations; recipes and decoders are application-owned.

package/docs/parser-api.md

@@ -13,9 +13,6 @@ const task = defineParser({
     minimum: z.number().nonnegative(),
     owner: z.string().nullable().optional(),
   }),
-  fields: {
-    minimum: { type: "money", aliases: ["ARR"] },
-  },
 });
 
 type Output = InferOutput<typeof task>;
@@ -34,7 +31,21 @@ const metadata = task.toJSON();
 const serialized = JSON.stringify(task); // Uses the same metadata representation.
 ```
 
-Validation returns a discriminated `ValidationResult<T>`. Invalid task definitions throw `TypeError` at definition time with a schema path. Invalid example values return issues rather than being silently normalized. There is no `parse(text)` inference method yet.
+Validation returns a discriminated `ValidationResult<T>`. Invalid task definitions throw `TypeError` at definition time with a schema path. Invalid example values return issues rather than being silently normalized. The task definition has no inference method; call `parse` on the generated model instead.
+
+## Arguments and return value
+
+`defineParser(config: ParserConfig<Output>): ParserDefinition<Output>` is exported from `@matchbox-ai/core`.
+
+| Argument | Required | Contract                                              |
+| -------- | -------- | ----------------------------------------------------- |
+| `input`  | Yes.     | `z.ZodString`, with supported constraints.            |
+| `output` | Yes.     | A supported structured Zod schema.                    |
+| `fields` | No.      | `Record<string, FieldMetadata>`; inert metadata only. |
+
+`FieldMetadata` has optional `type: string`, `aliases: readonly string[]`, and `description: string`. Metadata never enables money parsing or other domain behavior.
+
+The result exposes `kind: "parser"`, `input`, `output`, `validateInput(unknown)`, `validateOutput(unknown)`, and `toJSON()`. Validation returns `{ success: true, data }` or `{ success: false, issues }`, where every `ValidationIssue` has `code`, `path`, and `message`. `InferOutput<typeof task>` derives the validated output type. `ParserMetadata` is the detached JSON representation shown below.
 
 ## Supported schemas
 
@@ -70,17 +81,6 @@ The serialized definition contains:
 
 Metadata is captured at definition time. Every `toJSON()` call returns a detached copy. Treat the supplied Zod schemas as immutable after definition; use Zod's schema-building methods to create a new task when its contract changes.
 
-The format version identifies Matchbox's metadata representation, not a model or dataset version. A future loader must reject unsupported versions. This ticket provides serialization only; it does not reconstruct executable validators from JSON.
+The format version identifies Matchbox's metadata representation, not a model or dataset version. Artifact loading rejects unsupported format versions. The generated wrapper imports the authored task to validate model output.
 
 Zod's global metadata registry is intentionally excluded. `.meta()` must not override structural keywords or inject values that are not JSON. Descriptions, aliases, and semantic types belong in the optional `fields` object for now. Those hints are inert, copied JSON. Matchbox does not infer a field-to-output mapping or normalize money/countries from their names.
-
-## Dependency boundary
-
-Zod 4.6.3 is an explicit, pinned core dependency and is external to the Rolldown bundle. The API uses a small isolated portion of Zod Core's schema/check definitions to reject unsupported behavior before JSON conversion. Zod upgrades must run the contract suite, including negative cases.
-
-Validation uses Zod's non-JIT path. The browser smoke check exercises definition, validation, and metadata serialization through the built package. This is the task-authoring API; its dependency size is not a claim about the future generated inference runtime.
-
-## References
-
-- [Zod JSON Schema conversion](https://zod.dev/json-schema) documents representable schemas and conversion controls.
-- [Zod Core](https://zod.dev/packages/core) documents schema and check introspection.

package/docs/pipelines.md

@@ -1,11 +1,10 @@
 # Explicit pipelines
 
-The parser defines valid input and output. The pipeline defines how the model learns. Matchbox requires pipeline.ts for conventional tasks; the scaffold writes it visibly.
+The parser defines valid input and output. The pipeline defines how the model learns. Matchbox requires `pipeline.ts` or `pipeline/pipeline.ts` for conventional tasks; the scaffold writes it visibly.
 
 ```ts
-import { definePipeline, wordTokens, fieldClassifier } from "@matchbox-ai/train";
+import { definePipeline, fieldClassifier } from "@matchbox-ai/train";
 export default definePipeline({
-  input: wordTokens(),
   prediction: fieldClassifier(),
   acceptance: { minAccuracy: 0.95, maxBytes: 64000 },
 });
@@ -18,13 +17,13 @@ An explicit token pipeline uses application-owned supervision and decoding:
 ```ts
 import { definePipeline, tokenClassifier } from "@matchbox-ai/train";
 export default definePipeline({
-  prediction: tokenClassifier({ recipe: "./lib/recipe.ts", decode: "./lib/decode.ts" }),
+  prediction: tokenClassifier(),
   acceptance: { minAccuracy: 0.85, maxBytes: 24000 },
 });
 ```
 
-Paths resolve relative to the task. The recipe supplies tokenizer, labels, readout, and annotation alignment. The decoder receives labeled spans and returns a candidate output or null. Matchbox verifies training annotations decode to the supplied training outputs. Recipe code runs during training; decoder code ships with the browser artifact and must remain browser-safe.
+Matchbox discovers `recipe.ts` or `recipe/recipe.ts`, and `decode.ts` or `decode/decode.ts`. Explicit path overrides resolve relative to the task directory. The recipe supplies tokenizer, labels, readout, and annotation alignment. The decoder receives labeled spans and returns a candidate output or null. Matchbox verifies training annotations decode to the supplied training outputs. Recipe code runs during training; decoder code ships with the browser artifact and must remain browser-safe.
 
 These are two existing, evaluated presets. Decimal codecs, arbitrary graphs, automatic architecture search, and hidden domain normalizers are not implemented. New primitives should demonstrate their limitations and held-out behavior before becoming defaults.
 
-Programmatic training uses await train("money", { onProgress }) from @matchbox-ai/train and follows the same validation and packaging path as the CLI. Importing the authoring helpers does not initialize native TensorFlow; training loads it when invoked.
+Programmatic training uses await train("money", { onProgress }) from @matchbox-ai/train and follows the same validation and packaging path as the CLI. Importing the authoring helpers does not initialize native Burn; training loads it when invoked.

package/docs/primitives/README.md

@@ -2,18 +2,19 @@
 
 Import these from @matchbox-ai/train:
 
-| Primitive       | Contract                                                                       |
-| --------------- | ------------------------------------------------------------------------------ |
-| definePipeline  | Validates an explicit pipeline declaration and acceptance thresholds.          |
-| wordTokens      | Selects literal word features for field classification.                        |
-| fieldClassifier | Selects independent categorical output heads over training values.             |
-| tokenClassifier | Selects the sequence trainer with explicit recipe and decoder module paths.    |
-| tokenize        | Shares the portable tokenizer with annotation generators.                      |
-| train           | Runs project discovery, native training, validation, and packaging.            |
-| evaluate        | Evaluates a parser against examples and a supplied output-validation function. |
-| SequenceRecipe  | Types application-owned sequence supervision.                                  |
-| OutputDecoder   | Types a browser-safe sequence-to-output decoder.                               |
+| Primitive       | Contract                                                                                |
+| --------------- | --------------------------------------------------------------------------------------- |
+| definePipeline  | Validates an explicit pipeline declaration and acceptance thresholds.                   |
+| fieldClassifier | Selects independent categorical output heads over training values.                      |
+| tokenClassifier | Selects the sequence trainer with conventionally discovered recipe and decoder modules. |
+| tokenize        | Shares the portable tokenizer with annotation generators.                               |
+| train           | Runs project discovery, native training, validation, and packaging.                     |
+| evaluate        | Evaluates a parser against examples and a supplied output-validation function.          |
+| SequenceRecipe  | Types application-owned sequence supervision.                                           |
+| OutputDecoder   | Types a browser-safe sequence-to-output decoder.                                        |
 
 The field classifier derives its field names and value domains from training outputs. The output schema still independently validates predictions. The sequence recipe owns tokenization, so it cannot also specify a separate input encoder.
 
 Tensor shapes, exported weight readers, and architecture-specific predictors belong to @matchbox-ai/core/internal. That subpath supports coordinated framework packages and research tests; application code should use the documented task, pipeline, and runtime interfaces.
+
+See [token supervision](../reference/supervision.md) for explicit rejection examples and optional training token masking.

package/docs/project-structure.md

@@ -13,8 +13,10 @@ my-app/
       data/train.jsonl
       evals/validation.jsonl
       evals/test.jsonl
-      evals/baseline.ts        # Optional.
-      lib/                    # Optional application-owned helpers.
+      recipe.ts               # Token-classifier supervision.
+      decode/
+        decode.ts             # Browser-side output construction.
+        number-words.ts       # Decoder helper.
   .matchbox/money/
     model.matchbox
     model.ts
@@ -22,7 +24,7 @@ my-app/
     report.json
 ```
 
-The CLI discovers these paths. Multiple tasks require an explicit name; it never silently chooses the first one. Commands can also target a project directory, task directory, or explicit matchbox.config.ts. A task-local config may override paths; ordinary examples need only parser.ts and pipeline.ts.
+The CLI discovers these paths. Multiple tasks require an explicit name; it never silently chooses the first one. Commands can also target a project directory, task directory, or explicit matchbox.config.ts. A task-local config may override paths; field-classifier tasks need parser and pipeline modules; token classifiers also need recipe and decode modules.
 
 Generated artifacts are ignored. Validation gates packaging; test data does not select the model. Data generators live in the project-level scripts/ directory and write only matchbox/<task>/data/, preserving evals/ independently.
 
@@ -39,3 +41,15 @@ export default {
 ```
 
 The config overrides paths and acceptance settings. Keep the encoder, supervision, prediction strategy, and decoder explicit in `pipeline.ts`. The current loader supports TypeScript configuration; there is no separate JSON config format.
+
+## Named entry points
+
+Parser, pipeline, recipe, and decode modules each use the same convention: `X.ts` for a small module, or `X/X.ts` alongside supporting files. For example, `decode.ts` can become `decode/decode.ts` without changing `tokenClassifier()` in the pipeline. Matchbox rejects both forms existing at once. No `index.ts` is required or discovered.
+
+`tokenClassifier()` discovers the recipe and decoder by name. The recipe still explicitly defines tokenization, labels, readout, and supervision. The decoder still owns application-specific output construction. File discovery makes no learning or normalization choices.
+
+Keep helpers with the module that owns them. Shared domain code can have its own named folder: the filter example's `countries/` is used by the schema, training recipe, browser decoder. It is ordinary application code, not another framework-discovered entry point.
+
+Import named authored modules directly, such as `./decode/decode`. Task directories do not need a barrel exporting training and runtime internals together. Applications consume the generated `.matchbox/<task>/model.ts` wrapper.
+
+Generated wrappers import the concrete parser and decoder entry files. Recipe and training dependencies remain outside those browser imports.

package/docs/react.md

@@ -1,16 +1,58 @@
-# @matchbox-ai/core/react
+# React integration
 
-Load typed Matchbox parsers in React with `useMatchbox`.
+`useMatchbox` loads a parser and exposes initialization state. Use a stable loader outside the component. The generated TypeScript module needs no bundler plugin.
 
 ```tsx
+"use client";
+
+import { useState } from "react";
 import { useMatchbox } from "@matchbox-ai/core/react";
 
-const loadFilters = () => import("./generated/filters.matchbox");
+const loadMoney = () => import("./.matchbox/money/model");
+
+export function MoneyInput() {
+  const { parse, status, error } = useMatchbox(loadMoney);
+  const [message, setMessage] = useState("");
 
-function Search() {
-  const parser = useMatchbox(loadFilters);
-  return <input onChange={async (event) => console.log(await parser.parse(event.target.value))} />;
+  return (
+    <div>
+      <button
+        disabled={status !== "ready"}
+        onClick={async () => {
+          try {
+            const result = await parse("twenty dollars");
+            setMessage(
+              result.status === "ok"
+                ? `${result.value.amount} ${result.value.currency}`
+                : result.reason,
+            );
+          } catch (cause) {
+            setMessage(String(cause));
+          }
+        }}
+      >
+        Parse twenty dollars
+      </button>
+      <output aria-live="polite">{error ?? message}</output>
+    </div>
+  );
 }
 ```
 
-The hook exposes loading, ready, and error state. Keep the loader outside the component. React is a peer dependency of this package; core and training do not depend on React. See the [end-to-end guide](end-to-end.md).
+The path above assumes a component at the app root. Adjust it for your file's location. In Next.js, keep inference in a client component as shown. For direct artifact imports in Vite, see [the Vite plugin](reference/vite.md).
+
+## useMatchbox contract
+
+`useMatchbox<Output>(loader: () => Promise<{ default: MatchboxParser<Output> }>)` is exported from `@matchbox-ai/core/react`.
+
+| Return field | Type                                               | Behavior                                        |
+| ------------ | -------------------------------------------------- | ----------------------------------------------- |
+| `status`     | `"loading"`, `"ready"`, or `"error"`.              | Reflects the current loader and initialization. |
+| `error`      | `string` or `null`.                                | Loading error message.                          |
+| `parse`      | `(input: string) => Promise<ParseResult<Output>>`. | Runs inference; call failures reject.           |
+
+The hook calls optional `load` on mount. It does not store prediction results, debounce requests, catch parse errors, or dispose shared module instances. Applications own those behaviors. React 19 is the current peer requirement.
+
+## Continuous input
+
+For parsing while typing, keep the latest input in state and start a parse in an effect. Ignore results from an effect that has already cleaned up so an older prediction cannot overwrite a newer one. The hook's `status` describes model initialization, not an individual prediction.

package/docs/reference/README.md

@@ -0,0 +1,25 @@
+# API reference
+
+The examples use these public exports. Each page lists imports, arguments, return values, and failure behavior.
+
+| Package                      | Export                                                             | Reference                                           |
+| ---------------------------- | ------------------------------------------------------------------ | --------------------------------------------------- |
+| `@matchbox-ai/core`          | `defineParser`, parser and validation types                        | [Task definition](../parser-api.md)                 |
+| `@matchbox-ai/core`          | `parseDatasets`, dataset types                                     | [Datasets](../dataset-format.md)                    |
+| `@matchbox-ai/core/runtime`  | `createParser`, `MatchboxParser`, `ParseResult`                    | [Runtime](runtime.md)                               |
+| `@matchbox-ai/core/runtime`  | `Token`, `TaggedToken`, `SequenceDecoder`                          | [Token supervision](supervision.md)                 |
+| `@matchbox-ai/core/runtime`  | `compileClauses`, `Predicate`, `FilterExpression`                  | [Filter compilation](runtime.md#compileclauses)     |
+| `@matchbox-ai/core/react`    | `useMatchbox`                                                      | [React](../react.md)                                |
+| `@matchbox-ai/core/vite`     | `matchbox`                                                         | [Vite](vite.md)                                     |
+| `@matchbox-ai/train`         | `definePipeline`, `Pipeline`, `fieldClassifier`, `tokenClassifier` | [Pipeline API](pipeline.md)                         |
+| `@matchbox-ai/train`         | `SequenceRecipe`, `OutputDecoder`, `tokenize`                      | [Token supervision](supervision.md)                 |
+| `@matchbox-ai/train`         | `train`, `TrainingConfig`                                          | [Training API](training.md)                         |
+| `@matchbox-ai/train`         | `evaluate`                                                         | [Evaluation API](evaluation.md)                     |
+| `@matchbox-ai/train/project` | `discover`, `listTasks`, `loadConfig`, `loadArtifact`              | [Project tooling](configuration.md#project-tooling) |
+| `matchbox-ai`                | Eight CLI commands                                                 | [CLI reference](../cli.md)                          |
+
+`@matchbox-ai/core/internal` is for coordinated framework packages and research tests. Application code should use the entry points above.
+
+`version` is exported from `@matchbox-ai/core` and contains the installed core package version.
+
+Inputs are currently strings. Support for structured features requires a separate input and encoder contract.

package/docs/reference/configuration.md

@@ -0,0 +1,43 @@
+# Project configuration
+
+Conventional tasks need parser and pipeline entry points, each authored as `X.ts` or `X/X.ts`. Add `matchbox.config.ts` only to override paths or acceptance settings. The default export must satisfy `TrainingConfig` from `@matchbox-ai/train`.
+
+```ts
+import type { TrainingConfig } from "@matchbox-ai/train";
+export default {
+  train: "../../datasets/training.jsonl",
+  validation: "../../datasets/validation.jsonl",
+  eval: "../../datasets/test.jsonl",
+} satisfies TrainingConfig;
+```
+
+Paths resolve relative to the config's directory, normally `matchbox/<task>/`. Task config overrides pipeline acceptance values, which override defaults.
+
+| Field         | Type                                   | Default                                                                                      |
+| ------------- | -------------------------------------- | -------------------------------------------------------------------------------------------- |
+| `task`        | String path.                           | `parser.ts` or `parser/parser.ts`.                                                           |
+| `train`       | String path.                           | `./data/train.jsonl`.                                                                        |
+| `validation`  | String path.                           | `./evals/validation.jsonl`.                                                                  |
+| `eval`        | String path.                           | `./evals/test.jsonl`.                                                                        |
+| `output`      | Path ending in `.matchbox`.            | Project-level `.matchbox/<task>/model.matchbox`.                                             |
+| `minAccuracy` | Number from 0 to 1.                    | `0.95`, unless set in the pipeline.                                                          |
+| `maxBytes`    | Positive number.                       | `64000`, unless set in the pipeline.                                                         |
+| `challenges`  | Optional JSON path.                    | `./evals/challenges.json` if present.                                                        |
+| `sequence`    | `{ recipe: string, decoder: string }`. | Derived from the token pipeline. This is a legacy configuration route; prefer `pipeline.ts`. |
+
+Unknown properties are rejected. JSON configuration files are not supported. Challenge JSON is an array of `{ input: string, output: null }` used by sequence reports.
+
+## Project tooling
+
+These APIs support CLI and build integrations. Import from `@matchbox-ai/train/project` in Node/Bun only.
+
+| Function       | Arguments                       | Return value                                                                                                                                           |
+| -------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `discover`     | `target?: string, cwd?: string` | Promise of an absolute task directory or config path. Walks ancestors from `cwd` (default `process.cwd()`). Multiple tasks require an explicit target. |
+| `listTasks`    | Application root directory.     | Promise of sorted task names immediately under `matchbox/`, recognizing both parser entry forms.                                                       |
+| `loadConfig`   | Task directory or config path.  | Promise of `{ config, root, pipelinePath }` with defaults and overrides resolved.                                                                      |
+| `loadArtifact` | Task directory or config path.  | Promise of `{ artifact, task, decode, parser, inspect, output, config, root }`. Requires a previously trained artifact.                                |
+
+`loadArtifact().inspect(input)` returns model diagnostics and the candidate, without representing a final validated answer. These details differ by strategy. Use `parser.parse(input)` for application behavior. Discovery, missing files, invalid config, and incompatible artifacts reject their promises.
+
+`loadConfig` resolves task, recipe, and decoder module paths to absolute filenames. An extensionless override such as `task: "./schema"` resolves `schema.ts` or `schema/schema.ts`. An explicit filename bypasses conventional lookup for that module. Both conventional forms existing produces an error.

package/docs/reference/evaluation.md

@@ -0,0 +1,21 @@
+# Evaluation API
+
+```ts
+import { evaluate } from "@matchbox-ai/train";
+const metrics = await evaluate(parser, examples, (value) => task.validateOutput(value).success);
+```
+
+Arguments are a `MatchboxParser<unknown>`, nonempty `readonly DatasetExample<unknown>[]`, and `(value: unknown) => boolean` for output validation. It returns a promise for the metrics below. It does not train, select thresholds, dispose the parser, or enforce an acceptance gate. Caller errors and parser rejections propagate. Pass a nonempty set; empty input currently produces non-finite ratio metrics.
+
+| Field                | Meaning                                                                          |
+| -------------------- | -------------------------------------------------------------------------------- |
+| `examples`           | Number evaluated.                                                                |
+| `exactAccuracy`      | Fraction whose complete output matches.                                          |
+| `invalidOutputRate`  | Fraction of evaluated inputs producing an accepted output that fails validation. |
+| `accepted`           | Count returning `status: "ok"`.                                                  |
+| `acceptedAccuracy`   | Exact accuracy among accepted results, or `null` if none.                        |
+| `abstentionRate`     | Fraction returning uncertainty.                                                  |
+| `correctAbstentions` | Count correctly abstaining on rows with `output: null`.                          |
+| `failures`           | Array of `{ input, expected, actual }`; uncertainty appears as `actual: null`.   |
+
+Object key order does not affect exact matching; array order does. See [evaluation](../evaluation.md) for separate training, validation, test, and challenge sets.

package/docs/reference/pipeline.md

@@ -0,0 +1,43 @@
+# Pipeline API
+
+```ts
+import { definePipeline, fieldClassifier, tokenClassifier } from "@matchbox-ai/train";
+```
+
+## definePipeline
+
+`definePipeline(pipeline: Pipeline): Pipeline` validates a declaration and returns it. Invalid declarations throw a Zod validation error. It does not train or load Burn.
+
+| Property                 | Type                                   | Required | Behavior                                       |
+| ------------------------ | -------------------------------------- | -------- | ---------------------------------------------- |
+| `prediction`             | Field or token classifier declaration. | Yes.     | Chooses one of the two implemented strategies. |
+| `acceptance.minAccuracy` | Number from 0 to 1.                    | No.      | Minimum validation exact accuracy for export.  |
+| `acceptance.maxBytes`    | Positive number.                       | No.      | Maximum serialized model size in bytes.        |
+
+Unknown properties are rejected. See [configuration](configuration.md) for resolved defaults and overrides.
+
+## fieldClassifier
+
+`fieldClassifier(): { kind: "field-classifier" }` declares independent categorical predictions over values observed in training outputs. It uses literal word features fitted from training data only. Word order is discarded; unfamiliar vocabulary can cause uncertainty.
+
+```ts
+export default definePipeline({
+  prediction: fieldClassifier(),
+  acceptance: { minAccuracy: 0.95 },
+});
+```
+
+Use it for small, finite output domains. A numeric field is still a finite class, so this strategy cannot produce an unseen number. It is not a general JSON generator.
+
+## tokenClassifier
+
+`tokenClassifier(options?)` returns a token classifier declaration. It discovers `recipe.ts` or `recipe/recipe.ts`, and `decode.ts` or `decode/decode.ts`. Both modules remain authored and required for training. Optional `recipe` and `decode` string paths override these locations, relative to the task directory even when the pipeline lives in `pipeline/pipeline.ts`. Explicit filenames are used exactly; extensionless paths follow the named file-or-folder convention.
+
+```ts
+export default definePipeline({
+  prediction: tokenClassifier(),
+  acceptance: { minAccuracy: 0.9 },
+});
+```
+
+The recipe owns tokenization and training supervision. The decoder ships to the browser. See [their contracts](supervision.md). Schemas do not select encodings, dictionaries, or normalization rules.