@pipelex/sdk 0.18.1 → 0.19.0

package/docs/build-routes.md

@@ -0,0 +1,101 @@
+# Build routes (`/v1/build/*`)
+
+The three build projections — `buildInputs`, `buildOutput`, `buildRunner` — take a **closure** of MTHDS files, select **one pipe** in it, and project something from that pipe. They are Pipelex API extensions, not MTHDS Protocol routes: the standard specifies no projection surface, so these are ours to shape.
+
+> **`prepareInputs` no longer reads the inputs template.** It resolves its signature from the input-form descriptor on the validate report (see [input-preparation.md](./input-preparation.md#signature-driven-asset-identification)), so nothing inside this SDK calls these routes. They are kept for external consumers, and their retirement across the workspace is tracked separately.
+
+## The shared envelope
+
+Every build request supplies the closure **either** as inline `files` **or** as a `method_ref` — never both:
+
+```ts
+interface MthdsFileItem {
+  content: string;
+  source?: string; // provenance label — threaded onto this file's diagnostics
+}
+
+interface CrateRequestBase {
+  files?: MthdsFileItem[];
+  method_ref?: string; // address form server-resolved (pipelex-api >= 0.21.0); registry form reserved → 501
+}
+
+interface BuildRequestBase extends CrateRequestBase {
+  pipe_ref?: string; // QUALIFIED `domain.pipe_code`; omit to default to the closure's `main_pipe`
+  method_id?: never; // the build routes have no by-id form — expand with getMethodClosure
+}
+```
+
+The selector half is `CrateRequestBase`, shared verbatim with the [crate routes](./crate-routes.md) (`resolve` / `codegen`) — the same split the server makes (`MthdsPipeRequest(MthdsFilesRequest)`). What the build routes add is the pipe selector. An **address-form** `method_ref` (`github.com/<owner>/<repo>[/<selector>][@<tag>]`) is resolved by the server through the same fetch path as a `method_ref` run — the package's real relative paths feed the per-file `source` labels; any non-address reference stays reserved and answers `501`.
+
+The build routes deliberately take **no `method_id`** — the hosted tooling selector covers `validate` / `resolve` / `codegen` only, and these routes are frozen, being replaced by the codegen surface. The type pins the field to `never`, and `buildInputs` additionally rejects a stray runtime `method_id` with an error naming the migration (its old client-side by-id expansion was deleted). To build from a stored method, expand it first: `buildInputs({ files: await client.getMethodClosure(methodId) })`.
+
+`source` is why the envelope exists. A closure is a set of files, and a diagnostic that cannot say _which_ file it came from is much harder to act on. Pass a filename (or any label) per file and, **when the engine can attribute the diagnostic to a file**, it comes back as `source` on the corresponding `validation_errors[]` item.
+
+Treat that attribution as best-effort, not a guarantee — which is why `ValidationErrorItem.source` is typed optional and why the sample below reads `err.source ?? "?"`. Some arms don't populate it: graph-level `dry_run` and `pipe_factory` items have no single owning file, and a `main_pipe` naming a nonexistent pipe currently reports its provenance in the message prose while leaving the structured field unset. The syntax-error arm, the common case, does thread it.
+
+`pipe_ref` is **qualified** — `smoke.echo`, not `echo`. Omitting it defaults to the closure's declared `main_pipe`, which fails with a `422` when the closure declares **none**, or declares **several** across its domains. When you already know the pipe, name it: it is one field, and it removes the whole default-resolution failure mode.
+
+> `MthdsFileItem` is not the same type as `MthdsFile` (the one `validateFiles()` takes). `/v1/validate` spells the provenance label `uri` and carries it in a parallel `mthds_sources` array; this envelope spells it `source` and carries it inline. The two collapse into one only if `/v1/validate` ever migrates onto `files[]` — a protocol-level change owned by the MTHDS standard, not ours to make here.
+
+## The response is a verdict, not a payload
+
+Each route returns a **discriminated 200**, following `validate`'s discipline: an unresolvable closure is the _successful product_ of the call (the request was well-formed; the library was not), so it rides a 200 with `is_valid: false`, never a 4xx.
+
+**Branch on `is_valid` before reading the arm.** This is the easy thing to get wrong — a consumer that only catches throws will render a success over an unusable result, because nothing threw:
+
+```ts
+const result = await client.buildInputs({ files: [{ content: src, source: "method.mthds" }] });
+
+if (!result.is_valid) {
+  // 200, but the closure did not resolve. `validation_errors[]` says why, and each
+  // item's `source` names the file it came from.
+  for (const err of result.validation_errors) {
+    console.error(`${err.source ?? "?"}: ${err.message}`);
+  }
+  return;
+}
+
+// Narrowed to the valid arm.
+console.log(result.pipe_ref); // the RESOLVED qualified ref, e.g. "smoke.echo"
+console.log(result.inputs); // the template
+```
+
+The valid arm always echoes two refs: `pipe_ref` is what the server **resolved** (always qualified), and `requested_pipe_ref` is what you **submitted** — absent entirely when you omitted it and the server defaulted to `main_pipe`.
+
+## What throws
+
+Only a **no-verdict** condition, and it throws the typed `ApiResponseError` — branch on its `status`, never on its message:
+
+| Status        | Cause                                                                                                                                                                                                                                                 |
+| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `422`         | The pipe selector does not resolve: an unknown `pipe_ref`, or an omitted one on a closure with no `main_pipe` — or several. Also an output concept that cannot be rendered (`buildOutput`), and a requested pipe the dry-run skipped (`buildRunner`). |
+| `501`         | Registry-form `method_ref` — reserved, not implemented (the address form resolves).                                                                                                                                                                   |
+| `401` / `403` | Auth.                                                                                                                                                                                                                                                 |
+| `5xx`         | Server fault.                                                                                                                                                                                                                                         |
+
+Note the split: a bad **closure** is a 200 verdict; a bad **request** throws. "Your `pipe_ref` names nothing in this closure" says nothing is wrong with the closure — you asked for a pipe that isn't there — so it is a request error.
+
+## The payload field follows `format`
+
+Both `buildInputs` and `buildOutput` carry their payload in **one of two fields**, chosen by the requested `format`. The field the format did not select is **absent** from the body, not `null`.
+
+| Route         | `format`                     | Field           | Type                      |
+| ------------- | ---------------------------- | --------------- | ------------------------- |
+| `buildInputs` | `json` _(default)_           | `inputs`        | `Record<string, unknown>` |
+| `buildInputs` | `toml`                       | `inputs_toml`   | `string`                  |
+| `buildOutput` | `schema` _(default)_, `json` | `output`        | `Record<string, unknown>` |
+| `buildOutput` | `python`                     | `output_python` | `string`                  |
+
+The split is deliberate, not an accident of serialization. TOML cannot ride as a parsed object without losing exactly what makes it worth asking for — its concept comments and key order — and Python source is text by nature. Collapsing either into the JSON case would have meant stringifying the object case too, and the object shape is what consumers actually want.
+
+`buildInputs` also takes `explicit` (default `false`), which emits the ceremonial `{concept, content}` envelope per input instead of the light shape.
+
+## `buildRunner` is the odd one out
+
+It is the only build route that still **dry-runs** the closure, so:
+
+- It keeps `allow_signatures` (accept unresolved pipe signatures as pending rather than invalid). The other two dropped it — the flag only ever parameterized the sweep, and they no longer sweep.
+- Its valid arm carries `python_code` plus a stamped `structures` projection (`directory`, `artifacts[]`, `lock`, `lock_filename`) — write the artifacts and the lock under `directory`, relative to the runner script, and the script runs against them.
+- **Omitting `pipe_ref` sweeps the WHOLE closure**, not just the pipe it defaults to. The default can only be resolved _after_ the sweep has run, so there is no way to scope it in advance. A broken _sibling_ pipe can therefore sink the request — which is the honest answer for a caller who did not say which pipe they meant. Pass `pipe_ref` to scope the sweep.
+
+`buildInputs` and `buildOutput` are **static**: they read the pipe's _declared_ IO off a resolved crate. A valid verdict from them says the closure is structurally sound — never that the pipe runs. Runnability is `validate`'s vocabulary, and `buildRunner`'s.

package/docs/crate-routes.md

@@ -0,0 +1,197 @@
+# Crate routes (`/v1/resolve`, `/v1/codegen`)
+
+Two routes project a **closure** of MTHDS files into the artifacts downstream tooling actually consumes: `resolve` emits the **normalized library crate**, and `codegen` projects that crate into **stamped typed artifacts** plus their lock. Like the [build routes](./build-routes.md), they are Pipelex API extensions rather than MTHDS Protocol operations — but note the ownership split: the _crate_ is standard-owned (the MTHDS Library Crate Format), while the HTTP surface serving it, and every type projection on top of it, are ours.
+
+> **Both crate routes are served on every hosted origin, and by any `pipelex-api` runner.** On the hosted plane a route is reachable only when the gateway's API-key allowlist and the platform's tooling proxy both list its path — each enumerates routes explicitly, and an unlisted path answers a gateway `403 {"message":"Forbidden"}`, refused before any service sees the request, so not even an RFC 7807 problem body. `resolve` and `codegen` are listed by both.
+>
+> Measured 2026-08-23 against `api.pipelex.com` (`pipelex-hosted@0.10.1`) with a real API key: an empty body to either route comes back as an RFC 7807 `422` naming the missing fields — a request-shape verdict only a route that reached the service can produce — and a real closure comes back as a `200` carrying the artifacts. `api-dev.pipelex.com` has served them since 2026-08-13. Use a key when re-measuring: unauthenticated, every path answers `401` whether or not it is allowlisted, so a keyless probe cannot tell the two states apart.
+>
+> **`lint` / `format` are the exception and answer `403` on both origins.** That blocks nothing, because linting and formatting `.mthds` are toolchain capabilities rather than hosted ones: `plxt` carries both, and the post-edit hook this repo builds (`npm run build:hook`, vendored into `pipelex-plugins`) runs them offline through `@pipelex/tools-wasm` with no credentials — from the published package, `client.lint` / `client.format` are the documented fallback, against a runner. Exposing them on the hosted origins is a known, non-critical item on the platform's list, tracked in [`wip/hosted-exposure-crate-and-tools-routes.md`](https://github.com/Pipelex/pipelex-sdk-js/blob/main/wip/hosted-exposure-crate-and-tools-routes.md).
+
+## The shared envelope
+
+Both take the same closure selector — inline `files`, a `method_ref`, or a hosted `method_id`, exactly one (the strict tooling XOR):
+
+```ts
+interface CrateRequestBase {
+  files?: MthdsFileItem[]; // [{ content, source? }] — `source` is the provenance label
+  method_ref?: string; // address form server-resolved (pipelex-api >= 0.21.0); registry form reserved → 501
+}
+
+interface PipelexHostedToolingExtensions {
+  method_id?: string; // hosted-only — the platform resolves the stored method server-side
+}
+```
+
+This is the same `MthdsFileItem` and the same `source` semantics the build routes use, so the [notes there](./build-routes.md#the-shared-envelope) apply verbatim: pass a filename per file and, when the engine can attribute a diagnostic to one, it comes back as `source` on the corresponding `validation_errors[]` item.
+
+An **address-form** `method_ref` (`github.com/<owner>/<repo>[/<selector>][@<tag>]`) is resolved by the server through the same fetch path as a `method_ref` run: the repository fetched at the tag, the package located by manifest identity, the package's real relative paths feeding the per-file `source` labels. Any non-address reference stays reserved and answers `501`.
+
+`method_id` is the hosted platform's selector, and it is a **pass-through**: nothing is expanded client-side — the platform resolves the id against the org's catalog and injects the stored source before the runner sees the request. It is meaningless against a bare runner (no catalog), and on `api.pipelex.com` its availability follows the platform deploy that adds the tooling-route transform. An unknown or foreign-org id is a `404` (indistinguishable by design); a stored method with no MTHDS source is a `422`.
+
+Supplying **no** selector or **more than one** is a request-shape `422` — the tooling routes are stateless, so there is no linkage exception; a second selector could only be ignored, which is the worst contract of the three. The SDK does not model the XOR in the type system — the union would force the overwhelmingly common `{ files }` call site to pick a branch for no gain, and the server's answer is a typed `ApiResponseError` either way.
+
+The old advice to expand a stored method client-side (`resolve({ files: await client.getMethodClosure(methodId) })`) remains valid — `getMethodClosure` stays public as the local expansion utility, and it is what a caller uses against a bare runner or on the routes with no by-id form (`/v1/build/*`, `prepareInputs`).
+
+## `resolve` — the normalized crate
+
+Resolution is a first-class language operation alongside validation: the closure is loaded and statically validated, then emitted with fully qualified refs, refinement flattened, natives materialized, and a fingerprint set.
+
+```ts
+const result = await client.resolve({ files: [{ content: src, source: "method.mthds" }] });
+
+if (!result.is_valid) {
+  for (const err of result.validation_errors) console.error(`${err.source ?? "?"}: ${err.message}`);
+  return;
+}
+
+console.log(result.crate.fingerprint); // rides INSIDE the crate, not beside it
+```
+
+Two things to internalize:
+
+- **`fingerprint` and `mthds_version` are crate members**, not siblings of `crate`.
+- **Compare `fingerprint` values; never hash the crate yourself.** The fingerprint is a property of the _logical_ crate, not of an encoding: the server hashes `{concepts, pipes, domains}` with provenance `source` stripped, excluding `source_map`, `mthds_version` and `fingerprint` itself. And these are not the bytes `pipelex resolve --format json` prints — the CLI pretty-prints, the route answers compact JSON. Same logical crate, different serialization. Hashing `JSON.stringify(result.crate)` and comparing it against `result.crate.fingerprint` will mismatch on every call, and will look like tampering.
+- **`crate` is typed as opaque transport** (`Record<string, unknown>`). Its schema belongs to the MTHDS standard; restating it here would be a second source of truth, free to drift from the one the server emits.
+
+`resolve` runs **no dry-run sweep**. A valid verdict says the library resolves, never that it runs — runnability is `validate`'s vocabulary.
+
+## `codegen` — stamped artifacts and the trust chain
+
+`codegen` resolves the same way, then projects the crate through **two explicit axes**:
+
+| Axis     | Values                                                  | Notes                                                          |
+| -------- | ------------------------------------------------------- | -------------------------------------------------------------- |
+| `kind`   | `types`                                                 | The crate's whole concept set. Per-pipe kinds are future work.  |
+| `target` | `ts-zod`, `python-pydantic`, `python-structures`         | `ts-zod` is the natural one for TypeScript consumers.           |
+
+```ts
+const result = await client.codegen({ files, kind: "types", target: "ts-zod" });
+if (!result.is_valid) return;
+
+for (const artifact of result.artifacts) await writeFile(artifact.path, artifact.content);
+await writeFile(result.lock_filename, result.lock); // "codegen.lock", beside the artifacts
+```
+
+**The trust chain is the reason the lock rides along.** Write every artifact at its `path` and the `lock` content as `lock_filename`, both **verbatim**, and the resulting tree is byte-identical to what a local `pipelex codegen types` run produces — same stamps, same lock — so the offline `pipelex codegen check` passes on it. Reformatting an artifact, or re-serializing the lock through your own TOML writer, breaks that chain. There is deliberately **no** server-side check route: the check is offline by design.
+
+`crate_fingerprint` and `engine_version` say what the artifacts were generated _from_ — the crate `resolve` would have returned for the same closure, and the pipelex engine that emitted them.
+
+The SDK stays transport-only: it hands you the artifacts and does not write files for you. Verifying a committed tree later is the other half of the chain, and that half needs no server at all — see [the offline check](#the-offline-check--runcodegencheck) below.
+
+### `pipe_ref` is rejected, not ignored
+
+`kind: "types"` is concept-set-wide, so passing `pipe_ref` alongside it is a request-shape **`422`**. Silently ignoring the selector would mislead a caller into believing the artifacts were narrowed to one pipe. The field exists on the request for the future per-pipe kinds.
+
+## The response is a verdict, not a payload
+
+Both routes return a **discriminated 200**, the same discipline as `validate` and the build routes: an unresolvable closure is the _successful product_ of the call (the request was well-formed; the library was not), so it rides a 200 with `is_valid: false` and the shared `CrateInvalidReport` — the very same invalid arm the build routes return, carrying the same structured `validation_errors[]`.
+
+**Branch on `is_valid` before reading the arm.** A consumer that only catches throws will render a success over an unusable result, because nothing threw.
+
+## What throws
+
+Only a **no-verdict** condition, as the typed `ApiResponseError` — branch on its `status`, never on its message:
+
+| Status        | Cause                                                                                                     |
+| ------------- | --------------------------------------------------------------------------------------------------------- |
+| `422`         | Request shape: no closure selector or more than one, an unknown `kind`/`target`, a `pipe_ref` on `kind: "types"`, a stored method with no MTHDS source. |
+| `404`         | Unknown or foreign-org `method_id` (indistinguishable by design).                                           |
+| `501`         | Registry-form `method_ref` — reserved, not implemented (the address form resolves).                         |
+| `401` / `403` | Auth.                                                                                                       |
+| `5xx`         | Server fault.                                                                                               |
+
+Note the split, same as the build routes: a bad **closure** is a 200 verdict; a bad **request** throws.
+
+(The offline check below is not a route and throws its own `CodegenLockError` instead — it never reaches the network.)
+
+## The offline check — `runCodegenCheck`
+
+A consumer that commits the generated tree needs a CI gate over it. That gate is `runCodegenCheck`, and it is **pure**: no filesystem, no network, no API key, no `PipelexApiClient`. You walk your own tree and hand in the text; the SDK owns the verdict.
+
+The reason it is a separate, engine-free step is the same reason there is no server-side check route. Regeneration is a **dev action** — it needs the engine, so it is `codegen` above. The check is a **CI action** — it needs only a hash function. Keeping them apart is what stops an upstream template improvement from reddening a consumer's CI on a tree nobody touched.
+
+```ts
+import { isStampableArtifactPath, runCodegenCheck } from "@pipelex/sdk";
+
+const report = await runCodegenCheck({
+  lockContent: await readFile("src/generated/echo/codegen.lock", "utf8"),
+  // Every file under the lock's directory, recursively, with paths relative to it.
+  files: await readTree("src/generated/echo", isStampableArtifactPath),
+});
+
+if (!report.isCurrent) {
+  for (const drift of report.drifts) {
+    console.error(`${drift.category}: ${drift.path} — ${drift.detail}`);
+  }
+  process.exit(1);
+}
+```
+
+A `codegen()` response feeds straight in with no mapping — `GeneratedArtifact` and `CodegenTreeFile` are structurally identical on purpose, so `runCodegenCheck({ lockContent: result.lock, files: result.artifacts })` type-checks and reports `isCurrent`. That is worth doing right after a regeneration, before writing anything to disk.
+
+The function is `async` because it hashes through **WebCrypto** (`crypto.subtle`) rather than `node:crypto`, so the check adds no Node builtin to the barrel's import graph. (That is not the same as the barrel being browser-bundleable today: `upload.ts` still names `node:fs/promises`, so a browser-targeting bundler must mark `node:*` external.) One caveat that never bites a CI script but should not surprise anyone: `crypto.subtle` is secure-context-only, so a browser page must be served over https (or localhost).
+
+### The algorithm
+
+Each locked artifact is located in the supplied tree and the hash of the body **below its stamp** is recomputed. Absent is a drift; a body that no longer hashes to what the stamp or the lock records is a drift. Then, in the other direction, a stamped file the lock does not track is a drift — the stale-artifact class a per-file stamp can never catch on its own, and the reason the lock rides along with the artifacts at all.
+
+The verdict is a structured report, never an exit code alone: `drifts` enumerates the drifting artifacts by category, and `isCurrent` is exactly `drifts.length === 0`. Drift order is deterministic — locked-artifact drifts first, then orphans, each sorted by path — so a consumer can print it, snapshot it, or diff two runs.
+
+The report also carries `crateFingerprint` and `engineVersion`, read off the lock header. The check itself never compares them against anything (that would need the engine), but surfacing them is what lets a caller ask the question the check deliberately does not: _is this committed tree even from the crate my method resolves to today?_ — a live `codegen()` response's `crate_fingerprint` is the value to compare against.
+
+### The drift taxonomy
+
+| Category      | What it means                                                                                         |
+| ------------- | ----------------------------------------------------------------------------------------------------- |
+| `missing`     | Listed in the lock, absent from the tree you handed in.                                               |
+| `modified`    | Present, stamp self-consistent, but the body no longer matches the **locked** hash — regenerate.      |
+| `hand-edited` | Present, but its stamp is gone, unparseable, or disagrees with the body below it — someone edited it. |
+| `orphan`      | A stamped file the lock does not track — yesterday's artifact, left behind. Remove or regenerate.     |
+
+Two properties of that table are load-bearing and are pinned by tests:
+
+- **At most one drift per locked path, and `hand-edited` outranks `modified`.** A hand edit trips both conditions; reporting it twice would print the same file under two contradictory categories.
+- **An orphan only has to _look_ stamped.** A stray whose stamp is corrupt below the begin-marker line still counts — using the stricter parse there would silently ignore exactly the stale file the lock exists to catch.
+
+The category values are the canonical strings `pipelex codegen check` uses, and so are the `detail` sentences, verbatim. A consumer switching between the CLI and this helper reads the same report.
+
+### Where it knowingly differs from the CLI
+
+Verdict parity is the design constraint, and the stamp-header text rules mirror Python's exactly — the line-boundary set, the strip set, the drive-prefix rule, and the comment-prefix gate over the header region are all matched deliberately, each pinned by a test. These differences remain:
+
+- **The projection line is shape-checked, not vocabulary-checked.** pipelex resolves `kind` and `target` against its own enums, which cannot lag its own emitter; an SDK copy can. So a stamp reading `projection: types / rust-serde` from a newer engine verifies here and would be `hand-edited` there. Tightening it would report every artifact of such a tree as a hand edit — the failure mode is worse than the gap, and it is pinned in both directions.
+- **The "not valid UTF-8" branch cannot arise.** `content` reaches the check already decoded, so the caller owns that verdict — see the decode obligation above.
+- **A TOML float is accepted as a lock version, where pipelex calls it malformed.** `lock_version = 1.0` is a float upstream and not an `int`, so the CLI refuses it; the TOML parser here decodes it to the same `1` as the integer, so this reader cannot tell them apart and reads version 1. Not deliberate so much as unavoidable, and unreachable with any emitter — the divergence is a no-verdict there against a verdict here, on a lock nothing writes.
+
+### The lock format version
+
+`codegen.lock` opens with `lock_version`, and this build reads version 1. The field exists because the reader is deliberately strict *within* a version — an unknown key is a malformed lock, not a field to ignore — and that strictness is unworkable *across* versions without it.
+
+Four rules, mirroring the CLI exactly:
+
+- **A lock with no `lock_version` key is version 1.** The field was introduced with version 1, so every lock written before it existed is already conformant and nothing on disk needs migrating.
+- **The version is read _before_ the key set is validated.** This is the load-bearing ordering. The other way round, a lock from a newer codegen is rejected over whichever new key it happens to carry — an opaque shape complaint about a key the writer was entitled to add, instead of a verdict naming the version and saying what to upgrade.
+- **A version this build does not know is refused**, with a message that names the version found and which side to upgrade. A version *greater* than the one read means the lock came from a newer codegen; anything else (`0`, negative, non-integer) is malformed rather than futuristic.
+- **Strictness within a known version stays.** An unexpected key in a version-1 lock is still a no-verdict.
+
+So the upgrade path is: publish an SDK that tolerates the new version *before* the pipelex release that starts writing it. A reader that has not learned a version yet fails loudly and actionably rather than silently guessing at a shape it was not written for.
+
+### The caller's obligations
+
+The pure input moves the tree-reading obligations onto you. Each one, unmet, produces a **wrong verdict** rather than an error — so none of them can be left implicit:
+
+- **Pass each file's text as read, without reformatting it.** The hash is over exact UTF-8 bytes, and the stamp parser requires the text to _start with_ the begin-marker line. Re-encoding, inserting a BOM, or running Prettier over a generated artifact all report `hand-edited`.
+- **Line endings are the one exception, and they are handled for you.** `\r\n` and lone `\r` are normalized to `\n` before anything is parsed or hashed, mirroring the universal-newline translation pipelex's own reader applies. A Windows checkout under `core.autocrlf=true` is therefore **not** a false hand-edit here, exactly as it is not one for the CLI. Committing generated artifacts with a `.gitattributes` entry (`src/generated/** -text`) is still worth doing for diff hygiene; it is not load-bearing for the verdict.
+- **Walk the whole tree, recursively, from the lock's directory**, and pass paths relative to it. An incomplete list yields a wrong verdict in either direction: an omitted _locked_ file is reported `missing` though it sits on disk, and an omitted _orphan_ is never seen at all — `isCurrent: true`, a false negative on precisely the drift class orphan detection exists for. Filter your walk with `isStampableArtifactPath` (or `STAMPABLE_ARTIFACT_SUFFIXES`) so it picks up exactly what the check considers; a file of any other type is skipped rather than rejected, which is what lets you park a sidecar such as `sources.json` beside the lock. Pruning vendor and VCS directories and skipping symlinks is walk policy and stays with you — moot for a per-method generated directory, which holds nothing else.
+- **Decode the bytes yourself.** `content` is already a `string`, so pipelex's "not valid UTF-8, therefore not generated output" branch cannot arise here. A file you could not decode is not generated output: report it yourself, or leave it out and accept the `missing` drift.
+
+### What it deliberately does not verify
+
+It never compares a stamp's `crate_fingerprint` against the lock's, and it never re-resolves the crate. Both need the engine, which is the whole point of the offline split — and both are questions the caller can ask itself with `crateFingerprint` in hand.
+
+### What the check throws
+
+`CodegenLockError`, and only for a **no-verdict** condition: a malformed lock, a lock format version this build cannot read, an unknown key in it, or a path — in the lock or in your `files` — that is not a safe canonical artifact path (absolute, drive-prefixed, backslashed, control-charactered, `..`-bearing, empty, duplicated, or, for a locked path, of a type codegen does not stamp). None of these is a drift: the check could not produce a verdict at all, which is a distinct outcome and typically a distinct exit code. It deliberately does not derive from `PipelineRequestError` — nothing was requested over the wire.
+
+A `"./types.ts"` spelling is worth calling out, because it is the one a hand-rolled walk produces by accident: left to resolve as-is it would report both a `missing` and an `orphan` for the same file, so it throws instead.

package/docs/input-preparation.md

@@ -0,0 +1,189 @@
+# Input preparation (`uploadFile` / `prepareInputs`)
+
+> **Status: implemented** (`src/upload.ts`, `src/prepare-inputs.ts`). This document records the contract, and is the public account of it. The raw `upload()` primitive described in [architecture.md](./architecture.md) is the wire call `uploadFile` and `prepareInputs` build on.
+>
+> **Current scope.** `prepareInputs` names the method the same three ways every other method-taking operation does — inline `files`, a `method_ref` address, or a stored `method_id` — exactly one per call, all three resolved server-side by the one `validate` call it composes. One piece remains deliberately deferred and additive (it does not change this contract): the opt-in ingest of `http(s)` URLs into storage — for now an `http(s)` URL at a file position always passes through unchanged.
+
+## Why this exists
+
+A hosted run cannot see the caller's filesystem or a browser's selected bytes. Turning caller-local assets into run-ready inputs is therefore the SDK's job, not the runner's — the SDK process is the only component that can read the local file or hold the browser bytes. Today that work is re-implemented by every consumer (read file → base64 → `POST /v1/upload` → rewrite the input to the returned URI). `prepareInputs` makes it one reusable, explicit operation.
+
+Preparation is **explicit and separate from running.** `execute` / `start` never silently upload local files. The payoff: file-access errors happen *before* a run exists, prepared inputs are inspectable and reusable across a model sweep or retries without re-uploading, and `start` keeps a deterministic JSON-input contract.
+
+## The two operations
+
+### `uploadFile` — single-asset convenience
+
+Uploads one asset and returns its upload record. It is the language-native convenience over the raw `upload()` wire call (base64 JSON body), assembling the record client-side.
+
+- **Accepted sources (every runtime — browser, Node, edge):** `Blob`, `File`, `ArrayBuffer`, `Uint8Array`.
+- **Path strings are Node-only.** A filesystem path string is read from disk, which only a process that owns a filesystem can do. In a non-Node runtime a path string fails instructively rather than being misread — the SDK does not silently treat it as text or as a URL.
+- A string that is an **HTTP(S) URL** or an existing **`pipelex-storage://` URI** is not a local asset and passes through in any runtime (see pass-through rules below).
+- Open file objects and streams are **deferred** — they can be added later without removing anything.
+
+> **Runtime acceptance vs. bundling.** These source types are accepted at *runtime* in every environment, but at the *packaging* level `@pipelex/sdk` is **Node-first**: it statically references `node:fs/promises` (to read path strings), so bundling the SDK for a browser or edge target currently requires marking `node:*` external. This is a deliberate stance — every consumer runs the SDK server-side in Node (Next.js Server Actions, the MCP server, the plugins hook) — revisited only if the SDK is ever imported into a browser client bundle or an edge route.
+
+The returned **upload record** guarantees, beyond the source identity:
+
+| Field | Guarantee |
+| --- | --- |
+| `uri` | The `pipelex-storage://` reference for the uploaded asset. |
+| `contentType` (MIME) | Known client-side at upload time. |
+| `size` (bytes) | Known client-side at upload time. |
+| `filename` | Already in the wire model. |
+| checksum | **Not present.** Within-preparation dedup relies on source identity, not hashing; cross-preparation dedup is a hosted storage-policy concern (Phase 5). |
+
+The MIME type and size are known client-side, so the record is assembled without extending the `/v1/upload` response.
+
+### `prepareInputs` — signature-driven input preparation
+
+```
+client.prepareInputs({ files,      pipe_ref?, inputs }) → PreparedInputs
+client.prepareInputs({ method_ref, pipe_ref?, inputs }) → PreparedInputs
+client.prepareInputs({ method_id,  pipe_ref?, inputs }) → PreparedInputs
+```
+
+Takes the **method** as exactly one of three selectors, the optional target **pipe**, and the caller's `inputs`; resolves the pipe's declared input signature; interprets the inputs top-down against it; uploads the file-bearing values; and returns `PreparedInputs`. Per input, the caller may submit **either** the compact value **or** the explicit `{ concept, content }` envelope — see "[Compact or explicit-envelope inputs](#compact-or-explicit-envelope-inputs)" below:
+
+- `inputs` — a **copy** of the caller's inputs with each asset reference replaced by the canonical content shape carrying `pipelex-storage://` in its `url` field (see "Rewritten-input shape" below). Copy-on-write: the caller's original object is never mutated.
+- `uploads` — one upload record per prepared asset (the `uploadFile` record shape), exposing `uri` so callers can log which source became which reference without reverse-engineering the rewritten object.
+
+The prepared `inputs` are passed to the existing run lifecycle unchanged.
+
+#### The three selectors
+
+Exactly one per call. The type pins the other two to `never`, so a second selector is a compile error; an untyped caller gets an `InputPreparationError` naming the three forms. **Empty is absent** — `files: []`, `method_ref: ""`, `method_id: "  "` — mirroring the run options' rule, so an empty selector may sit beside a real one without tripping the XOR.
+
+| Selector | What it is | Who resolves it |
+| --- | --- | --- |
+| `files` | the inline MTHDS closure (`{content, source?}` entries) | nobody — inline |
+| `method_ref` | a published method's address, `github.com/<owner>/<repo>[/<selector>][@<tag>]` | the runner, server-side (pipelex-api >= 0.21.0 fetches the repository at the tag) |
+| `method_id` | a stored method's catalog id (`mt_…`) | the hosted platform, which injects the stored source before the runner sees the request |
+
+Nothing is expanded client-side: `method_id` here is a **pass-through**, the same rule every other id-taking operation follows. `getMethodClosure` stays available for callers that want the files in hand, but it is no longer a step on the way to preparing inputs.
+
+#### Where the signature comes from
+
+One `POST /v1/validate` per call, whatever the selector, asking for the **input-form descriptor**:
+
+```
+validate(<selector or contents>, allowSignatures = true, …, views = ["input_form"])
+```
+
+`allowSignatures: true` is deliberate. Preparation needs a pipe's *declared* inputs, and a bundle mid-authoring with an unresolved signature somewhere else must not be refused inputs for a pipe whose inputs are declared — whether the bundle runs is the run's verdict, not preparation's. An `is_valid: false` verdict still means the closure does not load, which is a preparation failure.
+
+A `method_ref` makes the server clone a repository first; `validate` needs no special budget for it, because the route already defaults to the 20-minute execute ceiling.
+
+#### Pipe selection
+
+`validate` has no pipe selector — its report describes every pipe, keyed by qualified `pipe_ref` — so the helper picks one, in this order:
+
+1. **`pipe_ref` when given.** Qualified-only: `domain.pipe_code`. A bare code, or a ref the method does not declare, is an `InputPreparationError` listing the qualified refs — one step to fix. The helper never grows a searched `pipe_code`: search is a run-route affordance, and the descriptor is keyed by qualified refs. An `alias->domain.pipe_code` ref is refused for a different reason: the descriptor describes the method's *own* pipes, and an alias names one belonging to a dependency package, which the report never carries a descriptor for — the run route takes such a ref, preparation cannot.
+2. **The report's resolved default** (`default_pipe_ref`): the qualified ref of the pipe a selector-less run of *this same method* would execute, manifest-aware for a fetched package. It is read on the field's **presence**, never on its truthiness, and has three arms:
+   - a **non-empty string** is the default and is walked — and if the descriptor does not describe it, preparation refuses rather than falling through, since one report keyed by one pipe set naming a pipe it does not describe is the report contradicting itself;
+   - a **stated `null`** — or any stated value that is not a non-empty string — is the server saying it determined no entry pipe: no blueprint declares a `main_pipe`, or the package manifest names a pipe the closure does not declare or declares in several domains. A run naming no pipe is refused in exactly those cases, so preparation refuses too and asks for `pipe_ref`. Steps 3 and 4 are **not** consulted behind it;
+   - an **absent field** means the runner predates it, and only then do steps 3 and 4 stand.
+3. **The bundle's declared `main_pipe`** — behind an absent field only — read defensively from the opaque `bundle_blueprint` and qualified by its `domain`.
+4. **The single pipe** — behind an absent field only — when the method declares exactly one.
+5. Otherwise an `InputPreparationError` naming the candidates and asking for `pipe_ref`.
+
+> **The manifest-only `main_pipe` gap is what step 2 closes.** A published package may name its entry pipe in `METHODS.toml` alone — `github.com/Pipelex/methods/documents` and `.../image_generation` do — and the validate report carries no manifest; the runner resolves the manifest server-side and states the result in `default_pipe_ref`, so such a package prepares with no `pipe_ref`. A manifest naming a pipe the closure does not declare, or declares in several domains, is the stated `null` instead: the run would fail on that code, and preparation says so before anything is uploaded.
+
+## `getMethodClosure` — the explicit expansion utility
+
+```
+client.getMethodClosure(methodId) → MthdsFileItem[]
+```
+
+`getMethodClosure` is the public **local expansion utility** — a client-side semantic layer over `getMethod` (the platform has no route that returns a parsed closure). It fetches the method, parses its polymorphic `mthds` source with [`methodSourceToContents`](#methodsourcetocontents--the-canonical-source-parser), and labels each resulting file with the `method_id` as its `source` provenance.
+
+Reach for it whenever you want the files **in hand** — to edit, to diff, or to feed a route with no by-id form:
+
+```
+client.buildInputs({ files: await client.getMethodClosure("mt_…"), pipe_ref?, format?, explicit? })
+```
+
+`/v1/build/*` is the last such family, and it is being retired. Every other method-taking operation — `execute` / `start`, `validate` / `resolve` / `codegen`, and now `prepareInputs` — takes a `method_id` **natively**, as a server pass-through: the platform resolves it and injects the stored source before the runner sees the request (see [architecture.md → hosted run extensions](./architecture.md#hosted-run-extensions-method_id)). That is the uniform rule — *an id option is always a server pass-through, and any client-side expansion is the caller's own explicit call*. An untyped caller still passing `method_id` to `buildInputs` gets a teaching error naming this migration (`PipelineRequestError`); for typed callers the field is a compile error.
+
+- **Requires an API key.** The methods catalog is org-scoped to the key's org, so resolution only works with an authenticated Pipelex-product key.
+- **Unknown or foreign-org id → the `getMethod` `404`** (`ApiResponseError`, `code: "not_found"`), propagated unchanged. An id from another org is indistinguishable from a nonexistent one — both 404.
+- **A real, in-org method whose source parses to nothing → `EmptyMethodSourceError`** — the row exists but has no runnable MTHDS source yet. This is a distinct outcome from the 404 (see "[Error and capability behavior](#error-and-capability-behavior)"), and `getMethodClosure` is now its only raiser: `prepareInputs` passes the id to the server, which answers a sourceless method with a `422`.
+
+### `methodSourceToContents` — the canonical source parser
+
+```
+methodSourceToContents(mthds: string) → string[]
+```
+
+A stored method's `mthds` field is **polymorphic**: it is either the raw single-bundle source, or a JSON array of `{ name, content }` file entries (a multi-file closure). `methodSourceToContents` is the one canonical parser that turns either shape into a flat list of file contents — a verbatim port of the platform's `_method_source_to_contents`, so the SDK and the runtime read a stored source identically. It is exported from the barrel for consumers (e.g. `pipelex-mcp`) that need the parse without the fetch. Blank entries are dropped; a source that parses to no non-blank content yields `[]`, which is what `getMethodClosure` turns into `EmptyMethodSourceError`.
+
+## Compact or explicit-envelope inputs
+
+Each input may be submitted in **either** of two shapes, and preparation treats them equivalently:
+
+- **Compact** — the bare value: a source string / bytes / canonical `{ url }` content (e.g. `photo: "https://…/p.png"`).
+- **Explicit envelope** — the `{ concept, content }` shape that `buildInputs({ explicit: true })` returns per input (e.g. `photo: { concept: "native.Image", content: { url: "https://…/p.png" } }`). This is the default template the hosted console and MCP hand agents to fill, so an agent that fills the template can hand it straight back.
+
+When a value is an envelope (a plain object whose keys are **exactly** `concept` and `content` — matching the runtime's `_is_explicit` in `input_shaper.py`), preparation unwraps `content`, interprets it exactly as the compact value would be, and **re-wraps** the result — so the concept annotation rides through to the run. The runtime accepts the envelope as a run input (`input_shaper.py` `_is_explicit` / `_shape_explicit`), and file-reference resolution (`input_normalizer.py`) runs over the already-shaped content, so a re-wrapped `{ concept, content: { url: "pipelex-storage://…" } }` resolves identically to the compact `{ url }`. The envelope's `content` may itself be a scalar, a canonical file content, a list, or a structured object nesting file fields — the same top-down walk applies underneath.
+
+## Signature-driven asset identification
+
+The SDK **must not** guess that every string resembling a path is an asset — that would make ordinary text inputs environment-dependent and could upload unintended files. Interpretation comes from the method's **declared signature**, never from a value's shape alone. This mirrors the runtime's own top-down interpretation (`InputShaper`) combined with the file-reference resolution of `input_normalizer` in `pipelex`, so local and hosted execution read the same inputs the same way. (A caller value may be compact or the explicit `{ concept, content }` envelope — see "[Compact or explicit-envelope inputs](#compact-or-explicit-envelope-inputs)"; the walk below is over the compact value, i.e. the envelope's `content`.)
+
+The declared signature is the **input-form descriptor** — the MTHDS standard's `input_form` artifact, carried on the validate report under `views: ["input_form"]`. It states the kind of every input at every depth, so a file position is a *fact of the method*, never an inference from the value in front of it.
+
+The walk is discriminated on the node's `kind`:
+
+| Node kind | What the walk does |
+| --- | --- |
+| `document`, `image` | A **file position.** The caller's value there is resolved: a local path, `data:` URL, or bytes is uploaded and rewritten to canonical content carrying `pipelex-storage://` in `url`; an `http(s)` URL or an existing storage URI passes through; a canonical `{ url }` content dict has its `url` resolved and its other keys preserved. |
+| `object` | Walks the declared `fields` by name against a plain-object value. Keys the descriptor does not name are copied through untouched. **Optional fields (`required: false`) are walked when present.** |
+| `list` | Walks `item` against each element of an array value. |
+| everything else (`text`, `prose`, `date`, `number`, `boolean`, `enum`, `unknown`) | Passes through at any depth. |
+
+Two consequences worth stating outright, because they are what the descriptor buys:
+
+- **Whether a concept is native or refines `Document` / `Image` is the descriptor's statement** (`refines`), never a shape test. A concept refining `native.Document` arrives as `kind: "document"` and is prepared as one.
+- **`unknown` is not interpreted.** It is the standard's escape hatch for a `Dynamic` / `Composite` input, and the walk does not enter it — including a canonical file dict nested inside one. The signature genuinely declares no file there, and uploading on the strength of a `url` key is precisely the guess this helper stopped making. A caller with such an input uploads with `uploadFile` first and passes the storage URI.
+
+A repeated reference to the **same source** within one preparation is uploaded once and rewritten consistently (within-preparation dedup by source identity). A caller value whose shape disagrees with the node — a scalar at an `object`, a non-array at a `list` — passes through for the run to reject; preparation never second-guesses the signature.
+
+> **Why the descriptor, and why not the template.** Preparation used to read the *explicit inputs template* (`buildInputs` with `explicit: true`), whose file signal was a rendered `{"url": …}` dict. That signal is a side effect of a field being **named** `url` or `*_url`, not of the field being an Image or a Document — so a text field named `url` was read off the caller's disk and uploaded, and an **optional** nested file field, which the required-only template never rendered, was invisible and travelled to the runner as a literal path. Both were live misclassifications. The descriptor states the kind instead of implying it, and states it at every depth, so both edges fall out correctly rather than needing special cases. It costs the same one round trip, it is the standard's own artifact rather than a presentation of it, and it is what `@pipelex/mthds-form`, `pipelex-app`, `mthds-ui` and `pipelex-mcp` already derive from. The template remains what it always was — a fill-in scaffold for a person or an agent — and is projected from this same descriptor.
+
+### Pass-through rules
+
+| Source at a file-bearing input | Action |
+| --- | --- |
+| Local path (Node) / data URL / bytes | Upload → rewrite to `pipelex-storage://` |
+| Existing `pipelex-storage://` URI | Already prepared — pass through unchanged |
+| HTTP(S) URL | Pass through unchanged, **unless** the caller explicitly asks to ingest it into Pipelex storage |
+
+## Rewritten-input shape: `url` carries the URI
+
+The runtime's canonical image/document content stores its reference in a **`url`** field. Preparation emits inputs the runtime interprets natively, so a rewritten input keeps the canonical content shape with `url` holding the `pipelex-storage://` value — exactly what the runtime's `input_normalizer` writes.
+
+The "uploaded reference is named `uri`" decision applies to the **upload surface**: the raw upload result and each upload record expose the storage reference as `uri`. Preparation must **not** invent a `uri` field inside rewritten inputs — that would produce inputs the runtime does not recognize.
+
+## Error and capability behavior
+
+Upload is a **hosted Pipelex-product capability**, even though the SDK can be pointed at other base URLs. A deployment that does not support upload must produce a specific, actionable error — preparation must never silently leave a local path in place and let a later run fail obscurely.
+
+The contract distinguishes these semantic outcomes, each a typed subclass of `InputPreparationError` (catch the base to handle any preparation failure, a subclass to branch on category):
+
+- **an unresolvable signature** (`InputPreparationError`) — the closure did not validate (`is_valid: false`, carrying the first error's message), the report carried no `input_form` descriptor, or no pipe could be chosen (an unknown or bare `pipe_ref`, or no default). A *no-verdict* condition from `/v1/validate` — a malformed selector, an unknown or foreign-org `method_id`, no package at the address, auth, a server fault — stays an `ApiResponseError` and propagates unchanged;
+- **empty method source** (`EmptyMethodSourceError`, carries `methodId`) — `getMethodClosure` found the stored method but its `mthds` source parses to nothing (the row exists, no runnable source yet). Distinct from the `getMethod` `404` for an unknown/foreign id, which stays an `ApiResponseError`. `prepareInputs` never raises it: it hands the id to the server, which answers a sourceless method with a `422`;
+- **invalid local source** (`InvalidLocalSourceError`) — missing, unreadable, or a path string outside Node;
+- **rejected asset** (`RejectedAssetError`) — the server refused it (e.g. a `413` past the service-defined size cap — see "Storage policy" — surfaced as a clear rejection, not a raw transport error);
+- **unsupported server capability** (`UnsupportedUploadCapabilityError`) — the configured deployment has no upload route;
+- **authentication / authorization failure** (`UploadAuthenticationError`) — `401` / `403`;
+- **transport failure** (`UploadTransportError`) — a network or server fault, a malformed data URL payload, or any other unexpected upload failure.
+
+All preparation failures are raised **before any run is created**.
+
+## Storage policy (inherited, Phase 1)
+
+The SDK ships against **today's route behavior**: a service-defined size cap (hosted default 50 MiB via `MAX_UPLOAD_MIB`, rejected with `413`), auth required, per-user keys, and nothing else — no MIME validation, retention, quotas, dedup, or cleanup. The SDK documents limits as **service-defined** and surfaces server rejections as clear "rejected asset" errors; it does **not** hardcode a client-side cap. Real storage policy (retention, quotas, org scoping, cleanup) is a later hosted-owner deliverable.
+
+## Stability across the future endpoint move
+
+The public abstraction sits deliberately **above** the HTTP route. Callers depend on `uploadFile` / `prepareInputs`, the `uri` result field, and the `pipelex-storage://` scheme — never on which backend service owns the route. The current transport is `POST /v1/upload` on `pipelex-api`; when hosted storage upload later moves to `pipelex-platform` (together with its paired resolution route, as one storage domain), the public path and wire shape are kept compatible so released SDK versions keep working, and any wire-protocol change is absorbed inside the SDK's upload transport. `uploadFile`, `prepareInputs`, and the prepared run-input shape stay stable across that move.