@pipelex/sdk 0.18.1 → 0.19.0

package/docs/run-results.md

@@ -0,0 +1,159 @@
+# Reading a run's results
+
+A completed run hands back one object, `RunResults` (`src/runs.ts`), and every field of it is described here. The same object comes back from `startAndWaitForResult`, from `waitForResult(runId)`, and from the `completed` arm of `getRunResult(runId)` — one accessor set whichever path ran, which is the point of the type.
+
+Two paths produce it. Against the hosted API the SDK starts a durable run and polls `GET /v1/runs/{id}/results`, where the platform relays the run's S3 artifacts verbatim. Against a bare `pipelex-api` runner, which has no run store, the SDK falls back to the blocking `POST /v1/execute` and maps the runner's native `pipe_output` onto the same shape, lifting the artifacts that ride it onto their own fields. `startAndWaitForResult` picks between the two from the `GET /v1/version` handshake, so a consumer does not choose.
+
+| field | type | hosted (durable) path | bare-runner (blocking) path |
+|---|---|---|---|
+| `pipeline_run_id` | `string` | the run store's id | the runner's own id for the call |
+| `main_stuff` | `unknown` | the `main_stuff.json` artifact | resolved out of the returned working memory |
+| `working_memory` | `DictWorkingMemory \| null` | the `working_memory.json` artifact | lifted off `pipe_output` |
+| `graph_spec` | `unknown` | the `graphspec.json` artifact | lifted off `pipe_output` |
+| `graph_assembly_error` | `string \| null \| undefined` | absent until the platform relays it | lifted off `pipe_output` |
+| `tokens_usages` | `TokensUsageRecord[] \| null` | the `tokens_usages.json` artifact | lifted off `pipe_output` |
+| `usage_assembly_error` | `string \| null` | relayed | lifted off `pipe_output` |
+| `pipe_output` | `DictPipeOutput \| null \| undefined` | absent | the runner's whole native output |
+
+## `pipeline_run_id` — the durable handle
+
+The run id is what makes a run readable after the process that started it has gone. `start` returns it in its acknowledgement before the run finishes, and every lifecycle read takes it: `getRunStatus(runId)` for the status row, `getRunResult(runId)` for a single result lookup, `waitForResult(runId)` to resume polling a run an earlier session started. It is also what a `RunTimeoutError` leaves you with — the run keeps executing server-side, so the timeout is a reason to re-poll by id, not a reason to run the method again.
+
+```ts
+const ack = await client.start({ pipe_code: "my_domain.summarize", inputs: { text: "..." } });
+console.log(ack.pipeline_run_id); // persist this — it outlives the process
+
+// …later, in another process
+const results = await client.waitForResult(ack.pipeline_run_id);
+```
+
+Against a bare runner the id identifies the call the runner just answered, but there is no run store behind it: the lifecycle routes are absent, so re-reading it raises `RunLifecycleUnavailableError`. Durable resumption is a hosted capability.
+
+## `main_stuff` — the output
+
+`main_stuff` is the resolved content of the run's main output and is always present for a completed run. On the hosted path it is the `main_stuff.json` artifact; on the blocking path the SDK resolves it out of the returned working memory through the response's `main_stuff_name`. Both deliver the same content shape, so there is no shape-guessing and no path-dependent branch to write. A completed run that cannot deliver one throws `MissingMainStuffError` rather than handing back a half-filled result.
+
+It is typed `unknown` because the content is polymorphic: a structured output arrives as an object of the concept's fields, and a multiple output as the envelope `{ items: [...] }` that the runtime's `ListContent` serialises to. Every content type serialises to an object, natives included — a text output is `{ "text": "…" }` and a number `{ "number": 0 }` — so a guard written for a bare `""` or `0` never fires, and an empty multiple output is `{ "items": [] }` rather than `[]`. Narrow it where you read it, ideally through the types generated for the method rather than a hand-written cast.
+
+For a multiple output that means reading `items` off the object and mapping the generated per-concept parser over its members, because codegen emits a parser per concept and no wrapper type for the envelope. **Do not rely on the parser to catch the mistake for you.** The generated schemas are not strict, so a concept with a required field rejects the envelope loudly, while one whose fields are all optional parses it to `{}` and discards the output in silence.
+
+```ts
+const state = await client.getRunResult(runId);
+
+switch (state.state) {
+  case "completed": {
+    // `state.result` is the RunResults; `main_stuff` is the output content.
+    const summary = state.result.main_stuff as { title: string; bullets: string[] };
+    console.log(summary.title, summary.bullets.length);
+    break;
+  }
+  case "running":
+    console.log(`not finished — poll again in ${state.retry_after_seconds ?? 2}s`);
+    break;
+  case "failed":
+    console.error(`run ended as ${state.status}: ${state.message}`);
+    break;
+}
+```
+
+`getRunResult` is the single-shot lookup and returns that discriminated state. `waitForResult(runId)` drives the same lookup in a loop, honouring the server's `Retry-After`, and returns the `RunResults` directly — throwing `RunFailedError` on a terminal non-completed status and `RunTimeoutError` when the budget runs out.
+
+## `working_memory` — every named stuff of the run
+
+`working_memory` is the run's whole working memory: every named stuff the run held when it finished, as `{ root, aliases }`. `root` maps each stuff's name to the stuff itself — its concept ref, a string such as `native.Text`, and its content — and `aliases` maps further names onto root keys. It holds the inputs the run was given and the intermediates it produced as well as the main output, which is what a consumer needs to show what went into a run beside what came out, or to repopulate the inputs of a run it restores.
+
+It reads the same on both paths. On the hosted path it is the `working_memory.json` artifact, relayed verbatim; on the blocking path the SDK lifts it off `pipe_output`. On the hosted path it is `null` when the artifact had not been written when the results were delivered. On the blocking path it is always there, because the SDK resolves `main_stuff` out of it and a response without one throws `MissingMainStuffError` first.
+
+`main_stuff` is one of its entries — the content of the stuff the run names as its main output — already resolved. Read `main_stuff` for the output and `working_memory` for everything else; there is no need to find the main output in the working memory by hand. The type, `DictWorkingMemory`, mirrors the MTHDS standard's `DictWorkingMemory` field for field.
+
+```ts
+for (const [name, stuff] of Object.entries(results.working_memory?.root ?? {})) {
+  console.log(name, stuff.concept); // e.g. "text native.Text"
+}
+```
+
+**Older runs carry the concept as an object.** A runtime older than pipelex 0.60.0 wrote each stuff's `concept` as the full concept object rather than its ref string: every hosted run whose `working_memory.json` was written before the hosted plane moved to that release, and every run of a bare runner still pinned below it. The artifact is relayed as it was written and never migrated. The type deliberately stays the standard's `concept: string`, which is what every run from 0.60.0 on carries, rather than widening to describe relics — so a consumer listing runs that may predate the release reads `concept` as `unknown` and narrows it itself:
+
+```ts
+const concept: unknown = stuff.concept;
+const ref = typeof concept === "string" ? concept : undefined; // undefined: a pre-0.60.0 concept object
+```
+
+## `graph_spec` — the executed graph
+
+`graph_spec` is the graph the run actually executed: `meta.mode` is `"live"`, and there is one node per pipe with its execution status, its start and end timestamps, its inputs and outputs, and the inference models and cost attributed to it. It is the same document a local `pipelex` run writes as `graphspec.json`, so anything that reads one of those files reads this value unchanged.
+
+The field is typed `unknown` by a standing ruling ([`architecture.md`](./architecture.md#standard-artifacts-on-the-validate-report)): the canonical declaration is `GraphSpec` in `@pipelex/mthds-ui`, which carries a React peer dependency that a server-side SDK has no business taking, and the MTHDS standard declares nothing this SDK could import instead. The value is relayed verbatim either way — the typing says where the schema lives, not that the content is uncertain.
+
+**Rendering it.** `@pipelex/mthds-ui` ships the viewer that consumes it, and its `GraphSpec` type is the cast to use on the consumer side. The viewer must be loaded client-side only, because ReactFlow touches browser globals at module evaluation, so the import goes through `next/dynamic` with `ssr: false` rather than a static one. It also fills its parent, so that parent needs `position: relative` and a height of its own or the graph renders at zero height:
+
+```tsx
+"use client";
+
+import dynamic from "next/dynamic";
+import type { RunResults } from "@pipelex/sdk";
+import type { GraphSpec } from "@pipelex/mthds-ui/graph";
+
+const GraphViewer = dynamic(
+  () => import("@pipelex/mthds-ui/graph/react").then((m) => m.GraphViewer),
+  { ssr: false },
+);
+
+export function RunGraph({ results }: { results: RunResults }) {
+  if (!results.graph_spec) return null;
+  return (
+    <div style={{ position: "relative", height: "600px" }}>
+      <GraphViewer graphspec={results.graph_spec as GraphSpec} />
+    </div>
+  );
+}
+```
+
+**Keeping it.** The value is plain JSON, so persisting it is a write; there is no SDK helper and none is needed. Keeping it is worth doing for anything you may have to explain later, because it is the only record of what the run did pipe by pipe:
+
+```ts
+import { writeFile } from "node:fs/promises";
+
+await writeFile("graphspec.json", JSON.stringify(results.graph_spec, null, 2));
+```
+
+**When it is null.** On the hosted path, the artifact may not have been written when the results were delivered. On either path, the runner may have assembled no graph at all — which is what the next field is for.
+
+## `graph_assembly_error` — why there is no graph
+
+`graph_assembly_error` is the graph's twin of `usage_assembly_error`, and it exists for the same reason: a null `graph_spec` alone cannot say whether graph assembly was off, broke, or simply had not finished writing. When the runner's assembly failed, this field carries the runner's message.
+
+On the blocking path the SDK lifts it off `pipe_output`, beside the graph itself. **On the hosted path the key is absent, so the field reads `undefined` rather than `null`**: the platform's results body relays no such key and the SDK parses that body as it arrives, so the failure the bare runner reports is not yet observable through the hosted API. The field is declared ahead of that relay so consumers have one accessor to write against and nothing breaks the day the wire gains the key — the value appears on its own, with no SDK change. Until then, treat `undefined` on the hosted path as "no information", not as "assembly succeeded", and compare with `!= null` rather than `=== null` so that the same branch keeps working once the key arrives.
+
+```ts
+if (results.graph_assembly_error != null) {
+  console.warn("graph assembly failed for this run:", results.graph_assembly_error);
+} else if (results.graph_spec == null) {
+  // no graph: assembly was off, the artifact was not written, or (hosted) the error is not relayed
+}
+```
+
+## `tokens_usages` and `usage_assembly_error` — what the run consumed
+
+The usage pair reports what each inference call consumed and cost — one `TokensUsageRecord` per call, in completion order — and reads identically on both paths. The null-versus-empty semantics, the cost rules (`null` is unrated, `0` is priced at zero), the non-additive token categories and the pre-contract artifacts that still type-check all have their own page: [`run-usage.md`](./run-usage.md).
+
+For the run's totals, call `summarizeUsage(results)` rather than adding the records up by hand: it returns the total cost, the input and output token totals and a per-pipe rollup, with a `state` that tells a run with records from one that did no inference and from one whose usage is unavailable. See [Summarizing a run](./run-usage.md#summarizing-a-run--summarizeusage).
+
+## `pipe_output` — the runner's native output
+
+`pipe_output` is the bare runner's whole native output, and it is present on the blocking path only — the hosted results body carries no such key, so on that path it reads `undefined`. It is supplementary: `main_stuff`, `working_memory`, the graph and the usage pair are all lifted out of it onto fields that read the same on both paths, so a consumer that wants every named stuff of the run reads `working_memory`, not `pipe_output.working_memory`, and its code keeps working against the hosted API. What `pipe_output` adds is the runner's output exactly as it arrived. It is typed `DictPipeOutput`, which is extension-open — the runner's Pipelex extension fields are reachable through the index signature without casting the whole value away.
+
+## Produced files
+
+A run that produces an image, a PDF or a document does not embed the bytes. The content inside `main_stuff` carries the file's durable reference — a `pipelex-storage://` URI, in the content's `url` — beside a `public_url` the storage provider signed when the run wrote the file. **That signed link is short-lived and must not be stored**: it expires on the provider's own schedule, so a link persisted in a database or rendered into a cached page stops working without warning, while the `pipelex-storage://` reference beside it is permanent and is what belongs in your records.
+
+To read the bytes, mint a fresh link from the reference:
+
+```ts
+const resolved = await client.resolveStorageUrl({ uri: "pipelex-storage://..." });
+// { url, expires_at, content_type } — fetch `url` now; re-resolve for the next reader.
+```
+
+The same rule holds in a browser: resolve on the server, hand the client a link it uses immediately, and never let a presigned URL outlive the request it was minted for.
+
+Bringing the files down is the SDK's job, not something to re-implement over `resolveStorageUrl`: `collectArtifacts` lists a result's references without touching the network, `resolveArtifacts` mints fresh links for a whole list in one bulk call, `fetchArtifact` streams one file within safe bounds, and `downloadArtifacts` saves everything a run produced under a directory, by run id even days later. All four are on their own page, [`artifact-download.md`](./artifact-download.md). The upload direction — turning local files into `pipelex-storage://` references before a run — is the mirror of this and has its own page too, [`input-preparation.md`](./input-preparation.md).

package/docs/run-usage.md

@@ -0,0 +1,155 @@
+# Run usage — reading what a run consumed
+
+A completed run reports what its inference calls consumed as a list of `TokensUsageRecord` objects on `RunResults`, one per inference call, in the order the calls completed. This page covers how to read them, what each field means, the edge cases the type is deliberately shaped around, and `summarizeUsage`, which folds them into one run-level summary under those rules.
+
+The wire shape is not this SDK's invention: it is specified in Pipelex's protocol spec, under "TokensUsage records on run artifacts", and the `TokensUsageRecord` interface in `src/runs.ts` is a client-side mirror of it. `pipelex-sdk` (Python) carries the same mirror. The record is a Pipelex runtime concept rather than part of the MTHDS standard — the MTHDS protocol itself says nothing about usage reporting.
+
+## Reading the records
+
+```ts
+const result = await client.startAndWaitForResult({
+  pipe_code: "my_domain.summarize",
+  inputs: { text: "..." },
+});
+
+if (result.tokens_usages) {
+  for (const record of result.tokens_usages) {
+    console.log(record.pipe_code, record.inference_model_name, record.nb_tokens_by_category, record.cost);
+  }
+}
+```
+
+For the run's totals, do not add the records up by hand: [`summarizeUsage`](#summarizing-a-run--summarizeusage) does it under the rules below.
+
+The accessor is the same whichever path ran. `startAndWaitForResult` picks a path from the `GET /v1/version` handshake:
+
+- **Hosted (durable) path** — the records come from the runner's `tokens_usages.json` artifact, which `GET /v1/runs/{id}/results` unpacks onto the results body as top-level keys and relays verbatim.
+- **Bare runner (blocking) path** — the records ride the execute response's extension-open `pipe_output` as Pipelex extension fields; the SDK lifts them onto the same two top-level fields.
+
+Because the runtime emits both surfaces through one helper, the two cannot structurally diverge.
+
+## Field reference
+
+| field | type | meaning |
+|---|---|---|
+| `model_type` | `string \| null` | Kind of inference. Known values: `llm`, `img_gen`, `extract`, `search`. |
+| `inference_model_name` | `string \| null` | Human model name (e.g. `gpt-4o`). |
+| `inference_model_id` | `string \| null` | Provider/platform model id (e.g. `gpt-4o-2024-11-20`). |
+| `pipe_code` | `string \| null` | The pipe that made the call — what makes per-pipe cost attribution possible. |
+| `job_category` | `string \| null` | Known values: `llm_job`, `img_gen_job`, `extract_job`, `search_job`, `jinja2_job`, `mock_job`. |
+| `unit_job_id` | `string \| null` | Known values: `llm_gen_text`, `llm_gen_object`, `img_gen_text_to_image`, `extract_pages`, `search_sourced_answer`, `search_structured`. |
+| `nb_tokens_by_category` | `Record<string, number> \| null` | Raw provider-reported token counts, keyed by token category (`input`, `input_cached`, `output`, `output_reasoning`, …). |
+| `cost` | `number \| null` | Computed USD cost of this call. |
+| `started_at` | `string \| null` | ISO 8601. |
+| `completed_at` | `string \| null` | ISO 8601. |
+
+Two traps worth naming explicitly:
+
+- **Token categories are not additive.** `input` is the joined total and `input_cached` is a *subset* of it. Summing every category double-counts the cached tokens.
+- **Duration is not shipped.** Derive it from the `started_at` / `completed_at` pair.
+
+### Enum-ish fields are open sets
+
+`model_type`, `job_category`, `unit_job_id`, and the `nb_tokens_by_category` keys are plain `string`, never frozen union types, and the values listed above are *known* values rather than an exhaustive set. This is deliberate: the runtime can add an inference kind without breaking any SDK consumer. Switch on them defensively — do not assume the list is closed.
+
+## Cost semantics
+
+`cost` is a server-computed USD total for that one call. The rate table behind it is not a contract field and does not cross the wire, so there is nothing to recompute client-side and no risk of a client's arithmetic disagreeing with the runtime's own reporting — the figure comes from the same cost engine that produces the local CLI cost table. (Pre-contract artifacts are the one exception: they carry a raw `unit_costs` table, which is a relic rather than an API — see [Old artifacts type-check too](#old-artifacts-type-check-too).)
+
+- `cost === null` means the model has **no rate table at all** — an own-GPU model, a mock run, a dry run.
+- `cost === 0` means a rate table existed and priced the call at zero.
+
+Those are different facts; `record.cost ?? 0` conflates them, which is fine for a sum but wrong for "was this call priced?".
+
+There is no per-category cost breakdown and no run-level aggregate on the wire. The run total is the sum of the records, and [`summarizeUsage`](#summarizing-a-run--summarizeusage) computes it with the `null` / `0` distinction kept.
+
+## Null and empty semantics
+
+`tokens_usages` is null whenever usage assembly produced no list at all, which happens for three different reasons:
+
+- usage assembly was **off** for the run;
+- usage assembly **broke** (an event-read failure);
+- on the hosted path, the run was **delivered before the artifact existed**.
+
+It is `[]` when assembly ran, succeeded, and no inference happened, and non-empty otherwise. An empty list is a run that did no inference, so the run's total cost is `0` and its token totals are zero, never `null`: `null` stays reserved for calls that were not rated.
+
+`usage_assembly_error` is the **only** field that distinguishes the broken case from the other two — they are otherwise indistinguishable on the wire. A caller that needs to tell "we have no usage data because something failed" from "there was nothing to report" must branch on `usage_assembly_error`, not on `tokens_usages` alone:
+
+```ts
+if (result.usage_assembly_error != null) {
+  console.warn("usage assembly failed for this run:", result.usage_assembly_error);
+} else if (result.tokens_usages == null) {
+  // usage was off, or this run predates the artifact
+} else if (result.tokens_usages.length === 0) {
+  // ran, but no inference happened
+}
+```
+
+## Summarizing a run — `summarizeUsage`
+
+`summarizeUsage(results)` folds a run's usage pair into one run-level reading and a per-pipe rollup, applying every rule on this page so that no consumer has to re-derive them. It is pure: it does no I/O, needs no client, and leaves its input untouched. It takes the whole `RunResults`, or any object carrying the `tokens_usages` / `usage_assembly_error` pair.
+
+```ts
+import { summarizeUsage } from "@pipelex/sdk";
+
+const usage = summarizeUsage(result);
+
+switch (usage.state) {
+  case "records": {
+    const cost = usage.total_cost_usd === null ? "not rated" : `$${usage.total_cost_usd.toFixed(4)}`;
+    console.log(`${usage.calls} calls, ${cost}${usage.cost_partial ? " (partial)" : ""}`);
+    for (const pipe of usage.by_pipe) {
+      console.log(pipe.pipe_code ?? "(unattributed)", pipe.total_cost_usd, pipe.calls);
+    }
+    break;
+  }
+  case "no_inference":
+    console.log("no inference happened: $0");
+    break;
+  case "unavailable":
+    console.log(usage.assembly_error ?? "usage was not reported for this run");
+    break;
+}
+```
+
+| field | type | meaning |
+|---|---|---|
+| `state` | `"records" \| "no_inference" \| "unavailable"` | Which reading of `tokens_usages` the summary describes. Read it first. |
+| `total_cost_usd` | `number \| null` | Sum of the priced calls' costs, in USD. |
+| `cost_partial` | `boolean` | True when priced and unrated calls are mixed, so the total covers the priced calls only and is a lower bound. |
+| `tokens` | `{ input: number \| null; output: number \| null }` | The two additive token totals, each summed over the records that reported it. A category no record reported is `null`, which is different from a reported `0`. |
+| `calls` | `number` | Number of records summarized, one per inference call. |
+| `assembly_error` | `string \| null` | `usage_assembly_error` as relayed. |
+| `by_pipe` | `PipeUsageSummary[]` | One row per `pipe_code`, each carrying `pipe_code`, `total_cost_usd`, `cost_partial`, `tokens` and `calls`, folded over that pipe's calls exactly as the run level is. |
+
+The three states follow the [null and empty semantics](#null-and-empty-semantics) above:
+
+| `state` | `tokens_usages` | `total_cost_usd` | `tokens` | `calls` | `by_pipe` |
+|---|---|---|---|---|---|
+| `records` | a non-empty list | the priced sum, or `null` when no call was priced | the summed totals | the record count | one row per pipe |
+| `no_inference` | `[]` | `0` | `{ input: 0, output: 0 }` | `0` | `[]` |
+| `unavailable` | `null` or absent | `null` | `{ input: null, output: null }` | `0` | `[]` |
+
+A `null` `total_cost_usd` therefore means one of two things, and `state` tells them apart: under `records` no call was rated, and under `unavailable` nothing is known. Within `unavailable`, `assembly_error` is still the only sign that usage assembly broke rather than being off or not yet written.
+
+`by_pipe` puts the most expensive pipe first. Priced pipes come by cost, descending, and unrated pipes after every priced one; a tie breaks on the call count, descending, and then on the pipe code. The calls the runtime did not attribute to a pipe (`pipe_code: null`) form one group of their own, which sorts after the named pipes when everything else ties.
+
+A [pre-contract record](#old-artifacts-type-check-too) carries no `cost` and no `pipe_code`, so it counts as unrated and unattributed. Its legacy `job_metadata` and `unit_costs` are never read, so an old artifact shows up as a partial or `null` total rather than as a figure the SDK guessed.
+
+## Old artifacts type-check too
+
+Durable artifacts written before this contract shipped are relayed verbatim and never migrated. `TokensUsageRecord` therefore keeps **every field optional** and carries an index signature (`[extension: string]: unknown`) — a pre-contract record is a valid value:
+
+- `cost` arrives absent (it did not exist yet — the record carried a raw `unit_costs` rate table instead);
+- `pipe_code` arrives absent (it was still nested inside a `job_metadata` object rather than flattened onto the record);
+- the legacy `job_metadata` and `unit_costs` stay reachable through the index signature.
+
+Those legacy fields are **not** contract fields. They exist on old records only, and reading them is reading a relic — a record the current runtime emits never carries them. Treat their presence as a signal that you are looking at an old artifact, not as an API.
+
+Conversely, a record the current runtime emits always carries the **full key set**: a field with no value is an explicit `null`, never an omitted key. You can read any field without an existence check — though TypeScript will still narrow the optional types, so keep the null handling.
+
+## What is deliberately absent
+
+The runtime's internal reporting models carry execution plumbing — `job_metadata`, `otel_context`, `trace_context`, `session_id`, `request_id`, `user_id`, `pipe_run_id`, `content_generation_job_id` — that is dropped at the boundary: on a record emitted under this contract, finding one of these is reading a leak, not a contract field. This is enforced upstream by leak-regression tests in `pipelex` and a conformance leak guard that walks relayed records at any nesting depth. Pre-contract artifacts are the documented exemption — relayed verbatim, they legitimately still carry `job_metadata` and `unit_costs`, and the leak guard does not run on them.
+
+One consequence worth knowing: the record shape is **invariant** with respect to server-side telemetry and tracing settings, because the only fields that varied with them are precisely the ones the boundary drops. You never get a structurally different record because an operator changed an observability setting.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pipelex/sdk",
-  "version": "0.18.1",
+  "version": "0.19.0",
   "description": "TypeScript SDK for the Pipelex hosted API — execute methods, manage runs, and call the product surface.",
   "license": "MIT",
   "repository": {
@@ -36,7 +36,8 @@
     "prepare": "tsc"
   },
   "files": [
-    "dist/"
+    "dist/",
+    "docs/"
   ],
   "engines": {
     "node": ">=22.12.0"