@pipelex/sdk 0.37.0 → 0.39.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -4
- package/dist/cli/run.js +8 -111
- package/dist/cli/run.js.map +1 -1
- package/dist/cli/script.d.ts +1 -1
- package/dist/client.d.ts +33 -10
- package/dist/client.js +111 -44
- package/dist/client.js.map +1 -1
- package/dist/error-models.d.ts +64 -18
- package/dist/error-models.js +20 -9
- package/dist/error-models.js.map +1 -1
- package/dist/errors.d.ts +114 -22
- package/dist/errors.js +114 -21
- package/dist/errors.js.map +1 -1
- package/dist/execute-result.js +4 -4
- package/dist/execute-result.js.map +1 -1
- package/dist/index.d.ts +3 -3
- package/dist/index.js +3 -2
- package/dist/index.js.map +1 -1
- package/dist/prepare-inputs.d.ts +13 -0
- package/dist/prepare-inputs.js +86 -41
- package/dist/prepare-inputs.js.map +1 -1
- package/dist/product-models.d.ts +13 -5
- package/dist/run-may-have-started.d.ts +31 -0
- package/dist/run-may-have-started.js +138 -0
- package/dist/run-may-have-started.js.map +1 -0
- package/dist/runs.d.ts +95 -7
- package/dist/runs.js +18 -0
- package/dist/runs.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/architecture.md +3 -2
- package/docs/cli.md +1 -1
- package/docs/errors.md +85 -16
- package/docs/input-preparation.md +18 -0
- package/docs/run-results.md +39 -0
- package/package.json +3 -3
package/docs/errors.md
CHANGED
|
@@ -30,26 +30,28 @@ try {
|
|
|
30
30
|
|
|
31
31
|
`errorVerdictOf` checks the shape rather than the class: it returns the pair from any `Error` carrying a boolean `retryable` and a known `errorDomain`, and `undefined` otherwise. So it reads an error raised by another copy of the SDK installed beside yours, a subclass of your own, and an `mthds` `ApiResponseError` whose runner sent both members. Every class also exposes the two members directly, and they derive from the abstract `PipelexRequestError`, itself a `PipelineRequestError`, so `instanceof PipelineRequestError` still matches all of them.
|
|
32
32
|
|
|
33
|
-
**"Retryable" means a retry can succeed,
|
|
33
|
+
**"Retryable" means a retry can succeed, and never one that could start a second paid run.** The verdict says whether asking again can plausibly succeed, and on most routes it says nothing about whether the first attempt had an effect. A request that may create a run, a `start` or a blocking `execute`, is the exception: when it failed in a way that may have started a run anyway, the error's `runMayHaveStarted` is `true` and its `retryable` is `false`, whatever the server sent and whatever the status says, since asking again could start the run twice. [A failed run start](#a-failed-run-start--runmayhavestarted) says when that is and what to tell a person.
|
|
34
34
|
|
|
35
|
-
**A request the gateway cut off: `isGatewayCutOff`.** The hosted gateway gives up on a request it waits on at about 30 seconds and answers `503` or `504`, while the server may still be handling it: a `start` whose server is fetching a `method_ref`'s package
|
|
35
|
+
**A request the gateway cut off: `isGatewayCutOff`.** The hosted gateway gives up on a request it waits on at about 30 seconds and answers `503` or `504`, while the server may still be handling it: a blocking `execute`, or a `start` whose server is fetching a `method_ref`'s package. `isGatewayCutOff(error, elapsedMs)` tells such a failure from a refusal: it is `true` when, at least ~28 seconds after the request was sent, the failure is a `503` or `504` answer or the client's own time limit (`ApiUnreachableError` with the code `ABORT_TIMEOUT`), and `false` before then and for any other failure. The blocking `execute` applies it and throws `PipelineExecuteTimeoutError`, timing the request on the monotonic clock `performance.now`, since a change of the system's time moves `Date.now`. A caller no longer needs it to learn whether a failed start may have created a run: the error's `runMayHaveStarted` says so whatever the timing. The fetch API cannot say whether `ABORT_TIMEOUT` ran out before the connection was made, so the rule reads it as the gateway's; with Node's own dispatcher it never does past the threshold, since undici gives up a connection that does not come after ten seconds (`UND_ERR_CONNECT_TIMEOUT`), the TLS handshake included. The Python SDK's `is_gateway_cut_off` is the same rule.
|
|
36
36
|
|
|
37
37
|
**What carries no verdict.** `errorVerdictOf` returns `undefined` for what is not an SDK failure: the `RangeError` or `TypeError` the SDK raises for an argument of the wrong type or range before any request is sent (a `timeoutMs` out of range, a malformed `appInfo`, an artifact selection that is not an array), which names a bug in the calling code; and the caller's own abort, which the client rethrows untouched. The per-reference errors inside a `downloadArtifacts` verdict are values with a `code`, not thrown errors, and carry none either. An argument the client refuses for what it asks, such as two method selectors given to `validate` or `execute`, or a base URL with a path, is a `RequestArgumentError`, which carries a verdict like every SDK error.
|
|
38
38
|
|
|
39
39
|
### Each class's verdict
|
|
40
40
|
|
|
41
|
+
On an error from a request that may have started a run, `retryable` is `false` whatever this table says ([below](#a-failed-run-start--runmayhavestarted)).
|
|
42
|
+
|
|
41
43
|
| Class | `errorDomain` | `retryable` | Why |
|
|
42
44
|
|---|---|---|---|
|
|
43
45
|
| `ApiResponseError` | the server's, else the fallback | the server's, else no for a sent `input` or `config` domain, else the fallback | See [below](#a-refused-requests-verdict). |
|
|
44
46
|
| `RequestArgumentError` | `input` | no | The client refused a call's arguments before sending any request: no run source given to `execute` or `start`, run sources or method selectors that exclude each other, a selector rule of `validate`, an empty `validateFiles`, a reserved key in `extra`, a `publishMethod` without its token, a `getMethodVersion` version that is not a positive integer, a selector `parseMethodSelector` refuses, or an artifact selection that is empty or names an unknown artifact. Its options take an optional `verdict`, and the client declares `config` for a base URL that is not host-only, since it typically comes from `PIPELEX_BASE_URL`. That refusal shows the base URL's scheme and host alone and names the other parts it carried (credentials, a path, a query, a fragment) without their text, since those are where a secret travels in a URL; a value that is not an http or https URL is not shown at all. It is a `PipelineRequestError`, as the bare refusal it replaced was, and a refusal the standard's own run-source check raises keeps that original as `cause`. |
|
|
45
47
|
| `ApiUnreachableError` | `config` | yes | The address or the network must be checked, and a later attempt can get through. With the code `ABORT_TIMEOUT`, the SDK's own request timeout, it is `runtime`: the API took the request and did not answer in time. |
|
|
46
|
-
| `PipelineExecuteTimeoutError` | `input` | no | The blocking `execute` cannot outlast the gateway's limit; start the run and poll instead. |
|
|
48
|
+
| `PipelineExecuteTimeoutError` | `input` | no | The blocking `execute` cannot outlast the gateway's limit; start the run and poll instead. The run goes on, so its `runMayHaveStarted` is always `true`. |
|
|
47
49
|
| `RunFailedError` | the report's | the report's | See [a failed run](#a-failed-run--runerrorreport). |
|
|
48
50
|
| `RunTimeoutError`, `RunStillRunningError` | `runtime` | yes | The run is still going; waiting again, by its id, can succeed. |
|
|
49
51
|
| `RunLifecycleUnavailableError` | `config` | no | The base URL points at a bare runner without the run lifecycle. |
|
|
50
52
|
| `MissingMainStuffError` | `runtime` | no | The API broke its own contract on a completed run. |
|
|
51
53
|
| `PagingNotTerminatingError` | `runtime` | no | A server minting cursors forever; `pageLimit` is the ceiling `iterateMethods` stopped at. |
|
|
52
|
-
| `InputPreparationError` and `EmptyMethodSourceError`, `InvalidLocalSourceError`, `InvalidInputValueError`, `MethodLoadError`, `RejectedAssetError` | `input` | no | The asset, the inputs or the method selector must change. The family's base takes an optional `verdict` in its options, so a subclass of your own can declare another. |
|
|
54
|
+
| `InputPreparationError` and `EmptyMethodSourceError`, `InvalidLocalSourceError`, `InvalidInputValueError`, `UploadNotAllowedError`, `MethodLoadError`, `RejectedAssetError` | `input` | no | The asset, the inputs or the method selector must change. The family's base takes an optional `verdict` in its options, so a subclass of your own can declare another. |
|
|
53
55
|
| `UnsupportedUploadCapabilityError`, `UploadAuthenticationError` | `config` | no | The deployment or the credential must change. |
|
|
54
56
|
| `UploadTransportError` | the cause's, else by `code` | the cause's, else by `code` | `uploadFile` wraps the `ApiResponseError` or `ApiUnreachableError` the client threw, and takes its verdict, so a wrapped `402` is `config` and not retryable. Without such a cause, which is always the case from `uploadWithGrant`: `timeout`, `storage_timeout` and `conflict` are `runtime` and retryable, `conflict` because storage documents its `409 ConditionalRequestConflict` as retryable and the grant is not spent by it (a spent grant is the `412` of a `RejectedAssetError` with code `grant_used`); `server_error` is `runtime`, and retryable when [the fallback table](#a-refused-requests-verdict) would call its `status` retryable, any `5xx` but the `501` storage answers for a request it does not implement, or when it carries no status; `unexpected` is `runtime`, and retryable when that table would call its `status` retryable, which is the `408` or `429` storage answers when it times out or throttles the request rather than refusing the file, and not retryable for any other status or none; `unreachable` is `config` and retryable; `redirected` is `config` and not retryable; `invalid_grant_url` is `runtime` and not retryable. |
|
|
55
57
|
| `ArtifactOperationError`, `ScopeUnavailableError` | `runtime` | no | The family's base also takes an optional `verdict`: the SDK declares `input` where it refuses an argument (a `scope`, a bound, a location) and `config` where the environment refuses the download (a runtime with no filesystem, a directory that cannot be created). |
|
|
@@ -79,9 +81,54 @@ The fallback reads the HTTP status, the platform's `code`, and whether the body
|
|
|
79
81
|
|
|
80
82
|
The members the server sent are read first, and the fallback answers only for one it did not send: a runner's problem usually carries its own, the platform sends them on the problems it renders itself once its deployment classifies them, and a route that sends neither, as the platform's Lambda-backed routes do not yet, gets the fallback for both. **Nothing on the error says which of the two a value came from**: `problemDocument` keeps the document whole, so a reader that must tell a server-sent verdict from a derived one reads it there, as the live suite `tests/e2e/error-verdicts.e2e.ts` does. A rejected API key always gets the fallback's verdict: the hosted gateway's authorizer refuses it before the platform sees the request, with a bare `403` whose body is `{"message":"Forbidden"}`, which carries no `code`, no `type` and no verdict, so the error reads it as `config` and not retryable from the status alone.
|
|
81
83
|
|
|
82
|
-
**The verdicts are a contract with the Python SDK.** `pipelex-sdk` must apply the same fallback
|
|
84
|
+
**The verdicts are a contract with the Python SDK.** `pipelex-sdk` must apply the same fallback, declare the same verdict on each class it shares with this package, and read a failed run start the same way. `tests/fixtures/error-verdicts.json` holds every case the two must agree on, the fallback's rows, the server-sent resolution, each class's variants and the answers to a run-creating request (its `run_start` member, each row sent through a real `start` or `execute`), and this package's test suite drives its own code through it. The file is the source: `make shared-files` at the repository's root writes its byte for byte identical copy to `python/tests/fixtures/error-verdicts.json`, `make check-shared-files` fails on a stale copy, and the Python suite drives its own code through the same cases (`python/docs/errors.md`). A class or a variant only one SDK has is marked `only_in` in the file, and the other suite skips it. So a change to a verdict is made in the file here, copied, and then made in both packages.
|
|
85
|
+
|
|
86
|
+
The shapes are not this SDK's invention. The report is the runner's `ErrorReport` (in `pipelex`), stored by the platform exactly as the runner wrote it and served through the runner's STRICT disclosure, as [below](#what-the-platform-serves-of-a-report), and the problem members are the ones the hosted-envelope spec names, `conformance/specs/pipelex-hosted-envelope.md` in the `conformance` repository, where the cross-repo specs sit beside the tests that verify them. The types live in `src/error-models.ts`; `pipelex-sdk` (Python) carries the same fields under snake_case names.
|
|
87
|
+
|
|
88
|
+
### A failed run start — `runMayHaveStarted`
|
|
89
|
+
|
|
90
|
+
Every error the SDK throws carries a third member beside the verdict, `runMayHaveStarted`, a boolean that is never `undefined`. It is `true` when the request that failed may have started a run whose id never reached the caller, and `false` otherwise, which includes every error from a request that cannot start a run. Read it with `runMayHaveStartedOf`, which takes anything a `catch` holds and, like `errorVerdictOf`, checks the shape rather than the class: it is `true` only for an `Error` whose `runMayHaveStarted` is `true`, so it reads an error raised by another copy of the SDK, and an `mthds` error, which never carries the member, reads as `false`.
|
|
91
|
+
|
|
92
|
+
**The requests that may create a run** are `start` and the blocking `execute`, and inside `startAndWaitForResult` the start and the blocking execute it falls back to against a bare runner. Once `onStarted` has been called the run exists and the caller holds its id, so an error while the composite waits started nothing: its `runMayHaveStarted` is `false`, and its `retryable` is about waiting again on that id, never about calling the composite again. `RunStillRunningError`, the `202` a blocking execute answers for a run still going, names its run, so its member is `false` too; a `202` that names no run is an answer the SDK could not read, which may have started one.
|
|
93
|
+
|
|
94
|
+
A failure of a request that may create a run may have started one in these cases, and only these:
|
|
95
|
+
|
|
96
|
+
| Failure | May have started | Why |
|
|
97
|
+
|---|---|---|
|
|
98
|
+
| `ApiUnreachableError` that proves nothing was sent | no | The connection's set-up failed, so no request followed. |
|
|
99
|
+
| Any other `ApiUnreachableError`, a body that could not be decoded included | yes | The request may have left, and the answer was lost or unreadable. |
|
|
100
|
+
| `ApiResponseError` with a `2xx` status | yes | The server accepted the request; only its answer could not be read, which includes a `start` acknowledgement or an `execute` `202` without a string `pipeline_run_id`, and an `execute` answer that is not the completed run. |
|
|
101
|
+
| `ApiResponseError` with a `408` | yes | A proxy in front of the platform may answer it for a request it forwarded. |
|
|
102
|
+
| `ApiResponseError` with a `503` and the code `pipeline_start_unavailable` | no | The platform never reached the runner, and recorded the run as `FAILED`. |
|
|
103
|
+
| `ApiResponseError` with a `501` | no | The deployment does not serve the route. |
|
|
104
|
+
| `ApiResponseError` with any other `5xx` | yes | The server or a gateway failed while handling the request. |
|
|
105
|
+
| `ApiResponseError` with any other status | no | The server refused the request before running anything. |
|
|
106
|
+
| `PipelineExecuteTimeoutError` | yes | The gateway cut the execute off, and the run goes on. |
|
|
107
|
+
| `RunLifecycleUnavailableError`, `RequestArgumentError`, an input preparation error | no | Nothing that starts a run was sent. |
|
|
108
|
+
|
|
109
|
+
**The reading errs on the safe side.** A false "may have started" costs a look at the run history; a false "did not start" costs a second paid run. So every `5xx` counts unless the server says otherwise, as the `501` and the coded `503` do. A `503` without that code can follow a run the runner already started, since the platform answers a failure of its own storage with one after it has dispatched the start. Timing plays no part: a `503` or `504` that came back at once reads the same as one at the gateway's limit.
|
|
83
110
|
|
|
84
|
-
|
|
111
|
+
**What proves nothing was sent** is a failure of the connection's set-up, which no request follows: the name lookup or the connect call failing (the system calls `getaddrinfo` and `connect`: an unknown host, a refused connection, no route to the host or the network, the system's connect time limit), undici's own connect time limit, which covers the TLS handshake (`UND_ERR_CONNECT_TIMEOUT`), and a server certificate the TLS handshake refused. Several addresses each failing so, which Node reports together as an `AggregateError`, prove it too. Anything else may have come once the request had left: a reset (`ECONNRESET`), which Node reports alike whether the connection dropped during the TLS handshake or after the request; another TLS failure; the SDK's own time limit (`ABORT_TIMEOUT`), which fetch cannot place before or after the connection was made; and a body that arrived and could not be decoded.
|
|
112
|
+
|
|
113
|
+
**A run that may have started is never retryable.** When `runMayHaveStarted` is `true`, `retryable` is `false`, whatever the problem document sent and whatever the fallback table says; `errorDomain` is unchanged, and the server's own members stay on `problemDocument`. So a caller that branches on `retryable` alone never offers a retry that could start a second paid run. A caller that reads the member can say more: tell the person the run may exist, and to look it up in the run history (`listRuns`) before starting it again.
|
|
114
|
+
|
|
115
|
+
```ts
|
|
116
|
+
import { errorVerdictOf, runMayHaveStartedOf } from "@pipelex/sdk";
|
|
117
|
+
|
|
118
|
+
try {
|
|
119
|
+
await client.start({ method_id: "mt_abc123", inputs });
|
|
120
|
+
} catch (err) {
|
|
121
|
+
if (runMayHaveStartedOf(err)) {
|
|
122
|
+
return tell("The run may have started. Check the run history before starting it again.");
|
|
123
|
+
}
|
|
124
|
+
if (errorVerdictOf(err)?.retryable) return scheduleRetry();
|
|
125
|
+
throw err;
|
|
126
|
+
}
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
**What lifts the rule** is a start that is safe to retry. Once the platform replays a start sent again under the same `Idempotency-Key`, the SDK can send one on every start and report such a failure as retryable under the same key. The SDK sends no key today.
|
|
130
|
+
|
|
131
|
+
The Python SDK carries the same member as `run_may_have_started`, with the reader `run_may_have_started_of`, and reads every failure the same way; the `run_start` rows of the shared case file hold the two to the same answer for every status.
|
|
85
132
|
|
|
86
133
|
## A failed run — `RunErrorReport`
|
|
87
134
|
|
|
@@ -127,27 +174,48 @@ The report's fields, all optional because the runner owns the shape:
|
|
|
127
174
|
|
|
128
175
|
| Field | Meaning |
|
|
129
176
|
|---|---|
|
|
130
|
-
| `message` | What went wrong
|
|
177
|
+
| `message` | What went wrong. For a run failure it is where the failing pipe is named. A hosted read serves the author's copy only when `caller_facing_message` is `true`, and the placeholder `An internal error occurred.` otherwise. |
|
|
131
178
|
| `error_domain` | Who can fix it: `input` (the caller — a malformed method, a bad input), `config` (a configuration change — a model the backend does not serve, a missing secret), `runtime` (nobody beforehand — a provider outage). **A branch field.** |
|
|
132
179
|
| `type_uri` | The stable URI naming the error class, the same on every occurrence. **A branch field.** |
|
|
133
180
|
| `retryable` | Whether running it again can succeed. Absent means unknown, which is not `false`. **A branch field.** |
|
|
134
|
-
| `user_action` | The next step, as `{ kind, detail }`: `detail` is the advice in words, `kind` its category (`wait_and_retry`, `check_billing`, `check_credentials`, `change_input`, `change_model`, `contact_support`, `unknown`). |
|
|
181
|
+
| `user_action` | The next step, as `{ kind, detail }`: `detail` is the advice in words, `kind` its category (`wait_and_retry`, `check_billing`, `check_credentials`, `change_input`, `change_model`, `contact_support`, `unknown`). A hosted read serves it only beside caller-facing copy. |
|
|
135
182
|
| `title` | The stable human label of the error class. |
|
|
136
183
|
| `error_type` | The runner's exception class name — an open set, for display and support, never matched against. |
|
|
137
184
|
| `error_category` | The finer class of an inference failure: `transient`, `configuration`, `content`, `capacity`, `ambiguous`, `unknown`. |
|
|
138
|
-
| `model`, `provider` | The model and provider an inference failure involved. |
|
|
139
|
-
| `provider_metadata` | What the provider's SDK said: `provider`, `sdk_exception_type`, `message`, `status_code`, `request_id` (the provider's own), `retry_after_seconds`, `provider_error_code`. |
|
|
185
|
+
| `model`, `provider` | The model and provider an inference failure involved. Absent on a hosted read. |
|
|
186
|
+
| `provider_metadata` | What the provider's SDK said: `provider`, `sdk_exception_type`, `message`, `status_code`, `request_id` (the provider's own), `retry_after_seconds`, `provider_error_code`. A hosted read keeps only `status_code` and `retry_after_seconds`. |
|
|
140
187
|
| `validation_errors` | The structured diagnostics of a method that failed validation, the same [items](#a-validation-item--validationerroritem) the validate report carries. |
|
|
141
|
-
| `migration` | A pending configuration migration that explains the failure, present only when the runner's scan found one. |
|
|
142
|
-
| `caller_facing_message` | `true` when `message` was written as caller-facing copy. |
|
|
188
|
+
| `migration` | A pending configuration migration that explains the failure, present only when the runner's scan found one. Absent on a hosted read. |
|
|
189
|
+
| `caller_facing_message` | `true` when `message` was written as caller-facing copy. A hosted read keeps it on a caller-facing report and leaves it out otherwise. |
|
|
143
190
|
|
|
144
191
|
A field a newer runner adds is reachable through the interface's index signature before this SDK names it.
|
|
145
192
|
|
|
146
193
|
**The report is checked field by field, never cast.** Every read that hands one back applies the same check: a value that is not an object reads as no report (`null`); each named field is kept when it has the type the table above gives it and is absent otherwise, so a `message` that is a number or a `retryable` of `"no"` never reaches you as typed; `null` is kept wherever the type allows it, and an empty string is kept, since deciding that a blank field says nothing is presentation. `provider_metadata` and `migration` are checked one level down, `validation_errors` keeps only the items that are objects with a string `category` and `message`, and members the SDK does not name, in the report or in a nested object, are relayed as sent. **`user_action` is kept whole or not at all**: an object whose `kind` and `detail` are both strings, which is what its type promises. This is the one place the check differs from the Python SDK's, which keeps a partial action, its stored-report action being a lenient type of its own. On the run records, the check runs only when the record carries the `error` key, so a server that does not serve it still leaves it absent.
|
|
147
194
|
|
|
148
|
-
**
|
|
195
|
+
**A run the platform ends or settles itself carries the platform's report.** Some endings have no runner to write a report, and the platform writes one instead, in the runner's shape, so it reads like any other:
|
|
196
|
+
|
|
197
|
+
| `error_type` | When | `error_domain` | `retryable` | `user_action.kind` |
|
|
198
|
+
|---|---|---|---|---|
|
|
199
|
+
| `WorkflowTaskKeptFailingError` | The run's workflow task kept failing for the whole failure window, and the platform ended the run `FAILED`. While the window runs, the status read says so as a `workflow_task_failing` degradation ([run-results.md](./run-results.md#a-degraded-status--degraded-and-degradation)). | `runtime` | `false` | `contact_support` |
|
|
200
|
+
| `RunTimedOutError` | The workflow engine timed the run out, and it ended `TIMED_OUT`. | `runtime` | `false` | `contact_support` |
|
|
201
|
+
| `RunWorkflowNotFoundError` | The run's workflow cannot be found: it aged out of the engine's retention, or the run was lost while starting. The run ends `FAILED`. | `runtime` | `true` | `wait_and_retry` |
|
|
202
|
+
|
|
203
|
+
These reports are caller-facing copy: they quote nothing of the worker's own failure text, their `caller_facing_message` is `true`, and so they keep their message and their next step on a hosted read. The one exception is a `WorkflowTaskKeptFailingError` report rebuilt from a termination an earlier platform recorded, whose message quotes the worker's failure: it is not caller-facing, so it reads as the placeholder.
|
|
204
|
+
|
|
205
|
+
**A cancelled, terminated or timed-out run holds a report when one was sent.** Its `error` holds the report a completion callback carried, or, on a `TIMED_OUT` run, the `RunTimedOutError` report the platform writes for the timeout. A run the platform ended because its workflow kept failing is `FAILED`, never `TERMINATED`.
|
|
206
|
+
|
|
207
|
+
**The report is `null` when the run has none**: a completed run, one whose start failed, a blocking `execute` run, whose answer carried its reason, one cancelled, terminated or given its status by hand with no completion callback carrying a report, and one that ended before the platform wrote reports of its own. The status and the `detail` sentence still say what happened, and the absence of a report says nothing about why.
|
|
208
|
+
|
|
209
|
+
### What the platform serves of a report
|
|
210
|
+
|
|
211
|
+
**The platform serves the runner's STRICT projection, and the SDK strips nothing further.** The run row keeps the runner's report whole, as the record of the people who operate the platform, but every read that hands it to a caller (the status read, the run records and the results read's `409`) serves it through the rule the hosted runner applies to an error on `execute`:
|
|
212
|
+
|
|
213
|
+
- `error_type`, `title`, `type_uri`, `error_domain`, `error_category`, `retryable` and `validation_errors` pass through when present;
|
|
214
|
+
- `message` and `user_action` pass through only when the report's `caller_facing_message` is `true`, which the served report then keeps; otherwise `message` is the placeholder `An internal error occurred.`, and `user_action` and `caller_facing_message` are absent;
|
|
215
|
+
- `provider_metadata` keeps only `status_code` and `retry_after_seconds`, and is absent when neither is present;
|
|
216
|
+
- `model`, `provider` and `migration` are absent.
|
|
149
217
|
|
|
150
|
-
|
|
218
|
+
So `caller_facing_message` is how a consumer tells the author's copy from the placeholder, and `message` can be shown as it comes: it never carries the worker's internal text or a provider's raw answer. `RunFailedError.message`, being the `detail`, already contains the served message. Every deployed environment serves the projection; a platform run locally with `ERROR_DISCLOSURE=verbose` serves the report whole, which is why the type still names every member the runner writes. Deciding what a person sees is still the consumer's decision.
|
|
151
219
|
|
|
152
220
|
**Numbers in `provider_metadata` may arrive as strings.** A report read back from the platform's run store carries `status_code` and `retry_after_seconds` as strings (`"404"`, `"1.5"`), because the store returns its numbers as decimals that the platform serializes as strings; a problem document a runner renders carries them as numbers. Both are in the type, so read them with `Number(...)`.
|
|
153
221
|
|
|
@@ -155,13 +223,14 @@ A field a newer runner adds is reachable through the interface's index signature
|
|
|
155
223
|
|
|
156
224
|
Every non-2xx answer from a `/v1` route is a problem document, and the SDK throws an `ApiResponseError` whose fields are its members. Each field is `undefined` when the document did not carry it, and a member of the wrong type (a numeric `type`, a `retryable` of `"no"`, a `user_action` without a `detail`) reads as absent rather than as a wrong value. The nested members are checked one level down, exactly as a [stored report's](#a-failed-run--runerrorreport) are: `providerMetadata` and `migration` keep each named field only when it has its type, `validationErrors` keeps only the items that are objects with a string `category` and `message`, and each item of `errors` keeps its `field`, `code` and `detail` only when they are strings; members the SDK does not name are relayed as sent. The verdict is the exception: `errorDomain` and `retryable` are always decided, the server's value when it sent a valid one and otherwise [as the verdict section says](#a-refused-requests-verdict).
|
|
157
225
|
|
|
158
|
-
**An answer the SDK could not read is an `ApiResponseError` too.** A `2xx` whose body is not JSON, such as a gateway's HTML page or an empty body where the route answers one, is no result the SDK can hand back, so every route throws an `ApiResponseError` for it instead of letting the runtime's `SyntaxError` escape. So is a `2xx` whose body is JSON but not the object the route answers — `null`, a number, a string, an array — on every protocol, extension and run-lifecycle route, on `health()`, and on the product routes whose answer the SDK or its caller reads before anything else (`listMethods`, `getMethod`, `createMethod`, `writeDraft`, `renameMethod`, `publishMethod`, `listMethodVersions`, `getMethodVersion`, `listRuns`, `getRunDetail`, `upload`, `requestUploadGrant`, `resolveStorageUrls`), instead of the `TypeError` that reading it would raise. So is an object missing the member those reads depend on: a page of `listMethods`, `listMethodVersions` or `listRuns` whose `items` is not an array or whose `next_cursor` is neither a string nor `null`, which `iterateMethods` and `iterateRuns` could not walk or would follow forever; a publish result whose `outcome` is not `published`, `unchanged` or `refused`, or whose `method` is not an object, which a caller branching on `outcome` could not read; an `upload` answer with no non-empty string `uri`, which names no stored file; and a stored method or a published version whose `mthds` field is neither a string nor absent (a `null` one is still no source, which `getMethodClosure` reports as `EmptyMethodSourceError`) or whose `python` field is not the serialized `[{ name, content }]` list, as `getMethod`, `createMethod`, `writeDraft`, `renameMethod`, `publishMethod` and `getMethodVersion` answer it: that is server data no change to the call fixes, not an argument the caller passed. Its `status` is the `2xx`, `responseBody` the raw text, `cause` what made the answer unreadable (the parse failure, or `mthds`'s refusal of the `python` field; nothing for a body that parsed to something the route does not answer), `problemDocument` and the problem members are `undefined`, and its verdict is the fallback's for a status no refusal carries: `runtime`, not retryable. A product route that answers a `2xx` with no body at all, such as `revokePipelexApiKey`, still resolves to `undefined`, and the other product routes hand their decoded body back as it came.
|
|
226
|
+
**An answer the SDK could not read is an `ApiResponseError` too.** A `2xx` whose body is not JSON, such as a gateway's HTML page or an empty body where the route answers one, is no result the SDK can hand back, so every route throws an `ApiResponseError` for it instead of letting the runtime's `SyntaxError` escape. So is a `2xx` whose body is JSON but not the object the route answers — `null`, a number, a string, an array — on every protocol, extension and run-lifecycle route, on `health()`, and on the product routes whose answer the SDK or its caller reads before anything else (`listMethods`, `getMethod`, `createMethod`, `writeDraft`, `renameMethod`, `publishMethod`, `listMethodVersions`, `getMethodVersion`, `listRuns`, `getRunDetail`, `upload`, `requestUploadGrant`, `resolveStorageUrls`), instead of the `TypeError` that reading it would raise. So is an object missing the member those reads depend on: a page of `listMethods`, `listMethodVersions` or `listRuns` whose `items` is not an array or whose `next_cursor` is neither a string nor `null`, which `iterateMethods` and `iterateRuns` could not walk or would follow forever; a publish result whose `outcome` is not `published`, `unchanged` or `refused`, or whose `method` is not an object, which a caller branching on `outcome` could not read; an `upload` answer with no non-empty string `uri`, which names no stored file; and a stored method or a published version whose `mthds` field is neither a string nor absent (a `null` one is still no source, which `getMethodClosure` reports as `EmptyMethodSourceError`) or whose `python` field is not the serialized `[{ name, content }]` list, as `getMethod`, `createMethod`, `writeDraft`, `renameMethod`, `publishMethod` and `getMethodVersion` answer it: that is server data no change to the call fixes, not an argument the caller passed. Its `status` is the `2xx`, `responseBody` the raw text, `cause` what made the answer unreadable (the parse failure, or `mthds`'s refusal of the `python` field; nothing for a body that parsed to something the route does not answer), `problemDocument` and the problem members are `undefined`, and its verdict is the fallback's for a status no refusal carries: `runtime`, not retryable. On `start` and `execute` its `runMayHaveStarted` is `true`, since the server accepted the request; that includes a start acknowledgement whose `pipeline_run_id` is not a string, and an `execute` `202` that names no run, which leave the caller a run it could not follow, and an `execute` answer that is not the completed run: one whose `pipeline_run_id` is not a string, or whose `pipe_output` carries no string `pipeline_run_id` or no `working_memory` with `root` and `aliases` objects, the members the Python SDK's model of the answer requires. A product route that answers a `2xx` with no body at all, such as `revokePipelexApiKey`, still resolves to `undefined`, and the other product routes hand their decoded body back as it came.
|
|
159
227
|
|
|
160
228
|
| Field | Wire member | Meaning |
|
|
161
229
|
|---|---|---|
|
|
162
230
|
| `errorDomain` | `error_domain`, else the fallback | Who can fix it: `input`, `config` or `runtime`. Always decided. **A branch field.** |
|
|
163
231
|
| `type` | `type` | The stable URI naming the error class. **A branch field.** |
|
|
164
|
-
| `retryable` | `retryable`, else `false` beside a sent `input` or `config` domain, else the fallback | Whether retrying the same request can succeed. Always decided
|
|
232
|
+
| `retryable` | `retryable`, else `false` beside a sent `input` or `config` domain, else the fallback | Whether retrying the same request can succeed. Always decided, and `false` whenever `runMayHaveStarted` is `true`. |
|
|
233
|
+
| `runMayHaveStarted` | none | Whether a `start` or a blocking `execute` that failed may have started a run anyway ([above](#a-failed-run-start--runmayhavestarted)). Always decided. |
|
|
165
234
|
| `userAction` | `user_action` | The next step, `{ kind, detail }`. |
|
|
166
235
|
| `requestId` | `request_id`, else the `X-Request-ID` header | The id to hand to support: it finds the server's log lines for the request. |
|
|
167
236
|
| `title` | `title` | The stable human label of the error class. |
|
|
@@ -143,6 +143,21 @@ The route selects the pipe, and the ref it answers with is the one prepared —
|
|
|
143
143
|
|
|
144
144
|
There is no single-pipe fallback: a method that declares one pipe and no `main_pipe` is refused a selector-less run, and a selector-less preparation with it.
|
|
145
145
|
|
|
146
|
+
#### Preparing without uploads
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
client.prepareInputs({ method_id, inputs, allowUploads: false }) → PreparedInputs
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
`allowUploads: false` prepares the inputs without uploading anything, for a server that prepares inputs on a caller's behalf and holds no files of its own, such as a hosted MCP connector. Such a server must never read a file from its own disk because a caller named a path, and in the default mode the read happens inside the walk, where no seam the server controls could refuse it first. Under this option:
|
|
153
|
+
|
|
154
|
+
- a value at a file position that would need an upload — a local path, a `data:` URL, bytes (`Blob`, `File`, `ArrayBuffer`, `Uint8Array`), whether bare, inside canonical `{ url }` content or inside an explicit envelope — throws `UploadNotAllowedError` **before** anything is read from disk, decoded or sent to the upload route. A `data:` URL is refused without being decoded, so a malformed one is refused the same way;
|
|
155
|
+
- `http(s)` URLs and `pipelex-storage://` references pass through and are rewritten exactly as in the default mode, and `uploads` is always empty;
|
|
156
|
+
- a value no file input takes (a number, an object that is neither bytes nor `{ url }` content) is still an `InvalidInputValueError`, as in the default mode, since it would need no upload, only a correction;
|
|
157
|
+
- everything that is not a file position is untouched, as always: the walk is the same descriptor walk.
|
|
158
|
+
|
|
159
|
+
The error names the input as `inputName` and where inside it the value sits as `inputPath`: the input's name for a top-level file, then `.field` for each declared field and `[index]` for each list element (`dossier.attachments[1]`). An explicit envelope adds nothing to the path, since its `content` is the input's value. `valueKind` says what the value is, `local_path`, `data_url` or `bytes`, so a consumer can tell its caller what to send instead; the value itself is never in the message or on the error, since it is the caller's data. The option is `true` when omitted, must be a boolean when given — anything else is refused with an `InputPreparationError` before any request, since reading a stray `"false"` as permission would upload exactly what the caller meant to keep — and is not sent to the server. The option covers `prepareInputs` alone: `uploadFile` is an upload by definition.
|
|
160
|
+
|
|
146
161
|
## `getMethodClosure` — the explicit expansion utility
|
|
147
162
|
|
|
148
163
|
```
|
|
@@ -214,6 +229,8 @@ A repeated reference to the **same source** within one preparation is uploaded o
|
|
|
214
229
|
| Existing `pipelex-storage://` URI | Already prepared — pass through unchanged |
|
|
215
230
|
| HTTP(S) URL | Pass through unchanged, **unless** the caller explicitly asks to ingest it into Pipelex storage |
|
|
216
231
|
|
|
232
|
+
Under `allowUploads: false`, the first row is refused with `UploadNotAllowedError` instead (see "[Preparing without uploads](#preparing-without-uploads)"), and the other two are unchanged.
|
|
233
|
+
|
|
217
234
|
## Rewritten-input shape: `url` carries the URI
|
|
218
235
|
|
|
219
236
|
The runtime's canonical image/document content stores its reference in a **`url`** field. Preparation emits inputs the runtime interprets natively, so a rewritten input keeps the canonical content shape with `url` holding the `pipelex-storage://` value — exactly what the runtime's `input_normalizer` writes.
|
|
@@ -231,6 +248,7 @@ The contract distinguishes these semantic outcomes, each a typed subclass of `In
|
|
|
231
248
|
- **a method that does not load** (`MethodLoadError`) — the pipe I/O answer's `is_valid: false` arm. Its message names the first validation item's message, else the answer's own `message`; `validationErrors` holds the answer's items, each kept when it carries a string `category` and `message`, and `serverMessage` the answer's own `message` when it is a string, so a consumer lists every item rather than reading the first from a sentence. The method must change, not the inputs;
|
|
232
249
|
- **invalid local source** (`InvalidLocalSourceError`) — missing, unreadable, or a path string outside Node;
|
|
233
250
|
- **invalid input value** (`InvalidInputValueError`) — a value at a file input that cannot be turned into a file to upload: a `data:` URL that does not decode (no comma, bad base64, bad percent-encoding), or a value of a type no file input takes, such as a number or an object that is neither bytes nor `{url}` content. It is raised while the inputs are read, before anything is uploaded. With `InvalidLocalSourceError`, it is how a consumer tells the inputs' own mistakes from a method that does not load, though all three are `InputPreparationError`s with the same verdict;
|
|
251
|
+
- **upload not allowed** (`UploadNotAllowedError`, carries `inputName`, `inputPath` and `valueKind`) — under `allowUploads: false`, a value at a file input that would need an upload: a local path, a `data:` URL or bytes. It is raised before anything is read, decoded or uploaded (see "[Preparing without uploads](#preparing-without-uploads)");
|
|
234
252
|
- **rejected asset** (`RejectedAssetError`) — the server refused it (e.g. a `413` past the service-defined size cap — see "Storage policy" — surfaced as a clear rejection with `code` `too_large`, not a raw transport error), or storage refused an upload with a grant (a used or expired grant, or a file that differs from what the grant signed — see the table above for each `code`);
|
|
235
253
|
- **unsupported server capability** (`UnsupportedUploadCapabilityError`) — the configured deployment has no upload route;
|
|
236
254
|
- **authentication / authorization failure** (`UploadAuthenticationError`) — `401` / `403`;
|
package/docs/run-results.md
CHANGED
|
@@ -76,6 +76,45 @@ It is never called on the blocking path, a bare runner's `POST /v1/execute` or t
|
|
|
76
76
|
|
|
77
77
|
**Knowing whether a run may exist.** `onStarting`, the other callback `StartAndWaitForResultOptions` takes, is called right before each request that may create a run is sent: `POST /v1/start`, and the blocking `POST /v1/execute` of a bare runner or of the fallback to it, so it is called twice when a runner that looked hosted refuses the start. Until it is called no run exists, and once the `signal` has aborted it is not called and no such request is sent, so a caller that stops waiting before then can say that no run was started; from then until `onStarted`, a run may exist that the API has not yet named. It runs synchronously and its return value is ignored; an exception it throws propagates before the request is sent. The `pipelex-sdk` command words its interrupt message by it ([`cli.md`](cli.md)).
|
|
78
78
|
|
|
79
|
+
**A failed start says whether a run may exist.** An error from the start, or from the blocking execute, carries `runMayHaveStarted`: `true` when the request may have created a run although the call failed, so a person looks the run up in the run history before starting it again, and then its `retryable` is `false` ([`errors.md`](errors.md#a-failed-run-start--runmayhavestarted)). Once `onStarted` has been called the run exists and the caller holds its id, so an error while the composite waits started nothing: its `runMayHaveStarted` is `false`, and its `retryable` is about waiting again on that id with `waitForResult`, never about calling `startAndWaitForResult` again, which would start a second run.
|
|
80
|
+
|
|
81
|
+
## A degraded status — `degraded` and `degradation`
|
|
82
|
+
|
|
83
|
+
`getRunStatus(runId)` reads a run's status row, which is how a caller shows a run while it runs. It returns a `RunRead` (`src/runs.ts`): the run record plus `degraded`, `degradation`, `request_id` and `retry_after_seconds`, the last parsed from the `Retry-After` header.
|
|
84
|
+
|
|
85
|
+
**`degraded` means the read is not a plain fresh reading, and `degradation.reason` says why.** So `degraded` alone no longer means the run is fine; branch on the reason.
|
|
86
|
+
|
|
87
|
+
| `reason` | What holds | `status` | What to tell the person |
|
|
88
|
+
|---|---|---|---|
|
|
89
|
+
| `status_unavailable` | The platform could not reach the workflow engine. | The last one recorded. | Nothing about the run, which may be fine: the status is not fresh, and the next read comes after `retry_after_seconds`. |
|
|
90
|
+
| `workflow_task_failing` | The workflow task that drives the run has failed `failed_attempts` times in a row since `failing_since`, so the run is not making progress. | Live. | The run is stuck. The workflow engine keeps retrying, and if the task is still failing at `ends_at` the platform ends the run `FAILED`, with a `WorkflowTaskKeptFailingError` report ([errors.md](./errors.md#a-failed-run--runerrorreport)). |
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
const run = await client.getRunStatus(runId);
|
|
94
|
+
if (run.degraded) {
|
|
95
|
+
// A platform older than the degradation sends none; `degraded` then meant only this.
|
|
96
|
+
const reason = run.degradation?.reason ?? "status_unavailable";
|
|
97
|
+
if (reason === "workflow_task_failing") {
|
|
98
|
+
const ends = run.degradation?.ends_at ?? "its time limit";
|
|
99
|
+
console.warn(`The run is stuck (${run.degradation?.failed_attempts} failures in a row); it ends at ${ends}.`);
|
|
100
|
+
} else if (reason === "status_unavailable") {
|
|
101
|
+
console.warn(`The status is not fresh; reading again in ${run.retry_after_seconds ?? 5}s.`);
|
|
102
|
+
} else {
|
|
103
|
+
console.warn(`The status read is degraded (${reason}).`); // a reason this SDK does not know yet
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
- **`failing_since` and `ends_at` are `null` when the platform could not find the recorded failure** in the workflow's history. It then does not end the run itself, and the run's own time limit ends it `TIMED_OUT`, with a `RunTimedOutError` report. Both are always `null` for `status_unavailable`, and so is `failed_attempts`.
|
|
109
|
+
- **`message` is the platform's sentence.** For `workflow_task_failing` it says how many attempts have failed in a row, that the run is not making progress and when the platform will end it. It does not quote the worker's own failure text, which only a platform run locally with `ERROR_DISCLOSURE=verbose` adds, so a product surface may show it; `failed_attempts` and `ends_at` carry the same facts as fields. Never branch on its wording.
|
|
110
|
+
- **`reason` is read open**, typed `DegradationReason | (string & {})`: a reason a later platform adds arrives as its raw string rather than failing the read, and a consumer that does not know it should not tell the person the run is fine.
|
|
111
|
+
- **The degradation is checked field by field, so it never fails the poll.** A `degradation` that is not an object, or whose `reason` is not a string, reads as `null`, since it cannot be branched on; `message`, `failing_since` and `ends_at` are kept when they are strings and `failed_attempts` when it is an integer, `null` kept for each, and absent otherwise; members this SDK does not name are relayed as sent. `degradation` is `null` whenever `degraded` is false, and absent on a platform that predates it. `degraded` is left as the server sent it.
|
|
112
|
+
- **`request_id` is the id of the request that started the run**, the one to quote to support beside the run id. `getRunDetail` carries it too; it is `null` for a run saved before the platform recorded it.
|
|
113
|
+
|
|
114
|
+
**The wait loops never see a degradation, and need not.** `waitForResult` and `startAndWaitForResult` poll the results route, whose in-flight answer is a `202` with no body. That route settles the run the same way the status read does, so a waiter receives a stuck run's end as a `RunFailedError` carrying the `WorkflowTaskKeptFailingError` report within one poll of `ends_at`. A caller that wants to show the degradation while it waits reads `getRunStatus` beside the wait.
|
|
115
|
+
|
|
116
|
+
Both SDKs read a status body the same way, held to it by a case file they share, `tests/fixtures/run-status-reads.json`.
|
|
117
|
+
|
|
79
118
|
## `main_stuff` — the output
|
|
80
119
|
|
|
81
120
|
`main_stuff` is the resolved content of the run's main output and is always present for a completed run, unless the read's `artifacts` selection left it out. On the hosted path it is the `main_stuff.json` artifact; on the blocking path the SDK resolves it out of the returned working memory through the response's `main_stuff_name`. Both deliver the same content shape, so there is no shape-guessing and no path-dependent branch to write. A completed run that cannot deliver one throws `MissingMainStuffError` rather than handing back a half-filled result.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@pipelex/sdk",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.39.0",
|
|
4
4
|
"description": "TypeScript SDK for the Pipelex hosted API — execute methods, manage runs, and call the product surface.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -61,11 +61,11 @@
|
|
|
61
61
|
},
|
|
62
62
|
"devDependencies": {
|
|
63
63
|
"@eslint/js": "^9.39.4",
|
|
64
|
-
"@pipelex/tools-wasm": "0.
|
|
64
|
+
"@pipelex/tools-wasm": "0.5.0",
|
|
65
65
|
"@types/node": "^22.12.0",
|
|
66
66
|
"@vitest/coverage-v8": "^4.1.9",
|
|
67
67
|
"dependency-cruiser": "^17.4.3",
|
|
68
|
-
"esbuild": "^0.
|
|
68
|
+
"esbuild": "^0.28.2",
|
|
69
69
|
"eslint": "^9.39.4",
|
|
70
70
|
"globals": "^16.5.0",
|
|
71
71
|
"prettier": "^3.8.4",
|