@pipelex/sdk 0.51.0 → 0.53.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -19
- package/dist/answer-reading.d.ts +70 -0
- package/dist/answer-reading.js +106 -0
- package/dist/answer-reading.js.map +1 -0
- package/dist/artifacts.js +45 -32
- package/dist/artifacts.js.map +1 -1
- package/dist/cli/present.d.ts +11 -14
- package/dist/cli/present.js +41 -37
- package/dist/cli/present.js.map +1 -1
- package/dist/cli/run.js +8 -7
- package/dist/cli/run.js.map +1 -1
- package/dist/cli/script.d.ts +1 -1
- package/dist/cli/source.js +2 -2
- package/dist/cli/source.js.map +1 -1
- package/dist/client.d.ts +254 -213
- package/dist/client.js +542 -472
- package/dist/client.js.map +1 -1
- package/dist/codegen-check.d.ts +21 -15
- package/dist/codegen-check.js +54 -27
- package/dist/codegen-check.js.map +1 -1
- package/dist/errors.d.ts +298 -455
- package/dist/errors.js +426 -534
- package/dist/errors.js.map +1 -1
- package/dist/exact-json.d.ts +8 -12
- package/dist/exact-json.js +19 -12
- package/dist/exact-json.js.map +1 -1
- package/dist/hooks/check-core.d.ts +12 -7
- package/dist/hooks/check-core.js +20 -14
- package/dist/hooks/check-core.js.map +1 -1
- package/dist/hooks/validate-stage.js +12 -3
- package/dist/hooks/validate-stage.js.map +1 -1
- package/dist/index.d.ts +4 -5
- package/dist/index.js +7 -6
- package/dist/index.js.map +1 -1
- package/dist/method-selector.js +7 -5
- package/dist/method-selector.js.map +1 -1
- package/dist/method-trees.d.ts +6 -7
- package/dist/method-trees.js +31 -37
- package/dist/method-trees.js.map +1 -1
- package/dist/models.d.ts +33 -210
- package/dist/models.js +2 -1
- package/dist/models.js.map +1 -1
- package/dist/prepare-inputs.d.ts +2 -2
- package/dist/prepare-inputs.js +51 -54
- package/dist/prepare-inputs.js.map +1 -1
- package/dist/problem-items.d.ts +19 -0
- package/dist/problem-items.js +52 -0
- package/dist/problem-items.js.map +1 -0
- package/dist/product-models.d.ts +43 -29
- package/dist/records.d.ts +12 -0
- package/dist/records.js +23 -0
- package/dist/records.js.map +1 -0
- package/dist/runs.d.ts +49 -38
- package/dist/runs.js +45 -19
- package/dist/runs.js.map +1 -1
- package/dist/transport-reading.d.ts +18 -0
- package/dist/{run-may-have-started.js → transport-reading.js} +15 -44
- package/dist/transport-reading.js.map +1 -0
- package/dist/upload-grant.d.ts +9 -6
- package/dist/upload-grant.js +42 -29
- package/dist/upload-grant.js.map +1 -1
- package/dist/upload.js +53 -33
- package/dist/upload.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/architecture.md +19 -19
- package/docs/artifact-download.md +6 -6
- package/docs/cli.md +1 -1
- package/docs/crate-routes.md +13 -14
- package/docs/errors.md +154 -267
- package/docs/exact-integers.md +1 -1
- package/docs/files.md +13 -13
- package/docs/hook-bundle.md +15 -1
- package/docs/input-preparation.md +14 -14
- package/docs/method-copy.md +35 -33
- package/docs/method-examples.md +13 -14
- package/docs/method-trees.md +27 -28
- package/docs/organization-branding.md +14 -16
- package/docs/run-results.md +8 -8
- package/package.json +1 -1
- package/dist/error-models.d.ts +0 -302
- package/dist/error-models.js +0 -203
- package/dist/error-models.js.map +0 -1
- package/dist/error-verdicts.d.ts +0 -46
- package/dist/error-verdicts.js +0 -120
- package/dist/error-verdicts.js.map +0 -1
- package/dist/run-may-have-started.d.ts +0 -31
- package/dist/run-may-have-started.js.map +0 -1
package/docs/exact-integers.md
CHANGED
|
@@ -29,4 +29,4 @@ A number with a fraction or an exponent stays a double. A float with no fraction
|
|
|
29
29
|
|
|
30
30
|
Both functions rest on the JSON source-text proposal, which every runtime the package supports (Node 22.12 and later) provides: the reader on the source text `JSON.parse` hands its reviver, the writer on `JSON.rawJSON`. The reader is that one `JSON.parse` call with a reviver, so it reads strings, escapes, repeated keys and `__proto__` exactly as `JSON.parse` does; a text holding no run of 16 digits, and so no integer past the safe range, is read without the reviver.
|
|
31
31
|
|
|
32
|
-
On a runtime without them, neither ever rounds. The reader refuses a number past the safe range that reads as a whole number with `ExactJsonUnsupportedError` (`
|
|
32
|
+
On a runtime without them, neither ever rounds. The reader refuses a number past the safe range that reads as a whole number with `ExactJsonUnsupportedError` (`client.tool_outdated`, naming the JavaScript runtime), since without the source text it cannot tell `12345678901234567891` from `1.2345678901234567e19`; the client reports it as the `ApiResponseError` of an answer it cannot read (`client.response_unreadable`), with that refusal as `cause`. The writer refuses a `bigint` with the `TypeError` `JSON.stringify` throws for one, which the client reports as `RequestArgumentError` before anything is sent.
|
package/docs/files.md
CHANGED
|
@@ -79,13 +79,13 @@ Every member is optional, and each is sent only when given, so an empty `q`, cur
|
|
|
79
79
|
|
|
80
80
|
`methodId`, `runId` and `createdBy` choose which list serves the page, and with none of them it is the organization's. The other members filter that list.
|
|
81
81
|
|
|
82
|
-
**An empty `format` or `mediaType` is refused before any request** with a `RequestArgumentError`, since a list with no value would be sent as no parameter at all and list every format; a value that is not a list of strings is a `TypeError`, a bug in the calling code. Everything else the API checks itself: a `limit` out of range, an unknown `kind`, more than 100 values of `format` or `mediaType`, or an id that cannot be one is a `422`; a date bound that is not an instant with an offset, a range whose start is after its end, or a cursor of another list or other filters is a `400`; and a run that is not the organization's is a `404` `
|
|
82
|
+
**An empty `format` or `mediaType` is refused before any request** with a `RequestArgumentError`, since a list with no value would be sent as no parameter at all and list every format; a value that is not a list of strings is a `TypeError`, a bug in the calling code. Everything else the API checks itself: a `limit` out of range, an unknown `kind`, more than 100 values of `format` or `mediaType`, or an id that cannot be one is a `422`; a date bound that is not an instant with an offset, a range whose start is after its end, or a cursor of another list or other filters is a `400`; and a run that is not the organization's is a `404` `run.not_found`. Each is an `ApiResponseError` ([errors.md](./errors.md)).
|
|
83
83
|
|
|
84
84
|
## Paging
|
|
85
85
|
|
|
86
86
|
**Only a `null` `nextCursor` ends a list.** The platform applies the filters to the rows a page reads and never refills the page, so a filtered page can hold fewer rows than `limit`, or none, and still carry a cursor; a client keeps asking until the cursor is `null`. **A cursor is bound to the list and the filters that issued it**, so a client passes it back with the same query, and passing it to another is a `400`.
|
|
87
87
|
|
|
88
|
-
`iterateFiles` does both: it sends the same query with every page, keeps going through an empty page that carries a cursor, stops on a `null` cursor or on a cursor the server failed to advance, and throws `PagingNotTerminatingError` past a page ceiling no real list reaches, through the loop `iterateMethods`, `iterateRuns` and `iterateMethodExamples` share. A page whose `items` is not an array, or whose `next_cursor` is neither a string nor `null`, is an `ApiResponseError` of an answer the SDK cannot read (`
|
|
88
|
+
`iterateFiles` does both: it sends the same query with every page, keeps going through an empty page that carries a cursor, stops on a `null` cursor or on a cursor the server failed to advance, and throws `PagingNotTerminatingError` past a page ceiling no real list reaches, through the loop `iterateMethods`, `iterateRuns` and `iterateMethodExamples` share. A page whose `items` is not an array, or whose `next_cursor` is neither a string nor `null`, is an `ApiResponseError` of an answer the SDK cannot read (`client.response_unreadable`), never a page the iterator walks.
|
|
89
89
|
|
|
90
90
|
## One file and its uses
|
|
91
91
|
|
|
@@ -99,7 +99,7 @@ if (detail.uses.next_cursor !== null) {
|
|
|
99
99
|
}
|
|
100
100
|
```
|
|
101
101
|
|
|
102
|
-
The uses are paged inside the answer, so their cursor keeps the platform's spelling, `uses.next_cursor`, and goes back as `usesCursor`; `usesLimit` is 1 to 200, 50 by default. A record whose upload looked unfinished past the upload grant's lifespan is settled by the read, which answers it `stored` when its bytes are there. A file that is not the organization's is a `404` `not_found`. An answer whose `uses` is not a page of `items` and `next_cursor` is an `ApiResponseError` of an answer the SDK cannot read (`
|
|
102
|
+
The uses are paged inside the answer, so their cursor keeps the platform's spelling, `uses.next_cursor`, and goes back as `usesCursor`; `usesLimit` is 1 to 200, 50 by default. A record whose upload looked unfinished past the upload grant's lifespan is settled by the read, which answers it `stored` when its bytes are there. A file that is not the organization's is a `404` `pipelex.storage.not_found`. An answer whose `uses` is not a page of `items` and `next_cursor` is an `ApiResponseError` of an answer the SDK cannot read (`client.response_unreadable`).
|
|
103
103
|
|
|
104
104
|
## The records of an address
|
|
105
105
|
|
|
@@ -110,7 +110,7 @@ const { items } = await client.getFilesByUri(input.uri, { runId: run.pipeline_ru
|
|
|
110
110
|
const shown = items[0]?.name ?? input.uri.split("/").pop();
|
|
111
111
|
```
|
|
112
112
|
|
|
113
|
-
A `uri` that is not a non-empty string is refused before any request. The platform answers `
|
|
113
|
+
A `uri` that is not a non-empty string is refused before any request. The platform answers `422` `pipelex.storage.uri_invalid` for a malformed address, `403` `pipelex.org.mismatch` for another organization's, `404` `run.not_found` for a run that is not the organization's and `422` `request.invalid` for an address over 512 characters.
|
|
114
114
|
|
|
115
115
|
## Renaming a file
|
|
116
116
|
|
|
@@ -121,7 +121,7 @@ const renamed = await client.renameFile("fl_0199", { name: "March invoice" });
|
|
|
121
121
|
console.log(renamed.name); // "March invoice.pdf"
|
|
122
122
|
```
|
|
123
123
|
|
|
124
|
-
**An `admin` or a `member` renames any file; any other role, a runner among them, renames only the files whose record names it as `created_by`**: its own uploads and the outputs of the runs it started. Past that the platform answers `403` `
|
|
124
|
+
**An `admin` or a `member` renames any file; any other role, a runner among them, renames only the files whose record names it as `created_by`**: its own uploads and the outputs of the runs it started. Past that the platform answers `403` `auth.forbidden`, its `detail` naming the caller's role. A `name` that is not a string is refused before any request; a name over 512 characters or with nothing left once cleaned is a `422` `request.invalid`, and a file that is not the organization's a `404` `pipelex.storage.not_found`.
|
|
125
125
|
|
|
126
126
|
## Links that name the file
|
|
127
127
|
|
|
@@ -140,11 +140,11 @@ for (const link of items) {
|
|
|
140
140
|
|
|
141
141
|
**Each link is signed with a `Content-Disposition` naming the file**, so a download saves under the name a person sees rather than the bytes' hash. It opens in place for an image, a PDF or plain text and downloads for every other type, SVG, HTML and XML included, so a file that could run scripts never opens from a link. A file named by its id is linked under its record's name; an address under the name of its first stored record, the caller's before another member's, whose id the item's `file_id` then names.
|
|
142
142
|
|
|
143
|
-
**A link is a bearer capability.** Keep it out of logs and ask again for a fresh one rather than storing it. The SDK keeps it out of its own errors: an answer it cannot read, one that is not one item per requested file or whose item carries neither a link nor an error, is an `ApiResponseError` whose `responseBody` replaces every `url` member and every link's query with `[redacted]` ([errors.md](./errors.md)). An item that does not name exactly one of `file_id` and `uri` as a non-empty string, or a `files` that is not a list, is refused before any request. The platform answers `422` for more than 100 files, and a `500` `
|
|
143
|
+
**A link is a bearer capability.** Keep it out of logs and ask again for a fresh one rather than storing it. The SDK keeps it out of its own errors: an answer it cannot read, one that is not one item per requested file or whose item carries neither a link nor an error, is an `ApiResponseError` whose `responseBody` replaces every `url` member and every link's query with `[redacted]` ([errors.md](./errors.md)). An item that does not name exactly one of `file_id` and `uri` as a non-empty string, or a `files` that is not a list, is refused before any request. The platform answers `422` `request.invalid` for more than 100 files, and a `500` `pipelex.storage.not_configured` when it cannot sign at all.
|
|
144
144
|
|
|
145
145
|
## A zip of files
|
|
146
146
|
|
|
147
|
-
The platform cannot build a zip inside a request, so a zip takes two calls. `requestFileArchive` queues the archive and answers at once, `queued`; `getFileArchive` reads it until its `state` is `ready`, whose `url` downloads the zip, or `failed`, whose `failure` says why. **An archive is never a file**: no list shows it, and it is gone a day after it was asked for, when a read of it answers `404` `not_found` as for an id never seen.
|
|
147
|
+
The platform cannot build a zip inside a request, so a zip takes two calls. `requestFileArchive` queues the archive and answers at once, `queued`; `getFileArchive` reads it until its `state` is `ready`, whose `url` downloads the zip, or `failed`, whose `failure` says why. **An archive is never a file**: no list shows it, and it is gone a day after it was asked for, when a read of it answers `404` `pipelex.storage.not_found` as for an id never seen.
|
|
148
148
|
|
|
149
149
|
The request names the files by exactly one of two members, and both are refused before any request when neither or both is given:
|
|
150
150
|
|
|
@@ -177,20 +177,20 @@ else console.warn(archive.failure?.code, archive.failure?.detail);
|
|
|
177
177
|
| `failure` | A `FileArchiveFailure`, `{ code, detail }`, on a `failed` archive; `null` otherwise. |
|
|
178
178
|
| `url`, `url_expires_at` | The link to the zip and when it lapses, on a `ready` archive; `null` otherwise. |
|
|
179
179
|
|
|
180
|
-
**A selection past a cap is a `422` `
|
|
180
|
+
**A selection past a cap is a `422` `pipelex.files.archive_too_large`**, checked before anything is written or queued: at most 500 files, or 2,000,000,000 bytes in all. Its problem's `limit` member, read from the error's `problem` and typed `FileArchiveLimit`, names the cap, `max_files` or `max_total_bytes`, its `value` and what the selection `reached`, so a client can tell the person how far over the selection is and split it. A run's count stops as soon as it passes the cap, so for a run `reached` is one more than the cap.
|
|
181
181
|
|
|
182
182
|
```ts
|
|
183
183
|
try {
|
|
184
184
|
await client.requestFileArchive({ file_ids: selected });
|
|
185
185
|
} catch (err) {
|
|
186
|
-
if (err instanceof ApiResponseError && err.code === "
|
|
187
|
-
const limit = err.
|
|
186
|
+
if (err instanceof ApiResponseError && err.code === "pipelex.files.archive_too_large") {
|
|
187
|
+
const limit = err.problem.limit as FileArchiveLimit;
|
|
188
188
|
console.warn(`${limit.name}: ${limit.reached} over ${limit.value}`);
|
|
189
189
|
}
|
|
190
190
|
}
|
|
191
191
|
```
|
|
192
192
|
|
|
193
|
-
The other refusals of a request: `422` `
|
|
193
|
+
The other refusals of a request: `422` `request.invalid` for more than 1,000 ids listed or an id longer than one can be; `404` `pipelex.storage.not_found` for a file that is not the organization's, the `detail` naming up to five of them, or a run whose files hold nothing stored to zip; `404` `run.not_found` for a run that is not the organization's; `409` `run.not_finished`, `retry` `yes`, for a run whose files are not all recorded yet, which the same request clears a few seconds after the run ends, and a `409` with `retry` `no` for a selected file whose upload has not finished; and `503` `runtime.unavailable`, `retry` `yes`, when the archive could not be queued. Each is an `ApiResponseError` whose problem's `domain` and `retry` say what to do ([errors.md](./errors.md)).
|
|
194
194
|
|
|
195
195
|
## A run's files
|
|
196
196
|
|
|
@@ -233,7 +233,7 @@ const added = await client.copyFileIntoDraft("mt_abc123", {
|
|
|
233
233
|
let draftToken = added.method.updated_at;
|
|
234
234
|
```
|
|
235
235
|
|
|
236
|
-
**With `expected_updated_at`, the write is a compare-and-swap on the draft token**, refused with `409` `
|
|
236
|
+
**With `expected_updated_at`, the write is a compare-and-swap on the draft token**, refused with `409` `pipelex.catalog.update_conflict` when the draft has moved on; without it, the file is added to the draft as it stands, never erasing someone else's save. It takes a bare catalog id, as `writeDraft` does, and refuses a `file_id` or a `path` that is not a non-empty string, or an `expected_updated_at` that is not a string, before any request. It is a write of the method, so the platform answers `403` `auth.forbidden` to a role that may not change methods, such as a runner; and `404` for an unknown method or a file that is not the organization's, `409` `pipelex.catalog.being_deleted`, `409` for a file whose upload has not finished or whose stored bytes do not match its record, `422` `method.package_invalid` for a path the draft cannot hold, such as `.env` or `../logo.png`, and `422` `method.package_invalid` past the deployment's limits. An answer whose `blob` is neither arm's, or whose `file` or `method` is not an object, is an answer the SDK cannot read.
|
|
237
237
|
|
|
238
238
|
## Uploading a test file
|
|
239
239
|
|
|
@@ -255,4 +255,4 @@ const prepared = await client.prepareInputs({
|
|
|
255
255
|
|
|
256
256
|
**The pair is checked before any file is read**, as the upload route checks it, so a request the route would refuse with a `422` is an `InputPreparationError` instead: a `kind` that is neither `uploaded` nor `test`, a `method_id` that is not a catalog id or carries a version suffix, `kind: "test"` without a `method_id`, and a `method_id` without `kind: "test"`. `null` reads as absent. `prepareInputs` names the members it refuses `upload.kind` and `upload.method_id`. `requestUploadGrant` sends its description as given, so the grant route's own `422` refuses a pair it would not take, and its refusals of a test file are the `ApiResponseError` the route answered.
|
|
257
257
|
|
|
258
|
-
**A test file the route refuses for what it claims is a `TestFileRefusedError`**, whose `
|
|
258
|
+
**A test file the route refuses for what it claims is a `TestFileRefusedError`**, whose `reason` says why: `role_not_permitted`, the `403` `auth.forbidden` of a caller whose role may not change methods, such as a runner, which an admin of the organization must change, or which the caller avoids by uploading the file as `uploaded`; `method_not_found`, the `404` `pipelex.catalog.not_found` of a method that is not one of the organization's; and `method_being_deleted`, the `409` `pipelex.catalog.being_deleted` of a method being erased. It carries the `status`, the `filename` it was sending and the `methodId` it named, and wraps the `ApiResponseError` as `cause`, whose problem it carries. These refusals are read off the platform's code, never the status alone: a `403` of any other code is still a refused credential (`UploadAuthenticationError`), and a `404` is still a deployment without upload (`UnsupportedUploadCapabilityError`) unless the upload named a method and the platform answered `pipelex.catalog.not_found`, rather than the `404` of a route it does not serve ([input-preparation.md](./input-preparation.md#error-and-capability-behavior)).
|
package/docs/hook-bundle.md
CHANGED
|
@@ -1,11 +1,25 @@
|
|
|
1
1
|
# The `.mthds` hook bundle
|
|
2
2
|
|
|
3
|
-
`dist-hooks/check.mjs` is the post-edit hook that checks a `.mthds` file after a coding agent edits it. It is built from `src/hooks/` by `npm run build:hook` and vendored by `pipelex-plugins`, which runs it after edits in Claude Code, Codex and Mistral Vibe (see [`architecture.md`](./architecture.md#hook-bundle-
|
|
3
|
+
`dist-hooks/check.mjs` is the post-edit hook that checks a `.mthds` file after a coding agent edits it. It is built from `src/hooks/` by `npm run build:hook` and vendored by `pipelex-plugins`, which runs it after edits in Claude Code, Codex and Mistral Vibe (see [`architecture.md`](./architecture.md#hook-bundle-dist-hookscheckmjs) for how it is built and distributed). This page covers which file the hook checks on each platform, and in particular how it finds the file a Codex patch wrote.
|
|
4
4
|
|
|
5
5
|
## What the hook does with a file
|
|
6
6
|
|
|
7
7
|
Each file goes through three stages: a local lint that blocks on any diagnostic, a local format that rewrites the file in place when its layout changes, and a validation by the hosted API (`POST /v1/validate`) that blocks on an invalid verdict and adds a non-blocking note about pending pipe signatures. The hook fails open: when the engine cannot load, when `PIPELEX_API_KEY` is unset or the API cannot be reached, the stage that could not run passes silently and the others stand. When several files are checked, one block among them wins and the notes are joined, and each file is read when its turn comes, so an edit made to it while an earlier file was being validated is checked rather than overwritten. The hook always exits 0, since its verdict is on stdout.
|
|
8
8
|
|
|
9
|
+
An invalid verdict blocks with the server's Markdown rendering of it, which the hook always asks for, forwarded verbatim. When the server sends none, the hook words the block itself: the file that failed, the verdict's message, then one line for each fault the verdict lists in `errors[]`, each an MTHDS problem item named by its code, its sentence and where it happened, with its suggested fix, when it carries one, on a line of its own. The place and the fix are worded by the `mthds` package's `formatLocation` and `formatSuggestedFix`, as every MTHDS client words them, and a fix of a safety the hook does not know is shown as unsafe:
|
|
10
|
+
|
|
11
|
+
```text
|
|
12
|
+
Validation failed for /work/methods/greeting/greeting.mthds:
|
|
13
|
+
|
|
14
|
+
The method does not validate.
|
|
15
|
+
|
|
16
|
+
- `method.input_undeclared`: Pipe 'greet_visitor' reads names its inputs do not declare: visitor. (pipe 'greet_visitor', file 'greeting.mthds')
|
|
17
|
+
- `method.model_unknown`: The llm model 'gpt-5.1' is not one the runtime knows. (pipe 'greet_visitor')
|
|
18
|
+
- Suggested fix (safe, rename-model): Use the model 'gpt-5' in place of 'gpt-5.1'.
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
The hook reads each item member by member, with the SDK's `readProblemItems`, as every verdict of the SDK is read ([`errors.md`](./errors.md#a-verdicts-problem-items)): an entry with no string `code` or `detail` names no fault and is skipped, and a `location` or a `suggested_fix` the standard's own check refuses is left out of its item, so a malformed member never costs the agent the fault it belongs to. The verdict, never its wording, decides the block: an invalid verdict whose faults cannot be read still blocks, with its message. The older `validation_errors[]` list is not read.
|
|
22
|
+
|
|
9
23
|
The validation sends the method's tree around the file, the tree a run of the method would send: the edited file's directory listed by the `mthds` package's lister under the MTHDS standard's membership rule, so it is rooted at the nearest `METHODS.toml` at or above that directory, else at the directory itself, and holds every file the rule keeps — every bundle of the method, and every file a step reads, such as the template a `PipeDocGen` step names in `template_file`, under its tree path — while hidden entries, `node_modules`, `__pycache__`, what a `.gitignore` inside the tree keeps out and a run's inputs file never travel ([`method-trees.md`](./method-trees.md#the-four-forms-of-a-tree)), and neither does a file that looks like a secret ([below](#secrets-stay-home)). A method that fills a template thus validates as it would load from disk, its template checked by the dry run, and a step whose template file the tree does not hold makes the verdict say so, which blocks until the file is written. The client sends the tree inline when small and by hash otherwise, uploading the files the API lacks. The whole validation runs under one ten-second deadline, the version handshake, the blob route, the uploads and the validation request together, and the deadline stops whichever of them is under way when it runs out. Validation passes silently, as when the API cannot be reached, when that deadline runs out, when the tree holds a symbolic link or a directory the lister cannot read, when the edited file is not in its own tree because the rule keeps it out, when a file of the tree changes between its listing and its reading, when a file of the tree fails to upload, and past the hook's own caps: 100 files, 5 MiB for one of them or 12 MiB in all, each checked while the tree is listed, before any file is read.
|
|
10
24
|
|
|
11
25
|
## Secrets stay home
|
|
@@ -68,7 +68,7 @@ The file is described, never sent. Beside the description, the request takes `ki
|
|
|
68
68
|
- **`stored: true`**: the organization already holds these bytes. Nothing is to be sent: `uri` is the file's `pipelex-storage://` reference now, `file_id` its record, and `url`, `headers` and `expires_at` are `null`. The same person asking again for the same bytes under the same name and kind gets their existing record back.
|
|
69
69
|
- **`stored: false`**: a presigned, create-only `PUT` for the object. `url` is where to send it, `headers` are the signed headers to send unchanged, `uri` is the reference the object will carry, `file_id` its record, `pending` until the bytes land, and `expires_at` is when storage stops accepting the `PUT`, a few minutes later.
|
|
70
70
|
|
|
71
|
-
With a `sha256`, the object is named after the bytes, `<org>/assets/<sha256>.<ext>`, and the grant signs the hash as `x-amz-checksum-sha256`, so storage holds the body to exactly those bytes. Without one, the grant always names a new object under a fresh name and answers `stored: false`, as every grant did before hashes, so a client that cannot hash still uploads; a client that can hash always should. The platform types the object by the name's extension and never stores the declared `content_type`, which only serves a name with no extension. The wire is snake_case like every product route, so the description can travel to the server and the grant back to the page as JSON without a remap. A declared `size` over the limit is a `413` `ApiResponseError` whose `code` is `
|
|
71
|
+
With a `sha256`, the object is named after the bytes, `<org>/assets/<sha256>.<ext>`, and the grant signs the hash as `x-amz-checksum-sha256`, so storage holds the body to exactly those bytes. Without one, the grant always names a new object under a fresh name and answers `stored: false`, as every grant did before hashes, so a client that cannot hash still uploads; a client that can hash always should. The platform types the object by the name's extension and never stores the declared `content_type`, which only serves a name with no extension. The wire is snake_case like every product route, so the description can travel to the server and the grant back to the page as JSON without a remap. A declared `size` over the limit is a `413` `ApiResponseError` whose `code` is `request.too_large`; a `sha256` that is not 64 lowercase hexadecimal digits is a `422` `request.invalid`, and so is a `size` other than the one the organization holds those bytes at, which means the hash was computed over other bytes than the ones described; a deployment without the route answers a `404`; and a `2xx` whose body is not a JSON object is an `ApiResponseError` the SDK cannot read, `client.response_unreadable`, rather than a grant.
|
|
72
72
|
|
|
73
73
|
**A grant with a target is a bearer capability.** Until it expires, whoever holds it can write that one object, so the SDK never logs it, and its caller should not either. The route never replays a grant for a repeated request: a caller that lost one asks for a new one, which, with a `sha256`, answers the same file record.
|
|
74
74
|
|
|
@@ -92,7 +92,7 @@ A page and the server that holds the key split the work this way: the page calls
|
|
|
92
92
|
|
|
93
93
|
**Storage's refusals** map onto the input-preparation errors, so a caller that already handles `uploadFile` needs no new case:
|
|
94
94
|
|
|
95
|
-
| Storage answers | Error | `
|
|
95
|
+
| Storage answers | Error | `reason` | Meaning |
|
|
96
96
|
| --- | --- | --- | --- |
|
|
97
97
|
| nothing sent, the grant answered `stored: true` | none | | Stored already; `uri` names the file. |
|
|
98
98
|
| `2xx` | none | | Stored; `uri` names the object. |
|
|
@@ -104,16 +104,16 @@ A page and the server that holds the key split the work this way: the page calls
|
|
|
104
104
|
| `400 BadDigest` | `RejectedAssetError` | `store_refused` | The body's bytes differ from the ones whose SHA-256 the grant was asked with. Send the file the grant was requested for, or describe the file again and ask for a new grant. |
|
|
105
105
|
| `400 RequestTimeout` | `UploadTransportError` | `storage_timeout` | Storage stopped waiting for the body and wrote nothing, so the same grant can be retried while it is valid. |
|
|
106
106
|
| `409 ConditionalRequestConflict` on every attempt | `UploadTransportError` | `conflict` | Another write of the same object was still in progress after every pause. Whether the object was written is unknown, so the same-grant retry applies. A single `409` followed by any other answer is read as that answer. |
|
|
107
|
-
| `408`, `429` | `UploadTransportError` | `unexpected` | Storage timed out or throttled the request rather than refusing the file, so
|
|
107
|
+
| `408`, `429` | `UploadTransportError` | `unexpected` | Storage timed out or throttled the request rather than refusing the file, so its problem is `runtime.unavailable`, `retry` `yes`: a later attempt can succeed, and the same-grant retry applies. |
|
|
108
108
|
| any other `4xx` | `RejectedAssetError` | `store_refused` | Storage's own message is relayed. |
|
|
109
|
-
| `501` | `UploadTransportError` | `server_error` | Storage does not implement the request it was sent, so it wrote nothing, and a retry meets the same answer:
|
|
109
|
+
| `501` | `UploadTransportError` | `server_error` | Storage does not implement the request it was sent, so it wrote nothing, and a retry meets the same answer: its problem is `runtime.unsupported`, `retry` `no`. |
|
|
110
110
|
| any other `5xx` | `UploadTransportError` | `server_error` | Whether the object was written is unknown: retrying with the same grant either stores it or finds it stored, which for a grant asked for without a hash is the `412` of a used grant. |
|
|
111
111
|
| a redirect | `UploadTransportError` | `redirected` | Refused, not followed. |
|
|
112
112
|
| no answer within the time limit | `UploadTransportError` | `timeout` | The limit above ran out before storage answered. Whether the object was written is unknown, so the same-grant retry applies. |
|
|
113
113
|
| no response | `UploadTransportError` | `unreachable` | Storage unreachable. In a browser a refused cross-origin request looks exactly like this, so the message also points at the page's CSP `connect-src`. If the connection failed after the file went out, the object may have been written, so the message gives the same-grant retry too. |
|
|
114
114
|
| nothing sent | `UploadTransportError` | `invalid_grant_url` | The grant is not stored and its `url` is not an absolute `http(s)` URL free of user info, so it is refused before any request. A browser would resolve a relative URL against the page and send the credential there. |
|
|
115
115
|
|
|
116
|
-
`RejectedAssetError.
|
|
116
|
+
`RejectedAssetError.reason` and `UploadTransportError.reason` say what the SDK saw, beside the problem's `code` — to request a new grant on `grant_used` or `grant_expired`, to retry with the same grant on `timeout`, or on `server_error` or `unexpected` when the problem's `retry` is `yes` (a `501` is not), or to tell a timeout from an unreachable host — rather than the message, the status or the error's `name`, since an expired grant and an unsigned header both answer `403 AccessDenied` and only S3's message tells them apart. The SDK sets `reason` on every error it raises. `RejectedAssetError.status` is storage's, and its `filename` is the `File`'s name, or the object name the grant's `uri` ends in for a nameless `Blob`. `UploadTransportError.status` is storage's status when storage answered, and undefined when it did not. Neither error wraps storage's response, and only S3's error code and message reach a message: its error body can echo the signed request, credential included, so it is never kept, and only its first 16 KiB are read, which is where S3 writes its code and message. **No error carries the grant's URL.** The runtime's network error can: Node and Chrome name the whole URL when they cannot build a request from it, and Bun puts it on the error's `path`. So an unreachable storage is described by its origin and by the error names and codes along the cause chain (`TypeError, caused by Error ENOTFOUND`), and the runtime's error is neither relayed nor kept as `cause`. A caller's abort propagates as the signal's own reason, unwrapped, including one that lands while storage's error body is still arriving.
|
|
117
117
|
|
|
118
118
|
### `prepareInputs` — signature-driven input preparation
|
|
119
119
|
|
|
@@ -158,7 +158,7 @@ The route resolves the closure, **selects the pipe**, and answers with that pipe
|
|
|
158
158
|
|
|
159
159
|
A `directory`, `tree` or `contents` too large to send inline is uploaded first, by hash, and a blob that fails to upload throws `MethodFileTransferError` before the route is asked. A `method_ref` makes the server clone a repository first, so the call gets the crate routes' fetch-sized budget. On the hosted API the gateway caps a request at about 30 seconds whatever the client allows, so a cold clone can answer a `502` `ApiResponseError`; a retry clears it once the runner has cached the clone.
|
|
160
160
|
|
|
161
|
-
`prepareInputs` needs an API that serves `POST /v1/pipe-io`. Against one that does not, the route's `404` (a runner) or `403` (a hosted origin whose gateway does not list it yet) propagates as an `ApiResponseError`. It also needs the route's pipe-selection refusals
|
|
161
|
+
`prepareInputs` needs an API that serves `POST /v1/pipe-io`. Against one that does not, the route's `404` (a runner) or `403` (a hosted origin whose gateway does not list it yet) propagates as an `ApiResponseError`. It also needs the route's pipe-selection refusals coded, as step 3 of "Pipe selection" below describes, which `pipelex-api` does from v0.90.0: against an older runner a refused selection reaches the caller as that `ApiResponseError` rather than an `InputPreparationError`.
|
|
162
162
|
|
|
163
163
|
#### Pipe selection
|
|
164
164
|
|
|
@@ -166,7 +166,7 @@ The route selects the pipe, and the ref it answers with is the one prepared —
|
|
|
166
166
|
|
|
167
167
|
1. **`pipe_ref` when given.** Qualified-only: `domain.pipe_code`, normalized (trimmed, and blank is absent) and checked **before any request**. A bare code is refused with an `InputPreparationError` asking for the qualified form, since a request names a pipe by its qualified ref and search is a run-route affordance this helper never grows. An `alias->domain.pipe_code` ref is refused for a different reason: the alias names a dependency package's pipe, and preparation covers the method's *own* pipes, whose input form the pipe I/O route describes. The run route takes such a ref; preparation cannot.
|
|
168
168
|
2. **Otherwise the method's own entry pipe**, chosen by the route: the `main_pipe` of the `METHODS.toml` at the tree's root, whether the tree was sent or fetched with a package, else the closure's `main_pipe` when exactly one domain declares one. A method that names its entry pipe in `METHODS.toml` alone — the published `github.com/Pipelex/methods/documents` does — therefore prepares with no `pipe_ref`.
|
|
169
|
-
3. **A selection the route refuses** — a `pipe_ref` that names no pipe of the method, a method with no entry pipe, one whose domains declare several, a manifest `main_pipe` the closure does not declare or declares in several domains — comes back as a `422` whose
|
|
169
|
+
3. **A selection the route refuses** — a `pipe_ref` that names no pipe of the method, a method with no entry pipe, one whose domains declare several, a manifest `main_pipe` the closure does not declare or declares in several domains — comes back as a `422` whose code is `request.pipe_not_found`, `request.pipe_unspecified` or `request.pipe_ambiguous`. The helper turns exactly those into an `InputPreparationError` carrying the server's problem and its detail, which names the candidates where there are some, with the `ApiResponseError` as its `cause`. The run route refuses a selector-less run in the same cases, so preparation never succeeds for a pipe the run would not take.
|
|
170
170
|
|
|
171
171
|
There is no single-pipe fallback: a method that declares one pipe and no `main_pipe` is refused a selector-less run, and a selector-less preparation with it.
|
|
172
172
|
|
|
@@ -256,20 +256,20 @@ Upload is a **hosted Pipelex-product capability**, even though the SDK can be po
|
|
|
256
256
|
|
|
257
257
|
The contract distinguishes these semantic outcomes, each a typed subclass of `InputPreparationError` (catch the base to handle any preparation failure, a subclass to branch on category):
|
|
258
258
|
|
|
259
|
-
- **an unresolvable signature** (`InputPreparationError`) — a bare or alias-qualified `pipe_ref`, refused before any request; a closure that does not load (`MethodLoadError`, below); or a pipe selection the route refused (its `422`
|
|
260
|
-
- **a method that does not load** (`MethodLoadError`) — the pipe I/O answer's `is_valid: false` arm. Its
|
|
259
|
+
- **an unresolvable signature** (`InputPreparationError`) — a bare or alias-qualified `pipe_ref`, refused before any request; a closure that does not load (`MethodLoadError`, below); or a pipe selection the route refused (its `422` `request.pipe_not_found`, `request.pipe_unspecified` or `request.pipe_ambiguous`, carrying the server's problem). Any other failure from `/v1/pipe-io` — a malformed selector, an over-limit file, an unknown or foreign-org `method_id`, no package at the address or one with no `.mthds` file, auth, a server fault, a deployment that does not serve the route — stays an `ApiResponseError` and propagates unchanged;
|
|
260
|
+
- **a method that does not load** (`MethodLoadError`) — the pipe I/O answer's `is_valid: false` arm. Its problem is `method.invalid`, whose `errors` are the answer's problem items, each read member by member and kept when it carries a string `code` and `detail`, and its message names the first item's `detail`, else the answer's own `message`; `serverMessage` is the answer's own `message` when it is a string, so a consumer lists every item rather than reading the first from a sentence. The method must change, not the inputs;
|
|
261
261
|
- **invalid local source** (`InvalidLocalSourceError`) — missing, unreadable, or a path string outside Node;
|
|
262
262
|
- **invalid input value** (`InvalidInputValueError`, carries `inputName` and `inputPath`) — a value at a file input that cannot be turned into a file to upload: a `data:` URL that does not decode (no comma, bad base64, bad percent-encoding), or a value of a type no file input takes, such as a number or an object that is neither bytes nor `{url}` content. It is raised while the inputs are read, before anything is uploaded. It names the input and the path inside it as `UploadNotAllowedError` does (`Input "dossier.attachments[1]" holds a number, which no file input takes: …`), describes the value by its kind and never quotes it, and lists the forms a file input takes in the mode it ran in. With `InvalidLocalSourceError`, it is how a consumer tells the inputs' own mistakes from a method that does not load, though all three are `InputPreparationError`s with the same verdict;
|
|
263
263
|
- **upload not allowed** (`UploadNotAllowedError`, carries `inputName`, `inputPath` and `valueKind`) — under `allowUploads: false`, a value at a file input that would need an upload: a local path, a `data:` URL or bytes. It is raised before anything is read, decoded or uploaded (see "[Preparing without uploads](#preparing-without-uploads)");
|
|
264
|
-
- **rejected asset** (`RejectedAssetError`) — the server refused it (e.g. a `413` past the service-defined size cap — see "Storage policy" — surfaced as a clear rejection with `
|
|
265
|
-
- **refused test file** (`TestFileRefusedError`, carries `
|
|
266
|
-
- **unsupported server capability** (`UnsupportedUploadCapabilityError`) — the configured deployment has no upload route: any other `404`, the platform's
|
|
264
|
+
- **rejected asset** (`RejectedAssetError`) — the server refused it (e.g. a `413` past the service-defined size cap — see "Storage policy" — surfaced as a clear rejection with `reason` `too_large`, not a raw transport error), or storage refused an upload with a grant (a used or expired grant, or a file that differs from what the grant signed — see the table above for each `reason`);
|
|
265
|
+
- **refused test file** (`TestFileRefusedError`, carries `reason`, `status`, `filename` and `methodId`) — the upload route refused a test file for what it claims, never for its bytes: `role_not_permitted`, the `403` `auth.forbidden` of a caller whose role may not change methods; `method_not_found`, the `404` `pipelex.catalog.not_found` of a method that is not the organization's; `method_being_deleted`, the `409` `pipelex.catalog.being_deleted` of a method being erased. Each is read off the platform's code, never the status alone, and the error carries the platform's problem;
|
|
266
|
+
- **unsupported server capability** (`UnsupportedUploadCapabilityError`) — the configured deployment has no upload route: any other `404`, the platform's own for a route it does not serve or a bare runner's, which is no problem document;
|
|
267
267
|
- **authentication / authorization failure** (`UploadAuthenticationError`) — a `401`, or a `403` other than a test file's `role_not_permitted`;
|
|
268
|
-
- **transport failure** (`UploadTransportError`) — a network or server fault, or any other unexpected upload failure; its `status` is the HTTP status when a response produced it, and its `
|
|
268
|
+
- **transport failure** (`UploadTransportError`) — a network or server fault, or any other unexpected upload failure; its `status` is the HTTP status when a response produced it, and its `reason` says which failure it was. From `uploadFile` the reason is `server_error` for a `5xx`, `timeout` when the client's own request timeout ran out, `unreachable` when no response came back, and `unexpected` for any other status or failure, a `2xx` answer the SDK cannot read included, such as one with no string `uri`; `uploadWithGrant`'s reasons are in the table above.
|
|
269
269
|
|
|
270
270
|
**Which file.** Every upload failure `uploadFile` raises names the file it was sending, as it sent it: `RejectedAssetError`, `TestFileRefusedError`, `UploadAuthenticationError`, `UnsupportedUploadCapabilityError` and `UploadTransportError` each carry `filename`, so a consumer that prepared several files says which one failed without parsing a message. `uploadWithGrant` sends a grant's file, not a named one, and leaves `filename` undefined on the `UploadTransportError` it raises.
|
|
271
271
|
|
|
272
|
-
Each of these carries
|
|
272
|
+
Each of these carries its problem, as every SDK error does ([errors.md](./errors.md#each-classs-problem)): the family's own refusals are `client.argument_invalid`, `input.file_unreadable` and `input.invalid`, an unsupported deployment is `runtime.unsupported`, a refused credential `auth.invalid_credential` or `auth.missing_credential`, a refused test file carries the platform's problem, and a transport failure carries the problem of the error it wraps, so a `402` plan refusal wrapped by `uploadFile` keeps the platform's `pipelex.billing.*` code, or, wrapping none, the problem its `reason` gives (`timeout`, `storage_timeout` and `conflict` are `runtime.unavailable`, `server_error` reads storage's status, a `501` being `runtime.unsupported`). A pipe I/O answer whose `is_valid` is not a boolean, whose `input_form` does not describe the pipe it selected, or whose descriptor for that pipe is malformed where the walk reads it or names a kind this SDK does not know, is a plain `InputPreparationError` whose problem is `client.response_unreadable`, since the route broke its own contract. The descriptor is checked whole, once, before any caller value is read or anything uploaded, so the refusal does not depend on which inputs the caller supplied, and its message names the pipe, where in the descriptor the fault sits (`fields[3].item`) and what was found there.
|
|
273
273
|
|
|
274
274
|
All preparation failures are raised **before any run is created**.
|
|
275
275
|
|
package/docs/method-copy.md
CHANGED
|
@@ -30,13 +30,13 @@ Both take a bare catalog id, as every method route does, and refuse a suffixed o
|
|
|
30
30
|
|
|
31
31
|
**The copy's body is closed.** The client sends `destination_org_id` and, when given, `name` and `expected_draft_updated_at`, and nothing else; a `null` reads as not given. `destination_org_id` is the canonical id of the destination, as a membership's `org_id` gives it; `name` is the copy's name, the original's when it is left out; `expected_draft_updated_at` is the original's draft token the caller last saw (`MethodData.updated_at`). A `destination_org_id` that is not a non-empty string, and a `name` or a token given that is not a string, are refused with a `RequestArgumentError` before any request.
|
|
32
32
|
|
|
33
|
-
**Only a web session may copy.** A Pipelex API key belongs to one organization and cannot speak for another, so the platform refuses a copy sent with one `403` `forbidden`, whatever its role. The listing is a read, open to every member of the method's organization and to every credential.
|
|
33
|
+
**Only a web session may copy.** A Pipelex API key belongs to one organization and cannot speak for another, so the platform refuses a copy sent with one `403` `auth.forbidden`, whatever its role. The listing is a read, open to every member of the method's organization and to every credential.
|
|
34
34
|
|
|
35
35
|
## What a copy holds
|
|
36
36
|
|
|
37
37
|
| Part of the method | In the copy |
|
|
38
38
|
|---|---|
|
|
39
|
-
| The draft's files | The same tree, held in the destination, so the copy's `draft_digest` equals the original's |
|
|
39
|
+
| The draft's files | The same tree, held in the destination, so the copy's `draft_digest` equals the original's; a draft holding no files copies to an empty draft, its `draft_digest` `null` |
|
|
40
40
|
| `name` | The original's, unless the request names another |
|
|
41
41
|
| `description` | The original's, which comes from the same tree |
|
|
42
42
|
| `input_data` | The original's form inputs, every stored file they name in the original's organization copied into the destination |
|
|
@@ -44,46 +44,48 @@ Both take a bare catalog id, as every method route does, and refuse a suffixed o
|
|
|
44
44
|
| Published versions | None: the copy starts unpublished, its `latest_version` and `latest_published` `null`, and its own publish makes version 1 |
|
|
45
45
|
| Runs | None: a run stays in the organization that ran it and was billed for it |
|
|
46
46
|
| `method_id`, `org_id`, `created_by_user_id`, `created_at` | A new id, the destination, the copier and the moment of the copy |
|
|
47
|
-
| `copied_from` | `{ org_id, method_id, draft_digest }` of the original, as it stood at the copy |
|
|
47
|
+
| `copied_from` | `{ org_id, method_id, draft_digest }` of the original, as it stood at the copy, its `draft_digest` `null` when the original's draft held no files |
|
|
48
48
|
|
|
49
|
-
**The copy takes the draft, never the latest published version.** The draft is what the person sees in the editor, and the copy goes through its own first publish anyway; an invalid draft is copied as it is, and the copy's publish refuses it as it would any draft. With `expected_draft_updated_at`, the copy takes exactly the draft the caller last saw: a draft that moved since is a `409` `
|
|
49
|
+
**The copy takes the draft, never the latest published version.** The draft is what the person sees in the editor, and the copy goes through its own first publish anyway; an invalid draft is copied as it is, and the copy's publish refuses it as it would any draft. With `expected_draft_updated_at`, the copy takes exactly the draft the caller last saw: a draft that moved since is a `409` `pipelex.catalog.update_conflict`, and nothing is copied. A client that has just saved the editor's pending edits sends the token that save answered.
|
|
50
50
|
|
|
51
|
-
**A stored file named by an input is copied.** A `pipelex-storage://` address resolves only inside the organization it names, so every such address in the original's organization, in the form inputs and in each example's inputs, is copied into the destination's `assets/` under a new address, and the value is rewritten to it; a file named twice is copied once. A duplicate keeps
|
|
51
|
+
**A stored file named by an input is copied.** A `pipelex-storage://` address resolves only inside the organization it names, so every such address in the original's organization, in the form inputs and in each example's inputs, is copied into the destination's `assets/` under a new address, and the value is rewritten to it; a file named twice is copied once. A duplicate keeps every address of the organization as it is, an upload's above all, since the organization is the same, and copies only the files the original method's own runs stored, under `pipelex-storage://<org_id>/<method_id>/…`, which erasing the original would sweep away. An address of another organization, a malformed one and one whose file is gone are kept unchanged. The destination records each copied file among its files, attributed to the copier. `stored_files_copied` counts the distinct files copied.
|
|
52
52
|
|
|
53
53
|
## The verdict
|
|
54
54
|
|
|
55
|
-
The answer is a `200` discriminated on `outcome`, as `publishMethod`'s is, because a bundle the
|
|
55
|
+
The answer is a `200` discriminated on `outcome`, as `publishMethod`'s is, because a bundle the copy cannot rely on is a verdict about the method, not a failure of the request. Branch on `outcome`, never on the status:
|
|
56
56
|
|
|
57
57
|
| `outcome` | Members | When |
|
|
58
58
|
|---|---|---|
|
|
59
59
|
| `copied` | `method`, the copy, a `MethodData` with its `copied_from`; `examples_copied`; `stored_files_copied` | The copy exists in the destination. |
|
|
60
|
-
| `refused` | `reason`, `bundle_names_other_storage`; `message`, a sentence for a person; `references`, every `{ path, uri }` the destination cannot read, ordered by path and then by reference | A `.mthds` file of the draft names a stored file outside the destination, which every run there would refuse. Nothing was written. |
|
|
60
|
+
| `refused` | `reason`, `bundle_names_other_storage`; `message`, a sentence for a person; `references`, every `{ path, uri }` the destination cannot read and, in a duplicate, every one under the original's run files, ordered by path and then by reference | A `.mthds` file of the draft names a stored file outside the destination, which every run there would refuse, or, in a duplicate, a file under the original method's own run files, which erasing the original would remove. Nothing was written. |
|
|
61
61
|
|
|
62
|
-
A
|
|
62
|
+
**A duplicate is held to one more rule.** Every reference to the original's organization is one the destination can read, the original's storage being the destination's, but a reference under the original method's own run files, `pipelex-storage://<org_id>/<method_id>/…`, names a file one of the original's runs stored, and erasing the original sweeps those away: every run of the duplicate would then fail to fetch it. So such a reference is refused too, with the same verdict and the same reason, beside any the destination cannot read. It is the rule the copy's inputs follow: an input naming such a file is re-homed, while a bundle naming one is refused, since the platform rewrites a copy's inputs but never its bundles. Every other reference of the organization, an upload under its `assets/` or another method's run file, passes in a duplicate as it does for the original. A copy into another organization needs no such rule, since it already refuses every reference to the original's organization. The `message` says why for each kind of reference.
|
|
63
63
|
|
|
64
|
-
|
|
64
|
+
A refused copy is fixed in the original: the person removes or replaces each reference the verdict names, then copies again. The platform does not rewrite the bundle, since that would change the author's code and the tree's digest without asking them.
|
|
65
|
+
|
|
66
|
+
The client checks the member a caller branches on before it hands the answer back: an `outcome` that is neither arm's, a `copied` answer whose `method` is not an object, and a `refused` one whose `references` is not an array are each an answer the SDK cannot read, thrown as an `ApiResponseError` whose problem is `client.response_unreadable`.
|
|
65
67
|
|
|
66
68
|
## What is refused, and where
|
|
67
69
|
|
|
68
|
-
A non-2xx answer means no verdict could be produced, and none of them leaves anything a reader sees. Each is an `ApiResponseError
|
|
70
|
+
A non-2xx answer means no verdict could be produced, and none of them leaves anything a reader sees, but for the `503` whose `detail` names the copy (below). Each is an `ApiResponseError` whose `problem` is the platform's problem document; branch on its `code` and its `retry`, never on the status alone.
|
|
69
71
|
|
|
70
72
|
| Status | `code` | When |
|
|
71
73
|
|---|---|---|
|
|
72
|
-
| `403` | `forbidden` | The request is not from a web session. |
|
|
73
|
-
| `
|
|
74
|
-
| `404` | `
|
|
75
|
-
| `
|
|
76
|
-
| `409` | `
|
|
77
|
-
| `409` |
|
|
78
|
-
| `
|
|
79
|
-
| `
|
|
80
|
-
| `422` | `
|
|
81
|
-
| `
|
|
82
|
-
| `
|
|
83
|
-
| `
|
|
84
|
-
| `503` | `
|
|
85
|
-
|
|
86
|
-
`MethodErrorCode` names `
|
|
74
|
+
| `403` | `auth.forbidden` | The request is not from a web session, or the caller's role changes no method, `runner` or a role the platform does not know, in the method's organization or in the destination: the role must change. |
|
|
75
|
+
| `404` | `pipelex.org.not_found` | The destination is one the caller does not belong to, one whose invitation they have not accepted, one still being set up, or one that does not exist. The platform answers them alike, so it never says whether an organization exists. |
|
|
76
|
+
| `404` | `pipelex.catalog.not_found` | An unknown method. The destination is judged first, so an unknown method in a destination the caller may not write to reads `pipelex.org.not_found`. |
|
|
77
|
+
| `409` | `pipelex.catalog.being_deleted` | The original's erasure has started. |
|
|
78
|
+
| `409` | `pipelex.catalog.update_conflict` | The draft moved since `expected_draft_updated_at`; `location.field` names that member. Read the method again and copy with the fresh token. |
|
|
79
|
+
| `409` | | The method's organization holds no record of the draft's tree, or no longer holds every file of it: its `domain` is `runtime` and its `retry` `no`, a stored record contradicting itself that neither the caller nor a retry can mend. No registered code names this refusal yet; it is not the `request.idempotency_conflict` of an idempotency key reused with other arguments (below). |
|
|
80
|
+
| `413` | `request.too_large` | A copied example, or the copy's row, does not fit in one stored item. The row keeps room for the summary a publish adds, so a `name` that would leave none is refused. |
|
|
81
|
+
| `422` | `request.invalid` | A body that does not validate, such as a name holding a lone surrogate. |
|
|
82
|
+
| `422` | `pipelex.catalog.copy_too_large` | The copy passes one of its bounds on the stored files it copies. Its problem's `limit` member, typed `MethodCopyLimit`, names the bound, `max_stored_files` or `max_stored_bytes`, its `value` and what the copy `reached`. Removing examples, or the files they hold, brings it under. |
|
|
83
|
+
| `502` | `runtime.unavailable` | The platform could not read the caller's memberships. Its `retry` is `yes`. |
|
|
84
|
+
| `500` | `pipelex.storage.not_configured` | The platform has no usable credentials for storage. Its `retry` is `no`. |
|
|
85
|
+
| `503` | `pipelex.storage.unavailable` | Storage failed a read, a copy or a write. Its `retry` is `yes`. |
|
|
86
|
+
| `503` | `runtime.unavailable` | The copy did not finish inside its deadline, a stored file it copied was written over meanwhile, or DynamoDB failed a read or a write. Its `retry` is `yes`. When DynamoDB failed both the write of the copy's row and the read that would confirm it, the `detail` names the copy's method id and says that a retry may make a second copy: the copy may stand in the destination, without the original's marker of it and without the destination's records of its files, so look for that id in the destination before copying again. |
|
|
87
|
+
|
|
88
|
+
`MethodErrorCode` names `pipelex.org.not_found` and `pipelex.catalog.copy_too_large` beside the other method codes ([errors.md](./errors.md#the-method-codes)).
|
|
87
89
|
|
|
88
90
|
```ts
|
|
89
91
|
import { ApiResponseError } from "@pipelex/sdk";
|
|
@@ -92,8 +94,8 @@ import type { MethodCopyLimit } from "@pipelex/sdk";
|
|
|
92
94
|
try {
|
|
93
95
|
await client.copyMethod("mt_abc123", { destination_org_id: "org_team" });
|
|
94
96
|
} catch (err) {
|
|
95
|
-
if (err instanceof ApiResponseError && err.code === "
|
|
96
|
-
const limit = err.
|
|
97
|
+
if (err instanceof ApiResponseError && err.code === "pipelex.catalog.copy_too_large") {
|
|
98
|
+
const limit = err.problem.limit as MethodCopyLimit;
|
|
97
99
|
console.error(`${limit.name}: ${limit.reached} of at most ${limit.value}`);
|
|
98
100
|
} else throw err;
|
|
99
101
|
}
|
|
@@ -107,11 +109,11 @@ try {
|
|
|
107
109
|
|---|---|
|
|
108
110
|
| The first under the key | The copy is made, or refused, and its answer is stored under the key for 24 hours, scoped to the caller and the method's organization, unless it is a `5xx` or a refusal for its timing (`408`, `425` or `429`). |
|
|
109
111
|
| The same arguments under the same key, once the first was answered with a verdict, or with a `4xx` refusal other than one for its timing | The stored answer, and nothing more is copied: a copy whose answer was lost, as on a client timeout, makes one copy, not two, and a copy refused for its content is refused again. |
|
|
110
|
-
| The same arguments under the same key, after a `5xx` or a refusal for its timing, such as a `429` | A fresh attempt: neither is stored, the key was freed, and the failed copy left nothing visible. |
|
|
111
|
-
| The same arguments under the same key, while the first is still being handled, or while its stored answer cannot be read | A `409` `idempotency_key_in_use`, `
|
|
112
|
-
| Other arguments under the same key | A `409` `
|
|
112
|
+
| The same arguments under the same key, after a `5xx` or a refusal for its timing, such as a `429` | A fresh attempt: neither is stored, the key was freed, and the failed copy left nothing visible, but for the `503` whose `detail` names the copy, which may stand. |
|
|
113
|
+
| The same arguments under the same key, while the first is still being handled, or while its stored answer cannot be read | A `409` `request.idempotency_key_in_use`, `retry` `yes`, answered by the key and never stored: send it again under the same key after its `retryAfter`. |
|
|
114
|
+
| Other arguments under the same key | A `409` `request.idempotency_conflict`, `retry` `no`. A changed copy, such as one with a fresh draft token, goes under a new key. |
|
|
113
115
|
|
|
114
|
-
Without a key, a copy whose answer was lost cannot be told from one that never ran, and sending it again may make a second copy. Even with one, a platform process
|
|
116
|
+
Without a key, a copy whose answer was lost cannot be told from one that never ran, and sending it again may make a second copy. Even with one, two rare windows can make a second copy on a retry. A platform process can die after writing the copy and before storing its answer, a window the creation of a method shares. And the `503` whose `detail` names the copy released the key while the copy may stand, which is why it names the copy: look for that method id in the destination before copying again.
|
|
115
117
|
|
|
116
118
|
## The copies of a method
|
|
117
119
|
|
|
@@ -119,8 +121,8 @@ The original keeps a marker of each copy made of it, and `listMethodCopies` page
|
|
|
119
121
|
|
|
120
122
|
Pass the page's `nextCursor` back as `cursor` to continue; `null` is the last page. A method never copied lists an empty page, as does a copy, which names its original through `copied_from` and has no copies of its own. A marker stays when its copy is erased, and the copy then reads as not found. The marker is written once the copy exists, and a failure to write it does not fail the copy, so a copy can, rarely, be missing from the list. The original's erasure and its discard delete its markers with it.
|
|
121
123
|
|
|
122
|
-
The listing answers `404` `not_found` for an unknown method, `409` `
|
|
124
|
+
The listing answers `404` `pipelex.catalog.not_found` for an unknown method, `409` `pipelex.catalog.being_deleted` while its erasure runs, and `422` `request.invalid` for a cursor it did not issue for that method or a `limit` outside 1 to 100. A page whose `items` is not an array or whose `next_cursor` is neither a string nor `null` is an answer the SDK cannot read.
|
|
123
125
|
|
|
124
126
|
## `copied_from` on a method
|
|
125
127
|
|
|
126
|
-
Every read of a method, `getMethod` and the `method` of every write, carries `copied_from`: `{ org_id, method_id, draft_digest }` of the original on a copy, `null` on a method created afresh, and absent from a platform that predates copies. It is written once, when the copy is made, and never rewritten, so it names the draft the copy started from even after the original has changed or been erased. It holds opaque ids only, never a name, and the original's organization may be one the reader does not belong to. The list rows (`MethodSummary`) do not carry it.
|
|
128
|
+
Every read of a method, `getMethod` and the `method` of every write, carries `copied_from`: `{ org_id, method_id, draft_digest }` of the original on a copy, `null` on a method created afresh, and absent from a platform that predates copies. It is written once, when the copy is made, and never rewritten, so it names the draft the copy started from even after the original has changed or been erased; its `draft_digest` is `null` for a draft that held no tree. It holds opaque ids only, never a name, and the original's organization may be one the reader does not belong to. The list rows (`MethodSummary`) do not carry it.
|
package/docs/method-examples.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# A method's saved examples
|
|
2
2
|
|
|
3
|
-
An **example** is a named set of inputs saved on a stored method, so the method can be run on them again in one gesture, together with the latest run of exactly those inputs. A method has any number of them. Like the method itself, an example belongs to the organization: every role of it reads and runs any example of any of its methods, every role that may change its methods (`admin` and `member`) saves, edits and deletes any of them and records their latest runs, and `created_by_user_id` only records who saved it. A runner is refused every example write with a `403` `
|
|
3
|
+
An **example** is a named set of inputs saved on a stored method, so the method can be run on them again in one gesture, together with the latest run of exactly those inputs. A method has any number of them. Like the method itself, an example belongs to the organization: every role of it reads and runs any example of any of its methods, every role that may change its methods (`admin` and `member`) saves, edits and deletes any of them and records their latest runs, and `created_by_user_id` only records who saved it. A runner is refused every example write with a `403` `auth.forbidden`. The client reaches them on the hosted API's `/v1/methods/{method_id}/examples` routes, and the types are in `src/product-models.ts`.
|
|
4
4
|
|
|
5
5
|
## The example
|
|
6
6
|
|
|
@@ -32,13 +32,13 @@ An example holds no output and no baseline: its result is its latest run's, read
|
|
|
32
32
|
|
|
33
33
|
Every method takes a bare catalog id and refuses a suffixed one (`mt_x@3`) with `RequestArgumentError` before sending, as the other method routes do, and refuses a method id or an example id that is empty, `.` or `..`, or that holds a lone surrogate, the same way, as every route that puts an id in its path does ([What is refused](#what-is-refused-and-where)). The bodies are closed on the platform, so the client sends the fields the route takes and nothing else an untyped caller adds.
|
|
34
34
|
|
|
35
|
-
**Every example the client hands back has been checked.** Each route answering one, the page rows and the verdicts' `example` included, checks it once, against every field the Python SDK's model requires, with the strict type that model holds it to: string `example_id`, `method_id`, `name`, `created_by_user_id`, `created_at` and `updated_at`, object `inputs`, `input_revision` an integer within `Number.MAX_SAFE_INTEGER` either way, `latest_run_id` present as a string or `null`, and `kept_by_assistant` a boolean. An example breaking one is an answer the SDK cannot read, thrown as an `ApiResponseError` (`
|
|
35
|
+
**Every example the client hands back has been checked.** Each route answering one, the page rows and the verdicts' `example` included, checks it once, against every field the Python SDK's model requires, with the strict type that model holds it to: string `example_id`, `method_id`, `name`, `created_by_user_id`, `created_at` and `updated_at`, object `inputs`, `input_revision` an integer within `Number.MAX_SAFE_INTEGER` either way, `latest_run_id` present as a string or `null`, and `kept_by_assistant` a boolean. An example breaking one is an answer the SDK cannot read, thrown as an `ApiResponseError` (`client.response_unreadable`) whose message names the field, so a missing `latest_run_id` never reads as an example with no latest run, and a revision of `"2"` or `true` is never read as one. Both SDKs refuse the same answers but one: a revision spelled `2.0`, which JavaScript reads as the number `2` and cannot tell from it, while the Python SDK refuses the float.
|
|
36
36
|
|
|
37
37
|
**The list is paged by an opaque cursor.** Pass `nextCursor` back as `cursor`; `null` is the last page. The platform reads examples whole and stops a read at 1 MB, so a page may be short while `nextCursor` is set: a caller that wants every example follows the cursor, which `iterateMethodExamples` does through the loop `iterateMethods` and `iterateRuns` share, stopping on a `null` cursor or one the server failed to advance and throwing `PagingNotTerminatingError` past the same page ceiling.
|
|
38
38
|
|
|
39
39
|
## Revisions and the latest run
|
|
40
40
|
|
|
41
|
-
**New inputs raise the revision and clear the latest run.** An edit whose `inputs` differ from the stored ones sets `input_revision` to the next number and `latest_run_id` to `null`, since the run no longer shows what the inputs produce, and `kept_by_assistant` to `false`, since those inputs are a person's. The platform compares the inputs as canonical JSON with sorted keys, so the same inputs sent again, in another key order included, change nothing, while `3` becoming `3.0` is a change. A rename alone keeps both. The edit takes no revision or token of its own: concurrent edits are serialized on the platform, which answers a
|
|
41
|
+
**New inputs raise the revision and clear the latest run.** An edit whose `inputs` differ from the stored ones sets `input_revision` to the next number and `latest_run_id` to `null`, since the run no longer shows what the inputs produce, and `kept_by_assistant` to `false`, since those inputs are a person's. The platform compares the inputs as canonical JSON with sorted keys, so the same inputs sent again, in another key order included, change nothing, while `3` becoming `3.0` is a change. A rename alone keeps both. The edit takes no revision or token of its own: concurrent edits are serialized on the platform, which answers a `503` `runtime.unavailable`, `retry` `yes`, with nothing written when they kept landing under one.
|
|
42
42
|
|
|
43
43
|
**A run is recorded against the revision the client names.** To run an example and keep its result, read the example, start a run with its `inputs`, and record that run with the `input_revision` you read:
|
|
44
44
|
|
|
@@ -54,7 +54,7 @@ if (verdict.outcome === "stale") {
|
|
|
54
54
|
}
|
|
55
55
|
```
|
|
56
56
|
|
|
57
|
-
A run started this way sends the inputs the example holds, digit for digit, an integer past `Number.MAX_SAFE_INTEGER` included ([below](#numbers-in-an-examples-inputs)). The write is conditioned on the example still holding that revision. **A stale run is a verdict, not an error**: a run outlasting an edit is the expected course of events whenever someone edits an example while one of its runs is in flight, so the platform answers a `200` discriminated on `outcome`, as it does for a publish, and both arms carry the example as it stands. Branch on `outcome`; an answer whose `outcome` is neither, or whose `example` is not an example the client can read, is an answer the SDK cannot read, thrown as an `ApiResponseError` (`
|
|
57
|
+
A run started this way sends the inputs the example holds, digit for digit, an integer past `Number.MAX_SAFE_INTEGER` included ([below](#numbers-in-an-examples-inputs)). The write is conditioned on the example still holding that revision. **A stale run is a verdict, not an error**: a run outlasting an edit is the expected course of events whenever someone edits an example while one of its runs is in flight, so the platform answers a `200` discriminated on `outcome`, as it does for a publish, and both arms carry the example as it stands. Branch on `outcome`; an answer whose `outcome` is neither, or whose `example` is not an example the client can read, is an answer the SDK cannot read, thrown as an `ApiResponseError` (`client.response_unreadable`). The platform trusts the revision and does not compare the run's own inputs with the example's, so record a run only against the revision whose inputs it was started with. A run recorded when the example is created goes in `createMethodExample`'s `run_id` instead.
|
|
58
58
|
|
|
59
59
|
## The build assistant's example
|
|
60
60
|
|
|
@@ -94,7 +94,7 @@ The platform stores an example's inputs as JSON text and answers them exactly as
|
|
|
94
94
|
|
|
95
95
|
## What is refused, and where
|
|
96
96
|
|
|
97
|
-
The client refuses before sending, with `RequestArgumentError` (`
|
|
97
|
+
The client refuses before sending, with `RequestArgumentError` (`client.argument_invalid`):
|
|
98
98
|
|
|
99
99
|
- a suffixed method id, on every method;
|
|
100
100
|
- a method id or an example id that is empty, `.` or `..`, or that is not a string: the URL resolves a dot segment before the request leaves, so an example id `..` would address the method itself, and deleting it would erase the method, while an empty example id would address the examples list;
|
|
@@ -106,17 +106,16 @@ The client refuses before sending, with `RequestArgumentError` (`input`, not ret
|
|
|
106
106
|
- an `iterateMethodExamples` whose method id any method above refuses, at the call itself rather than when the iterator is first pulled;
|
|
107
107
|
- a body JSON cannot carry, such as inputs holding a cycle, or a `bigint` on a runtime without `JSON.rawJSON` ([exact-integers.md](./exact-integers.md)).
|
|
108
108
|
|
|
109
|
-
The platform refuses the rest, each as an `ApiResponseError` whose `code` names it (see [errors.md](./errors.md)):
|
|
109
|
+
The platform refuses the rest, each as an `ApiResponseError` whose problem's `code` names it (see [errors.md](./errors.md)):
|
|
110
110
|
|
|
111
111
|
| `code` | Status | When |
|
|
112
112
|
|---|---|---|
|
|
113
|
-
| `not_found` | `404` | An unknown method or example, an example already deleted included. |
|
|
114
|
-
| `
|
|
115
|
-
| `
|
|
116
|
-
| `
|
|
117
|
-
| `
|
|
118
|
-
| `
|
|
119
|
-
| `
|
|
120
|
-
| `malformed_request` | `400` | A cursor the listing did not issue for this method. |
|
|
113
|
+
| `pipelex.catalog.not_found` | `404` | An unknown method or example, an example already deleted included. |
|
|
114
|
+
| `run.not_found` | `404` | A `run_id`, on the create or the latest-run write, that is not one of this method's runs in the caller's organization; nothing is written. |
|
|
115
|
+
| `auth.forbidden` | `403` | Any example write by a role that may not change methods, such as a runner, which an admin of the organization must change. |
|
|
116
|
+
| `pipelex.catalog.being_deleted` | `409` | The method's erasure has started. Its examples are erased with it. |
|
|
117
|
+
| `request.too_large` | `413` | An example larger than one stored item holds. |
|
|
118
|
+
| `input.reference_not_found` | `422` | A save, the create, an inputs edit or the assistant's, whose inputs hold a file a run made whose object is no longer stored, since that run was deleted; its `missing_files` member names every such address, and nothing is saved. The platform copies a run's file an example takes into the organization's store, so the example outlives the run that made it. |
|
|
119
|
+
| `request.invalid` | `422` | A name empty, blank or over 200 characters, inputs that are not a JSON object, a `limit` outside 1 to 100, an example id longer than one can be (64 characters), or a cursor the listing did not issue for this method. |
|
|
121
120
|
|
|
122
121
|
Every write is `403` to a read-only (`purpose=sandbox`) key, which may still read the examples.
|