@pipelex/sdk 0.48.1 → 0.50.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +29 -11
- package/dist/cli/script.d.ts +1 -1
- package/dist/client.d.ts +341 -18
- package/dist/client.js +615 -28
- package/dist/client.js.map +1 -1
- package/dist/errors.d.ts +100 -12
- package/dist/errors.js +49 -7
- package/dist/errors.js.map +1 -1
- package/dist/index.d.ts +6 -5
- package/dist/index.js +5 -4
- package/dist/index.js.map +1 -1
- package/dist/prepare-inputs.d.ts +12 -1
- package/dist/prepare-inputs.js +26 -3
- package/dist/prepare-inputs.js.map +1 -1
- package/dist/product-models.d.ts +677 -22
- package/dist/product-models.js +2 -2
- package/dist/upload-grant.d.ts +134 -40
- package/dist/upload-grant.js +328 -136
- package/dist/upload-grant.js.map +1 -1
- package/dist/upload.d.ts +39 -11
- package/dist/upload.js +115 -8
- package/dist/upload.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/architecture.md +7 -5
- package/docs/errors.md +12 -3
- package/docs/files.md +258 -0
- package/docs/input-preparation.md +57 -23
- package/docs/method-examples.md +36 -3
- package/docs/organization-branding.md +102 -0
- package/package.json +1 -1
package/docs/files.md
ADDED
|
@@ -0,0 +1,258 @@
|
|
|
1
|
+
# An organization's files
|
|
2
|
+
|
|
3
|
+
A **file** is the record the hosted platform keeps of a stored object: a member's upload, a test file made for a method, or a file a run made that reached its result. The bytes stay where they were written, and a run, a form or an example still points at them by their `pipelex-storage://` address; the record adds what the bytes lack, the name a person sees, a kind, a type, a size and where the file came from. A file belongs to the organization: every role of it reads every file of it, as every role reads every run. The client reaches them on the hosted API's `/v1/files` routes, a run's files on `GET /v1/runs/{run_id}/files` and the copy into a method's draft on `POST /v1/methods/{method_id}/draft/files/from-file`, and the types are in `src/product-models.ts`. The platform's own contract is `platform/docs/files.md` in `pipelex-server`.
|
|
4
|
+
|
|
5
|
+
The point of the list for a program is the `uri` on every row: a run's inputs take it as they take any address, so an agent or an app can find "the invoice uploaded this morning" and run a method on it without uploading it again.
|
|
6
|
+
|
|
7
|
+
## A row — `FileSummary`
|
|
8
|
+
|
|
9
|
+
| Field | Meaning |
|
|
10
|
+
|---|---|
|
|
11
|
+
| `file_id` | The record's id, `fl_…`. |
|
|
12
|
+
| `uri` | The `pipelex-storage://` address of the bytes. |
|
|
13
|
+
| `name` | The name shown and used for downloads. It starts as the name the file arrived with and changes with a rename. |
|
|
14
|
+
| `media_type` | The MIME type the object is stored under. |
|
|
15
|
+
| `format` | The format key the list's `format` filter matches: `pdf`, `docx`, `image`, `html` and the rest of the runtime's table. |
|
|
16
|
+
| `size` | The byte size. |
|
|
17
|
+
| `kind` | A `FileKind`: `uploaded`, a person's upload or a run's inline input; `generated`, a file a run made whose result reaches it; `test`, a test input made for one method. |
|
|
18
|
+
| `origin` | A `FileOrigin`, who or what made the file, each member `null` where the kind fills no value: `surface`, the surface an upload came through as its `User-Agent` declared it (`app`, `mcp`, `cli`, `n8n`, `hook`, `sdk`, `raw_http`), which is provenance to show and never authority; `run_id`, the run that made it; `method_id`, the method whose run made it or the method a test file was made for; `method_version`, the version number or `"draft"` that ran. |
|
|
19
|
+
| `created_at` | The ISO-8601 instant the record was made. |
|
|
20
|
+
| `created_by` | The member the file is attributed to: the uploader, or the member who started the run that made it. |
|
|
21
|
+
|
|
22
|
+
**A list never holds a file a run made along the way** (an intermediate file, which the run's result does not reach), **nor an upload whose bytes have not been seen stored**, nor anything of another organization.
|
|
23
|
+
|
|
24
|
+
## The whole record — `FileRecord`
|
|
25
|
+
|
|
26
|
+
A file read by its id or its address, renamed, or listed among a run's files is answered whole: a `FileRecord` is a `FileSummary` with five more fields.
|
|
27
|
+
|
|
28
|
+
| Field | Meaning |
|
|
29
|
+
|---|---|
|
|
30
|
+
| `source_uri` | The address the record was written under before another run or an example took the file and the platform copied it into the organization's `assets/`, `null` until then. The record keeps its `file_id`, its names and its place on every list when it moves; its `uri` becomes the copy's. |
|
|
31
|
+
| `original_name` | The name the file arrived with, never changed. A rename adds this name's extension to the new name when it lacks it ([below](#renaming-a-file)). |
|
|
32
|
+
| `sha256` | The lowercase hexadecimal hash of the bytes, or `null` for a file stored before every writer sent one. |
|
|
33
|
+
| `role` | A `FileRole` for a generated file, `final` (it reached the run's result) or `intermediate` (made along the way); `null` for every other kind. |
|
|
34
|
+
| `state` | A `FileState`: `stored`, or `pending` while an upload's bytes have not been seen. Only a read by id or by address can answer a `pending` record, since no list holds one. |
|
|
35
|
+
|
|
36
|
+
## The methods
|
|
37
|
+
|
|
38
|
+
| Method | Route | Answer |
|
|
39
|
+
|---|---|---|
|
|
40
|
+
| `listFiles(query?)` | `GET /v1/files` | One `FilePage`, `{ items, nextCursor }`, newest first. |
|
|
41
|
+
| `iterateFiles(query?)` | the same, page after page | Every file the query selects, as an async iterator. |
|
|
42
|
+
| `getFile(fileId, { usesLimit?, usesCursor? })` | `GET /v1/files/{file_id}` | A `FileDetail`: the whole record, the methods whose lists hold it and one page of its uses. |
|
|
43
|
+
| `getFilesByUri(uri, { runId? })` | `GET /v1/files/by-uri` | A `FileRecordList`, `{ items }`: every record holding the address, the one to show first. |
|
|
44
|
+
| `renameFile(fileId, { name })` | `PATCH /v1/files/{file_id}` | The `FileRecord` as it now stands. |
|
|
45
|
+
| `linkFiles(files)` | `POST /v1/files/links` | A `FileLinksResult`, `{ items }`: one `FileLink` per requested file, in order, each a link or an error. |
|
|
46
|
+
| `requestFileArchive(request)` | `POST /v1/files/archives` | The `FileArchive`, `queued`. |
|
|
47
|
+
| `getFileArchive(archiveId)` | `GET /v1/files/archives/{archive_id}` | The `FileArchive` as it stands, with a link to the zip once it is `ready`. |
|
|
48
|
+
| `listRunFiles(runId, { limit?, cursor? })` | `GET /v1/runs/{run_id}/files` | One `RunFilesPage`: the run's inputs, outputs, intermediate files and web links, with `nextCursor`. |
|
|
49
|
+
| `copyFileIntoDraft(methodId, { file_id, path, expected_updated_at? })` | `POST /v1/methods/{method_id}/draft/files/from-file` | A `DraftFileFromFileResult`, discriminated on `blob`. |
|
|
50
|
+
|
|
51
|
+
Every method putting an id in its path refuses one that is empty, `.` or `..`, or that holds a lone surrogate, with `RequestArgumentError` before sending, as every route of the SDK does, and every body is closed on the platform, so the client sends the fields the route takes and nothing else an untyped caller adds, a `null` member reading as not given. Every role of the organization reads every file, link and archive of it; only a rename and the copy into a draft are refused to some roles, each saying which below.
|
|
52
|
+
|
|
53
|
+
```ts
|
|
54
|
+
const page = await client.listFiles({ q: "invoice", kind: "uploaded", format: ["pdf"] });
|
|
55
|
+
for (const file of page.items) console.log(file.name, file.uri);
|
|
56
|
+
|
|
57
|
+
// A method's files: what its runs and examples took or made.
|
|
58
|
+
for await (const file of client.iterateFiles({ methodId: "mt_abc123" })) {
|
|
59
|
+
if (file.kind === "test") console.log("test input", file.name);
|
|
60
|
+
}
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
## The query — `ListFilesQuery`
|
|
64
|
+
|
|
65
|
+
Every member is optional, and each is sent only when given, so an empty `q`, cursor or date is sent and refused by the API rather than dropped into an unfiltered list. The members follow the SDK's camelCase, as `ListRunsQuery`'s do, and each is sent under the API's own name.
|
|
66
|
+
|
|
67
|
+
| Member | Sent as | Meaning |
|
|
68
|
+
|---|---|---|
|
|
69
|
+
| `methodId` | `method_id` | The method's list, by its bare catalog id: the files its runs and examples took or made. |
|
|
70
|
+
| `runId` | `run_id` | The run's own list: its final outputs and its inline inputs, **in the order of their addresses** rather than newest first. Beside `methodId`, the run's list when the method is the run's own, and an empty page otherwise. The files a run took from the organization as inputs are not in it. |
|
|
71
|
+
| `createdBy` | `created_by` | One member's files by user id, or `"me"` for the caller's own: their uploads and their runs' outputs. |
|
|
72
|
+
| `kind` | `kind` | `uploaded`, `generated` or `test`. |
|
|
73
|
+
| `format` | `format`, repeated | Only the files of one of these format keys, one parameter per value (`format=pdf&format=image`), at most 100. |
|
|
74
|
+
| `mediaType` | `media_type`, repeated | Only the files of one of these MIME types, one parameter per value, at most 100. |
|
|
75
|
+
| `createdFrom`, `createdTo` | `created_from`, `created_to` | ISO-8601 **instants** with an offset, both inclusive. A bare date or a naive timestamp is a `400`, since only the caller knows which time zone's day it means. |
|
|
76
|
+
| `q` | `q` | A case-insensitive search in the name and the name the file arrived with, at most 200 characters. |
|
|
77
|
+
| `limit` | `limit` | Page size, 1 to 200, 50 by default. |
|
|
78
|
+
| `cursor` | `cursor` | The previous page's `nextCursor`. `iterateFiles` takes no cursor: it follows its own. |
|
|
79
|
+
|
|
80
|
+
`methodId`, `runId` and `createdBy` choose which list serves the page, and with none of them it is the organization's. The other members filter that list.
|
|
81
|
+
|
|
82
|
+
**An empty `format` or `mediaType` is refused before any request** with a `RequestArgumentError`, since a list with no value would be sent as no parameter at all and list every format; a value that is not a list of strings is a `TypeError`, a bug in the calling code. Everything else the API checks itself: a `limit` out of range, an unknown `kind`, more than 100 values of `format` or `mediaType`, or an id that cannot be one is a `422`; a date bound that is not an instant with an offset, a range whose start is after its end, or a cursor of another list or other filters is a `400`; and a run that is not the organization's is a `404` `run_not_found`. Each is an `ApiResponseError` ([errors.md](./errors.md)).
|
|
83
|
+
|
|
84
|
+
## Paging
|
|
85
|
+
|
|
86
|
+
**Only a `null` `nextCursor` ends a list.** The platform applies the filters to the rows a page reads and never refills the page, so a filtered page can hold fewer rows than `limit`, or none, and still carry a cursor; a client keeps asking until the cursor is `null`. **A cursor is bound to the list and the filters that issued it**, so a client passes it back with the same query, and passing it to another is a `400`.
|
|
87
|
+
|
|
88
|
+
`iterateFiles` does both: it sends the same query with every page, keeps going through an empty page that carries a cursor, stops on a `null` cursor or on a cursor the server failed to advance, and throws `PagingNotTerminatingError` past a page ceiling no real list reaches, through the loop `iterateMethods`, `iterateRuns` and `iterateMethodExamples` share. A page whose `items` is not an array, or whose `next_cursor` is neither a string nor `null`, is an `ApiResponseError` of an answer the SDK cannot read (`runtime`, not retryable), never a page the iterator walks.
|
|
89
|
+
|
|
90
|
+
## One file and its uses
|
|
91
|
+
|
|
92
|
+
`getFile` answers a `FileDetail`: the file's `FileRecord`, its `method_ids`, the methods whose lists hold it, and `uses`, a `FileUsePage` of the examples that hold its address, then the runs that took it. Each `FileUse` names the `uri` it holds, its `holder_type` (`example` or `run`), its `holder_id`, the `method_id` it belongs to and its `created_at`. A use names an address, so every record holding that address shows the same uses.
|
|
93
|
+
|
|
94
|
+
```ts
|
|
95
|
+
const detail = await client.getFile("fl_0199");
|
|
96
|
+
for (const use of detail.uses.items) console.log(use.holder_type, use.holder_id);
|
|
97
|
+
if (detail.uses.next_cursor !== null) {
|
|
98
|
+
const more = await client.getFile("fl_0199", { usesCursor: detail.uses.next_cursor });
|
|
99
|
+
}
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
The uses are paged inside the answer, so their cursor keeps the platform's spelling, `uses.next_cursor`, and goes back as `usesCursor`; `usesLimit` is 1 to 200, 50 by default. A record whose upload looked unfinished past the upload grant's lifespan is settled by the read, which answers it `stored` when its bytes are there. A file that is not the organization's is a `404` `not_found`. An answer whose `uses` is not a page of `items` and `next_cursor` is an `ApiResponseError` of an answer the SDK cannot read (`runtime`, not retryable).
|
|
103
|
+
|
|
104
|
+
## The records of an address
|
|
105
|
+
|
|
106
|
+
Several records hold one address when several members uploaded the same bytes, each under the name they gave it. `getFilesByUri` answers every one, **the first being the one a surface shows**: with `runId`, the records of the member who started that run come first, so a run's inputs show the name its starter gave the file; then the caller's; then the rest, each group oldest first. A record whose upload has not finished is answered with its `state`, `pending`. **An address no record holds answers an empty `items`**, which is how a client tells a file stored before the platform kept records, never an error.
|
|
107
|
+
|
|
108
|
+
```ts
|
|
109
|
+
const { items } = await client.getFilesByUri(input.uri, { runId: run.pipeline_run_id });
|
|
110
|
+
const shown = items[0]?.name ?? input.uri.split("/").pop();
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
A `uri` that is not a non-empty string is refused before any request. The platform answers `400` `invalid_storage_uri` for a malformed address, `403` `forbidden` for another organization's, `404` `run_not_found` for a run that is not the organization's and `422` for an address over 512 characters.
|
|
114
|
+
|
|
115
|
+
## Renaming a file
|
|
116
|
+
|
|
117
|
+
`renameFile` changes the file's `name` alone, never its bytes, its address or its `original_name`, and answers the record as it now stands. **Read the saved name from the answer**, since the platform holds every name to one rule: it removes the characters no list or header can show (control and format characters, a right-to-left override among them), trims the ends, refuses a name with nothing left, and adds the file's own extension, the one its `original_name` ends with, when the name does not already end with it, case aside. `March invoice` becomes `March invoice.pdf`, `March.docx` becomes `March.docx.pdf`, and a file whose original name has no extension, `Makefile` or `.env`, gains none. A form can show the name the rule will save as the person types, from the record's `original_name`.
|
|
118
|
+
|
|
119
|
+
```ts
|
|
120
|
+
const renamed = await client.renameFile("fl_0199", { name: "March invoice" });
|
|
121
|
+
console.log(renamed.name); // "March invoice.pdf"
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
**An `admin` or a `member` renames any file; any other role, a runner among them, renames only the files whose record names it as `created_by`**: its own uploads and the outputs of the runs it started. Past that the platform answers `403` `role_not_permitted`, its `detail` naming the caller's role. A `name` that is not a string is refused before any request; a name over 512 characters or with nothing left once cleaned is a `422`, and a file that is not the organization's a `404` `not_found`.
|
|
125
|
+
|
|
126
|
+
## Links that name the file
|
|
127
|
+
|
|
128
|
+
`linkFiles` asks for fresh links to read files, each named by exactly one of its record's id, `{ file_id }`, and its address, `{ uri }`, at most 100 in one request. The answer holds exactly one `FileLink` per requested file, in the request's order. **Branch on each item's `error`, never on the status**: the request is a `200` whenever every file got a verdict.
|
|
129
|
+
|
|
130
|
+
- A linked file carries `url`, a presigned `GET` valid for fifteen minutes, its `expires_at`, the `name` a download saves under and its `media_type`, with `error` `null`.
|
|
131
|
+
- A file that got no link carries `error`, a `FileLinkError` `{ code, detail }`, and `null` `url`, `expires_at`, `name` and `media_type`: `not_found` for an unknown file id, `conflict` for a file whose upload has not finished, `invalid_storage_uri` for a malformed address and `forbidden` for another organization's. `code` is typed as a string, so a code a later platform adds still reads.
|
|
132
|
+
|
|
133
|
+
```ts
|
|
134
|
+
const { items } = await client.linkFiles([{ file_id: "fl_0199" }, { uri: input.uri }]);
|
|
135
|
+
for (const link of items) {
|
|
136
|
+
if (link.error === null) window.open(link.url);
|
|
137
|
+
else console.warn(link.error.code, link.error.detail);
|
|
138
|
+
}
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
**Each link is signed with a `Content-Disposition` naming the file**, so a download saves under the name a person sees rather than the bytes' hash. It opens in place for an image, a PDF or plain text and downloads for every other type, SVG, HTML and XML included, so a file that could run scripts never opens from a link. A file named by its id is linked under its record's name; an address under the name of its first stored record, the caller's before another member's, whose id the item's `file_id` then names.
|
|
142
|
+
|
|
143
|
+
**A link is a bearer capability.** Keep it out of logs and ask again for a fresh one rather than storing it. The SDK keeps it out of its own errors: an answer it cannot read, one that is not one item per requested file or whose item carries neither a link nor an error, is an `ApiResponseError` whose `responseBody` replaces every `url` member and every link's query with `[redacted]` ([errors.md](./errors.md)). An item that does not name exactly one of `file_id` and `uri` as a non-empty string, or a `files` that is not a list, is refused before any request. The platform answers `422` for more than 100 files, and a `500` `config` when it cannot sign at all.
|
|
144
|
+
|
|
145
|
+
## A zip of files
|
|
146
|
+
|
|
147
|
+
The platform cannot build a zip inside a request, so a zip takes two calls. `requestFileArchive` queues the archive and answers at once, `queued`; `getFileArchive` reads it until its `state` is `ready`, whose `url` downloads the zip, or `failed`, whose `failure` says why. **An archive is never a file**: no list shows it, and it is gone a day after it was asked for, when a read of it answers `404` `not_found` as for an id never seen.
|
|
148
|
+
|
|
149
|
+
The request names the files by exactly one of two members, and both are refused before any request when neither or both is given:
|
|
150
|
+
|
|
151
|
+
- `file_ids`, a non-empty list of the organization's file ids, in the order the zip holds them. An id given twice is zipped once, and a file made along the way is allowed.
|
|
152
|
+
- `run_id`, a run of the organization, whose zip holds what went in and what came out of it: the stored files of its inputs, then its final outputs. Never a file it made along the way, nor an input given as a web link, which the platform holds no bytes of.
|
|
153
|
+
|
|
154
|
+
`name`, optional, is the name the zip downloads under. The platform makes it a download's name, taking out what no file system accepts and ending it in `.zip`; omitted, or with nothing usable left, it is `files-<run_id>.zip` for a run and `files-<YYYYMMDD-HHMMSS>.zip`, the request's instant in UTC, otherwise. Read the saved name from the archive's `name`.
|
|
155
|
+
|
|
156
|
+
```ts
|
|
157
|
+
let archive = await client.requestFileArchive({ file_ids: selected, name: "March invoices" });
|
|
158
|
+
while (archive.state === "queued" || archive.state === "building") {
|
|
159
|
+
await new Promise((resolve) => setTimeout(resolve, 2000));
|
|
160
|
+
archive = await client.getFileArchive(archive.archive_id);
|
|
161
|
+
}
|
|
162
|
+
if (archive.state === "ready") window.location.assign(archive.url!);
|
|
163
|
+
else console.warn(archive.failure?.code, archive.failure?.detail);
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
**Branch on `state`.** `queued` and `building` mean read it again shortly: a zip of a few files is ready within seconds, and the largest the caps allow within a few minutes. The SDK has no waiting helper, since how long to wait and how to show it is the caller's. `ready` carries `url`, a presigned `GET` valid for fifteen minutes and signed with a `Content-Disposition` naming the archive's `name`, with `url_expires_at`, `archive_bytes`, the zip's own size, and `ready_at`. Each read signs a fresh link, so a caller that let one lapse reads the archive again. `failed` carries a `FileArchiveFailure`: `file_gone`, a selected file deleted before the zip was ready, or `build_failed`, and the remedy for both is a new request. An archive still `queued` or `building` an hour after it was asked for reads as `failed`. `url` and `url_expires_at` are `null` in every state but `ready`, and an answer the SDK cannot read hides the link in its error as `linkFiles`' does.
|
|
167
|
+
|
|
168
|
+
| Field | Meaning |
|
|
169
|
+
|---|---|
|
|
170
|
+
| `archive_id` | `arc_…`, the id `getFileArchive` takes. |
|
|
171
|
+
| `name` | The name the zip downloads under. |
|
|
172
|
+
| `state` | A `FileArchiveState`: `queued`, `building`, `ready` or `failed`. |
|
|
173
|
+
| `source`, `run_id` | `files` for a request by `file_ids`, `run` for one by `run_id`, whose `run_id` the archive then carries; `null` otherwise. |
|
|
174
|
+
| `file_count`, `total_bytes` | How many files the zip holds and their total size before compression. |
|
|
175
|
+
| `created_at`, `expires_at` | When it was asked for, and when it is gone, a day later. |
|
|
176
|
+
| `archive_bytes`, `ready_at` | The zip's own size and when it was ready; `null` until then. |
|
|
177
|
+
| `failure` | A `FileArchiveFailure`, `{ code, detail }`, on a `failed` archive; `null` otherwise. |
|
|
178
|
+
| `url`, `url_expires_at` | The link to the zip and when it lapses, on a `ready` archive; `null` otherwise. |
|
|
179
|
+
|
|
180
|
+
**A selection past a cap is a `422` `file_archive_too_large`**, checked before anything is written or queued: at most 500 files, or 2,000,000,000 bytes in all. Its problem document's `limit` member, read from the error's `problemDocument` and typed `FileArchiveLimit`, names the cap, `max_files` or `max_total_bytes`, its `value` and what the selection `reached`, so a client can tell the person how far over the selection is and split it. A run's count stops as soon as it passes the cap, so for a run `reached` is one more than the cap.
|
|
181
|
+
|
|
182
|
+
```ts
|
|
183
|
+
try {
|
|
184
|
+
await client.requestFileArchive({ file_ids: selected });
|
|
185
|
+
} catch (err) {
|
|
186
|
+
if (err instanceof ApiResponseError && err.code === "file_archive_too_large") {
|
|
187
|
+
const limit = err.problemDocument?.limit as FileArchiveLimit;
|
|
188
|
+
console.warn(`${limit.name}: ${limit.reached} over ${limit.value}`);
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
The other refusals of a request: `422` `validation_failed` for more than 1,000 ids listed or an id longer than one can be; `404` `not_found` for a file that is not the organization's, the `detail` naming up to five of them, or a run whose files hold nothing stored to zip; `404` `run_not_found` for a run that is not the organization's; `409` `conflict`, an `input` one, not retryable, for a selected file whose upload has not finished, and a `runtime` one, retryable, for a run whose files are not all recorded yet, which the same request clears a few seconds after the run ends; and `503` `service_unavailable`, retryable, when the archive could not be queued. Each is an `ApiResponseError` whose `errorDomain` and `retryable` say what to do ([errors.md](./errors.md)).
|
|
194
|
+
|
|
195
|
+
## A run's files
|
|
196
|
+
|
|
197
|
+
`listRunFiles` answers a `RunFilesPage` of what went in, what came out and what was made along the way, sorted by where each file sits in the run. Each `RunFileEntry` carries its `mark` (`input`, `final` or `intermediate`), the `input_name` of the run's input that held an input given by its address (`null` for an inline input and for every file the run made), its `uri` and the `FileRecord` a surface shows for it, `null` for an input no record holds.
|
|
198
|
+
|
|
199
|
+
| Member | Meaning |
|
|
200
|
+
|---|---|
|
|
201
|
+
| `pipeline_run_id` | The run. |
|
|
202
|
+
| `state` | A `RunFilesState`: `listing` until the platform has recorded every file of the run, a few seconds after it ends, then `listed`. |
|
|
203
|
+
| `inputs` | What went in: the inputs given by their address, with the record of the member who started the run first, then the inline inputs. |
|
|
204
|
+
| `outputs` | The run's final outputs. |
|
|
205
|
+
| `intermediates` | The files the run made along the way, which no list of the organization's files holds. |
|
|
206
|
+
| `links` | The inputs given as a web link, each a `RunInputLink` with its `input_name`, its `url` and the `warning` that replaying the run needs the link to still serve the same file. The platform holds no record or bytes of them. |
|
|
207
|
+
| `nextCursor` | The cursor of the next page, or `null` on the last, read from the platform's `next_cursor` as every page of the SDK reads it. |
|
|
208
|
+
|
|
209
|
+
```ts
|
|
210
|
+
let page = await client.listRunFiles(runId);
|
|
211
|
+
while (page.state === "listing") {
|
|
212
|
+
await new Promise((resolve) => setTimeout(resolve, 1000));
|
|
213
|
+
page = await client.listRunFiles(runId);
|
|
214
|
+
}
|
|
215
|
+
for (const entry of page.outputs) console.log(entry.file?.name ?? entry.uri);
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
**Take the list as whole only once `state` is `listed`.** The run's own records, its outputs, its intermediate files and its inline inputs, are paged by address with `limit` (1 to 200) and `cursor`; the inputs given by address and the links come on the first page only. Pass `nextCursor` back as `cursor` until it is `null`. A run that is not the organization's is a `404`, exactly as one that does not exist; a run id holding a character outside letters, digits, `_` and `-`, or a cursor another listing issued, is a `400`; and a `limit` outside 1 to 200 is a `422`. A page whose `state` is neither, whose four lists are not arrays, or whose `next_cursor` is neither a string nor `null`, is an `ApiResponseError` of an answer the SDK cannot read.
|
|
219
|
+
|
|
220
|
+
## Adding a file to a method's draft
|
|
221
|
+
|
|
222
|
+
`copyFileIntoDraft` copies one of the organization's files into a method's draft at `path`, such as `assets/logo.png`, replacing a file already there: the app's "Add to method files". The bytes are copied by the storage service itself, so they never cross the client or the API whatever their size, and the method file then lives apart from its source: erasing the file's record, the run that made it or the method it came from leaves it readable.
|
|
223
|
+
|
|
224
|
+
The answer is a `DraftFileFromFileResult`, a `200` discriminated on `blob`: `copied`, or `already_held` when the organization's blob store already held the bytes and nothing was copied. Both carry the `file` as the draft's tree now names it, `{ path, sha256, size }`, and the `method` as the draft write stored it, a `MethodData` whose `updated_at` is the new draft token.
|
|
225
|
+
|
|
226
|
+
```ts
|
|
227
|
+
const method = await client.getMethod("mt_abc123");
|
|
228
|
+
const added = await client.copyFileIntoDraft("mt_abc123", {
|
|
229
|
+
file_id: "fl_0199",
|
|
230
|
+
path: "assets/logo.png",
|
|
231
|
+
expected_updated_at: method.updated_at,
|
|
232
|
+
});
|
|
233
|
+
let draftToken = added.method.updated_at;
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
**With `expected_updated_at`, the write is a compare-and-swap on the draft token**, refused with `409` `method_update_conflict` when the draft has moved on; without it, the file is added to the draft as it stands, never erasing someone else's save. It takes a bare catalog id, as `writeDraft` does, and refuses a `file_id` or a `path` that is not a non-empty string, or an `expected_updated_at` that is not a string, before any request. It is a write of the method, so the platform answers `403` `role_not_permitted` to a role that may not change methods, such as a runner; and `404` for an unknown method or a file that is not the organization's, `409` `method_being_deleted`, `409` `conflict` for a file whose upload has not finished or whose stored bytes do not match its record, `422` `invalid_method_files` for a path the draft cannot hold, such as `.env` or `../logo.png`, and `422` `method_tree_too_large` past the deployment's limits. An answer whose `blob` is neither arm's, or whose `file` or `method` is not an object, is an answer the SDK cannot read.
|
|
237
|
+
|
|
238
|
+
## Uploading a test file
|
|
239
|
+
|
|
240
|
+
What an upload is recorded as travels with it to `POST /v1/upload` or `POST /v1/upload/grant`: its `kind`, `uploaded` when omitted, and for a `test` file the bare catalog id of the method it is for, `method_id`. A test file is a statement about a method, so the platform holds it to what changing that method takes. The pair is spelled the same on every surface that sends it: `UploadInput` (the raw `upload`), `UploadGrantInput` (`requestUploadGrant`, [input-preparation.md](./input-preparation.md#uploading-with-a-grant--requestuploadgrant--uploadwithgrant)), `UploadFileOptions` (`uploadFile`) and `prepareInputs`' nested `upload` member, which applies to every file the preparation uploads. `UploadKind` names the two kinds, and `UploadKindOptions` the pair.
|
|
241
|
+
|
|
242
|
+
```ts
|
|
243
|
+
// One file the build assistant made to try a method:
|
|
244
|
+
await client.uploadFile("cases/invoice.pdf", { kind: "test", method_id: "mt_abc123" });
|
|
245
|
+
|
|
246
|
+
// Every file a preparation uploads, recorded as test files of the method it prepares:
|
|
247
|
+
const prepared = await client.prepareInputs({
|
|
248
|
+
method_id: "mt_abc123",
|
|
249
|
+
inputs: { invoice: "cases/invoice.pdf" },
|
|
250
|
+
upload: { kind: "test", method_id: "mt_abc123" },
|
|
251
|
+
});
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
`upload` is nested so that its `method_id`, the method a test file is for, never reads as the request's own `method_id` selector, which names the method whose inputs are prepared and excludes `method_ref` and the method's tree.
|
|
255
|
+
|
|
256
|
+
**The pair is checked before any file is read**, as the upload route checks it, so a request the route would refuse with a `422` is an `InputPreparationError` instead: a `kind` that is neither `uploaded` nor `test`, a `method_id` that is not a catalog id or carries a version suffix, `kind: "test"` without a `method_id`, and a `method_id` without `kind: "test"`. `null` reads as absent. `prepareInputs` names the members it refuses `upload.kind` and `upload.method_id`. `requestUploadGrant` sends its description as given, so the grant route's own `422` refuses a pair it would not take, and its refusals of a test file are the `ApiResponseError` the route answered.
|
|
257
|
+
|
|
258
|
+
**A test file the route refuses for what it claims is a `TestFileRefusedError`**, whose `code` says why: `role_not_permitted`, the `403` of a caller whose role may not change methods, such as a runner, which an admin of the organization must change (`config`), or which the caller avoids by uploading the file as `uploaded`; `method_not_found`, the `404` of a method that is not one of the organization's (`input`); and `method_being_deleted`, the `409` of a method being erased (`input`). It carries the `status`, the `filename` it was sending and the `methodId` it named, and wraps the `ApiResponseError` as `cause`, whose verdict it takes. These refusals are read off the platform's code, never the status alone: a `403` without `role_not_permitted` is still a refused credential (`UploadAuthenticationError`), and a `404` is still a deployment without upload (`UnsupportedUploadCapabilityError`) unless the upload named a method and the platform answered its domain miss, `not_found` with the `input` domain, rather than the `config` `404` of a route it does not serve ([input-preparation.md](./input-preparation.md#error-and-capability-behavior)).
|
|
@@ -20,6 +20,7 @@ Uploads one asset and returns its upload record. It is the language-native conve
|
|
|
20
20
|
- **Path strings are Node-only.** A filesystem path string is read from disk, which only a process that owns a filesystem can do. In a non-Node runtime a path string fails instructively rather than being misread — the SDK does not silently treat it as text or as a URL.
|
|
21
21
|
- A string that is an **HTTP(S) URL** or an existing **`pipelex-storage://` URI** is not a local asset and passes through in any runtime (see pass-through rules below).
|
|
22
22
|
- Open file objects and streams are **deferred** — they can be added later without removing anything.
|
|
23
|
+
- **A test file.** `kind: "test"` with the `method_id` of the method it is for, among the options, records the upload as that method's test file rather than a person's file, sent with it to `POST /v1/upload`; the pair is checked before the asset is read ([files.md](./files.md#uploading-a-test-file)).
|
|
23
24
|
|
|
24
25
|
> **Runtime acceptance vs. bundling.** These source types are accepted at *runtime* in every environment, and the main `@pipelex/sdk` entry bundles for a browser or edge target with nothing marked external: such a bundle resolves the Node host to its empty variant, a module holding no Node built-in ([architecture.md](./architecture.md#the-node-host)), so path strings stay Node-only, and where the host is absent a path string is refused with `InvalidLocalSourceError`, never read. The client still belongs on a server, since it holds an API key (Next.js Server Actions, the MCP server, the plugins hook), so a page imports the main entry for its error classes and types, and a page that must store a file uses the upload grant below instead, whose sender ships from the `@pipelex/sdk/upload` entry.
|
|
25
26
|
|
|
@@ -39,54 +40,78 @@ The MIME type and size are known client-side, so the record is assembled without
|
|
|
39
40
|
|
|
40
41
|
`uploadFile` needs the client, and the client needs the API key. A browser page holds the bytes but must never hold the key, so the work is split in two: the side holding the key requests a **grant** for one file, hands it to the page, and the page sends the file straight to storage with it. The bytes cross neither the credential-holding server nor the API gateway, so the gateway's request quota, which caps a base64 `/v1/upload` near 7.5 MiB, does not apply: the file may be as large as the service's own limit, which the grant reports as `max_bytes`.
|
|
41
42
|
|
|
42
|
-
**
|
|
43
|
+
**Describing the file** happens where the bytes are, with the standalone `describeUpload(file, { filename?, maxBytes? })`, which reads a `File`'s name, type and size and computes the SHA-256 of its bytes with Web Crypto:
|
|
43
44
|
|
|
44
45
|
```ts
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
});
|
|
50
|
-
// { uri, url, headers, expires_at, max_bytes }
|
|
46
|
+
import { describeUpload } from "@pipelex/sdk/upload";
|
|
47
|
+
|
|
48
|
+
const description = await describeUpload(file);
|
|
49
|
+
// { filename, content_type, size, sha256 }
|
|
51
50
|
```
|
|
52
51
|
|
|
53
|
-
|
|
52
|
+
A nameless `Blob` takes its name from `filename`, which also replaces a `File`'s own, and a file with no name at all is refused with an `InputPreparationError` before it is read. The whole file is read into memory once to hash it; a file that cannot be read, such as one changed on disk since it was picked, is an `InputPreparationError` whose `cause` is the runtime's error.
|
|
53
|
+
|
|
54
|
+
`sha256` is left out, rather than the call failing, in two cases, and the grant then works as it does for any request without a hash, below:
|
|
54
55
|
|
|
55
|
-
**The
|
|
56
|
+
- **A file larger than `maxBytes`** is described without its hash and never read. The bound is 52428800 bytes (50 MiB) by default, the platform's own default limit on one upload, because reading a file of several gigabytes picked by mistake would freeze or crash a browser tab before the grant route could refuse its size. Without the hash, the grant request carries the size alone. Where the deployment's limit is no larger than `maxBytes`, as it is with the default on a deployment at the platform's default limit, the route's `413` refuses it with its own message. On a deployment whose grants report a larger `max_bytes`, a file between `maxBytes` and that limit is granted as any request without a hash is: it is uploaded under a fresh name, with no deduplication and no checksum for storage to hold its body to, and the description's missing `sha256` is how the caller sees it. A caller on such a deployment passes its limit, which every grant reports as `max_bytes`, as `maxBytes` to hash up to it, and a `maxBytes` that is not a positive number is refused with an `InputPreparationError` before anything is read.
|
|
57
|
+
- **Where the runtime offers no Web Crypto digest**: a browser withholds `crypto.subtle` from a page served over plain `http` from any host but the local one. The upload still succeeds, only without the deduplication.
|
|
56
58
|
|
|
57
|
-
**
|
|
59
|
+
**Requesting the grant** is a client method over `POST /v1/upload/grant`, which takes the description as it is:
|
|
58
60
|
|
|
59
61
|
```ts
|
|
60
|
-
|
|
62
|
+
const grant = await client.requestUploadGrant(description);
|
|
63
|
+
// { file_id, uri, stored, url, headers, expires_at, max_bytes }
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
The file is described, never sent. Beside the description, the request takes `kind`, `uploaded` by default or `test` for a test input made for one method, which then names that method's bare catalog id in `method_id`. The grant (`UploadGrant`) is one shape with nullable fields, and **a caller branches on `stored`, never on whether `url` is present**:
|
|
67
|
+
|
|
68
|
+
- **`stored: true`**: the organization already holds these bytes. Nothing is to be sent: `uri` is the file's `pipelex-storage://` reference now, `file_id` its record, and `url`, `headers` and `expires_at` are `null`. The same person asking again for the same bytes under the same name and kind gets their existing record back.
|
|
69
|
+
- **`stored: false`**: a presigned, create-only `PUT` for the object. `url` is where to send it, `headers` are the signed headers to send unchanged, `uri` is the reference the object will carry, `file_id` its record, `pending` until the bytes land, and `expires_at` is when storage stops accepting the `PUT`, a few minutes later.
|
|
61
70
|
|
|
62
|
-
|
|
71
|
+
With a `sha256`, the object is named after the bytes, `<org>/assets/<sha256>.<ext>`, and the grant signs the hash as `x-amz-checksum-sha256`, so storage holds the body to exactly those bytes. Without one, the grant always names a new object under a fresh name and answers `stored: false`, as every grant did before hashes, so a client that cannot hash still uploads; a client that can hash always should. The platform types the object by the name's extension and never stores the declared `content_type`, which only serves a name with no extension. The wire is snake_case like every product route, so the description can travel to the server and the grant back to the page as JSON without a remap. A declared `size` over the limit is a `413` `ApiResponseError` whose `code` is `payload_too_large`; a `sha256` that is not 64 lowercase hexadecimal digits is a `422`, and so is a `size` other than the one the organization holds those bytes at (`blob_mismatch`), which means the hash was computed over other bytes than the ones described; a deployment without the route answers a `404`; and a `2xx` whose body is not a JSON object is an `ApiResponseError` the SDK cannot read, `runtime` and not retryable, rather than a grant.
|
|
72
|
+
|
|
73
|
+
**A grant with a target is a bearer capability.** Until it expires, whoever holds it can write that one object, so the SDK never logs it, and its caller should not either. The route never replays a grant for a repeated request: a caller that lost one asks for a new one, which, with a `sha256`, answers the same file record.
|
|
74
|
+
|
|
75
|
+
**Sending the file** is the standalone `uploadWithGrant(grant, file, { signal?, timeoutMs? })`, which takes a `Blob` or a `File` and returns `{ uri, file_id }`:
|
|
76
|
+
|
|
77
|
+
```ts
|
|
78
|
+
import { describeUpload, uploadWithGrant, RejectedAssetError } from "@pipelex/sdk/upload";
|
|
79
|
+
|
|
80
|
+
const { uri, file_id } = await uploadWithGrant(grant, file);
|
|
63
81
|
```
|
|
64
82
|
|
|
65
|
-
|
|
83
|
+
For a grant that answers `stored: true` it sends nothing, reads nothing and returns at once. Otherwise it sends the file as the raw body of a `PUT` to `grant.url`, with `grant.headers` exactly as received. The grant signed `Content-Length` too, and the body sets it, so the file must be exactly the declared size. Redirects are refused rather than followed, because a presigned URL is valid only at the address it was signed for. **For a grant that is not stored, the `uri` names nothing until the `PUT` has succeeded**, so fill a form field or a run input with it only after `uploadWithGrant` resolves.
|
|
84
|
+
|
|
85
|
+
Two of storage's answers are no failure. A `412` refuses the create-only `PUT` because the object already exists; when the grant names the file by its bytes, that object can only hold those bytes, stored there meanwhile by another upload of the same file, such as the same file dropped twice at once, or by the platform's own copy of a run's file holding them, made when another run took it. So the call resolves as stored, with the grant's `uri` and `file_id`, the answer a `stored: true` grant gives, and the platform settles the file's record once it sees the bytes, as for any upload. The SDK tells such a grant by its signed headers, never by the shape of its key: the platform signs `x-amz-checksum-sha256` on every grant asked for with a hash and on none asked for without one, whose `412` keeps its meaning of a used grant. A `409 ConditionalRequestConflict` says another write of the object was in flight, which storage documents as retryable, so the call sends the `PUT` again after pauses of 250 ms, 500 ms, 1 s and 2 s, within its time limit, and fails with `conflict` only when every attempt meets it.
|
|
66
86
|
|
|
67
|
-
|
|
87
|
+
A page and the server that holds the key split the work this way: the page calls `describeUpload` and sends the description to the server, the server passes it to `requestUploadGrant` unchanged and hands the grant back, and the page calls `uploadWithGrant`. The server never holds the bytes.
|
|
68
88
|
|
|
69
|
-
**The
|
|
89
|
+
**The time limit.** The whole exchange with storage runs under a time limit — the request, the file going out, the answer coming back and the read of an error body — so a stalled upload settles even when the caller passes no signal. By default the limit is 60 s plus 1 s for every started 128 KiB of the file, which assumes about 1 Mbit/s and comes to 460 s for a 50 MiB file. `timeoutMs` replaces the default, for a slower link or a caller that wants a tighter bound; it must be a positive number no larger than 2147483647, the longest delay a timer honours, or the call throws an `InputPreparationError` before anything is sent. A `signal` ends the upload whenever it fires, so it can shorten the limit but never lengthen it, and a caller's abort still propagates as the signal's own reason even when the limit ran out too. When the limit runs out before storage answers, the call throws an `UploadTransportError` whose `code` is `timeout` and whose message names the seconds allowed. When it runs out while storage's error body is still arriving, or during the pause after a conflict, storage has already answered, so the call settles as that answer instead, classified from its status and whatever part of the body arrived. After a timeout, whether storage stored the file is unknown: retrying with the same grant before it expires either stores the file or finds it stored, which for a grant asked for without a hash is the `412` of a used grant, and then the grant's `uri` names the file. The grant's expiry does not bound the upload, because storage checks the signature when the request starts, not when it ends.
|
|
90
|
+
|
|
91
|
+
**The browser-safe entry.** `describeUpload`, `uploadWithGrant`, the grant types and the three error classes they raise (`InputPreparationError`, `RejectedAssetError`, `UploadTransportError`) are exported from `@pipelex/sdk/upload`, whose import graph reaches no Node builtin, no `undici` and not the client, so a page bundles it with nothing marked external. They are the same functions and the same classes the main entry exports, so an `instanceof` check agrees whichever entry the error came through. The unit suite bundles both entries for the browser with esbuild and fails if anything Node-only becomes reachable.
|
|
70
92
|
|
|
71
93
|
**Storage's refusals** map onto the input-preparation errors, so a caller that already handles `uploadFile` needs no new case:
|
|
72
94
|
|
|
73
95
|
| Storage answers | Error | `code` | Meaning |
|
|
74
96
|
| --- | --- | --- | --- |
|
|
97
|
+
| nothing sent, the grant answered `stored: true` | none | | Stored already; `uri` names the file. |
|
|
75
98
|
| `2xx` | none | | Stored; `uri` names the object. |
|
|
76
|
-
| `412 PreconditionFailed
|
|
99
|
+
| `412 PreconditionFailed`, for a grant that names its bytes | none | | Stored meanwhile, by another upload of the same bytes or by the platform's copy of a run's file: the object can only hold those bytes, so the call resolves with the grant's `uri` and `file_id`. |
|
|
100
|
+
| `412 PreconditionFailed`, for a grant asked for without a hash | `RejectedAssetError` | `grant_used` | The grant was already used. Such a grant writes one object, once — so after an earlier attempt that failed at storage, the grant's `uri` may already name the file. |
|
|
77
101
|
| `403 SignatureDoesNotMatch` | `RejectedAssetError` | `signature_mismatch` | The file's size, content type or metadata differ from what the grant signed. |
|
|
78
102
|
| `403 AccessDenied`, "Request has expired" | `RejectedAssetError` | `grant_expired` | The grant expired. |
|
|
79
103
|
| `403 AccessDenied`, headers "not signed" | `RejectedAssetError` | `unsigned_header` | The request carried a storage header the grant did not sign. |
|
|
104
|
+
| `400 BadDigest` | `RejectedAssetError` | `store_refused` | The body's bytes differ from the ones whose SHA-256 the grant was asked with. Send the file the grant was requested for, or describe the file again and ask for a new grant. |
|
|
80
105
|
| `400 RequestTimeout` | `UploadTransportError` | `storage_timeout` | Storage stopped waiting for the body and wrote nothing, so the same grant can be retried while it is valid. |
|
|
81
|
-
| `409 ConditionalRequestConflict` | `UploadTransportError` | `conflict` | Another
|
|
106
|
+
| `409 ConditionalRequestConflict` on every attempt | `UploadTransportError` | `conflict` | Another write of the same object was still in progress after every pause. Whether the object was written is unknown, so the same-grant retry applies. A single `409` followed by any other answer is read as that answer. |
|
|
82
107
|
| `408`, `429` | `UploadTransportError` | `unexpected` | Storage timed out or throttled the request rather than refusing the file, so the error is retryable: a later attempt can succeed, and the same-grant retry applies. |
|
|
83
108
|
| any other `4xx` | `RejectedAssetError` | `store_refused` | Storage's own message is relayed. |
|
|
84
109
|
| `501` | `UploadTransportError` | `server_error` | Storage does not implement the request it was sent, so it wrote nothing, and a retry meets the same answer: the error is not retryable. |
|
|
85
|
-
| any other `5xx` | `UploadTransportError` | `server_error` | Whether the object was written is unknown: retrying with the same grant either stores it or
|
|
110
|
+
| any other `5xx` | `UploadTransportError` | `server_error` | Whether the object was written is unknown: retrying with the same grant either stores it or finds it stored, which for a grant asked for without a hash is the `412` of a used grant. |
|
|
86
111
|
| a redirect | `UploadTransportError` | `redirected` | Refused, not followed. |
|
|
87
112
|
| no answer within the time limit | `UploadTransportError` | `timeout` | The limit above ran out before storage answered. Whether the object was written is unknown, so the same-grant retry applies. |
|
|
88
113
|
| no response | `UploadTransportError` | `unreachable` | Storage unreachable. In a browser a refused cross-origin request looks exactly like this, so the message also points at the page's CSP `connect-src`. If the connection failed after the file went out, the object may have been written, so the message gives the same-grant retry too. |
|
|
89
|
-
| nothing sent | `UploadTransportError` | `invalid_grant_url` | The grant
|
|
114
|
+
| nothing sent | `UploadTransportError` | `invalid_grant_url` | The grant is not stored and its `url` is not an absolute `http(s)` URL free of user info, so it is refused before any request. A browser would resolve a relative URL against the page and send the credential there. |
|
|
90
115
|
|
|
91
116
|
`RejectedAssetError.code` and `UploadTransportError.code` are what a caller branches on — to request a new grant on `grant_used` or `grant_expired`, to retry with the same grant on `timeout`, or on `server_error` or `unexpected` when the error is `retryable` (a `501` is not), or to tell a timeout from an unreachable host — rather than the message, the status or the error's `name`, since an expired grant and an unsigned header both answer `403 AccessDenied` and only S3's message tells them apart. The SDK sets `code` on every error it raises. `RejectedAssetError.status` is storage's, and its `filename` is the `File`'s name, or the object name the grant's `uri` ends in for a nameless `Blob`. `UploadTransportError.status` is storage's status when storage answered, and undefined when it did not. Neither error wraps storage's response, and only S3's error code and message reach a message: its error body can echo the signed request, credential included, so it is never kept, and only its first 16 KiB are read, which is where S3 writes its code and message. **No error carries the grant's URL.** The runtime's network error can: Node and Chrome name the whole URL when they cannot build a request from it, and Bun puts it on the error's `path`. So an unreachable storage is described by its origin and by the error names and codes along the cause chain (`TypeError, caused by Error ENOTFOUND`), and the runtime's error is neither relayed nor kept as `cause`. A caller's abort propagates as the signal's own reason, unwrapped, including one that lands while storage's error body is still arriving.
|
|
92
117
|
|
|
@@ -160,6 +185,14 @@ client.prepareInputs({ method_id, inputs, allowUploads: false }) → PreparedInp
|
|
|
160
185
|
|
|
161
186
|
The error names the input as `inputName` and where inside it the value sits as `inputPath`: the input's name for a top-level file, then `.field` for each declared field and `[index]` for each list element (`dossier.attachments[1]`). An explicit envelope adds nothing to the path, since its `content` is the input's value. `valueKind` says what the value is, `local_path`, `data_url` or `bytes`, so a consumer can tell its caller what to send instead; the value itself is never in the message or on the error, since it is the caller's data. The option is `true` when omitted, must be a boolean when given — anything else is refused with an `InputPreparationError` before any request, since reading a stray `"false"` as permission would upload exactly what the caller meant to keep — and is not sent to the server. The option covers `prepareInputs` alone: `uploadFile` is an upload by definition.
|
|
162
187
|
|
|
188
|
+
#### Recording the uploads as test files
|
|
189
|
+
|
|
190
|
+
```
|
|
191
|
+
client.prepareInputs({ method_id, inputs, upload: { kind: "test", method_id } }) → PreparedInputs
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
`upload` says what every file the preparation uploads is recorded as, sent with each upload: its `kind` and, for a test file, the bare catalog id of the method it is for, such as the inputs a build assistant makes to try a method. Omitted, every upload is a person's file. It is a member of its own so that its `method_id` never reads as the request's own `method_id` selector, and it is checked once, before the pipe I/O request, as `uploadFile` checks its options: anything the upload route would refuse with a `422` is an `InputPreparationError` naming `upload.kind` or `upload.method_id`, and an `upload` that is not an object is refused the same way. A file the route refuses for what a test file claims is a `TestFileRefusedError` ([files.md](./files.md#uploading-a-test-file)). It is not sent to the pipe I/O route, and under `allowUploads: false` it records nothing, since nothing is uploaded.
|
|
195
|
+
|
|
163
196
|
## A method's files in hand
|
|
164
197
|
|
|
165
198
|
Every method-taking operation — `execute` / `start`, `validate` / `resolve` / `codegen` / `pipeIo`, and `prepareInputs` — takes a `method_id` **natively** on the hosted API, as a server pass-through, its version suffix included (`mt_…`, `mt_…@<n>`, `mt_…@draft`): the platform resolves it and injects the tree of the version it names before the runner sees the request (see [architecture.md → hosted run extensions](./architecture.md#hosted-run-extensions-method_id)). That is the uniform rule — *an id option is always a server pass-through, and any client-side expansion is the caller's own explicit call*.
|
|
@@ -229,13 +262,14 @@ The contract distinguishes these semantic outcomes, each a typed subclass of `In
|
|
|
229
262
|
- **invalid input value** (`InvalidInputValueError`, carries `inputName` and `inputPath`) — a value at a file input that cannot be turned into a file to upload: a `data:` URL that does not decode (no comma, bad base64, bad percent-encoding), or a value of a type no file input takes, such as a number or an object that is neither bytes nor `{url}` content. It is raised while the inputs are read, before anything is uploaded. It names the input and the path inside it as `UploadNotAllowedError` does (`Input "dossier.attachments[1]" holds a number, which no file input takes: …`), describes the value by its kind and never quotes it, and lists the forms a file input takes in the mode it ran in. With `InvalidLocalSourceError`, it is how a consumer tells the inputs' own mistakes from a method that does not load, though all three are `InputPreparationError`s with the same verdict;
|
|
230
263
|
- **upload not allowed** (`UploadNotAllowedError`, carries `inputName`, `inputPath` and `valueKind`) — under `allowUploads: false`, a value at a file input that would need an upload: a local path, a `data:` URL or bytes. It is raised before anything is read, decoded or uploaded (see "[Preparing without uploads](#preparing-without-uploads)");
|
|
231
264
|
- **rejected asset** (`RejectedAssetError`) — the server refused it (e.g. a `413` past the service-defined size cap — see "Storage policy" — surfaced as a clear rejection with `code` `too_large`, not a raw transport error), or storage refused an upload with a grant (a used or expired grant, or a file that differs from what the grant signed — see the table above for each `code`);
|
|
232
|
-
- **
|
|
233
|
-
- **
|
|
265
|
+
- **refused test file** (`TestFileRefusedError`, carries `code`, `status`, `filename` and `methodId`) — the upload route refused a test file for what it claims, never for its bytes: `role_not_permitted`, the `403` of a caller whose role may not change methods; `method_not_found`, the `404` the platform answers as `not_found` in the `input` domain for a method that is not the organization's; `method_being_deleted`, the `409` of a method being erased. Each is read off the platform's code, never the status alone;
|
|
266
|
+
- **unsupported server capability** (`UnsupportedUploadCapabilityError`) — the configured deployment has no upload route: any other `404`, the platform's `config` one for a route it does not serve or a bare runner's, which carries no code;
|
|
267
|
+
- **authentication / authorization failure** (`UploadAuthenticationError`) — a `401`, or a `403` other than a test file's `role_not_permitted`;
|
|
234
268
|
- **transport failure** (`UploadTransportError`) — a network or server fault, or any other unexpected upload failure; its `status` is the HTTP status when a response produced it, and its `code` says which failure it was. From `uploadFile` the code is `server_error` for a `5xx`, `timeout` when the client's own request timeout ran out, `unreachable` when no response came back, and `unexpected` for any other status or failure, a `2xx` answer the SDK cannot read included, such as one with no string `uri`; `uploadWithGrant`'s codes are in the table above.
|
|
235
269
|
|
|
236
|
-
**Which file.** Every upload failure `uploadFile` raises names the file it was sending, as it sent it: `RejectedAssetError`, `UploadAuthenticationError`, `UnsupportedUploadCapabilityError` and `UploadTransportError` each carry `filename`, so a consumer that prepared several files says which one failed without parsing a message. `uploadWithGrant` sends a grant's file, not a named one, and leaves `filename` undefined on the `UploadTransportError` it raises.
|
|
270
|
+
**Which file.** Every upload failure `uploadFile` raises names the file it was sending, as it sent it: `RejectedAssetError`, `TestFileRefusedError`, `UploadAuthenticationError`, `UnsupportedUploadCapabilityError` and `UploadTransportError` each carry `filename`, so a consumer that prepared several files says which one failed without parsing a message. `uploadWithGrant` sends a grant's file, not a named one, and leaves `filename` undefined on the `UploadTransportError` it raises.
|
|
237
271
|
|
|
238
|
-
Each of these carries the verdict every SDK error does, `retryable` and `errorDomain` ([errors.md](./errors.md#each-classs-verdict)): the family's own is `input` and not retryable, an unsupported deployment and a refused credential are `config`, and a transport failure takes the verdict of the error it wraps, so a `402` plan refusal wrapped by `uploadFile` is `config` and not retryable, or, wrapping none, the verdict its `code` gives (`timeout`, `storage_timeout` and `conflict` retryable, `server_error` retryable unless storage answered a `501`, `unreachable` and `redirected` the environment's). A pipe I/O answer whose `is_valid` is not a boolean, whose `input_form` does not describe the pipe it selected, or whose descriptor for that pipe is malformed where the walk reads it or names a kind this SDK does not know, is a plain `InputPreparationError` with a `runtime` verdict, not retryable, since the route broke its own contract. The descriptor is checked whole, once, before any caller value is read or anything uploaded, so the refusal does not depend on which inputs the caller supplied, and its message names the pipe, where in the descriptor the fault sits (`fields[3].item`) and what was found there.
|
|
272
|
+
Each of these carries the verdict every SDK error does, `retryable` and `errorDomain` ([errors.md](./errors.md#each-classs-verdict)): the family's own is `input` and not retryable, an unsupported deployment and a refused credential are `config`, a refused test file takes the platform's verdict, `config` for a role and `input` for a method, and a transport failure takes the verdict of the error it wraps, so a `402` plan refusal wrapped by `uploadFile` is `config` and not retryable, or, wrapping none, the verdict its `code` gives (`timeout`, `storage_timeout` and `conflict` retryable, `server_error` retryable unless storage answered a `501`, `unreachable` and `redirected` the environment's). A pipe I/O answer whose `is_valid` is not a boolean, whose `input_form` does not describe the pipe it selected, or whose descriptor for that pipe is malformed where the walk reads it or names a kind this SDK does not know, is a plain `InputPreparationError` with a `runtime` verdict, not retryable, since the route broke its own contract. The descriptor is checked whole, once, before any caller value is read or anything uploaded, so the refusal does not depend on which inputs the caller supplied, and its message names the pipe, where in the descriptor the fault sits (`fields[3].item`) and what was found there.
|
|
239
273
|
|
|
240
274
|
All preparation failures are raised **before any run is created**.
|
|
241
275
|
|
package/docs/method-examples.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# A method's saved examples
|
|
2
2
|
|
|
3
|
-
An **example** is a named set of inputs saved on a stored method, so the method can be run on them again in one gesture, together with the latest run of exactly those inputs. A method has any number of them. Like the method itself, an example belongs to the organization:
|
|
3
|
+
An **example** is a named set of inputs saved on a stored method, so the method can be run on them again in one gesture, together with the latest run of exactly those inputs. A method has any number of them. Like the method itself, an example belongs to the organization: every role of it reads and runs any example of any of its methods, every role that may change its methods (`admin` and `member`) saves, edits and deletes any of them and records their latest runs, and `created_by_user_id` only records who saved it. A runner is refused every example write with a `403` `role_not_permitted`. The client reaches them on the hosted API's `/v1/methods/{method_id}/examples` routes, and the types are in `src/product-models.ts`.
|
|
4
4
|
|
|
5
5
|
## The example
|
|
6
6
|
|
|
@@ -13,6 +13,7 @@ An **example** is a named set of inputs saved on a stored method, so the method
|
|
|
13
13
|
| `input_revision` | Starts at 1 and rises by one each time the inputs change. |
|
|
14
14
|
| `latest_run_id` | The latest run of the current inputs, or `null`. It may name a run deleted since, whose read then answers `404`. |
|
|
15
15
|
| `created_by_user_id`, `created_at`, `updated_at` | Who saved it, when, and when it last changed. |
|
|
16
|
+
| `kept_by_assistant` | Whether the build assistant keeps this example's inputs current with its test inputs: `true` on an example the assistant's save created, `false` on every other, and `false` from the first inputs change a person makes. A rename leaves it as it is ([below](#the-build-assistants-example)). |
|
|
16
17
|
|
|
17
18
|
An example holds no output and no baseline: its result is its latest run's, read by the run's id (`getRunResult`).
|
|
18
19
|
|
|
@@ -26,17 +27,18 @@ An example holds no output and no baseline: its result is its latest run's, read
|
|
|
26
27
|
| `getMethodExample(methodId, exampleId)` | `GET /v1/methods/{id}/examples/{example_id}` | The `MethodExample`. |
|
|
27
28
|
| `updateMethodExample(methodId, exampleId, { name?, inputs? })` | `PATCH /v1/methods/{id}/examples/{example_id}` | The `MethodExample` as it now stands. |
|
|
28
29
|
| `recordMethodExampleLatestRun(methodId, exampleId, { run_id, input_revision })` | `PUT /v1/methods/{id}/examples/{example_id}/latest-run` | A `MethodExampleLatestRunResult`: `{ outcome: "recorded" \| "stale", example }`. |
|
|
30
|
+
| `saveAssistantMethodExample(methodId, { name, inputs, example_id?, input_revision? })` | `POST /v1/methods/{id}/examples/assistant-save` | A `MethodExampleAssistantSaveResult`: `{ outcome: "kept" \| "created", example }`. |
|
|
29
31
|
| `deleteMethodExample(methodId, exampleId)` | `DELETE /v1/methods/{id}/examples/{example_id}` | Nothing; the platform answers `204`. |
|
|
30
32
|
|
|
31
33
|
Every method takes a bare catalog id and refuses a suffixed one (`mt_x@3`) with `RequestArgumentError` before sending, as the other method routes do, and refuses a method id or an example id that is empty, `.` or `..`, or that holds a lone surrogate, the same way, as every route that puts an id in its path does ([What is refused](#what-is-refused-and-where)). The bodies are closed on the platform, so the client sends the fields the route takes and nothing else an untyped caller adds.
|
|
32
34
|
|
|
33
|
-
**Every example the client hands back has been checked.** Each route answering one, the page rows and the
|
|
35
|
+
**Every example the client hands back has been checked.** Each route answering one, the page rows and the verdicts' `example` included, checks it once, against every field the Python SDK's model requires, with the strict type that model holds it to: string `example_id`, `method_id`, `name`, `created_by_user_id`, `created_at` and `updated_at`, object `inputs`, `input_revision` an integer within `Number.MAX_SAFE_INTEGER` either way, `latest_run_id` present as a string or `null`, and `kept_by_assistant` a boolean. An example breaking one is an answer the SDK cannot read, thrown as an `ApiResponseError` (`runtime`, not retryable) whose message names the field, so a missing `latest_run_id` never reads as an example with no latest run, and a revision of `"2"` or `true` is never read as one. Both SDKs refuse the same answers but one: a revision spelled `2.0`, which JavaScript reads as the number `2` and cannot tell from it, while the Python SDK refuses the float.
|
|
34
36
|
|
|
35
37
|
**The list is paged by an opaque cursor.** Pass `nextCursor` back as `cursor`; `null` is the last page. The platform reads examples whole and stops a read at 1 MB, so a page may be short while `nextCursor` is set: a caller that wants every example follows the cursor, which `iterateMethodExamples` does through the loop `iterateMethods` and `iterateRuns` share, stopping on a `null` cursor or one the server failed to advance and throwing `PagingNotTerminatingError` past the same page ceiling.
|
|
36
38
|
|
|
37
39
|
## Revisions and the latest run
|
|
38
40
|
|
|
39
|
-
**New inputs raise the revision and clear the latest run.** An edit whose `inputs` differ from the stored ones sets `input_revision` to the next number and `latest_run_id` to `null`, since the run no longer shows what the inputs produce. The platform compares the inputs as canonical JSON with sorted keys, so the same inputs sent again, in another key order included, change nothing, while `3` becoming `3.0` is a change. A rename alone keeps both. The edit takes no revision or token of its own: concurrent edits are serialized on the platform, which answers a retryable `503` with nothing written when they kept landing under one.
|
|
41
|
+
**New inputs raise the revision and clear the latest run.** An edit whose `inputs` differ from the stored ones sets `input_revision` to the next number and `latest_run_id` to `null`, since the run no longer shows what the inputs produce, and `kept_by_assistant` to `false`, since those inputs are a person's. The platform compares the inputs as canonical JSON with sorted keys, so the same inputs sent again, in another key order included, change nothing, while `3` becoming `3.0` is a change. A rename alone keeps both. The edit takes no revision or token of its own: concurrent edits are serialized on the platform, which answers a retryable `503` with nothing written when they kept landing under one.
|
|
40
42
|
|
|
41
43
|
**A run is recorded against the revision the client names.** To run an example and keep its result, read the example, start a run with its `inputs`, and record that run with the `input_revision` you read:
|
|
42
44
|
|
|
@@ -54,6 +56,34 @@ if (verdict.outcome === "stale") {
|
|
|
54
56
|
|
|
55
57
|
A run started this way sends the inputs the example holds, digit for digit, an integer past `Number.MAX_SAFE_INTEGER` included ([below](#numbers-in-an-examples-inputs)). The write is conditioned on the example still holding that revision. **A stale run is a verdict, not an error**: a run outlasting an edit is the expected course of events whenever someone edits an example while one of its runs is in flight, so the platform answers a `200` discriminated on `outcome`, as it does for a publish, and both arms carry the example as it stands. Branch on `outcome`; an answer whose `outcome` is neither, or whose `example` is not an example the client can read, is an answer the SDK cannot read, thrown as an `ApiResponseError` (`runtime`, not retryable). The platform trusts the revision and does not compare the run's own inputs with the example's, so record a run only against the revision whose inputs it was started with. A run recorded when the example is created goes in `createMethodExample`'s `run_id` instead.
|
|
56
58
|
|
|
59
|
+
## The build assistant's example
|
|
60
|
+
|
|
61
|
+
The build assistant writes test inputs for the method it is building and keeps one example of the method current with them, so that a person finds its latest inputs saved and can run them again. **A person's edit is never written over.** `saveAssistantMethodExample` is that save, and `kept_by_assistant` says whose an example is: `true` on an example the assistant's save created, `false` from the first inputs change made through `updateMethodExample`, after which the example is the person's. A rename leaves the flag as it is, and a create through `createMethodExample` is a person's and never flagged.
|
|
62
|
+
|
|
63
|
+
The assistant names the example it read and the revision it read it at. Find its example as the method's newest with `kept_by_assistant` true, the last such in `listMethodExamples`' oldest-first order, and send its `example_id` with its `input_revision`:
|
|
64
|
+
|
|
65
|
+
```ts
|
|
66
|
+
let assistants: MethodExample | undefined;
|
|
67
|
+
for await (const example of client.iterateMethodExamples(methodId)) {
|
|
68
|
+
if (example.kept_by_assistant) assistants = example;
|
|
69
|
+
}
|
|
70
|
+
const saved = await client.saveAssistantMethodExample(methodId, {
|
|
71
|
+
name: "Assistant test inputs",
|
|
72
|
+
inputs: testInputs,
|
|
73
|
+
...(assistants && {
|
|
74
|
+
example_id: assistants.example_id,
|
|
75
|
+
input_revision: assistants.input_revision,
|
|
76
|
+
}),
|
|
77
|
+
});
|
|
78
|
+
if (saved.outcome === "created") {
|
|
79
|
+
// A person took the previous example, or there was none: saved.example is the assistant's now.
|
|
80
|
+
}
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
The platform writes the inputs into the named example only while it still exists, still carries the flag and still holds that revision, in one conditional write it never retries, and answers `kept` with the example, its revision raised and its latest run cleared as any inputs change does, its name untouched since a person may have chosen it. Otherwise, when a person's edit, another save or a deletion landed first, and when no example is named, which is how the first save on a method creates the assistant's example, it saves the inputs as a new example flagged as the assistant's under `name`, at revision 1, and answers `created`. **Both are a `200` verdict**, since a person taking the example while the assistant works is the expected course of events; branch on `outcome`. A retry of a save that landed answers `kept` and writes nothing again. The SDK has no helper finding the assistant's example, since the caller already lists the examples it shows.
|
|
84
|
+
|
|
85
|
+
The client sends `name`, `inputs` and, when given, `example_id` with `input_revision`, and nothing else, a `null` reading as not given. `name` is the name a new example takes, so it is required even when the save names an example, whose own name a `kept` save leaves alone.
|
|
86
|
+
|
|
57
87
|
## Numbers in an example's inputs
|
|
58
88
|
|
|
59
89
|
The platform stores an example's inputs as JSON text and answers them exactly as they were saved, but JavaScript reads every JSON number as a double, which holds an integer exactly only up to `Number.MAX_SAFE_INTEGER` (2^53 - 1). Inputs saved by another client, the Python SDK or a raw request, can hold a larger one, such as `12345678901234567891`, which a double reads as `12345678901234567000`. Sent back, to `updateMethodExample` or as a run's inputs, the rounded number would overwrite the saved inputs, clearing the latest run, or run the method on inputs nobody saved and record that run under the revision that holds the real ones, since the platform trusts the revision it is given.
|
|
@@ -72,6 +102,7 @@ The client refuses before sending, with `RequestArgumentError` (`input`, not ret
|
|
|
72
102
|
- a `createMethodExample` without its `name` or its `inputs` (a `null` one counting as not given), or whose `run_id` is given but is not a non-empty string, a `null` reading as not given;
|
|
73
103
|
- an `updateMethodExample` naming neither `name` nor `inputs` (a `null` field counts as not given, as the platform reads it);
|
|
74
104
|
- a `recordMethodExampleLatestRun` whose `run_id` is not a non-empty string or whose `input_revision` is not a positive integer, the example id checked first, then the run id, then the revision;
|
|
105
|
+
- a `saveAssistantMethodExample` without its `name` or its `inputs`, whose `example_id` is given but is not a non-empty string, whose `input_revision` is given but is not a positive integer, or that gives only one of the two;
|
|
75
106
|
- an `iterateMethodExamples` whose method id any method above refuses, at the call itself rather than when the iterator is first pulled;
|
|
76
107
|
- a body JSON cannot carry, such as inputs holding a cycle, or a `bigint` on a runtime without `JSON.rawJSON` ([exact-integers.md](./exact-integers.md)).
|
|
77
108
|
|
|
@@ -81,8 +112,10 @@ The platform refuses the rest, each as an `ApiResponseError` whose `code` names
|
|
|
81
112
|
|---|---|---|
|
|
82
113
|
| `not_found` | `404` | An unknown method or example, an example already deleted included. |
|
|
83
114
|
| `run_not_found` | `404` | A `run_id`, on the create or the latest-run write, that is not one of this method's runs in the caller's organization; nothing is written. |
|
|
115
|
+
| `role_not_permitted` | `403` | Any example write by a role that may not change methods, such as a runner, which an admin of the organization must change. |
|
|
84
116
|
| `method_being_deleted` | `409` | The method's erasure has started. Its examples are erased with it. |
|
|
85
117
|
| `payload_too_large` | `413` | An example larger than one stored item holds. |
|
|
118
|
+
| `input_files_missing` | `422` | A save, the create, an inputs edit or the assistant's, whose inputs hold a file a run made whose object is no longer stored, since that run was deleted; its `missing_files` member names every such address, and nothing is saved. The platform copies a run's file an example takes into the organization's store, so the example outlives the run that made it. |
|
|
86
119
|
| `validation_failed` | `422` | A name empty, blank or over 200 characters, inputs that are not a JSON object, a `limit` outside 1 to 100, or an example id longer than one can be (64 characters). |
|
|
87
120
|
| `malformed_request` | `400` | A cursor the listing did not issue for this method. |
|
|
88
121
|
|