@extuitive/skill 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +514 -0
- package/bin/cli.mjs +932 -0
- package/package.json +40 -0
- package/skills/extuitive/SKILL.md +63 -0
- package/skills/extuitive/references/connect.md +75 -0
- package/skills/extuitive/references/init.md +83 -0
- package/skills/extuitive/references/select.md +89 -0
- package/skills/extuitive/references/tools.md +252 -0
- package/skills/extuitive/references/upload-status.md +102 -0
- package/skills/extuitive/references/upload.md +160 -0
- package/skills/extuitive/scripts/upload.mjs +340 -0
- package/src/constants.mjs +63 -0
- package/src/doctor.mjs +513 -0
- package/src/exec.mjs +139 -0
- package/src/hosts.mjs +350 -0
- package/src/install.mjs +541 -0
- package/src/mcp-setup.mjs +476 -0
- package/src/zip.mjs +246 -0
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# `/extuitive upload-status` — how is an upload going
|
|
2
|
+
|
|
3
|
+
Reports where an Extuitive upload has got to, and finishes by saying what was actually
|
|
4
|
+
accepted.
|
|
5
|
+
|
|
6
|
+
A `batchId` given with the command arrives in `$ARGUMENTS`.
|
|
7
|
+
|
|
8
|
+
## Why this is not just "did the transfer finish"
|
|
9
|
+
|
|
10
|
+
Bytes arriving at storage is not the same as a file being accepted. Every file is checked
|
|
11
|
+
after it lands, and it can still be rejected. `READY` is the only status that means a file
|
|
12
|
+
uploaded successfully, and it arrives some time after the transfer completes.
|
|
13
|
+
|
|
14
|
+
So "the upload finished" is never the answer on its own. The answer is how many reached
|
|
15
|
+
`READY`, and which ones did not, and why.
|
|
16
|
+
|
|
17
|
+
## Steps
|
|
18
|
+
|
|
19
|
+
### 1. Find the batch
|
|
20
|
+
|
|
21
|
+
In order of preference:
|
|
22
|
+
|
|
23
|
+
1. The `batchId` the person gave you, or the one from a `create_upload_batch` call earlier in
|
|
24
|
+
this conversation.
|
|
25
|
+
2. Otherwise call `list_upload_batches` for the workspace. It returns batches newest first
|
|
26
|
+
with `fileCount`, `pending`, and `statusCounts` per batch. The first entry is the current
|
|
27
|
+
one. This is also the right call when someone uploaded through a browser link, since you
|
|
28
|
+
never saw a `batchId` for that.
|
|
29
|
+
|
|
30
|
+
If you do not know which workspace, use the one chosen earlier in this conversation, or call
|
|
31
|
+
`list_workspaces` first when there was none. A `batchId` belongs to the workspace it was
|
|
32
|
+
created in, so a batch from before a switch needs the workspace it was made in rather than the
|
|
33
|
+
current one.
|
|
34
|
+
|
|
35
|
+
A single file uploaded on its own has no `batchId` at all. Use `get_upload_content` with its
|
|
36
|
+
`contentId` instead — same status rules apply.
|
|
37
|
+
|
|
38
|
+
### 2. Poll
|
|
39
|
+
|
|
40
|
+
Call `get_upload_batch_content` with the `batchId` about **every 5 seconds** until `settled`
|
|
41
|
+
is `true`.
|
|
42
|
+
|
|
43
|
+
Give up after about **5 minutes**. Say that you stopped waiting and what the counts were when
|
|
44
|
+
you did. Do not describe an unfinished batch as finished.
|
|
45
|
+
|
|
46
|
+
`list_upload_batches` is the cheaper call if you only need "is it done yet" across a whole
|
|
47
|
+
batch — it returns no file rows. Use `get_upload_batch_content` when you need to name
|
|
48
|
+
individual files.
|
|
49
|
+
|
|
50
|
+
### 3. Report periodically, not only at the end
|
|
51
|
+
|
|
52
|
+
This is the part that matters most, and it is the one that is easy to skip.
|
|
53
|
+
|
|
54
|
+
**While polling, tell the person where things stand roughly every 15–30 seconds.** Something
|
|
55
|
+
as short as "18 of 30 accepted, 12 still being checked" is enough. A silent two-minute wait is
|
|
56
|
+
indistinguishable from a hang, and the person cannot tell whether to keep waiting.
|
|
57
|
+
|
|
58
|
+
**When `settled` becomes `true`, report once more and stop polling.** That final report needs:
|
|
59
|
+
|
|
60
|
+
- how many reached `READY`, out of how many in the batch
|
|
61
|
+
- every `REJECTED` file by name, each with its `rejectionReason`
|
|
62
|
+
- anything left in a non-final state, if you stopped on the timeout rather than on `settled`
|
|
63
|
+
|
|
64
|
+
Then stop. Do not keep polling a settled batch.
|
|
65
|
+
|
|
66
|
+
### 4. Read the numbers correctly
|
|
67
|
+
|
|
68
|
+
`statusCounts`, `pending`, and `settled` always describe the **whole batch**. `count`
|
|
69
|
+
describes only the rows the call returned.
|
|
70
|
+
|
|
71
|
+
That distinction bites when you filter. Asking for `status: ["READY"]` gives you just the
|
|
72
|
+
accepted rows, which is often what you want for a list — but `count` is then the number of
|
|
73
|
+
accepted files, not the batch size. Read progress from `statusCounts` and `pending`, never
|
|
74
|
+
from `count`.
|
|
75
|
+
|
|
76
|
+
Only `READY`, `REJECTED`, and `ABORTED` are final. `VALIDATING` and `EXPIRED` are both still
|
|
77
|
+
counted in `pending`, so a batch is not settled while either remains.
|
|
78
|
+
|
|
79
|
+
| Status | Means | Final |
|
|
80
|
+
| --- | --- | --- |
|
|
81
|
+
| `CREATED` | Destination minted, bytes not sent yet | no |
|
|
82
|
+
| `UPLOADING` | Transfer in progress | no |
|
|
83
|
+
| `VALIDATING` | Landed, being checked | no |
|
|
84
|
+
| `READY` | Accepted. A `READY` image carries a short-lived `url` | yes |
|
|
85
|
+
| `REJECTED` | Refused — report `rejectionReason` | yes |
|
|
86
|
+
| `ABORTED` | Abandoned | yes |
|
|
87
|
+
| `EXPIRED` | Destination went unused; can still land | no |
|
|
88
|
+
|
|
89
|
+
## When a batch will never settle
|
|
90
|
+
|
|
91
|
+
A batch whose files sit at `CREATED` forever was opened by something that could not send the
|
|
92
|
+
bytes. That happens when `create_upload_batch` is called from a host with no way to make HTTP
|
|
93
|
+
requests. Say so plainly rather than polling for five minutes: the fix is
|
|
94
|
+
`create_browser_upload_link`, and the person uploads from their browser instead.
|
|
95
|
+
|
|
96
|
+
A file stuck at `UPLOADING` on a large video usually means its multipart upload was never
|
|
97
|
+
completed — `complete_upload` has to be called with the part ETags before the object is
|
|
98
|
+
assembled. `list_upload_parts` shows what storage actually holds.
|
|
99
|
+
|
|
100
|
+
## More detail
|
|
101
|
+
|
|
102
|
+
`tools.md` has every tool's arguments and the full error vocabulary.
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# `/extuitive upload` — send creative into a workspace
|
|
2
|
+
|
|
3
|
+
Uploads a set of local files to an Extuitive workspace and reports which ones were accepted.
|
|
4
|
+
|
|
5
|
+
Any folder or file paths the person gave follow the command in `$ARGUMENTS`. If they gave
|
|
6
|
+
none, ask what to upload before doing anything else.
|
|
7
|
+
|
|
8
|
+
## Before you start
|
|
9
|
+
|
|
10
|
+
**The MCP tools never carry file bytes.** `create_upload_batch` returns presigned storage
|
|
11
|
+
URLs; whoever has the files sends the bytes to those URLs directly.
|
|
12
|
+
|
|
13
|
+
So decide which case you are in. The question is whether you can **read the person's files**,
|
|
14
|
+
which is not the same as whether you can run code:
|
|
15
|
+
|
|
16
|
+
- **You can open the paths they gave you.** Use `../scripts/upload.mjs`, as below.
|
|
17
|
+
- **You cannot.** Do not call `create_upload_batch` — a batch you cannot fill sits unfinished
|
|
18
|
+
and later reads as a failed upload. Call `create_browser_upload_link`, give the person the
|
|
19
|
+
link, and skip to step 6.
|
|
20
|
+
|
|
21
|
+
Some hosts run your code in a container that holds this skill but not the person's disk, so a
|
|
22
|
+
working `node` and a readable `../scripts/upload.mjs` prove nothing about `~/creative/`. If a
|
|
23
|
+
path they named does not open, you are in the second case — say so and hand over the link,
|
|
24
|
+
rather than asking them to attach thirty files to the conversation.
|
|
25
|
+
|
|
26
|
+
If the Extuitive tools are missing entirely, read `init.md` instead.
|
|
27
|
+
|
|
28
|
+
## 1. Pick the workspace
|
|
29
|
+
|
|
30
|
+
If one was already chosen earlier in this conversation, use it, and name it in the first thing
|
|
31
|
+
you say — a mistaken choice is cheap to correct before the files move and expensive after.
|
|
32
|
+
|
|
33
|
+
Otherwise call `list_workspaces`. If there is more than one and the person did not say which,
|
|
34
|
+
ask; `select.md` covers how to lay the options out.
|
|
35
|
+
|
|
36
|
+
Ask even when two rows share a `facebookAdAccountId`. They are two workspaces, and the files
|
|
37
|
+
land in whichever one you use. **Do not pick by `role` or `isOwner`** — those govern who can
|
|
38
|
+
reconnect Meta and say nothing about uploading, so a workspace where they are `viewer` may
|
|
39
|
+
take files that the one where they are `owner` refuses. Describe both rows and let them
|
|
40
|
+
choose.
|
|
41
|
+
|
|
42
|
+
If it comes back **empty**, they have no workspace rather than no access. Read `connect.md`.
|
|
43
|
+
There is nothing to upload into yet.
|
|
44
|
+
|
|
45
|
+
## 2. Check the files against the limits
|
|
46
|
+
|
|
47
|
+
Call `get_upload_limits` for the workspace. It returns `maxFiles`, `multipartThresholdBytes`,
|
|
48
|
+
`recommendedPartBytes`, `maxParts`, and separate `maxBytes` and `contentTypes` for images and
|
|
49
|
+
for video — the two ceilings differ by orders of magnitude, so check each file against the one
|
|
50
|
+
for its own kind.
|
|
51
|
+
|
|
52
|
+
Never hardcode these numbers. They are server-owned and they change.
|
|
53
|
+
|
|
54
|
+
Set aside anything oversized or of a disallowed type and tell the person which and why. Do not
|
|
55
|
+
silently drop files.
|
|
56
|
+
|
|
57
|
+
## 3. Open the batch
|
|
58
|
+
|
|
59
|
+
Call `create_upload_batch` with every remaining file declared at once: `fileName`,
|
|
60
|
+
`contentType`, and the real size in `bytes`. Split into several batches if there are more than
|
|
61
|
+
`maxFiles`.
|
|
62
|
+
|
|
63
|
+
You get back a `batchId` and one destination per file, in the order you sent them. Keep the
|
|
64
|
+
`batchId` — step 6 and `upload-status.md` both need it.
|
|
65
|
+
|
|
66
|
+
If it refuses with `workspace_access_denied`, that workspace will not take these files no
|
|
67
|
+
matter what its row said. Move to another workspace they have, and tell them the batch is in
|
|
68
|
+
that one instead — including if it contradicts a workspace you named earlier. Do not retry the
|
|
69
|
+
refused id, and do not explain the refusal in terms of their role; it does not follow from it.
|
|
70
|
+
|
|
71
|
+
Each destination is one of two kinds:
|
|
72
|
+
|
|
73
|
+
- **`PUT`** — the whole file in one request. Carries `url`, `headers`, and `expiresIn`.
|
|
74
|
+
- **`MULTIPART`** — a large video. Carries `uploadId` and `partBytes`, and needs step 4 first.
|
|
75
|
+
|
|
76
|
+
## 4. Sign the parts of any MULTIPART file
|
|
77
|
+
|
|
78
|
+
Skip this if every destination is a `PUT`. Images never take the multipart path.
|
|
79
|
+
|
|
80
|
+
Work out the part count yourself: `ceil(bytes / partBytes)`, using the `bytes` you declared in
|
|
81
|
+
step 3 and the `partBytes` on that destination. You do not need to touch the file to do this.
|
|
82
|
+
|
|
83
|
+
Then call `sign_upload_part` for each part number from 1 to N. Each returns `{ url, headers,
|
|
84
|
+
expiresIn }`. Signing needs only the part number — no bytes, no checksum — so sign them all up
|
|
85
|
+
front, before any transfer starts.
|
|
86
|
+
|
|
87
|
+
Attach them to the destination as a `parts` array, each entry `{ partNumber, url, headers }`.
|
|
88
|
+
|
|
89
|
+
A 4 GB video is roughly 256 parts. If that is more calls than you want to make in one go, say
|
|
90
|
+
so and offer `create_browser_upload_link`, which handles large video without any of this.
|
|
91
|
+
|
|
92
|
+
## 5. Send the bytes
|
|
93
|
+
|
|
94
|
+
Write a plan file pairing each local path with its destination:
|
|
95
|
+
|
|
96
|
+
```json
|
|
97
|
+
{
|
|
98
|
+
"workspaceId": "<workspace id>",
|
|
99
|
+
"files": [
|
|
100
|
+
{
|
|
101
|
+
"path": "/abs/path/one.png",
|
|
102
|
+
"destination": { "method": "PUT", "...": "..." }
|
|
103
|
+
},
|
|
104
|
+
{
|
|
105
|
+
"path": "/abs/path/big.mp4",
|
|
106
|
+
"destination": {
|
|
107
|
+
"method": "MULTIPART",
|
|
108
|
+
"uploadId": "...",
|
|
109
|
+
"partBytes": 16777216,
|
|
110
|
+
"parts": [{ "partNumber": 1, "url": "...", "headers": {} }]
|
|
111
|
+
}
|
|
112
|
+
}
|
|
113
|
+
]
|
|
114
|
+
}
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
Run the script using the absolute path of the skill directory this file sits in:
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
node <skill directory>/scripts/upload.mjs plan.json
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
It prints a JSON report with `uploaded`, `multipart`, `failed`, `needsResign`, and
|
|
124
|
+
`needsPartResign`. It handles retries, concurrency, and the CRC32C checksum each multipart
|
|
125
|
+
chunk needs. It holds no credential and makes no MCP call — it only sends bytes to URLs you
|
|
126
|
+
signed.
|
|
127
|
+
|
|
128
|
+
Three things it hands back rather than solving itself:
|
|
129
|
+
|
|
130
|
+
- **`multipart`** lists each large file with its `uploadId` and the `{ PartNumber, ETag }`
|
|
131
|
+
array storage returned. Call `complete_upload` once per entry, passing those parts through
|
|
132
|
+
exactly as given. Note the capitalisation — `PartNumber` and `ETag` are S3's own casing and
|
|
133
|
+
the only fields in this surface that are not camelCase. **A multipart file is not uploaded
|
|
134
|
+
until you do this.**
|
|
135
|
+
- **`needsResign`** lists `contentId`s whose presigned PUT expired. Every URL in a batch is
|
|
136
|
+
signed at the same moment and lasts about 30 minutes, so in a large batch the last files can
|
|
137
|
+
expire while the first are still going. Call `resign_upload` for each, put the fresh
|
|
138
|
+
destination in a new plan, and run the script again for just those.
|
|
139
|
+
- **`needsPartResign`** is the same problem on one part of a multipart file. Call
|
|
140
|
+
`sign_upload_part` again for that part number and re-run with just that file.
|
|
141
|
+
|
|
142
|
+
## 6. Report what was actually accepted
|
|
143
|
+
|
|
144
|
+
**A finished transfer is not an accepted file.** The bytes arriving only means storage has
|
|
145
|
+
them; the file is then checked, and it can still be rejected.
|
|
146
|
+
|
|
147
|
+
Poll `get_upload_batch_content` with the `batchId` about every 5 seconds until `settled` is
|
|
148
|
+
`true`. Give up after about 5 minutes and say so rather than claiming success.
|
|
149
|
+
|
|
150
|
+
**Report as you go — do not go silent.** Roughly every 15–30 seconds, say how many have
|
|
151
|
+
reached `READY` out of the batch, so a long upload does not look like a hang. Report once more
|
|
152
|
+
when `settled` is `true`.
|
|
153
|
+
|
|
154
|
+
`upload-status.md` covers the polling, the status table, and the reporting rules in full. Read
|
|
155
|
+
it rather than repeating the logic here.
|
|
156
|
+
|
|
157
|
+
## 7. Stop there
|
|
158
|
+
|
|
159
|
+
Report what landed and wait. Uploading is not an instruction to do anything further with the
|
|
160
|
+
files.
|
|
@@ -0,0 +1,340 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Sends creative files to storage for a batch the MCP tools already opened.
|
|
4
|
+
*
|
|
5
|
+
* This exists because the MCP tools deliberately never carry bytes: `create_upload_batch`
|
|
6
|
+
* returns presigned destinations and something has to PUT to them. Doing that by hand from a
|
|
7
|
+
* chat turn is not practical — every part of a large video needs a CRC32C over its own
|
|
8
|
+
* bytes, plus retries.
|
|
9
|
+
*
|
|
10
|
+
* **This script holds no credential and makes no MCP call.** It is handed presigned URLs and
|
|
11
|
+
* it sends bytes to them. Signing, opening the batch, completing a multipart upload, and
|
|
12
|
+
* reporting the outcome are all tool calls the agent makes, which is what keeps the access
|
|
13
|
+
* token in the host's credential store where it belongs. An earlier version called
|
|
14
|
+
* `sign_upload_part` itself over raw HTTP and therefore needed a token in its plan; that was
|
|
15
|
+
* a choice this script made, not something the protocol required, and it is gone.
|
|
16
|
+
*
|
|
17
|
+
* Usage: node upload.mjs plan.json (or: cat plan.json | node upload.mjs -)
|
|
18
|
+
*
|
|
19
|
+
* Plan:
|
|
20
|
+
* {
|
|
21
|
+
* "workspaceId": "...",
|
|
22
|
+
* "files": [
|
|
23
|
+
* { "path": "/abs/a.png", "destination": { …a PUT entry from create_upload_batch… } },
|
|
24
|
+
* {
|
|
25
|
+
* "path": "/abs/big.mp4",
|
|
26
|
+
* "destination": {
|
|
27
|
+
* "method": "MULTIPART",
|
|
28
|
+
* "contentId": "...", "fileName": "...", "uploadId": "...", "partBytes": 16777216,
|
|
29
|
+
* // One entry per part, from sign_upload_part. Parts are numbered from 1.
|
|
30
|
+
* "parts": [{ "partNumber": 1, "url": "...", "headers": { … } }]
|
|
31
|
+
* }
|
|
32
|
+
* }
|
|
33
|
+
* ]
|
|
34
|
+
* }
|
|
35
|
+
*
|
|
36
|
+
* Prints a JSON report on stdout. Exit code is 0 when every file landed, 1 otherwise.
|
|
37
|
+
*/
|
|
38
|
+
import { readFile, open, stat } from "node:fs/promises";
|
|
39
|
+
|
|
40
|
+
/** How many files move at once. Files, not parts: a multipart file is already several requests. */
|
|
41
|
+
const FILE_CONCURRENCY = 4;
|
|
42
|
+
const ATTEMPTS = 4;
|
|
43
|
+
const RETRY_BASE_MS = 500;
|
|
44
|
+
|
|
45
|
+
/** Castagnoli polynomial, reversed. Not the CRC32 used by zip — they are not interchangeable. */
|
|
46
|
+
const CRC32C_POLYNOMIAL = 0x82f63b78;
|
|
47
|
+
|
|
48
|
+
let crcTable = null;
|
|
49
|
+
|
|
50
|
+
function getCrcTable() {
|
|
51
|
+
if (crcTable !== null) {
|
|
52
|
+
return crcTable;
|
|
53
|
+
}
|
|
54
|
+
const table = new Uint32Array(256);
|
|
55
|
+
for (let index = 0; index < 256; index += 1) {
|
|
56
|
+
let value = index;
|
|
57
|
+
for (let bit = 0; bit < 8; bit += 1) {
|
|
58
|
+
value = (value & 1) === 1 ? (value >>> 1) ^ CRC32C_POLYNOMIAL : value >>> 1;
|
|
59
|
+
}
|
|
60
|
+
table[index] = value >>> 0;
|
|
61
|
+
}
|
|
62
|
+
crcTable = table;
|
|
63
|
+
return table;
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
/**
|
|
67
|
+
* The value S3 expects for `x-amz-checksum-crc32c`: big-endian four bytes, base64.
|
|
68
|
+
*
|
|
69
|
+
* Byte order is not cosmetic. Little-endian produces a well-formed base64 string that S3
|
|
70
|
+
* rejects on every part, which reads as a signing problem rather than a checksum one.
|
|
71
|
+
*/
|
|
72
|
+
function crc32cBase64(bytes) {
|
|
73
|
+
const table = getCrcTable();
|
|
74
|
+
let crc = 0xffffffff;
|
|
75
|
+
for (let index = 0; index < bytes.length; index += 1) {
|
|
76
|
+
crc = (crc >>> 8) ^ table[(crc ^ bytes[index]) & 0xff];
|
|
77
|
+
}
|
|
78
|
+
const checksum = (crc ^ 0xffffffff) >>> 0;
|
|
79
|
+
|
|
80
|
+
const view = Buffer.alloc(4);
|
|
81
|
+
view.writeUInt32BE(checksum, 0);
|
|
82
|
+
return view.toString("base64");
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
function sleep(ms) {
|
|
86
|
+
return new Promise((resolve) => setTimeout(resolve, ms));
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
class UploadFailure extends Error {
|
|
90
|
+
constructor(message, options = {}) {
|
|
91
|
+
super(message);
|
|
92
|
+
this.name = "UploadFailure";
|
|
93
|
+
this.needsResign = options.needsResign === true;
|
|
94
|
+
this.partNumber = options.partNumber ?? null;
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
/**
|
|
99
|
+
* Retries only what retrying can fix.
|
|
100
|
+
*
|
|
101
|
+
* A 403 on a presigned URL is almost always an expired signature, and re-sending the same
|
|
102
|
+
* URL will fail identically however many times it is tried — it needs a new signature, which
|
|
103
|
+
* only the agent can ask for. Reporting it as needing a re-sign is the useful answer;
|
|
104
|
+
* retrying it would just spend four attempts arriving at the same place.
|
|
105
|
+
*/
|
|
106
|
+
async function sendWithRetry(label, send, { partNumber = null } = {}) {
|
|
107
|
+
let lastError = null;
|
|
108
|
+
|
|
109
|
+
for (let attempt = 1; attempt <= ATTEMPTS; attempt += 1) {
|
|
110
|
+
try {
|
|
111
|
+
const response = await send();
|
|
112
|
+
if (response.ok === true) {
|
|
113
|
+
return response;
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
if (response.status === 403) {
|
|
117
|
+
throw new UploadFailure(`${label}: signature rejected or expired (403).`, {
|
|
118
|
+
needsResign: true,
|
|
119
|
+
partNumber,
|
|
120
|
+
});
|
|
121
|
+
}
|
|
122
|
+
if (response.status < 500 && response.status !== 429) {
|
|
123
|
+
throw new UploadFailure(`${label}: refused with ${response.status}.`);
|
|
124
|
+
}
|
|
125
|
+
lastError = new UploadFailure(`${label}: ${response.status} from storage.`);
|
|
126
|
+
} catch (error) {
|
|
127
|
+
if (error instanceof UploadFailure && error.needsResign === true) {
|
|
128
|
+
throw error;
|
|
129
|
+
}
|
|
130
|
+
lastError = error instanceof Error ? error : new Error(String(error));
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
if (attempt < ATTEMPTS) {
|
|
134
|
+
await sleep(RETRY_BASE_MS * 2 ** (attempt - 1));
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
throw lastError ?? new UploadFailure(`${label}: failed.`);
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
/** One request, whole file. Headers go verbatim: the signature covers them. */
|
|
142
|
+
async function sendSinglePut(file, destination) {
|
|
143
|
+
const body = await readFile(file.path);
|
|
144
|
+
|
|
145
|
+
await sendWithRetry(destination.fileName, () =>
|
|
146
|
+
fetch(destination.url, {
|
|
147
|
+
method: "PUT",
|
|
148
|
+
headers: destination.headers ?? {},
|
|
149
|
+
body,
|
|
150
|
+
}),
|
|
151
|
+
);
|
|
152
|
+
|
|
153
|
+
return {
|
|
154
|
+
kind: "put",
|
|
155
|
+
contentId: destination.contentId,
|
|
156
|
+
fileName: destination.fileName,
|
|
157
|
+
};
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
/**
|
|
161
|
+
* Send every chunk of a multipart upload against signatures the agent already obtained.
|
|
162
|
+
*
|
|
163
|
+
* The part count is checked rather than trusted. The agent derives it from the `bytes` it
|
|
164
|
+
* declared in the manifest, so a file edited between declaring and uploading would leave the
|
|
165
|
+
* last chunk unsent and the object silently short — cheaper to refuse here than to discover
|
|
166
|
+
* after `complete_upload` assembled something wrong.
|
|
167
|
+
*/
|
|
168
|
+
async function sendMultipart(file, destination) {
|
|
169
|
+
const { size } = await stat(file.path);
|
|
170
|
+
const partBytes = destination.partBytes;
|
|
171
|
+
|
|
172
|
+
if (typeof partBytes !== "number" || partBytes <= 0) {
|
|
173
|
+
throw new UploadFailure(`${destination.fileName}: destination has no usable partBytes.`);
|
|
174
|
+
}
|
|
175
|
+
|
|
176
|
+
const signedParts = Array.isArray(destination.parts) === true ? destination.parts : [];
|
|
177
|
+
if (signedParts.length === 0) {
|
|
178
|
+
throw new UploadFailure(
|
|
179
|
+
`${destination.fileName}: no presigned parts in the plan. Call sign_upload_part for parts 1..N and put them on the destination as "parts".`,
|
|
180
|
+
);
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
const expectedCount = Math.max(1, Math.ceil(size / partBytes));
|
|
184
|
+
if (signedParts.length !== expectedCount) {
|
|
185
|
+
throw new UploadFailure(
|
|
186
|
+
`${destination.fileName}: ${signedParts.length} presigned part(s) for a file needing ${expectedCount} at ${partBytes} bytes each. The file on disk is ${size} bytes; re-declare it and sign again.`,
|
|
187
|
+
);
|
|
188
|
+
}
|
|
189
|
+
|
|
190
|
+
const byNumber = new Map();
|
|
191
|
+
for (const part of signedParts) {
|
|
192
|
+
byNumber.set(part.partNumber, part);
|
|
193
|
+
}
|
|
194
|
+
|
|
195
|
+
const parts = [];
|
|
196
|
+
const handle = await open(file.path, "r");
|
|
197
|
+
|
|
198
|
+
try {
|
|
199
|
+
for (let partNumber = 1; partNumber <= expectedCount; partNumber += 1) {
|
|
200
|
+
const signed = byNumber.get(partNumber);
|
|
201
|
+
if (signed === undefined) {
|
|
202
|
+
throw new UploadFailure(
|
|
203
|
+
`${destination.fileName}: no presigned URL for part ${partNumber}.`,
|
|
204
|
+
{ partNumber },
|
|
205
|
+
);
|
|
206
|
+
}
|
|
207
|
+
|
|
208
|
+
const offset = (partNumber - 1) * partBytes;
|
|
209
|
+
const length = Math.min(partBytes, size - offset);
|
|
210
|
+
const chunk = Buffer.alloc(length);
|
|
211
|
+
await handle.read(chunk, 0, length, offset);
|
|
212
|
+
|
|
213
|
+
// Computed over the exact bytes being sent. The upstream signed this header, so S3
|
|
214
|
+
// rejects the part if it is absent or does not match.
|
|
215
|
+
const checksum = crc32cBase64(chunk);
|
|
216
|
+
|
|
217
|
+
const response = await sendWithRetry(
|
|
218
|
+
`${destination.fileName} part ${partNumber}`,
|
|
219
|
+
() =>
|
|
220
|
+
fetch(signed.url, {
|
|
221
|
+
method: "PUT",
|
|
222
|
+
headers: { ...signed.headers, "x-amz-checksum-crc32c": checksum },
|
|
223
|
+
body: chunk,
|
|
224
|
+
}),
|
|
225
|
+
{ partNumber },
|
|
226
|
+
);
|
|
227
|
+
|
|
228
|
+
const eTag = response.headers.get("etag");
|
|
229
|
+
if (eTag === null) {
|
|
230
|
+
throw new UploadFailure(
|
|
231
|
+
`${destination.fileName} part ${partNumber}: storage returned no ETag.`,
|
|
232
|
+
{ partNumber },
|
|
233
|
+
);
|
|
234
|
+
}
|
|
235
|
+
parts.push({ PartNumber: partNumber, ETag: eTag });
|
|
236
|
+
}
|
|
237
|
+
} finally {
|
|
238
|
+
await handle.close();
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
return {
|
|
242
|
+
kind: "multipart",
|
|
243
|
+
contentId: destination.contentId,
|
|
244
|
+
fileName: destination.fileName,
|
|
245
|
+
uploadId: destination.uploadId,
|
|
246
|
+
parts,
|
|
247
|
+
};
|
|
248
|
+
}
|
|
249
|
+
|
|
250
|
+
async function sendOne(file) {
|
|
251
|
+
const destination = file.destination;
|
|
252
|
+
return destination.method === "MULTIPART"
|
|
253
|
+
? sendMultipart(file, destination)
|
|
254
|
+
: sendSinglePut(file, destination);
|
|
255
|
+
}
|
|
256
|
+
|
|
257
|
+
async function run(plan) {
|
|
258
|
+
const queue = [...plan.files];
|
|
259
|
+
const uploaded = [];
|
|
260
|
+
const multipart = [];
|
|
261
|
+
const failed = [];
|
|
262
|
+
const needsResign = [];
|
|
263
|
+
const needsPartResign = [];
|
|
264
|
+
|
|
265
|
+
async function worker() {
|
|
266
|
+
for (;;) {
|
|
267
|
+
const file = queue.shift();
|
|
268
|
+
if (file === undefined) {
|
|
269
|
+
return;
|
|
270
|
+
}
|
|
271
|
+
|
|
272
|
+
const destination = file.destination ?? {};
|
|
273
|
+
|
|
274
|
+
try {
|
|
275
|
+
const result = await sendOne(file);
|
|
276
|
+
if (result.kind === "multipart") {
|
|
277
|
+
multipart.push({
|
|
278
|
+
contentId: result.contentId,
|
|
279
|
+
fileName: result.fileName,
|
|
280
|
+
uploadId: result.uploadId,
|
|
281
|
+
parts: result.parts,
|
|
282
|
+
});
|
|
283
|
+
} else {
|
|
284
|
+
uploaded.push({ contentId: result.contentId, fileName: result.fileName });
|
|
285
|
+
}
|
|
286
|
+
} catch (error) {
|
|
287
|
+
const entry = {
|
|
288
|
+
fileName: destination.fileName ?? file.path,
|
|
289
|
+
contentId: destination.contentId,
|
|
290
|
+
error: error instanceof Error ? error.message : String(error),
|
|
291
|
+
};
|
|
292
|
+
failed.push(entry);
|
|
293
|
+
|
|
294
|
+
if (error instanceof UploadFailure && error.needsResign === true) {
|
|
295
|
+
if (destination.method === "MULTIPART") {
|
|
296
|
+
needsPartResign.push({
|
|
297
|
+
fileName: entry.fileName,
|
|
298
|
+
contentId: entry.contentId,
|
|
299
|
+
uploadId: destination.uploadId,
|
|
300
|
+
partNumber: error.partNumber,
|
|
301
|
+
});
|
|
302
|
+
} else {
|
|
303
|
+
needsResign.push(entry.contentId);
|
|
304
|
+
}
|
|
305
|
+
}
|
|
306
|
+
}
|
|
307
|
+
}
|
|
308
|
+
}
|
|
309
|
+
|
|
310
|
+
await Promise.all(
|
|
311
|
+
Array.from({ length: Math.min(FILE_CONCURRENCY, plan.files.length) }, worker),
|
|
312
|
+
);
|
|
313
|
+
|
|
314
|
+
return { uploaded, multipart, failed, needsResign, needsPartResign };
|
|
315
|
+
}
|
|
316
|
+
|
|
317
|
+
async function readPlan(source) {
|
|
318
|
+
if (source === "-" || source === undefined) {
|
|
319
|
+
const chunks = [];
|
|
320
|
+
for await (const chunk of process.stdin) {
|
|
321
|
+
chunks.push(chunk);
|
|
322
|
+
}
|
|
323
|
+
return JSON.parse(Buffer.concat(chunks).toString("utf8"));
|
|
324
|
+
}
|
|
325
|
+
return JSON.parse(await readFile(source, "utf8"));
|
|
326
|
+
}
|
|
327
|
+
|
|
328
|
+
const plan = await readPlan(process.argv[2]);
|
|
329
|
+
if (Array.isArray(plan.files) === false || plan.files.length === 0) {
|
|
330
|
+
console.error("Plan needs a non-empty files array.");
|
|
331
|
+
process.exit(2);
|
|
332
|
+
}
|
|
333
|
+
|
|
334
|
+
const report = await run(plan);
|
|
335
|
+
console.log(JSON.stringify(report, null, 2));
|
|
336
|
+
|
|
337
|
+
// Transfers finishing is not the same as files being accepted. Every entry in `multipart`
|
|
338
|
+
// still needs a complete_upload call, and everything still needs polling with
|
|
339
|
+
// get_upload_batch_content; this only reports what reached storage.
|
|
340
|
+
process.exit(report.failed.length === 0 ? 0 : 1);
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Values that appear in more than one place, defined once.
|
|
3
|
+
*/
|
|
4
|
+
|
|
5
|
+
/**
|
|
6
|
+
* The name people type. This is the unscoped launcher package on npm (`packages/extuitive`),
|
|
7
|
+
* which depends on `@extuitive/skill` — the package this file ships in — and does nothing
|
|
8
|
+
* but import `bin/cli.mjs`. The usage text shows the launcher's name because that is the
|
|
9
|
+
* command, not the implementation.
|
|
10
|
+
*/
|
|
11
|
+
export const PACKAGE_NAME = "extuitive";
|
|
12
|
+
|
|
13
|
+
/** `owner/repo`, the one string that has to change if the repository moves. */
|
|
14
|
+
export const GITHUB_REPO = "fl100inc/extuitive-skill";
|
|
15
|
+
|
|
16
|
+
/**
|
|
17
|
+
* How to invoke this tool, for every message that tells someone to run it again.
|
|
18
|
+
*
|
|
19
|
+
* `npx extuitive` resolves the launcher from the npm registry, so it works from any
|
|
20
|
+
* directory with nothing installed. `npx github:fl100inc/extuitive-skill` still runs the
|
|
21
|
+
* unreleased `main` for anyone who wants that, but it is not what a fix message should
|
|
22
|
+
* suggest.
|
|
23
|
+
*/
|
|
24
|
+
export const NPX_COMMAND = `npx ${PACKAGE_NAME}`;
|
|
25
|
+
|
|
26
|
+
/** Overridable with `--endpoint` for development against a local dev server. */
|
|
27
|
+
export const DEFAULT_MCP_ENDPOINT = "https://www.extuitive.com/mcp";
|
|
28
|
+
|
|
29
|
+
/**
|
|
30
|
+
* The name every host registers the server under.
|
|
31
|
+
*
|
|
32
|
+
* Not one of Claude Code's reserved names (`workspace`, `claude-in-chrome`, `computer-use`,
|
|
33
|
+
* `Claude Preview`, `Claude Browser`), which it refuses at add time.
|
|
34
|
+
*/
|
|
35
|
+
export const MCP_SERVER_NAME = "extuitive";
|
|
36
|
+
|
|
37
|
+
/**
|
|
38
|
+
* One skill, whose directory name is also its command name.
|
|
39
|
+
*
|
|
40
|
+
* The CLI hosts key a skill on its directory, so this string is what someone types:
|
|
41
|
+
* `/extuitive` in Claude Code, `$extuitive` in Codex. Claude Desktop has no prefix at all
|
|
42
|
+
* and selects on the description instead, which is why `invocationNote` exists on a host
|
|
43
|
+
* rather than a prefix being assumed everywhere.
|
|
44
|
+
*
|
|
45
|
+
* The individual jobs are arguments to it rather than skills of their own — `/extuitive
|
|
46
|
+
* upload` — which is why there is only one entry here. It must equal the `name` in
|
|
47
|
+
* SKILL.md's frontmatter.
|
|
48
|
+
*/
|
|
49
|
+
export const SKILL_NAMES = ["extuitive"];
|
|
50
|
+
|
|
51
|
+
/** The subcommands the skill routes, used for the usage text and nothing else. */
|
|
52
|
+
export const SKILL_COMMANDS = ["init", "select", "upload", "upload-status", "connect"];
|
|
53
|
+
|
|
54
|
+
/**
|
|
55
|
+
* What to suggest typing once the install is done.
|
|
56
|
+
*
|
|
57
|
+
* Natural language rather than `$extuitive init`, because that is how the skill is meant to
|
|
58
|
+
* be reached and because the slash-vs-dollar prefix differs by host. The first one routes to
|
|
59
|
+
* `init`, which verifies the connection end to end and is the right first thing to do; the
|
|
60
|
+
* second is the job most people installed it for. Both phrases appear in SKILL.md's
|
|
61
|
+
* description so the host's skill matcher recognises them.
|
|
62
|
+
*/
|
|
63
|
+
export const EXAMPLE_PROMPTS = ["Check my Extuitive connection", "Upload these images to Extuitive"];
|