pi-tsgw 0.0.13 → 0.0.15
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +57 -0
- package/extensions/index.ts +5 -0
- package/extensions/mcp.ts +26 -0
- package/extensions/models/vendors/deepseek.ts +10 -5
- package/extensions/upload/README.md +146 -0
- package/extensions/upload/errors.ts +15 -0
- package/extensions/upload/index.ts +62 -0
- package/extensions/upload/local-file.ts +93 -0
- package/extensions/upload/protocol.ts +177 -0
- package/extensions/upload/transport.ts +216 -0
- package/extensions/upload/uploader.ts +214 -0
- package/package.json +6 -2
- package/skills/delegate-model-selection/SKILL.md +50 -0
package/README.md
CHANGED
|
@@ -37,6 +37,7 @@ Example:
|
|
|
37
37
|
{
|
|
38
38
|
"tsgw": {
|
|
39
39
|
"baseUrl": "https://aih.example.net",
|
|
40
|
+
"mcpUrl": "https://xn--oht.cc/mcp",
|
|
40
41
|
"tsSearch": "live",
|
|
41
42
|
"traceHeaders": false,
|
|
42
43
|
"excludeModels": ["gpt-*", "claude-*"],
|
|
@@ -69,6 +70,7 @@ catalog may fall back to static models until Pi refreshes it.
|
|
|
69
70
|
| Setting (`settings.json` `tsgw.*`) | Default | Meaning |
|
|
70
71
|
| --- | --- | --- |
|
|
71
72
|
| `baseUrl` | placeholder | Gateway root (the `/v1` suffix is normalized away) |
|
|
73
|
+
| `mcpUrl` | `https://xn--oht.cc/mcp` | Optional `ts_oht` HTTP(S) endpoint override; independent of the model gateway |
|
|
72
74
|
| `tsSearch` | `live` | GPT: `cached` or `live`; unset, legacy `off`, and invalid values fall back to `live`. Supported Grok models always use live search. |
|
|
73
75
|
| `traceHeaders` | `false` | Adds `AH-Thread-Id` / `AH-Trace-Id` headers for gateway-side tracing |
|
|
74
76
|
| `includeModels` | unset | Re-include override: matching model ids are kept even when also matched by `excludeModels` (exact id or `prefix-*` glob) |
|
|
@@ -179,6 +181,61 @@ buttons. Busy sessions disable the buttons. The terminal CLI shows the same
|
|
|
179
181
|
statuses as text; use the slash commands there. Pi Web may not show extension
|
|
180
182
|
statuses until the session runtime has started.
|
|
181
183
|
|
|
184
|
+
## Subagent orchestration skill
|
|
185
|
+
|
|
186
|
+
The package ships [delegate-model-selection](skills/delegate-model-selection/SKILL.md),
|
|
187
|
+
which encourages suitable tasks to run through background subagents and provides
|
|
188
|
+
model/thinking-level guidance, including independent Kimi + Astra reviews.
|
|
189
|
+
Task splitting, parallelism, validation, and escalation remain the agent's choice.
|
|
190
|
+
|
|
191
|
+
The plugin maintains the reference table in the same `SKILL.md`
|
|
192
|
+
for direct use, without asking agents to check live rankings or data freshness.
|
|
193
|
+
It is guidance, not a model whitelist or fixed execution sequence. Prices are
|
|
194
|
+
API reference rates, not subscription charges. GLM guidance uses 5.3, with Flash
|
|
195
|
+
and FlashX as alternatives. Thinking levels use Pi names: M3 recommends `high`,
|
|
196
|
+
which maps to adaptive like its other non-off levels.
|
|
197
|
+
|
|
198
|
+
Pi discovers the skill through the package manifest after installation and
|
|
199
|
+
`/reload`, unless skills are disabled or filtered by the host/package settings.
|
|
200
|
+
Loading only `extensions/index.ts` does not load package skills. A host-provided
|
|
201
|
+
subagent tool with model selection and available `tsgw` models is required; this
|
|
202
|
+
package does not supply that tool or change dispatch settings, user rules, or
|
|
203
|
+
permissions.
|
|
204
|
+
|
|
205
|
+
## Default MCP registration
|
|
206
|
+
|
|
207
|
+
The plugin registers `ts_oht` at `https://xn--oht.cc/mcp` with `codemode`
|
|
208
|
+
exposure on each load. No MCP configuration is required to use this default.
|
|
209
|
+
An optional `tsgw.mcpUrl` in the user-level settings changes the plugin's endpoint;
|
|
210
|
+
it is independent of `tsgw.baseUrl` and is read once at initialization. Invalid
|
|
211
|
+
explicit URLs skip this registration with a safe warning, rather than silently
|
|
212
|
+
connecting to the default. Older hosts without `registerMcpServer` skip it.
|
|
213
|
+
|
|
214
|
+
Pi manages connections, OAuth sign-in and credentials. Use `/mcp login ts_oht`
|
|
215
|
+
inside a session if sign-in is required. The plugin does not read or write
|
|
216
|
+
`mcp.json` or `mcp-auth.json`, and does not carry credentials. Keep the same
|
|
217
|
+
server name and exact URL to reuse the host's existing OAuth scope.
|
|
218
|
+
|
|
219
|
+
A same-name `ts_oht` in the effective global/project `mcp.json` takes precedence
|
|
220
|
+
over the plugin registration, including `enabled: false`; the plugin does not
|
|
221
|
+
create a second connection or bypass that choice. Use a complete same-name
|
|
222
|
+
file configuration to override the URL, exposure, timeout, auth, or enabled state.
|
|
223
|
+
Changes to a plugin-provided server apply only to the current session; they are
|
|
224
|
+
not saved. Shell `pi mcp` commands do not load plugins and cannot see servers
|
|
225
|
+
provided only by this registration. Reload or restart after configuration changes.
|
|
226
|
+
|
|
227
|
+
## Local file upload
|
|
228
|
+
|
|
229
|
+
The `upload_local_file` tool uploads an authorized file from the Pi host directly
|
|
230
|
+
to S3 using the already-connected `ts_oht` MCP for signing. It supports single
|
|
231
|
+
PUTs and streaming multipart uploads, explicit recovery, custom download expiry
|
|
232
|
+
and inline/attachment links. It uses the host-managed connection and stores no credentials.
|
|
233
|
+
|
|
234
|
+
Upload support requires a host with `ctx.executeTool` (tested with Pi 1.1.0) and
|
|
235
|
+
callable `ts_oht` upload tools. File access uses host OS permissions, not a
|
|
236
|
+
sandbox. See [upload usage and safety boundaries](extensions/upload/README.md)
|
|
237
|
+
before enabling it in an environment with sensitive files.
|
|
238
|
+
|
|
182
239
|
## Development
|
|
183
240
|
|
|
184
241
|
```bash
|
package/extensions/index.ts
CHANGED
|
@@ -28,6 +28,8 @@ import {
|
|
|
28
28
|
type CatalogStatus,
|
|
29
29
|
} from "./status.ts";
|
|
30
30
|
import type { ThinkingLevel, WebSearchMode } from "./models/_tools.ts";
|
|
31
|
+
import { registerUploadTool } from "./upload/index.ts";
|
|
32
|
+
import { registerTsgwMcp } from "./mcp.ts";
|
|
31
33
|
|
|
32
34
|
/**
|
|
33
35
|
* The data needed by provider hooks, copied while the lifecycle context is
|
|
@@ -55,6 +57,7 @@ interface RequestModel {
|
|
|
55
57
|
*/
|
|
56
58
|
interface TsgwSettings {
|
|
57
59
|
baseUrl?: string;
|
|
60
|
+
mcpUrl?: string;
|
|
58
61
|
tsSearch?: string;
|
|
59
62
|
traceHeaders?: boolean;
|
|
60
63
|
includeModels?: string[];
|
|
@@ -195,7 +198,9 @@ async function resolveEffectiveModels(
|
|
|
195
198
|
}
|
|
196
199
|
|
|
197
200
|
export default async function registerTsgw(pi: ExtensionAPI): Promise<void> {
|
|
201
|
+
registerUploadTool(pi);
|
|
198
202
|
const settings = readTsgwSettings();
|
|
203
|
+
registerTsgwMcp(pi, settings.mcpUrl);
|
|
199
204
|
const root = rootForRuntime(settings.baseUrl);
|
|
200
205
|
const defaultSearchMode: WebSearchMode =
|
|
201
206
|
settings.tsSearch === "cached" || settings.tsSearch === "live"
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
2
|
+
|
|
3
|
+
export const TS_OHT_NAME = "ts_oht";
|
|
4
|
+
export const TS_OHT_URL = "https://xn--oht.cc/mcp";
|
|
5
|
+
|
|
6
|
+
/** Session default only; Pi owns file overrides, connections and OAuth. */
|
|
7
|
+
export function registerTsgwMcp(pi: ExtensionAPI, configuredUrl?: unknown): void {
|
|
8
|
+
const value = configuredUrl === undefined ? TS_OHT_URL : configuredUrl;
|
|
9
|
+
try {
|
|
10
|
+
if (typeof value !== "string" || value.trim() !== value) throw new Error();
|
|
11
|
+
const url = new URL(value);
|
|
12
|
+
if (!["https:", "http:"].includes(url.protocol) || url.username || url.password || url.hash)
|
|
13
|
+
throw new Error();
|
|
14
|
+
} catch {
|
|
15
|
+
// Invalid explicit configuration must not connect to a different default.
|
|
16
|
+
// Never include URLs or parser errors in ordinary logs.
|
|
17
|
+
console.error("TSGW: invalid mcpUrl; ts_oht registration skipped.");
|
|
18
|
+
return;
|
|
19
|
+
}
|
|
20
|
+
if (typeof pi.registerMcpServer !== "function") {
|
|
21
|
+
console.error("TSGW: this Pi host cannot register MCP servers; ts_oht registration skipped.");
|
|
22
|
+
return;
|
|
23
|
+
}
|
|
24
|
+
// Preserve the exact URL because Pi binds OAuth to server name + URL.
|
|
25
|
+
pi.registerMcpServer(TS_OHT_NAME, { url: value as string, exposure: "codemode" });
|
|
26
|
+
}
|
|
@@ -9,7 +9,12 @@ const DEEPSEEK_RESPONSES_COMPAT = {
|
|
|
9
9
|
|
|
10
10
|
/**
|
|
11
11
|
* DeepSeek(深度求索)系列:V4.1 Flash 与 V4 Pro。
|
|
12
|
-
*
|
|
12
|
+
* 协议文档:https://api-docs.deepseek.com/guides/responses_api
|
|
13
|
+
* 定价依据(2026-10-16):https://api-docs.deepseek.com/zh-cn/quick_start/pricing/
|
|
14
|
+
* 国内高峰价(元/百万 tokens):Flash 输入/输出/命中 2/8/0.04,Pro 9/27/0.30。
|
|
15
|
+
* 按项目固定 0.1475 USD/CNY 折算;静态 cost 用高峰价保守估算,不按请求时间切换。
|
|
16
|
+
* 空闲时段为半价;高峰为北京时间工作日(不含中国法定节假日)9–12、14–18。
|
|
17
|
+
* cacheWrite=0 表示官方未单列附加缓存写入费用;非网关实际计费。
|
|
13
18
|
*/
|
|
14
19
|
export function deepseekModels(root: string): ProviderModelConfig[] {
|
|
15
20
|
const v1 = `${root}/v1`;
|
|
@@ -21,7 +26,7 @@ export function deepseekModels(root: string): ProviderModelConfig[] {
|
|
|
21
26
|
baseUrl: v1,
|
|
22
27
|
reasoning: true,
|
|
23
28
|
input: ["text", "image"],
|
|
24
|
-
cost: { input: 0.
|
|
29
|
+
cost: { input: 0.295, output: 1.18, cacheRead: 0.0059, cacheWrite: 0 },
|
|
25
30
|
contextWindow: 1000000,
|
|
26
31
|
maxTokens: 384000,
|
|
27
32
|
compat: DEEPSEEK_RESPONSES_COMPAT,
|
|
@@ -43,9 +48,9 @@ export function deepseekModels(root: string): ProviderModelConfig[] {
|
|
|
43
48
|
reasoning: true,
|
|
44
49
|
input: ["text"],
|
|
45
50
|
cost: {
|
|
46
|
-
input:
|
|
47
|
-
output:
|
|
48
|
-
cacheRead: 0.
|
|
51
|
+
input: 1.3275,
|
|
52
|
+
output: 3.9825,
|
|
53
|
+
cacheRead: 0.04425,
|
|
49
54
|
cacheWrite: 0,
|
|
50
55
|
},
|
|
51
56
|
contextWindow: 1000000,
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# Local file uploads
|
|
2
|
+
|
|
3
|
+
`upload_local_file` is a local Pi tool. It orchestrates the host's existing
|
|
4
|
+
`ts_oht.upload_file` and `ts_oht.get_file_url` MCP tools, then streams raw bytes
|
|
5
|
+
directly to the returned S3 targets. No new MCP server/client, S3 SDK, runtime
|
|
6
|
+
dependency, MCP key storage or S3 credentials are introduced.
|
|
7
|
+
|
|
8
|
+
## Requirements and authorization
|
|
9
|
+
|
|
10
|
+
- Pi with `ctx.executeTool` (tested with 1.1.0). The package's older provider
|
|
11
|
+
compatibility floor is not a promise of upload support on older hosts.
|
|
12
|
+
- The plugin registers `ts_oht` with the default public endpoint; optional
|
|
13
|
+
`tsgw.mcpUrl` overrides it. A same-name global/project MCP file entry wins,
|
|
14
|
+
including a disabled entry. Pi manages the connection and OAuth; the upload
|
|
15
|
+
tool still requires its upload/URL tools to be connected and callable. Hidden,
|
|
16
|
+
disabled, blocked or not-yet-connected tools produce an error; the tool does
|
|
17
|
+
not activate them, create its own connection, or read/modify MCP configuration.
|
|
18
|
+
- The current service's `mode` / `uploads[]` contract is required. The old
|
|
19
|
+
`uploadUrl`-only contract is not supported.
|
|
20
|
+
- The tool uses the Pi process's OS file permissions, not a sandbox or the
|
|
21
|
+
built-in `read` tool's path approvals. Authorize **file exfiltration**, not just
|
|
22
|
+
reading. Gate `upload_local_file` in your permission extension where needed;
|
|
23
|
+
both the outer tool and nested MCP calls pass through host permission hooks.
|
|
24
|
+
- Paths belong to the plugin machine/container, not necessarily your browser's
|
|
25
|
+
computer. Relative paths use the session cwd; `~/` expands to the host home.
|
|
26
|
+
Symlinks resolve to their canonical regular file; directories/devices/FIFOs
|
|
27
|
+
are rejected. There is no implicit cwd-only allowlist.
|
|
28
|
+
- The server advertises a 300-second planning/completion limit. Configure the
|
|
29
|
+
host MCP timeout to at least 330 seconds if needed. The plugin cannot override
|
|
30
|
+
that per call and never edits the user's settings. The host default may end a
|
|
31
|
+
call earlier. The local 330-second MCP deadline bounds waiting, not remote work.
|
|
32
|
+
|
|
33
|
+
## Tool interface
|
|
34
|
+
|
|
35
|
+
```ts
|
|
36
|
+
upload_local_file({
|
|
37
|
+
path: string,
|
|
38
|
+
contentType?: string,
|
|
39
|
+
expiresIn?: number, // integer 1–604800; default 86400
|
|
40
|
+
disposition?: "inline" | "attachment", // default inline
|
|
41
|
+
resume?: string // unchanged receipt from a failed call
|
|
42
|
+
})
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
`contentType` must not contain charset/other parameters. The default is a
|
|
46
|
+
best-effort extension mapping, falling back to `application/octet-stream`;
|
|
47
|
+
this is not content sniffing. The program reads the real size and timestamps,
|
|
48
|
+
never taking these from model-supplied arguments. File names must satisfy the
|
|
49
|
+
service's UTF-8 name restrictions.
|
|
50
|
+
|
|
51
|
+
- 1 byte through 32 MiB: read a bounded snapshot and send one raw PUT.
|
|
52
|
+
- Above 32 MiB through the service's current 5 TiB safety ceiling: stream the
|
|
53
|
+
returned part ranges with at most **3 workers**, without buffering the whole
|
|
54
|
+
file or a whole large part. The backend may impose lower limits; the extreme
|
|
55
|
+
size is a protocol bound, not a claim of endpoint acceptance/performance.
|
|
56
|
+
- Validate mode, key/uid, part geometry, unique part numbers, sizes, HTTP method,
|
|
57
|
+
headers and expiration. A one-part remaining plan is still multipart.
|
|
58
|
+
- Follow no redirects and perform no automatic PUT retries. Targets must use
|
|
59
|
+
public HTTPS on port 443. DNS addresses are checked and pinned for each PUT;
|
|
60
|
+
private/local targets, credential/hop-by-hop headers and non-PUT methods are
|
|
61
|
+
refused. Download and upload targets must share an origin.
|
|
62
|
+
- After all multipart PUTs finish, call `upload_file` **once** with the same
|
|
63
|
+
metadata to reconcile and complete. Only the service's `uploaded:true` counts
|
|
64
|
+
as a completed multipart object. No automatic completion loop is used.
|
|
65
|
+
- Default requests reuse the service's 24-hour inline URL if still valid.
|
|
66
|
+
Attachment/custom lifetime (or an expired default link) is signed only after
|
|
67
|
+
upload success using `get_file_url`.
|
|
68
|
+
|
|
69
|
+
The pre-signed PUT deadline is bounded by its signature expiry and a
|
|
70
|
+
size-dependent total timeout. User cancellation stops local PUTs and waits for
|
|
71
|
+
all workers to settle before releasing the file/lock. MCP cancellation does
|
|
72
|
+
**not** guarantee cancellation of service-side work. The plugin does not retry
|
|
73
|
+
MCP calls; the host's transport can independently reconnect/re-authenticate.
|
|
74
|
+
|
|
75
|
+
## Failures and explicit recovery
|
|
76
|
+
|
|
77
|
+
Results carry `uploaded` (`true`, `false`, or `null` when unknown),
|
|
78
|
+
`uploadStatus`, `phase`, `linkStatus`, and safe errors. Never interpret merely
|
|
79
|
+
receiving a key or a signed URL as a completed upload.
|
|
80
|
+
|
|
81
|
+
| Situation | Result / recovery |
|
|
82
|
+
| --- | --- |
|
|
83
|
+
| Invalid input, missing file or unavailable MCP | Error; no automatic connection, alternate authentication or retry |
|
|
84
|
+
| Part failure, timeout or cancellation | Stop the batch; preserve `key`, `uid` and `resume` when known; do not auto-resign or retry |
|
|
85
|
+
| Completion response lost | Unknown, not failed; explicit recovery checks the previous key first |
|
|
86
|
+
| Upload succeeded but link signing failed | `uploaded:true`, `linkStatus:failed`; preserve key and resume; a still-valid default link is labelled `fallback`, not the requested link |
|
|
87
|
+
| Single PUT outcome unknown | Recovery checks the old key only; never repeats the PUT or requests a new object |
|
|
88
|
+
|
|
89
|
+
To recover, call the same tool with the returned `resume` receipt, the original
|
|
90
|
+
path, and desired link options. The receipt contains metadata/identity and the
|
|
91
|
+
object key, **not URLs, credentials or file contents**. It is an opaque recovery
|
|
92
|
+
record, not an authentication token. Do not edit it or discard it and blindly
|
|
93
|
+
start a fresh upload.
|
|
94
|
+
|
|
95
|
+
Recovery first calls `get_file_url` for the old key. If the object exists, it
|
|
96
|
+
returns the link without reading/re-uploading the file (also works when the
|
|
97
|
+
local file has since been removed). Only an explicit remote `not-found` allows
|
|
98
|
+
an unfinished multipart upload to proceed. Other errors leave the outcome
|
|
99
|
+
unknown. A completed upload whose object was deleted is not recreated.
|
|
100
|
+
|
|
101
|
+
For multipart recovery the file's canonical path, device/inode, size, mtime,
|
|
102
|
+
ctime and content type must still match. The service re-plans by the original
|
|
103
|
+
metadata and returns only missing parts. The client requires the **same key and
|
|
104
|
+
uid** before sending bytes. If the service no longer has the old task, it may
|
|
105
|
+
create a different plan during this lookup; that plan is refused without
|
|
106
|
+
uploading bytes, and the server's existing cleanup handles it. There is no
|
|
107
|
+
service-side resume-by-uid endpoint to avoid this planning side effect.
|
|
108
|
+
|
|
109
|
+
The tool checks file identity and timestamps before/after streamed ranges and
|
|
110
|
+
before completion. This detects normal changes, not malicious timestamp
|
|
111
|
+
spoofing; it is not a content hash or snapshot guarantee for mutable large
|
|
112
|
+
files. Do not edit an in-progress file. A source change stops completion; the
|
|
113
|
+
client never deletes or aborts objects/tasks itself. Service-side cleanup may
|
|
114
|
+
still reconcile uploaded parts independently.
|
|
115
|
+
|
|
116
|
+
A process-local metadata lock prevents overlapping calls through this module,
|
|
117
|
+
including while a timed-out host call is still settling. It is **not** a
|
|
118
|
+
cross-process/client lock. Stop other clients and wait for outstanding PUTs
|
|
119
|
+
before resuming. The server shares bucket access among all authorized clients;
|
|
120
|
+
there is no per-user object isolation.
|
|
121
|
+
|
|
122
|
+
## Links, storage and verification
|
|
123
|
+
|
|
124
|
+
Download URLs are temporary bearer credentials. Anyone holding one can access
|
|
125
|
+
the object. Tool results intentionally include the requested link, but the
|
|
126
|
+
plugin never logs signed URLs, headers, raw MCP results or network errors and
|
|
127
|
+
never persists recovery state on disk. Host transcripts and trusted tool hooks
|
|
128
|
+
can see tool results; handle/export them accordingly. `inline` can execute
|
|
129
|
+
active content such as HTML on the storage domain; use `attachment` as needed.
|
|
130
|
+
|
|
131
|
+
The private temporary bucket is cleaned after approximately seven days.
|
|
132
|
+
Re-signing a URL does not extend object retention; deletion/credential expiry
|
|
133
|
+
may invalidate a link earlier than its stated expiry.
|
|
134
|
+
|
|
135
|
+
Automated tests use synthetic files and isolated host/MCP fixtures; they never
|
|
136
|
+
use personal configuration or the production gateway. Real endpoint verification
|
|
137
|
+
must be explicitly authorized and reported separately from mocked host tests.
|
|
138
|
+
|
|
139
|
+
After authorization, `TSGW_UPLOAD_LIVE=1 npm run test:upload:live` runs the real Pi
|
|
140
|
+
SDK with the host's existing `ts_oht` configuration/authentication and a local
|
|
141
|
+
fixture model (no inference request). It creates two small files and one 33 MiB
|
|
142
|
+
multipart object, verifies downloaded SHA-256, inline/attachment and custom
|
|
143
|
+
lifetimes, interrupts a part batch, and simulates losing the completion response
|
|
144
|
+
before explicit recovery. It does not modify client/server configuration or
|
|
145
|
+
remove remote objects; existing server retention cleans them. Local fixtures
|
|
146
|
+
and in-memory session state are discarded. No signed links are printed.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
/** Only these messages may reach tool results; never expose raw network/MCP errors. */
|
|
2
|
+
export class UploadError extends Error {
|
|
3
|
+
constructor(readonly code: string, message: string) {
|
|
4
|
+
super(message);
|
|
5
|
+
this.name = "UploadError";
|
|
6
|
+
}
|
|
7
|
+
}
|
|
8
|
+
|
|
9
|
+
export function invalidResponse(): never {
|
|
10
|
+
throw new UploadError("invalid-response", "The upload service returned an invalid plan; no further upload was attempted.");
|
|
11
|
+
}
|
|
12
|
+
|
|
13
|
+
export function checkAbort(signal: AbortSignal): void {
|
|
14
|
+
if (signal.aborted) throw new UploadError("cancelled", "Upload cancelled. Do not automatically repeat the upload.");
|
|
15
|
+
}
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
2
|
+
import { Type } from "typebox";
|
|
3
|
+
import { UploadError } from "./errors.ts";
|
|
4
|
+
import { object, UPLOAD_TOOL, URL_TOOL } from "./protocol.ts";
|
|
5
|
+
import { uploadLocalFile, type UploadDependencies } from "./uploader.ts";
|
|
6
|
+
|
|
7
|
+
const linkSchema = Type.Object({ key: Type.String(), url: Type.String(), expiresAt: Type.String(), expiresIn: Type.Integer() });
|
|
8
|
+
|
|
9
|
+
/** No MCP registration or credential/configuration access: use the caller's live tools. */
|
|
10
|
+
export function registerUploadTool(pi: ExtensionAPI, dependencies: Pick<UploadDependencies, "put" | "remoteTimeoutMs"> = {}): void {
|
|
11
|
+
pi.registerTool({
|
|
12
|
+
name: "upload_local_file",
|
|
13
|
+
label: "Upload local file",
|
|
14
|
+
description: "Upload an explicitly authorized local file from the Pi host to private S3 via the connected ts_oht MCP. " +
|
|
15
|
+
"Reads OS-accessible files; not a browser upload or sandbox. Up to 32 MiB uses one PUT; larger files use streaming multipart with 3 workers. " +
|
|
16
|
+
"Returns a temporary bearer download URL (default 24h inline; HTML may execute). Storage is cleaned after about 7 days. " +
|
|
17
|
+
"No automatic retries. On failure/unknown outcome preserve key and resume receipt; only resume explicitly with the returned receipt, " +
|
|
18
|
+
"never start a fresh upload blindly. Stop other clients uploading this same file before resuming. Do not log presigned URLs.",
|
|
19
|
+
parameters: Type.Object({
|
|
20
|
+
path: Type.String({ minLength: 1, description: "Path in the plugin environment; relative to the session cwd. Only regular files." }),
|
|
21
|
+
contentType: Type.Optional(Type.String({ description: "MIME type without charset; inferred from extension or application/octet-stream." })),
|
|
22
|
+
expiresIn: Type.Optional(Type.Integer({ minimum: 1, maximum: 604800, description: "Download URL lifetime in seconds; default 86400. Does not extend storage retention." })),
|
|
23
|
+
disposition: Type.Optional(Type.Union([Type.Literal("inline"), Type.Literal("attachment")])),
|
|
24
|
+
resume: Type.Optional(Type.String({ minLength: 1, maxLength: 8192, description: "Unmodified recovery receipt from a previous failed call; checks the old object before resuming. No credentials." })),
|
|
25
|
+
}),
|
|
26
|
+
outputSchema: Type.Object({
|
|
27
|
+
uploaded: Type.Union([Type.Boolean(), Type.Null()]),
|
|
28
|
+
uploadStatus: Type.Union([Type.Literal("not-started"), Type.Literal("incomplete"), Type.Literal("unknown"), Type.Literal("uploaded")]),
|
|
29
|
+
phase: Type.Union([Type.Literal("local"), Type.Literal("prepare"), Type.Literal("upload"), Type.Literal("complete"), Type.Literal("link")]),
|
|
30
|
+
linkStatus: Type.Union([Type.Literal("not-ready"), Type.Literal("ready"), Type.Literal("failed")]),
|
|
31
|
+
key: Type.Optional(Type.String()), uid: Type.Optional(Type.String()), url: Type.Optional(Type.String()),
|
|
32
|
+
expiresAt: Type.Optional(Type.String()), expiresIn: Type.Optional(Type.Integer()),
|
|
33
|
+
disposition: Type.Optional(Type.Union([Type.Literal("inline"), Type.Literal("attachment")])),
|
|
34
|
+
fallback: Type.Optional(linkSchema), resume: Type.Optional(Type.String()),
|
|
35
|
+
error: Type.Optional(Type.Object({ code: Type.String(), message: Type.String() })),
|
|
36
|
+
}),
|
|
37
|
+
annotations: { readOnlyHint: false, destructiveHint: false, idempotentHint: false, openWorldHint: true },
|
|
38
|
+
async execute(_id, params, signal, onUpdate, ctx) {
|
|
39
|
+
const result = await uploadLocalFile(params, ctx.cwd, signal ?? ctx.signal ?? new AbortController().signal, {
|
|
40
|
+
...dependencies,
|
|
41
|
+
async call(name, args, operationSignal) {
|
|
42
|
+
if (typeof ctx.executeTool !== "function" || !Array.isArray(ctx.tools))
|
|
43
|
+
throw new UploadError("unsupported-host", "This upload tool needs a Pi host with ctx.executeTool (tested with 1.1.0).");
|
|
44
|
+
if (![UPLOAD_TOOL, URL_TOOL].includes(name) || !ctx.tools.some(tool => tool.name === name))
|
|
45
|
+
throw new UploadError("mcp-unavailable", "The required ts_oht tool is not connected or callable. Enable it in the host; no connection was created.");
|
|
46
|
+
const outcome = await ctx.executeTool(name, args, { signal: operationSignal });
|
|
47
|
+
// Pi wraps the raw MCP CallToolResult in result.structuredContent.
|
|
48
|
+
// Never bypass host redaction by recovering an absent structured result from text.
|
|
49
|
+
if (outcome.result.structuredContent === undefined)
|
|
50
|
+
throw new UploadError("mcp-unavailable", "The host blocked the call or withheld its structured MCP result.");
|
|
51
|
+
const raw = object(outcome.result.structuredContent);
|
|
52
|
+
return outcome.isError ? { ...raw, isError: true } : raw;
|
|
53
|
+
},
|
|
54
|
+
onProgress(progress) {
|
|
55
|
+
onUpdate?.({ content: [{ type: "text", text: `Upload: ${progress.phase} (${progress.completedParts}/${progress.totalParts} parts)` }], details: progress });
|
|
56
|
+
},
|
|
57
|
+
});
|
|
58
|
+
return { content: [{ type: "text", text: JSON.stringify(result) }], details: result,
|
|
59
|
+
structuredContent: { ...result }, ...(result.error ? { isError: true } : {}) };
|
|
60
|
+
},
|
|
61
|
+
});
|
|
62
|
+
}
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
import { constants } from "node:fs";
|
|
2
|
+
import { open, realpath, stat, type FileHandle } from "node:fs/promises";
|
|
3
|
+
import { createHash } from "node:crypto";
|
|
4
|
+
import { basename, extname, resolve } from "node:path";
|
|
5
|
+
import { homedir } from "node:os";
|
|
6
|
+
import { MAX_BYTES, type FileMetadata } from "./protocol.ts";
|
|
7
|
+
import { checkAbort, UploadError } from "./errors.ts";
|
|
8
|
+
|
|
9
|
+
const MIME: Record<string, string> = {
|
|
10
|
+
".txt": "text/plain", ".md": "text/markdown", ".csv": "text/csv", ".html": "text/html",
|
|
11
|
+
".json": "application/json", ".pdf": "application/pdf", ".zip": "application/zip",
|
|
12
|
+
".gz": "application/gzip", ".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg",
|
|
13
|
+
".gif": "image/gif", ".webp": "image/webp", ".svg": "image/svg+xml", ".mp3": "audio/mpeg",
|
|
14
|
+
".wav": "audio/wav", ".ogg": "audio/ogg", ".mp4": "video/mp4", ".webm": "video/webm",
|
|
15
|
+
};
|
|
16
|
+
|
|
17
|
+
type FileStat = Awaited<ReturnType<typeof fileStat>>;
|
|
18
|
+
const fileStat = (handle: FileHandle) => handle.stat({ bigint: true });
|
|
19
|
+
const identity = (s: FileStat): string =>
|
|
20
|
+
[s.dev, s.ino, s.size, s.mtimeNs, s.ctimeNs].join(":");
|
|
21
|
+
|
|
22
|
+
export interface LocalFile {
|
|
23
|
+
metadata: FileMetadata;
|
|
24
|
+
fingerprint: string;
|
|
25
|
+
assertUnchanged(): Promise<void>;
|
|
26
|
+
bytes(offset: number, size: number, signal: AbortSignal): AsyncIterable<Uint8Array>;
|
|
27
|
+
snapshot(signal: AbortSignal): Promise<Uint8Array>;
|
|
28
|
+
close(): Promise<void>;
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
/** OS file access, NOT a sandbox or the built-in read tool's permission policy. */
|
|
32
|
+
export async function openLocalFile(path: string, cwd: string, contentType?: string): Promise<LocalFile> {
|
|
33
|
+
let handle: FileHandle | undefined;
|
|
34
|
+
try {
|
|
35
|
+
if (!path || path.includes("\0")) throw new UploadError("invalid-path", "A local file path is required.");
|
|
36
|
+
const fullPath = await realpath(resolve(cwd, path.startsWith("~/") ? `${homedir()}/${path.slice(2)}` : path));
|
|
37
|
+
// NONBLOCK prevents a race replacing a regular file with a FIFO from hanging open().
|
|
38
|
+
if (!(await stat(fullPath)).isFile()) throw new UploadError("not-file", "Only regular files can be uploaded.");
|
|
39
|
+
handle = await open(fullPath, constants.O_RDONLY | constants.O_NONBLOCK | (constants.O_NOFOLLOW ?? 0));
|
|
40
|
+
const opened = handle;
|
|
41
|
+
const initial = await fileStat(opened);
|
|
42
|
+
if (!initial.isFile()) throw new UploadError("not-file", "Only regular files can be uploaded.");
|
|
43
|
+
const size = Number(initial.size);
|
|
44
|
+
if (!Number.isSafeInteger(size) || size < 1 || size > MAX_BYTES)
|
|
45
|
+
throw new UploadError("invalid-size", "Files must contain 1 byte to 5 TiB. Backend limits may be lower.");
|
|
46
|
+
const filename = basename(fullPath);
|
|
47
|
+
if (Buffer.byteLength(filename) > 255 || filename.trim() !== filename || /[\u0000-\u001f\u007f-\u009f\\%?#:<>"|*]/.test(filename))
|
|
48
|
+
throw new UploadError("invalid-filename", "The file name cannot be represented by the upload service.");
|
|
49
|
+
const mime = contentType ?? MIME[extname(filename).toLowerCase()] ?? "application/octet-stream";
|
|
50
|
+
if (mime.length > 127 || !/^[A-Za-z0-9!#$&^_.+-]+\/[A-Za-z0-9!#$&^_.+-]+$/.test(mime))
|
|
51
|
+
throw new UploadError("invalid-content-type", "contentType must be a MIME type without parameters.");
|
|
52
|
+
const metadata: FileMetadata = { filename, contentType: mime.toLowerCase(), size,
|
|
53
|
+
mtimeMs: Number(initial.mtimeNs) / 1e6, ctimeMs: Number(initial.ctimeNs) / 1e6 };
|
|
54
|
+
const stamp = identity(initial);
|
|
55
|
+
const fingerprint = createHash("sha256").update(JSON.stringify([fullPath, stamp, metadata])).digest("hex");
|
|
56
|
+
const assertUnchanged = async (): Promise<void> => {
|
|
57
|
+
try {
|
|
58
|
+
if (identity(await fileStat(opened)) !== stamp || identity(await stat(fullPath, { bigint: true })) !== stamp)
|
|
59
|
+
throw new Error();
|
|
60
|
+
} catch { throw new UploadError("file-changed", "The source file changed. Its old parts must not be reused or completed."); }
|
|
61
|
+
};
|
|
62
|
+
async function* bytes(offset: number, count: number, signal: AbortSignal): AsyncIterable<Uint8Array> {
|
|
63
|
+
checkAbort(signal);
|
|
64
|
+
await assertUnchanged();
|
|
65
|
+
let position = offset;
|
|
66
|
+
while (position < offset + count) {
|
|
67
|
+
checkAbort(signal);
|
|
68
|
+
// Explicit positions allow concurrent parts without sharing the descriptor's cursor.
|
|
69
|
+
const buffer = Buffer.allocUnsafe(Math.min(256 * 1024, offset + count - position));
|
|
70
|
+
const { bytesRead } = await opened.read(buffer, 0, buffer.length, position);
|
|
71
|
+
if (bytesRead === 0) throw new UploadError("file-changed", "The source file was truncated during upload.");
|
|
72
|
+
position += bytesRead;
|
|
73
|
+
yield buffer.subarray(0, bytesRead);
|
|
74
|
+
}
|
|
75
|
+
await assertUnchanged();
|
|
76
|
+
}
|
|
77
|
+
await assertUnchanged();
|
|
78
|
+
return {
|
|
79
|
+
metadata, fingerprint, assertUnchanged, bytes,
|
|
80
|
+
async snapshot(signal) {
|
|
81
|
+
const data = new Uint8Array(size);
|
|
82
|
+
let offset = 0;
|
|
83
|
+
for await (const block of bytes(0, size, signal)) { data.set(block, offset); offset += block.byteLength; }
|
|
84
|
+
return data;
|
|
85
|
+
},
|
|
86
|
+
close: () => opened.close(),
|
|
87
|
+
};
|
|
88
|
+
} catch (error) {
|
|
89
|
+
await handle?.close().catch(() => {});
|
|
90
|
+
if (error instanceof UploadError) throw error;
|
|
91
|
+
throw new UploadError("file-unavailable", "Cannot read the local file in the plugin's environment.");
|
|
92
|
+
}
|
|
93
|
+
}
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
import { UploadError, invalidResponse } from "./errors.ts";
|
|
2
|
+
|
|
3
|
+
export const SINGLE_LIMIT = 32 * 1024 ** 2;
|
|
4
|
+
export const MAX_BYTES = 5 * 1024 ** 4;
|
|
5
|
+
export const UPLOAD_TOOL = "mcp__ts_oht__upload_file";
|
|
6
|
+
export const URL_TOOL = "mcp__ts_oht__get_file_url";
|
|
7
|
+
|
|
8
|
+
export interface FileMetadata {
|
|
9
|
+
filename: string;
|
|
10
|
+
contentType: string;
|
|
11
|
+
size: number;
|
|
12
|
+
mtimeMs: number;
|
|
13
|
+
ctimeMs: number;
|
|
14
|
+
}
|
|
15
|
+
export type Download = {
|
|
16
|
+
key: string;
|
|
17
|
+
url: string;
|
|
18
|
+
expiresAt: string;
|
|
19
|
+
expiresIn: number;
|
|
20
|
+
};
|
|
21
|
+
export interface Target {
|
|
22
|
+
partNumber: number;
|
|
23
|
+
offset: number;
|
|
24
|
+
size: number;
|
|
25
|
+
url: string;
|
|
26
|
+
method: "PUT";
|
|
27
|
+
headers: Record<string, string>;
|
|
28
|
+
expiresAt: string;
|
|
29
|
+
}
|
|
30
|
+
export interface Plan extends Download {
|
|
31
|
+
mode: "single" | "multipart";
|
|
32
|
+
uploaded: boolean;
|
|
33
|
+
uploads: Target[];
|
|
34
|
+
uid?: string;
|
|
35
|
+
partSize?: number;
|
|
36
|
+
totalParts?: number;
|
|
37
|
+
}
|
|
38
|
+
export interface Receipt {
|
|
39
|
+
version: 1;
|
|
40
|
+
key: string;
|
|
41
|
+
uid?: string;
|
|
42
|
+
mode: "single" | "multipart";
|
|
43
|
+
phase: "upload" | "complete" | "link";
|
|
44
|
+
fingerprint: string;
|
|
45
|
+
metadata: FileMetadata;
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
export function object(value: unknown): Record<string, unknown> {
|
|
49
|
+
if (!value || typeof value !== "object" || Array.isArray(value)) invalidResponse();
|
|
50
|
+
return value as Record<string, unknown>;
|
|
51
|
+
}
|
|
52
|
+
function text(value: unknown, max = 16384): string {
|
|
53
|
+
if (typeof value !== "string" || !value || value.length > max) invalidResponse();
|
|
54
|
+
return value;
|
|
55
|
+
}
|
|
56
|
+
function integer(value: unknown, max: number): number {
|
|
57
|
+
if (!Number.isSafeInteger(value) || (value as number) < 1 || (value as number) > max) invalidResponse();
|
|
58
|
+
return value as number;
|
|
59
|
+
}
|
|
60
|
+
export function validKey(value: unknown): string {
|
|
61
|
+
const key = text(value, 1024);
|
|
62
|
+
if (Buffer.byteLength(key) > 1024 || /[\u0000-\u001f\u007f-\u009f\\%?#]/.test(key) || key.split("/").some(p => !p || p === "." || p === "..")) invalidResponse();
|
|
63
|
+
return key;
|
|
64
|
+
}
|
|
65
|
+
function httpsUrl(value: unknown): string {
|
|
66
|
+
const input = text(value);
|
|
67
|
+
try {
|
|
68
|
+
const url = new URL(input);
|
|
69
|
+
if (url.protocol !== "https:" || url.username || url.password || url.hash || (url.port && url.port !== "443")) invalidResponse();
|
|
70
|
+
return input;
|
|
71
|
+
} catch { return invalidResponse(); }
|
|
72
|
+
}
|
|
73
|
+
function expiry(value: unknown): string {
|
|
74
|
+
const date = text(value, 64);
|
|
75
|
+
if (!Number.isFinite(Date.parse(date))) invalidResponse();
|
|
76
|
+
return date;
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
/** The host adapter supplies the MCP CallToolResult, NOT AgentToolCallOutcome. */
|
|
80
|
+
export function mcpData(value: unknown): Record<string, unknown> {
|
|
81
|
+
const result = object(value);
|
|
82
|
+
let data: unknown = result.structuredContent;
|
|
83
|
+
if (data === undefined && Array.isArray(result.content)) {
|
|
84
|
+
const blocks = result.content.filter(b => b && b.type === "text");
|
|
85
|
+
if (blocks.length === 1 && typeof blocks[0].text === "string" && blocks[0].text.length <= 4 * 1024 ** 2) {
|
|
86
|
+
try { data = JSON.parse(blocks[0].text); } catch { /* Reject non-JSON, never echo it. */ }
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
const body = object(data);
|
|
90
|
+
if (result.isError === true || body.error !== undefined) {
|
|
91
|
+
const error = body.error && typeof body.error === "object" ? body.error as Record<string, unknown> : {};
|
|
92
|
+
// Only a definite not-found allows multipart recovery to proceed.
|
|
93
|
+
const code = error.code === "not-found" ? "not-found" : "mcp-error";
|
|
94
|
+
throw new UploadError(code, code === "not-found"
|
|
95
|
+
? "The previous object does not exist."
|
|
96
|
+
: "The remote tool failed; its raw response was withheld to protect credentials.");
|
|
97
|
+
}
|
|
98
|
+
return body;
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
export function download(value: unknown, expectedKey?: string, expectedSeconds = 86400): Download {
|
|
102
|
+
const data = object(value);
|
|
103
|
+
const result = {
|
|
104
|
+
key: validKey(data.key), url: httpsUrl(data.url),
|
|
105
|
+
expiresAt: expiry(data.expiresAt), expiresIn: integer(data.expiresIn, 604800),
|
|
106
|
+
};
|
|
107
|
+
if ((expectedKey && result.key !== expectedKey) || result.expiresIn !== expectedSeconds) invalidResponse();
|
|
108
|
+
return result;
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
export function planIdentity(value: unknown, size: number, previous?: Receipt): Pick<Plan, "key" | "mode" | "uploaded" | "uid" | "partSize" | "totalParts"> {
|
|
112
|
+
const data = object(value);
|
|
113
|
+
const key = validKey(data.key);
|
|
114
|
+
if (previous && key !== previous.key)
|
|
115
|
+
throw new UploadError("session-changed", "The old multipart task is no longer available. A different plan was refused; no bytes were sent to it.");
|
|
116
|
+
const mode = size <= SINGLE_LIMIT ? "single" : "multipart";
|
|
117
|
+
if (data.mode !== mode || typeof data.uploaded !== "boolean" ||
|
|
118
|
+
!Array.isArray(data.uploads) || data.uploads.length > 1024) invalidResponse();
|
|
119
|
+
const result: Pick<Plan, "key" | "mode" | "uploaded" | "uid" | "partSize" | "totalParts"> = { key, mode, uploaded: data.uploaded };
|
|
120
|
+
if (result.mode === "multipart") {
|
|
121
|
+
result.uid = text(data.uid, 64);
|
|
122
|
+
if (!/^[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(result.uid)) invalidResponse();
|
|
123
|
+
if (previous && previous.uid !== result.uid) invalidResponse();
|
|
124
|
+
result.partSize = integer(data.partSize, 5 * 1024 ** 3);
|
|
125
|
+
result.totalParts = integer(data.totalParts, 1024);
|
|
126
|
+
if (result.partSize < 5 * 1024 ** 2 || Math.ceil(size / result.partSize) !== result.totalParts ||
|
|
127
|
+
data.remainingParts !== data.uploads.length) invalidResponse();
|
|
128
|
+
} else if (result.uploaded || data.uploads.length !== 1) invalidResponse();
|
|
129
|
+
if (result.uploaded !== (data.uploads.length === 0)) invalidResponse();
|
|
130
|
+
return result;
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
export function plan(value: unknown, size: number, previous?: Receipt): Plan {
|
|
134
|
+
const identity = planIdentity(value, size, previous);
|
|
135
|
+
const data = object(value);
|
|
136
|
+
const link = download(data);
|
|
137
|
+
const result: Plan = { ...identity, ...link, uploads: [] };
|
|
138
|
+
const seen = new Set<number>();
|
|
139
|
+
for (const item of data.uploads as unknown[]) {
|
|
140
|
+
const part = object(item);
|
|
141
|
+
const number = integer(part.partNumber, result.totalParts ?? 1);
|
|
142
|
+
const offset = (number - 1) * (result.partSize ?? size);
|
|
143
|
+
const bytes = Math.min(result.partSize ?? size, size - offset);
|
|
144
|
+
if (seen.has(number) || part.offset !== offset || part.size !== bytes || part.method !== "PUT") invalidResponse();
|
|
145
|
+
seen.add(number);
|
|
146
|
+
const url = httpsUrl(part.url);
|
|
147
|
+
if (new URL(url).origin !== new URL(link.url).origin) invalidResponse();
|
|
148
|
+
const headers = object(part.headers);
|
|
149
|
+
if (Object.keys(headers).length > 64 || Object.values(headers).some(v => typeof v !== "string" || v.length > 8192)) invalidResponse();
|
|
150
|
+
result.uploads.push({ partNumber: number, offset, size: bytes, url, method: "PUT",
|
|
151
|
+
headers: headers as Record<string, string>, expiresAt: expiry(part.expiresAt) });
|
|
152
|
+
}
|
|
153
|
+
return result;
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
/** No URLs, headers, S3 credentials or MCP secrets are included in a receipt. */
|
|
157
|
+
export function encodeReceipt(receipt: Receipt): string {
|
|
158
|
+
return Buffer.from(JSON.stringify(receipt)).toString("base64url");
|
|
159
|
+
}
|
|
160
|
+
export function decodeReceipt(token: string): Receipt {
|
|
161
|
+
try {
|
|
162
|
+
if (!token || token.length > 8192 || !/^[A-Za-z0-9_-]+$/.test(token)) invalidResponse();
|
|
163
|
+
const data = object(JSON.parse(Buffer.from(token, "base64url").toString("utf8")));
|
|
164
|
+
const meta = object(data.metadata);
|
|
165
|
+
if (data.version !== 1 || !["single", "multipart"].includes(String(data.mode)) ||
|
|
166
|
+
!["upload", "complete", "link"].includes(String(data.phase)) ||
|
|
167
|
+
typeof data.fingerprint !== "string" || !/^[a-f0-9]{64}$/.test(data.fingerprint) ||
|
|
168
|
+
typeof meta.mtimeMs !== "number" || !Number.isFinite(meta.mtimeMs) ||
|
|
169
|
+
typeof meta.ctimeMs !== "number" || !Number.isFinite(meta.ctimeMs)) invalidResponse();
|
|
170
|
+
const metadata: FileMetadata = { filename: text(meta.filename, 255), contentType: text(meta.contentType, 127),
|
|
171
|
+
size: integer(meta.size, MAX_BYTES), mtimeMs: meta.mtimeMs, ctimeMs: meta.ctimeMs };
|
|
172
|
+
const mode = data.mode as Receipt["mode"];
|
|
173
|
+
if ((metadata.size <= SINGLE_LIMIT) !== (mode === "single")) invalidResponse();
|
|
174
|
+
return { version: 1, key: validKey(data.key), mode, phase: data.phase as Receipt["phase"],
|
|
175
|
+
uid: mode === "multipart" ? text(data.uid, 64) : undefined, fingerprint: data.fingerprint, metadata };
|
|
176
|
+
} catch { throw new UploadError("invalid-resume", "Invalid resume receipt. Do not replace it with a new upload attempt."); }
|
|
177
|
+
}
|
|
@@ -0,0 +1,216 @@
|
|
|
1
|
+
import { Resolver } from "node:dns/promises";
|
|
2
|
+
import { request, type RequestOptions } from "node:https";
|
|
3
|
+
import type { ClientRequest, IncomingMessage } from "node:http";
|
|
4
|
+
import { isIP } from "node:net";
|
|
5
|
+
import { UploadError } from "./errors.ts";
|
|
6
|
+
|
|
7
|
+
export interface PutTarget {
|
|
8
|
+
url: string;
|
|
9
|
+
method: "PUT";
|
|
10
|
+
headers: Record<string, string>;
|
|
11
|
+
expiresAt: string;
|
|
12
|
+
size: number;
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
type Address = { address: string; family: 4 | 6 };
|
|
16
|
+
/** Narrow seams for offline transport tests; production always uses Node DNS and HTTPS. */
|
|
17
|
+
export interface TransportDeps {
|
|
18
|
+
resolve(host: string, signal: AbortSignal): Promise<Address[]>;
|
|
19
|
+
request(options: RequestOptions, response: (response: IncomingMessage) => void): ClientRequest;
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
const messages = {
|
|
23
|
+
"http-error": "Upload server rejected the bytes. Do not automatically repeat the upload.",
|
|
24
|
+
redirect: "Upload redirects are not allowed.",
|
|
25
|
+
"network-error": "Upload transport or byte stream failed. Do not automatically repeat the upload.",
|
|
26
|
+
timeout: "Upload deadline exceeded. Do not automatically repeat the upload.",
|
|
27
|
+
cancelled: "Upload cancelled. Do not automatically repeat the upload.",
|
|
28
|
+
"invalid-target": "Upload target is not a valid public HTTPS destination.",
|
|
29
|
+
"invalid-headers": "Upload headers are not permitted or do not match the byte count.",
|
|
30
|
+
} as const;
|
|
31
|
+
type Code = keyof typeof messages;
|
|
32
|
+
const failure = (code: Code) => new UploadError(code, messages[code]);
|
|
33
|
+
|
|
34
|
+
/** Reject special-use IPv4 and non-global/transition/documentation IPv6. */
|
|
35
|
+
export function isPublicAddress(address: string): boolean {
|
|
36
|
+
if (isIP(address) === 4) {
|
|
37
|
+
const [a, b, c] = address.split(".").map(Number) as [number, number, number, number];
|
|
38
|
+
return !(a === 0 || a === 10 || a === 127 || a >= 224 ||
|
|
39
|
+
(a === 100 && b >= 64 && b <= 127) || (a === 169 && b === 254) ||
|
|
40
|
+
(a === 172 && b >= 16 && b <= 31) || (a === 192 && b === 168) ||
|
|
41
|
+
(a === 192 && b === 0) || (a === 192 && b === 88 && c === 99) ||
|
|
42
|
+
(a === 198 && (b === 18 || b === 19)) || (a === 198 && b === 51 && c === 100) ||
|
|
43
|
+
(a === 203 && b === 0 && c === 113));
|
|
44
|
+
}
|
|
45
|
+
if (isIP(address) !== 6 || address.includes("%")) return false;
|
|
46
|
+
const canonical = new URL(`https://[${address}]/`).hostname.slice(1, -1);
|
|
47
|
+
const parts = canonical.split(":");
|
|
48
|
+
const first = Number.parseInt(parts[0]!, 16);
|
|
49
|
+
const second = Number.parseInt(parts[1] || "0", 16);
|
|
50
|
+
return first >= 0x2000 && first <= 0x3fff &&
|
|
51
|
+
// 2001::/23 includes Teredo, benchmarking, ORCHID and protocol assignments.
|
|
52
|
+
!(first === 0x2001 && second < 0x200) &&
|
|
53
|
+
!(first === 0x2001 && second === 0xdb8) && first !== 0x2002 &&
|
|
54
|
+
!(first === 0x3fff && second < 0x1000);
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
async function resolvePublic(host: string, signal: AbortSignal): Promise<Address[]> {
|
|
58
|
+
const resolver = new Resolver();
|
|
59
|
+
const cancel = () => resolver.cancel();
|
|
60
|
+
signal.addEventListener("abort", cancel, { once: true });
|
|
61
|
+
try {
|
|
62
|
+
if (signal.aborted) throw failure("cancelled");
|
|
63
|
+
const results = await Promise.allSettled([resolver.resolve4(host), resolver.resolve6(host)]);
|
|
64
|
+
return results.flatMap((result, index) => result.status === "fulfilled"
|
|
65
|
+
? result.value.map(address => ({ address, family: (index === 0 ? 4 : 6) as 4 | 6 })) : []);
|
|
66
|
+
} finally {
|
|
67
|
+
signal.removeEventListener("abort", cancel);
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
function validate(target: PutTarget): { url: URL; headers: Record<string, string>; deadline: number } {
|
|
72
|
+
let url: URL;
|
|
73
|
+
try { url = new URL(target.url); } catch { throw failure("invalid-target"); }
|
|
74
|
+
const expiry = Date.parse(target.expiresAt);
|
|
75
|
+
if (target.method !== "PUT" || !Number.isSafeInteger(target.size) || target.size < 0 ||
|
|
76
|
+
url.protocol !== "https:" || url.username || url.password || url.hash ||
|
|
77
|
+
(url.port && url.port !== "443") || !Number.isFinite(expiry) || expiry <= Date.now()) {
|
|
78
|
+
throw failure("invalid-target");
|
|
79
|
+
}
|
|
80
|
+
const headers: Record<string, string> = Object.create(null) as Record<string, string>;
|
|
81
|
+
for (const [key, value] of Object.entries(target.headers)) {
|
|
82
|
+
const name = key.toLowerCase();
|
|
83
|
+
if (!(name === "content-type" || name === "content-length" || name === "if-none-match" ||
|
|
84
|
+
/^x-amz-[a-z0-9-]+$/.test(name)) || name in headers || typeof value !== "string" ||
|
|
85
|
+
/[^\x20-\x7e]/.test(value) || (name === "if-none-match" && value !== "*") ||
|
|
86
|
+
(name === "content-length" && value !== String(target.size))) throw failure("invalid-headers");
|
|
87
|
+
headers[name] = value;
|
|
88
|
+
}
|
|
89
|
+
headers["content-length"] = String(target.size);
|
|
90
|
+
// Total budget, not a socket-idle timeout: allow at least 330s, then 64 KiB/s.
|
|
91
|
+
const budget = Math.max(330_000, Math.ceil(target.size / 65_536) * 1000);
|
|
92
|
+
return { url, headers, deadline: Math.min(expiry, Date.now() + budget) };
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
export async function putBytes(
|
|
96
|
+
target: PutTarget, body: AsyncIterable<Uint8Array> | Uint8Array, signal: AbortSignal,
|
|
97
|
+
): Promise<void> {
|
|
98
|
+
return putBytesWithDeps(target, body, signal, { resolve: resolvePublic, request });
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
export async function putBytesWithDeps(
|
|
102
|
+
target: PutTarget, body: AsyncIterable<Uint8Array> | Uint8Array, signal: AbortSignal, deps: TransportDeps,
|
|
103
|
+
): Promise<void> {
|
|
104
|
+
let iterator: AsyncIterator<Uint8Array> | undefined;
|
|
105
|
+
let req: ClientRequest | undefined;
|
|
106
|
+
let response: IncomingMessage | undefined;
|
|
107
|
+
let requestClosed: Promise<void> | undefined;
|
|
108
|
+
let pumping: Promise<void> | undefined;
|
|
109
|
+
let timer: ReturnType<typeof setTimeout> | undefined;
|
|
110
|
+
const controller = new AbortController();
|
|
111
|
+
let error: UploadError | undefined;
|
|
112
|
+
let rejectStopped!: (error: UploadError) => void;
|
|
113
|
+
const stopped = new Promise<never>((_, reject) => { rejectStopped = reject; });
|
|
114
|
+
void stopped.catch(() => {});
|
|
115
|
+
const stop = (code: Code) => {
|
|
116
|
+
if (error) return;
|
|
117
|
+
error = failure(code);
|
|
118
|
+
controller.abort();
|
|
119
|
+
response?.destroy();
|
|
120
|
+
req?.destroy();
|
|
121
|
+
rejectStopped(error);
|
|
122
|
+
};
|
|
123
|
+
const abort = () => stop("cancelled");
|
|
124
|
+
try {
|
|
125
|
+
iterator = body instanceof Uint8Array ? (async function* () { yield body; })() : body[Symbol.asyncIterator]();
|
|
126
|
+
signal.addEventListener("abort", abort, { once: true });
|
|
127
|
+
if (signal.aborted) { stop("cancelled"); throw error; }
|
|
128
|
+
const { url, headers, deadline } = validate(target);
|
|
129
|
+
const arm = () => {
|
|
130
|
+
const remaining = deadline - Date.now();
|
|
131
|
+
if (remaining <= 0) stop("timeout");
|
|
132
|
+
else timer = setTimeout(arm, Math.min(remaining, 2_147_483_647));
|
|
133
|
+
};
|
|
134
|
+
arm();
|
|
135
|
+
const host = url.hostname.replace(/^\[|\]$/g, "");
|
|
136
|
+
const addresses = isIP(host) ? [{ address: host, family: isIP(host) as 4 | 6 }]
|
|
137
|
+
: await deps.resolve(host, controller.signal);
|
|
138
|
+
if (error) throw error;
|
|
139
|
+
if (!addresses.length) throw failure("network-error");
|
|
140
|
+
if (addresses.some(item => !isPublicAddress(item.address))) throw failure("invalid-target");
|
|
141
|
+
const pinned = addresses[0]!;
|
|
142
|
+
let sent = false;
|
|
143
|
+
let resolveResponse!: () => void;
|
|
144
|
+
const received = new Promise<void>(resolve => { resolveResponse = resolve; });
|
|
145
|
+
req = deps.request({
|
|
146
|
+
protocol: "https:", hostname: host, port: 443, method: "PUT",
|
|
147
|
+
path: url.pathname + url.search, headers, agent: false, rejectUnauthorized: true,
|
|
148
|
+
lookup: (_hostname, options, callback) => {
|
|
149
|
+
if (options.all) callback(null, [pinned]);
|
|
150
|
+
else callback(null, pinned.address, pinned.family);
|
|
151
|
+
},
|
|
152
|
+
}, incoming => {
|
|
153
|
+
response = incoming;
|
|
154
|
+
incoming.on("error", () => stop("network-error"));
|
|
155
|
+
const status = incoming.statusCode ?? 0;
|
|
156
|
+
if (status >= 300 && status < 400) { stop("redirect"); return; }
|
|
157
|
+
if (status < 200 || status >= 300) { stop("http-error"); return; }
|
|
158
|
+
if (!sent) { stop("network-error"); return; }
|
|
159
|
+
incoming.on("aborted", () => stop("network-error"));
|
|
160
|
+
incoming.on("end", resolveResponse);
|
|
161
|
+
incoming.on("close", () => { if (!incoming.complete) stop("network-error"); });
|
|
162
|
+
incoming.resume();
|
|
163
|
+
});
|
|
164
|
+
const active = req;
|
|
165
|
+
requestClosed = new Promise(resolve => active.once("close", resolve));
|
|
166
|
+
active.on("error", () => stop("network-error"));
|
|
167
|
+
active.on("close", () => { if (!response?.complete) stop("network-error"); });
|
|
168
|
+
pumping = (async () => {
|
|
169
|
+
let count = 0;
|
|
170
|
+
while (!error) {
|
|
171
|
+
const item = await iterator!.next();
|
|
172
|
+
if (error) break;
|
|
173
|
+
if (item.done) {
|
|
174
|
+
if (count !== target.size) throw failure("network-error");
|
|
175
|
+
await new Promise<void>((resolve, reject) => {
|
|
176
|
+
active.end(() => { sent = true; resolve(); });
|
|
177
|
+
void stopped.catch(reject);
|
|
178
|
+
});
|
|
179
|
+
return;
|
|
180
|
+
}
|
|
181
|
+
if (!(item.value instanceof Uint8Array) || item.value.byteLength > target.size - count) {
|
|
182
|
+
throw failure("network-error");
|
|
183
|
+
}
|
|
184
|
+
count += item.value.byteLength;
|
|
185
|
+
// Bounded writes and callback backpressure; never concatenate the upload.
|
|
186
|
+
for (let offset = 0; offset < item.value.byteLength && !error; offset += 65_536) {
|
|
187
|
+
await new Promise<void>((resolve, reject) => {
|
|
188
|
+
const interrupted = () => finish(error);
|
|
189
|
+
const finish = (err?: Error | null) => {
|
|
190
|
+
controller.signal.removeEventListener("abort", interrupted);
|
|
191
|
+
err ? reject(err) : resolve();
|
|
192
|
+
};
|
|
193
|
+
controller.signal.addEventListener("abort", interrupted, { once: true });
|
|
194
|
+
try { active.write(item.value.subarray(offset, offset + 65_536), finish); }
|
|
195
|
+
catch { finish(failure("network-error")); }
|
|
196
|
+
});
|
|
197
|
+
}
|
|
198
|
+
}
|
|
199
|
+
})();
|
|
200
|
+
void pumping.catch(() => stop("network-error"));
|
|
201
|
+
await Promise.race([stopped, Promise.all([pumping, received])]);
|
|
202
|
+
} catch (caught) {
|
|
203
|
+
if (!error) stop(caught instanceof UploadError && caught.code in messages ? caught.code as Code : "network-error");
|
|
204
|
+
} finally {
|
|
205
|
+
if (timer) clearTimeout(timer);
|
|
206
|
+
signal.removeEventListener("abort", abort);
|
|
207
|
+
controller.abort();
|
|
208
|
+
response?.destroy();
|
|
209
|
+
req?.destroy();
|
|
210
|
+
// Do not return while the request or an in-flight producer is still running.
|
|
211
|
+
await pumping?.catch(() => {});
|
|
212
|
+
try { await iterator?.return?.(); } catch { if (!error) error = failure("network-error"); }
|
|
213
|
+
await requestClosed;
|
|
214
|
+
}
|
|
215
|
+
if (error) throw error;
|
|
216
|
+
}
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
import { createHash } from "node:crypto";
|
|
2
|
+
import { openLocalFile, type LocalFile } from "./local-file.ts";
|
|
3
|
+
import { putBytes } from "./transport.ts";
|
|
4
|
+
import { checkAbort, UploadError } from "./errors.ts";
|
|
5
|
+
import { decodeReceipt, download, encodeReceipt, mcpData, plan, planIdentity, SINGLE_LIMIT, UPLOAD_TOOL, URL_TOOL,
|
|
6
|
+
type Download, type FileMetadata, type Receipt, type Target } from "./protocol.ts";
|
|
7
|
+
|
|
8
|
+
export interface UploadParams {
|
|
9
|
+
path: string;
|
|
10
|
+
contentType?: string;
|
|
11
|
+
expiresIn?: number;
|
|
12
|
+
disposition?: "inline" | "attachment";
|
|
13
|
+
resume?: string;
|
|
14
|
+
}
|
|
15
|
+
export interface UploadResult {
|
|
16
|
+
uploaded: boolean | null;
|
|
17
|
+
uploadStatus: "not-started" | "incomplete" | "unknown" | "uploaded";
|
|
18
|
+
phase: "local" | "prepare" | "upload" | "complete" | "link";
|
|
19
|
+
linkStatus: "not-ready" | "ready" | "failed";
|
|
20
|
+
key?: string;
|
|
21
|
+
uid?: string;
|
|
22
|
+
url?: string;
|
|
23
|
+
expiresAt?: string;
|
|
24
|
+
expiresIn?: number;
|
|
25
|
+
disposition?: "inline" | "attachment";
|
|
26
|
+
fallback?: Download;
|
|
27
|
+
resume?: string;
|
|
28
|
+
error?: { code: string; message: string };
|
|
29
|
+
}
|
|
30
|
+
export interface Progress { phase: UploadResult["phase"]; completedParts: number; totalParts: number }
|
|
31
|
+
export interface UploadDependencies {
|
|
32
|
+
/** Returns the raw MCP CallToolResult. Authentication/connection belong to the host. */
|
|
33
|
+
call(tool: string, args: Record<string, unknown>, signal: AbortSignal): Promise<unknown>;
|
|
34
|
+
put?: (target: Target, body: AsyncIterable<Uint8Array> | Uint8Array, signal: AbortSignal) => Promise<void>;
|
|
35
|
+
open?: typeof openLocalFile;
|
|
36
|
+
remoteTimeoutMs?: number;
|
|
37
|
+
onProgress?: (progress: Progress) => void;
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
// Scope matches the service's metadata-based identity, including aliases/hard links.
|
|
41
|
+
// The lock remains held until late host calls settle; it is not a distributed lock.
|
|
42
|
+
const active = new Set<string>();
|
|
43
|
+
const lockKey = (metadata: FileMetadata) => createHash("sha256").update(JSON.stringify(metadata)).digest("hex");
|
|
44
|
+
|
|
45
|
+
export async function uploadLocalFile(params: UploadParams, cwd: string, signal: AbortSignal,
|
|
46
|
+
deps: UploadDependencies): Promise<UploadResult> {
|
|
47
|
+
let file: LocalFile | undefined;
|
|
48
|
+
let receipt: Receipt | undefined;
|
|
49
|
+
let key: string | undefined;
|
|
50
|
+
let fallback: Download | undefined;
|
|
51
|
+
const pending = new Set<Promise<unknown>>();
|
|
52
|
+
const state: UploadResult = { uploaded: false, uploadStatus: "not-started", phase: "local", linkStatus: "not-ready" };
|
|
53
|
+
const seconds = params.expiresIn ?? 86400;
|
|
54
|
+
const disposition = params.disposition ?? "inline";
|
|
55
|
+
const put = deps.put ?? putBytes;
|
|
56
|
+
const open = deps.open ?? openLocalFile;
|
|
57
|
+
const notify = (completedParts = 0, totalParts = 0) => {
|
|
58
|
+
// Rendering/progress observers must never change upload outcome.
|
|
59
|
+
try { deps.onProgress?.({ phase: state.phase, completedParts, totalParts }); } catch { /* UI only */ }
|
|
60
|
+
};
|
|
61
|
+
const claim = (metadata: FileMetadata) => {
|
|
62
|
+
const candidate = lockKey(metadata);
|
|
63
|
+
if (active.has(candidate)) throw new UploadError("busy", "An upload for this file is still in flight. Wait for it to stop before resuming.");
|
|
64
|
+
active.add(candidate);
|
|
65
|
+
key = candidate;
|
|
66
|
+
};
|
|
67
|
+
const call = async (tool: string, args: Record<string, unknown>): Promise<Record<string, unknown>> => {
|
|
68
|
+
checkAbort(signal);
|
|
69
|
+
const controller = new AbortController();
|
|
70
|
+
const combined = AbortSignal.any([signal, controller.signal]);
|
|
71
|
+
const timer = setTimeout(() => controller.abort(), deps.remoteTimeoutMs ?? 330_000);
|
|
72
|
+
let stop: (() => void) | undefined;
|
|
73
|
+
// Host hooks/connection waits may ignore cancellation. Bound waiting, retain the lock.
|
|
74
|
+
const work = Promise.resolve().then(() => {
|
|
75
|
+
checkAbort(combined);
|
|
76
|
+
return deps.call(tool, args, combined);
|
|
77
|
+
});
|
|
78
|
+
pending.add(work);
|
|
79
|
+
void work.then(() => pending.delete(work), () => pending.delete(work));
|
|
80
|
+
const stopped = new Promise<never>((_, reject) => {
|
|
81
|
+
stop = () => reject(new UploadError(signal.aborted ? "cancelled" : "timeout",
|
|
82
|
+
"Remote result is unknown. Do not automatically repeat the operation."));
|
|
83
|
+
combined.addEventListener("abort", stop, { once: true });
|
|
84
|
+
if (combined.aborted) stop();
|
|
85
|
+
});
|
|
86
|
+
try { return mcpData(await Promise.race([work, stopped])); }
|
|
87
|
+
finally {
|
|
88
|
+
clearTimeout(timer);
|
|
89
|
+
if (stop) combined.removeEventListener("abort", stop);
|
|
90
|
+
}
|
|
91
|
+
};
|
|
92
|
+
const acceptLink = (link: Download): UploadResult => ({ uploaded: true, uploadStatus: "uploaded",
|
|
93
|
+
phase: "link", linkStatus: "ready", ...link, disposition, ...(receipt?.uid ? { uid: receipt.uid } : {}) });
|
|
94
|
+
const getLink = async (objectKey: string) => {
|
|
95
|
+
const link = download(await call(URL_TOOL, { key: objectKey, expiresIn: seconds, disposition }), objectKey, seconds);
|
|
96
|
+
if (Date.parse(link.expiresAt) <= Date.now()) throw new UploadError("expired-link", "The service returned an expired download link; no automatic re-sign was attempted.");
|
|
97
|
+
return link;
|
|
98
|
+
};
|
|
99
|
+
try {
|
|
100
|
+
if (params.resume !== undefined) {
|
|
101
|
+
receipt = decodeReceipt(params.resume);
|
|
102
|
+
state.uploaded = receipt.phase === "link" ? true : null;
|
|
103
|
+
state.uploadStatus = receipt.phase === "link" ? "uploaded" : "unknown";
|
|
104
|
+
}
|
|
105
|
+
checkAbort(signal);
|
|
106
|
+
if (!Number.isSafeInteger(seconds) || seconds < 1 || seconds > 604800 || !["inline", "attachment"].includes(disposition))
|
|
107
|
+
throw new UploadError("invalid-options", "expiresIn must be an integer from 1 to 604800; disposition must be inline or attachment.");
|
|
108
|
+
if (receipt) {
|
|
109
|
+
claim(receipt.metadata);
|
|
110
|
+
state.phase = "link";
|
|
111
|
+
notify();
|
|
112
|
+
try { return acceptLink(await getLink(receipt.key)); }
|
|
113
|
+
catch (error) {
|
|
114
|
+
// Never replay a single PUT or a completed upload, even when the object is gone.
|
|
115
|
+
if (!(error instanceof UploadError) || error.code !== "not-found" || receipt.mode === "single" || receipt.phase === "link") throw error;
|
|
116
|
+
}
|
|
117
|
+
state.uploaded = false;
|
|
118
|
+
state.uploadStatus = "incomplete";
|
|
119
|
+
}
|
|
120
|
+
file = await open(params.path, cwd, params.contentType ?? receipt?.metadata.contentType);
|
|
121
|
+
if (receipt && (file.fingerprint !== receipt.fingerprint || JSON.stringify(file.metadata) !== JSON.stringify(receipt.metadata)))
|
|
122
|
+
throw new UploadError("file-changed", "The file no longer matches the resume receipt. Old parts must not be reused.");
|
|
123
|
+
if (!receipt) claim(file.metadata);
|
|
124
|
+
// Single uploads use a fixed snapshot; multipart uploads use bounded streaming reads.
|
|
125
|
+
const snapshot = file.metadata.size <= SINGLE_LIMIT ? await file.snapshot(signal) : undefined;
|
|
126
|
+
await file.assertUnchanged();
|
|
127
|
+
state.phase = "prepare";
|
|
128
|
+
// Preparing multipart may also complete a pre-existing server-side task.
|
|
129
|
+
if (file.metadata.size > SINGLE_LIMIT) {
|
|
130
|
+
state.uploaded = null;
|
|
131
|
+
state.uploadStatus = "unknown";
|
|
132
|
+
}
|
|
133
|
+
notify();
|
|
134
|
+
const rawPlan = await call(UPLOAD_TOOL, { ...file.metadata });
|
|
135
|
+
const identity = planIdentity(rawPlan, file.metadata.size, receipt);
|
|
136
|
+
receipt = { version: 1, key: identity.key, uid: identity.uid, mode: identity.mode,
|
|
137
|
+
phase: identity.uploaded ? "link" : "upload", fingerprint: file.fingerprint, metadata: file.metadata };
|
|
138
|
+
if (identity.uploaded) {
|
|
139
|
+
state.uploaded = true;
|
|
140
|
+
state.uploadStatus = "uploaded";
|
|
141
|
+
state.phase = "link";
|
|
142
|
+
}
|
|
143
|
+
const prepared = plan(rawPlan, file.metadata.size, receipt);
|
|
144
|
+
fallback = { key: prepared.key, url: prepared.url, expiresAt: prepared.expiresAt, expiresIn: prepared.expiresIn };
|
|
145
|
+
if (!prepared.uploaded) {
|
|
146
|
+
await file.assertUnchanged();
|
|
147
|
+
state.phase = "upload";
|
|
148
|
+
state.uploadStatus = "incomplete";
|
|
149
|
+
const batch = new AbortController();
|
|
150
|
+
const batchSignal = AbortSignal.any([signal, batch.signal]);
|
|
151
|
+
let cursor = 0;
|
|
152
|
+
let completed = 0;
|
|
153
|
+
let failure: unknown;
|
|
154
|
+
const worker = async () => {
|
|
155
|
+
try {
|
|
156
|
+
while (cursor < prepared.uploads.length) {
|
|
157
|
+
checkAbort(batchSignal);
|
|
158
|
+
const target = prepared.uploads[cursor++]!;
|
|
159
|
+
state.uploaded = null;
|
|
160
|
+
state.uploadStatus = "unknown";
|
|
161
|
+
await put(target, snapshot ?? file!.bytes(target.offset, target.size, batchSignal), batchSignal);
|
|
162
|
+
completed++;
|
|
163
|
+
notify(completed, prepared.uploads.length);
|
|
164
|
+
}
|
|
165
|
+
} catch (error) { if (!failure) failure = error; batch.abort(); }
|
|
166
|
+
};
|
|
167
|
+
await Promise.allSettled(Array.from({ length: Math.min(3, prepared.uploads.length) }, worker));
|
|
168
|
+
if (failure) throw failure;
|
|
169
|
+
if (prepared.mode === "multipart") {
|
|
170
|
+
await file.assertUnchanged();
|
|
171
|
+
checkAbort(signal);
|
|
172
|
+
state.phase = "complete";
|
|
173
|
+
receipt.phase = "complete";
|
|
174
|
+
notify(completed, prepared.uploads.length);
|
|
175
|
+
// One reconciliation/complete call; never loop or resend a part automatically.
|
|
176
|
+
const completion = await call(UPLOAD_TOOL, { ...file.metadata });
|
|
177
|
+
const completedIdentity = planIdentity(completion, file.metadata.size, receipt);
|
|
178
|
+
if (!completedIdentity.uploaded) {
|
|
179
|
+
receipt.phase = "upload";
|
|
180
|
+
state.uploaded = false;
|
|
181
|
+
state.uploadStatus = "incomplete";
|
|
182
|
+
throw new UploadError("incomplete-upload", "The service still reports missing parts. Resume explicitly; no part was retried.");
|
|
183
|
+
}
|
|
184
|
+
state.uploaded = true;
|
|
185
|
+
state.uploadStatus = "uploaded";
|
|
186
|
+
state.phase = "link";
|
|
187
|
+
receipt.phase = "link";
|
|
188
|
+
const completedPlan = plan(completion, file.metadata.size, receipt);
|
|
189
|
+
fallback = { key: completedPlan.key, url: completedPlan.url, expiresAt: completedPlan.expiresAt, expiresIn: completedPlan.expiresIn };
|
|
190
|
+
}
|
|
191
|
+
}
|
|
192
|
+
state.uploaded = true;
|
|
193
|
+
state.uploadStatus = "uploaded";
|
|
194
|
+
state.phase = "link";
|
|
195
|
+
receipt.phase = "link";
|
|
196
|
+
notify();
|
|
197
|
+
if (seconds === 86400 && disposition === "inline" && Date.parse(fallback.expiresAt) > Date.now()) return acceptLink(fallback);
|
|
198
|
+
return acceptLink(await getLink(receipt.key));
|
|
199
|
+
} catch (error) {
|
|
200
|
+
const safe = error instanceof UploadError ? error : new UploadError("operation-failed",
|
|
201
|
+
"The upload operation failed. Raw errors were withheld; do not automatically retry.");
|
|
202
|
+
return { ...state, linkStatus: state.phase === "link" ? "failed" : "not-ready",
|
|
203
|
+
...(receipt ? { key: receipt.key, uid: receipt.uid, resume: encodeReceipt(receipt) } : {}),
|
|
204
|
+
...(state.uploaded === true && fallback && Date.parse(fallback.expiresAt) > Date.now() ? { fallback } : {}),
|
|
205
|
+
error: { code: safe.code, message: safe.message } };
|
|
206
|
+
} finally {
|
|
207
|
+
await file?.close().catch(() => {});
|
|
208
|
+
if (key) {
|
|
209
|
+
const held = key;
|
|
210
|
+
if (pending.size === 0) active.delete(held);
|
|
211
|
+
else void Promise.allSettled([...pending]).then(() => active.delete(held));
|
|
212
|
+
}
|
|
213
|
+
}
|
|
214
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-tsgw",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.15",
|
|
4
4
|
"description": "Pi extension for the TOSSP AIH gateway: per-vendor model catalog and thinking operations for openai-completions, openai-responses, anthropic-messages, and google-generative-ai wire protocols.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"keywords": [
|
|
@@ -18,10 +18,14 @@
|
|
|
18
18
|
"pi": {
|
|
19
19
|
"extensions": [
|
|
20
20
|
"./extensions/index.ts"
|
|
21
|
+
],
|
|
22
|
+
"skills": [
|
|
23
|
+
"./skills"
|
|
21
24
|
]
|
|
22
25
|
},
|
|
23
26
|
"scripts": {
|
|
24
|
-
"test": "tsc -p tsconfig.test.json && node dist/extensions/index.test.js && node dist/extensions/status.test.js && node dist/extensions/status-integration.test.js && node dist/extensions/models/gateway-catalog-metadata.test.js && node dist/extensions/models/operations.test.js && node dist/extensions/models/catalog.test.js && node dist/extensions/models/web-search.test.js && node dist/extensions/models/gateway-catalog.test.js && node dist/extensions/models/gateway-catalog-isolation.test.js && node dist/extensions/models/vendor-strategies.test.js && node dist/extensions/host-integration.test.js"
|
|
27
|
+
"test": "tsc -p tsconfig.test.json && node dist/extensions/index.test.js && node dist/extensions/status.test.js && node dist/extensions/status-integration.test.js && node dist/extensions/models/gateway-catalog-metadata.test.js && node dist/extensions/models/operations.test.js && node dist/extensions/models/catalog.test.js && node dist/extensions/models/web-search.test.js && node dist/extensions/models/gateway-catalog.test.js && node dist/extensions/models/gateway-catalog-isolation.test.js && node dist/extensions/models/vendor-strategies.test.js && node dist/extensions/host-integration.test.js && node dist/extensions/mcp.test.js && node dist/extensions/mcp-host-integration.test.js && node dist/extensions/skills.test.js && node dist/extensions/upload/protocol.test.js && node dist/extensions/upload/uploader.test.js && node dist/extensions/upload/transport.test.js && node dist/extensions/upload/host-integration.test.js",
|
|
28
|
+
"test:upload:live": "tsc -p tsconfig.test.json && node dist/extensions/upload/live.test.js"
|
|
25
29
|
},
|
|
26
30
|
"peerDependencies": {
|
|
27
31
|
"@earendil-works/pi-coding-agent": ">=0.82.0",
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: delegate-model-selection
|
|
3
|
+
description: 当本技能存在时,应更倾向于把适合委派的任务编排为子代理背景执行。本技能定义编排中的模型与思考深度选择,应尽量遵循;具体任务拆分、并行方式与升级策略由模型结合现场自行判断。
|
|
4
|
+
license: MIT
|
|
5
|
+
compatibility: 需要宿主提供子代理工具及可用的 tsgw 模型。
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# 子代理编排与模型选型
|
|
9
|
+
|
|
10
|
+
当本技能存在时,应更倾向于把适合委派的任务编排为子代理背景执行,并尽量遵循本技能的模型与思考深度定义。具体编排由模型根据任务自行判断。
|
|
11
|
+
|
|
12
|
+
## 选型参考
|
|
13
|
+
|
|
14
|
+
结合任务、套餐、速度和费用参考下表,同档顺序不是单纯按评分排序;表格不是白名单或固定派遣路线。
|
|
15
|
+
|
|
16
|
+
评分为 AA Intelligence Index v4.3.2 原始分;价格单位为美元/百万 tokens,是 API 参考单价而非套餐扣费;思考深度使用 Pi 档位。
|
|
17
|
+
|
|
18
|
+
| 任务等级 | 模型 | 思考深度 | 评分 | 输入价 | 输出价 |
|
|
19
|
+
| --- | --- | --- | ---: | ---: | ---: |
|
|
20
|
+
| 轻量执行 | `MiniMax-M3` | `high` | 29† | 0.30 | 1.20 |
|
|
21
|
+
| 轻量执行 | `gpt-6-luna` | `high` | 33 | 0.10 | 0.50 |
|
|
22
|
+
| 通用工程 | `glm-5.3` | `low` | 34 | 1.18 | 4.13 |
|
|
23
|
+
| 通用工程 | `glm-5.3` | `high` | 待评 | 1.18 | 4.13 |
|
|
24
|
+
| 通用工程 | `glm-5.3` | `max` | 45 | 1.18 | 4.13 |
|
|
25
|
+
| 通用工程 | `gpt-6-luna` | `max` | 38 | 0.10 | 0.50 |
|
|
26
|
+
| 通用工程(低成本备选) | `glm-5.3-flash` | `max` | 42 | 0.118 | 0.413 |
|
|
27
|
+
| 通用工程 | `gpt-6.1-sol` | `medium` | 48 | 2.00 | 10.00 |
|
|
28
|
+
| 按量付费补位 | `deepseek-flash` | `max` | 39 | 0.1475–0.295 | 0.59–1.18 |
|
|
29
|
+
| 复杂任务 | `gpt-6.1-sol` | `high` | 50 | 2.00 | 10.00 |
|
|
30
|
+
| 高难任务 | `gpt-6.1-sol` | `xhigh` | 51 | 2.00 | 10.00 |
|
|
31
|
+
| 高难任务 | `gpt-6-astra` | `high` | 51 | 10.00 | 50.00 |
|
|
32
|
+
| 高难任务 | `gpt-6-astra` | `xhigh` | 52 | 10.00 | 50.00 |
|
|
33
|
+
| 极难任务 | `gpt-6.1-sol` | `max` | 52 | 2.00 | 10.00 |
|
|
34
|
+
| 极难任务 | `gpt-6-astra` | `max` | 53 | 10.00 | 50.00 |
|
|
35
|
+
| 双模型复核专用 | `kimi-k3` | `high` | 待评 | 3.00 | 15.00 |
|
|
36
|
+
|
|
37
|
+
- † MiniMax 的 29 分为模型级参考,Pi high 映射为 adaptive;待评项不借用其他型号或档位的分数。
|
|
38
|
+
- GLM 5.3 三型号始终思考,实际 effort 为 low/high/max;Pi medium 映射 high,xhigh 映射 max。FlashX 可作未定档备选,评分待评,输入/输出参考价为 0.295/1.0325。
|
|
39
|
+
- GPT 价格对应输入 ≤272K,超过后输入 ×2、输出 ×1.5;MiniMax 输入超过 512K 为 0.60/2.40。
|
|
40
|
+
- DeepSeek 采用国内价(2026-10-16 核验,Blinko #604):输入 1–2 元、输出 4–8 元/百万 tokens,按固定 0.1475 USD/CNY 折算;区间为低谷/高峰价。静态目录采用区间高端,不自动按时段切换;高峰为北京时间工作日(不含中国法定节假日)9–12、14–18。
|
|
41
|
+
- GLM 依据 Blinko #929,同样以固定 0.1475 USD/CNY 折算国内人民币标价。其余数值沿用既有选型资料。
|
|
42
|
+
|
|
43
|
+
## 必要边界
|
|
44
|
+
|
|
45
|
+
- **MiniMax-M3** 只承担步骤明确的小任务。参考使用 Pi 的 high;非 off 档位均映射为 adaptive,不代表不同能力档。adaptive 不是 Pi 档位,此映射不适用于 M3.1 Flash Preview。
|
|
46
|
+
- **deepseek-flash** 作为按量付费补位时使用 max。
|
|
47
|
+
- **Kimi** 不承担长程执行;双模型复核固定为 **Kimi+Astra**,独立审查同一份材料。Kimi 使用 kimi-k3 / high,Astra 深度按任务选择。
|
|
48
|
+
- 用户明确要求、预算、权限及宿主实际能力优先;不可用的模型或档位如实说明。
|
|
49
|
+
|
|
50
|
+
宿主支持 `Agent.model = provider/modelId:level` 时,例如 `tsgw/MiniMax-M3:high`、`tsgw/glm-5.3:max`;其他宿主使用其实际格式。
|