cloudgpu-mcp 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +44 -1
- package/package.json +1 -1
- package/rules/cloudgpu.md +13 -0
- package/src/api.js +43 -6
- package/src/credentials.js +34 -0
- package/src/index.js +200 -2
- package/src/storage.js +129 -0
package/README.md
CHANGED
|
@@ -129,6 +129,10 @@ The same guidance is available as two [Agent Skills](https://agentskills.io):
|
|
|
129
129
|
| `upload` / `download` | Files and folders between your computer and the instance (SFTP) |
|
|
130
130
|
| `stop_instance` | Destroys the instance and stops billing |
|
|
131
131
|
| `list_models` | Models on the OpenAI-compatible API, for inference without renting a GPU |
|
|
132
|
+
| `run_job` | Starts a GPU, runs one command and destroys the GPU, all on CloudGPU's servers; safe to retry with the same `idempotency_key` |
|
|
133
|
+
| `get_job` / `list_jobs` | Job status, exit code, cost, the last ~4 KB of output and the job's history |
|
|
134
|
+
| `cancel_job` | Stops a job and destroys its GPU |
|
|
135
|
+
| `connect_account` / `finish_connect` | Gets a key without copying one: the user approves this assistant in the browser with a spending limit and a number of GPUs at once |
|
|
132
136
|
|
|
133
137
|
The server also provides a `train_on_cloudgpu` prompt.
|
|
134
138
|
|
|
@@ -140,13 +144,15 @@ The server also provides a `train_on_cloudgpu` prompt.
|
|
|
140
144
|
- **Results before shutdown.** After commands have run, `stop_instance` refuses until something has been downloaded, unless the user confirms `force=true`.
|
|
141
145
|
- **Secrets stay out of the chat.** The instance's SSH login is used inside this process and never returned to the model. Tool output is scrubbed of the API key and login details. Uploading `.ssh`, `.env`, key files or your whole home directory needs `allow_sensitive=true`.
|
|
142
146
|
- **The instance stays manageable.** Commands that change the root password, SSH configuration, firewall or the platform's monitoring agent are refused. CloudGPU shuts down instances that it can no longer monitor.
|
|
147
|
+
- **Jobs are not started twice.** `run_job` sends an `Idempotency-Key`. Resending the same job returns the same job instead of starting a second GPU, and the server refuses the same key with different parameters.
|
|
148
|
+
- **Keys with a budget.** A key from `connect_account` has a total spending limit, a number of GPUs it may run at once and an expiry date, all enforced by CloudGPU's servers. Revoke it at https://cloudgpu.app/api and it can no longer start anything.
|
|
143
149
|
- **Read-only mode.** `CLOUDGPU_READ_ONLY=1` registers only the look-up tools.
|
|
144
150
|
|
|
145
151
|
## Configuration
|
|
146
152
|
|
|
147
153
|
| Env | Default | |
|
|
148
154
|
|---|---|---|
|
|
149
|
-
| `CLOUDGPU_API_KEY` | (
|
|
155
|
+
| `CLOUDGPU_API_KEY` | (or `connect_account`) | From https://cloudgpu.app/api. Without it, a key saved by `finish_connect` in `~/.config/cloudgpu/credentials.json` is used |
|
|
150
156
|
| `CLOUDGPU_MAX_SPEND_USD` | `50` | Highest budget the agent may set for one instance |
|
|
151
157
|
| `CLOUDGPU_MAX_HOURS` | `24` | Longest time limit the agent may set |
|
|
152
158
|
| `CLOUDGPU_READ_ONLY` | off | `1` = look-up tools only |
|
|
@@ -156,6 +162,43 @@ The server also provides a `train_on_cloudgpu` prompt.
|
|
|
156
162
|
|
|
157
163
|
The key also works with the OpenAI-compatible API at `https://cloudgpu.app/v1`. Call `list_models` (or `GET /v1/models`) to see the available model IDs. Model prices are on [cloudgpu.app/api](https://cloudgpu.app/api).
|
|
158
164
|
|
|
165
|
+
## Optional persistent files and Workspaces (development)
|
|
166
|
+
|
|
167
|
+
These tools require persistent storage to be explicitly enabled on the server;
|
|
168
|
+
they return `storage_not_enabled` until it is ready. Discover server capabilities
|
|
169
|
+
at `/v1/public/compute-capabilities`. Read `get_storage` for current included allowance, quota, prices and retention. Storage fees are separate from GPU Job budgets. Uploads above the included allowance require `accept_storage_charges=true` after user agreement; the default is false. Budget-limited connect-link keys cannot enable paid storage. Files expose `expires_at`; downloading does not renew retention. Versions containing expired files become unavailable, while active Jobs defer deletion.
|
|
170
|
+
|
|
171
|
+
- `get_compute_catalog` and `quote_compute` use public discovery and server quotes,
|
|
172
|
+
including required starting balance and `supports_jobs`. Pass the server quote to
|
|
173
|
+
`run_job` as `quote_id`; the original `estimate_cost` remains a local quote for
|
|
174
|
+
interactive instance rentals.
|
|
175
|
+
- `get_storage`, `list_files`, `list_workspaces`, `list_workspace_versions` and
|
|
176
|
+
`get_workspace_version` show saved files, versions and usage.
|
|
177
|
+
- `upload_artifact` and `download_artifact` transfer persistent files, including
|
|
178
|
+
datasets/models, without returning storage credentials to the assistant.
|
|
179
|
+
- `save_workspace` saves regular project files as an immutable version. It refuses
|
|
180
|
+
known credential paths and symlinks, and skips `.git`, `node_modules`,
|
|
181
|
+
`__pycache__` and `.venv`. Save dependency manifests so the next GPU can rebuild
|
|
182
|
+
its environment. The returned `workspace_version_id` can be passed to `run_job`.
|
|
183
|
+
- `delete_file` and `delete_workspace_version` require explicit user confirmation
|
|
184
|
+
and refuse active references.
|
|
185
|
+
|
|
186
|
+
This restores project files into a fresh directory before the Job command. It is
|
|
187
|
+
not full-disk restore and does not automatically save edits or collect outputs.
|
|
188
|
+
Continue saving results from the command itself. Large-model restore speed and
|
|
189
|
+
cross-supplier cache behavior must be measured before relying on them. Backend restore plans travel via SSH stdin, never in
|
|
190
|
+
Job command arguments or saved scripts; the user command starts after restoration.
|
|
191
|
+
This does not protect transfer-time memory from root on the GPU. Failed deletions
|
|
192
|
+
or expired uploads may remain pending while the server verifies cleanup; quota is
|
|
193
|
+
released only after confirmed removal. With storage disabled, normal catalog,
|
|
194
|
+
account, instance and Job tools keep working without a storage provider.
|
|
195
|
+
|
|
196
|
+
|
|
197
|
+
The developer acceptance script `node scripts/acceptance.mjs` is read-only by
|
|
198
|
+
default. Its `--paid` mode is for an explicitly authorized internal test account
|
|
199
|
+
only; it can create two Jobs capped at $0.03 each. Do not run paid mode as part of
|
|
200
|
+
ordinary package tests.
|
|
201
|
+
|
|
159
202
|
## Without MCP
|
|
160
203
|
|
|
161
204
|
The server wraps CloudGPU's public REST API, which any agent or script can call with the same key:
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "cloudgpu-mcp",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"mcpName": "io.github.mrhy7176/cloudgpu-mcp",
|
|
5
5
|
"description": "MCP server and command-line tool for cloudgpu.app: let Claude Code, Cursor, Cline, opencode, goose and other MCP clients (or you, from a terminal) rent a GPU per minute, run a job, fetch the results and shut it down, with spending limits the platform enforces.",
|
|
6
6
|
"type": "module",
|
package/rules/cloudgpu.md
CHANGED
|
@@ -32,6 +32,19 @@ Copy this file into your project as `AGENTS.md`, `.cursor/rules/cloudgpu.md` or
|
|
|
32
32
|
- **Do not** change the root password, edit the SSH configuration, change the firewall, or stop the platform's monitoring agent (`cgpu-*` processes). The platform must keep its access to the instance; if that access is lost, the platform shuts the instance down. The MCP server refuses such commands.
|
|
33
33
|
- Never use the GPU for cryptocurrency mining or anything the [acceptable use policy](https://cloudgpu.app/aup) forbids. Instances that do so are shut down.
|
|
34
34
|
|
|
35
|
+
## Jobs: one command, start to finish on CloudGPU
|
|
36
|
+
|
|
37
|
+
For a batch task that is one command (a training script, a render, a batch of inference), `run_job` is simpler than renting an instance: CloudGPU starts the GPU, runs the command, and destroys the GPU when it finishes, even if you disconnect.
|
|
38
|
+
|
|
39
|
+
- Get the user's yes on the GPU, the budget and the time limit first, as for renting.
|
|
40
|
+
- The GPU's disk is erased when the job ends. The command must upload its results itself (for example to the user's own S3 bucket, with credentials the user gave for that) or print them. `get_job` returns the last ~4 KB of output.
|
|
41
|
+
- If `run_job` times out or the connection drops, call it again with the same `idempotency_key`. You get the same job back; a second GPU is not started.
|
|
42
|
+
- Check progress with `get_job` every minute or so. `needs_attention` means CloudGPU stopped the job to check something by hand; tell the user and do not resubmit it.
|
|
43
|
+
|
|
44
|
+
## No API key?
|
|
45
|
+
|
|
46
|
+
Call `connect_account` with a name for yourself and the spending limit the user wants. Show the user the link; they sign in and approve. Then call `finish_connect`. The key is saved on this computer and never shown in the chat. The user can revoke it at https://cloudgpu.app/api.
|
|
47
|
+
|
|
35
48
|
## Finishing (always)
|
|
36
49
|
|
|
37
50
|
1. Download every result: model weights, adapters, checkpoints, logs, metrics. Destroying an instance erases its disk, and CloudGPU has no persistent storage.
|
package/src/api.js
CHANGED
|
@@ -18,8 +18,11 @@ export class CloudGpuApi {
|
|
|
18
18
|
this.userAgent = userAgent;
|
|
19
19
|
}
|
|
20
20
|
|
|
21
|
-
|
|
22
|
-
|
|
21
|
+
// opts.idempotencyKey: sent as Idempotency-Key; such a POST is safe to retry because the server returns the
|
|
22
|
+
// same job for the same key instead of starting a second GPU.
|
|
23
|
+
// opts.noAuth: endpoints used before there is a key (agent connect).
|
|
24
|
+
async request(method, path, body, opts = {}) {
|
|
25
|
+
if (!this.apiKey && !opts.noAuth) {
|
|
23
26
|
throw new ApiError(
|
|
24
27
|
401,
|
|
25
28
|
"missing_api_key",
|
|
@@ -27,14 +30,17 @@ export class CloudGpuApi {
|
|
|
27
30
|
);
|
|
28
31
|
}
|
|
29
32
|
let lastErr;
|
|
30
|
-
// Retry only on network errors and 502/503/504; never on 4xx
|
|
31
|
-
|
|
33
|
+
// Retry only on network errors and 502/503/504; never on 4xx. A POST is retried only when it carries an
|
|
34
|
+
// Idempotency-Key, because only then can it not start a second GPU.
|
|
35
|
+
const retryable = method === "GET" || Boolean(opts.idempotencyKey);
|
|
36
|
+
const attempts = retryable ? 3 : 1;
|
|
32
37
|
for (let i = 0; i < attempts; i++) {
|
|
33
38
|
try {
|
|
34
39
|
const res = await this.fetch(this.baseUrl + path, {
|
|
35
40
|
method,
|
|
36
41
|
headers: {
|
|
37
|
-
Authorization: `Bearer ${this.apiKey}
|
|
42
|
+
...(this.apiKey && !opts.noAuth ? { Authorization: `Bearer ${this.apiKey}` } : {}),
|
|
43
|
+
...(opts.idempotencyKey ? { "Idempotency-Key": opts.idempotencyKey } : {}),
|
|
38
44
|
"Content-Type": "application/json",
|
|
39
45
|
"User-Agent": this.userAgent,
|
|
40
46
|
},
|
|
@@ -50,7 +56,7 @@ export class CloudGpuApi {
|
|
|
50
56
|
if (!res.ok) {
|
|
51
57
|
const e = json && json.error ? json.error : {};
|
|
52
58
|
const err = new ApiError(res.status, e.code || `http_${res.status}`, e.message || `HTTP ${res.status}`);
|
|
53
|
-
if (
|
|
59
|
+
if (retryable && res.status >= 502 && res.status <= 504 && i < attempts - 1) {
|
|
54
60
|
lastErr = err;
|
|
55
61
|
await sleep(1000 * (i + 1));
|
|
56
62
|
continue;
|
|
@@ -94,6 +100,37 @@ export class CloudGpuApi {
|
|
|
94
100
|
models() {
|
|
95
101
|
return this.request("GET", "/v1/models").then((r) => r.data || []);
|
|
96
102
|
}
|
|
103
|
+
catalog() { return this.request("GET", "/v1/public/gpu-catalog", undefined, { noAuth: !this.apiKey }); }
|
|
104
|
+
quote(body) { return this.request("POST", "/v1/quotes", body, { noAuth: !this.apiKey }); }
|
|
105
|
+
|
|
106
|
+
// ── jobs: start a GPU, run one command, destroy the GPU (server-side) ──
|
|
107
|
+
createJob(body, idempotencyKey) {
|
|
108
|
+
if (!idempotencyKey) throw new Error("createJob needs an idempotency key");
|
|
109
|
+
return this.request("POST", "/v1/jobs", body, { idempotencyKey });
|
|
110
|
+
}
|
|
111
|
+
job(id) {
|
|
112
|
+
return this.request("GET", `/v1/jobs/${encodeURIComponent(id)}`);
|
|
113
|
+
}
|
|
114
|
+
jobs(limit = 20) {
|
|
115
|
+
return this.request("GET", `/v1/jobs?limit=${encodeURIComponent(limit)}`).then((r) => r.data || []);
|
|
116
|
+
}
|
|
117
|
+
jobLogs(id) {
|
|
118
|
+
return this.request("GET", `/v1/jobs/${encodeURIComponent(id)}/logs`);
|
|
119
|
+
}
|
|
120
|
+
jobEvents(id) {
|
|
121
|
+
return this.request("GET", `/v1/jobs/${encodeURIComponent(id)}/events`).then((r) => r.data || []);
|
|
122
|
+
}
|
|
123
|
+
cancelJob(id) {
|
|
124
|
+
return this.request("POST", `/v1/jobs/${encodeURIComponent(id)}/cancel`, {});
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
// ── agent connect: get a key with a budget, approved by the account owner in the browser ──
|
|
128
|
+
connectStart(body) {
|
|
129
|
+
return this.request("POST", "/v1/agent/connect", body, { noAuth: true });
|
|
130
|
+
}
|
|
131
|
+
connectToken(deviceCode) {
|
|
132
|
+
return this.request("POST", "/v1/agent/connect/token", { device_code: deviceCode }, { noAuth: true });
|
|
133
|
+
}
|
|
97
134
|
}
|
|
98
135
|
|
|
99
136
|
export function sleep(ms) {
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
// Where a key obtained through "connect_account" is kept, so the user does not have to paste it anywhere.
|
|
2
|
+
// File: ~/.config/cloudgpu/credentials.json (or $CLOUDGPU_CONFIG_DIR), readable only by the user.
|
|
3
|
+
// CLOUDGPU_API_KEY in the environment always wins over the file.
|
|
4
|
+
import fs from "node:fs";
|
|
5
|
+
import os from "node:os";
|
|
6
|
+
import path from "node:path";
|
|
7
|
+
|
|
8
|
+
export function configDir(env = process.env) {
|
|
9
|
+
return env.CLOUDGPU_CONFIG_DIR || path.join(os.homedir(), ".config", "cloudgpu");
|
|
10
|
+
}
|
|
11
|
+
|
|
12
|
+
export function loadStoredKey(env = process.env) {
|
|
13
|
+
try {
|
|
14
|
+
const j = JSON.parse(fs.readFileSync(path.join(configDir(env), "credentials.json"), "utf8"));
|
|
15
|
+
if (j && typeof j.api_key === "string" && j.api_key.trim()) {
|
|
16
|
+
if (j.expires_at && Number(j.expires_at) * 1000 < Date.now()) return null;
|
|
17
|
+
return j;
|
|
18
|
+
}
|
|
19
|
+
} catch {
|
|
20
|
+
// no file or unreadable: no stored key
|
|
21
|
+
}
|
|
22
|
+
return null;
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
export function saveStoredKey(cred, env = process.env) {
|
|
26
|
+
const dir = configDir(env);
|
|
27
|
+
fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
|
|
28
|
+
const file = path.join(dir, "credentials.json");
|
|
29
|
+
const tmp = file + ".tmp";
|
|
30
|
+
fs.writeFileSync(tmp, JSON.stringify(cred, null, 2), { mode: 0o600 });
|
|
31
|
+
fs.renameSync(tmp, file);
|
|
32
|
+
fs.chmodSync(file, 0o600);
|
|
33
|
+
return file;
|
|
34
|
+
}
|
package/src/index.js
CHANGED
|
@@ -16,6 +16,9 @@ import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"
|
|
|
16
16
|
import { z } from "zod";
|
|
17
17
|
import { CloudGpuApi, ApiError } from "./api.js";
|
|
18
18
|
import { SshPool, exec, upload, download, writeFile } from "./ssh.js";
|
|
19
|
+
import { loadStoredKey, saveStoredKey } from "./credentials.js";
|
|
20
|
+
import crypto from "node:crypto";
|
|
21
|
+
import { registerStorageTools } from "./storage.js";
|
|
19
22
|
import {
|
|
20
23
|
checkCommand,
|
|
21
24
|
sensitiveReason,
|
|
@@ -27,12 +30,14 @@ import {
|
|
|
27
30
|
shq,
|
|
28
31
|
} from "./guards.js";
|
|
29
32
|
|
|
30
|
-
const VERSION = "0.
|
|
33
|
+
const VERSION = "0.5.0";
|
|
31
34
|
const HERE = path.dirname(fileURLToPath(import.meta.url));
|
|
32
35
|
const RULES = fs.readFileSync(path.join(HERE, "..", "rules", "cloudgpu.md"), "utf8");
|
|
33
36
|
|
|
37
|
+
// A key from connect_account (saved on this computer) is used when CLOUDGPU_API_KEY is not set.
|
|
38
|
+
const stored = loadStoredKey();
|
|
34
39
|
const cfg = {
|
|
35
|
-
apiKey: (process.env.CLOUDGPU_API_KEY || "").trim(),
|
|
40
|
+
apiKey: (process.env.CLOUDGPU_API_KEY || "").trim() || (stored ? stored.api_key : ""),
|
|
36
41
|
baseUrl: process.env.CLOUDGPU_BASE_URL || "https://cloudgpu.app",
|
|
37
42
|
maxSpend: num(process.env.CLOUDGPU_MAX_SPEND_USD, 50),
|
|
38
43
|
maxHours: num(process.env.CLOUDGPU_MAX_HOURS, 24),
|
|
@@ -53,9 +58,12 @@ const session = new Map();
|
|
|
53
58
|
const INSTRUCTIONS = `CloudGPU (cloudgpu.app) rents GPUs billed per minute from a prepaid balance.
|
|
54
59
|
Workflow: get_account and list_gpus -> estimate_cost -> show the quote to the user and get an explicit yes -> create_instance -> get_status until running -> upload -> run_command (background for long jobs) -> tail_logs -> download results -> stop_instance -> report the actual cost.
|
|
55
60
|
Rules: never rent without the user's yes on a quote; always set a budget and time limit (the platform shuts the instance down when either is hit); download results before stop_instance because destroying an instance erases its disk; never change passwords, SSH config, the firewall or the platform's monitoring agent on the instance; if anything fails, download what exists and stop the instance rather than leaving it running.
|
|
61
|
+
Jobs: for a batch task (one command that writes its results somewhere), run_job does the whole thing on CloudGPU's servers: start a GPU, run the command, destroy the GPU, with the same per-job budget and time limit. Check it with get_job. The GPU's disk is erased when the job ends, so the command must upload its results itself (for example to the user's own S3 bucket) or print them; get_job returns the last ~4 KB of output.
|
|
62
|
+
No API key yet? connect_account gives the user a link to approve this assistant with a spending limit; then finish_connect stores the key on this computer.
|
|
56
63
|
The same API key also works for the OpenAI-compatible model API at ${cfg.baseUrl}/v1 (see list_models).`;
|
|
57
64
|
|
|
58
65
|
const server = new McpServer({ name: "cloudgpu", version: VERSION }, { instructions: INSTRUCTIONS });
|
|
66
|
+
registerStorageTools(server, { api, wrap, ok, fail, readOnly: cfg.readOnly, expandHome });
|
|
59
67
|
|
|
60
68
|
// ── helpers ──────────────────────────────────────────────────────────────
|
|
61
69
|
|
|
@@ -120,6 +128,20 @@ function cleanOut(s, secrets) {
|
|
|
120
128
|
|
|
121
129
|
// ── account and catalogue ────────────────────────────────────────────────
|
|
122
130
|
|
|
131
|
+
server.registerTool("get_compute_catalog", {
|
|
132
|
+
title: "Public GPU catalog", description: "GPU catalog, templates and billing without an API key. Does not rent anything.",
|
|
133
|
+
inputSchema: {}, annotations: { readOnlyHint: true, openWorldHint: true },
|
|
134
|
+
}, wrap(async () => ok(await api.catalog())));
|
|
135
|
+
server.registerTool("quote_compute", {
|
|
136
|
+
title: "Server-side compute quote", description: "Five-minute server quote: separates estimated task cost from required starting balance and marks supports_jobs. No GPU is started. Pass its quote_id to run_job.",
|
|
137
|
+
inputSchema: { min_vram_gb: z.number().int().min(0).default(0), gpu_model: z.string().optional(), gpu_count: z.number().int().min(1).max(8).default(1),
|
|
138
|
+
template: z.string().optional(), disk_gb: z.number().int().optional(), estimated_duration_seconds: z.number().int().positive(), max_budget_usd: z.number().positive() },
|
|
139
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
140
|
+
}, wrap(async ({ estimated_duration_seconds, max_budget_usd, ...requirements }) => {
|
|
141
|
+
const q = await api.quote({ requirements, estimated_duration_seconds, max_budget_usd });
|
|
142
|
+
return ok({ ...q, quote_id: q.id });
|
|
143
|
+
}));
|
|
144
|
+
|
|
123
145
|
server.registerTool(
|
|
124
146
|
"get_account",
|
|
125
147
|
{
|
|
@@ -248,6 +270,118 @@ server.registerTool(
|
|
|
248
270
|
}),
|
|
249
271
|
);
|
|
250
272
|
|
|
273
|
+
// ── connect (get a key with a budget, approved in the browser) ──────────────
|
|
274
|
+
|
|
275
|
+
let pendingConnect = null;
|
|
276
|
+
|
|
277
|
+
server.registerTool(
|
|
278
|
+
"connect_account",
|
|
279
|
+
{
|
|
280
|
+
title: "Connect this assistant to a CloudGPU account",
|
|
281
|
+
description:
|
|
282
|
+
"Use when there is no API key. Starts an approval request and returns a link. The user opens it, signs in to cloudgpu.app and approves this assistant " +
|
|
283
|
+
"with a spending limit and a maximum number of GPUs at once; CloudGPU's servers enforce both. Then call finish_connect. No money is spent.",
|
|
284
|
+
inputSchema: {
|
|
285
|
+
agent_name: z.string().min(1).max(64).describe("How the user will recognise this assistant, e.g. 'Claude Code on Yang's laptop'"),
|
|
286
|
+
spend_limit_usd: z.number().min(1).max(10000).describe("Total this key may ever spend, in USD; ask the user"),
|
|
287
|
+
max_concurrent: z.number().int().min(1).max(10).default(1).describe("GPUs or jobs this key may run at once"),
|
|
288
|
+
expires_in_days: z.number().int().min(1).max(90).default(30),
|
|
289
|
+
},
|
|
290
|
+
annotations: { readOnlyHint: false, destructiveHint: false, idempotentHint: false, openWorldHint: true },
|
|
291
|
+
},
|
|
292
|
+
wrap(async ({ agent_name, spend_limit_usd, max_concurrent = 1, expires_in_days = 30 }) => {
|
|
293
|
+
const r = await api.connectStart({ agent_name, spend_limit_usd, max_concurrent, expires_in_days });
|
|
294
|
+
pendingConnect = { device_code: r.device_code, until: Date.now() + Number(r.expires_in || 600) * 1000, interval: Number(r.interval || 5) };
|
|
295
|
+
return ok({
|
|
296
|
+
link: r.verification_uri_complete,
|
|
297
|
+
code: r.user_code,
|
|
298
|
+
expires_in_seconds: r.expires_in,
|
|
299
|
+
next: "Show the link to the user and ask them to open it, sign in and approve. Then call finish_connect.",
|
|
300
|
+
});
|
|
301
|
+
}),
|
|
302
|
+
);
|
|
303
|
+
|
|
304
|
+
server.registerTool(
|
|
305
|
+
"finish_connect",
|
|
306
|
+
{
|
|
307
|
+
title: "Finish connecting",
|
|
308
|
+
description:
|
|
309
|
+
"After the user approved the link from connect_account: waits up to about a minute for the approval, then stores the new key on this computer " +
|
|
310
|
+
"(~/.config/cloudgpu/credentials.json, readable only by the user) and uses it from now on. The key itself is never shown.",
|
|
311
|
+
inputSchema: {},
|
|
312
|
+
annotations: { readOnlyHint: false, destructiveHint: false, idempotentHint: true, openWorldHint: true },
|
|
313
|
+
},
|
|
314
|
+
wrap(async () => {
|
|
315
|
+
if (!pendingConnect) return fail("No approval is pending. Call connect_account first.");
|
|
316
|
+
const deadline = Math.min(Date.now() + 60_000, pendingConnect.until);
|
|
317
|
+
for (;;) {
|
|
318
|
+
try {
|
|
319
|
+
const cred = await api.connectToken(pendingConnect.device_code);
|
|
320
|
+
const file = saveStoredKey({
|
|
321
|
+
api_key: cred.api_key,
|
|
322
|
+
key_id: cred.key_id,
|
|
323
|
+
agent_name: cred.agent_name,
|
|
324
|
+
spend_limit_usd: cred.spend_limit_usd,
|
|
325
|
+
max_concurrent: cred.max_concurrent,
|
|
326
|
+
expires_at: cred.expires_at,
|
|
327
|
+
base_url: cfg.baseUrl,
|
|
328
|
+
});
|
|
329
|
+
cfg.apiKey = cred.api_key;
|
|
330
|
+
api.apiKey = cred.api_key;
|
|
331
|
+
pendingConnect = null;
|
|
332
|
+
return ok({
|
|
333
|
+
connected: true,
|
|
334
|
+
agent_name: cred.agent_name,
|
|
335
|
+
spend_limit_usd: cred.spend_limit_usd,
|
|
336
|
+
max_concurrent: cred.max_concurrent,
|
|
337
|
+
expires_at: cred.expires_at,
|
|
338
|
+
saved_to: file,
|
|
339
|
+
note: "The account owner can revoke this key at any time at https://cloudgpu.app/api.",
|
|
340
|
+
});
|
|
341
|
+
} catch (err) {
|
|
342
|
+
if (err instanceof ApiError && err.code === "authorization_pending" && Date.now() + pendingConnect.interval * 1000 < deadline) {
|
|
343
|
+
await new Promise((r) => setTimeout(r, pendingConnect.interval * 1000));
|
|
344
|
+
continue;
|
|
345
|
+
}
|
|
346
|
+
if (err instanceof ApiError && err.code === "authorization_pending") {
|
|
347
|
+
return fail("Not approved yet. Ask the user to open the link and approve, then call finish_connect again.");
|
|
348
|
+
}
|
|
349
|
+
if (err instanceof ApiError && (err.code === "access_denied" || err.code === "expired_token")) pendingConnect = null;
|
|
350
|
+
throw err;
|
|
351
|
+
}
|
|
352
|
+
}
|
|
353
|
+
}),
|
|
354
|
+
);
|
|
355
|
+
|
|
356
|
+
// ── jobs (read) ───────────────────────────────────────────────────────────
|
|
357
|
+
|
|
358
|
+
server.registerTool(
|
|
359
|
+
"get_job",
|
|
360
|
+
{
|
|
361
|
+
title: "Job status",
|
|
362
|
+
description:
|
|
363
|
+
"Status of a job (queued / provisioning / running / stopping / succeeded / failed / cancelled / needs_attention), exit code, cost so far, " +
|
|
364
|
+
"the last ~4 KB of the command's output and the job's event history.",
|
|
365
|
+
inputSchema: { job_id: z.string() },
|
|
366
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
367
|
+
},
|
|
368
|
+
wrap(async ({ job_id }) => {
|
|
369
|
+
const [job, logs, events] = await Promise.all([api.job(job_id), api.jobLogs(job_id).catch(() => null), api.jobEvents(job_id).catch(() => [])]);
|
|
370
|
+
return ok({ ...job, output_tail: logs ? logs.tail : null, events });
|
|
371
|
+
}),
|
|
372
|
+
);
|
|
373
|
+
|
|
374
|
+
server.registerTool(
|
|
375
|
+
"list_jobs",
|
|
376
|
+
{
|
|
377
|
+
title: "List jobs",
|
|
378
|
+
description: "Recent jobs on this account with status and cost.",
|
|
379
|
+
inputSchema: { limit: z.number().int().min(1).max(100).default(20) },
|
|
380
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
381
|
+
},
|
|
382
|
+
wrap(async ({ limit = 20 }) => ok({ jobs: await api.jobs(limit) })),
|
|
383
|
+
);
|
|
384
|
+
|
|
251
385
|
// ── instances (read) ─────────────────────────────────────────────────────
|
|
252
386
|
|
|
253
387
|
server.registerTool(
|
|
@@ -346,6 +480,70 @@ if (!cfg.readOnly) {
|
|
|
346
480
|
}),
|
|
347
481
|
);
|
|
348
482
|
|
|
483
|
+
server.registerTool(
|
|
484
|
+
"run_job",
|
|
485
|
+
{
|
|
486
|
+
title: "Run a job on a GPU",
|
|
487
|
+
description:
|
|
488
|
+
"Costs money. CloudGPU starts a GPU, runs one bash command on it, then destroys the GPU, all on its servers, so it finishes even if this assistant disconnects. " +
|
|
489
|
+
"max_spend_usd and max_hours are enforced by the servers. Retrying with the same idempotency_key never starts a second GPU; this tool generates one if you leave it out and returns it. " +
|
|
490
|
+
"The disk is erased when the job ends: the command must upload its own results (for example to the user's storage) or print them. " +
|
|
491
|
+
"Requires the user's explicit yes on the GPU, budget and time limit.",
|
|
492
|
+
inputSchema: {
|
|
493
|
+
command: z.string().min(1).max(32768).describe("bash command or script to run on the GPU"),
|
|
494
|
+
gpu_model: z.string().optional().describe("GPU model exactly as list_gpus returns it"),
|
|
495
|
+
template: z.string().optional().describe("Template id from list_templates"),
|
|
496
|
+
gpu_count: z.number().int().min(1).max(8).default(1),
|
|
497
|
+
disk_gb: z.number().int().optional().describe("Larger system disk: 100, 200 or 500; omit for the standard disk"),
|
|
498
|
+
workspace_version_id: z.string().optional().describe("Immutable Workspace version to restore before the command; requires enabled persistent storage"),
|
|
499
|
+
quote_id: z.string().optional().describe("Server quote from quote_compute, starting quote_; binds configuration and checks price at submission"),
|
|
500
|
+
max_spend_usd: z.number().positive().describe(`Budget for this job in USD (at most ${cfg.maxSpend})`),
|
|
501
|
+
max_hours: z.number().positive().describe(`Hard time limit in hours (at most ${cfg.maxHours})`),
|
|
502
|
+
user_confirmed: z.boolean().describe("true only if the user explicitly approved the GPU, budget and time limit"),
|
|
503
|
+
idempotency_key: z.string().max(128).optional().describe("Reuse the same value when retrying the same job"),
|
|
504
|
+
},
|
|
505
|
+
annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: true, openWorldHint: true },
|
|
506
|
+
},
|
|
507
|
+
wrap(async ({ command, gpu_model, template, gpu_count = 1, disk_gb, workspace_version_id, quote_id, max_spend_usd, max_hours, user_confirmed, idempotency_key }) => {
|
|
508
|
+
if (user_confirmed !== true) {
|
|
509
|
+
return fail("Not started: tell the user the GPU, budget and time limit, wait for an explicit yes, then call again with user_confirmed=true.");
|
|
510
|
+
}
|
|
511
|
+
if (!gpu_model && !template && !quote_id) return fail("Give gpu_model, template or a server quote_id from quote_compute.");
|
|
512
|
+
if (max_spend_usd > cfg.maxSpend) {
|
|
513
|
+
return fail(`max_spend_usd ${max_spend_usd} is above this server's cap of ${cfg.maxSpend} (CLOUDGPU_MAX_SPEND_USD).`);
|
|
514
|
+
}
|
|
515
|
+
if (max_hours > cfg.maxHours) {
|
|
516
|
+
return fail(`max_hours ${max_hours} is above this server's cap of ${cfg.maxHours} (CLOUDGPU_MAX_HOURS).`);
|
|
517
|
+
}
|
|
518
|
+
const refused = checkCommand(command);
|
|
519
|
+
if (refused) return fail(refused);
|
|
520
|
+
const key = idempotency_key || crypto.randomUUID();
|
|
521
|
+
const body = { command, gpu_count, max_minutes: Math.ceil(max_hours * 60), max_spend_usd };
|
|
522
|
+
if (gpu_model) body.gpu_model = gpu_model;
|
|
523
|
+
if (template) body.template = template;
|
|
524
|
+
if (disk_gb) body.disk_gb = disk_gb;
|
|
525
|
+
if (workspace_version_id) body.workspace_version_id = workspace_version_id;
|
|
526
|
+
if (quote_id) body.quote_id = quote_id;
|
|
527
|
+
const job = await api.createJob(body, key);
|
|
528
|
+
return ok({
|
|
529
|
+
...job,
|
|
530
|
+
idempotency_key: key,
|
|
531
|
+
next: "Call get_job at intervals (every minute or so, not in a tight loop) until status is succeeded, failed or cancelled, then report the cost.",
|
|
532
|
+
});
|
|
533
|
+
}),
|
|
534
|
+
);
|
|
535
|
+
|
|
536
|
+
server.registerTool(
|
|
537
|
+
"cancel_job",
|
|
538
|
+
{
|
|
539
|
+
title: "Cancel a job",
|
|
540
|
+
description: "Stops a job: if a GPU was started it is destroyed and billing stops. Output not yet uploaded by the command is lost.",
|
|
541
|
+
inputSchema: { job_id: z.string() },
|
|
542
|
+
annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: true, openWorldHint: true },
|
|
543
|
+
},
|
|
544
|
+
wrap(async ({ job_id }) => ok(await api.cancelJob(job_id))),
|
|
545
|
+
);
|
|
546
|
+
|
|
349
547
|
server.registerTool(
|
|
350
548
|
"run_command",
|
|
351
549
|
{
|
package/src/storage.js
ADDED
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
// 持久文件直接流向对象存储;长期 key 只发给 CloudGPU,不发给签名 URL 的主机。
|
|
2
|
+
import fs from "node:fs";
|
|
3
|
+
import fsp from "node:fs/promises";
|
|
4
|
+
import path from "node:path";
|
|
5
|
+
import { pipeline } from "node:stream/promises";
|
|
6
|
+
import { Readable, Transform } from "node:stream";
|
|
7
|
+
import { sensitiveReason } from "./guards.js";
|
|
8
|
+
import { z } from "zod";
|
|
9
|
+
|
|
10
|
+
export class PersistentStorage {
|
|
11
|
+
constructor(api, fetchImpl = globalThis.fetch) { this.api = api; this.fetch = fetchImpl; }
|
|
12
|
+
async upload(local, purpose = "artifact", name = path.basename(local), acceptStorageCharges = false) {
|
|
13
|
+
const stat = await fsp.stat(local);
|
|
14
|
+
if (!stat.isFile()) throw new Error("Upload needs a regular file.");
|
|
15
|
+
const file = await this.api.request("POST", "/v1/files", { name, purpose, size_bytes: stat.size, accept_storage_charges: acceptStorageCharges === true });
|
|
16
|
+
try {
|
|
17
|
+
for (let i = 1, offset = 0; i <= file.part_count; i++) {
|
|
18
|
+
const part = await this.api.request("POST", `/v1/files/${file.file_id}/parts/${i}`, {});
|
|
19
|
+
if (!String(part.url).startsWith("https://")) throw new Error("Storage must use HTTPS.");
|
|
20
|
+
const len = Number(part.content_length);
|
|
21
|
+
if (!Number.isSafeInteger(len) || len < 0 || offset + len > stat.size || (len === 0 && stat.size !== 0)) throw new Error("Invalid upload reservation.");
|
|
22
|
+
const stream = len === 0 ? Buffer.alloc(0) : fs.createReadStream(local, { start: offset, end: offset + len - 1 });
|
|
23
|
+
try {
|
|
24
|
+
const res = await this.fetch(part.url, { method: "PUT", headers: { "Content-Length": String(len) }, body: stream, duplex: "half", redirect: "error" });
|
|
25
|
+
if (!res.ok) throw new Error("Part upload failed.");
|
|
26
|
+
if (res.body) await res.body.cancel();
|
|
27
|
+
} catch { throw new Error("Storage part upload failed; no credentials were sent to the API response."); }
|
|
28
|
+
finally { if (stream.destroy) stream.destroy(); }
|
|
29
|
+
offset += len;
|
|
30
|
+
}
|
|
31
|
+
const after = await fsp.stat(local);
|
|
32
|
+
if (after.size !== stat.size || after.mtimeMs !== stat.mtimeMs) throw new Error("File changed during upload; snapshot was not saved.");
|
|
33
|
+
return await this.api.request("POST", `/v1/files/${file.file_id}/complete`, {});
|
|
34
|
+
} catch (e) {
|
|
35
|
+
await this.api.request("DELETE", `/v1/files/${file.file_id}`).catch(() => {});
|
|
36
|
+
throw e;
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
async download(id, local) {
|
|
40
|
+
const transfer = await this.api.request("POST", `/v1/files/${encodeURIComponent(id)}/download`, {});
|
|
41
|
+
if (!String(transfer.url).startsWith("https://")) throw new Error("Storage must use HTTPS.");
|
|
42
|
+
const expected = Number(transfer.size_bytes);
|
|
43
|
+
if (!Number.isSafeInteger(expected) || expected < 0) throw new Error("Invalid file size.");
|
|
44
|
+
await fsp.mkdir(path.dirname(local), { recursive: true });
|
|
45
|
+
const handle = await fsp.open(local, "wx", 0o600); // 不覆盖已有结果。
|
|
46
|
+
try {
|
|
47
|
+
const res = await this.fetch(transfer.url, { redirect: "error" });
|
|
48
|
+
if (!res.ok || !res.body) throw new Error("Download failed.");
|
|
49
|
+
let received = 0;
|
|
50
|
+
const bounded = new Transform({ transform(chunk, _encoding, done) {
|
|
51
|
+
received += chunk.length;
|
|
52
|
+
done(received > expected ? new Error("Storage returned excess bytes.") : null, received > expected ? undefined : chunk);
|
|
53
|
+
} });
|
|
54
|
+
await pipeline(Readable.fromWeb(res.body), bounded, handle.createWriteStream());
|
|
55
|
+
if ((await fsp.stat(local)).size !== Number(transfer.size_bytes)) throw new Error("Downloaded size mismatch.");
|
|
56
|
+
return { local_path: local, bytes: transfer.size_bytes };
|
|
57
|
+
} catch {
|
|
58
|
+
await handle.close().catch(() => {}); await fsp.unlink(local).catch(() => {});
|
|
59
|
+
throw new Error("Storage download failed; the partial file was removed.");
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
async saveWorkspace(localDir, name, workspaceId, acceptStorageCharges = false) {
|
|
63
|
+
const root = path.resolve(localDir), selected = [];
|
|
64
|
+
const walk = async dir => {
|
|
65
|
+
for (const e of await fsp.readdir(dir, { withFileTypes: true })) {
|
|
66
|
+
if ([".git", "node_modules", "__pycache__", ".venv"].includes(e.name)) continue;
|
|
67
|
+
const full = path.join(dir, e.name);
|
|
68
|
+
if (e.isSymbolicLink()) throw new Error("Workspace contains a symlink; choose regular project files.");
|
|
69
|
+
if (sensitiveReason(full)) throw new Error("Workspace contains a credential file. Remove it before saving.");
|
|
70
|
+
if (e.isDirectory()) await walk(full);
|
|
71
|
+
else if (e.isFile()) selected.push({ full, rel: path.relative(root, full).split(path.sep).join("/") });
|
|
72
|
+
}
|
|
73
|
+
};
|
|
74
|
+
await walk(root);
|
|
75
|
+
// Job runner 当前一次恢复最多 100 个文件;不悄悄保存一个不能用的工作区。
|
|
76
|
+
if (selected.length < 1 || selected.length > 100) throw new Error("Workspace must contain 1–100 regular files; binary environments and full disk images are not supported.");
|
|
77
|
+
const created = [], manifest = {};
|
|
78
|
+
try {
|
|
79
|
+
for (const file of selected) {
|
|
80
|
+
const saved = await this.upload(file.full, "workspace", file.rel, acceptStorageCharges);
|
|
81
|
+
created.push(saved.file_id); manifest[file.rel] = saved.file_id;
|
|
82
|
+
}
|
|
83
|
+
const ws = workspaceId ? { workspace_id: workspaceId } : await this.api.request("POST", "/v1/workspaces", { name });
|
|
84
|
+
const version = await this.api.request("POST", `/v1/workspaces/${encodeURIComponent(ws.workspace_id)}/versions`, manifest);
|
|
85
|
+
return { ...version, workspace_version_id: version.version_id };
|
|
86
|
+
} catch (e) {
|
|
87
|
+
for (const id of created) await this.api.request("DELETE", `/v1/files/${id}`).catch(() => {});
|
|
88
|
+
throw e;
|
|
89
|
+
}
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
export function registerStorageTools(server, { api, wrap, ok, fail, readOnly, expandHome }) {
|
|
94
|
+
const client = new PersistentStorage(api);
|
|
95
|
+
server.registerTool("get_storage", { title: "Persistent storage quota", description: "Storage availability, used bytes, reserved uploads and account quota. No money is spent.", inputSchema: {}, annotations: { readOnlyHint: true, openWorldHint: true } }, wrap(async () => ok(await api.request("GET", "/v1/storage"))));
|
|
96
|
+
server.registerTool("list_files", { title: "List persistent files", description: "Persistent artifacts, model/data files and Workspace files. Availability depends on storage being enabled.", inputSchema: {}, annotations: { readOnlyHint: true, openWorldHint: true } }, wrap(async () => ok(await api.request("GET", "/v1/files"))));
|
|
97
|
+
server.registerTool("list_workspaces", { title: "List Workspaces", description: "Saved project Workspaces; versions are immutable file manifests.", inputSchema: {}, annotations: { readOnlyHint: true, openWorldHint: true } }, wrap(async () => ok(await api.request("GET", "/v1/workspaces"))));
|
|
98
|
+
server.registerTool("list_workspace_versions", { title: "List Workspace versions", description: "List saved immutable versions of an owned Workspace.", inputSchema: { workspace_id: z.string() }, annotations: { readOnlyHint: true } }, wrap(async ({ workspace_id }) => ok(await api.request("GET", `/v1/workspaces/${encodeURIComponent(workspace_id)}/versions`))));
|
|
99
|
+
server.registerTool("get_workspace_version", { title: "Read Workspace version", description: "Read the path-to-file manifest for an owned version.", inputSchema: { version_id: z.string() }, annotations: { readOnlyHint: true } }, wrap(async ({ version_id }) => ok(await api.request("GET", `/v1/workspace-versions/${encodeURIComponent(version_id)}`))));
|
|
100
|
+
if (readOnly) return;
|
|
101
|
+
server.registerTool("delete_workspace_version", {
|
|
102
|
+
title: "Delete Workspace version", description: "Deletes a saved version, releasing its file references. Refuses versions pinned by active Jobs. Ask the user first.",
|
|
103
|
+
inputSchema: { version_id: z.string(), user_confirmed: z.boolean() }, annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: true },
|
|
104
|
+
}, wrap(async ({ version_id, user_confirmed }) => user_confirmed === true ? ok(await api.request("DELETE", `/v1/workspace-versions/${encodeURIComponent(version_id)}`)) : fail("Ask the user before deleting the Workspace version.")));
|
|
105
|
+
|
|
106
|
+
server.registerTool("upload_artifact", {
|
|
107
|
+
title: "Save a persistent artifact", description: "Uploads a local regular file in resumable-sized parts. Consumes storage quota; retention and pricing depend on the account policy. Credential files are refused.",
|
|
108
|
+
inputSchema: { local_path: z.string(), purpose: z.enum(["artifact", "model", "dataset"]).default("artifact"), accept_storage_charges: z.boolean().default(false).describe("Read get_storage pricing first. Set true only after the user agrees to paid storage above the included allowance.") },
|
|
109
|
+
annotations: { readOnlyHint: false, destructiveHint: false, openWorldHint: true },
|
|
110
|
+
}, wrap(async ({ local_path, purpose = "artifact", accept_storage_charges = false }) => {
|
|
111
|
+
const local = path.resolve(expandHome(local_path));
|
|
112
|
+
if (sensitiveReason(local)) return fail("Credential files cannot be saved as artifacts.");
|
|
113
|
+
if ((await fsp.lstat(local)).isSymbolicLink()) return fail("Choose a regular file, not a symlink.");
|
|
114
|
+
return ok(await client.upload(local, purpose, path.basename(local), accept_storage_charges));
|
|
115
|
+
}));
|
|
116
|
+
server.registerTool("download_artifact", {
|
|
117
|
+
title: "Download a persistent file", description: "Streams a persistent file to this computer. Never overwrites an existing local file.",
|
|
118
|
+
inputSchema: { file_id: z.string(), local_path: z.string() }, annotations: { readOnlyHint: false, destructiveHint: false, openWorldHint: true },
|
|
119
|
+
}, wrap(async ({ file_id, local_path }) => ok(await client.download(file_id, path.resolve(expandHome(local_path))))));
|
|
120
|
+
server.registerTool("save_workspace", {
|
|
121
|
+
title: "Save a project Workspace", description: "Saves 1–100 regular project files as an immutable version. Refuses secrets and symlinks; skips .git, node_modules, __pycache__ and .venv. Save dependency manifests, not the operating system. Pass the returned workspace_version_id to run_job.",
|
|
122
|
+
inputSchema: { local_dir: z.string(), name: z.string().min(1).max(120), workspace_id: z.string().optional(), accept_storage_charges: z.boolean().default(false).describe("Read get_storage pricing and obtain user agreement before enabling paid storage.") },
|
|
123
|
+
annotations: { readOnlyHint: false, destructiveHint: false, openWorldHint: true },
|
|
124
|
+
}, wrap(async ({ local_dir, name, workspace_id, accept_storage_charges = false }) => ok(await client.saveWorkspace(expandHome(local_dir), name, workspace_id, accept_storage_charges))));
|
|
125
|
+
server.registerTool("delete_file", {
|
|
126
|
+
title: "Delete a persistent file", description: "Permanently deletes a persistent file or cancels an unfinished upload. Files pinned by a Workspace version cannot be deleted.",
|
|
127
|
+
inputSchema: { file_id: z.string(), user_confirmed: z.boolean() }, annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: true, openWorldHint: true },
|
|
128
|
+
}, wrap(async ({ file_id, user_confirmed }) => user_confirmed === true ? ok(await api.request("DELETE", `/v1/files/${encodeURIComponent(file_id)}`)) : fail("Ask the user before deleting the file.")));
|
|
129
|
+
}
|