@runpod/mcp-server 3.1.0 → 3.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/dist/{chunk-OM6RHPQC.mjs → chunk-WGK7D2YR.mjs} +370 -9
- package/dist/chunk-WGK7D2YR.mjs.map +1 -0
- package/dist/http.cjs +392 -17
- package/dist/http.cjs.map +1 -1
- package/dist/http.mjs +24 -10
- package/dist/http.mjs.map +1 -1
- package/dist/stdio.cjs +387 -16
- package/dist/stdio.cjs.map +1 -1
- package/dist/stdio.mjs +387 -16
- package/dist/stdio.mjs.map +1 -1
- package/dist/tools.cjs +369 -8
- package/dist/tools.cjs.map +1 -1
- package/dist/tools.d.cts +3 -0
- package/dist/tools.d.ts +3 -0
- package/dist/tools.mjs +1 -1
- package/package.json +1 -1
- package/dist/chunk-OM6RHPQC.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# @runpod/mcp-server
|
|
2
2
|
|
|
3
|
+
## 3.2.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- 2fb68de: Default REST v2 base URL is now `https://api.runpod.io/v2` (the documented canonical host) instead of `https://v2-rest.runpod.io/v2`. Both hosts serve the same API; deployments that pin `RUNPOD_REST_V2_API_URL` are unaffected. Server instructions updated to match.
|
|
8
|
+
- 773a4ce: Add `get-capacity`: GPU capacity across host CUDA versions as one matrix call. Agents picking an endpoint's `allowedCudaVersions`/`minCudaVersion` (or diagnosing a capacity-starved endpoint) previously had to reverse-engineer per-version stock by calling the GraphQL `gpuTypes.lowestPrice` query once per CUDA version — an undocumented idiom nobody discovers organically. The default mode returns, per GPU type, overall stock plus AVAILABLE/UNAVAILABLE per host-reported CUDA version in a single credential-free catalog query; passing `cudaVersions` deep-probes those versions instead, returning graded stock (High/Medium/Low/Out) and the lowest on-demand price per version. Nothing is hidden by default — out-of-stock versions appear as explicit `Out` cells (set `includeUnavailable: false` to drop no-stock GPUs), and a transiently failing probe reports a per-version `probeErrors` entry instead of failing the whole call. Works on both v1 and v2 APIs — both use the public GraphQL catalog, since the v2 REST catalog's CUDA support is a `minCudaVersion` floor filter on availability, not the exact per-version breakdown, graded stock, or per-version pricing this tool returns.
|
|
9
|
+
- 14b1f88: Add server instructions (sent in the `initialize` response) pointing agents at the machine-readable API contracts: the REST v2 OpenAPI spec at api.runpod.io/v2/openapi.json and the GraphQL schema reference at graphql-spec.runpod.io. The MCP tools are curated projections of the underlying APIs; this gives agents a discoverable escape hatch when a tool doesn't expose the field or parameter they need, instead of a dead end. Also moves the `capabilities` object to the `ServerOptions` argument of the `McpServer` constructor, where the SDK actually reads it.
|
|
10
|
+
|
|
11
|
+
### Patch Changes
|
|
12
|
+
|
|
13
|
+
- 58c1218: `create-pod` now returns a clean local 400 when `name` is omitted on a v2 GPU create without a `templateId`, matching the v2 spec (`CreatePodRequest` requires `name`) instead of surfacing the API's raw 422. Template deploys (name inherited from the template) and CPU pods (routed to v1) are unaffected.
|
|
14
|
+
- cd52f94: Surface HTTP-level GraphQL failures by status instead of as parse errors. The GraphQL helper (behind `list-gpu-types`, `list-data-centers`, `list-hub-repos`, `list-public-endpoints`, `deploy-hub-repo`, and `set-endpoint-gpus`) previously called `response.json()` unconditionally, so a 429 or 5xx returning an HTML error page surfaced as an opaque `Unexpected token '<'` parse error. Non-OK responses now throw the same `HttpError` as the REST client — status and body named, with the `RateLimit` wait hint on 429 — completing the treatment the REST client received. A non-OK body that still carries a GraphQL `errors` array keeps the readable GraphQL message as an `HttpError` with the status attached (`Runpod GraphQL Error: 400 - <message>`), except on 401/429 where the re-auth and rate-limit hints take precedence. The 401 re-auth hint is now applied only by call sites that actually sent a credential (REST, authed GraphQL, the SSE log reader) — the credential-free public GraphQL path no longer advises re-authenticating for a 401 that cannot be about the caller's API key. `HttpError` messages now also cap embedded bodies at 2 KB (`.body` keeps the full text), so a large proxy/WAF error page can't flood an agent's context.
|
|
15
|
+
- deae681: Make 429s actionable. A bare "rate limit exceeded" invites an immediate retry — the worst possible agent response. On a 429 from the REST or serverless client, the error now names the exhausted quota window and how long to wait, read from the API's `Retry-After` and `RateLimit` headers, e.g. `(rate limited — the hour quota is exhausted; wait ~29 minutes (1724s) before retrying, and pace bulk operations)`. Several windows can report zero remaining at once, so the hint reports the longest wait either header justifies rather than the first or the nearest — answering a 30-minute lockout with "retry in 12s" is the failure this exists to prevent. When part of the header couldn't be read, the wait is qualified "at least" rather than presented as the answer. Falls back to generic back-off guidance when neither header is usable, as on the v1 REST API, which sends neither. The log-streaming tools, which build their own error, carry the same hint. GraphQL-backed tools (catalog, hub, public endpoints, endpoint GPUs) are unchanged — that helper doesn't inspect the response status yet. `HttpError.status`/`.body` are unchanged for programmatic callers.
|
|
16
|
+
|
|
3
17
|
## 3.1.0
|
|
4
18
|
|
|
5
19
|
### Minor Changes
|
|
@@ -2,11 +2,150 @@
|
|
|
2
2
|
import fetch from "node-fetch";
|
|
3
3
|
import { randomUUID } from "crypto";
|
|
4
4
|
|
|
5
|
+
// src/_shared/rate-limit.ts
|
|
6
|
+
var GENERIC_RATE_LIMIT_HINT = "rate limited \u2014 back off before retrying, and pace bulk operations";
|
|
7
|
+
var MAX_PLAUSIBLE_WAIT_S = 7 * 86400;
|
|
8
|
+
function remainingRequests(raw) {
|
|
9
|
+
const value = raw?.trim();
|
|
10
|
+
return value && /^\d+$/.test(value) ? Number(value) : null;
|
|
11
|
+
}
|
|
12
|
+
function waitSeconds(raw) {
|
|
13
|
+
const value = raw?.trim();
|
|
14
|
+
if (!value || !/^\d+$/.test(value)) return null;
|
|
15
|
+
const seconds = Number(value);
|
|
16
|
+
return seconds <= MAX_PLAUSIBLE_WAIT_S ? seconds : null;
|
|
17
|
+
}
|
|
18
|
+
var WINDOW_NAME_RE = /^(?:"([\x20-\x21\x23-\x5b\x5d-\x7e]{1,32})"|([A-Za-z*][A-Za-z0-9!#$%&'*+.^_`|~:/-]{0,31}))$/;
|
|
19
|
+
function windowName(raw) {
|
|
20
|
+
const match = WINDOW_NAME_RE.exec(raw.trim());
|
|
21
|
+
return match ? match[1] ?? match[2] : null;
|
|
22
|
+
}
|
|
23
|
+
var QUOTED_VALUE_RE = /^"(?:[\x20-\x21\x23-\x5b\x5d-\x7e]|\\["\\])*"$/;
|
|
24
|
+
var DISPLAY_VALUE_RE = /^%"(?:[\x20-\x21\x23-\x24\x26-\x5b\x5d-\x7e]|\\|%[0-9a-f]{2})*"$/;
|
|
25
|
+
function isWellFormedParam(param) {
|
|
26
|
+
const eq = param.indexOf("=");
|
|
27
|
+
const key = (eq === -1 ? param : param.slice(0, eq)).trim();
|
|
28
|
+
if (key.includes('"')) return false;
|
|
29
|
+
if (eq === -1) return true;
|
|
30
|
+
const value = param.slice(eq + 1).trim();
|
|
31
|
+
if (value.startsWith('"')) return QUOTED_VALUE_RE.test(value);
|
|
32
|
+
if (value.startsWith('%"')) return DISPLAY_VALUE_RE.test(value);
|
|
33
|
+
return !/[\s"]/.test(value);
|
|
34
|
+
}
|
|
35
|
+
function splitOutsideStrings(input, sep) {
|
|
36
|
+
const parts = [];
|
|
37
|
+
let current = "";
|
|
38
|
+
let inString = false;
|
|
39
|
+
let inDisplay = false;
|
|
40
|
+
let escaped = false;
|
|
41
|
+
let prev = "";
|
|
42
|
+
for (const ch of input) {
|
|
43
|
+
if (escaped) escaped = false;
|
|
44
|
+
else if (inString && !inDisplay && ch === "\\") escaped = true;
|
|
45
|
+
else if (ch === '"') {
|
|
46
|
+
if (!inString) inDisplay = prev === "%";
|
|
47
|
+
inString = !inString;
|
|
48
|
+
} else if (ch === sep && !inString) {
|
|
49
|
+
parts.push(current);
|
|
50
|
+
current = "";
|
|
51
|
+
prev = ch;
|
|
52
|
+
continue;
|
|
53
|
+
}
|
|
54
|
+
current += ch;
|
|
55
|
+
prev = ch;
|
|
56
|
+
}
|
|
57
|
+
parts.push(current);
|
|
58
|
+
return parts;
|
|
59
|
+
}
|
|
60
|
+
function readParams(params) {
|
|
61
|
+
const values = /* @__PURE__ */ new Map();
|
|
62
|
+
let unreadable = false;
|
|
63
|
+
let valuelessAt = -1;
|
|
64
|
+
for (const [index, param] of params.entries()) {
|
|
65
|
+
const eq = param.indexOf("=");
|
|
66
|
+
const key = (eq === -1 ? param : param.slice(0, eq)).trim().toLowerCase();
|
|
67
|
+
if (!key) continue;
|
|
68
|
+
if (eq === -1) {
|
|
69
|
+
if (key === "r") unreadable = true;
|
|
70
|
+
if (key === "t") values.delete("t");
|
|
71
|
+
valuelessAt = index;
|
|
72
|
+
continue;
|
|
73
|
+
}
|
|
74
|
+
if ((key === "r" || key === "t") && (values.has(key) || valuelessAt !== -1 && index > valuelessAt)) {
|
|
75
|
+
unreadable = true;
|
|
76
|
+
}
|
|
77
|
+
values.set(key, param.slice(eq + 1));
|
|
78
|
+
}
|
|
79
|
+
return { values, unreadable };
|
|
80
|
+
}
|
|
81
|
+
function parseExhaustedWindows(header) {
|
|
82
|
+
if (!header) return { windows: [], dropped: false };
|
|
83
|
+
const windows = [];
|
|
84
|
+
let dropped = false;
|
|
85
|
+
for (const item of splitOutsideStrings(header, ",")) {
|
|
86
|
+
if (item.trim() === "") continue;
|
|
87
|
+
const [rawName, ...params] = splitOutsideStrings(item, ";");
|
|
88
|
+
if (!params.every(isWellFormedParam)) {
|
|
89
|
+
dropped = true;
|
|
90
|
+
continue;
|
|
91
|
+
}
|
|
92
|
+
const { values, unreadable } = readParams(params);
|
|
93
|
+
if (unreadable) {
|
|
94
|
+
dropped = true;
|
|
95
|
+
continue;
|
|
96
|
+
}
|
|
97
|
+
const remaining = remainingRequests(values.get("r"));
|
|
98
|
+
if (remaining === null) {
|
|
99
|
+
dropped = true;
|
|
100
|
+
continue;
|
|
101
|
+
}
|
|
102
|
+
if (remaining !== 0) continue;
|
|
103
|
+
windows.push({
|
|
104
|
+
window: windowName(rawName),
|
|
105
|
+
resetS: waitSeconds(values.get("t"))
|
|
106
|
+
});
|
|
107
|
+
}
|
|
108
|
+
return { windows, dropped };
|
|
109
|
+
}
|
|
110
|
+
function humanWait(seconds) {
|
|
111
|
+
if (seconds < 120) return `~${seconds}s`;
|
|
112
|
+
if (seconds < 3600)
|
|
113
|
+
return `~${Math.ceil(seconds / 60)} minutes (${seconds}s)`;
|
|
114
|
+
if (seconds < 2 * 86400) {
|
|
115
|
+
const hours = Math.ceil(seconds / 3600);
|
|
116
|
+
return `~${hours} hour${hours === 1 ? "" : "s"} (${seconds}s)`;
|
|
117
|
+
}
|
|
118
|
+
return `~${Math.ceil(seconds / 86400)} days (${seconds}s)`;
|
|
119
|
+
}
|
|
120
|
+
function rateLimitHint(rateLimitHeader, retryAfterHeader) {
|
|
121
|
+
const { windows, dropped } = parseExhaustedWindows(rateLimitHeader);
|
|
122
|
+
windows.sort((a, b) => (b.resetS ?? -1) - (a.resetS ?? -1));
|
|
123
|
+
const longestWindow = windows[0];
|
|
124
|
+
const waitCandidates = [
|
|
125
|
+
waitSeconds(retryAfterHeader),
|
|
126
|
+
longestWindow?.resetS
|
|
127
|
+
].filter((n) => typeof n === "number" && n > 0);
|
|
128
|
+
const waitS = waitCandidates.length ? Math.max(...waitCandidates) : null;
|
|
129
|
+
const retryAfterUnreadable = typeof retryAfterHeader === "string" && retryAfterHeader.trim() !== "" && waitSeconds(retryAfterHeader) === null;
|
|
130
|
+
if (!longestWindow && waitS === null) {
|
|
131
|
+
if (!retryAfterUnreadable) return GENERIC_RATE_LIMIT_HINT;
|
|
132
|
+
}
|
|
133
|
+
const namedClause = longestWindow?.window ? `the ${longestWindow.window} quota is exhausted` : "";
|
|
134
|
+
if (waitS === null) {
|
|
135
|
+
const cause = namedClause ? `${namedClause}, but the response gave no usable reset time` : "the response gave no usable reset time";
|
|
136
|
+
return `rate limited \u2014 ${cause}; back off substantially before retrying, and pace bulk operations`;
|
|
137
|
+
}
|
|
138
|
+
const hedge = dropped || retryAfterUnreadable || windows.some((w) => w.resetS === null) ? "at least " : "";
|
|
139
|
+
const wait = `wait ${hedge}${humanWait(waitS)} before retrying, and pace bulk operations`;
|
|
140
|
+
return `rate limited \u2014 ${namedClause ? `${namedClause}; ` : ""}${wait}`;
|
|
141
|
+
}
|
|
142
|
+
|
|
5
143
|
// src/_shared/http.ts
|
|
144
|
+
var EXPIRED_CREDENTIAL_HINT = "the Runpod API rejected the credential \u2014 the API key may be expired or revoked; re-authenticate or update RUNPOD_API_KEY";
|
|
6
145
|
var HttpError = class extends Error {
|
|
7
|
-
constructor(prefix, status, body) {
|
|
8
|
-
const
|
|
9
|
-
super(`${prefix}: ${status} - ${
|
|
146
|
+
constructor(prefix, status, body, hint) {
|
|
147
|
+
const shownBody = body.length > 2048 ? `${body.slice(0, 2048)}\u2026 [truncated ${body.length - 2048} chars]` : body;
|
|
148
|
+
super(`${prefix}: ${status} - ${shownBody}${hint ? ` (${hint})` : ""}`);
|
|
10
149
|
this.name = "HttpError";
|
|
11
150
|
this.status = status;
|
|
12
151
|
this.body = body;
|
|
@@ -41,7 +180,11 @@ function createHttpClient(deps) {
|
|
|
41
180
|
throw new HttpError(
|
|
42
181
|
deps.errorPrefix,
|
|
43
182
|
response.status,
|
|
44
|
-
await response.text()
|
|
183
|
+
await response.text(),
|
|
184
|
+
response.status === 429 ? rateLimitHint(
|
|
185
|
+
response.headers.get("ratelimit"),
|
|
186
|
+
response.headers.get("retry-after")
|
|
187
|
+
) : response.status === 401 ? EXPIRED_CREDENTIAL_HINT : void 0
|
|
45
188
|
);
|
|
46
189
|
}
|
|
47
190
|
return isJsonContentType(response.headers.get("content-type")) ? response.json() : { success: true, status: response.status };
|
|
@@ -100,7 +243,15 @@ async function readSseSnapshot(fetchImpl, url, headers, opts) {
|
|
|
100
243
|
throw new HttpError(
|
|
101
244
|
"Runpod API Error",
|
|
102
245
|
res.status,
|
|
103
|
-
await res.text().catch(() => "")
|
|
246
|
+
await res.text().catch(() => ""),
|
|
247
|
+
// This throw bypasses createHttpClient, so the status hints are built
|
|
248
|
+
// here too or these two tools alone return a bare 429/401. The 401
|
|
249
|
+
// hint applies: this request always carries Authorization (see the
|
|
250
|
+
// headers bound in runtime.ts streamSse).
|
|
251
|
+
res.status === 429 ? rateLimitHint(
|
|
252
|
+
res.headers.get("ratelimit"),
|
|
253
|
+
res.headers.get("retry-after")
|
|
254
|
+
) : res.status === 401 ? EXPIRED_CREDENTIAL_HINT : void 0
|
|
104
255
|
);
|
|
105
256
|
}
|
|
106
257
|
for await (const chunk of res.body ?? []) {
|
|
@@ -359,7 +510,7 @@ function restV1Base(env) {
|
|
|
359
510
|
return env.RUNPOD_REST_API_URL ?? "https://rest.runpod.io/v1";
|
|
360
511
|
}
|
|
361
512
|
function restV2Base(env) {
|
|
362
|
-
return env.RUNPOD_REST_V2_API_URL ?? "https://
|
|
513
|
+
return env.RUNPOD_REST_V2_API_URL ?? "https://api.runpod.io/v2";
|
|
363
514
|
}
|
|
364
515
|
function serverlessBase(env) {
|
|
365
516
|
return env.RUNPOD_SERVERLESS_API_URL ?? "https://api.runpod.ai/v2";
|
|
@@ -530,7 +681,7 @@ function resolveBackend(opts) {
|
|
|
530
681
|
}
|
|
531
682
|
|
|
532
683
|
// src/tools/runtime.ts
|
|
533
|
-
var MCP_SERVER_VERSION = true ? "3.
|
|
684
|
+
var MCP_SERVER_VERSION = true ? "3.2.0" : "dev";
|
|
534
685
|
var SESSION_ID = randomUUID();
|
|
535
686
|
function trackingHeaders(server, ctx) {
|
|
536
687
|
const info = server.server.getClientVersion();
|
|
@@ -580,6 +731,36 @@ async function graphqlRequest(query, tracking, fetchImpl, url, options) {
|
|
|
580
731
|
...options?.variables ? { variables: options.variables } : {}
|
|
581
732
|
})
|
|
582
733
|
});
|
|
734
|
+
if (!response.ok) {
|
|
735
|
+
const bodyText = await response.text();
|
|
736
|
+
let gqlErrors;
|
|
737
|
+
try {
|
|
738
|
+
gqlErrors = JSON.parse(bodyText).errors;
|
|
739
|
+
} catch {
|
|
740
|
+
}
|
|
741
|
+
if (Array.isArray(gqlErrors) && gqlErrors.length > 0 && response.status !== 401 && response.status !== 429) {
|
|
742
|
+
throw new HttpError(
|
|
743
|
+
"Runpod GraphQL Error",
|
|
744
|
+
response.status,
|
|
745
|
+
gqlErrors.map((e) => String(e?.message ?? JSON.stringify(e))).join(", ")
|
|
746
|
+
);
|
|
747
|
+
}
|
|
748
|
+
throw new HttpError(
|
|
749
|
+
"Runpod GraphQL Error",
|
|
750
|
+
response.status,
|
|
751
|
+
bodyText,
|
|
752
|
+
response.status === 429 ? rateLimitHint(
|
|
753
|
+
response.headers.get("ratelimit"),
|
|
754
|
+
response.headers.get("retry-after")
|
|
755
|
+
) : (
|
|
756
|
+
// Only when this request actually carried the caller's API key: the
|
|
757
|
+
// public path sends no Authorization header, so a 401 from that
|
|
758
|
+
// host (WAF, misconfigured RUNPOD_PUBLIC_GRAPHQL_URL) says nothing
|
|
759
|
+
// about the credential — same rationale as observeUnauthorized.
|
|
760
|
+
response.status === 401 && options?.apiKey ? EXPIRED_CREDENTIAL_HINT : void 0
|
|
761
|
+
)
|
|
762
|
+
);
|
|
763
|
+
}
|
|
583
764
|
const result = await response.json();
|
|
584
765
|
if (result.errors && result.errors.length > 0) {
|
|
585
766
|
throw new Error(
|
|
@@ -996,6 +1177,178 @@ function registerCatalogTools(server, rt) {
|
|
|
996
1177
|
return jsonReply2(result);
|
|
997
1178
|
}
|
|
998
1179
|
);
|
|
1180
|
+
server.tool(
|
|
1181
|
+
"get-capacity",
|
|
1182
|
+
"GPU capacity across host CUDA versions, as a matrix. Use this to choose an endpoint's allowedCudaVersions/minCudaVersion (or diagnose/widen a capacity-starved one) and to distinguish capacity problems from compatibility problems. Default mode is one call returning, per GPU type, overall stock plus AVAILABLE/UNAVAILABLE per host-reported CUDA version. Pass cudaVersions to deep-probe instead: one stock lookup per listed version, returning graded stock (High/Medium/Low/Out) and the lowest on-demand price per version; a probe that fails transiently reports a per-version error instead of failing the call. Nothing is hidden by default \u2014 set includeUnavailable:false to drop GPUs with no stock on any listed version. Credential-free public catalog data; works on both v1 and v2 APIs. Stock is live, so page cursors can shift between calls. Note: the matrix reflects host-reported versions, but endpoint allowedCudaVersions only accepts values from the platform enum \u2014 check create-endpoint/update-endpoint for the accepted list.",
|
|
1183
|
+
{
|
|
1184
|
+
...listPaginationParams,
|
|
1185
|
+
cudaVersions: z3.array(z3.string().regex(/^\d{1,2}\.\d{1,2}$/)).min(1).max(12).optional().describe(
|
|
1186
|
+
'Deep-probe these host CUDA versions (e.g. ["12.8", "13.0"], max 12): one stock lookup per version returns graded stock and price. Omit for the single-call AVAILABLE/UNAVAILABLE matrix across all versions the fleet currently reports.'
|
|
1187
|
+
),
|
|
1188
|
+
includeUnavailable: z3.boolean().optional().describe(
|
|
1189
|
+
'Out-of-stock GPUs are included by default (explicit "Out" cells / all-UNAVAILABLE rows, sorted last). Set false to hide GPUs with no stock on any listed version.'
|
|
1190
|
+
),
|
|
1191
|
+
gpuTypeIds: z3.array(z3.string()).optional().describe(
|
|
1192
|
+
"Filter to GPU types matching any of these ids or display names (case-insensitive substring, e.g. ['NVIDIA GeForce RTX 4090'] or ['4090', 'H200'])."
|
|
1193
|
+
),
|
|
1194
|
+
gpuCount: z3.number().int().min(1).max(8).optional().describe("GPUs per worker/pod to check stock for (default 1)."),
|
|
1195
|
+
secureCloudOnly: z3.boolean().optional().describe("Restrict the stock lookup to Secure Cloud hosts.")
|
|
1196
|
+
},
|
|
1197
|
+
{ title: "Get GPU capacity by CUDA version", ...READ_ONLY },
|
|
1198
|
+
async (params) => {
|
|
1199
|
+
const gpuCount = Math.min(
|
|
1200
|
+
8,
|
|
1201
|
+
Math.max(1, Math.floor(Number(params.gpuCount)) || 1)
|
|
1202
|
+
);
|
|
1203
|
+
const secureArg = params.secureCloudOnly ? ", secureCloud: true" : "";
|
|
1204
|
+
const filterTerms = (params.gpuTypeIds ?? []).filter((t) => typeof t === "string" && t.trim().length > 0).map((t) => t.toLowerCase());
|
|
1205
|
+
const matchesFilter = (gpu) => {
|
|
1206
|
+
if (filterTerms.length === 0) return true;
|
|
1207
|
+
const id = gpu.id.toLowerCase();
|
|
1208
|
+
const name = gpu.displayName.toLowerCase();
|
|
1209
|
+
return filterTerms.some(
|
|
1210
|
+
(term) => id.includes(term) || name.includes(term)
|
|
1211
|
+
);
|
|
1212
|
+
};
|
|
1213
|
+
const stockPriority = {
|
|
1214
|
+
High: 3,
|
|
1215
|
+
Medium: 2,
|
|
1216
|
+
Low: 1
|
|
1217
|
+
};
|
|
1218
|
+
if (!params.cudaVersions?.length) {
|
|
1219
|
+
const data = await graphql(`
|
|
1220
|
+
query {
|
|
1221
|
+
gpuTypes {
|
|
1222
|
+
id
|
|
1223
|
+
displayName
|
|
1224
|
+
memoryInGb
|
|
1225
|
+
secureCloud
|
|
1226
|
+
communityCloud
|
|
1227
|
+
lowestPrice(input: { gpuCount: ${gpuCount}${secureArg} }) {
|
|
1228
|
+
stockStatus
|
|
1229
|
+
uninterruptablePrice
|
|
1230
|
+
gpuTypeCudaVersions {
|
|
1231
|
+
cudaVersion
|
|
1232
|
+
availability
|
|
1233
|
+
}
|
|
1234
|
+
}
|
|
1235
|
+
}
|
|
1236
|
+
}
|
|
1237
|
+
`);
|
|
1238
|
+
let rows2 = data.gpuTypes.filter((gpu) => gpu.id !== "unknown" && matchesFilter(gpu)).map((gpu) => {
|
|
1239
|
+
const cuda = {};
|
|
1240
|
+
for (const c of gpu.lowestPrice?.gpuTypeCudaVersions ?? []) {
|
|
1241
|
+
cuda[c.cudaVersion] = c.availability;
|
|
1242
|
+
}
|
|
1243
|
+
return {
|
|
1244
|
+
id: gpu.id,
|
|
1245
|
+
displayName: gpu.displayName,
|
|
1246
|
+
memoryGb: gpu.memoryInGb,
|
|
1247
|
+
secureCloud: gpu.secureCloud,
|
|
1248
|
+
communityCloud: gpu.communityCloud,
|
|
1249
|
+
stockStatus: gpu.lowestPrice?.stockStatus || "unavailable",
|
|
1250
|
+
pricePerHr: gpu.lowestPrice?.uninterruptablePrice ?? null,
|
|
1251
|
+
cudaVersions: cuda
|
|
1252
|
+
};
|
|
1253
|
+
});
|
|
1254
|
+
const availableCount = (r) => Object.values(r.cudaVersions).filter((a) => a === "AVAILABLE").length;
|
|
1255
|
+
if (params.includeUnavailable === false)
|
|
1256
|
+
rows2 = rows2.filter((r) => availableCount(r) > 0);
|
|
1257
|
+
rows2.sort((a, b) => {
|
|
1258
|
+
const diff = availableCount(b) - availableCount(a);
|
|
1259
|
+
if (diff !== 0) return diff;
|
|
1260
|
+
const stockDiff = (stockPriority[b.stockStatus] || 0) - (stockPriority[a.stockStatus] || 0);
|
|
1261
|
+
if (stockDiff !== 0) return stockDiff;
|
|
1262
|
+
return b.memoryGb - a.memoryGb;
|
|
1263
|
+
});
|
|
1264
|
+
return capListResult(rows2, {
|
|
1265
|
+
limit: params.limit,
|
|
1266
|
+
cursor: params.cursor
|
|
1267
|
+
});
|
|
1268
|
+
}
|
|
1269
|
+
const versions = [...new Set(params.cudaVersions)].slice(0, 12);
|
|
1270
|
+
const invalid = versions.filter((v) => !/^\d{1,2}\.\d{1,2}$/.test(v));
|
|
1271
|
+
if (invalid.length > 0) {
|
|
1272
|
+
return jsonReply2({
|
|
1273
|
+
error: `Invalid CUDA version format: ${invalid.join(", ")}. Use "major.minor" strings like "12.8".`,
|
|
1274
|
+
status: 400
|
|
1275
|
+
});
|
|
1276
|
+
}
|
|
1277
|
+
const perVersion = await Promise.allSettled(
|
|
1278
|
+
versions.map(
|
|
1279
|
+
(v) => graphql(`
|
|
1280
|
+
query {
|
|
1281
|
+
gpuTypes {
|
|
1282
|
+
id
|
|
1283
|
+
displayName
|
|
1284
|
+
memoryInGb
|
|
1285
|
+
secureCloud
|
|
1286
|
+
communityCloud
|
|
1287
|
+
lowestPrice(input: { gpuCount: ${gpuCount}, allowedCudaVersions: ["${v}"]${secureArg} }) {
|
|
1288
|
+
stockStatus
|
|
1289
|
+
uninterruptablePrice
|
|
1290
|
+
}
|
|
1291
|
+
}
|
|
1292
|
+
}
|
|
1293
|
+
`)
|
|
1294
|
+
)
|
|
1295
|
+
);
|
|
1296
|
+
const probeErrors = {};
|
|
1297
|
+
const byId = /* @__PURE__ */ new Map();
|
|
1298
|
+
versions.forEach((v, i) => {
|
|
1299
|
+
const settled = perVersion[i];
|
|
1300
|
+
if (settled.status === "rejected") {
|
|
1301
|
+
probeErrors[v] = settled.reason instanceof Error ? settled.reason.message : String(settled.reason);
|
|
1302
|
+
return;
|
|
1303
|
+
}
|
|
1304
|
+
for (const gpu of settled.value.gpuTypes) {
|
|
1305
|
+
if (gpu.id === "unknown" || !matchesFilter(gpu)) continue;
|
|
1306
|
+
let row = byId.get(gpu.id);
|
|
1307
|
+
if (!row) {
|
|
1308
|
+
row = {
|
|
1309
|
+
id: gpu.id,
|
|
1310
|
+
displayName: gpu.displayName,
|
|
1311
|
+
memoryGb: gpu.memoryInGb,
|
|
1312
|
+
secureCloud: gpu.secureCloud,
|
|
1313
|
+
communityCloud: gpu.communityCloud,
|
|
1314
|
+
cudaVersions: {}
|
|
1315
|
+
};
|
|
1316
|
+
byId.set(gpu.id, row);
|
|
1317
|
+
}
|
|
1318
|
+
const stock = gpu.lowestPrice?.stockStatus;
|
|
1319
|
+
row.cudaVersions[v] = !stock || stock === "Out" ? { stock: "Out", pricePerHr: null } : {
|
|
1320
|
+
stock,
|
|
1321
|
+
pricePerHr: gpu.lowestPrice?.uninterruptablePrice ?? null
|
|
1322
|
+
};
|
|
1323
|
+
}
|
|
1324
|
+
});
|
|
1325
|
+
let rows = [...byId.values()];
|
|
1326
|
+
const inStockCount = (r) => Object.values(r.cudaVersions).filter((c) => c.stock !== "Out").length;
|
|
1327
|
+
if (params.includeUnavailable === false)
|
|
1328
|
+
rows = rows.filter((r) => inStockCount(r) > 0);
|
|
1329
|
+
rows.sort((a, b) => {
|
|
1330
|
+
const diff = inStockCount(b) - inStockCount(a);
|
|
1331
|
+
if (diff !== 0) return diff;
|
|
1332
|
+
const best = (r) => Math.max(
|
|
1333
|
+
0,
|
|
1334
|
+
...Object.values(r.cudaVersions).map(
|
|
1335
|
+
(c) => stockPriority[c.stock] || 0
|
|
1336
|
+
)
|
|
1337
|
+
);
|
|
1338
|
+
const bestDiff = best(b) - best(a);
|
|
1339
|
+
if (bestDiff !== 0) return bestDiff;
|
|
1340
|
+
return b.memoryGb - a.memoryGb;
|
|
1341
|
+
});
|
|
1342
|
+
return capListResult(
|
|
1343
|
+
rows,
|
|
1344
|
+
{ limit: params.limit, cursor: params.cursor },
|
|
1345
|
+
{
|
|
1346
|
+
probedCudaVersions: versions,
|
|
1347
|
+
...Object.keys(probeErrors).length > 0 ? { probeErrors } : {}
|
|
1348
|
+
}
|
|
1349
|
+
);
|
|
1350
|
+
}
|
|
1351
|
+
);
|
|
999
1352
|
server.tool(
|
|
1000
1353
|
"get-data-center",
|
|
1001
1354
|
"Get details for a single data center by id. v2-only \u2014 returns a 501 notice on the v1 API (use list-data-centers there).",
|
|
@@ -1680,7 +2033,9 @@ function registerPodTools(server, rt) {
|
|
|
1680
2033
|
"create-pod",
|
|
1681
2034
|
'Create a new GPU/CPU pod on Runpod. Pass gpuTypeIds for a GPU pod, or computeType:"CPU" for a CPU pod (CPU pods are served by the v1 API for now). Pass either imageName or templateId (templateId deploys from an existing template \u2014 its name, image, start command, ports, env, disk, volume, and registry credential are used as defaults, and any field you also pass explicitly replaces the template value for that field, e.g. passing env replaces the whole env set rather than merging; pass containerRegistryAuthId to override or supply a private-image credential, or an empty string to deploy with none). If the user specifies neither an image nor a template, recommend the "Runpod Pytorch 2.8.0" image (runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404) as the default \u2014 it has the most up-to-date CUDA and PyTorch versions.',
|
|
1682
2035
|
{
|
|
1683
|
-
name: z7.string().optional().describe(
|
|
2036
|
+
name: z7.string().optional().describe(
|
|
2037
|
+
"Name for the pod. Required on v2 unless deploying from a templateId (the template supplies its name as the default)."
|
|
2038
|
+
),
|
|
1684
2039
|
imageName: z7.string().optional().describe("Docker image to use. Optional when templateId is provided."),
|
|
1685
2040
|
templateId: z7.string().optional().describe(
|
|
1686
2041
|
"Deploy from an existing template (see list-templates / get-template). The template supplies name, image, start command, ports, env, disk, volume, and registry credential as defaults; any field you also pass replaces the template value for that field (whole-field, not a merge). Override the registry credential with containerRegistryAuthId. Requires the v2 REST API. Still pass gpuTypeIds or computeType to choose the compute \u2014 a template does not pin the pod compute."
|
|
@@ -1739,6 +2094,12 @@ function registerPodTools(server, rt) {
|
|
|
1739
2094
|
status: 400
|
|
1740
2095
|
});
|
|
1741
2096
|
}
|
|
2097
|
+
if (!wantsCpu && !params.name && !params.templateId) {
|
|
2098
|
+
return jsonReply2({
|
|
2099
|
+
error: "Provide a name for the pod (deploying from a templateId inherits the template name).",
|
|
2100
|
+
status: 400
|
|
2101
|
+
});
|
|
2102
|
+
}
|
|
1742
2103
|
if (wantsCpu) {
|
|
1743
2104
|
if (params.templateId) {
|
|
1744
2105
|
return jsonReply2({
|
|
@@ -3020,4 +3381,4 @@ export {
|
|
|
3020
3381
|
restVersionSummary,
|
|
3021
3382
|
registerTools
|
|
3022
3383
|
};
|
|
3023
|
-
//# sourceMappingURL=chunk-
|
|
3384
|
+
//# sourceMappingURL=chunk-WGK7D2YR.mjs.map
|