@sandboxrouter/sdk 0.6.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +39 -39
- package/dist/billing.js +2 -1
- package/dist/catalog.d.ts +18 -1
- package/dist/catalog.js +31 -9
- package/dist/check.d.ts +2 -1
- package/dist/check.js +13 -6
- package/dist/cli.js +26 -12
- package/dist/cost.js +22 -2
- package/dist/index.d.ts +1 -1
- package/dist/index.js +1 -1
- package/dist/router.d.ts +26 -2
- package/dist/router.js +40 -11
- package/dist/verified.js +16 -16
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -23,7 +23,7 @@ Runs as plain JavaScript on Node 20+. (In TypeScript 5.2+, or Node 24+, `await u
|
|
|
23
23
|
|
|
24
24
|
Set the keys of the providers you use. `create()` starts the sandbox on the cheapest one that fits: list price for the size you asked for, corrected where we measured what a provider really bills. Providers within 10% of each other on price go fastest first, by start times measured where you run the router. Providers that failed their last real-account check go last. Ties are alphabetical, and no provider pays for placement. If that provider is down, busy or out of credit, the next one takes it, and every attempt is on `sbx.attempts`. The sandbox stays on that provider until you `kill()` it.
|
|
25
25
|
|
|
26
|
-
Site and docs: [sandboxrouter.
|
|
26
|
+
Site and docs: [sandboxrouter.ai](https://sandboxrouter.ai) · npm: [`@sandboxrouter/sdk`](https://www.npmjs.com/package/@sandboxrouter/sdk)
|
|
27
27
|
|
|
28
28
|
## Why
|
|
29
29
|
|
|
@@ -38,46 +38,46 @@ AI agents need a disposable computer to run code a model wrote: install packages
|
|
|
38
38
|
|
|
39
39
|
## Prices
|
|
40
40
|
|
|
41
|
-
List price while running, 2 vCPU / 4 GB, cheapest first; waitlisted sign-ups follow. The router ranks by these, corrected where we measured what a provider really bills, and tries providers whose last real-account check failed last. "Up to" = CPU billed only while busy, shown with every core busy. "Not run yet" = built from the provider's docs and tested against mocks only. Click a provider for its billing details; more in [Reading the price table](https://sandboxrouter.
|
|
41
|
+
List price while running, 2 vCPU / 4 GB, cheapest first; waitlisted sign-ups follow. The router ranks by these, corrected where we measured what a provider really bills, and tries providers whose last real-account check failed last. "Up to" = CPU billed only while busy, shown with every core busy. "Not run yet" = built from the provider's docs and tested against mocks only. Click a provider for its billing details; more in [Reading the price table](https://sandboxrouter.ai/docs#price-table).
|
|
42
42
|
|
|
43
43
|
| Provider | $/hr | Plan needed | Real-account check | Source |
|
|
44
44
|
|---|---|---|---|---|
|
|
45
|
-
| [Agent37](https://sandboxrouter.
|
|
46
|
-
| [Boat](https://sandboxrouter.
|
|
47
|
-
| [Maritime](https://sandboxrouter.
|
|
48
|
-
| [Sailboxes (Sail Research)](https://sandboxrouter.
|
|
49
|
-
| [Northflank](https://sandboxrouter.
|
|
50
|
-
| [Morph Cloud](https://sandboxrouter.
|
|
51
|
-
| [exe.dev](https://sandboxrouter.
|
|
52
|
-
| [Novita AI Agent Sandbox](https://sandboxrouter.
|
|
53
|
-
| [Freestyle](https://sandboxrouter.
|
|
54
|
-
| [Beam](https://sandboxrouter.
|
|
55
|
-
| [CodeSandbox SDK (Together)](https://sandboxrouter.
|
|
56
|
-
| [Fly.io Sprites](https://sandboxrouter.
|
|
57
|
-
| [Blaxel](https://sandboxrouter.
|
|
58
|
-
| [HOPX](https://sandboxrouter.
|
|
59
|
-
| [Superserve](https://sandboxrouter.
|
|
60
|
-
| [Tensorlake](https://sandboxrouter.
|
|
61
|
-
| [Upstash Box](https://sandboxrouter.
|
|
62
|
-
| [Cloudflare Sandbox](https://sandboxrouter.
|
|
63
|
-
| [Amazon Bedrock AgentCore Code Interpreter](https://sandboxrouter.
|
|
64
|
-
| [Modal](https://sandboxrouter.
|
|
65
|
-
| [AWS Lambda MicroVMs](https://sandboxrouter.
|
|
66
|
-
| [Deno Sandbox](https://sandboxrouter.
|
|
67
|
-
| [Runloop](https://sandboxrouter.
|
|
68
|
-
| [Vercel Sandbox](https://sandboxrouter.
|
|
69
|
-
| [InstaBox (sign-up waitlisted)](https://sandboxrouter.
|
|
70
|
-
| [Koyeb Sandboxes (not taking new accounts)](https://sandboxrouter.
|
|
71
|
-
| [Dedalus Machines (sign-up waitlisted)](https://sandboxrouter.
|
|
72
|
-
| [LangSmith Sandboxes (sign-up waitlisted)](https://sandboxrouter.
|
|
73
|
-
| [OpenScale Sandboxes](https://sandboxrouter.
|
|
74
|
-
| [Azure Container Apps dynamic sessions](https://sandboxrouter.
|
|
75
|
-
| [Daytona](https://sandboxrouter.
|
|
76
|
-
| [Alibaba Cloud Agent Sandbox](https://sandboxrouter.
|
|
77
|
-
| [E2B](https://sandboxrouter.
|
|
78
|
-
| [Docker (local)](https://sandboxrouter.
|
|
79
|
-
|
|
80
|
-
Checked 2026-11-04 to 2026-11-
|
|
45
|
+
| [Agent37](https://sandboxrouter.ai/providers/agent37) | $0.007 | none | passed (2026-11-13) | [pricing](https://www.agent37.com/docs/agents-api/billing) |
|
|
46
|
+
| [Boat](https://sandboxrouter.ai/providers/boat) | $0.018 | $20/mo (credited) | not run yet | [pricing](https://docs.boat.dev/pricing.md) |
|
|
47
|
+
| [Maritime](https://sandboxrouter.ai/providers/maritime) | $0.022 | $20/mo (credited) | passed (2026-11-13) | [pricing](https://maritime.sh/pricing) |
|
|
48
|
+
| [Sailboxes (Sail Research)](https://sandboxrouter.ai/providers/sail) | up to $0.062 | none | passed (2026-11-13) | [pricing](https://docs.sailresearch.com/sailboxes-pricing.md) |
|
|
49
|
+
| [Northflank](https://sandboxrouter.ai/providers/northflank) | $0.067 | none | not run yet | [pricing](https://northflank.com/pricing) |
|
|
50
|
+
| [Morph Cloud](https://sandboxrouter.ai/providers/morph) | $0.100 | none | not run yet | [pricing](https://cloud.morph.so/web/pricing) |
|
|
51
|
+
| [exe.dev](https://sandboxrouter.ai/providers/exedev) | $0.105 | $15/mo | not run yet | [pricing](https://exe.dev/docs/standalone-vms.md) |
|
|
52
|
+
| [Novita AI Agent Sandbox](https://sandboxrouter.ai/providers/novita) | $0.117 | none | passed (2026-11-13) | [pricing](https://docs.novita.ai/guides/sandbox-pricing) |
|
|
53
|
+
| [Freestyle](https://sandboxrouter.ai/providers/freestyle) | $0.134 | none | passed (2026-11-13) | [pricing](https://www.freestyle.sh/docs/vms/pricing-and-limits) |
|
|
54
|
+
| [Beam](https://sandboxrouter.ai/providers/beam) | $0.140 | $20/mo (credited) | not run yet | [pricing](https://www.beam.cloud/pricing) |
|
|
55
|
+
| [CodeSandbox SDK (Together)](https://sandboxrouter.ai/providers/codesandbox) | $0.149 | none | not run yet | [pricing](https://docs.together.ai/docs/together-code-sandbox-legacy) |
|
|
56
|
+
| [Fly.io Sprites](https://sandboxrouter.ai/providers/sprites) | up to $0.164 | none | passed (2026-11-13) | [pricing](https://fly.io/sprites) |
|
|
57
|
+
| [Blaxel](https://sandboxrouter.ai/providers/blaxel) | $0.166 | none | passed (2026-11-13) | [pricing](https://blaxel.ai/pricing) |
|
|
58
|
+
| [HOPX](https://sandboxrouter.ai/providers/hopx) | $0.166 | none | not run yet | [pricing](https://hopx.ai/pricing) |
|
|
59
|
+
| [Superserve](https://sandboxrouter.ai/providers/superserve) | $0.166 | maybe $20/mo (credited) | passed (2026-11-13) | [pricing](https://superserve.ai/pricing) |
|
|
60
|
+
| [Tensorlake](https://sandboxrouter.ai/providers/tensorlake) | up to $0.200 | none | not run yet | [pricing](https://www.tensorlake.ai/pricing) |
|
|
61
|
+
| [Upstash Box](https://sandboxrouter.ai/providers/upstash) | up to $0.200 | none | not run yet | [pricing](https://upstash.com/pricing/box) |
|
|
62
|
+
| [Cloudflare Sandbox](https://sandboxrouter.ai/providers/cloudflare) | up to $0.216 | $5/mo | passed (2026-11-13) | [pricing](https://developers.cloudflare.com/containers/platform/pricing/) |
|
|
63
|
+
| [Amazon Bedrock AgentCore Code Interpreter](https://sandboxrouter.ai/providers/agentcore) | up to $0.217 | none | passed (2026-11-13) | [pricing](https://aws.amazon.com/bedrock/agentcore/pricing/) |
|
|
64
|
+
| [Modal](https://sandboxrouter.ai/providers/modal) | up to $0.238 | none | passed (2026-11-13) | [pricing](https://modal.com/pricing) |
|
|
65
|
+
| [AWS Lambda MicroVMs](https://sandboxrouter.ai/providers/lambda) | $0.252 | none | passed (2026-11-13) | [pricing](https://aws.amazon.com/lambda/pricing/) |
|
|
66
|
+
| [Deno Sandbox](https://sandboxrouter.ai/providers/deno) | up to $0.300 | $20/mo | not run yet | [pricing](https://deno.com/sandbox) |
|
|
67
|
+
| [Runloop](https://sandboxrouter.ai/providers/runloop) | $0.317 | none | not run yet | [pricing](https://www.runloop.ai/pricing) |
|
|
68
|
+
| [Vercel Sandbox](https://sandboxrouter.ai/providers/vercel) | up to $0.341 | none | passed (2026-11-13) | [pricing](https://vercel.com/docs/vercel-sandbox/pricing) |
|
|
69
|
+
| [InstaBox (sign-up waitlisted)](https://sandboxrouter.ai/providers/instabox) | $0.017 | none | not run yet | [pricing](https://instabox.sh/docs/pricing.md) |
|
|
70
|
+
| [Koyeb Sandboxes (not taking new accounts)](https://sandboxrouter.ai/providers/koyeb) | $0.058 | $29/mo | not run yet | [pricing](https://www.koyeb.com/docs/reference/instances) |
|
|
71
|
+
| [Dedalus Machines (sign-up waitlisted)](https://sandboxrouter.ai/providers/dedalus) | $0.149 | none | not run yet | [pricing](https://docs.dedaluslabs.ai/dcs/dm/pricing) |
|
|
72
|
+
| [LangSmith Sandboxes (sign-up waitlisted)](https://sandboxrouter.ai/providers/langsmith) | $0.189 | none | not run yet | [pricing](https://www.langchain.com/pricing) |
|
|
73
|
+
| [OpenScale Sandboxes](https://sandboxrouter.ai/providers/openscale) | skipped (default image is 2 vCPU / 1 GB; a bigger size needs image.openscale) | none | passed (2026-11-13) | [pricing](https://openscale.so/sandboxes) |
|
|
74
|
+
| [Azure Container Apps dynamic sessions](https://sandboxrouter.ai/providers/azure) | skipped (max 1 vCPU) | none | not run yet | [pricing](https://learn.microsoft.com/en-us/azure/container-apps/billing#dynamic-sessions) |
|
|
75
|
+
| [Daytona](https://sandboxrouter.ai/providers/daytona) | skipped (default image is 1 vCPU / 1 GB; a bigger size needs image.daytona) | none | passed (2026-11-13) | [pricing](https://www.daytona.io/pricing) |
|
|
76
|
+
| [Alibaba Cloud Agent Sandbox](https://sandboxrouter.ai/providers/alibaba) | skipped (max 2 GB memory) | none | not run yet | [pricing](https://www.alibabacloud.com/help/en/agent-sandbox/product-overview/billing-overview) |
|
|
77
|
+
| [E2B](https://sandboxrouter.ai/providers/e2b) | skipped (default image is 2 vCPU / 0.5 GB; a bigger size needs image.e2b) | none | passed (2026-11-13) | [pricing](https://e2b.dev/pricing) |
|
|
78
|
+
| [Docker (local)](https://sandboxrouter.ai/providers/docker) | free | none | passed (2026-11-13) | [pricing](https://docs.docker.com/engine/) |
|
|
79
|
+
|
|
80
|
+
Checked 2026-11-04 to 2026-11-13.
|
|
81
81
|
|
|
82
82
|
## Install
|
|
83
83
|
|
|
@@ -89,7 +89,7 @@ Then the SDK of each provider you use (many need none) and its keys: both are li
|
|
|
89
89
|
|
|
90
90
|
## More
|
|
91
91
|
|
|
92
|
-
- [Docs](https://sandboxrouter.
|
|
92
|
+
- [Docs](https://sandboxrouter.ai/docs): routing, fallback, streaming, files, pause, the HTTP server and Python.
|
|
93
93
|
- [Reference](docs/reference.md): the full library, CLI and HTTP API, desktops and phones, the real-account compatibility table, and how each adapter maps the spec.
|
|
94
94
|
- [CONTRIBUTING.md](CONTRIBUTING.md): run a provider for real and send the result, fix a price, or add a provider.
|
|
95
95
|
- [CHANGELOG.md](CHANGELOG.md) · [SECURITY.md](SECURITY.md) · [MIT](LICENSE)
|
package/dist/billing.js
CHANGED
package/dist/catalog.d.ts
CHANGED
|
@@ -2,10 +2,16 @@ import type { Capability, ProviderId, SandboxSpec } from "./types.js";
|
|
|
2
2
|
export interface ProviderInfo {
|
|
3
3
|
id: ProviderId;
|
|
4
4
|
name: string;
|
|
5
|
-
/** List price while running, USD. */
|
|
5
|
+
/** List price while running, USD, and the rules that turn running time into a bill. */
|
|
6
6
|
pricing: {
|
|
7
7
|
vcpuHour: number;
|
|
8
8
|
gbHour: number;
|
|
9
|
+
/** Each sandbox is billed for at least this many seconds of running time. Default: no minimum. */
|
|
10
|
+
minSeconds?: number;
|
|
11
|
+
/** Running time is rounded up to a multiple of this many seconds. Default 1 (per second). */
|
|
12
|
+
incrementSeconds?: number;
|
|
13
|
+
/** A fixed charge per sandbox created, USD, on top of running time. */
|
|
14
|
+
perSandboxUsd?: number;
|
|
9
15
|
note?: string;
|
|
10
16
|
source: string;
|
|
11
17
|
checked: string;
|
|
@@ -258,6 +264,17 @@ export declare function cloudflareInstance(vcpu: number, memoryGb: number): {
|
|
|
258
264
|
readonly vcpu: 4;
|
|
259
265
|
readonly memoryGb: 12;
|
|
260
266
|
};
|
|
267
|
+
/** How a provider turns running time into a bill: minimum, rounding and fixed per-sandbox charge. */
|
|
268
|
+
export declare function billingRules(id: ProviderId): {
|
|
269
|
+
minSeconds: number;
|
|
270
|
+
incrementSeconds: number;
|
|
271
|
+
perSandboxUsd: number;
|
|
272
|
+
};
|
|
273
|
+
/**
|
|
274
|
+
* What a provider bills for one sandbox that ran `seconds` at `hourlyUsd`: running time rounded up to
|
|
275
|
+
* the provider's increment, at least its minimum, plus any fixed per-sandbox charge.
|
|
276
|
+
*/
|
|
277
|
+
export declare function billedUsd(id: ProviderId, hourlyUsd: number, seconds: number): number;
|
|
261
278
|
export declare function info(id: ProviderId): ProviderInfo;
|
|
262
279
|
export declare const DEFAULT_VCPU = 2;
|
|
263
280
|
export declare const DEFAULT_MEMORY_GB = 4;
|
package/dist/catalog.js
CHANGED
|
@@ -91,7 +91,7 @@ const ENTRIES = [
|
|
|
91
91
|
gbHour: 0.0162,
|
|
92
92
|
note: "per second, plus disk above 5 GiB",
|
|
93
93
|
source: "https://www.daytona.io/pricing",
|
|
94
|
-
checked: "2026-11-
|
|
94
|
+
checked: "2026-11-13",
|
|
95
95
|
},
|
|
96
96
|
capabilities: ["pause", "custom-image"],
|
|
97
97
|
limits: { maxVcpu: 4, maxMemoryGb: 8 },
|
|
@@ -119,6 +119,7 @@ const ENTRIES = [
|
|
|
119
119
|
id: "vercel",
|
|
120
120
|
name: "Vercel Sandbox",
|
|
121
121
|
pricing: {
|
|
122
|
+
minSeconds: 60,
|
|
122
123
|
vcpuHour: 0.128,
|
|
123
124
|
gbHour: 0.0212,
|
|
124
125
|
note: "iad1; active CPU, provisioned RAM in minimum 1-minute increments (2 GB/vCPU); full-CPU estimate; Hobby free within quotas, Pro $20/mo usage credit; creations $0.60/million and retained snapshots $0.08/GB-month extra",
|
|
@@ -153,7 +154,7 @@ const ENTRIES = [
|
|
|
153
154
|
gbHour: 0.021875,
|
|
154
155
|
note: "bills actual CPU and memory use; this is the full-use price; storage extra",
|
|
155
156
|
source: "https://fly.io/sprites",
|
|
156
|
-
checked: "2026-11-
|
|
157
|
+
checked: "2026-11-13",
|
|
157
158
|
},
|
|
158
159
|
capabilities: [],
|
|
159
160
|
// Not published per sprite; 8 GB is what accounts report getting by default.
|
|
@@ -221,7 +222,7 @@ ENTRIES.push({
|
|
|
221
222
|
gbHour: 0,
|
|
222
223
|
note: "fixed sizes billed per second ($0.018/h for 2 vCPU / 4 GB up to $0.20/h for 16 / 32); needs a plan from $20/mo, paid back as VM time",
|
|
223
224
|
source: "https://docs.boat.dev/pricing.md",
|
|
224
|
-
checked: "2026-11-
|
|
225
|
+
checked: "2026-11-13",
|
|
225
226
|
},
|
|
226
227
|
// "pause" here is Boat's stop: the disk is kept, running processes end.
|
|
227
228
|
capabilities: ["pause", "custom-image"],
|
|
@@ -251,7 +252,7 @@ ENTRIES.push({
|
|
|
251
252
|
gbHour: 0.00833,
|
|
252
253
|
note: "per second on Northflank Cloud, ephemeral disk included; sandboxes run as deployment services (microVMs)",
|
|
253
254
|
source: "https://northflank.com/pricing",
|
|
254
|
-
checked: "2026-11-
|
|
255
|
+
checked: "2026-11-13",
|
|
255
256
|
},
|
|
256
257
|
capabilities: ["custom-image"],
|
|
257
258
|
limits: { maxVcpu: 8, maxMemoryGb: 16 },
|
|
@@ -295,7 +296,7 @@ ENTRIES.push({
|
|
|
295
296
|
gbHour: 0.00945,
|
|
296
297
|
note: "per second; CPU billed on actual use (I/O wait free), memory on peak; fixed 2 vCPU / 8 GB session, this is the 100%-CPU price",
|
|
297
298
|
source: "https://aws.amazon.com/bedrock/agentcore/pricing/",
|
|
298
|
-
checked: "2026-11-
|
|
299
|
+
checked: "2026-11-13",
|
|
299
300
|
},
|
|
300
301
|
capabilities: [],
|
|
301
302
|
limits: { maxVcpu: 2, maxMemoryGb: 8 },
|
|
@@ -308,6 +309,8 @@ ENTRIES.push({
|
|
|
308
309
|
id: "azure",
|
|
309
310
|
name: "Azure Container Apps dynamic sessions",
|
|
310
311
|
pricing: {
|
|
312
|
+
minSeconds: 3600,
|
|
313
|
+
incrementSeconds: 3600,
|
|
311
314
|
vcpuHour: 0.03,
|
|
312
315
|
gbHour: 0,
|
|
313
316
|
note: "flat $0.03 per session-hour (built-in Shell pools), billed per allocated session in 1-hour increments, East US; needs a session pool you create once",
|
|
@@ -343,11 +346,13 @@ ENTRIES.push({
|
|
|
343
346
|
id: "agent37",
|
|
344
347
|
name: "Agent37",
|
|
345
348
|
pricing: {
|
|
349
|
+
minSeconds: 60,
|
|
350
|
+
incrementSeconds: 60,
|
|
346
351
|
vcpuHour: 0.8 / 730,
|
|
347
352
|
gbHour: 0.7 / 730,
|
|
348
353
|
note: "whole-minute billing: monthly shape total / 730 / 60; running estimate includes default disk (2 vCPU / 4 GB / 4 GB disk = $4.76/mo); sleeping/stopped disk $0.09/GB-mo; gVisor; $5 minimum top-up, wallet must hold one running day",
|
|
349
354
|
source: "https://www.agent37.com/docs/agents-api/billing",
|
|
350
|
-
checked: "2026-11-
|
|
355
|
+
checked: "2026-11-13",
|
|
351
356
|
},
|
|
352
357
|
capabilities: ["pause", "custom-image"],
|
|
353
358
|
limits: { maxVcpu: 4, maxMemoryGb: 32 },
|
|
@@ -387,6 +392,8 @@ ENTRIES.push({
|
|
|
387
392
|
id: "codesandbox",
|
|
388
393
|
name: "CodeSandbox SDK (Together)",
|
|
389
394
|
pricing: {
|
|
395
|
+
minSeconds: 60,
|
|
396
|
+
incrementSeconds: 60,
|
|
390
397
|
vcpuHour: 0.0743,
|
|
391
398
|
gbHour: 0,
|
|
392
399
|
note: "fixed tiers: Pico 2 vCPU / 1 GB $0.0743/h, Nano 2 / 4 $0.1486/h, up to 64 / 128 $4.7552/h; credits $0.01486, billed per started minute; Build free, 10 concurrent; Scale $170/mo incl. 1,100 credits",
|
|
@@ -415,14 +422,15 @@ ENTRIES.push({
|
|
|
415
422
|
id: "sail",
|
|
416
423
|
name: "Sailboxes (Sail Research)",
|
|
417
424
|
pricing: {
|
|
425
|
+
perSandboxUsd: 0.01,
|
|
418
426
|
vcpuHour: 0.015,
|
|
419
427
|
gbHour: 0.008,
|
|
420
|
-
note: "
|
|
428
|
+
note: "on what the box uses (CPU time, RAM it holds, disk $0.0007/GiB-h), sampled per second and not rounded up; RAM counts the VM's file cache, up to its memory limit (16 GiB at least), so file-heavy work can hold more than you asked for; paused/sleeping free; $0.01 per new Sailbox on the free plan, waived on Pro and Enterprise",
|
|
421
429
|
source: "https://docs.sailresearch.com/sailboxes-pricing.md",
|
|
422
|
-
checked: "2026-11-
|
|
430
|
+
checked: "2026-11-13",
|
|
423
431
|
},
|
|
424
432
|
capabilities: ["pause"],
|
|
425
|
-
limits: { maxVcpu: 64, maxMemoryGb:
|
|
433
|
+
limits: { maxVcpu: 64, maxMemoryGb: 256 },
|
|
426
434
|
credentials: [["SAIL_API_KEY"]],
|
|
427
435
|
docs: "https://docs.sailresearch.com/sailboxes.md",
|
|
428
436
|
}, {
|
|
@@ -661,6 +669,20 @@ export const BOAT_SIZES = [
|
|
|
661
669
|
export function cloudflareInstance(vcpu, memoryGb) {
|
|
662
670
|
return CLOUDFLARE_INSTANCES.find((t) => t.vcpu >= vcpu && t.memoryGb >= memoryGb) ?? CLOUDFLARE_INSTANCES[CLOUDFLARE_INSTANCES.length - 1];
|
|
663
671
|
}
|
|
672
|
+
/** How a provider turns running time into a bill: minimum, rounding and fixed per-sandbox charge. */
|
|
673
|
+
export function billingRules(id) {
|
|
674
|
+
const p = CATALOG.find((c) => c.id === id)?.pricing;
|
|
675
|
+
return { minSeconds: p?.minSeconds ?? 0, incrementSeconds: p?.incrementSeconds ?? 1, perSandboxUsd: p?.perSandboxUsd ?? 0 };
|
|
676
|
+
}
|
|
677
|
+
/**
|
|
678
|
+
* What a provider bills for one sandbox that ran `seconds` at `hourlyUsd`: running time rounded up to
|
|
679
|
+
* the provider's increment, at least its minimum, plus any fixed per-sandbox charge.
|
|
680
|
+
*/
|
|
681
|
+
export function billedUsd(id, hourlyUsd, seconds) {
|
|
682
|
+
const r = billingRules(id);
|
|
683
|
+
const billed = Math.max(r.minSeconds, Math.ceil(Math.max(0, seconds) / r.incrementSeconds) * r.incrementSeconds);
|
|
684
|
+
return (hourlyUsd * billed) / 3600 + r.perSandboxUsd;
|
|
685
|
+
}
|
|
664
686
|
export function info(id) {
|
|
665
687
|
const p = CATALOG.find((c) => c.id === id);
|
|
666
688
|
if (!p)
|
package/dist/check.d.ts
CHANGED
|
@@ -17,7 +17,8 @@ export interface CheckReport {
|
|
|
17
17
|
ok: boolean;
|
|
18
18
|
/**
|
|
19
19
|
* Set when the run says nothing about the provider: the account running it couldn't pay (402: no
|
|
20
|
-
* balance, plan not started)
|
|
20
|
+
* balance, plan not started), or the provider's SDK isn't installed where it ran. Such a run isn't
|
|
21
|
+
* saved, so the provider keeps its last real result.
|
|
21
22
|
*/
|
|
22
23
|
inconclusive?: string;
|
|
23
24
|
}
|
package/dist/check.js
CHANGED
|
@@ -42,21 +42,28 @@ export async function checkProvider(router, provider, opts = {}) {
|
|
|
42
42
|
}
|
|
43
43
|
};
|
|
44
44
|
let sbx;
|
|
45
|
-
let
|
|
45
|
+
let notRun;
|
|
46
46
|
await step("create", async () => {
|
|
47
47
|
try {
|
|
48
48
|
sbx = await router.create({ timeoutMinutes: 10, ...fixedSize(provider), ...opts.spec }, { provider, fallback: false });
|
|
49
49
|
}
|
|
50
50
|
catch (err) {
|
|
51
|
-
|
|
51
|
+
const msg = err.message;
|
|
52
|
+
if (err.attempts?.at(-1)?.status === 402) {
|
|
53
|
+
notRun = "the account running the check can't pay (402)";
|
|
54
|
+
return { status: "skip", detail: `the account can't pay (402), so this run says nothing about ${provider}: ${msg}` };
|
|
55
|
+
}
|
|
56
|
+
// The provider's SDK isn't installed where the check runs: that says nothing about the provider.
|
|
57
|
+
const pkg = /needs the "([^"]+)" package/.exec(msg)?.[1];
|
|
58
|
+
if (!pkg)
|
|
52
59
|
throw err;
|
|
53
|
-
|
|
54
|
-
return { status: "skip", detail:
|
|
60
|
+
notRun = `the machine running the check doesn't have the "${pkg}" package`;
|
|
61
|
+
return { status: "skip", detail: `${notRun}, so this run says nothing about ${provider}: npm i ${pkg}` };
|
|
55
62
|
}
|
|
56
63
|
return `${sbx.id}, ~$${sbx.estimatedHourlyUsd}/hr`;
|
|
57
64
|
});
|
|
58
|
-
if (
|
|
59
|
-
return { provider, results, ok: false, inconclusive:
|
|
65
|
+
if (notRun)
|
|
66
|
+
return { provider, results, ok: false, inconclusive: notRun };
|
|
60
67
|
if (!sbx)
|
|
61
68
|
return { provider, results, ok: false };
|
|
62
69
|
const box = sbx;
|
package/dist/cli.js
CHANGED
|
@@ -18,10 +18,12 @@ Usage:
|
|
|
18
18
|
sandboxrouter providers [--vcpu N] [--memory GB] price table + which keys are set
|
|
19
19
|
sandboxrouter route [spec flags] [route flags] dry run: the order providers would be tried
|
|
20
20
|
sandboxrouter run "<cmd>" [spec flags] [route flags] [--keep]
|
|
21
|
-
sandboxrouter check --provider ID[,ID] | --all [--json] [--save [dir]] [--by GITHUB_USER]
|
|
21
|
+
sandboxrouter check --provider ID[,ID] | --all [--json] [--save [dir]] [--retry] [--out FILE] [--by GITHUB_USER]
|
|
22
22
|
test providers for real: exec, env, UTF-8, streaming, files, timeout, pause,
|
|
23
23
|
kill; exits 1 if any step fails. --all = every configured provider;
|
|
24
|
-
--save writes results/conformance/<id>.json for the compatibility table
|
|
24
|
+
--save writes results/conformance/<id>.json for the compatibility table;
|
|
25
|
+
--retry runs a failed provider once more and keeps that result;
|
|
26
|
+
--out writes every report as JSON (with the first try's failure, if retried)
|
|
25
27
|
sandboxrouter ls [--json] sandboxes this machine started that weren't killed
|
|
26
28
|
sandboxrouter gc [--all | --older-than MIN] [--dry-run]
|
|
27
29
|
kill leaked sandboxes: by default those past their lifetime
|
|
@@ -73,6 +75,7 @@ const { values, positionals } = parseArgs({
|
|
|
73
75
|
label: { type: "string" },
|
|
74
76
|
all: { type: "boolean" },
|
|
75
77
|
save: { type: "boolean" },
|
|
78
|
+
retry: { type: "boolean" },
|
|
76
79
|
"older-than": { type: "string" },
|
|
77
80
|
"dry-run": { type: "boolean" },
|
|
78
81
|
minutes: { type: "string" },
|
|
@@ -161,7 +164,8 @@ async function main() {
|
|
|
161
164
|
const list = c.listHourlyUsd !== c.estimatedHourlyUsd ? ` (list $${c.listHourlyUsd.toFixed(4)})` : "";
|
|
162
165
|
const size = `${c.size.vcpu} vCPU / ${c.size.memoryGb} GB`;
|
|
163
166
|
const speed = c.medianCreateMs !== undefined ? `${c.medianCreateMs} ms` : "";
|
|
164
|
-
const
|
|
167
|
+
const fee = c.perSandboxUsd ? ` + $${c.perSandboxUsd}/sandbox` : "";
|
|
168
|
+
const cols = [`$${c.estimatedHourlyUsd.toFixed(4)}/hr${list}${fee}`, size, speed, tested].filter(Boolean).join(" ");
|
|
165
169
|
console.log(`${mark} ${c.provider.padEnd(11)} ${cols}${c.skip ? ` (skipped: ${c.skip})` : ""}`);
|
|
166
170
|
}
|
|
167
171
|
return;
|
|
@@ -208,16 +212,23 @@ async function main() {
|
|
|
208
212
|
const saveDir = values.save ? (rest[0] ?? "results/conformance") : undefined;
|
|
209
213
|
const mark = { pass: "ok ", fail: "FAIL", skip: "skip", info: "note" };
|
|
210
214
|
const reports = [];
|
|
215
|
+
const run = (p) => checkProvider(router, p, {
|
|
216
|
+
spec,
|
|
217
|
+
onStep: (r) => {
|
|
218
|
+
if (!values.json)
|
|
219
|
+
console.log(` ${mark[r.status]} ${r.step.padEnd(32)} ${String(r.ms).padStart(6)} ms ${r.detail ?? ""}`);
|
|
220
|
+
},
|
|
221
|
+
});
|
|
211
222
|
for (const p of ids) {
|
|
212
223
|
if (!values.json)
|
|
213
224
|
console.log(`\n${p}`);
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
}
|
|
220
|
-
}
|
|
225
|
+
let report = await run(p);
|
|
226
|
+
if (values.retry && !report.ok && !report.inconclusive) {
|
|
227
|
+
const failed = report.results.find((r) => r.status === "fail");
|
|
228
|
+
if (!values.json)
|
|
229
|
+
console.log(` failed; trying ${p} once more`);
|
|
230
|
+
report = { ...(await run(p)), firstTry: failed ? `${failed.step}: ${failed.detail ?? "failed"}` : "failed" };
|
|
231
|
+
}
|
|
221
232
|
reports.push(report);
|
|
222
233
|
if (report.inconclusive) {
|
|
223
234
|
if (!values.json)
|
|
@@ -226,16 +237,19 @@ async function main() {
|
|
|
226
237
|
}
|
|
227
238
|
if (saveDir) {
|
|
228
239
|
mkdirSync(saveDir, { recursive: true });
|
|
229
|
-
const
|
|
240
|
+
const { firstTry: _, ...result } = report;
|
|
241
|
+
const saved = publishableCheck({ tool: "sandboxrouter-check", version, date: new Date().toISOString(), platform: `${platform()}-${arch()}`, ...(by ? { by } : {}), ...result });
|
|
230
242
|
writeFileSync(`${saveDir}/${p}.json`, JSON.stringify(saved, null, 2) + "\n");
|
|
231
243
|
}
|
|
232
244
|
}
|
|
245
|
+
if (values.out)
|
|
246
|
+
writeFileSync(values.out, JSON.stringify(reports, null, 2) + "\n");
|
|
233
247
|
if (values.json)
|
|
234
248
|
console.log(JSON.stringify(reports, null, 2));
|
|
235
249
|
else {
|
|
236
250
|
const unpaid = reports.filter((r) => r.inconclusive).length;
|
|
237
251
|
const ran = reports.length - unpaid;
|
|
238
|
-
console.log(`\n${reports.filter((r) => r.ok).length}/${ran} passed${unpaid ? `; ${unpaid} not run (the account can't pay)` : ""}`);
|
|
252
|
+
console.log(`\n${reports.filter((r) => r.ok).length}/${ran} passed${unpaid ? `; ${unpaid} not run (the account can't pay, or the provider's package isn't installed)` : ""}`);
|
|
239
253
|
}
|
|
240
254
|
if (reports.some((r) => !r.ok && !r.inconclusive))
|
|
241
255
|
process.exitCode = 1;
|
package/dist/cost.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { readdirSync, readFileSync } from "node:fs";
|
|
2
2
|
import { join } from "node:path";
|
|
3
|
-
import { benchmarksPublishable, fittingSpec, hourlyCost } from "./catalog.js";
|
|
3
|
+
import { benchmarksPublishable, billingRules, fittingSpec, hourlyCost } from "./catalog.js";
|
|
4
4
|
import { Router } from "./router.js";
|
|
5
5
|
/**
|
|
6
6
|
* The size the sandbox really has, as "<vcpus> <memoryMb>". nproc and MemTotal can show the whole host
|
|
@@ -123,6 +123,21 @@ export const BILL_READERS = {
|
|
|
123
123
|
source: `sail /sailboxes/spend cost_usd_nanos (estimate at list rates; ${fees}; ${Math.round(row.vcpu_seconds ?? 0)} vCPU-s, ${Math.round(row.memory_gib_seconds ?? 0)} GiB-s)`,
|
|
124
124
|
};
|
|
125
125
|
},
|
|
126
|
+
async agent37(e) {
|
|
127
|
+
const h = { Authorization: `Bearer ${process.env.AGENT37_API_KEY}` };
|
|
128
|
+
const api = process.env.AGENT37_API_URL ?? "https://api.agent37.com";
|
|
129
|
+
const q = new URLSearchParams({ from: e.startedAt.slice(0, 10), to: e.endedAt.slice(0, 10) });
|
|
130
|
+
const j = await getJson(`${api}/v1/usage?${q}`, h);
|
|
131
|
+
const id = e.sandboxId.replace(/^agent37:/, "");
|
|
132
|
+
const row = (j.instances ?? []).find((i) => i.id === id);
|
|
133
|
+
if (!row)
|
|
134
|
+
return { source: "agent37 /v1/usage: not posted yet" };
|
|
135
|
+
return {
|
|
136
|
+
usd: round((row.total_micros ?? 0) / 1e6),
|
|
137
|
+
seconds: (row.awake_minutes ?? 0) * 60,
|
|
138
|
+
source: `agent37 /v1/usage total_micros (billed in whole minutes: ${row.awake_minutes ?? 0} awake)`,
|
|
139
|
+
};
|
|
140
|
+
},
|
|
126
141
|
async e2b() {
|
|
127
142
|
return { source: "no billing API: compare the console usage tab (console.e2b.dev, Usage)" };
|
|
128
143
|
},
|
|
@@ -195,12 +210,17 @@ export function billedVsList(dir = "results/cost") {
|
|
|
195
210
|
const predicted = e.predictedUsd;
|
|
196
211
|
if (!usd || !predicted || usd <= 0 || predicted <= 0 || !benchmarksPublishable(e.provider))
|
|
197
212
|
continue;
|
|
213
|
+
// A fixed per-sandbox charge (Sail's $0.01) isn't an hourly rate: take it out, or a 10-minute run
|
|
214
|
+
// turns it into "bills 46% over list" for every hour after.
|
|
215
|
+
const running = usd - billingRules(e.provider).perSandboxUsd;
|
|
216
|
+
if (running <= 0)
|
|
217
|
+
continue;
|
|
198
218
|
if (/integer cents/i.test(e.bill.source) && predicted < 1)
|
|
199
219
|
continue;
|
|
200
220
|
const date = run.date.slice(0, 10);
|
|
201
221
|
const prev = latest.get(e.provider);
|
|
202
222
|
if (!prev || date >= prev.date)
|
|
203
|
-
latest.set(e.provider, { ratio: Math.round((
|
|
223
|
+
latest.set(e.provider, { ratio: Math.round((running / predicted) * 1000) / 1000, date });
|
|
204
224
|
}
|
|
205
225
|
}
|
|
206
226
|
return [...latest].map(([provider, v]) => ({ provider, ...v })).sort((a, b) => a.provider.localeCompare(b.provider));
|
package/dist/index.d.ts
CHANGED
|
@@ -6,7 +6,7 @@ export { Router, RoutedSandbox, defaultProviders, defaultComputerProviders, pars
|
|
|
6
6
|
export { VERIFIED, FAILING } from "./verified.js";
|
|
7
7
|
export { BILLED_VS_LIST } from "./billing.js";
|
|
8
8
|
export { validate, splitRequest, SPEC_KEYS, ROUTE_KEYS, STRATEGIES } from "./validate.js";
|
|
9
|
-
export { CATALOG, hourlyCost, unfit, info, type ProviderInfo } from "./catalog.js";
|
|
9
|
+
export { CATALOG, hourlyCost, unfit, info, billingRules, billedUsd, type ProviderInfo } from "./catalog.js";
|
|
10
10
|
export { createApp, HttpError, ERROR_STATUS, ORPHAN_GRACE_MS, type ServerHooks } from "./server.js";
|
|
11
11
|
export { Client, ClientError, RemoteSandbox, type CreateResponse } from "./client.js";
|
|
12
12
|
export { RoutedComputer, createComputer, splitComputerRequest, DEFAULT_COMPUTER_TIMEOUT_S, DEFAULT_COMPUTER_DEADLINE_S, type ComputerRoute, planComputer, computerCatalog, computerHourly, COMPUTER_ENTRIES, type ComputerCatalogEntry, type ComputerCandidate, type ComputerProviderId, type ActionResult, type ComputerAction, type ComputerActionName, type ComputerInfo, type ComputerOs, type ComputerProvider, type ComputerSpec, type ProviderComputer, } from "./computers.js";
|
package/dist/index.js
CHANGED
|
@@ -6,7 +6,7 @@ export { Router, RoutedSandbox, defaultProviders, defaultComputerProviders, pars
|
|
|
6
6
|
export { VERIFIED, FAILING } from "./verified.js";
|
|
7
7
|
export { BILLED_VS_LIST } from "./billing.js";
|
|
8
8
|
export { validate, splitRequest, SPEC_KEYS, ROUTE_KEYS, STRATEGIES } from "./validate.js";
|
|
9
|
-
export { CATALOG, hourlyCost, unfit, info } from "./catalog.js";
|
|
9
|
+
export { CATALOG, hourlyCost, unfit, info, billingRules, billedUsd } from "./catalog.js";
|
|
10
10
|
export { createApp, HttpError, ERROR_STATUS, ORPHAN_GRACE_MS } from "./server.js";
|
|
11
11
|
export { Client, ClientError, RemoteSandbox } from "./client.js";
|
|
12
12
|
export { RoutedComputer, createComputer, splitComputerRequest, DEFAULT_COMPUTER_TIMEOUT_S, DEFAULT_COMPUTER_DEADLINE_S, planComputer, computerCatalog, computerHourly, COMPUTER_ENTRIES, } from "./computers.js";
|
package/dist/router.d.ts
CHANGED
|
@@ -29,6 +29,11 @@ export interface Candidate {
|
|
|
29
29
|
verified: boolean;
|
|
30
30
|
/** Failed a step of its last real-account check (as of this release, or `RouterOptions.failing`). */
|
|
31
31
|
failing: boolean;
|
|
32
|
+
/**
|
|
33
|
+
* A fixed charge per sandbox, USD, on top of `estimatedHourlyUsd` (Sail's free plan: $0.01). Ranking
|
|
34
|
+
* spreads it over the sandbox's lifetime (`timeoutMinutes`, default 30), so it weighs more on short ones.
|
|
35
|
+
*/
|
|
36
|
+
perSandboxUsd?: number;
|
|
32
37
|
/** Why this provider won't be tried, if it won't. */
|
|
33
38
|
skip?: string;
|
|
34
39
|
}
|
|
@@ -63,6 +68,18 @@ export interface RouterOptions {
|
|
|
63
68
|
requireVerified?: boolean;
|
|
64
69
|
/** Billed / list-price ratio per provider, behind `estimatedHourlyUsd`. Default: the ratios in `BILLED_VS_LIST`, from saved cost runs. */
|
|
65
70
|
billedVsList?: Partial<Record<ProviderId, number>>;
|
|
71
|
+
/**
|
|
72
|
+
* When a create runs well past a provider's usual start time, try the next provider sooner than
|
|
73
|
+
* `createTimeoutSeconds`. The wait becomes `factor` x the provider's median start time (or 1.5x the
|
|
74
|
+
* slowest of its recent starts, if longer), at least `minSeconds`. It applies only when the caller
|
|
75
|
+
* didn't set `createTimeoutSeconds`, the router has a start time for the provider (3 of its own creates,
|
|
76
|
+
* or the bench file), and another provider is left to try. Default `{ factor: 5, minSeconds: 10 }`;
|
|
77
|
+
* `false` always waits the full `createTimeoutSeconds`.
|
|
78
|
+
*/
|
|
79
|
+
quickFallback?: false | {
|
|
80
|
+
factor?: number;
|
|
81
|
+
minSeconds?: number;
|
|
82
|
+
};
|
|
66
83
|
}
|
|
67
84
|
/** Desktop-only providers. Filled in as adapters are added. */
|
|
68
85
|
export declare function defaultComputerProviders(): ComputerProvider[];
|
|
@@ -77,6 +94,7 @@ export declare class Router {
|
|
|
77
94
|
/** Create times this router measured, most recent last. */
|
|
78
95
|
private readonly live;
|
|
79
96
|
private computerOnly;
|
|
97
|
+
private readonly quick;
|
|
80
98
|
/** The local record of sandboxes this router created, or undefined when turned off. */
|
|
81
99
|
readonly ledger?: Ledger;
|
|
82
100
|
constructor(opts?: RouterOptions);
|
|
@@ -94,6 +112,11 @@ export declare class Router {
|
|
|
94
112
|
private whyNone;
|
|
95
113
|
/** Start a sandbox on the best available provider, falling back down the list on failure. */
|
|
96
114
|
create(spec?: SandboxSpec, route?: RouteOptions): Promise<RoutedSandbox>;
|
|
115
|
+
/**
|
|
116
|
+
* Seconds to wait for this provider's create: the caller's `createTimeoutSeconds`, or (see
|
|
117
|
+
* `quickFallback`) a few times its usual start time when another provider is left to try.
|
|
118
|
+
*/
|
|
119
|
+
private waitFor;
|
|
97
120
|
/** Remember a successful create's time, for the speed tie-break and strategy "fastest". */
|
|
98
121
|
private observe;
|
|
99
122
|
private record;
|
|
@@ -161,9 +184,10 @@ export declare function parseId(id: string): {
|
|
|
161
184
|
* The default order. Providers whose last real-account check failed go last. The rest go cheapest
|
|
162
185
|
* first by effective price (list price x what the provider really billed, where measured), except that
|
|
163
186
|
* providers within PRICE_TIE of the cheapest remaining one count as a tie and go fastest first (by
|
|
164
|
-
* median create time; unknown speed after known), then cheapest, then alphabetical.
|
|
187
|
+
* median create time; unknown speed after known), then cheapest, then alphabetical. A fixed
|
|
188
|
+
* per-sandbox charge counts as spread over `lifetimeHours` (the sandbox's timeoutMinutes).
|
|
165
189
|
*/
|
|
166
|
-
export declare function balanced(list: Candidate[]): Candidate[];
|
|
190
|
+
export declare function balanced(list: Candidate[], lifetimeHours?: number): Candidate[];
|
|
167
191
|
/** Seconds to wait for one provider's create before trying the next (RouteOptions.createTimeoutSeconds). */
|
|
168
192
|
export declare const DEFAULT_CREATE_TIMEOUT_S = 120;
|
|
169
193
|
/** Seconds a whole create may take over every provider it tries (RouteOptions.deadlineSeconds). */
|
package/dist/router.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { CATALOG, DEFAULT_MEMORY_GB, DEFAULT_VCPU, accessLabel, fixedSize, hourlyCost, unfit } from "./catalog.js";
|
|
1
|
+
import { CATALOG, DEFAULT_MEMORY_GB, DEFAULT_VCPU, accessLabel, billingRules, fixedSize, hourlyCost, unfit } from "./catalog.js";
|
|
2
2
|
import { RoutedComputer, createComputer, planComputer, } from "./computers.js";
|
|
3
3
|
import { loadBench, median } from "./bench-store.js";
|
|
4
4
|
import { Ledger, defaultLedgerFile } from "./ledger.js";
|
|
@@ -103,6 +103,7 @@ export class Router {
|
|
|
103
103
|
/** Create times this router measured, most recent last. */
|
|
104
104
|
live = new Map();
|
|
105
105
|
computerOnly;
|
|
106
|
+
quick;
|
|
106
107
|
/** The local record of sandboxes this router created, or undefined when turned off. */
|
|
107
108
|
ledger;
|
|
108
109
|
constructor(opts = {}) {
|
|
@@ -117,6 +118,7 @@ export class Router {
|
|
|
117
118
|
this.billed =
|
|
118
119
|
opts.billedVsList ?? Object.fromEntries(Object.entries(BILLED_VS_LIST).map(([id, b]) => [id, b.ratio]));
|
|
119
120
|
this.computerOnly = opts.computers ?? (opts.providers ? [] : defaultComputerProviders());
|
|
121
|
+
this.quick = opts.quickFallback === false ? false : { factor: 5, minSeconds: 10, ...opts.quickFallback };
|
|
120
122
|
}
|
|
121
123
|
/** Registered and has credentials. False for providers this router doesn't know. */
|
|
122
124
|
isConfigured(id) {
|
|
@@ -161,6 +163,9 @@ export class Router {
|
|
|
161
163
|
verified: this.verified.has(id),
|
|
162
164
|
failing: this.failing.has(id),
|
|
163
165
|
};
|
|
166
|
+
const fee = billingRules(id).perSandboxUsd;
|
|
167
|
+
if (fee)
|
|
168
|
+
c.perSandboxUsd = fee;
|
|
164
169
|
if (!p)
|
|
165
170
|
c.skip = "no adapter registered";
|
|
166
171
|
else {
|
|
@@ -176,7 +181,8 @@ export class Router {
|
|
|
176
181
|
// depends on who the provider is, only on its price, what it really billed, how fast it started for
|
|
177
182
|
// this router and whether its last real-account check failed. An explicit list keeps the caller's order.
|
|
178
183
|
if (!explicit) {
|
|
179
|
-
const
|
|
184
|
+
const hours = (spec.timeoutMinutes ?? 30) / 60;
|
|
185
|
+
const byCost = (a, b) => rankUsd(a, hours) - rankUsd(b, hours) || a.provider.localeCompare(b.provider);
|
|
180
186
|
const byList = (a, b) => a.listHourlyUsd - b.listHourlyUsd || a.provider.localeCompare(b.provider);
|
|
181
187
|
if (strategy === "fastest") {
|
|
182
188
|
list.sort((a, b) => (a.medianCreateMs ?? Infinity) - (b.medianCreateMs ?? Infinity) || byCost(a, b));
|
|
@@ -188,7 +194,7 @@ export class Router {
|
|
|
188
194
|
list.sort((a, b) => Number(b.verified) - Number(a.verified) || byCost(a, b));
|
|
189
195
|
}
|
|
190
196
|
else {
|
|
191
|
-
return balanced(list);
|
|
197
|
+
return balanced(list, hours);
|
|
192
198
|
}
|
|
193
199
|
}
|
|
194
200
|
return list;
|
|
@@ -233,22 +239,23 @@ export class Router {
|
|
|
233
239
|
}
|
|
234
240
|
const attempts = [];
|
|
235
241
|
const tries = route.fallback === false ? usable.slice(0, 1) : usable;
|
|
236
|
-
const limitS = route.createTimeoutSeconds ?? DEFAULT_CREATE_TIMEOUT_S;
|
|
237
242
|
const deadline = Date.now() + (route.deadlineSeconds ?? DEFAULT_CREATE_DEADLINE_S) * 1000;
|
|
238
|
-
for (const c of tries) {
|
|
243
|
+
for (const [i, c] of tries.entries()) {
|
|
239
244
|
const left = deadline - Date.now();
|
|
240
245
|
if (left <= 0) {
|
|
241
246
|
throw new RouterError(`create ran out of time (deadlineSeconds) after trying ${attempts.map((a) => a.provider).join(", ")}: ${attempts.map((a) => `${a.provider}: ${a.error}`).join(" | ")}`, attempts, "timeout");
|
|
242
247
|
}
|
|
243
248
|
const t0 = Date.now();
|
|
244
249
|
const pending = this.provider(c.provider).create(spec);
|
|
250
|
+
const { limitS, typicalMs } = this.waitFor(c.provider, route, i === tries.length - 1);
|
|
245
251
|
const waitS = Math.min(limitS, left / 1000);
|
|
246
252
|
const price = { estimatedHourlyUsd: c.estimatedHourlyUsd, listHourlyUsd: c.listHourlyUsd, size: c.size };
|
|
247
253
|
let timedOut = false;
|
|
248
254
|
try {
|
|
249
255
|
const box = await withDeadline(pending, waitS * 1000, () => {
|
|
250
256
|
timedOut = true;
|
|
251
|
-
|
|
257
|
+
const usual = typicalMs !== undefined && waitS < (route.createTimeoutSeconds ?? DEFAULT_CREATE_TIMEOUT_S) ? `; it usually starts in ${Number((typicalMs / 1000).toFixed(1))}s` : "";
|
|
258
|
+
return new ProviderError(c.provider, `create timed out after ${Number(waitS.toFixed(1))}s${usual}`);
|
|
252
259
|
});
|
|
253
260
|
attempts.push({ provider: c.provider, ok: true, ms: Date.now() - t0 });
|
|
254
261
|
this.observe(c.provider, Date.now() - t0);
|
|
@@ -273,6 +280,22 @@ export class Router {
|
|
|
273
280
|
}
|
|
274
281
|
throw new RouterError(`all providers failed: ${attempts.map((a) => `${a.provider}: ${a.error}`).join(" | ")}`, attempts, "all_failed");
|
|
275
282
|
}
|
|
283
|
+
/**
|
|
284
|
+
* Seconds to wait for this provider's create: the caller's `createTimeoutSeconds`, or (see
|
|
285
|
+
* `quickFallback`) a few times its usual start time when another provider is left to try.
|
|
286
|
+
*/
|
|
287
|
+
waitFor(id, route, last) {
|
|
288
|
+
const full = route.createTimeoutSeconds ?? DEFAULT_CREATE_TIMEOUT_S;
|
|
289
|
+
if (!this.quick || last || route.createTimeoutSeconds !== undefined)
|
|
290
|
+
return { limitS: full };
|
|
291
|
+
const live = this.live.get(id);
|
|
292
|
+
const samples = live && live.length >= LIVE_MIN_SAMPLES ? live : loadBench(this.benchFile)[id]?.createMs;
|
|
293
|
+
const typicalMs = median(samples);
|
|
294
|
+
if (typicalMs === undefined || !samples?.length)
|
|
295
|
+
return { limitS: full };
|
|
296
|
+
const s = Math.max(this.quick.factor * typicalMs, 1.5 * Math.max(...samples)) / 1000;
|
|
297
|
+
return { limitS: Math.min(full, Math.max(this.quick.minSeconds, s)), typicalMs };
|
|
298
|
+
}
|
|
276
299
|
/** Remember a successful create's time, for the speed tie-break and strategy "fastest". */
|
|
277
300
|
observe(id, ms) {
|
|
278
301
|
const xs = this.live.get(id) ?? [];
|
|
@@ -416,18 +439,20 @@ function round(n) {
|
|
|
416
439
|
* The default order. Providers whose last real-account check failed go last. The rest go cheapest
|
|
417
440
|
* first by effective price (list price x what the provider really billed, where measured), except that
|
|
418
441
|
* providers within PRICE_TIE of the cheapest remaining one count as a tie and go fastest first (by
|
|
419
|
-
* median create time; unknown speed after known), then cheapest, then alphabetical.
|
|
442
|
+
* median create time; unknown speed after known), then cheapest, then alphabetical. A fixed
|
|
443
|
+
* per-sandbox charge counts as spread over `lifetimeHours` (the sandbox's timeoutMinutes).
|
|
420
444
|
*/
|
|
421
|
-
export function balanced(list) {
|
|
422
|
-
const
|
|
445
|
+
export function balanced(list, lifetimeHours = 0.5) {
|
|
446
|
+
const cost = (c) => rankUsd(c, lifetimeHours);
|
|
447
|
+
const byCost = (a, b) => cost(a) - cost(b) || a.provider.localeCompare(b.provider);
|
|
423
448
|
const bySpeed = (a, b) => (a.medianCreateMs ?? Infinity) - (b.medianCreateMs ?? Infinity) || byCost(a, b);
|
|
424
449
|
const order = (group) => {
|
|
425
450
|
const rest = [...group].sort(byCost);
|
|
426
451
|
const out = [];
|
|
427
452
|
while (rest.length) {
|
|
428
|
-
const limit = rest[0]
|
|
453
|
+
const limit = cost(rest[0]) * (1 + PRICE_TIE);
|
|
429
454
|
// Always take at least the cheapest, so the loop ends whatever the prices are.
|
|
430
|
-
const tie = rest.filter((c, i) => i === 0 || c
|
|
455
|
+
const tie = rest.filter((c, i) => i === 0 || cost(c) <= limit);
|
|
431
456
|
out.push(...tie.sort(bySpeed));
|
|
432
457
|
rest.splice(0, tie.length);
|
|
433
458
|
}
|
|
@@ -435,6 +460,10 @@ export function balanced(list) {
|
|
|
435
460
|
};
|
|
436
461
|
return [...order(list.filter((c) => !c.failing)), ...order(list.filter((c) => c.failing))];
|
|
437
462
|
}
|
|
463
|
+
/** USD per hour a candidate is ranked by: its hourly estimate plus any per-sandbox charge spread over the lifetime. */
|
|
464
|
+
function rankUsd(c, lifetimeHours) {
|
|
465
|
+
return c.estimatedHourlyUsd + (c.perSandboxUsd ?? 0) / Math.max(lifetimeHours, 1 / 60);
|
|
466
|
+
}
|
|
438
467
|
/** Seconds to wait for one provider's create before trying the next (RouteOptions.createTimeoutSeconds). */
|
|
439
468
|
export const DEFAULT_CREATE_TIMEOUT_S = 120;
|
|
440
469
|
/** Seconds a whole create may take over every provider it tries (RouteOptions.deadlineSeconds). */
|
package/dist/verified.js
CHANGED
|
@@ -1,20 +1,20 @@
|
|
|
1
1
|
export const VERIFIED = {
|
|
2
|
-
agent37: "2026-11-
|
|
3
|
-
agentcore: "2026-11-
|
|
4
|
-
lambda: "2026-11-
|
|
5
|
-
blaxel: "2026-11-
|
|
6
|
-
cloudflare: "2026-11-
|
|
7
|
-
daytona: "2026-11-
|
|
8
|
-
e2b: "2026-11-
|
|
9
|
-
sprites: "2026-11-
|
|
10
|
-
freestyle: "2026-11-
|
|
11
|
-
maritime: "2026-11-
|
|
12
|
-
modal: "2026-11-
|
|
13
|
-
novita: "2026-11-
|
|
14
|
-
openscale: "2026-11-
|
|
15
|
-
sail: "2026-11-
|
|
16
|
-
superserve: "2026-11-
|
|
17
|
-
vercel: "2026-11-
|
|
2
|
+
agent37: "2026-11-13",
|
|
3
|
+
agentcore: "2026-11-13",
|
|
4
|
+
lambda: "2026-11-13",
|
|
5
|
+
blaxel: "2026-11-13",
|
|
6
|
+
cloudflare: "2026-11-13",
|
|
7
|
+
daytona: "2026-11-13",
|
|
8
|
+
e2b: "2026-11-13",
|
|
9
|
+
sprites: "2026-11-13",
|
|
10
|
+
freestyle: "2026-11-13",
|
|
11
|
+
maritime: "2026-11-13",
|
|
12
|
+
modal: "2026-11-13",
|
|
13
|
+
novita: "2026-11-13",
|
|
14
|
+
openscale: "2026-11-13",
|
|
15
|
+
sail: "2026-11-13",
|
|
16
|
+
superserve: "2026-11-13",
|
|
17
|
+
vercel: "2026-11-13",
|
|
18
18
|
};
|
|
19
19
|
/** Providers whose last saved real-account check failed a step. The default strategy tries them last. */
|
|
20
20
|
export const FAILING = {};
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sandboxrouter/sdk",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.8.0",
|
|
4
4
|
"description": "One API for every sandbox provider: route AI agents' code by price, speed or capability, with automatic fallback. Free and open source.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -175,7 +175,7 @@
|
|
|
175
175
|
"@aws-sdk/client-appstream": "^3.1162.0",
|
|
176
176
|
"@aws-sdk/credential-provider-node": "^3.972.93",
|
|
177
177
|
"@blaxel/core": "^0.3.34",
|
|
178
|
-
"@daytonaio/sdk": "^0.
|
|
178
|
+
"@daytonaio/sdk": "^0.230.0",
|
|
179
179
|
"@e2b/desktop": "^2.5.0",
|
|
180
180
|
"@fly/sprites": "^0.2.4",
|
|
181
181
|
"@hopx-ai/sdk": "^0.5.1",
|
|
@@ -201,5 +201,5 @@
|
|
|
201
201
|
"type": "git",
|
|
202
202
|
"url": "git+https://github.com/gigialc/sandboxrouter.git"
|
|
203
203
|
},
|
|
204
|
-
"homepage": "https://sandboxrouter.
|
|
204
|
+
"homepage": "https://sandboxrouter.ai"
|
|
205
205
|
}
|