@runblackcat/sdk 0.1.0-alpha.57 → 0.1.0-alpha.71

package/README.md

@@ -26,8 +26,8 @@ are in [docs/PRODUCT.md](docs/PRODUCT.md).
   rather than a connection.
 - **Queue and backpressure** — a request the fleet has no room for waits durably
   instead of failing, and is placed by a background drain as capacity arrives.
-  Queued work is charged against tenant quota exactly like running work, so the
-  quota is the backpressure limit and no second limiter exists.
+  Fleet capacity and the durable queue are the only backpressure; there are no
+  per-tenant quotas.
 - **Demand-driven capacity** — the placer sizes the fleet from the backlog it
   cannot place, through a provider seam, and retires idle workers back to zero.
 - **Forkable templates** — build a Docker image into a validated state, memory,
@@ -49,7 +49,7 @@ are in [docs/PRODUCT.md](docs/PRODUCT.md).
 | | |
 |---|---|
 | **Gateway** | Stateless. The single authenticated entry point; resolves sandbox→node and proxies. Scales horizontally. |
-| **Placer** | Single writer. Bin-packs by memory and vCPU, proposes placements, owns quotas, the queue and the node registry. Sole writer of substrate desired capacity. |
+| **Placer** | Single writer. Bin-packs by memory and vCPU, proposes placements, owns the queue and the node registry. Sole writer of substrate desired capacity. |
 | **Node agent** | One per host. Firecracker lifecycle, jailer, networking, UFFD handler. **Final authority on admission** — the placer proposes, the node accepts or rejects. |
 | **Guest agent** | Inside each sandbox, reached over vsock. Exec, file I/O, template-declared boot services. |
 | **Plugin supervisor (in development)** | Runs operator-installed, digest-pinned job-source plugins outside sandbox hosts. Each installation has its own identity, secret scope and sandbox grant. A small Rust SDK calls the scoped private API. |
@@ -71,7 +71,7 @@ flowchart TB
   subgraph ctrl["Control host"]
     direction LR
     GW["blackcat-gateway<br/>auth · capabilities · routing<br/>listens :443"]
-    PL["blackcat-placer<br/>single writer · best-fit<br/>quotas · node registry"]
+    PL["blackcat-placer<br/>single writer · best-fit<br/>queue · node registry"]
     DB[("placer.sqlite")]
     GW <-->|"Unix socket"| PL
     PL --- DB
@@ -158,8 +158,7 @@ construction. Everything else is a network hop.
 ## Capacity and observability
 
 Creation is admitted durably and answered immediately. A request the fleet cannot
-place becomes a **queued acquisition** — it keeps its sandbox identity and its
-quota charge, and a background drain places it as capacity appears. Callers poll
+place becomes a **queued acquisition** — it keeps its sandbox identity, and a background drain places it as capacity appears. Callers poll
 the claim they already hold:
 
 ```
@@ -169,14 +168,27 @@ GET  /v1/sandboxes/acquisitions/{id}       -> 202 pending … -> 200 {"state":"p
 POST /v1/sandboxes/{id}/renew {ttl_secs}   -> 200 {"expires_at":…, "renewable_until":…}
 ```
 
+To start many identical sandboxes, add `"count": N` (1 to 1000) to the create.
+`request_id` then names the batch and member `i` is `request_id.i`. The whole batch
+is recorded in one durable transaction and answered `202` at once, with no
+connection held through restores; a retry returns the same members.
+
+```
+POST /v1/sandboxes {request_id,template,count,lease:{…}}
+                                            -> 202 {"request_id":…, "count":N, "counts":{"pending":N}, "members":[…]}
+GET  /v1/sandboxes/batches/{request_id}    -> 202 while any member is pending, then 200
+GET  /v1/sandboxes/acquisitions/{id}.{i}   -> one member, as an ordinary claim
+```
+
 `request_id` is caller-supplied and is the idempotency boundary, so repeating a
 create is always safe: it returns the same claim rather than allocating a second
 sandbox.
 
-Backpressure uses separate tenant limits for running allocations and retained
-snapshots. Pending/resuming work reserves running capacity; hibernating and
-resuming transitions reserve both classes until they commit. A queued acquisition
-that waits past the queue deadline is torn down by the ordinary expiry reconciler.
+Backpressure is fleet capacity and the durable queue; there are no per-tenant
+quotas. A queued acquisition that waits past the queue deadline is torn down by
+the ordinary expiry reconciler. Request-rate limits are operator-set fleet state
+(`blackcat admin rate-limits get|set`): token mint and OIDC limits are on by
+default, every workload limit is unlimited until set.
 
 The placer sizes the fleet from that backlog through the `CapacityProvider`
 seam, and is the **only** writer of substrate desired capacity — it raises the
@@ -194,12 +206,53 @@ blackcat_demand_unsatisfied{tenant="…"}          queued acquisitions
 blackcat_demand_oldest_age_seconds               longest current wait
 blackcat_capacity_target_nodes                   workers last asked for
 blackcat_nodes{health="healthy"}                 workers actually registered
-blackcat_placement_rejected_total{reason="…"}    no_capacity | quota
+blackcat_placement_rejected_total{reason="…"}    no_capacity
 ```
 
 A fleet pinned at its ceiling shows up as `blackcat_capacity_target_nodes`
 persistently above `blackcat_nodes`.
 
+## Recommended production shape: bare metal
+
+Run workers on bare metal, one or a few large `.metal` hosts, and keep nested virtualization (a VM with `/dev/kvm`) as the quick-start
+default: it launches in under a minute and needs no special quota. On metal every guest page fault costs about 7.6 times less than
+nested, so the same fleet restores 2 to 3 times faster, fans out 30 times faster at 100 clones, costs a third as much per clone and
+does not lose VMs to nested VM-entry failures.
+
+| | bare metal (`c7i.metal-24xl`) | nested (`m8i.4xlarge`) |
+|---|---|---|
+| create p50 | 29 ms | 60 ms |
+| first exec | 33 ms | 67 ms |
+| snapshot restore | 13 ms | 43 ms |
+| fan-out of 100 clones | 0.6 s | 17.9 s |
+| cost per clone-hour | $0.0043 | $0.0132 |
+| VM-entry deaths | 0 | 2.9 per 1,000 hot restores, 325 per 1,000 at density |
+
+(Bake-off, 2 vCPU / 1 GiB app template, `contrib/bench/bakeoff/RESULTS.md`, "Huge pages". The hosts are different CPU
+generations and metal has 12 times the CPUs, so the fan-out column is not virtualization alone. Metal is cheaper per clone because it
+carries about 1,000 clones where the nested host was measured at 64.)
+
+North star on one `c7i.metal-48xl` (500 clones of a built-from-source web app at once, each running a short test then destroyed):
+99% done 112 s after the PR arrived, all 500 passed with no replacements and no VMM deaths, about $0.34 per PR from a stopped node,
+against 300 s, 264 replacements and about $1.00 on five `m8i.8xlarge`.
+
+Trade-offs, plainly:
+
+* **Boot.** A metal reboot has been quoted at about 7.5 minutes. Measured on a `c7i.metal-48xl`: a fresh launch was registered with SSM
+  in 74 s and a stopped node was serving again 47 s after `StartInstances`, but stopping it took 320 s (unbilled). Time from a
+  planner launch to an enrolled node on metal has not been measured (see
+  `contrib/bench/results/2026-10-04-wave-c-metal-northstar.md`). Nested hosts register in 40 to 45 s.
+* **Coarse sizes.** Metal is `.metal` sizes only: 96 or 192 vCPUs on the `c7i` family. A small or spiky fleet pays for a whole box
+  or runs nested.
+* **Thin Spot capacity.** Metal Spot pools are small; the planner falls back to on-demand when one refuses, and the price you plan
+  with is the cheaper Spot one.
+* **Large blast radius.** One host carries up to about a thousand clones. Losing it, or restarting its agent, hits all of them; keep
+  `fleet max-clones-per-host` as the cap you can accept and keep more than one host for anything that must survive a host loss.
+
+The capacity planner prefers metal when a reservation's `needed_by` allows its boot time and the price per clone is lower
+([docs/capacity-planning.md](docs/capacity-planning.md)); `blackcat_vm_entry_deaths` counts the VM-entry failures that make nested
+hosts unreliable at density. Setup: [docs/bare-mode.md](docs/bare-mode.md#recommended-production-shape-bare-metal).
+
 ## Install (alpha)
 
 Identical on bare metal and on AWS:
@@ -286,7 +339,8 @@ Linux x86-64/arm64, run `curl -fsSL https://runblackcat.dev/install.sh | sh`;
 see [CLI installation](docs/cli-install.md). The installer uses release
 checksums from the public
 [`blackcat-releases`](https://github.com/TreebeardHQ/blackcat-releases) mirror
-and needs neither Docker nor Rust on the client. The site itself lives in
+and needs neither Docker nor Rust on the client. To run a fleet on plain
+Debian or Ubuntu servers, see [Bare mode](docs/bare-mode.md). The site itself lives in
 [`site/`](site/) and publishes an allowlisted subset of `docs/`.
 `blackcat template list` shows launchable templates to members without their
 source; `blackcat sandbox claim alpine-e2e-matched` resolves the latest version

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@runblackcat/sdk",
-  "version": "0.1.0-alpha.57",
+  "version": "0.1.0-alpha.71",
   "description": "TypeScript SDK for the Blackcat sandbox gateway",
   "license": "Apache-2.0",
   "repository": "https://github.com/TreebeardHQ/blackcat",

package/sdk/dist/index.d.ts

@@ -21,6 +21,10 @@ export declare const CAPABILITY_OPS: {
     readonly RENEW: 4096;
 };
 export type CapabilityOps = number;
+export declare const BLACKCAT_ERROR_CODES: {
+    readonly SANDBOX_TRANSITIONING: "sandbox_transitioning";
+    readonly FLEET_UPGRADE_QUIESCING: "fleet_upgrade_quiescing";
+};
 export type FetchFunction = (input: RequestInfo | URL, init?: RequestInit & {
     tls?: {
         ca: string;
@@ -48,6 +52,17 @@ export interface TemplateDefaults {
     memory_mib: number;
     hibernate?: HibernateMode;
     on_interruption?: "evacuate" | "destroy";
+    /** MiB of virtio-mem region booted into the checkpoint; a multiple of 128. Defaults to the template's memory. */
+    hotplug_max_mib?: number;
+}
+/**
+ * The size a stored template version carries: `memory_mib` is the size it was
+ * captured at (boot memory plus what was plugged), and a sandbox can later
+ * take any size from `base_memory_mib` to `base_memory_mib + hotplug_max_mib`.
+ */
+export interface TemplateShapeDefaults extends TemplateDefaults {
+    base_memory_mib: number;
+    hotplug_max_mib: number;
 }
 export type HibernateMode = "memory";
 export type TemplateEgressPolicy = {
@@ -95,6 +110,9 @@ interface TemplateDockerSourceBase {
     defaults: TemplateDefaults;
     egress: TemplateEgressPolicy;
     restore_health: TemplateRestoreHealth;
+    /** Runs in the guest after it is ready and before the capture, so every
+     * clone restores with the app's caches and JIT already hot. Must exit 0. */
+    warmup?: TemplateRestoreHealth;
 }
 interface TemplatePreparation {
     parameters: Record<string, {
@@ -133,8 +151,11 @@ export interface UploadedImage {
     unchanged: boolean;
 }
 export interface TemplateSandboxSource {
+    /** `warmup` runs in the sandbox just before it is captured; omitted, the
+     * sandbox's template's own warm-up (if any) runs. */
     sandbox: {
         id: string;
+        warmup?: TemplateRestoreHealth;
     };
     /**
      * A promoted sandbox keeps the machine dimensions it was captured on, so
@@ -144,20 +165,33 @@ export interface TemplateSandboxSource {
     egress?: TemplateEgressPolicy;
 }
 export type TemplateSource = TemplateDockerSource | TemplateUploadedImageSource | TemplateSandboxSource;
+/** What a promotion adds to a template create: how it is warmed and packed. */
+export interface PromotionOptions {
+    /** Warm the new version onto this many more nodes in bulk, beyond the node
+     * that captured it (at most 99), or onto `"all"` the other healthy nodes.
+     * Nodes bound to a capacity reservation that follows the template are warmed
+     * regardless. */
+    prefetch_copies?: number | "all";
+    /** Millicores each clone of the new version reserves, at most 1000 per vCPU.
+     * Omitted, the version inherits what the sandbox's own template reserves. */
+    cpu_reservation_mcpu?: number;
+}
 export type CreateTemplateInput = {
     name: string;
     source: TemplateDockerSource;
 } | {
     name: string;
     source: TemplateUploadedImageSource;
-} | {
+} | ({
     name?: string;
     source: TemplateSandboxSource;
-};
+} & PromotionOptions);
 export interface Template {
     id: string;
     name: string;
     version: number;
+    /** `local_ready` while a promotion's chunks are still being published. */
+    state?: "local_ready" | "published";
     /** The gateway stores the complete source manifest as JSON text. */
     source: string;
     current: boolean;
@@ -187,27 +221,156 @@ export interface PrepareSandboxResponse {
 export interface CreateTemplateResponse {
     id: string;
     version: number;
+    /**
+     * `local_ready`: a promotion whose capture has been proved but whose chunks
+     * are still being published. The version is current and claimable, but only
+     * on the node that captured it; clones depend only on that node's files and
+     * are destroyed if the publication fails. `published`: every chunk is in the
+     * object store.
+     */
+    state?: "local_ready" | "published";
+    /**
+     * Capture/publish phase breakdown for a checkpoint promotion, when this
+     * create captured a running sandbox. Absent for a template created without
+     * a capture, and for a server that predates this field.
+     */
+    capture_timing?: {
+        capture_ms: number;
+        /** How long the source VM stood paused: the memory/state dump, the disk
+         * freeze and the resume. Sealing and upload happen after it. */
+        pause_ms: number;
+        /** Absent while the version is `local_ready`: publication is not over. */
+        publish_ms?: number;
+        /** Restoring a disposable clone from the capture and running the
+         * template's restore-health command in it; both overlap the upload. */
+        validation_restore_ms?: number;
+        validation_health_ms?: number;
+        /** Whether the capturing node was left holding the new version's memory
+         * and disk for fault-free restores. */
+        warmed?: {
+            memory: boolean;
+            disk: boolean;
+        };
+    };
+    /** Present for a promotion that asked for or achieved warming. */
+    warming?: {
+        capturing_node_warm: boolean;
+        prefetch_copies: number;
+        /** Pass to `templates.prefetchStatus` to follow the extra copies. */
+        prefetch_id?: string;
+        error?: string;
+    };
 }
-export type CreateSandboxInput = ({
-    template: string;
-    vcpus?: never;
-    memory_mib?: never;
-} | {
-    template: string;
-    vcpus: number;
-    memory_mib: number;
-}) & {
+export type CreateSandboxInput = {
+    /**
+     * The template name or version id. Omitted, it is the tenant's `default`
+     * template, which the gateway builds on the tenant's first use: that first
+     * create fails with a 503 `default_template_building` until the build is
+     * ready, and repeating the same request is how to wait for it.
+     */
+    template?: string;
+    /** Must equal the checkpoint's vCPUs, which are fixed at capture; omit it. */
+    vcpus?: number;
+    /**
+     * Guest memory in MiB: the checkpoint's captured size or larger, up to its
+     * boot memory plus hotplug region (the extra is plugged right after restore).
+     */
+    memory_mib?: number;
+    /** CPU entitlement in millicores, at most the checkpoint's vCPUs * 1000. */
+    cpu_mcpu?: number;
     /** Stable across retries of one acquisition; never reuse for another request. */
     request_id: string;
     ssh_public_key?: string;
     lease: SandboxLeaseInput;
+    /** A capacity reservation this create should draw a bound slot from. */
+    reservation?: string;
+    /**
+     * Ask for a capability scoped to each sandbox this request creates, returned
+     * in the response (`capability` on the receipt, or on each batch member), so
+     * a caller driving many sandboxes mints none of its own. `ops` must be within
+     * the creating token's; `ttl_secs` is capped at 900 and renewable like any
+     * token. Only the request that creates a sandbox gets its capability: a
+     * repeat of the same `request_id` returns none.
+     */
+    capability?: {
+        ops: string[] | CapabilityOps;
+        ttl_secs: number;
+    };
+};
+/** A capability scoped to one sandbox, as returned by a create. */
+export interface SandboxCapability {
+    token: string;
+    token_id: string;
+    ops: string[];
+    /** Epoch milliseconds. */
+    expires_at: number;
+    renewable_until: number;
+}
+/** A batch create: `request_id` names the batch, member `i` is `<request_id>.<i>`. */
+export type CreateSandboxBatchInput = CreateSandboxInput & {
+    /** How many identical sandboxes to create, 1 to 1000. */
+    count: number;
 };
+export type SandboxBatchMemberState = "pending" | "placed" | "failed" | "destroyed";
+export interface SandboxBatchMember {
+    /** Poll this member alone with `sandboxes.acquisition(request_id)`. */
+    request_id: string;
+    id: string;
+    state: SandboxBatchMemberState;
+    /** Present when the create asked for `capability`. */
+    capability?: SandboxCapability;
+}
+export interface SandboxBatch {
+    /** The batch's request id. */
+    request_id: string;
+    count: number;
+    /** Members in each state; a state with no members is absent. */
+    counts: Partial<Record<SandboxBatchMemberState, number>>;
+    members: SandboxBatchMember[];
+}
 export interface SandboxLeaseInput {
     /** Initial lease duration, between 60 seconds and 24 hours. */
     ttl_secs: number;
     /** Hard lifetime from acquisition, at least ttl_secs and at most 30 days. */
     renewable_for_secs: number;
 }
+export type CreateCapacityReservationInput = ({
+    /** One immutable template version, pinned for the reservation's life. */
+    template: string;
+    template_resource?: never;
+} | {
+    /** A template resource name: the reservation follows whichever version
+     * is current, even one that does not exist yet, and its bound nodes are
+     * warmed with each new version as it is activated. */
+    template_resource: string;
+    template?: never;
+}) & {
+    /** How many sandboxes of this template/shape to reserve. */
+    count: number;
+    vcpus: number;
+    memory_mib: number;
+    /** How long to hold the reservation, between 1 second and 4 hours. */
+    ttl_secs: number;
+};
+export interface CapacityReservation {
+    id: string;
+    state: "active" | "released" | "expired";
+    /** The version nodes are warmed with; null while a reservation following a
+     * template resource waits for the resource's first version. */
+    template: string | null;
+    /** The template resource this reservation follows, or null when pinned. */
+    template_resource: string | null;
+    count: number;
+    vcpus: number;
+    memory_mib: number;
+    /** Slots consumed by an actual sandbox create. */
+    claimed: number;
+    /** Slots bound to a specific node and not yet claimed. */
+    bound: number;
+    /** Bound slots whose node already has the template warm. */
+    warm: number;
+    expires_at: SystemTimeValue;
+}
 /**
  * Response returned by POST /v1/sandboxes (the gateway has no template field).
  * The gateway sends a `failed` receipt with HTTP 409, so `create` and
@@ -221,6 +384,14 @@ export type CreateSandboxResponse = {
     tier: "boot" | "warm_pool" | "restore";
     expires_at: SystemTimeValue;
     renewable_until: SystemTimeValue;
+    /**
+     * Phase-timing breakdown for the launch that placed this sandbox, once one
+     * has committed. Absent while still queued/pending, and for a server that
+     * predates this field.
+     */
+    restore_timing?: RestoreTiming;
+    /** Present when the create asked for `capability` and created this sandbox. */
+    capability?: SandboxCapability;
 } & ({
     state: "placed";
     node: string;
@@ -232,19 +403,96 @@ export type CreateSandboxResponse = {
     node: null;
     error: string;
 });
+/**
+ * Restore phase breakdown a node reported for a checkpoint launch (Proposal
+ * F, docs/restore-performance.md). `phase_ms` mirrors the node's own
+ * `checkpoint vm timing` log line; `chunk_source` reports how much of the
+ * checkpoint's memory/disk was fetched remotely vs. served locally, how many
+ * UFFD faults and NBD reads this restore actually served (both zero on the
+ * file-restore path), and the checkpoint's logical sizes. `memory_source` is
+ * `"file"` when a node-local materialized copy let this restore skip
+ * UFFD/NBD lazy faulting entirely (fault-free warm restore,
+ * docs/restore-performance.md), or `"uffd"` for the lazy chunk-source path
+ * every checkpoint restore used before it.
+ */
+export interface RestoreTiming {
+    proposal_ms: number;
+    restore_vm_ms: number;
+    commit_ms: number;
+    total_ms: number;
+    phase_ms: {
+        open: number;
+        network: number;
+        firecracker: number;
+        state: number;
+        disk: number;
+        pager: number;
+        load: number;
+        activation: number;
+        /** Milliseconds from the TAP being raised until the guest's gateway and resolver answered; absent from a node that predates it, 0 from a guest that does. */
+        network_ready?: number;
+    };
+    memory_source?: "file" | "uffd";
+    chunk_source: {
+        remote_requests: number;
+        remote_bytes: number;
+        local_requests?: number;
+        local_bytes?: number;
+        uffd_faults?: number;
+        uffd_fault_bytes?: number;
+        nbd_reads?: number;
+        nbd_read_bytes?: number;
+        logical_memory_bytes: number;
+        logical_disk_bytes: number;
+    };
+}
 export interface Sandbox {
     id: string;
     template: string;
     state: string;
     node?: string;
+    /** vCPUs the guest has now: the template's, until a cold resize reboots it onto another count. */
+    vcpus: number;
+    /** Current guest memory in MiB (boot memory plus what is plugged). */
+    memory_mib: number;
+    /** Current CPU entitlement in millicores. */
+    cpu_mcpu: number;
     expires_at: SystemTimeValue;
     renewable_until: SystemTimeValue;
 }
+/** A short-lived SSH key to authorize on a running sandbox. */
+export interface GrantSshKeyInput {
+    /** One `ssh-ed25519` public key line. */
+    public_key: string;
+    /** Lifetime in seconds, 1 to 86400. The key only matters while connecting. */
+    ttl_secs: number;
+}
+export interface SshKeyGrant {
+    /** Unix milliseconds at which the key stops authenticating. */
+    expires_at: number;
+    /** The login account in the sandbox. */
+    user: string;
+    /** The sandbox's SSH host public key; pin it before connecting. */
+    host_key: string;
+}
 export interface RenewSandboxResponse {
     id: string;
     expires_at: SystemTimeValue;
     renewable_until: SystemTimeValue;
 }
+/** The CPU a node runs on, as it reports it. */
+export interface CpuIdentity {
+    vendor: string;
+    model_name: string;
+    family: number;
+    model: number;
+    stepping: number;
+    microcode: string;
+    /** Hash of the CPUID feature leaves a guest's CPUID is derived from. */
+    features_hash: string;
+    /** The host is itself a guest (nested virtualization). */
+    nested: boolean;
+}
 /**
  * Tenant-scoped view returned by GET /v1/fleet. Residents and cached snapshots
  * are already filtered by the gateway to the caller's tenant.
@@ -262,6 +510,78 @@ export interface FleetNode {
     residents: string[];
     cached_snapshots: string[];
     interruption: FleetInterruption | null;
+    /** Charged versus measured memory and the node's pressure state. */
+    memory: FleetNodeMemory;
+    /** Whether the host runs with SMT active. */
+    smt_active: boolean;
+    /** The node's CPU; absent from nodes older than CPU-aware placement. */
+    cpu?: CpuIdentity | null;
+    /** 1000 charges file-backed clones in full; 2000 charges them at half. */
+    max_memory_ratio_milli: number;
+    max_vcpu_ratio_milli: number;
+    /** How the fleet decides what a sandbox costs. */
+    packing_mode?: "learned" | "manual";
+}
+export type MemoryPressure = "normal" | "high" | "critical";
+export interface FleetNodeMemory {
+    /** What admission charged for the node's residents. */
+    charged_mib: number;
+    /** The residents' sizes before the overcommit discount. */
+    plugged_mib: number;
+    /** Resident memory of the residents' VMMs, shared pages counted per mapper. */
+    measured_rss_mib: number;
+    /** Proportional memory the residents cost; compare with `charged_mib`. */
+    measured_pss_mib: number;
+    /** Memory only one resident maps (Private_Clean + Private_Dirty). */
+    measured_uss_mib: number;
+    host_available_mib: number;
+    psi_some_avg10_milli: number;
+    pressure: MemoryPressure;
+}
+export type FleetTrust = "single_trust" | "multi_tenant";
+export interface ResizeSandboxInput {
+    /**
+     * Target vCPU count. A count different from the sandbox's reboots the guest
+     * cold on its own disk: every process ends, template boot services restart,
+     * processes started by hand do not.
+     */
+    vcpus?: number;
+    /** Target guest memory in MiB: boot memory plus a multiple of 128. */
+    memory_mib?: number;
+    /** Target CPU entitlement in millicores. */
+    cpu_mcpu?: number;
+    /** Drop the guest's page caches before a shrink. Defaults to true. */
+    drop_caches?: boolean;
+    /** Stable across retries of one resize; generated when omitted. */
+    request_id?: string;
+}
+/** The size a resize reached. A shrink the guest could not fully honour reports what it reached. */
+export interface SandboxResize {
+    id: string;
+    state: "placed";
+    request_id: string;
+    /** True when the guest was rebooted to change its vCPU count. */
+    cold: boolean;
+    vcpus: number;
+    memory_mib: number;
+    cpu_mcpu: number;
+    /** False when the guest stopped short of the requested memory. */
+    reached_target: boolean;
+    elapsed_ms: number;
+    /** Cold resizes only: durable processes the reboot ended. */
+    processes_ended?: number;
+    /** Cold resizes only: where the reboot spent its time, in milliseconds. */
+    phases?: {
+        quiesce_ms: number;
+        stop_ms: number;
+        boot_ms: number;
+        plug_ms: number;
+        handshake_ms: number;
+        health_ms: number;
+    };
+    /** Cold resizes only: unexpired SSH grants installed again, and ones that failed. */
+    ssh_grants_restored?: number;
+    ssh_grants_failed?: number;
 }
 export interface FleetInterruption {
     reason: string;
@@ -269,6 +589,9 @@ export interface FleetInterruption {
 }
 export interface FleetResponse {
     nodes: FleetNode[];
+    fleet_trust: FleetTrust;
+    /** `native`, or a CPU template (`t2`, `t2s`, `t2cl`, `t2a`). */
+    cpu_baseline?: string;
 }
 export interface TemplateRefreshPolicy {
     interval_seconds: number;
@@ -395,6 +718,8 @@ export interface RenewTokenResponse {
 }
 export declare class BlackcatApiError extends Error {
     readonly status: number;
+    /** Stable gateway error code when one is provided. */
+    readonly code?: string;
     readonly retryAfter?: number;
     readonly requestId?: string;
     readonly details: unknown;
@@ -406,6 +731,7 @@ declare const MAX_PROCESS_STDIN: number;
 declare const MAX_EXEC_STDIN: number;
 export declare function epochMillis(value: unknown, field: string): number;
 export declare function parseCreateSandboxResponse(value: unknown, requestId: string): CreateSandboxResponse;
+export declare function parseSandboxBatch(value: unknown, requestId: string): SandboxBatch;
 export declare function assertSecureBlackcatGatewayUrl(baseUrl: string): void;
 declare class GatewayClient {
     readonly baseUrl: string;
@@ -476,6 +802,16 @@ declare class FleetResource {
     /** List the nodes and this tenant's residents/snapshots visible to INSPECT. */
     list(): Promise<FleetResponse>;
 }
+declare class ReservationsResource {
+    private readonly client;
+    constructor(client: GatewayClient);
+    /** Reserve capacity for a burst of sandboxes of one template/shape. */
+    create(input: CreateCapacityReservationInput): Promise<CapacityReservation>;
+    get(id: string): Promise<CapacityReservation>;
+    /** Release early. Unclaimed slots free immediately; bound nodes then fall
+     * back to the normal empty-worker scale-in grace period. */
+    release(id: string): Promise<CapacityReservation>;
+}
 declare class SandboxesResource {
     private readonly client;
     constructor(client: GatewayClient);
@@ -487,10 +823,25 @@ declare class SandboxesResource {
      * no automatic retry, new ID, or replacement allocation is synthesized here.
      */
     create(input: CreateSandboxInput): Promise<CreateSandboxResponse>;
+    /**
+     * Create `count` identical sandboxes under one request ID. Resolves as soon
+     * as the whole batch is durably recorded; nothing waits on a restore.
+     * Repeat the exact request to recover a lost response: member IDs are
+     * stable. Poll `batch` for counts by state, or a member with `acquisition`.
+     */
+    createBatch(input: CreateSandboxBatchInput): Promise<SandboxBatch>;
+    /** Counts by state and every member of a batch created with `createBatch`. */
+    batch(requestId: string): Promise<SandboxBatch>;
     acquisition(requestId: string): Promise<CreateSandboxResponse>;
     get(id: string): Promise<Sandbox>;
     /** Extend the soft expiry without moving the immutable hard deadline. */
     renew(id: string, ttl_secs: number): Promise<RenewSandboxResponse>;
+    /**
+     * Authorize `public_key` on a running sandbox for `ttl_secs`, whether or not
+     * it was created with a key. Needs the sandbox-scoped exec capability. Connect
+     * with `CONNECT /v1/sandboxes/{id}/ssh`, or let `blackcat sandbox ssh` do it.
+     */
+    grantSshKey(id: string, input: GrantSshKeyInput): Promise<SshKeyGrant>;
     exec(id: string, input: ExecInput): Promise<ExecResult>;
     execStream(id: string, input: ExecInput): AsyncGenerator<ExecStreamChunk>;
     startProcess(id: string, requestId: string, input: ProcessStartInput): Promise<ProcessRecord>;
@@ -528,10 +879,32 @@ declare class SandboxesResource {
     snapshot(id: string): Promise<{
         template: string;
     }>;
+    /**
+     * Pause a running sandbox's VM in memory: processes, memory and network
+     * state are kept, nothing is captured, and `resume` continues it where it
+     * stopped. Memory stays resident and charged (`hibernate` frees it); the CPU
+     * entitlement is released. Idempotent. Requests that need the guest are
+     * refused while it is paused.
+     */
+    pause(id: string): Promise<void>;
     hibernate(id: string, input?: {
         mode: HibernateMode;
     }): Promise<void>;
+    /**
+     * Change a running sandbox's memory (live, within its checkpoint's hotplug
+     * region) and CPU entitlement; processes keep running. A `vcpus` different
+     * from the sandbox's is a cold reboot instead (`cold: true`). Repeating the
+     * same `request_id` replays the result.
+     */
+    resize(id: string, input: ResizeSandboxInput): Promise<SandboxResize>;
+    /** Continue a paused sandbox, or restore a hibernated one. */
     resume(id: string): Promise<void>;
+    /**
+     * Destroy a sandbox. Resolves when its VM is dead and the node has released
+     * its capacity, not when the node has finished removing its network device,
+     * jail and cgroup: that follows in the background, usually within a second.
+     * Destroying a sandbox that is already destroyed succeeds.
+     */
     destroy(id: string): Promise<void>;
 }
 export declare class BlackcatClient extends GatewayClient {
@@ -541,6 +914,7 @@ export declare class BlackcatClient extends GatewayClient {
     readonly sandboxes: SandboxesResource;
     readonly tokens: TokensResource;
     readonly fleet: FleetResource;
+    readonly reservations: ReservationsResource;
     constructor(options: BlackcatClientOptions);
     /**
      * Create a client that uses a sandbox capability for resource operations.

package/sdk/dist/index.js

@@ -14,9 +14,17 @@ var CAPABILITY_OPS = {
   PROCESS_CANCEL: 2048,
   RENEW: 4096
 };
+var BLACKCAT_ERROR_CODES = {
+  SANDBOX_TRANSITIONING: "sandbox_transitioning",
+  FLEET_UPGRADE_QUIESCING: "fleet_upgrade_quiescing"
+};
+function withTemplate(input) {
+  return { ...input, template: input.template ?? "default" };
+}
 
 class BlackcatApiError extends Error {
   status;
+  code;
   retryAfter;
   requestId;
   details;
@@ -25,6 +33,11 @@ class BlackcatApiError extends Error {
     this.name = "BlackcatApiError";
     this.status = status;
     this.details = details;
+    if (typeof details === "object" && details !== null && !Array.isArray(details)) {
+      const code = details.code;
+      if (typeof code === "string")
+        this.code = code;
+    }
     this.retryAfter = retryAfter;
     this.requestId = typeof requestId === "string" && requestId.length === 32 && /^[a-f0-9]{32}$/i.test(requestId) ? requestId : undefined;
   }
@@ -83,6 +96,20 @@ function parseCreateSandboxResponse(value, requestId) {
   }
   return value;
 }
+var BATCH_STATES = ["pending", "placed", "failed", "destroyed"];
+function parseSandboxBatch(value, requestId) {
+  const malformed = () => new BlackcatProtocolError("sandbox batch is malformed or belongs to another request");
+  if (typeof value !== "object" || value === null || Array.isArray(value))
+    throw malformed();
+  const row = value;
+  if (row.request_id !== requestId || typeof row.count !== "number" || typeof row.counts !== "object" || row.counts === null || !Array.isArray(row.members) || row.members.length !== row.count || !row.members.every((member) => {
+    const m = member;
+    return typeof m === "object" && m !== null && typeof m.request_id === "string" && typeof m.id === "string" && BATCH_STATES.includes(String(m.state));
+  })) {
+    throw malformed();
+  }
+  return value;
+}
 function parseMintTokenResponse(value) {
   if (typeof value !== "object" || value === null || Array.isArray(value)) {
     throw new BlackcatProtocolError("token mint response is not an object");
@@ -395,6 +422,22 @@ class FleetResource {
   }
 }
 
+class ReservationsResource {
+  client;
+  constructor(client) {
+    this.client = client;
+  }
+  create(input) {
+    return this.client.json("POST", "/v1/capacity-reservations", input, "token");
+  }
+  get(id) {
+    return this.client.json("GET", `/v1/capacity-reservations/${pathSegment(id)}`, undefined, "token");
+  }
+  release(id) {
+    return this.client.json("DELETE", `/v1/capacity-reservations/${pathSegment(id)}`, undefined, "token");
+  }
+}
+
 class SandboxesResource {
   client;
   constructor(client) {
@@ -404,7 +447,13 @@ class SandboxesResource {
     return this.client.json("GET", "/v1/sandboxes", undefined, "token");
   }
   async create(input) {
-    return parseCreateSandboxResponse(await this.client.json("POST", "/v1/sandboxes", input, "token", false), input.request_id);
+    return parseCreateSandboxResponse(await this.client.json("POST", "/v1/sandboxes", withTemplate(input), "token", false), input.request_id);
+  }
+  async createBatch(input) {
+    return parseSandboxBatch(await this.client.json("POST", "/v1/sandboxes", withTemplate(input), "token", false), input.request_id);
+  }
+  async batch(requestId) {
+    return parseSandboxBatch(await this.client.json("GET", `/v1/sandboxes/batches/${pathSegment(requestId)}`, undefined, "token"), requestId);
   }
   async acquisition(requestId) {
     return parseCreateSandboxResponse(await this.client.json("GET", `/v1/sandboxes/acquisitions/${pathSegment(requestId)}`, undefined, "token"), requestId);
@@ -415,6 +464,9 @@ class SandboxesResource {
   renew(id, ttl_secs) {
     return this.client.json("POST", `/v1/sandboxes/${pathSegment(id)}/renew`, { ttl_secs }, "token");
   }
+  grantSshKey(id, input) {
+    return this.client.json("POST", `/v1/sandboxes/${pathSegment(id)}/ssh-keys`, input, "token");
+  }
   async exec(id, input) {
     return this.client.json("POST", `/v1/sandboxes/${pathSegment(id)}/exec`, input, "token", false, execDeadline(input)).then(execResult);
   }
@@ -596,11 +648,19 @@ class SandboxesResource {
   snapshot(id) {
     return this.client.json("POST", `/v1/sandboxes/${pathSegment(id)}/snapshot`, undefined, "token");
   }
+  pause(id) {
+    return this.client.raw("POST", `/v1/sandboxes/${pathSegment(id)}/pause`, undefined, "token").then(() => {
+      return;
+    });
+  }
   hibernate(id, input) {
     return this.client.raw("POST", `/v1/sandboxes/${pathSegment(id)}/hibernate`, input === undefined ? undefined : JSON.stringify(input), "token", input === undefined ? undefined : "application/json", false).then(() => {
       return;
     });
   }
+  resize(id, input) {
+    return this.client.json("POST", `/v1/sandboxes/${pathSegment(id)}/resize`, { ...input, request_id: input.request_id ?? `resize-${crypto.randomUUID()}` }, "token", false);
+  }
   resume(id) {
     return this.client.raw("POST", `/v1/sandboxes/${pathSegment(id)}/resume`, undefined, "token").then(() => {
       return;
@@ -618,6 +678,7 @@ class BlackcatClient extends GatewayClient {
   sandboxes;
   tokens;
   fleet;
+  reservations;
   constructor(options) {
     if (options.apiKey !== undefined)
       assertSecureBlackcatGatewayUrl(options.baseUrl);
@@ -628,6 +689,7 @@ class BlackcatClient extends GatewayClient {
     this.sandboxes = new SandboxesResource(this);
     this.tokens = new TokensResource(this);
     this.fleet = new FleetResource(this);
+    this.reservations = new ReservationsResource(this);
   }
   withToken(token, renewCapability) {
     const capability = typeof token === "string" ? undefined : token;
@@ -644,6 +706,7 @@ class BlackcatClient extends GatewayClient {
   }
 }
 export {
+  parseSandboxBatch,
   parseCreateSandboxResponse,
   epochMillis,
   assertSecureBlackcatGatewayUrl,
@@ -654,5 +717,6 @@ export {
   MAX_EXEC_STDIN,
   CAPABILITY_OPS,
   BlackcatClient,
-  BlackcatApiError
+  BlackcatApiError,
+  BLACKCAT_ERROR_CODES
 };