@runblackcat/sdk 0.1.0-alpha.100
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +35 -0
- package/README.md +406 -0
- package/package.json +29 -0
- package/sdk/dist/index.d.ts +1020 -0
- package/sdk/dist/index.js +751 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
Blackcat is dual-licensed by component.
|
|
2
|
+
|
|
3
|
+
SSPL-1.0 (Server Side Public License, version 1) — see LICENSE-SSPL
|
|
4
|
+
The server: everything that runs a fleet.
|
|
5
|
+
crates/blackcat-gateway
|
|
6
|
+
crates/blackcat-placer
|
|
7
|
+
crates/blackcat-agent
|
|
8
|
+
crates/blackcat-guest
|
|
9
|
+
crates/blackcat-local
|
|
10
|
+
crates/blackcat-aws
|
|
11
|
+
crates/blackcat-s3
|
|
12
|
+
crates/blackcat-plugin-supervisor
|
|
13
|
+
crates/blackcat-plugin-egress
|
|
14
|
+
crates/blackcat-github-actions
|
|
15
|
+
crates/blackcat-template-refresh
|
|
16
|
+
|
|
17
|
+
Apache-2.0 — see LICENSE-APACHE
|
|
18
|
+
The clients and shared libraries, and everything not listed above.
|
|
19
|
+
crates/blackcat-cli (source)
|
|
20
|
+
crates/blackcat-core
|
|
21
|
+
crates/blackcat-mcp
|
|
22
|
+
crates/blackcat-plugin
|
|
23
|
+
crates/blackcat-plugin-sdk
|
|
24
|
+
sdk/ (the TypeScript SDK, @runblackcat/sdk)
|
|
25
|
+
the conformance suite, scripts, docs and examples
|
|
26
|
+
|
|
27
|
+
Each crate's Cargo.toml states its license (SPDX: SSPL-1.0 or Apache-2.0).
|
|
28
|
+
|
|
29
|
+
Binaries: the `blackcat` CLI binary also embeds SSPL server crates (the local
|
|
30
|
+
fleet, node CA and maintenance executor link blackcat-gateway, blackcat-placer,
|
|
31
|
+
blackcat-guest, blackcat-local and blackcat-s3), so the distributed CLI binary
|
|
32
|
+
as a whole is subject to SSPL-1.0. The TypeScript SDK is Apache-2.0 throughout.
|
|
33
|
+
|
|
34
|
+
Releases up to and including v0.1.0-alpha.81 were published under Apache-2.0
|
|
35
|
+
and remain available under it.
|
package/README.md
ADDED
|
@@ -0,0 +1,406 @@
|
|
|
1
|
+
# blackcat
|
|
2
|
+
|
|
3
|
+
A self-hosted [Firecracker](https://firecracker-microvm.github.io/) microVM fleet:
|
|
4
|
+
multi-tenant sandboxes with snapshots, resumable workloads, and scale-to-zero.
|
|
5
|
+
|
|
6
|
+
**Agnostic core, paved AWS path.** The core knows about Linux, not about any cloud. AWS is
|
|
7
|
+
the one deployment path that comes fully assembled; everything else is possible with work.
|
|
8
|
+
|
|
9
|
+
## Status: hosted development, pre-release
|
|
10
|
+
|
|
11
|
+
The gateway, placer, node agent, guest, forkable checkpoint templates, durable
|
|
12
|
+
processes, previews and the TypeScript SDK are implemented, and a hot claim from
|
|
13
|
+
a checkpoint template restores in about 1.1 s on the development fleet. Nothing
|
|
14
|
+
is shipped to customers or released as open source.
|
|
15
|
+
|
|
16
|
+
What is missing before untrusted tenants can share workers is written down in
|
|
17
|
+
[docs/SECURITY.md](docs/SECURITY.md); the ranked backlog is
|
|
18
|
+
[docs/GROOMED.md](docs/GROOMED.md). Readiness levels and the product contract
|
|
19
|
+
are in [docs/PRODUCT.md](docs/PRODUCT.md).
|
|
20
|
+
|
|
21
|
+
## What it will do
|
|
22
|
+
|
|
23
|
+
- **Sandbox API** — acquire an isolated microVM with retry-safe request identity
|
|
24
|
+
and a pinned immutable template version. Creation never blocks: the
|
|
25
|
+
request is admitted durably and answered at once, so a caller holds a claim
|
|
26
|
+
rather than a connection.
|
|
27
|
+
- **Queue and backpressure** — a request the fleet has no room for waits durably
|
|
28
|
+
instead of failing, and is placed by a background drain as capacity arrives.
|
|
29
|
+
Fleet capacity and the durable queue are the only backpressure; there are no
|
|
30
|
+
per-tenant quotas.
|
|
31
|
+
- **Demand-driven capacity** — the placer sizes the fleet from the backlog it
|
|
32
|
+
cannot place, through a provider seam, and retires idle workers back to zero.
|
|
33
|
+
- **Forkable templates** — build a Docker image into a validated state, memory,
|
|
34
|
+
and disk checkpoint, then restore isolated writable clones with its declared
|
|
35
|
+
services already running. Sealed content-addressed chunks deduplicate within a
|
|
36
|
+
tenant and are fetched lazily into a bounded node cache.
|
|
37
|
+
- **Snapshot / resume** — UFFD lazy paging, so a resumed sandbox pages memory in on demand
|
|
38
|
+
instead of waiting for a full restore.
|
|
39
|
+
- **Scale to zero** — idle sandboxes snapshot to disk and release their RAM.
|
|
40
|
+
The next request resumes them when capacity is available.
|
|
41
|
+
- **HTTP previews** — templates declare named ports; each sandbox gets its own
|
|
42
|
+
HTTPS hostname, with share-link or public access and optional custom domains.
|
|
43
|
+
See [HTTP previews and custom domains](docs/preview-domains.md).
|
|
44
|
+
- **Multi-tenant** — tenants share hosts, isolated at the VM boundary by the Firecracker
|
|
45
|
+
jailer, seccomp, and explicit VM-to-VM network denial.
|
|
46
|
+
|
|
47
|
+
## Design in one page
|
|
48
|
+
|
|
49
|
+
| | |
|
|
50
|
+
|---|---|
|
|
51
|
+
| **Gateway** | Stateless. The single authenticated entry point; resolves sandbox→node and proxies. Scales horizontally. |
|
|
52
|
+
| **Placer** | Single writer. Bin-packs by memory and vCPU, proposes placements, owns the queue and the node registry. Sole writer of substrate desired capacity. |
|
|
53
|
+
| **Node agent** | One per host. Firecracker lifecycle, jailer, networking, UFFD handler. **Final authority on admission** — the placer proposes, the node accepts or rejects. |
|
|
54
|
+
| **Guest agent** | Inside each sandbox, reached over vsock. Exec, file I/O, template-declared boot services. |
|
|
55
|
+
| **Plugin supervisor (in development)** | Runs operator-installed, digest-pinned job-source plugins outside sandbox hosts. Each installation has its own identity, secret scope and sandbox grant. A small Rust SDK calls the scoped private API. |
|
|
56
|
+
|
|
57
|
+
Nodes **self-register** with a join token — there is no host inventory file anywhere.
|
|
58
|
+
Tenants are **API-managed rows**, not configuration, so an operator can provision
|
|
59
|
+
and use one without restarting the service.
|
|
60
|
+
|
|
61
|
+
No ZooKeeper, etcd, or Consul at any scale. Placement is sticky and sandboxes never move
|
|
62
|
+
while running, so the problem partitions without consensus.
|
|
63
|
+
|
|
64
|
+
## How it fits together
|
|
65
|
+
|
|
66
|
+
```mermaid
|
|
67
|
+
flowchart TB
|
|
68
|
+
CLI["blackcat-cli"]
|
|
69
|
+
SDK["TypeScript SDK"]
|
|
70
|
+
|
|
71
|
+
subgraph ctrl["Control host"]
|
|
72
|
+
direction LR
|
|
73
|
+
GW["blackcat-gateway<br/>auth · capabilities · routing<br/>listens :443"]
|
|
74
|
+
PL["blackcat-placer<br/>single writer · best-fit<br/>queue · node registry"]
|
|
75
|
+
DB[("placer.sqlite")]
|
|
76
|
+
GW <-->|"Unix socket"| PL
|
|
77
|
+
PL --- DB
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
subgraph plugins["Plugin host — planned integration"]
|
|
81
|
+
SUP["Plugin supervisor<br/>digest-pinned images · per-install identity"]
|
|
82
|
+
subgraph PIN["Per-install isolated bridge — no direct route"]
|
|
83
|
+
GHPL["GitHub Actions connector<br/>poll queued jobs · request JIT runners"]
|
|
84
|
+
CPX["Connector proxy<br/>GitHub DNS rules · pinned control target"]
|
|
85
|
+
GHPL -.->|"HTTPS CONNECT only"| CPX
|
|
86
|
+
end
|
|
87
|
+
EXT["Dedicated outbound bridge<br/>proxy only"]
|
|
88
|
+
RPX["Runner proxy<br/>GitHub DNS rules · no control target"]
|
|
89
|
+
SUP --> GHPL
|
|
90
|
+
SUP --> CPX
|
|
91
|
+
CPX -.-> EXT
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
SEC[("Plugin secret store<br/>local encrypted or AWS Secrets Manager")]
|
|
95
|
+
GITHUB["GitHub Actions<br/>job queue · results"]
|
|
96
|
+
|
|
97
|
+
OS[("Opaque object store<br/>S3 or filesystem")]
|
|
98
|
+
|
|
99
|
+
subgraph nodehost["Node host — one per machine, N machines"]
|
|
100
|
+
AG["blackcat-agent<br/>listens :8081"]
|
|
101
|
+
UF["UFFD handler<br/>one process per restore"]
|
|
102
|
+
FC["Firecracker + jailer"]
|
|
103
|
+
GU["blackcat-guest<br/>PID 1 inside the VM"]
|
|
104
|
+
RUN["Ephemeral Actions runner<br/>nonroot guest process"]
|
|
105
|
+
AG -->|spawns| UF
|
|
106
|
+
AG --> FC
|
|
107
|
+
UF -.->|"page faults"| FC
|
|
108
|
+
FC -->|vsock| GU
|
|
109
|
+
GU -->|"nonroot boot service"| RUN
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
CLI -->|HTTPS| GW
|
|
113
|
+
SDK -->|HTTPS| GW
|
|
114
|
+
CLI -.->|"admin plugin API"| GW
|
|
115
|
+
PL -.->|"installation state"| SUP
|
|
116
|
+
SUP -.->|"scoped Blackcat API"| GW
|
|
117
|
+
SUP -.->|"per-install secret refs"| SEC
|
|
118
|
+
EXT -.->|"GitHub API via proxy"| GITHUB
|
|
119
|
+
EXT -.->|"pinned private API via proxy"| GW
|
|
120
|
+
RPX -.->|"GitHub runner traffic"| GITHUB
|
|
121
|
+
RUN -.->|"one job + result"| RPX
|
|
122
|
+
GW ---|"metadata + grants"| OS
|
|
123
|
+
AG <-->|"short-lived object grants"| OS
|
|
124
|
+
GW -->|"node TLS"| AG
|
|
125
|
+
PL -->|"node TLS · propose"| AG
|
|
126
|
+
AG -->|"HTTPS · register, heartbeat, objects"| GW
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Dashed plugin links show the intended integration, not a deployed runner. The
|
|
130
|
+
connector observes external jobs and asks Blackcat for a sandbox; the external
|
|
131
|
+
source remains the job and result authority. Runner code executes only inside a
|
|
132
|
+
sandbox, without the connector's private key or plugin service credential. See
|
|
133
|
+
the [plugin system contract](docs/plugin-system.md) for the current boundary.
|
|
134
|
+
|
|
135
|
+
**Creating a sandbox.** The gateway checks the caller's capability token and opens a
|
|
136
|
+
durable acquisition receipt keyed by `(tenant, request_id)`, so a retry is the same
|
|
137
|
+
lease rather than a second VM. The placer picks candidate nodes by best fit over free
|
|
138
|
+
memory and vCPU — snapshot-cache affinity first — and **proposes** to each in turn. The
|
|
139
|
+
node accepts or rejects; it is the final admission authority, not the placer. On
|
|
140
|
+
acceptance the agent starts Firecracker under the jailer, and `blackcat-guest` boots as
|
|
141
|
+
PID 1 and serves the vsock protocol. Until the sandbox reports `placed`, the API answers
|
|
142
|
+
`202 pending` and it is not ready for commands.
|
|
143
|
+
|
|
144
|
+
Three things the picture above is drawn carefully to show:
|
|
145
|
+
|
|
146
|
+
- **The placer talks to nodes itself.** It is not hidden behind the gateway — it opens
|
|
147
|
+
its own node-TLS connections to propose placements.
|
|
148
|
+
- **Nodes receive only object-scoped grants.** The control plane authorizes a
|
|
149
|
+
specific artifact operation and the node transfers sealed bytes directly.
|
|
150
|
+
Nodes never receive a general object-store or tenant credential.
|
|
151
|
+
- **There are two TLS trust domains**, not one. Clients and agents reach the gateway
|
|
152
|
+
under the control-plane CA; the gateway and placer reach node agents under a separate
|
|
153
|
+
node CA issuing 24-hour leaves.
|
|
154
|
+
|
|
155
|
+
Gateway and placer speak over a Unix socket, so they are co-located on one host by
|
|
156
|
+
construction. Everything else is a network hop.
|
|
157
|
+
|
|
158
|
+
## Capacity and observability
|
|
159
|
+
|
|
160
|
+
Creation is admitted durably and answered immediately. A request the fleet cannot
|
|
161
|
+
place becomes a **queued acquisition** — it keeps its sandbox identity, and a background drain places it as capacity appears. Callers poll
|
|
162
|
+
the claim they already hold:
|
|
163
|
+
|
|
164
|
+
```
|
|
165
|
+
POST /v1/sandboxes {request_id,template,lease:{ttl_secs,renewable_for_secs}}
|
|
166
|
+
-> 202 {"request_id":…, "id":…, "state":"pending", "expires_at":…}
|
|
167
|
+
GET /v1/sandboxes/acquisitions/{id} -> 202 pending … -> 200 {"state":"placed","node":…}
|
|
168
|
+
POST /v1/sandboxes/{id}/renew {ttl_secs} -> 200 {"expires_at":…, "renewable_until":…}
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
Add `?wait_secs=N` (held at most 55 s) to the create or the lookup to have the gateway
|
|
172
|
+
hold a still-pending claim and answer as soon as it is placed or fails; the
|
|
173
|
+
placed receipt carries `ports`, so there is nothing to poll and no second call.
|
|
174
|
+
A claim still pending when the hold ends is answered `pending`; ask again. It
|
|
175
|
+
does not apply to a `count` batch: wait on each member by `request_id`.
|
|
176
|
+
|
|
177
|
+
To start many identical sandboxes, add `"count": N` (1 to 1000) to the create.
|
|
178
|
+
`request_id` then names the batch and member `i` is `request_id.i`. The whole batch
|
|
179
|
+
is recorded in one durable transaction and answered `202` at once, with no
|
|
180
|
+
connection held through restores; a retry returns the same members.
|
|
181
|
+
|
|
182
|
+
```
|
|
183
|
+
POST /v1/sandboxes {request_id,template,count,lease:{…}}
|
|
184
|
+
-> 202 {"request_id":…, "count":N, "counts":{"pending":N}, "members":[…]}
|
|
185
|
+
GET /v1/sandboxes/batches/{request_id} -> 202 while any member is pending, then 200
|
|
186
|
+
GET /v1/sandboxes/acquisitions/{id}.{i} -> one member, as an ordinary claim
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
`request_id` is caller-supplied and is the idempotency boundary, so repeating a
|
|
190
|
+
create is always safe: it returns the same claim rather than allocating a second
|
|
191
|
+
sandbox.
|
|
192
|
+
|
|
193
|
+
Backpressure is fleet capacity and the durable queue; there are no per-tenant
|
|
194
|
+
quotas. A queued acquisition that waits past the queue deadline is torn down by
|
|
195
|
+
the ordinary expiry reconciler. Request-rate limits are operator-set fleet state
|
|
196
|
+
(`blackcat admin rate-limits get|set`): session token-mint and OIDC limits are on by
|
|
197
|
+
default, every workload limit is unlimited until set.
|
|
198
|
+
|
|
199
|
+
The placer sizes the fleet from that backlog through the `CapacityProvider`
|
|
200
|
+
seam, and is the **only** writer of substrate desired capacity — it raises the
|
|
201
|
+
count from unplaceable demand and lowers it only by retiring workers it has
|
|
202
|
+
proven empty. An external autoscaling policy driving the same knob would fight
|
|
203
|
+
it, so deployments should cap the fleet (a maximum size) rather than scale it.
|
|
204
|
+
|
|
205
|
+
`GET /metrics` exposes Prometheus text to the **admin credential**. The series
|
|
206
|
+
are fleet-wide and keyed by tenant, so unlike `/v1/fleet` — which serves an
|
|
207
|
+
`INSPECT` holder its own rows only — there is no tenant-scoped view to hand out.
|
|
208
|
+
`/livez` and `/readyz` stay unauthenticated because they disclose nothing.
|
|
209
|
+
|
|
210
|
+
```
|
|
211
|
+
blackcat_demand_unsatisfied{tenant="…"} queued acquisitions
|
|
212
|
+
blackcat_demand_oldest_age_seconds longest current wait
|
|
213
|
+
blackcat_capacity_target_nodes workers last asked for
|
|
214
|
+
blackcat_nodes{health="healthy"} workers actually registered
|
|
215
|
+
blackcat_placement_rejected_total{reason="…"} no_capacity
|
|
216
|
+
blackcat_placements_total{host="…"} kept (hosts scale-in keeps), elastic
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
A fleet pinned at its ceiling shows up as `blackcat_capacity_target_nodes`
|
|
220
|
+
persistently above `blackcat_nodes`.
|
|
221
|
+
|
|
222
|
+
## Recommended production shape: bare metal
|
|
223
|
+
|
|
224
|
+
Run workers on bare metal, one or a few large `.metal` hosts, and keep nested virtualization (a VM with `/dev/kvm`) as the quick-start
|
|
225
|
+
default: it launches in under a minute and needs no special quota. On metal every guest page fault costs about 7.6 times less than
|
|
226
|
+
nested, so the same fleet restores 2 to 3 times faster, fans out 30 times faster at 100 clones, costs a third as much per clone and
|
|
227
|
+
does not lose VMs to nested VM-entry failures.
|
|
228
|
+
|
|
229
|
+
| | bare metal (`c7i.metal-24xl`) | nested (`m8i.4xlarge`) |
|
|
230
|
+
|---|---|---|
|
|
231
|
+
| create p50 | 29 ms | 60 ms |
|
|
232
|
+
| first exec | 33 ms | 67 ms |
|
|
233
|
+
| snapshot restore | 13 ms | 43 ms |
|
|
234
|
+
| fan-out of 100 clones | 0.6 s | 17.9 s |
|
|
235
|
+
| cost per clone-hour | $0.0043 | $0.0132 |
|
|
236
|
+
| VM-entry deaths | 0 | 2.9 per 1,000 hot restores, 325 per 1,000 at density |
|
|
237
|
+
|
|
238
|
+
(Bake-off, 2 vCPU / 1 GiB app template, `contrib/bench/bakeoff/RESULTS.md`, "Huge pages". The hosts are different CPU
|
|
239
|
+
generations and metal has 12 times the CPUs, so the fan-out column is not virtualization alone. Metal is cheaper per clone because it
|
|
240
|
+
carries about 1,000 clones where the nested host was measured at 64.)
|
|
241
|
+
|
|
242
|
+
North star on one `c7i.metal-48xl` (500 clones of a built-from-source web app at once, each running a short test then destroyed):
|
|
243
|
+
99% done 112 s after the PR arrived, all 500 passed with no replacements and no VMM deaths, about $0.34 per PR from a stopped node,
|
|
244
|
+
against 300 s, 264 replacements and about $1.00 on five `m8i.8xlarge`.
|
|
245
|
+
|
|
246
|
+
Trade-offs, plainly:
|
|
247
|
+
|
|
248
|
+
* **Boot.** A metal reboot has been quoted at about 7.5 minutes. Measured on a `c7i.metal-48xl`: a fresh launch was registered with SSM
|
|
249
|
+
in 74 s and a stopped node was serving again 47 s after `StartInstances`, but stopping it took 320 s (unbilled). Time from a
|
|
250
|
+
planner launch to an enrolled node on metal has not been measured (see
|
|
251
|
+
`contrib/bench/results/2026-10-04-wave-c-metal-northstar.md`). Nested hosts register in 40 to 45 s.
|
|
252
|
+
* **Coarse sizes.** Metal is `.metal` sizes only: 96 or 192 vCPUs on the `c7i` family. A small or spiky fleet pays for a whole box
|
|
253
|
+
or runs nested.
|
|
254
|
+
* **Thin Spot capacity.** Metal Spot pools are small; the planner falls back to on-demand when one refuses, and the price you plan
|
|
255
|
+
with is the cheaper Spot one.
|
|
256
|
+
* **Large blast radius.** One host carries up to about a thousand clones. Losing it, or restarting its agent, hits all of them; keep
|
|
257
|
+
`fleet max-clones-per-host` as the cap you can accept and keep more than one host for anything that must survive a host loss.
|
|
258
|
+
|
|
259
|
+
The capacity planner prefers metal when a reservation's `needed_by` allows its boot time and the price per clone is lower
|
|
260
|
+
([docs/capacity-planning.md](docs/capacity-planning.md)); `blackcat_vm_entry_deaths` counts the VM-entry failures that make nested
|
|
261
|
+
hosts unreliable at density. Setup: [docs/bare-mode.md](docs/bare-mode.md#recommended-production-shape-bare-metal).
|
|
262
|
+
|
|
263
|
+
## Install (alpha)
|
|
264
|
+
|
|
265
|
+
Identical on bare metal and on AWS:
|
|
266
|
+
|
|
267
|
+
```bash
|
|
268
|
+
blackcat init --bind 0.0.0.0:8080 --hostname control.example
|
|
269
|
+
# init installs and starts blackcat-placer and blackcat-gateway
|
|
270
|
+
|
|
271
|
+
# Copy /etc/blackcat/tls/ca.pem to each node. Join tokens are single-use, so
|
|
272
|
+
# mint a distinct token with `blackcat join-token` for each node.
|
|
273
|
+
blackcat join https://control.example:8080 \
|
|
274
|
+
--token <node-1-token> --ca-cert /path/to/ca.pem
|
|
275
|
+
blackcat join https://control.example:8080 \
|
|
276
|
+
--token <node-2-token> --ca-cert /path/to/ca.pem
|
|
277
|
+
|
|
278
|
+
# Fleet and workload operations use a tenant capability token, not the admin
|
|
279
|
+
# credential. Create a tenant/API key and mint that token through the API.
|
|
280
|
+
blackcat fleet status --control-plane https://control.example:8080 \
|
|
281
|
+
--ca-cert /path/to/ca.pem --token <capability-token>
|
|
282
|
+
blackcat template create --control-plane https://control.example:8080 \
|
|
283
|
+
--ca-cert /path/to/ca.pem --token <capability-token> --name base \
|
|
284
|
+
--docker <image> --vcpus <n> --memory-mib <m>
|
|
285
|
+
blackcat sandbox create --control-plane https://control.example:8080 \
|
|
286
|
+
--ca-cert /path/to/ca.pem --token <capability-token> --template <t> \
|
|
287
|
+
--request-id <stable-acquisition-id> --ttl-secs 3600 \
|
|
288
|
+
--renewable-for-secs 86400
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
Create request IDs identify logical acquisitions: reuse the ID and exact
|
|
292
|
+
parameters after an interrupted request, never for a new VM. A `pending`
|
|
293
|
+
receipt is not ready for commands. EXEC also requires an explicit positive
|
|
294
|
+
`--timeout-ms` (or JSON `timeout_ms`); timeout does not undo effects that already
|
|
295
|
+
happened. These breaking pre-trial contracts are being verified together; old
|
|
296
|
+
embedded guests and callers must be rebuilt/updated, not silently accepted.
|
|
297
|
+
|
|
298
|
+
AWS adds exactly two things: an AMI that registers itself using the EC2 instance identity
|
|
299
|
+
document, and a `CapacityProvider` that wraps an ASG. If AWS ever needs a *different*
|
|
300
|
+
path, the design has failed — there is a test for that.
|
|
301
|
+
|
|
302
|
+
**Bare-metal prerequisites:** Linux with `/dev/kvm`, root for the node agent, a hardened
|
|
303
|
+
host kernel, and an uncompressed guest `vmlinux` ≥ 5.18. Releases ship a known-good guest
|
|
304
|
+
kernel, because otherwise a new user's first task is a kernel build.
|
|
305
|
+
Control-plane `blackcat init` additionally requires systemd and Python 3 at
|
|
306
|
+
`/usr/bin/python3`. Init checks the interpreter before changing host state and
|
|
307
|
+
installs the shared bounded placer-listener gate for every gateway start.
|
|
308
|
+
The gate establishes startup ordering, not catalog or workload readiness.
|
|
309
|
+
Filesystem template nodes also need `e2fsprogs` (`mkfs.ext4` and `resize2fs`)
|
|
310
|
+
and GNU `cp` with sparse/reflink support. A compact template is verified before
|
|
311
|
+
its private sandbox copy is grown to the declared writable capacity; missing
|
|
312
|
+
growth tooling is a launch error, not permission to boot a smaller disk.
|
|
313
|
+
|
|
314
|
+
Generic plain-Linux installs use the default-feature `blackcat-agent-x86_64-musl`,
|
|
315
|
+
`blackcat-gateway-x86_64-musl`, and `blackcat-placer-x86_64-musl` release artifacts.
|
|
316
|
+
The AWS AMI uses the corresponding `-aws` variants.
|
|
317
|
+
|
|
318
|
+
## Authentication modes
|
|
319
|
+
|
|
320
|
+
Private/VPC installations keep the existing static authentication mode: an
|
|
321
|
+
administrator creates tenant API keys, each with a fixed operation set, and
|
|
322
|
+
those keys mint short-lived Blackcat capabilities within that set. Optional hosted access adds external
|
|
323
|
+
OIDC login for people while retaining those same capabilities as the workload
|
|
324
|
+
authorization boundary. OIDC is not required for a private cluster, and enabling
|
|
325
|
+
it does not expose admin or node authority.
|
|
326
|
+
|
|
327
|
+
The context/login/claim CLI and the exact trust, storage and public deployment
|
|
328
|
+
boundaries are specified in
|
|
329
|
+
[docs/public-access-auth.md](docs/public-access-auth.md). Public reachability is
|
|
330
|
+
network policy, not an application `public` switch.
|
|
331
|
+
For an identity provider that requires an exact pre-registered loopback
|
|
332
|
+
redirect, use `blackcat login --callback-port PORT` and register
|
|
333
|
+
`http://127.0.0.1:PORT/callback` on its public OIDC client. The default login
|
|
334
|
+
continues to choose a free ephemeral port.
|
|
335
|
+
Hosted CLI sessions use a revocable 12-hour access credential and a distinct
|
|
336
|
+
30-day, non-extendable refresh credential. The CLI renews and rotates both
|
|
337
|
+
automatically before commands; browser login is required again at the absolute
|
|
338
|
+
deadline or after revocation. See
|
|
339
|
+
[the session lifecycle and tradeoff](docs/hosted-session-renewal.md). The
|
|
340
|
+
[disposable dev trial runbook](docs/history/hosted-auth-dev-e2e.md#manual-cli-login-to-claim-window)
|
|
341
|
+
includes a macOS Docker harness for a manual login-to-claim test; it is not a
|
|
342
|
+
production deployment recipe.
|
|
343
|
+
|
|
344
|
+
For a native, globally usable `blackcat` command on Apple-silicon macOS or
|
|
345
|
+
Linux x86-64/arm64, run `curl -fsSL https://runblackcat.dev/install.sh | sh`;
|
|
346
|
+
see [CLI installation](docs/cli-install.md). The installer uses release
|
|
347
|
+
checksums from the public
|
|
348
|
+
[`blackcat-releases`](https://github.com/TreebeardHQ/blackcat-releases) mirror
|
|
349
|
+
and needs neither Docker nor Rust on the client. To run a fleet on plain
|
|
350
|
+
Debian or Ubuntu servers, see [Bare mode](docs/bare-mode.md). The site itself lives in
|
|
351
|
+
[`site/`](site/) and publishes an allowlisted subset of `docs/`.
|
|
352
|
+
`blackcat template list` shows launchable templates to members without their
|
|
353
|
+
source; `blackcat sandbox claim alpine-e2e-matched` resolves the latest version
|
|
354
|
+
and pins its ID for safe retries. Intel Mac binaries are not published.
|
|
355
|
+
|
|
356
|
+
## Documentation
|
|
357
|
+
|
|
358
|
+
- **[docs/PRODUCT.md](docs/PRODUCT.md)** — the product contract and readiness levels.
|
|
359
|
+
- **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)** — processes, crates, trust domains, a sandbox's life.
|
|
360
|
+
- **[docs/SECURITY.md](docs/SECURITY.md)** — isolation boundaries and known gaps.
|
|
361
|
+
- **[docs/fleet-upgrades.md](docs/fleet-upgrades.md)** — stage, preview, run, and resume compatible fleet upgrades.
|
|
362
|
+
- **[docs/GROOMED.md](docs/GROOMED.md)** — the ranked backlog.
|
|
363
|
+
- **[AGENTS.md](AGENTS.md)** — how to work in this repository.
|
|
364
|
+
- **[docs/forkable-templates.md](docs/forkable-templates.md)**,
|
|
365
|
+
**[docs/preview-domains.md](docs/preview-domains.md)**,
|
|
366
|
+
**[docs/public-access-auth.md](docs/public-access-auth.md)** — feature contracts.
|
|
367
|
+
- **docs/history/** — the original plan, the progress ledger, trial acceptance
|
|
368
|
+
records and measurements. Rationale lives there; it is not required reading.
|
|
369
|
+
|
|
370
|
+
## Prior art worth knowing
|
|
371
|
+
|
|
372
|
+
[e2b-dev/infra](https://github.com/e2b-dev/infra) (Apache-2.0) solves much of this and is
|
|
373
|
+
worth borrowing from — its AWS support is beta and it requires operating Nomad + Consul,
|
|
374
|
+
which is the gap blackcat targets. Firecracker's own UFFD reference handler, the
|
|
375
|
+
[rust-vmm](https://github.com/rust-vmm) crates, and Cloud Hypervisor are the other sources.
|
|
376
|
+
|
|
377
|
+
## Contributing
|
|
378
|
+
|
|
379
|
+
While the service is pre-0.01, **commit straight to `main`** — no PR review cycle,
|
|
380
|
+
no branch discipline. Use conventional commit titles (`feat:`, `fix:`, `docs:`,
|
|
381
|
+
`refactor:`, `test:`, `chore:`), and verify in proportion to risk before pushing.
|
|
382
|
+
|
|
383
|
+
```sh
|
|
384
|
+
git add <paths>
|
|
385
|
+
git commit -m "fix(gateway): ..."
|
|
386
|
+
git push origin main
|
|
387
|
+
```
|
|
388
|
+
|
|
389
|
+
Open a PR only when you want hosted CI to check something you cannot check
|
|
390
|
+
locally — the Rust toolchain is pinned in `rust-toolchain.toml`, so a container
|
|
391
|
+
gets you the same compiler without installing one:
|
|
392
|
+
|
|
393
|
+
```sh
|
|
394
|
+
docker run --rm -v "$PWD":/w -w /w rust:1.85.1 \
|
|
395
|
+
sh -c "cargo fmt --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test"
|
|
396
|
+
```
|
|
397
|
+
|
|
398
|
+
This will tighten up before a real release. See [AGENTS.md](AGENTS.md) for the
|
|
399
|
+
security and provider-boundary rules that do *not* relax.
|
|
400
|
+
|
|
401
|
+
## License
|
|
402
|
+
|
|
403
|
+
The server (gateway, placer, agent and the other fleet crates) is [SSPL-1.0](LICENSE-SSPL); the CLI
|
|
404
|
+
source, the TypeScript SDK and the shared client libraries are [Apache-2.0](LICENSE-APACHE). The distributed
|
|
405
|
+
`blackcat` CLI binary embeds server crates and is therefore subject to SSPL-1.0. See [LICENSE](LICENSE) for
|
|
406
|
+
the per-crate split. Releases up to v0.1.0-alpha.81 remain available under Apache-2.0.
|
package/package.json
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@runblackcat/sdk",
|
|
3
|
+
"version": "0.1.0-alpha.100",
|
|
4
|
+
"description": "TypeScript SDK for the Blackcat sandbox gateway",
|
|
5
|
+
"license": "Apache-2.0",
|
|
6
|
+
"repository": "https://github.com/TreebeardHQ/blackcat",
|
|
7
|
+
"type": "module",
|
|
8
|
+
"exports": {
|
|
9
|
+
".": {
|
|
10
|
+
"types": "./sdk/dist/index.d.ts",
|
|
11
|
+
"import": "./sdk/dist/index.js",
|
|
12
|
+
"default": "./sdk/dist/index.js"
|
|
13
|
+
}
|
|
14
|
+
},
|
|
15
|
+
"files": ["sdk/dist", "README.md", "LICENSE"],
|
|
16
|
+
"scripts": {
|
|
17
|
+
"build": "bun build sdk/src/index.ts --outdir sdk/dist --target node --format esm && tsc -p sdk/tsconfig.build.json",
|
|
18
|
+
"prepack": "bun run build",
|
|
19
|
+
"typecheck": "tsc -p sdk/tsconfig.json --noEmit",
|
|
20
|
+
"test": "bun test sdk/src/index.test.ts"
|
|
21
|
+
},
|
|
22
|
+
"devDependencies": {
|
|
23
|
+
"@types/bun": "1.3.12",
|
|
24
|
+
"typescript": "6.0.3"
|
|
25
|
+
},
|
|
26
|
+
"engines": {
|
|
27
|
+
"bun": ">=1.1"
|
|
28
|
+
}
|
|
29
|
+
}
|