@runblackcat/sdk 0.1.0-alpha.44
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -0
- package/README.md +341 -0
- package/package.json +28 -0
- package/sdk/dist/index.d.ts +503 -0
- package/sdk/dist/index.js +642 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
package/README.md
ADDED
|
@@ -0,0 +1,341 @@
|
|
|
1
|
+
# blackcat
|
|
2
|
+
|
|
3
|
+
A self-hosted [Firecracker](https://firecracker-microvm.github.io/) microVM fleet:
|
|
4
|
+
multi-tenant sandboxes with snapshots, resumable workloads, and scale-to-zero.
|
|
5
|
+
|
|
6
|
+
**Agnostic core, paved AWS path.** The core knows about Linux, not about any cloud. AWS is
|
|
7
|
+
the one deployment path that comes fully assembled; everything else is possible with work.
|
|
8
|
+
|
|
9
|
+
## Status: hosted development, pre-release
|
|
10
|
+
|
|
11
|
+
The gateway, placer, node agent, guest, forkable checkpoint templates, durable
|
|
12
|
+
processes, previews and the TypeScript SDK are implemented, and a hot claim from
|
|
13
|
+
a checkpoint template restores in about 1.1 s on the development fleet. Nothing
|
|
14
|
+
is shipped to customers or released as open source.
|
|
15
|
+
|
|
16
|
+
What is missing before untrusted tenants can share workers is written down in
|
|
17
|
+
[docs/SECURITY.md](docs/SECURITY.md); the ranked backlog is
|
|
18
|
+
[docs/GROOMED.md](docs/GROOMED.md). Readiness levels and the product contract
|
|
19
|
+
are in [docs/PRODUCT.md](docs/PRODUCT.md).
|
|
20
|
+
|
|
21
|
+
## What it will do
|
|
22
|
+
|
|
23
|
+
- **Sandbox API** — acquire an isolated microVM with retry-safe request identity
|
|
24
|
+
and a pinned immutable template version. Creation never blocks: the
|
|
25
|
+
request is admitted durably and answered at once, so a caller holds a claim
|
|
26
|
+
rather than a connection.
|
|
27
|
+
- **Queue and backpressure** — a request the fleet has no room for waits durably
|
|
28
|
+
instead of failing, and is placed by a background drain as capacity arrives.
|
|
29
|
+
Queued work is charged against tenant quota exactly like running work, so the
|
|
30
|
+
quota is the backpressure limit and no second limiter exists.
|
|
31
|
+
- **Demand-driven capacity** — the placer sizes the fleet from the backlog it
|
|
32
|
+
cannot place, through a provider seam, and retires idle workers back to zero.
|
|
33
|
+
- **Forkable templates** — build a Docker image into a validated state, memory,
|
|
34
|
+
and disk checkpoint, then restore isolated writable clones with its declared
|
|
35
|
+
services already running. Sealed content-addressed chunks deduplicate within a
|
|
36
|
+
tenant and are fetched lazily into a bounded node cache.
|
|
37
|
+
- **Snapshot / resume** — UFFD lazy paging, so a resumed sandbox pages memory in on demand
|
|
38
|
+
instead of waiting for a full restore.
|
|
39
|
+
- **Scale to zero** — idle sandboxes snapshot to disk and release their RAM.
|
|
40
|
+
The next request resumes them when capacity is available.
|
|
41
|
+
- **HTTP previews** — templates declare named ports; each sandbox gets its own
|
|
42
|
+
HTTPS hostname, with share-link or public access and optional custom domains.
|
|
43
|
+
See [HTTP previews and custom domains](docs/preview-domains.md).
|
|
44
|
+
- **Multi-tenant** — tenants share hosts, isolated at the VM boundary by the Firecracker
|
|
45
|
+
jailer, seccomp, and explicit VM-to-VM network denial.
|
|
46
|
+
|
|
47
|
+
## Design in one page
|
|
48
|
+
|
|
49
|
+
| | |
|
|
50
|
+
|---|---|
|
|
51
|
+
| **Gateway** | Stateless. The single authenticated entry point; resolves sandbox→node and proxies. Scales horizontally. |
|
|
52
|
+
| **Placer** | Single writer. Bin-packs by memory and vCPU, proposes placements, owns quotas, the queue and the node registry. Sole writer of substrate desired capacity. |
|
|
53
|
+
| **Node agent** | One per host. Firecracker lifecycle, jailer, networking, UFFD handler. **Final authority on admission** — the placer proposes, the node accepts or rejects. |
|
|
54
|
+
| **Guest agent** | Inside each sandbox, reached over vsock. Exec, file I/O, template-declared boot services. |
|
|
55
|
+
| **Plugin supervisor (in development)** | Runs operator-installed, digest-pinned job-source plugins outside sandbox hosts. Each installation has its own identity, secret scope and sandbox grant. A small Rust SDK calls the scoped private API. |
|
|
56
|
+
|
|
57
|
+
Nodes **self-register** with a join token — there is no host inventory file anywhere.
|
|
58
|
+
Tenants are **API-managed rows**, not configuration, so an operator can provision
|
|
59
|
+
and use one without restarting the service.
|
|
60
|
+
|
|
61
|
+
No ZooKeeper, etcd, or Consul at any scale. Placement is sticky and sandboxes never move
|
|
62
|
+
while running, so the problem partitions without consensus.
|
|
63
|
+
|
|
64
|
+
## How it fits together
|
|
65
|
+
|
|
66
|
+
```mermaid
|
|
67
|
+
flowchart TB
|
|
68
|
+
CLI["blackcat-cli"]
|
|
69
|
+
SDK["TypeScript SDK"]
|
|
70
|
+
|
|
71
|
+
subgraph ctrl["Control host"]
|
|
72
|
+
direction LR
|
|
73
|
+
GW["blackcat-gateway<br/>auth · capabilities · routing<br/>listens :443"]
|
|
74
|
+
PL["blackcat-placer<br/>single writer · best-fit<br/>quotas · node registry"]
|
|
75
|
+
DB[("placer.sqlite")]
|
|
76
|
+
GW <-->|"Unix socket"| PL
|
|
77
|
+
PL --- DB
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
subgraph plugins["Plugin host — planned integration"]
|
|
81
|
+
SUP["Plugin supervisor<br/>digest-pinned images · per-install identity"]
|
|
82
|
+
subgraph PIN["Per-install isolated bridge — no direct route"]
|
|
83
|
+
GHPL["GitHub Actions connector<br/>poll queued jobs · request JIT runners"]
|
|
84
|
+
CPX["Connector proxy<br/>GitHub DNS rules · pinned control target"]
|
|
85
|
+
GHPL -.->|"HTTPS CONNECT only"| CPX
|
|
86
|
+
end
|
|
87
|
+
EXT["Dedicated outbound bridge<br/>proxy only"]
|
|
88
|
+
RPX["Runner proxy<br/>GitHub DNS rules · no control target"]
|
|
89
|
+
SUP --> GHPL
|
|
90
|
+
SUP --> CPX
|
|
91
|
+
CPX -.-> EXT
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
SEC[("Plugin secret store<br/>local encrypted or AWS Secrets Manager")]
|
|
95
|
+
GITHUB["GitHub Actions<br/>job queue · results"]
|
|
96
|
+
|
|
97
|
+
OS[("Opaque object store<br/>S3 or filesystem")]
|
|
98
|
+
|
|
99
|
+
subgraph nodehost["Node host — one per machine, N machines"]
|
|
100
|
+
AG["blackcat-agent<br/>listens :8081"]
|
|
101
|
+
UF["UFFD handler<br/>one process per restore"]
|
|
102
|
+
FC["Firecracker + jailer"]
|
|
103
|
+
GU["blackcat-guest<br/>PID 1 inside the VM"]
|
|
104
|
+
RUN["Ephemeral Actions runner<br/>nonroot guest process"]
|
|
105
|
+
AG -->|spawns| UF
|
|
106
|
+
AG --> FC
|
|
107
|
+
UF -.->|"page faults"| FC
|
|
108
|
+
FC -->|vsock| GU
|
|
109
|
+
GU -->|"nonroot boot service"| RUN
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
CLI -->|HTTPS| GW
|
|
113
|
+
SDK -->|HTTPS| GW
|
|
114
|
+
CLI -.->|"admin plugin API"| GW
|
|
115
|
+
PL -.->|"installation state"| SUP
|
|
116
|
+
SUP -.->|"scoped Blackcat API"| GW
|
|
117
|
+
SUP -.->|"per-install secret refs"| SEC
|
|
118
|
+
EXT -.->|"GitHub API via proxy"| GITHUB
|
|
119
|
+
EXT -.->|"pinned private API via proxy"| GW
|
|
120
|
+
RPX -.->|"GitHub runner traffic"| GITHUB
|
|
121
|
+
RUN -.->|"one job + result"| RPX
|
|
122
|
+
GW ---|"metadata + grants"| OS
|
|
123
|
+
AG <-->|"short-lived object grants"| OS
|
|
124
|
+
GW -->|"node TLS"| AG
|
|
125
|
+
PL -->|"node TLS · propose"| AG
|
|
126
|
+
AG -->|"HTTPS · register, heartbeat, objects"| GW
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Dashed plugin links show the intended integration, not a deployed runner. The
|
|
130
|
+
connector observes external jobs and asks Blackcat for a sandbox; the external
|
|
131
|
+
source remains the job and result authority. Runner code executes only inside a
|
|
132
|
+
sandbox, without the connector's private key or plugin service credential. See
|
|
133
|
+
the [plugin system contract](docs/plugin-system.md) for the current boundary.
|
|
134
|
+
|
|
135
|
+
**Creating a sandbox.** The gateway checks the caller's capability token and opens a
|
|
136
|
+
durable acquisition receipt keyed by `(tenant, request_id)`, so a retry is the same
|
|
137
|
+
lease rather than a second VM. The placer picks candidate nodes by best fit over free
|
|
138
|
+
memory and vCPU — snapshot-cache affinity first — and **proposes** to each in turn. The
|
|
139
|
+
node accepts or rejects; it is the final admission authority, not the placer. On
|
|
140
|
+
acceptance the agent starts Firecracker under the jailer, and `blackcat-guest` boots as
|
|
141
|
+
PID 1 and serves the vsock protocol. Until the sandbox reports `placed`, the API answers
|
|
142
|
+
`202 pending` and it is not ready for commands.
|
|
143
|
+
|
|
144
|
+
Three things the picture above is drawn carefully to show:
|
|
145
|
+
|
|
146
|
+
- **The placer talks to nodes itself.** It is not hidden behind the gateway — it opens
|
|
147
|
+
its own node-TLS connections to propose placements.
|
|
148
|
+
- **Nodes receive only object-scoped grants.** The control plane authorizes a
|
|
149
|
+
specific artifact operation and the node transfers sealed bytes directly.
|
|
150
|
+
Nodes never receive a general object-store or tenant credential.
|
|
151
|
+
- **There are two TLS trust domains**, not one. Clients and agents reach the gateway
|
|
152
|
+
under the control-plane CA; the gateway and placer reach node agents under a separate
|
|
153
|
+
node CA issuing 24-hour leaves.
|
|
154
|
+
|
|
155
|
+
Gateway and placer speak over a Unix socket, so they are co-located on one host by
|
|
156
|
+
construction. Everything else is a network hop.
|
|
157
|
+
|
|
158
|
+
## Capacity and observability
|
|
159
|
+
|
|
160
|
+
Creation is admitted durably and answered immediately. A request the fleet cannot
|
|
161
|
+
place becomes a **queued acquisition** — it keeps its sandbox identity and its
|
|
162
|
+
quota charge, and a background drain places it as capacity appears. Callers poll
|
|
163
|
+
the claim they already hold:
|
|
164
|
+
|
|
165
|
+
```
|
|
166
|
+
POST /v1/sandboxes {request_id,template,lease:{ttl_secs,renewable_for_secs}}
|
|
167
|
+
-> 202 {"request_id":…, "id":…, "state":"pending", "expires_at":…}
|
|
168
|
+
GET /v1/sandboxes/acquisitions/{id} -> 202 pending … -> 200 {"state":"placed","node":…}
|
|
169
|
+
POST /v1/sandboxes/{id}/renew {ttl_secs} -> 200 {"expires_at":…, "renewable_until":…}
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
`request_id` is caller-supplied and is the idempotency boundary, so repeating a
|
|
173
|
+
create is always safe: it returns the same claim rather than allocating a second
|
|
174
|
+
sandbox.
|
|
175
|
+
|
|
176
|
+
Backpressure uses separate tenant limits for running allocations and retained
|
|
177
|
+
snapshots. Pending/resuming work reserves running capacity; hibernating and
|
|
178
|
+
resuming transitions reserve both classes until they commit. A queued acquisition
|
|
179
|
+
that waits past the queue deadline is torn down by the ordinary expiry reconciler.
|
|
180
|
+
|
|
181
|
+
The placer sizes the fleet from that backlog through the `CapacityProvider`
|
|
182
|
+
seam, and is the **only** writer of substrate desired capacity — it raises the
|
|
183
|
+
count from unplaceable demand and lowers it only by retiring workers it has
|
|
184
|
+
proven empty. An external autoscaling policy driving the same knob would fight
|
|
185
|
+
it, so deployments should cap the fleet (a maximum size) rather than scale it.
|
|
186
|
+
|
|
187
|
+
`GET /metrics` exposes Prometheus text to the **admin credential**. The series
|
|
188
|
+
are fleet-wide and keyed by tenant, so unlike `/v1/fleet` — which serves an
|
|
189
|
+
`INSPECT` holder its own rows only — there is no tenant-scoped view to hand out.
|
|
190
|
+
`/livez` and `/readyz` stay unauthenticated because they disclose nothing.
|
|
191
|
+
|
|
192
|
+
```
|
|
193
|
+
blackcat_demand_unsatisfied{tenant="…"} queued acquisitions
|
|
194
|
+
blackcat_demand_oldest_age_seconds longest current wait
|
|
195
|
+
blackcat_capacity_target_nodes workers last asked for
|
|
196
|
+
blackcat_nodes{health="healthy"} workers actually registered
|
|
197
|
+
blackcat_placement_rejected_total{reason="…"} no_capacity | quota
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
A fleet pinned at its ceiling shows up as `blackcat_capacity_target_nodes`
|
|
201
|
+
persistently above `blackcat_nodes`.
|
|
202
|
+
|
|
203
|
+
## Install (alpha)
|
|
204
|
+
|
|
205
|
+
Identical on bare metal and on AWS:
|
|
206
|
+
|
|
207
|
+
```bash
|
|
208
|
+
blackcat init --bind 0.0.0.0:8080 --hostname control.example
|
|
209
|
+
# init installs and starts blackcat-placer and blackcat-gateway
|
|
210
|
+
|
|
211
|
+
# Copy /etc/blackcat/tls/ca.pem to each node. Join tokens are single-use, so
|
|
212
|
+
# mint a distinct token with `blackcat join-token` for each node.
|
|
213
|
+
blackcat join https://control.example:8080 \
|
|
214
|
+
--token <node-1-token> --ca-cert /path/to/ca.pem
|
|
215
|
+
blackcat join https://control.example:8080 \
|
|
216
|
+
--token <node-2-token> --ca-cert /path/to/ca.pem
|
|
217
|
+
|
|
218
|
+
# Fleet and workload operations use a tenant capability token, not the admin
|
|
219
|
+
# credential. Create a tenant/API key and mint that token through the API.
|
|
220
|
+
blackcat fleet status --control-plane https://control.example:8080 \
|
|
221
|
+
--ca-cert /path/to/ca.pem --token <capability-token>
|
|
222
|
+
blackcat template create --control-plane https://control.example:8080 \
|
|
223
|
+
--ca-cert /path/to/ca.pem --token <capability-token> --name base \
|
|
224
|
+
--docker <image> --vcpus <n> --memory-mib <m>
|
|
225
|
+
blackcat sandbox create --control-plane https://control.example:8080 \
|
|
226
|
+
--ca-cert /path/to/ca.pem --token <capability-token> --template <t> \
|
|
227
|
+
--request-id <stable-acquisition-id> --ttl-secs 3600 \
|
|
228
|
+
--renewable-for-secs 86400
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
Create request IDs identify logical acquisitions: reuse the ID and exact
|
|
232
|
+
parameters after an interrupted request, never for a new VM. A `pending`
|
|
233
|
+
receipt is not ready for commands. EXEC also requires an explicit positive
|
|
234
|
+
`--timeout-ms` (or JSON `timeout_ms`); timeout does not undo effects that already
|
|
235
|
+
happened. These breaking pre-trial contracts are being verified together; old
|
|
236
|
+
embedded guests and callers must be rebuilt/updated, not silently accepted.
|
|
237
|
+
|
|
238
|
+
AWS adds exactly two things: an AMI that registers itself using the EC2 instance identity
|
|
239
|
+
document, and a `CapacityProvider` that wraps an ASG. If AWS ever needs a *different*
|
|
240
|
+
path, the design has failed — there is a test for that.
|
|
241
|
+
|
|
242
|
+
**Bare-metal prerequisites:** Linux with `/dev/kvm`, root for the node agent, a hardened
|
|
243
|
+
host kernel, and an uncompressed guest `vmlinux` ≥ 5.18. Releases ship a known-good guest
|
|
244
|
+
kernel, because otherwise a new user's first task is a kernel build.
|
|
245
|
+
Control-plane `blackcat init` additionally requires systemd and Python 3 at
|
|
246
|
+
`/usr/bin/python3`. Init checks the interpreter before changing host state and
|
|
247
|
+
installs the shared bounded placer-listener gate for every gateway start.
|
|
248
|
+
The gate establishes startup ordering, not catalog or workload readiness.
|
|
249
|
+
Filesystem template nodes also need `e2fsprogs` (`mkfs.ext4` and `resize2fs`)
|
|
250
|
+
and GNU `cp` with sparse/reflink support. A compact template is verified before
|
|
251
|
+
its private sandbox copy is grown to the declared writable capacity; missing
|
|
252
|
+
growth tooling is a launch error, not permission to boot a smaller disk.
|
|
253
|
+
|
|
254
|
+
Generic plain-Linux installs use the default-feature `blackcat-agent-x86_64-musl`,
|
|
255
|
+
`blackcat-gateway-x86_64-musl`, and `blackcat-placer-x86_64-musl` release artifacts.
|
|
256
|
+
The AWS AMI uses the corresponding `-aws` variants.
|
|
257
|
+
|
|
258
|
+
## Authentication modes
|
|
259
|
+
|
|
260
|
+
Private/VPC installations keep the existing static authentication mode: an
|
|
261
|
+
administrator creates tenant API keys, each with a fixed operation set, and
|
|
262
|
+
those keys mint short-lived Blackcat capabilities within that set. Optional hosted access adds external
|
|
263
|
+
OIDC login for people while retaining those same capabilities as the workload
|
|
264
|
+
authorization boundary. OIDC is not required for a private cluster, and enabling
|
|
265
|
+
it does not expose admin or node authority.
|
|
266
|
+
|
|
267
|
+
The context/login/claim CLI and the exact trust, storage and public deployment
|
|
268
|
+
boundaries are specified in
|
|
269
|
+
[docs/public-access-auth.md](docs/public-access-auth.md). Public reachability is
|
|
270
|
+
network policy, not an application `public` switch.
|
|
271
|
+
For an identity provider that requires an exact pre-registered loopback
|
|
272
|
+
redirect, use `blackcat login --callback-port PORT` and register
|
|
273
|
+
`http://127.0.0.1:PORT/callback` on its public OIDC client. The default login
|
|
274
|
+
continues to choose a free ephemeral port.
|
|
275
|
+
Hosted CLI sessions use a revocable 12-hour access credential and a distinct
|
|
276
|
+
30-day, non-extendable refresh credential. The CLI renews and rotates both
|
|
277
|
+
automatically before commands; browser login is required again at the absolute
|
|
278
|
+
deadline or after revocation. See
|
|
279
|
+
[the session lifecycle and tradeoff](docs/hosted-session-renewal.md). The
|
|
280
|
+
[disposable dev trial runbook](docs/history/hosted-auth-dev-e2e.md#manual-cli-login-to-claim-window)
|
|
281
|
+
includes a macOS Docker harness for a manual login-to-claim test; it is not a
|
|
282
|
+
production deployment recipe.
|
|
283
|
+
|
|
284
|
+
For a native, globally usable `blackcat` command on Apple-silicon macOS or
|
|
285
|
+
Linux x86-64/arm64, run `curl -fsSL https://runblackcat.dev/install.sh | sh`;
|
|
286
|
+
see [CLI installation](docs/cli-install.md). The installer uses release
|
|
287
|
+
checksums from the public
|
|
288
|
+
[`blackcat-releases`](https://github.com/TreebeardHQ/blackcat-releases) mirror
|
|
289
|
+
and needs neither Docker nor Rust on the client. The site itself lives in
|
|
290
|
+
[`site/`](site/) and publishes an allowlisted subset of `docs/`.
|
|
291
|
+
`blackcat template list` shows launchable templates to members without their
|
|
292
|
+
source; `blackcat sandbox claim alpine-e2e-matched` resolves the latest version
|
|
293
|
+
and pins its ID for safe retries. Intel Mac binaries are not published.
|
|
294
|
+
|
|
295
|
+
## Documentation
|
|
296
|
+
|
|
297
|
+
- **[docs/PRODUCT.md](docs/PRODUCT.md)** — the product contract and readiness levels.
|
|
298
|
+
- **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)** — processes, crates, trust domains, a sandbox's life.
|
|
299
|
+
- **[docs/SECURITY.md](docs/SECURITY.md)** — isolation boundaries and known gaps.
|
|
300
|
+
- **[docs/GROOMED.md](docs/GROOMED.md)** — the ranked backlog.
|
|
301
|
+
- **[AGENTS.md](AGENTS.md)** — how to work in this repository.
|
|
302
|
+
- **[docs/forkable-templates.md](docs/forkable-templates.md)**,
|
|
303
|
+
**[docs/preview-domains.md](docs/preview-domains.md)**,
|
|
304
|
+
**[docs/public-access-auth.md](docs/public-access-auth.md)** — feature contracts.
|
|
305
|
+
- **docs/history/** — the original plan, the progress ledger, trial acceptance
|
|
306
|
+
records and measurements. Rationale lives there; it is not required reading.
|
|
307
|
+
|
|
308
|
+
## Prior art worth knowing
|
|
309
|
+
|
|
310
|
+
[e2b-dev/infra](https://github.com/e2b-dev/infra) (Apache-2.0) solves much of this and is
|
|
311
|
+
worth borrowing from — its AWS support is beta and it requires operating Nomad + Consul,
|
|
312
|
+
which is the gap blackcat targets. Firecracker's own UFFD reference handler, the
|
|
313
|
+
[rust-vmm](https://github.com/rust-vmm) crates, and Cloud Hypervisor are the other sources.
|
|
314
|
+
|
|
315
|
+
## Contributing
|
|
316
|
+
|
|
317
|
+
While the service is pre-0.01, **commit straight to `main`** — no PR review cycle,
|
|
318
|
+
no branch discipline. Use conventional commit titles (`feat:`, `fix:`, `docs:`,
|
|
319
|
+
`refactor:`, `test:`, `chore:`), and verify in proportion to risk before pushing.
|
|
320
|
+
|
|
321
|
+
```sh
|
|
322
|
+
git add <paths>
|
|
323
|
+
git commit -m "fix(gateway): ..."
|
|
324
|
+
git push origin main
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
Open a PR only when you want hosted CI to check something you cannot check
|
|
328
|
+
locally — the Rust toolchain is pinned in `rust-toolchain.toml`, so a container
|
|
329
|
+
gets you the same compiler without installing one:
|
|
330
|
+
|
|
331
|
+
```sh
|
|
332
|
+
docker run --rm -v "$PWD":/w -w /w rust:1.85.1 \
|
|
333
|
+
sh -c "cargo fmt --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test"
|
|
334
|
+
```
|
|
335
|
+
|
|
336
|
+
This will tighten up before a real release. See [AGENTS.md](AGENTS.md) for the
|
|
337
|
+
security and provider-boundary rules that do *not* relax.
|
|
338
|
+
|
|
339
|
+
## License
|
|
340
|
+
|
|
341
|
+
Apache-2.0. See [LICENSE](LICENSE).
|
package/package.json
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@runblackcat/sdk",
|
|
3
|
+
"version": "0.1.0-alpha.44",
|
|
4
|
+
"description": "TypeScript SDK for the Blackcat sandbox gateway",
|
|
5
|
+
"license": "Apache-2.0",
|
|
6
|
+
"repository": "https://github.com/TreebeardHQ/blackcat",
|
|
7
|
+
"type": "module",
|
|
8
|
+
"exports": {
|
|
9
|
+
".": {
|
|
10
|
+
"types": "./sdk/dist/index.d.ts",
|
|
11
|
+
"import": "./sdk/dist/index.js",
|
|
12
|
+
"default": "./sdk/dist/index.js"
|
|
13
|
+
}
|
|
14
|
+
},
|
|
15
|
+
"files": ["sdk/dist", "README.md", "LICENSE"],
|
|
16
|
+
"scripts": {
|
|
17
|
+
"build": "bun build sdk/src/index.ts --outdir sdk/dist --target node --format esm && tsc -p sdk/tsconfig.build.json",
|
|
18
|
+
"typecheck": "tsc -p sdk/tsconfig.json --noEmit",
|
|
19
|
+
"test": "bun test sdk/src/index.test.ts"
|
|
20
|
+
},
|
|
21
|
+
"devDependencies": {
|
|
22
|
+
"@types/bun": "1.3.12",
|
|
23
|
+
"typescript": "6.0.3"
|
|
24
|
+
},
|
|
25
|
+
"engines": {
|
|
26
|
+
"bun": ">=1.1"
|
|
27
|
+
}
|
|
28
|
+
}
|