bulkhead-runtime 2026.4.5-beta.6 → 2026.4.5-beta.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +86 -480
- package/dist/package.json +30 -1
- package/package.json +30 -1
package/README.md
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
Each in its own OS namespace. Each with private memory, encrypted credentials, and an isolated filesystem.
|
|
7
7
|
**No Docker. No cloud. One `npm install`.**
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Built on battle-tested subsystems extracted from [OpenClaw](https://github.com/nicepkg/openclaw) — model fallback, error classification, API key rotation, SSRF protection, embedding pipeline, and more.
|
|
10
10
|
|
|
11
11
|
---
|
|
12
12
|
|
|
@@ -68,11 +68,11 @@ app.post("/api/agent", async (req, res) => {
|
|
|
68
68
|
|
|
69
69
|
### Teams -- per-team agents, per-team secrets
|
|
70
70
|
|
|
71
|
-
Different
|
|
71
|
+
Different teams, different databases, different cloud accounts. No credential leaks.
|
|
72
72
|
|
|
73
73
|
```typescript
|
|
74
|
-
const eng
|
|
75
|
-
const ops
|
|
74
|
+
const eng = await platform.createWorkspace("engineering");
|
|
75
|
+
const ops = await platform.createWorkspace("ops");
|
|
76
76
|
|
|
77
77
|
eng.skills.enable("github-pr");
|
|
78
78
|
ops.skills.enable("pagerduty");
|
|
@@ -81,22 +81,9 @@ await eng.credentials.store("github", { token: "ghp_eng..." });
|
|
|
81
81
|
await ops.credentials.store("pagerduty", { token: "pd_..." });
|
|
82
82
|
```
|
|
83
83
|
|
|
84
|
-
### Consulting -- client isolation, clean offboarding
|
|
85
|
-
|
|
86
|
-
One `deleteWorkspace()` wipes everything -- memory, credentials, sessions. Gone.
|
|
87
|
-
|
|
88
|
-
```typescript
|
|
89
|
-
const acme = await platform.createWorkspace("client-acme");
|
|
90
|
-
await acme.credentials.store("aws", { key: "...", secret: "..." });
|
|
91
|
-
await acme.run({ message: "Check staging and open a Jira ticket if it failed" });
|
|
92
|
-
|
|
93
|
-
// Client offboarded -- clean wipe
|
|
94
|
-
await platform.deleteWorkspace("client-acme");
|
|
95
|
-
```
|
|
96
|
-
|
|
97
84
|
### CI/CD -- ephemeral agents, zero state leaks
|
|
98
85
|
|
|
99
|
-
Spin up per job, per PR, per deploy.
|
|
86
|
+
Spin up per job, per PR, per deploy. Destroyed after.
|
|
100
87
|
|
|
101
88
|
```typescript
|
|
102
89
|
const ws = await platform.createWorkspace(`deploy-${Date.now()}`);
|
|
@@ -112,35 +99,15 @@ await platform.deleteWorkspace(ws.userId);
|
|
|
112
99
|
|
|
113
100
|
|
|
114
101
|
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
---
|
|
118
|
-
|
|
119
|
-
## Why Bulkhead Over Alternatives
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
| | Docker per user | E2B / Cloud | **Bulkhead Runtime** |
|
|
123
|
-
| -------------------------------- | ------------------ | -------------------- | --------------------------------------------------------- |
|
|
124
|
-
| **Isolation mechanism** | Container per user | Cloud VM per session | **Linux namespaces** |
|
|
125
|
-
| **Credential security** | DIY | Not built-in | **AES-256-GCM, never exposed to agent** |
|
|
126
|
-
| **Persistent memory** | DIY | DIY | **SQLite + vector embeddings per tenant** |
|
|
127
|
-
| **Embedding cache + batch** | DIY | DIY | **SQLite cache, batch with retry** |
|
|
128
|
-
| **SSRF protection** | DIY | DIY | **DNS-validated, private-IP blocking, fail-closed** |
|
|
129
|
-
| **Model fallback** | DIY | DIY | **Automatic multi-model fallback chains** |
|
|
130
|
-
| **API key rotation** | DIY | DIY | **Multi-key with rate-limit rotation** |
|
|
131
|
-
| **Parallel subagents** | DIY | DIY | **Built-in with depth limiting** |
|
|
132
|
-
| **Skills with secret injection** | DIY | DIY | **Credentials injected server-side** |
|
|
133
|
-
| **Infrastructure** | Docker daemon | Cloud API + billing | **Single npm package** |
|
|
134
|
-
| **Cold start** | ~2s | ~5-10s | **~50ms** |
|
|
135
|
-
| **Embeddable in your app** | No | No | **Yes — it's a library** |
|
|
136
|
-
| **License** | — | Proprietary | **MIT** |
|
|
102
|
+
When `workspace.run()` executes, Bulkhead spawns a **child process** with 5 layers of kernel isolation: user namespace, PID namespace, mount namespace (pivot_root), optional network namespace, and cgroups v2 resource limits. The agent **never runs in your application's process** and **cannot see anything outside its sandbox**.
|
|
137
103
|
|
|
104
|
+
> **[Read the full isolation architecture →](docs/isolation.md)**
|
|
138
105
|
|
|
139
106
|
---
|
|
140
107
|
|
|
141
108
|
## Single-User Mode
|
|
142
109
|
|
|
143
|
-
|
|
110
|
+
For prototyping or single-agent use. No platform needed.
|
|
144
111
|
|
|
145
112
|
```typescript
|
|
146
113
|
import { createRuntime } from "bulkhead-runtime";
|
|
@@ -155,448 +122,101 @@ const result = await runtime.run({
|
|
|
155
122
|
});
|
|
156
123
|
```
|
|
157
124
|
|
|
158
|
-
The agent runs inside a Linux namespace sandbox with full coding tools (read, write, edit, bash, grep, find, ls) and autonomous memory (`memory_store`, `memory_search`).
|
|
159
|
-
|
|
160
125
|
---
|
|
161
126
|
|
|
162
|
-
##
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
});
|
|
178
|
-
|
|
179
|
-
const bob = await platform.createWorkspace("bob", {
|
|
180
|
-
provider: "google",
|
|
181
|
-
model: "gemini-2.5-flash",
|
|
182
|
-
});
|
|
183
|
-
```
|
|
184
|
-
|
|
185
|
-
### What "isolated" actually means
|
|
186
|
-
|
|
187
|
-
When `workspace.run()` executes, Bulkhead spawns a **child process** with 5 layers of kernel isolation. The agent **never runs in your application's process**.
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
The agent gets coding tools (bash, file read/write/edit) because the mount namespace restricts its entire filesystem view. **It literally cannot see anything outside its sandbox.**
|
|
192
|
-
|
|
193
|
-
---
|
|
194
|
-
|
|
195
|
-
## Credential Security
|
|
196
|
-
|
|
197
|
-
Credentials are **AES-256-GCM encrypted** at rest. PBKDF2 key derivation with 100k iterations (SHA-512). The agent **never** sees raw secrets — not through tools, not through IPC, not through environment variables.
|
|
198
|
-
|
|
199
|
-
```typescript
|
|
200
|
-
await alice.credentials.store("github", { token: "ghp_alice_secret" });
|
|
201
|
-
await alice.credentials.store("openai", { apiKey: "sk-..." });
|
|
202
|
-
```
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
System environment variables (`PATH`, `HOME`, `NODE_ENV`) are protected from credential key collision. Skill IDs are validated against prototype pollution.
|
|
127
|
+
## Features
|
|
128
|
+
|
|
129
|
+
| Feature | What It Does | Docs |
|
|
130
|
+
|---------|-------------|------|
|
|
131
|
+
| **OS-level Isolation** | 5 layers: user, PID, mount, network namespaces + cgroups v2 | [docs/isolation.md](docs/isolation.md) |
|
|
132
|
+
| **Encrypted Credentials** | AES-256-GCM at rest, PBKDF2 key derivation, never exposed to agent | [docs/credentials.md](docs/credentials.md) |
|
|
133
|
+
| **Hybrid Memory** | Vector + FTS5 fusion, MMR diversity, temporal decay, 7-language query expansion | [docs/memory.md](docs/memory.md) |
|
|
134
|
+
| **Per-Workspace Skills** | Global registry, per-tenant enablement, credentials injected server-side | [docs/skills.md](docs/skills.md) |
|
|
135
|
+
| **Session Continuity** | Named sessions, JSONL transcripts, context pruning | [docs/sessions.md](docs/sessions.md) |
|
|
136
|
+
| **Model Fallback** | Automatic multi-model chains, error classification, cooldown tracking | [docs/fallback.md](docs/fallback.md) |
|
|
137
|
+
| **API Key Rotation** | Multi-key per provider, automatic rotation on rate limits | [docs/fallback.md](docs/fallback.md) |
|
|
138
|
+
| **Subagent Orchestration** | Parallel execution, depth limiting, registry | [docs/subagents.md](docs/subagents.md) |
|
|
139
|
+
| **Structured Logging** | JSON/pretty/compact, file output, subsystem tagging | [docs/logging.md](docs/logging.md) |
|
|
140
|
+
| **SSRF Protection** | DNS pinning, private IP blocking, hostname allowlists, fail-closed | [docs/memory.md](docs/memory.md) |
|
|
141
|
+
| **Embedding Pipeline** | Batch with retry, SQLite cache, 5 providers (including local Ollama) | [docs/memory.md](docs/memory.md) |
|
|
207
142
|
|
|
208
143
|
---
|
|
209
144
|
|
|
210
|
-
##
|
|
211
|
-
|
|
212
|
-
Skills are registered globally and **enabled per workspace**. Each workspace gets exactly the capabilities it needs — nothing more.
|
|
213
|
-
|
|
214
|
-
```typescript
|
|
215
|
-
const frontend = await platform.createWorkspace("team-frontend");
|
|
216
|
-
const backend = await platform.createWorkspace("team-backend");
|
|
217
|
-
|
|
218
|
-
frontend.skills.enable("github-issues");
|
|
219
|
-
backend.skills.enable("github-issues");
|
|
220
|
-
backend.skills.enable("db-migration");
|
|
221
|
-
|
|
222
|
-
await frontend.credentials.store("github", { token: "ghp_frontend_token" });
|
|
223
|
-
await backend.credentials.store("github", { token: "ghp_backend_token" });
|
|
224
|
-
await backend.credentials.store("database", { url: "postgres://prod:5432/app" });
|
|
225
|
-
```
|
|
226
|
-
|
|
227
|
-
```javascript
|
|
228
|
-
// skills/github-issues/execute.js
|
|
229
|
-
const params = JSON.parse(await readStdin());
|
|
230
|
-
const token = process.env.token;
|
|
231
|
-
const res = await fetch(`https://api.github.com/repos/${params.repo}/issues`, {
|
|
232
|
-
headers: { Authorization: `Bearer ${token}` },
|
|
233
|
-
});
|
|
234
|
-
console.log(JSON.stringify(await res.json()));
|
|
235
|
-
```
|
|
236
|
-
|
|
237
|
-
Skills run with a minimal env (`PATH`, `HOME`, `NODE_ENV` + credentials), 30s timeout, and 10 MB stdout/stderr cap.
|
|
238
|
-
|
|
239
|
-
---
|
|
240
|
-
|
|
241
|
-
## Memory Isolation
|
|
242
|
-
|
|
243
|
-
Each workspace has its own SQLite database. Memories never cross workspace boundaries — not by access control, **by physical separation**.
|
|
244
|
-
|
|
245
|
-
```typescript
|
|
246
|
-
await alice.memory.store("Project uses React and TypeScript");
|
|
247
|
-
await bob.memory.store("Project uses Vue and Python");
|
|
248
|
-
|
|
249
|
-
const search = await alice.memory.search("framework");
|
|
250
|
-
// → "React and TypeScript"
|
|
251
|
-
// Bob's data doesn't exist in Alice's universe.
|
|
252
|
-
```
|
|
253
|
-
|
|
254
|
-
### Autonomous Agent Memory
|
|
255
|
-
|
|
256
|
-
The agent decides what to remember. Memory persists across sessions, across restarts.
|
|
257
|
-
|
|
258
|
-
```typescript
|
|
259
|
-
// Session 1
|
|
260
|
-
await runtime.run({
|
|
261
|
-
message: "My name is Juan, I work in fintech, I prefer TypeScript",
|
|
262
|
-
sessionId: "onboarding",
|
|
263
|
-
});
|
|
264
|
-
|
|
265
|
-
// Session 2 — different session, memory persists
|
|
266
|
-
await runtime.run({
|
|
267
|
-
message: "Set up a new project for me",
|
|
268
|
-
sessionId: "new-project",
|
|
269
|
-
});
|
|
270
|
-
// Agent searches memory → finds preferences → scaffolds TypeScript project
|
|
271
|
-
```
|
|
272
|
-
|
|
273
|
-
**Hybrid search engine under the hood:**
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
| Stage | Algorithm |
|
|
277
|
-
| ------------------- | ---------------------------------------------------- |
|
|
278
|
-
| **Vector search** | Cosine similarity against stored embeddings |
|
|
279
|
-
| **Keyword search** | SQLite FTS5 with BM25 ranking |
|
|
280
|
-
| **Fusion** | Weighted merge of vector + keyword scores |
|
|
281
|
-
| **Temporal decay** | Exponential time-based score attenuation |
|
|
282
|
-
| **Diversity** | MMR (Maximal Marginal Relevance) re-ranking |
|
|
283
|
-
| **Query expansion** | 7-language keyword extraction (EN/ES/PT/ZH/JA/KO/AR) |
|
|
284
|
-
|
|
145
|
+
## Why Bulkhead Over Alternatives
|
|
285
146
|
|
|
286
|
-
|
|
147
|
+
| | Docker per user | E2B / Cloud | **Bulkhead Runtime** |
|
|
148
|
+
| -------------------------------- | ------------------ | -------------------- | -------------------------------------------- |
|
|
149
|
+
| **Isolation mechanism** | Container per user | Cloud VM per session | **Linux namespaces** |
|
|
150
|
+
| **Credential security** | DIY | Not built-in | **AES-256-GCM, never exposed to agent** |
|
|
151
|
+
| **Persistent memory** | DIY | DIY | **SQLite + vector embeddings per tenant** |
|
|
152
|
+
| **SSRF protection** | DIY | DIY | **DNS-validated, private-IP blocking** |
|
|
153
|
+
| **Model fallback** | DIY | DIY | **Automatic multi-model fallback chains** |
|
|
154
|
+
| **API key rotation** | DIY | DIY | **Multi-key with rate-limit rotation** |
|
|
155
|
+
| **Skills with secret injection** | DIY | DIY | **Credentials injected server-side** |
|
|
156
|
+
| **Infrastructure** | Docker daemon | Cloud API + billing | **Single npm package** |
|
|
157
|
+
| **Cold start** | ~2s | ~5-10s | **~50ms** |
|
|
158
|
+
| **Embeddable in your app** | No | No | **Yes — it's a library** |
|
|
159
|
+
| **License** | — | Proprietary | **MIT** |
|
|
287
160
|
|
|
288
161
|
---
|
|
289
162
|
|
|
290
|
-
##
|
|
291
|
-
|
|
292
|
-
```typescript
|
|
293
|
-
await workspace.run({
|
|
294
|
-
message: "Create a REST API for user management",
|
|
295
|
-
sessionId: "api-project",
|
|
296
|
-
});
|
|
297
|
-
|
|
298
|
-
// Later — agent sees the full conversation history
|
|
299
|
-
await workspace.run({
|
|
300
|
-
message: "Add input validation to those endpoints",
|
|
301
|
-
sessionId: "api-project",
|
|
302
|
-
});
|
|
303
|
-
```
|
|
304
|
-
|
|
305
|
-
Sessions are per-workspace, stored as JSONL transcripts with async locking.
|
|
306
|
-
|
|
307
|
-
---
|
|
163
|
+
## Provider Support
|
|
308
164
|
|
|
309
|
-
|
|
165
|
+
### LLM Providers
|
|
310
166
|
|
|
311
|
-
|
|
167
|
+
Any provider supported by [pi-ai](https://github.com/nicepkg/pi-ai):
|
|
312
168
|
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
169
|
+
| Provider | Example Model |
|
|
170
|
+
|----------|---------------|
|
|
171
|
+
| **Anthropic** | `claude-sonnet-4-20250514` |
|
|
172
|
+
| **Google** | `gemini-2.5-flash` |
|
|
173
|
+
| **OpenAI** | `gpt-4o` |
|
|
174
|
+
| **Groq** | `llama-3.3-70b-versatile` |
|
|
175
|
+
| **Cerebras** | `llama-3.3-70b` |
|
|
176
|
+
| **Mistral** | `mistral-large-latest` |
|
|
177
|
+
| **xAI** | `grok-3` |
|
|
319
178
|
|
|
320
|
-
|
|
179
|
+
### Embedding Providers
|
|
321
180
|
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
{ id: "docs", task: "Check all public APIs have JSDoc", label: "Documentation" },
|
|
330
|
-
],
|
|
331
|
-
maxConcurrent: 3,
|
|
332
|
-
run: async (task) => {
|
|
333
|
-
const r = await runtime.run({
|
|
334
|
-
message: task.task,
|
|
335
|
-
systemPrompt: `You are a ${task.label} expert.`,
|
|
336
|
-
});
|
|
337
|
-
return r.response;
|
|
338
|
-
},
|
|
339
|
-
});
|
|
340
|
-
```
|
|
181
|
+
| Provider | Default Model | Local |
|
|
182
|
+
|----------|--------------|-------|
|
|
183
|
+
| **OpenAI** | `text-embedding-3-small` | |
|
|
184
|
+
| **Gemini** | `gemini-embedding-001` | |
|
|
185
|
+
| **Voyage** | `voyage-3-lite` | |
|
|
186
|
+
| **Mistral** | `mistral-embed` | |
|
|
187
|
+
| **Ollama** | `nomic-embed-text` | **Yes** |
|
|
341
188
|
|
|
342
189
|
---
|
|
343
190
|
|
|
344
191
|
## Lifecycle Hooks
|
|
345
192
|
|
|
193
|
+
6 hook points for audit logging, billing, and custom logic:
|
|
194
|
+
|
|
346
195
|
```typescript
|
|
347
196
|
workspace.hooks.register("before_tool_call", async ({ toolName, input }) => {
|
|
348
197
|
await auditLog.write({ tool: toolName, input, timestamp: Date.now() });
|
|
349
198
|
});
|
|
350
199
|
|
|
351
200
|
workspace.hooks.register("after_agent_end", async ({ sessionId, result }) => {
|
|
352
|
-
await billing.recordUsage(workspace.
|
|
353
|
-
});
|
|
354
|
-
```
|
|
355
|
-
|
|
356
|
-
6 hook points: `session_start` · `session_end` · `before_agent_start` · `after_agent_end` · `before_tool_call` · `after_tool_call`
|
|
357
|
-
|
|
358
|
-
---
|
|
359
|
-
|
|
360
|
-
## Model Fallback & API Key Rotation
|
|
361
|
-
|
|
362
|
-
When a model fails, Bulkhead automatically falls back to the next candidate. The error classification engine — ported from [OpenClaw](https://github.com/nicepkg/openclaw)'s battle-tested failover system — covers rate limits, billing, auth, overload, timeout, model not found, context overflow, and provider-specific patterns (Bedrock, Groq, Azure, Ollama, Mistral, etc.). Providers in cooldown are skipped automatically. When a key is rate-limited, it rotates to the next one.
|
|
363
|
-
|
|
364
|
-
```typescript
|
|
365
|
-
const result = await runtime.run({
|
|
366
|
-
message: "Analyze this codebase",
|
|
367
|
-
provider: "anthropic",
|
|
368
|
-
model: "claude-sonnet-4-20250514",
|
|
369
|
-
|
|
370
|
-
fallbacks: ["openai/gpt-4o", "google/gemini-2.5-flash"],
|
|
371
|
-
|
|
372
|
-
apiKeys: [process.env.ANTHROPIC_KEY_1!, process.env.ANTHROPIC_KEY_2!],
|
|
201
|
+
await billing.recordUsage(workspace.userId, sessionId);
|
|
373
202
|
});
|
|
374
203
|
```
|
|
375
204
|
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
```bash
|
|
379
|
-
ANTHROPIC_API_KEY=sk-ant-...
|
|
380
|
-
ANTHROPIC_API_KEYS=sk-ant-key1,sk-ant-key2,sk-ant-key3
|
|
381
|
-
ANTHROPIC_API_KEY_1=sk-ant-...
|
|
382
|
-
ANTHROPIC_API_KEY_2=sk-ant-...
|
|
383
|
-
```
|
|
205
|
+
`session_start` · `session_end` · `before_agent_start` · `after_agent_end` · `before_tool_call` · `after_tool_call`
|
|
384
206
|
|
|
385
207
|
---
|
|
386
208
|
|
|
387
|
-
##
|
|
209
|
+
## Demos
|
|
388
210
|
|
|
389
|
-
|
|
211
|
+
12 runnable demos covering every feature. 4 require an API key, 8 run fully local.
|
|
390
212
|
|
|
391
|
-
|
|
392
|
-
import { resolveContextWindowInfo, evaluateContextWindowGuard } from "bulkhead-runtime";
|
|
393
|
-
|
|
394
|
-
const info = resolveContextWindowInfo({
|
|
395
|
-
modelContextWindow: model.contextWindow,
|
|
396
|
-
configContextTokens: 16_000,
|
|
397
|
-
});
|
|
213
|
+
> **[See all demos →](docs/demos.md)**
|
|
398
214
|
|
|
399
|
-
|
|
400
|
-
// guard.shouldWarn → true if below 32,000 tokens
|
|
401
|
-
// guard.shouldBlock → true if below 16,000 tokens
|
|
402
|
-
```
|
|
403
|
-
|
|
404
|
-
The runtime applies these guards automatically before every agent execution.
|
|
405
|
-
|
|
406
|
-
---
|
|
407
|
-
|
|
408
|
-
## Retry with Compaction
|
|
409
|
-
|
|
410
|
-
Transient errors (rate limits, timeouts, context overflow) trigger automatic retry with exponential backoff.
|
|
411
|
-
|
|
412
|
-
```typescript
|
|
413
|
-
const result = await runtime.run({
|
|
414
|
-
message: "Refactor the auth module",
|
|
415
|
-
maxRetries: 3,
|
|
416
|
-
});
|
|
417
|
-
// On context overflow → SDK compaction reduces history
|
|
418
|
-
// On 429/5xx → exponential backoff + jitter
|
|
419
|
-
```
|
|
420
|
-
|
|
421
|
-
---
|
|
422
|
-
|
|
423
|
-
## Embedding Pipeline
|
|
424
|
-
|
|
425
|
-
### Embedding Cache
|
|
426
|
-
|
|
427
|
-
Embeddings are cached in SQLite to avoid re-embedding unchanged content. Enabled by default.
|
|
428
|
-
|
|
429
|
-
```typescript
|
|
430
|
-
const memory = createSimpleMemoryManager({
|
|
431
|
-
dbDir: "/var/data/memory",
|
|
432
|
-
embeddingProvider: createEmbeddingProvider({ provider: "openai", apiKey: "..." }),
|
|
433
|
-
enableEmbeddingCache: true,
|
|
434
|
-
maxCacheEntries: 50_000,
|
|
435
|
-
});
|
|
436
|
-
```
|
|
437
|
-
|
|
438
|
-
### Batch Embedding with Retry
|
|
439
|
-
|
|
440
|
-
```typescript
|
|
441
|
-
import { embedBatchWithRetry } from "bulkhead-runtime";
|
|
442
|
-
|
|
443
|
-
const result = await embedBatchWithRetry(
|
|
444
|
-
["text 1", "text 2", "text 3"],
|
|
445
|
-
{
|
|
446
|
-
provider: embeddingProvider,
|
|
447
|
-
cache: memory.embeddingCache ?? undefined,
|
|
448
|
-
batchSize: 100,
|
|
449
|
-
concurrency: 2,
|
|
450
|
-
retryAttempts: 3,
|
|
451
|
-
},
|
|
452
|
-
);
|
|
453
|
-
// result: { embeddings: [...], cached: 150, computed: 50, errors: 0 }
|
|
454
|
-
```
|
|
455
|
-
|
|
456
|
-
### SSRF Protection
|
|
457
|
-
|
|
458
|
-
All embedding provider HTTP calls are protected against Server-Side Request Forgery by default. The SSRF engine — ported from [OpenClaw](https://github.com/nicepkg/openclaw) — resolves DNS and pins IPs before connecting, blocks private/link-local ranges, and enforces a hostname allowlist. Fail-closed by default.
|
|
459
|
-
|
|
460
|
-
```typescript
|
|
461
|
-
import { validateUrl, buildBaseUrlPolicy } from "bulkhead-runtime";
|
|
462
|
-
|
|
463
|
-
const provider = createEmbeddingProvider({
|
|
464
|
-
provider: "openai",
|
|
465
|
-
apiKey: "...",
|
|
466
|
-
enableSsrf: true,
|
|
467
|
-
});
|
|
468
|
-
|
|
469
|
-
await validateUrl("https://api.openai.com/v1/embeddings"); // OK
|
|
470
|
-
await validateUrl("http://169.254.1.1/steal"); // throws SSRF error
|
|
471
|
-
```
|
|
472
|
-
|
|
473
|
-
---
|
|
474
|
-
|
|
475
|
-
## File-based Memory Indexing
|
|
476
|
-
|
|
477
|
-
Automatically watches `MEMORY.md` and `memory/` directory for changes and re-indexes them into the memory system.
|
|
478
|
-
|
|
479
|
-
```typescript
|
|
480
|
-
import { createFileIndexer } from "bulkhead-runtime";
|
|
481
|
-
|
|
482
|
-
const indexer = createFileIndexer({
|
|
483
|
-
workspaceDir: "/path/to/workspace",
|
|
484
|
-
memory,
|
|
485
|
-
watchPaths: ["docs/"],
|
|
486
|
-
debounceMs: 2000,
|
|
487
|
-
});
|
|
488
|
-
|
|
489
|
-
indexer.start();
|
|
490
|
-
indexer.stop();
|
|
491
|
-
```
|
|
492
|
-
|
|
493
|
-
---
|
|
494
|
-
|
|
495
|
-
## Session Transcript Indexing
|
|
496
|
-
|
|
497
|
-
Indexes session transcripts into memory for cross-session search. Supports post-compaction re-indexing. Ported from [OpenClaw](https://github.com/nicepkg/openclaw)'s memory-core extension.
|
|
498
|
-
|
|
499
|
-
```typescript
|
|
500
|
-
import { createSessionIndexer } from "bulkhead-runtime";
|
|
501
|
-
|
|
502
|
-
const indexer = createSessionIndexer({
|
|
503
|
-
sessionsDir: path.join(stateDir, "sessions"),
|
|
504
|
-
memory,
|
|
505
|
-
deltaBytes: 4096,
|
|
506
|
-
deltaMessages: 10,
|
|
507
|
-
});
|
|
508
|
-
|
|
509
|
-
await indexer.indexAllSessions();
|
|
510
|
-
indexer.onTranscriptUpdate(sessionFile);
|
|
511
|
-
```
|
|
512
|
-
|
|
513
|
-
---
|
|
514
|
-
|
|
515
|
-
## Structured Logging
|
|
516
|
-
|
|
517
|
-
Structured JSON logging with file output, rotation, and configurable levels.
|
|
518
|
-
|
|
519
|
-
```typescript
|
|
520
|
-
import { configureLogger, createSubsystemLogger } from "bulkhead-runtime";
|
|
521
|
-
|
|
522
|
-
configureLogger({
|
|
523
|
-
level: "debug",
|
|
524
|
-
file: "/var/log/bulkhead-runtime.log",
|
|
525
|
-
maxFileBytes: 10 * 1024 * 1024,
|
|
526
|
-
json: true,
|
|
527
|
-
});
|
|
528
|
-
|
|
529
|
-
const log = createSubsystemLogger("my-module");
|
|
530
|
-
log.info("agent started", { userId: "alice", model: "claude-sonnet-4-20250514" });
|
|
531
|
-
```
|
|
532
|
-
|
|
533
|
-
Or via environment:
|
|
534
|
-
|
|
535
|
-
```bash
|
|
536
|
-
BULKHEAD_LOG_LEVEL=debug
|
|
537
|
-
BULKHEAD_LOG_FILE=/var/log/bulkhead-runtime.log
|
|
538
|
-
```
|
|
539
|
-
|
|
540
|
-
---
|
|
541
|
-
|
|
542
|
-
## Security Architecture
|
|
543
|
-
|
|
544
|
-
### 5 Layers of Sandbox Isolation
|
|
545
|
-
|
|
546
|
-
All layers are **fail-closed** — if any layer can't be applied, the sandbox refuses to start.
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
### Defense in Depth
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
| Defense | Mechanism |
|
|
554
|
-
| ----------------------------- | ---------------------------------------------------------------------------------------------------- |
|
|
555
|
-
| **Env allowlist** | Only `PATH`, `HOME`, `NODE_ENV` + the single API key the agent needs. Everything else dropped. |
|
|
556
|
-
| **Credential proxy** | Secrets decrypted server-side, injected into skill execution. Never sent over IPC. |
|
|
557
|
-
| **Path traversal blocklist** | `/proc`, `/sys`, `/home/`, `/etc/shadow`, `/run/docker.sock`, and more are blocked from bind mounts. |
|
|
558
|
-
| **Symlink rejection** | `additionalBinds` sources must not be symlinks (prevents TOCTOU attacks). |
|
|
559
|
-
| **IPC rate limiting** | 200 calls/sec per method. Prevents resource exhaustion from rogue agents. |
|
|
560
|
-
| **IPC buffer limit** | 50 MB max. Peer stops on overflow to prevent memory exhaustion. |
|
|
561
|
-
| **Prototype pollution guard** | `__proto__`, `constructor`, `prototype` rejected as skill/credential IDs. |
|
|
562
|
-
| **Stdout interception** | IPC uses a dedicated fd. All other stdout is redirected to stderr. |
|
|
563
|
-
| **Sensitive path validation** | `workspaceDir`, `projectDir`, `nodeExecutable`, `additionalBinds` all validated. |
|
|
564
|
-
| **Atomic writes** | Config, credentials, sessions, skill state — all use tmp+rename pattern. |
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
---
|
|
568
|
-
|
|
569
|
-
## Provider Support
|
|
570
|
-
|
|
571
|
-
### LLM Providers
|
|
572
|
-
|
|
573
|
-
Any provider supported by [pi-ai](https://github.com/nicepkg/pi-ai):
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
| Provider | Example Model |
|
|
577
|
-
| ------------- | -------------------------- |
|
|
578
|
-
| **Anthropic** | `claude-sonnet-4-20250514` |
|
|
579
|
-
| **Google** | `gemini-2.5-flash` |
|
|
580
|
-
| **OpenAI** | `gpt-4o` |
|
|
581
|
-
| **Groq** | `llama-3.3-70b-versatile` |
|
|
582
|
-
| **Cerebras** | `llama-3.3-70b` |
|
|
583
|
-
| **Mistral** | `mistral-large-latest` |
|
|
584
|
-
| **xAI** | `grok-3` |
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
### Embedding Providers
|
|
588
|
-
|
|
589
|
-
Optional — keyword search works without any API key.
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
| Provider | Default Model | Local |
|
|
593
|
-
| ----------- | ------------------------ | ------- |
|
|
594
|
-
| **OpenAI** | `text-embedding-3-small` | |
|
|
595
|
-
| **Gemini** | `gemini-embedding-001` | |
|
|
596
|
-
| **Voyage** | `voyage-3-lite` | |
|
|
597
|
-
| **Mistral** | `mistral-embed` | |
|
|
598
|
-
| **Ollama** | `nomic-embed-text` | **Yes** |
|
|
215
|
+
Highlights:
|
|
599
216
|
|
|
217
|
+
- **[demo-workspace-real](demos/demo-workspace-real.ts)** — Full multi-tenant DevOps platform with 2 teams, skills, memory, credentials, and live agent execution
|
|
218
|
+
- **[demo-subagents](demos/demo-subagents.ts)** — Orchestrator delegates to specialist sub-agents
|
|
219
|
+
- **[demo-fallback](demos/demo-fallback.ts)** — Model fallback chains with simulated failures
|
|
600
220
|
|
|
601
221
|
---
|
|
602
222
|
|
|
@@ -613,7 +233,6 @@ src/
|
|
|
613
233
|
│ ├── rootfs.ts Minimal rootfs with bind mounts
|
|
614
234
|
│ ├── ipc.ts Bidirectional JSON-RPC 2.0 over stdio
|
|
615
235
|
│ ├── seccomp.ts BPF syscall filter profiles
|
|
616
|
-
│ ├── seccomp-apply.ts seccomp-BPF application via C helper
|
|
617
236
|
│ ├── proxy-tools.ts memory/skill tools proxied to host via IPC
|
|
618
237
|
│ └── worker.ts Agent entry point inside sandbox
|
|
619
238
|
├── credentials/ AES-256-GCM encrypted store + credential proxy
|
|
@@ -621,24 +240,15 @@ src/
|
|
|
621
240
|
├── runtime/ createRuntime() — single-user mode
|
|
622
241
|
│ ├── failover-error.ts Error classification (ported from OpenClaw)
|
|
623
242
|
│ ├── model-fallback.ts Fallback chains + cooldown tracking
|
|
624
|
-
│ ├── context-guard.ts Context window guards (16K/32K thresholds)
|
|
625
|
-
│ ├── retry.ts Exponential backoff with jitter + retryAfterMs
|
|
626
243
|
│ ├── api-key-rotation.ts Per-provider key rotation
|
|
627
244
|
│ ├── subagent.ts Parallel execution + lifecycle + registry
|
|
628
|
-
│
|
|
629
|
-
│ ├── tool-result-truncation.ts Truncate oversized tool output
|
|
630
|
-
│ ├── memory-flush.ts Pre-compaction memory save
|
|
631
|
-
│ └── stream-adapters.ts Per-provider stream configuration
|
|
245
|
+
│ └── ... context-guard, retry, session-pruning, memory-flush
|
|
632
246
|
├── memory/ Hybrid search engine
|
|
633
247
|
│ ├── hybrid.ts Vector + FTS5 fusion scoring
|
|
634
248
|
│ ├── mmr.ts Maximal Marginal Relevance re-ranking
|
|
635
|
-
│ ├──
|
|
636
|
-
│ ├──
|
|
637
|
-
│
|
|
638
|
-
│ ├── embedding-batch.ts Batch embedding with retry
|
|
639
|
-
│ ├── file-indexer.ts File-based memory indexing with fs.watch
|
|
640
|
-
│ ├── session-indexer.ts Session transcript indexing
|
|
641
|
-
│ └── ssrf.ts SSRF protection for HTTP calls
|
|
249
|
+
│ ├── embeddings.ts 5-provider embedding support
|
|
250
|
+
│ ├── ssrf.ts SSRF protection for HTTP calls
|
|
251
|
+
│ └── ... temporal-decay, query-expansion, cache, indexers
|
|
642
252
|
├── hooks/ 6 lifecycle hook points
|
|
643
253
|
├── logging/ Structured JSON logging with file output
|
|
644
254
|
├── sessions/ File-based store with async locking
|
|
@@ -674,25 +284,21 @@ Every file is workspace-scoped. No shared state between tenants.
|
|
|
674
284
|
|
|
675
285
|
## Built on OpenClaw
|
|
676
286
|
|
|
677
|
-
Bulkhead Runtime stands on the shoulders of [OpenClaw](https://github.com/nicepkg/openclaw). Several production-critical subsystems were ported directly
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
| Subsystem | Origin |
|
|
681
|
-
| ----------------------------------------- | ------------------------------------------------------------------------- |
|
|
682
|
-
| **Model fallback & error classification** | `src/agents/model-fallback.ts`, `failover-error.ts`, `failover-policy.ts` |
|
|
683
|
-
| **API key rotation** | `src/agents/api-key-rotation.ts`, `live-auth-keys.ts` |
|
|
684
|
-
| **Context window guards** | `src/agents/context-window-guard.ts` |
|
|
685
|
-
| **Retry with backoff** | `src/infra/retry.ts` |
|
|
686
|
-
| **SSRF protection** | `src/infra/net/ssrf.ts`, `fetch-guard.ts` |
|
|
687
|
-
| **Embedding cache & batch** | `extensions/memory-core/src/memory/manager-embedding-ops.ts` |
|
|
688
|
-
| **File indexer** | `extensions/memory-core/src/memory/manager-sync-ops.ts` |
|
|
689
|
-
| **Session transcript indexer** | `extensions/memory-core/src/memory/manager-sync-ops.ts` |
|
|
690
|
-
| **Subagent orchestration** | `src/agents/subagent-*.ts` |
|
|
691
|
-
| **Structured logging** | `src/logging/logger.ts`, `subsystem.ts`, `levels.ts` |
|
|
692
|
-
| **Transcript events** | `src/sessions/transcript-events.ts` |
|
|
287
|
+
Bulkhead Runtime stands on the shoulders of [OpenClaw](https://github.com/nicepkg/openclaw). Several production-critical subsystems were ported directly:
|
|
693
288
|
|
|
289
|
+
| Subsystem | Origin |
|
|
290
|
+
|-----------|--------|
|
|
291
|
+
| **Model fallback & error classification** | `src/agents/model-fallback.ts`, `failover-error.ts` |
|
|
292
|
+
| **API key rotation** | `src/agents/api-key-rotation.ts`, `live-auth-keys.ts` |
|
|
293
|
+
| **Context window guards** | `src/agents/context-window-guard.ts` |
|
|
294
|
+
| **Retry with backoff** | `src/infra/retry.ts` |
|
|
295
|
+
| **SSRF protection** | `src/infra/net/ssrf.ts`, `fetch-guard.ts` |
|
|
296
|
+
| **Embedding cache & batch** | `extensions/memory-core/` |
|
|
297
|
+
| **File & session indexers** | `extensions/memory-core/` |
|
|
298
|
+
| **Subagent orchestration** | `src/agents/subagent-*.ts` |
|
|
299
|
+
| **Structured logging** | `src/logging/logger.ts`, `subsystem.ts` |
|
|
694
300
|
|
|
695
|
-
OpenClaw solved these problems in production. We extracted, adapted, and integrated them into Bulkhead's multi-tenant architecture.
|
|
301
|
+
OpenClaw solved these problems in production. We extracted, adapted, and integrated them into Bulkhead's multi-tenant architecture.
|
|
696
302
|
|
|
697
303
|
---
|
|
698
304
|
|
|
@@ -704,4 +310,4 @@ OpenClaw solved these problems in production. We extracted, adapted, and integra
|
|
|
704
310
|
|
|
705
311
|
## License
|
|
706
312
|
|
|
707
|
-
[MIT](LICENSE)
|
|
313
|
+
[MIT](LICENSE)
|
package/dist/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "bulkhead-runtime",
|
|
3
|
-
"version": "2026.4.5-beta.
|
|
3
|
+
"version": "2026.4.5-beta.8",
|
|
4
4
|
"description": "Multi-tenant AI agent runtime with OS-level isolation. Sandboxed execution, encrypted credentials, private memory per tenant — one server, no Docker.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -29,6 +29,35 @@
|
|
|
29
29
|
"@mariozechner/pi-ai": "0.55.3",
|
|
30
30
|
"@mariozechner/pi-coding-agent": "0.55.3"
|
|
31
31
|
},
|
|
32
|
+
"keywords": [
|
|
33
|
+
"ai-agent",
|
|
34
|
+
"ai-agents",
|
|
35
|
+
"agent",
|
|
36
|
+
"agent-runtime",
|
|
37
|
+
"autonomous-agent",
|
|
38
|
+
"multi-agent",
|
|
39
|
+
"ai",
|
|
40
|
+
"llm",
|
|
41
|
+
"ai-infrastructure",
|
|
42
|
+
"ai-security",
|
|
43
|
+
"agent-security",
|
|
44
|
+
"mcp",
|
|
45
|
+
"model-context-protocol",
|
|
46
|
+
"tool-use",
|
|
47
|
+
"sandbox",
|
|
48
|
+
"secure-sandbox",
|
|
49
|
+
"sandboxing",
|
|
50
|
+
"runtime",
|
|
51
|
+
"isolation",
|
|
52
|
+
"process-isolation",
|
|
53
|
+
"tenant-isolation",
|
|
54
|
+
"multi-tenant",
|
|
55
|
+
"multitenant",
|
|
56
|
+
"security",
|
|
57
|
+
"credentials",
|
|
58
|
+
"encrypted-credentials",
|
|
59
|
+
"containerless"
|
|
60
|
+
],
|
|
32
61
|
"engines": {
|
|
33
62
|
"node": ">=22.12.0"
|
|
34
63
|
},
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "bulkhead-runtime",
|
|
3
|
-
"version": "2026.4.5-beta.
|
|
3
|
+
"version": "2026.4.5-beta.8",
|
|
4
4
|
"description": "Multi-tenant AI agent runtime with OS-level isolation. Sandboxed execution, encrypted credentials, private memory per tenant — one server, no Docker.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -45,6 +45,35 @@
|
|
|
45
45
|
"typescript": "^5.9.3",
|
|
46
46
|
"vitest": "^4.1.2"
|
|
47
47
|
},
|
|
48
|
+
"keywords": [
|
|
49
|
+
"ai-agent",
|
|
50
|
+
"ai-agents",
|
|
51
|
+
"agent",
|
|
52
|
+
"agent-runtime",
|
|
53
|
+
"autonomous-agent",
|
|
54
|
+
"multi-agent",
|
|
55
|
+
"ai",
|
|
56
|
+
"llm",
|
|
57
|
+
"ai-infrastructure",
|
|
58
|
+
"ai-security",
|
|
59
|
+
"agent-security",
|
|
60
|
+
"mcp",
|
|
61
|
+
"model-context-protocol",
|
|
62
|
+
"tool-use",
|
|
63
|
+
"sandbox",
|
|
64
|
+
"secure-sandbox",
|
|
65
|
+
"sandboxing",
|
|
66
|
+
"runtime",
|
|
67
|
+
"isolation",
|
|
68
|
+
"process-isolation",
|
|
69
|
+
"tenant-isolation",
|
|
70
|
+
"multi-tenant",
|
|
71
|
+
"multitenant",
|
|
72
|
+
"security",
|
|
73
|
+
"credentials",
|
|
74
|
+
"encrypted-credentials",
|
|
75
|
+
"containerless"
|
|
76
|
+
],
|
|
48
77
|
"engines": {
|
|
49
78
|
"node": ">=22.12.0"
|
|
50
79
|
},
|