@enclave-vm/core 2.19.1 → 2.19.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. package/README.md +262 -105
  2. package/adapters/worker-pool/config.d.ts +15 -5
  3. package/adapters/worker-pool/protocol.d.ts +27 -1
  4. package/adapters/worker-pool/worker-pool-adapter.d.ts +24 -0
  5. package/adapters/worker-pool/worker-slot.d.ts +35 -6
  6. package/async-context.d.ts +34 -0
  7. package/babel/babel-wrapper.d.ts +21 -12
  8. package/babel/multi-file-transform.d.ts +3 -0
  9. package/babel/text-scan.d.ts +54 -0
  10. package/deadline-guard.d.ts +16 -0
  11. package/dynamic-import.d.ts +25 -0
  12. package/enclave-options.d.ts +62 -0
  13. package/enclave.d.ts +31 -7
  14. package/esm/index.mjs +4649 -774
  15. package/esm/worker.mjs +1136 -134
  16. package/execution-deadline.d.ts +6 -0
  17. package/global-function-gate.d.ts +34 -1
  18. package/index.d.ts +3 -3
  19. package/index.js +4690 -815
  20. package/interpreter/interpreter.d.ts +204 -6
  21. package/memory-tracker.d.ts +7 -0
  22. package/package.json +3 -3
  23. package/realm-airlock.d.ts +6 -1
  24. package/realm-memory.d.ts +89 -0
  25. package/realm-stack.d.ts +25 -0
  26. package/safe-runtime.d.ts +2 -0
  27. package/sandbox-rejections.d.ts +3 -1
  28. package/scoring/feature-extractor.d.ts +52 -1
  29. package/scoring/index.d.ts +2 -1
  30. package/scoring/scorers/index.d.ts +2 -1
  31. package/scoring/scorers/local-llm.scorer.d.ts +9 -3
  32. package/scoring/scorers/progressive.scorer.d.ts +28 -1
  33. package/scoring/scorers/rule-based.scorer.d.ts +11 -1
  34. package/scoring/types.d.ts +36 -8
  35. package/secure-proxy.d.ts +17 -3
  36. package/session/session.d.ts +8 -1
  37. package/session-types.d.ts +6 -4
  38. package/sidecar/index.d.ts +1 -1
  39. package/sidecar/reference-config.d.ts +18 -1
  40. package/sidecar/reference-resolver.d.ts +19 -1
  41. package/sidecar/reference-sidecar.d.ts +37 -0
  42. package/src/adapters/worker-pool/package.json +3 -3
  43. package/src/adapters/worker-pool/worker-script.js +2250 -428
  44. package/thrown-value.d.ts +59 -0
  45. package/types.d.ts +75 -19
  46. package/value-sanitizer.d.ts +6 -1
  47. package/worker.js +1136 -134
package/README.md CHANGED
@@ -17,7 +17,7 @@ The @enclave-vm/core package provides a hardened execution environment for runni
17
17
  | CVE Protection | Covers known vm2, isolated-vm, node-vm exploits |
18
18
  | Defense Layers | 6 (Pre-Scanner, AST, Transform, Scoring, VM/Worker Pool, Sanitization) |
19
19
 
20
- For the full security audit report, see [SECURITY-AUDIT.md](./SECURITY-AUDIT.md).
20
+ For the full security audit report, see [SECURITY-AUDIT.md](./SECURITY-AUDIT.md). The invariants of the sandbox by layer, and the ones that are still open, are listed in [SECURITY-TESTS.md](../../SECURITY-TESTS.md): in process the time and memory limits are the enclave's own accounting, not the engine's (see [How `timeout` Is Enforced](#how-timeout-is-enforced) and [What `memoryLimit` Counts](#what-memorylimit-counts)).
21
21
 
22
22
  ## Features
23
23
 
@@ -26,8 +26,8 @@ For the full security audit report, see [SECURITY-AUDIT.md](./SECURITY-AUDIT.md)
26
26
  - **Safe Tool Calls**: Secure `callTool()` execution with iteration and call limits
27
27
  - **Reference Sidecar**: Handle large data without embedding it in the script
28
28
  - **Configurable Security Levels**: From STRICT to PERMISSIVE based on trust level
29
- - **Resource Limits**: Timeout, iteration limits, and tool call limits
30
- - **AI Scoring Gate**: Semantic analysis detects exfiltration patterns and sensitive data access
29
+ - **Resource Limits**: Timeout, memory accounting, iteration limits, and tool call limits
30
+ - **AI Scoring Gate**: Optional heuristic risk score for a script, from its tool calls and argument names
31
31
  - **Zero Dependencies on Vulnerable Packages**: Uses Node.js native `vm` module
32
32
 
33
33
  ## Installation
@@ -80,13 +80,15 @@ enclave.dispose();
80
80
  ```typescript
81
81
  interface CreateEnclaveOptions {
82
82
  // Execution limits
83
- timeout?: number; // Execution timeout in ms (default: 5000)
83
+ timeout?: number; // Execution timeout in ms (default: 30000; by security level)
84
84
  toolCallGraceMs?: number; // How long a tool call in progress at the timeout may still take (default: same as timeout)
85
- maxToolCalls?: number; // Maximum tool calls allowed (default: 100)
86
- maxIterations?: number; // Maximum loop iterations (default: 10000)
85
+ maxToolCalls?: number; // Maximum tool calls allowed (default: 100; by security level)
86
+ maxIterations?: number; // Maximum loop iterations (default: 10000; by security level)
87
+ memoryLimit?: number; // Memory limit in bytes, as the enclave counts it (default: 1048576, 1 MB; 0: off)
87
88
 
88
89
  // Security
89
- securityLevel?: SecurityLevel; // STRICT | SECURE | STANDARD | PERMISSIVE
90
+ securityLevel?: SecurityLevel; // 'STRICT' | 'SECURE' | 'STANDARD' | 'PERMISSIVE' (default: 'STANDARD')
91
+ preset?: AstPreset; // AST validation preset (default: 'agentscript')
90
92
  validate?: boolean; // Enable AST validation (default: true)
91
93
  transform?: boolean; // Enable code transformation (default: true)
92
94
  realmGlobals?: 'allowlist' | 'security-level'; // What exists in the script's realm (see "What the Script's Realm Holds")
@@ -115,11 +117,39 @@ interface CreateEnclaveOptions {
115
117
  // Sidecar (large data handling)
116
118
  sidecar?: ReferenceSidecarOptions;
117
119
 
120
+ // Scoring gate
121
+ scoringGate?: ScoringGateConfig; // See "AI Scoring Gate" section
122
+
118
123
  // Double VM security layer
119
124
  doubleVm?: PartialDoubleVmConfig; // See "Double VM Security Layer" section
120
125
  }
121
126
  ```
122
127
 
128
+ ### Option Values
129
+
130
+ The constructor checks its options and throws a `TypeError` for a value outside the documented type or range:
131
+
132
+ - A limit is a finite integer in its range: `timeout` from 1 to 2147483647 (the longest delay a timer holds); `maxIterations`, `maxSanitizeDepth` and `maxSanitizeProperties` at least 1; `maxToolCalls`, `maxConsoleCalls`, `maxConsoleOutputBytes`, `maxGlobalFunctionCalls` and `memoryLimit` at least 0. `Infinity`, `NaN` and a value of another type (a numeric string, `null`) are refused. The one exception is `toolCallGraceMs`, a number of at least 0 for which `Infinity` means "wait without limit".
133
+ - A switch is a boolean: `validate`, `transform`, `sanitizeStackTraces`, `allowFunctionsInGlobals`, `allowBuiltins`, `doubleVm.enabled`, and the switches of `secureProxyConfig` and `doubleVm.parentValidation`.
134
+ - `securityLevel` is one of the four levels.
135
+ - An option passed as `undefined` is an option that was not set: it has its default, which is the security level's value where the level sets one. A wrapper can forward `{ maxIterations: opts.maxIterations }` whether or not its caller set it.
136
+
137
+ ### Limits on the Script Itself
138
+
139
+ - **Size.** A script is at most 5,242,880 characters. A longer one, or a script that is not a string, fails with `VALIDATION_ERROR` before anything parses it.
140
+ - **Dynamic import.** A script that contains `import()` or `import.meta` is refused before it runs. Every preset refuses it during validation; with `validate: false` it fails with `SECURITY_VIOLATION`. The word in a string, a comment or a property name is not the construct.
141
+ - **Iteration limit.** A script that declares `__maxIterations`, the runtime's name for the limit, is refused. With validation off the limit still holds: each loop is compared with a number in its own code.
142
+
143
+ ### What `memoryLimit` Counts
144
+
145
+ `memoryLimit` (default 1 MB; `0` switches it off) is the enclave's own accounting of the operations that make data grow. It is not a limit of the engine's heap.
146
+
147
+ - **Text**: concatenation (`a + b`, and `x += y`, which is charged like `x + y`) and the methods that multiply their input (`repeat`, `padStart`, `padEnd`, `join`).
148
+ - **Arrays**: no single array is larger than the limit holds, at 8 bytes per element, and what an operation adds is charged: `push`, `unshift`, `splice`, `fill`, the elements `concat`, `flat`, `flatMap` and a spread literal add, and what `Array.from` makes out of something that is not an array.
149
+ - **Not counted**: a copy is free, so a script can hold many arrays that are each within the limit; object literals, Maps and Sets are not counted.
150
+
151
+ A script that goes over the limit fails with `MEMORY_LIMIT_EXCEEDED`, and a script cannot switch the accounting off or reset it. With a limit set, a returned value whose serialized size is estimated above the limit (above 50 MB where the limit is larger) fails with `SERIALIZATION_LIMIT_EXCEEDED`. The limit is an estimate: where the host must survive whatever a script allocates, use the [worker pool](#worker-pool-adapter-optional), whose workers have a heap limit of the engine, or a child process or container with a memory limit of the operating system.
152
+
123
153
  ### Security Levels
124
154
 
125
155
  | Level | Description | Use Case |
@@ -130,10 +160,10 @@ interface CreateEnclaveOptions {
130
160
  | `PERMISSIVE` | Adds console for debugging | Internal/development |
131
161
 
132
162
  ```typescript
133
- import { Enclave, SecurityLevel } from '@enclave-vm/core';
163
+ import { Enclave } from '@enclave-vm/core';
134
164
 
135
165
  const enclave = new Enclave({
136
- securityLevel: SecurityLevel.STRICT,
166
+ securityLevel: 'STRICT',
137
167
  toolHandler: async (name, args) => {
138
168
  /* ... */
139
169
  },
@@ -240,7 +270,7 @@ import { Enclave } from '@enclave-vm/core';
240
270
 
241
271
  const enclave = new Enclave({
242
272
  preset: 'agentscript', // AST validation preset (default)
243
- securityLevel: SecurityLevel.STRICT, // Runtime security level
273
+ securityLevel: 'STRICT', // Runtime security level
244
274
  toolHandler: async (name, args) => {
245
275
  /* ... */
246
276
  },
@@ -312,23 +342,49 @@ A worker thread is **not a sandbox by itself**: it has the whole Node.js API and
312
342
 
313
343
  ### Worker Pool Configuration Options
314
344
 
315
- | Option | Default | Description |
316
- | ------------------------ | --------- | ------------------------------------------------------- |
317
- | `minWorkers` | 2 | Minimum warm workers |
318
- | `maxWorkers` | CPU count | Maximum concurrent workers |
319
- | `memoryLimitPerWorker` | 128MB | Heap limit of each worker (its old generation); 0: none |
320
- | `maxMessagesPerSecond` | 1000 | Rate limit per worker |
321
- | `maxExecutionsPerWorker` | 1000 | Recycle after N executions |
322
- | `maxQueueSize` | 100 | Maximum pending executions |
323
- | `idleTimeoutMs` | 30000 | Time before idle worker release |
345
+ | Option | Default | Description |
346
+ | ------------------------ | --------- | ---------------------------------------------------------------- |
347
+ | `minWorkers` | 2 | Minimum warm workers |
348
+ | `maxWorkers` | CPU count | Maximum concurrent workers |
349
+ | `memoryLimitPerWorker` | 128MB | Heap limit of each worker (its old generation); 0: none |
350
+ | `maxMessagesPerSecond` | 1000 | Messages a worker may send per second while it runs an execution |
351
+ | `maxPendingToolCalls` | 100 | Tool calls one execution may have in flight |
352
+ | `maxMessageSizeBytes` | 16MB | Largest single message a worker may send |
353
+ | `maxExecutionsPerWorker` | 1000 | Recycle after N executions |
354
+ | `maxQueueSize` | 100 | Maximum pending executions |
355
+ | `queueTimeoutMs` | 30000 | Longest wait for a free worker |
356
+ | `idleTimeoutMs` | 30000 | Time before idle worker release |
357
+
358
+ These are the defaults of `WorkerPoolConfig`. An enclave starts from the preset of its security level (the table above, exported as `WORKER_POOL_PRESETS`), and `workerPoolConfig` overrides single settings of it.
324
359
 
325
360
  Executions beyond `maxWorkers` wait in a queue (`maxQueueSize`, `queueTimeoutMs`). The wait counts against the script's `timeout`, so size `maxWorkers` for the concurrency you expect.
326
361
 
362
+ `maxMessagesPerSecond` is counted per execution: what an earlier execution sent from the same worker is not held against the next one, and the result and the worker's confirmation that it is idle are not counted. `maxMessageSizeBytes` is applied before a message is parsed. The limit an execution gets is never below what the enclave's own limits let one message carry (a tool request of `toolBridge.maxPayloadBytes`, a result of `memoryLimit`), so it decides where those are larger than it or off: with `memoryLimit: 0` it is the only bound on the size of a result.
363
+
364
+ ### How a Worker Execution Ends
365
+
366
+ A worker runs one execution at a time. After its result it confirms that it is idle, which it can only do when nothing of the script keeps its thread busy, and only then is it given to the next execution. A worker that does not confirm within one second is terminated and replaced; the result it sent stands.
367
+
368
+ An execution the pool ends itself ends its worker too, and the pool starts a new one. `run()` then returns `error.code` `EXECUTION_ERROR` with one of these in `error.name`:
369
+
370
+ | `error.name` | When |
371
+ | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
372
+ | `WorkerTimeoutError` | The script was still running five seconds after its `timeout` |
373
+ | `WorkerMemoryError` | The worker held more than its memory budget while it executed (see [The Worker Heap Limit](#the-worker-heap-limit)) |
374
+ | `MessageFloodError` | The execution sent more than `maxMessagesPerSecond` messages |
375
+ | `MessageSizeError` | A message from the worker was larger than the limit |
376
+ | `TooManyPendingCallsError` | More than `maxPendingToolCalls` tool calls were in flight |
377
+ | `MessageValidationError` | A message from the worker did not pass the protocol schema |
378
+ | `ExecutionAbortedError` | The pool was disposed while the execution was in flight, or the script's time had passed before it started (it is not run) |
379
+ | `QueueFullError`, `QueueTimeoutError` | No worker became free: the queue was full, or the wait exceeded `queueTimeoutMs` |
380
+
381
+ `dispose()` ends the executions in flight with `ExecutionAbortedError` and terminates every worker.
382
+
327
383
  ### The Worker Heap Limit
328
384
 
329
385
  `memoryLimitPerWorker` is the limit of the worker's old generation, passed to the engine as the worker's `resourceLimits`. The engine allows the young generation on top of it, and the pool sets that to 24 MB, so the most a worker can hold is the sum of the two: its budget. (Left to the engine the young generation of such a worker is 192 MB on Node.js 24 and 48 MB on Node.js 22, measured on a 64-bit machine.)
330
386
 
331
- **A heap flag on the process replaces it.** When the host process runs with `--max-old-space-size`, on the command line or in `NODE_OPTIONS`, that value becomes the heap limit of every worker thread, whatever limit a worker was created with (this is Node.js behaviour; `worker.resourceLimits` goes on reporting the limit that was asked for). The pool therefore reads the actual limit of each new worker. When the engine does not apply the configured one, the pool logs a warning once and enforces the limit itself: while a worker executes, it reads the worker's heap every 100 ms and terminates the worker when its heap and external memory exceed the budget. The execution fails with `WorkerMemoryError`. This is approximate, because a script can allocate between two reads: with a 64 MB limit and `--max-old-space-size=4096`, a script that kept allocating was stopped when its worker held about 290 MB, with the process at 0.5 GB resident; without the check it went on until the process was at 4.4 GB. Reading a worker's heap from outside needs Node.js 22.16 or later; on an older 22.x the pool can only warn that the limit is not enforced.
387
+ **A heap flag on the process replaces it.** When the host process runs with `--max-old-space-size`, on the command line or in `NODE_OPTIONS`, that value becomes the heap limit of every worker thread, whatever limit a worker was created with (this is Node.js behaviour; `worker.resourceLimits` goes on reporting the limit that was asked for). The pool therefore reads the actual limit of each new worker. When the engine does not apply the configured one, the pool logs a warning once and enforces the limit itself: while a worker executes, it reads the worker's heap every 100 ms and terminates the worker when its heap and external memory exceed the budget. The execution fails with `WorkerMemoryError`. Where the engine does enforce the heap limit, the same read bounds what the engine does not count: memory the worker holds outside its heap counts against the budget while it executes. This is approximate, because a script can allocate between two reads: with a 64 MB limit and `--max-old-space-size=4096`, a script that kept allocating was stopped when its worker held about 290 MB, with the process at 0.5 GB resident; without the check it went on until the process was at 4.4 GB. Reading a worker's heap from outside needs Node.js 22.16 or later; on an older 22.x the pool can only warn that the limit is not enforced.
332
388
 
333
389
  **The memory monitor** (`memoryCheckIntervalMs`) asks every worker for its memory use and recycles a worker that holds more than its budget. It compares what the worker itself holds, its heap plus its external memory. (`rss` is no measure of a worker: inside a worker thread it is the resident size of the whole process.)
334
390
 
@@ -360,7 +416,8 @@ Executions beyond `maxWorkers` wait in a queue (`maxQueueSize`, `queueTimeoutMs`
360
416
  - **`resourceLimits`**: a heap limit per worker, enforced by V8, or by the pool when a heap flag of the process overrides it ([The Worker Heap Limit](#the-worker-heap-limit))
361
417
  - **The host does not trust a worker**: every message is parsed without prototype pollution and checked against the protocol schema (types, identifier and tool-name patterns, sizes); a message that fails ends the execution and replaces the worker
362
418
  - **JSON-only messages**: no structured clone of objects in either direction
363
- - **Rate limiting**: message flood protection per worker
419
+ - **Rate and size limits**: the messages a worker may send per second, counted per execution, and the size of one message
420
+ - **Idle confirmation**: a worker is given to the next execution only when it has confirmed that nothing of the last script is left in it
364
421
  - **Pool recovery**: a worker that is terminated or dies is removed and replaced, so the pool is not left short. Running out of memory is the one way a worker can still take the process with it ([The Worker Heap Limit](#the-worker-heap-limit))
365
422
 
366
423
  ## Double VM Security Layer
@@ -473,7 +530,7 @@ const enclave = new Enclave({
473
530
 
474
531
  ## AI Scoring Gate
475
532
 
476
- The AI Scoring Gate provides semantic security analysis to detect attack patterns beyond what static AST validation can catch. It analyzes tool call patterns, sensitive data access, and exfiltration attempts.
533
+ The AI Scoring Gate gives a script a risk score from 0 to 100 before it runs and refuses the script when the score reaches a threshold. The score is a heuristic over features of the script's text: the tools it calls, their argument names and literal values, their order, and the loops they are in. It flags patterns that look like exfiltration, bulk access or access to sensitive fields; a low score is not evidence that a script is safe. The gate runs after AST validation and before execution, and is off unless configured.
477
534
 
478
535
  ### Configuration
479
536
 
@@ -482,31 +539,54 @@ const enclave = new Enclave({
482
539
  toolHandler: async (name, args) => {
483
540
  /* ... */
484
541
  },
485
- scoring: {
542
+ scoringGate: {
486
543
  scorer: 'rule-based', // 'disabled' | 'rule-based' | 'local-llm' | 'external-api'
487
- blockThreshold: 70, // Block if score >= 70
488
- warnThreshold: 40, // Warn if score >= 40
544
+ blockThreshold: 70, // Do not run the script if score >= 70
545
+ warnThreshold: 40, // Set scoringResult.warned if score >= 40
546
+ failOpen: true, // Run the script when scoring fails (default)
489
547
  },
490
548
  });
491
549
  ```
492
550
 
551
+ | Option | Default | Description |
552
+ | ---------------- | ------------------------------ | ----------------------------------------------------------------------------------------------- |
553
+ | `scorer` | `'disabled'` | Which scorer answers |
554
+ | `scoring` | - | Progressive scoring: the rule-based scorer first, a detailed scorer for scripts that score high |
555
+ | `blockThreshold` | 70 | A script with a score at or above it is not run |
556
+ | `warnThreshold` | 40 | A script with a score at or above it runs, with `scoringResult.warned` set |
557
+ | `failOpen` | `true` | Whether a script runs when the scorer cannot give its answer |
558
+ | `customRules` | - | Base scores of the rule-based scorer by rule id (0-100; 0 turns a rule off) |
559
+ | `toolNamespaces` | the enclave's `toolNamespaces` | The namespaces whose method calls are scored as tool calls |
560
+ | `localLlm` | - | Configuration of the `local-llm` scorer |
561
+ | `externalApi` | - | `endpoint`, `apiKey`, `timeoutMs`, `retries`, `headers` of the `external-api` scorer |
562
+ | `cache` | enabled, 5 minutes, 1,000 | Cache of scoring results by the SHA-256 digest of the script |
563
+ | `verbose` | `false` | Log scores and signals to the console |
564
+
565
+ A scoring failure is anything that keeps the configured scorer from giving its answer: a script the feature extractor cannot parse (`FeatureExtractionError`), a scorer that throws, and in progressive mode a detailed scorer that fails or is missing once the fast score called for it. With `failOpen: false` each of these refuses the script; with `failOpen: true` the script runs.
566
+
567
+ A refused script fails with `SCORING_BLOCKED`; `error.data` holds `totalScore`, `riskLevel` and `signals`. A result of a script the gate evaluated carries `scoringResult`.
568
+
569
+ A call through a tool namespace (`mail.send(args)`) is scored as the tool call it stands for (`callTool('mail.send', args)`): the gate of an enclave knows the enclave's `toolNamespaces`.
570
+
493
571
  ### Scorer Types
494
572
 
495
- | Type | Description | Latency |
496
- | -------------- | ------------------------------------------ | ------- |
497
- | `disabled` | No scoring (pass-through) | 0ms |
498
- | `rule-based` | Pure TypeScript rules, zero dependencies | ~1ms |
499
- | `local-llm` | On-device ML with HuggingFace transformers | ~5-10ms |
500
- | `external-api` | External API-based scoring | ~100ms |
573
+ | Type | What the score is computed from |
574
+ | -------------- | --------------------------------------------------------------------------------------------------------- |
575
+ | `disabled` | No scoring |
576
+ | `rule-based` | Eight TypeScript rules over the extracted features, zero dependencies |
577
+ | `local-llm` | Your `customAnalyzer`, or a built-in keyword heuristic over the extracted features (no model computes it) |
578
+ | `external-api` | The answer of your scoring service, which receives the extracted features |
579
+
580
+ The `local-llm` scorer loads a Hugging Face model at initialization when the optional dependency `@huggingface/transformers` is installed, but it never calls it: no score is computed from the model. Set `ENCLAVE_DISABLE_LOCAL_LLM_MODEL=1` to skip the load. Its `mode: 'similarity'` searches a VectoriaDB index that is created empty and that the configuration cannot fill, so it scores like the default mode.
501
581
 
502
582
  ### Custom Analyzer (Extensibility)
503
583
 
504
- For advanced use cases, you can provide a custom analyzer to integrate external LLMs or static code analyzers:
584
+ A custom analyzer replaces the keyword heuristic of the `local-llm` scorer, for example with a call to a model or a static analyzer of your own:
505
585
 
506
586
  ```typescript
507
- import { Enclave, CustomAnalyzer } from '@enclave-vm/core';
587
+ import { Enclave, type LocalLlmConfig } from '@enclave-vm/core';
508
588
 
509
- const myAnalyzer: CustomAnalyzer = {
589
+ const myAnalyzer: NonNullable<LocalLlmConfig['customAnalyzer']> = {
510
590
  async analyze(prompt, features) {
511
591
  // Call your external LLM or static analyzer
512
592
  const response = await fetch('https://my-analyzer-api.com/score', {
@@ -559,17 +639,15 @@ interface CustomAnalyzer {
559
639
 
560
640
  ## Reference Sidecar
561
641
 
562
- The Reference Sidecar is a powerful feature for handling large data in AgentScript without embedding it directly in the script. This is essential when:
563
-
564
- - Tool responses contain large datasets
565
- - You need to pass large data between tool calls
566
- - Script size must be kept small for security validation
642
+ The Reference Sidecar keeps large strings outside the sandbox. The script holds a short reference token in place of the data and can pass it on to another tool. This is for scripts that move large data from one tool to another, not for scripts that read it.
567
643
 
568
644
  ### How It Works
569
645
 
570
- 1. **Large Data Extraction**: When a tool returns large data, it's stored in a sidecar storage and replaced with a reference token (`__ref_abc123`)
571
- 2. **Lazy Resolution**: When the script accesses the reference, it's resolved just-in-time
572
- 3. **Safe Property Access**: Only specific properties can be accessed, preventing data exfiltration
646
+ 1. **Storing**: A string of at least `extractionThreshold` bytes is stored in the sidecar and replaced by a reference token (`__REF_<uuid>__`). This applies to a string literal in the script's source and to a tool result that is a string as a whole. A tool that returns an object or an array is returned as it is, with every string in it.
647
+ 2. **The script holds a token**: Nothing is resolved for the script, lazily or otherwise. The token is an ordinary short string, and the script has no way to read the stored data.
648
+ 3. **Resolving at the tool boundary**: A token in the arguments of a tool call, at any depth, is replaced with the stored data before the `toolHandler` is called.
649
+
650
+ The sidecar belongs to one execution and is disposed when `run()` returns.
573
651
 
574
652
  ### Enabling Sidecar
575
653
 
@@ -582,23 +660,36 @@ const enclave = new Enclave({
582
660
  enabled: true,
583
661
  maxTotalSize: 10 * 1024 * 1024, // 10MB total storage
584
662
  maxReferenceSize: 1 * 1024 * 1024, // 1MB per reference
585
- extractionThreshold: 1024, // Extract strings > 1KB
586
- maxResolvedSize: 5 * 1024 * 1024, // 5MB max resolved size
587
- allowComposites: false, // Block string concatenation with refs
663
+ extractionThreshold: 1024, // Store strings of 1KB and more
664
+ maxResolvedSize: 5 * 1024 * 1024, // 5MB resolved for one tool call
665
+ allowComposites: false, // Refuse string concatenation with refs
588
666
  },
589
667
  });
590
668
  ```
591
669
 
592
670
  ### Sidecar Options
593
671
 
594
- | Option | Type | Default | Description |
595
- | --------------------- | --------- | ------- | ------------------------------------------------ |
596
- | `enabled` | `boolean` | `false` | Enable the sidecar feature |
597
- | `maxTotalSize` | `number` | 10MB | Maximum total size of all stored references |
598
- | `maxReferenceSize` | `number` | 1MB | Maximum size of a single reference |
599
- | `extractionThreshold` | `number` | 1024 | Minimum string size to extract to sidecar |
600
- | `maxResolvedSize` | `number` | 5MB | Maximum size when resolving references |
601
- | `allowComposites` | `boolean` | `false` | Allow string concatenation with reference tokens |
672
+ | Option | Type | Default | Description |
673
+ | ---------------------- | --------- | -------------------- | ------------------------------------------------------------------------ |
674
+ | `enabled` | `boolean` | `false` | Enable the sidecar feature |
675
+ | `maxTotalSize` | `number` | by level | Maximum total size of all stored references |
676
+ | `maxReferenceSize` | `number` | by level | Maximum size of a single reference |
677
+ | `extractionThreshold` | `number` | by level | Size in bytes from which a string is stored |
678
+ | `maxResolvedSize` | `number` | by level | Maximum size the references in the arguments of one tool call resolve to |
679
+ | `maxTotalResolvedSize` | `number` | twice `maxTotalSize` | Maximum size of everything one execution resolves for its tool calls |
680
+ | `maxReferenceCount` | `number` | by level | Maximum number of references in one execution |
681
+ | `allowComposites` | `boolean` | by level | Allow string concatenation with reference tokens |
682
+
683
+ | Default | STRICT | SECURE | STANDARD | PERMISSIVE |
684
+ | --------------------- | ------- | ------- | -------- | ---------- |
685
+ | `maxTotalSize` | 16 MB | 64 MB | 256 MB | 1 GB |
686
+ | `maxReferenceSize` | 4 MB | 16 MB | 64 MB | 256 MB |
687
+ | `extractionThreshold` | 64 KB | 256 KB | 1 MB | 4 MB |
688
+ | `maxResolvedSize` | 8 MB | 32 MB | 128 MB | 512 MB |
689
+ | `maxReferenceCount` | 50 | 100 | 500 | 1,000 |
690
+ | `allowComposites` | `false` | `false` | `true` | `true` |
691
+
692
+ `maxTotalResolvedSize` bounds what one execution resolves in total: a stored value counts once, the first time it is resolved, and a concatenation of references counts every time it is built. A tool call whose arguments would go over it, or over `maxResolvedSize`, is refused (`REFERENCE_RESOLUTION_LIMIT`) and the handler is not called.
602
693
 
603
694
  ### Example with Sidecar
604
695
 
@@ -606,36 +697,38 @@ const enclave = new Enclave({
606
697
  const enclave = new Enclave({
607
698
  sidecar: {
608
699
  enabled: true,
609
- extractionThreshold: 100, // Extract strings > 100 bytes
610
- allowComposites: false, // Block: ref + "suffix" (security measure)
700
+ extractionThreshold: 100, // Store strings of 100 bytes and more
701
+ allowComposites: false, // Refuse: ref + "suffix"
611
702
  },
612
703
  toolHandler: async (name, args) => {
613
- if (name === 'getLargeData') {
614
- // Returns 1MB of data - automatically stored in sidecar
615
- return { data: 'x'.repeat(1024 * 1024) };
704
+ if (name === 'getLargeText') {
705
+ // A 1MB string: stored in the sidecar, the script gets a token
706
+ return 'x'.repeat(1024 * 1024);
707
+ }
708
+ if (name === 'saveText') {
709
+ // args.text is the 1MB string again: the token was resolved
710
+ return { saved: args.text.length };
616
711
  }
617
712
  },
618
713
  });
619
714
 
620
715
  const result = await enclave.run(`
621
- // Large data is transparently handled via sidecar
622
- const response = await callTool('getLargeData', {});
623
-
624
- // Access properties - resolved on demand
625
- return { hasData: response.data.length > 0 };
716
+ const text = await callTool('getLargeText', {});
717
+ // text is a reference token here, not the data
718
+ return await callTool('saveText', { text });
626
719
  `);
627
720
  ```
628
721
 
629
- ### Security Considerations
722
+ ### Concatenation
630
723
 
631
- The `allowComposites: false` setting (default) blocks string concatenation with reference tokens. This prevents attacks like:
724
+ With `allowComposites: false` (the default at `STRICT` and `SECURE`), concatenating a reference token with anything throws inside the script: a token can only be passed on as it is.
632
725
 
633
726
  ```javascript
634
- // BLOCKED when allowComposites: false
635
- const malicious = ref + '__proto__'; // Attempting prototype pollution
727
+ // Throws when allowComposites is false
728
+ const combined = ref + '-suffix';
636
729
  ```
637
730
 
638
- Set `allowComposites: true` only if you need to concatenate strings and understand the security implications.
731
+ With `allowComposites: true` (the default at `STANDARD` and `PERMISSIVE`), the concatenation yields a composite handle, and a tool that is passed the handle receives the concatenated text.
639
732
 
640
733
  ## Tool Namespaces
641
734
 
@@ -801,33 +894,55 @@ interface ExecutionResult<T> {
801
894
  error?: {
802
895
  message: string;
803
896
  name: string;
804
- code?: string; // Error code (e.g., 'VALIDATION_ERROR', 'TIMEOUT')
805
- stack?: string;
897
+ code?: string; // Error code (e.g., 'VALIDATION_ERROR'; see "Error Codes")
898
+ stack?: string; // Only for errors made outside the sandbox
806
899
  data?: Record<string, unknown>;
807
900
  };
808
901
  stats: {
809
902
  duration: number; // Execution time in ms
810
903
  toolCallCount: number; // Number of tool calls made
811
904
  iterationCount: number; // Number of loop iterations
905
+ memoryUsage?: number; // Peak tracked memory in bytes (when memoryLimit is set)
906
+ startTime: number;
907
+ endTime: number;
812
908
  };
909
+ scoringResult?: ScoringGateResult; // When a scoring gate evaluated the script
813
910
  }
814
911
  ```
815
912
 
816
- With `sanitizeStackTraces` on (the default for `STRICT` and `SECURE`), every returned `error.stack` is
817
- sanitized, whichever stage raised the error (validation, transformation, compilation, execution, tool
818
- calls) and whichever adapter ran the script: stack frames become `at [REDACTED]` and paths, URLs and
819
- file locations in the remaining lines are redacted.
913
+ **`result.error` is plain, bounded text.** Whatever a script threw, `name` and `message` are strings: `name`
914
+ of at most 256 characters, `message` of at most 16,384. A thrown string is the message; a thrown `null`,
915
+ `undefined` or other primitive is named in the message (`The script threw null`); an object without a
916
+ string `message` is reported as `The script threw a value that is not an error`. An object a script makes
917
+ to look like the memory-limit error is not reported as one. The message of a validation failure is cut to
918
+ 500 characters per issue and 32,768 in total; `error.data.issues` holds the issues.
919
+
920
+ **Errors made inside the sandbox have no `stack`.** No sandbox realm captures a stack trace, so
921
+ `error.stack` is `undefined` for the script and absent from `result.error` for what a script throws and
922
+ for what the runtime raises inside the sandbox (a limit, a failed tool call). This holds for the double VM
923
+ and the single VM, in process and in a worker. Errors the host makes outside a sandbox realm (an adapter
924
+ that fails, a worker pool error, an error of the `InterpreterAdapter`, which has no realm of its own) keep
925
+ their stack. With `sanitizeStackTraces` on (the default for `STRICT` and `SECURE`), such a stack is
926
+ sanitized, whichever stage raised the error: stack frames become `at [REDACTED]` and paths, URLs and file
927
+ locations in the remaining lines are redacted.
820
928
 
821
929
  ### Error Codes
822
930
 
823
- | Code | Description |
824
- | ------------------ | -------------------------------------- |
825
- | `VALIDATION_ERROR` | AST validation failed (dangerous code) |
826
- | `TIMEOUT` | Execution exceeded timeout |
827
- | `MAX_TOOL_CALLS` | Exceeded maximum tool calls |
828
- | `MAX_ITERATIONS` | Exceeded maximum loop iterations |
829
- | `TOOL_ERROR` | Tool handler threw an error |
830
- | `RUNTIME_ERROR` | General runtime error in script |
931
+ | Code | Description |
932
+ | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- |
933
+ | `VALIDATION_ERROR` | AST validation failed, or the script is too large or not a string |
934
+ | `SECURITY_VIOLATION` | The script contains a dynamic import (also with `validate: false`), or a blocked operation was recorded at `STRICT` or `SECURE` |
935
+ | `SCORING_BLOCKED` | Refused by the scoring gate |
936
+ | `DOUBLE_VM_EXECUTION_ERROR` | The script failed in the double VM (the default): an error it did not catch, a failed tool call, the tool-call or iteration limit, a timeout |
937
+ | `VM_EXECUTION_ERROR` | The same with `doubleVm: { enabled: false }`; with either VM, the timeout of a script that was only waiting |
938
+ | `MEMORY_LIMIT_EXCEEDED` | `memoryLimit` exceeded |
939
+ | `SERIALIZATION_LIMIT_EXCEEDED` | The returned value is larger than the memory limit allows |
940
+ | `EXECUTION_ERROR` | The adapter itself failed; with the worker pool `error.name` is the pool's error (`WorkerTimeoutError`, ...) |
941
+ | `ENCLAVE_ERROR` | Any other failure outside the sandbox, such as a script with a syntax error |
942
+ | `EXECUTION_TIMEOUT`, `STEP_LIMIT_EXCEEDED` | The timeout and the step budget of the `InterpreterAdapter` |
943
+
944
+ For the limits `error.message` says which one ended the script: `Script execution timed out ...`,
945
+ `Maximum iteration limit exceeded ...`, `Maximum tool call limit exceeded ...`.
831
946
 
832
947
  ### How `timeout` Is Enforced
833
948
 
@@ -837,9 +952,10 @@ file locations in the remaining lines are redacted.
837
952
  time has passed while it waited is not started: `run()` returns `ExecutionAbortedError`.
838
953
  - Until the script's first `await`, Node's `vm` interrupts it at the deadline.
839
954
  - After that, the script checks the clock itself: the transformer inserts a check at the start of
840
- every function and loop body, so synchronous work, recursion and promise chains that never yield
841
- are stopped at the deadline as well. A script that catches the timeout error cannot enter another
842
- function or loop iteration.
955
+ every function and loop body and in front of a parameter default, so synchronous work, recursion
956
+ and promise chains that never yield are stopped at the deadline as well. A script that catches
957
+ the timeout error cannot enter another function or loop iteration. The deadline is cooperative:
958
+ it is looked at between the steps of the script's own code.
843
959
  - A tool call in progress at the deadline is not cut off, and its result is delivered. From then on
844
960
  the next tool call or call into a host function is refused with `Execution aborted`. The same
845
961
  holds for an async function from `globals`.
@@ -851,7 +967,8 @@ file locations in the remaining lines are redacted.
851
967
  settling).
852
968
  - When `run()` has returned, for whatever reason, the script is finished for the host as well.
853
969
  What it left behind (a tool call it did not await, with a `then` on it) can no longer call a
854
- tool, a function from `globals` or the console.
970
+ tool, a function from `globals` or the console, and it ends within a bounded number of steps:
971
+ its next checks throw `Script execution has ended`.
855
972
  - A script that only waits is ended as well. Waiting takes no code: a thenable that never calls
856
973
  back (`return { then: () => 0 }`) is enough, and nothing runs that could look at the clock. So
857
974
  the enclave keeps count of what the host has in flight for the script (tool calls, async
@@ -859,8 +976,9 @@ file locations in the remaining lines are redacted.
859
976
  that count is zero. A sandbox has no timers and no I/O, so nothing else could wake the script.
860
977
  (With `adapter: 'worker_threads'` the pool ends such a script like a busy one: with its worker,
861
978
  five seconds after the deadline.)
862
- - Not covered: a single built-in call that does all its work natively (the memory limit bounds
863
- those) and a script run with `transform: false`. And the enclave cannot stop your
979
+ - Not covered: a single built-in call that does all its work natively, and a script run with
980
+ `transform: false`. The engine finishes one call into a built-in before the next check, so
981
+ `timeout` does not bound its duration. And the enclave cannot stop your
864
982
  `toolHandler`: a call it stops waiting for keeps running in the host, so a tool that changes
865
983
  something should be safe to repeat, or have a time limit of its own. For a
866
984
  stop the script cannot influence, use `adapter: 'worker_threads'`: a script that is still
@@ -877,11 +995,11 @@ file locations in the remaining lines are redacted.
877
995
  │
878
996
  ▼
879
997
  ┌─────────────────────────────────────────────────────────────┐
880
- │ Layer 0: Pre-Scanner (ast-guard) │
881
- │ ├── Blocks ReDoS patterns │
882
- │ ├── Blocks BiDi/Trojan source attacks │
883
- │ ├── Input size limits │
884
- │ └── Nesting depth limits │
998
+ │ Layer 0: Pre-Scanner (ast-guard, preset of the AST preset) │
999
+ │ ├── Input size, line and string limits │
1000
+ │ ├── Nesting depth limits │
1001
+ │ ├── Regex literals refused (agentscript) or analyzed │
1002
+ │ └── Bidirectional, invisible and look-alike characters │
885
1003
  └─────────────────────────────────────────────────────────────┘
886
1004
  │
887
1005
  ▼
@@ -889,8 +1007,9 @@ file locations in the remaining lines are redacted.
889
1007
  │ Layer 1: AST Validation (ast-guard) │
890
1008
  │ ├── NoEvalRule - Blocks eval(), Function() │
891
1009
  │ ├── NoGlobalAccessRule - Blocks dangerous globals │
892
- │ ├── DisallowedIdentifierRule - Blocks reserved prefixes │
893
- │ └── RequiredFunctionCallRule - Enforces callTool usage │
1010
+ │ ├── AllowedSyntaxRule - Syntax allowlist │
1011
+ │ ├── UnknownGlobalRule - Only the allowed globals │
1012
+ │ └── ReservedPrefixRule - Blocks reserved prefixes │
894
1013
  └─────────────────────────────────────────────────────────────┘
895
1014
  │
896
1015
  ▼
@@ -904,10 +1023,10 @@ file locations in the remaining lines are redacted.
904
1023
  │
905
1024
  ▼
906
1025
  ┌─────────────────────────────────────────────────────────────┐
907
- │ Layer 3: AI Scoring Gate │
908
- │ ├── Semantic security analysis via AST features │
909
- │ ├── Exfiltration pattern detection │
910
- │ ├── Sensitive field access tracking │
1026
+ │ Layer 3: AI Scoring Gate (optional, heuristic) │
1027
+ │ ├── Features of the script's tool calls │
1028
+ │ ├── Exfiltration-like tool sequences │
1029
+ │ ├── Sensitive field names │
911
1030
  │ └── Risk scoring (0-100) with thresholds │
912
1031
  └─────────────────────────────────────────────────────────────┘
913
1032
  │
@@ -927,9 +1046,9 @@ file locations in the remaining lines are redacted.
927
1046
  ▼
928
1047
  ┌─────────────────────────────────────────────────────────────┐
929
1048
  │ Layer 5: Output Sanitization │
930
- │ ├── Removes stack traces │
931
- │ ├── Sanitizes file paths │
932
- │ └── Truncates oversized outputs │
1049
+ │ ├── Returned values copied into plain data │
1050
+ │ ├── Errors as bounded text, no stack from the sandbox │
1051
+ │ └── Refuses oversized return values │
933
1052
  └─────────────────────────────────────────────────────────────┘
934
1053
  │
935
1054
  ▼
@@ -938,6 +1057,44 @@ file locations in the remaining lines are redacted.
938
1057
  └─────────────────────────────────────────────────────────────┘
939
1058
  ```
940
1059
 
1060
+ ### What the Pre-Scanner Checks
1061
+
1062
+ `Enclave` runs the pre-scanner on the script as it was written, before anything parses it, with the pre-scanner preset named like the AST `preset`:
1063
+
1064
+ | AST preset | Pre-scanner preset | Script | Lines | String literals | Regex literals | Bidirectional controls | Invisible characters |
1065
+ | ----------------------- | ------------------ | ---------------------------------------------------- | ------------------------------- | --------------------------- | ------------------------------------------- | ---------------------- | -------------------- |
1066
+ | `agentscript` (default) | `agentscript` | 100 KB, nesting 20 | 1,000 of at most 2,000 chars | 10 KB each, 100 KB in total | refused | refused | refused |
1067
+ | `strict` | `strict` | 500 KB, nesting 30 | 2,000 of at most 5,000 chars | 50 KB each, 500 KB in total | 10 of at most 100 chars, analyzed for ReDoS | refused | refused |
1068
+ | `secure` | `secure` | 1 MB, nesting 40 | 5,000 of at most 8,000 chars | 100 KB each, 1 MB in total | 20 of at most 200 chars, analyzed | refused | allowed |
1069
+ | `standard`, `babel` | `standard` | 5 MB, nesting 50 | 10,000 of at most 10,000 chars | 500 KB each, 5 MB in total | 30 of at most 500 chars, analyzed | allowed | allowed |
1070
+ | `permissive` | `permissive` | 5,242,880 chars (the enclave's maximum), nesting 100 | 100,000 of at most 50,000 chars | 1 MB each, 10 MB in total | allowed | allowed | allowed |
1071
+
1072
+ Every preset except `permissive` also refuses a null byte, more consecutive operators than its limit, and one of a fixed list of letters that look like ASCII letters (Cyrillic and Greek look-alikes of `a`, `c`, `e`, `o`, `p`, `x`, `y`, and the full-width `a` and `e`), anywhere in the text.
1073
+
1074
+ The `preScan` option picks another preset or changes single limits of the one in force; the pre-scanner's own absolute maximums still hold:
1075
+
1076
+ ```typescript
1077
+ // The limits enclaves had before 2.19.2, whatever the AST preset
1078
+ const enclave = new Enclave({ preScan: { preset: 'standard' } });
1079
+
1080
+ // The agentscript pre-scanner with room for larger string literals (for the reference sidecar, say).
1081
+ // The limit on one literal and the limit on all literals together are separate: raise both.
1082
+ const enclave = new Enclave({
1083
+ preScan: {
1084
+ config: {
1085
+ maxInputSize: 1024 * 1024,
1086
+ maxLineLength: 100000,
1087
+ maxStringLength: 512 * 1024,
1088
+ maxTotalStringContent: 1024 * 1024,
1089
+ },
1090
+ },
1091
+ });
1092
+ ```
1093
+
1094
+ A line is at most 100,000 characters under any configuration (the pre-scanner's absolute maximum), so a literal longer than that has to be a template literal over several lines.
1095
+
1096
+ With `validate: false` the pre-scanner does not run; a script is limited to 5,242,880 characters in any case.
1097
+
941
1098
  ## Runtime Notes
942
1099
 
943
1100
  ### Node.js 24 (V8 13.x)
@@ -1042,15 +1199,15 @@ the handler.
1042
1199
 
1043
1200
  ```bash
1044
1201
  # Run tests
1045
- nx test enclave-vm
1202
+ npx nx test core
1046
1203
 
1047
1204
  # Run tests with coverage
1048
- nx test enclave --coverage
1205
+ npx nx test core --coverage
1049
1206
  ```
1050
1207
 
1051
1208
  ## Related Packages
1052
1209
 
1053
- - [`ast-guard`](../ast-guard) - AST validation rules used by enclave
1210
+ - [`@enclave-vm/ast`](../ast) - AST validation rules used by enclave
1054
1211
 
1055
1212
  ## License
1056
1213