sandboxedjs 0.2.10 → 0.2.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +96 -63
- package/assets/logo.png +0 -0
- package/bin/sandboxedjs-egress.mjs +25 -10
- package/dist/index.cjs +1 -1
- package/dist/index.js +1 -1
- package/dist/service-worker.js +3 -2
- package/docs/agent/COMMANDS.md +85 -0
- package/docs/agent/DECISION-TREE.md +84 -0
- package/docs/agent/INVARIANTS.md +40 -0
- package/docs/agent/LAUNCH-PROMPT.md +37 -0
- package/docs/agent/LOOP.md +84 -0
- package/docs/agent/README.md +77 -0
- package/docs/agent/ROADMAP.md +37 -0
- package/docs/agent/STATE.md +93 -0
- package/docs/agent/tasks/00-verify-inherited-work.md +36 -0
- package/docs/agent/tasks/01-authoritative-metadata.md +34 -0
- package/docs/agent/tasks/02-native-dependencies.md +35 -0
- package/docs/agent/tasks/03-reproducible-inputs.md +27 -0
- package/docs/agent/tasks/04-build-frontends.md +27 -0
- package/docs/agent/tasks/05-registry-integration.md +27 -0
- package/docs/agent/tasks/06-package-cohorts.md +42 -0
- package/docs/agent/tasks/07-build-on-miss-boundary.md +31 -0
- package/docs/browser-runtime-architecture.md +142 -0
- package/docs/compatibility-implementation-plan.md +98 -0
- package/docs/developer-tool-packs.md +134 -0
- package/docs/frontend-automation.md +49 -0
- package/docs/fullstack-deployment.md +163 -0
- package/docs/handoff.md +275 -0
- package/docs/original-x64.md +39 -0
- package/docs/platform-hardening.md +49 -0
- package/docs/python/abi.md +97 -0
- package/docs/python/architecture.md +94 -0
- package/docs/python/baseline-inventory.md +54 -0
- package/docs/python/build-on-miss.md +198 -0
- package/docs/python/compatibility.md +206 -0
- package/docs/python/cross-build.md +354 -0
- package/docs/python/extensions.md +282 -0
- package/docs/python/release-gates.md +46 -0
- package/docs/python/virtual-sockets-plan.md +331 -0
- package/docs/runtime-lifecycle-fixes.md +39 -0
- package/docs/server-previews.md +268 -0
- package/docs/virtual-browser.md +120 -0
- package/package.json +5 -3
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# Browser runtime architecture
|
|
2
|
+
|
|
3
|
+
The current implementation is a JavaScript/POSIX compatibility runtime with
|
|
4
|
+
CPython compiled to WebAssembly. It is not a Linux kernel or an x86 emulator.
|
|
5
|
+
Keep this distinction in the public API and product claims.
|
|
6
|
+
|
|
7
|
+
## Existing execution paths
|
|
8
|
+
|
|
9
|
+
- JavaScript: the module engine and worker pods implement a subset of Node APIs.
|
|
10
|
+
- Python: CPython runs in a process worker and calls the shared virtual kernel.
|
|
11
|
+
- WASI: compiled wasm32-wasi programs use the existing WASI host.
|
|
12
|
+
- Browser preview: a service worker routes HTTP into the virtual network.
|
|
13
|
+
- Native Linux ELF executables and native Node addons: unsupported by these paths.
|
|
14
|
+
|
|
15
|
+
Browser execution needs suitable worker URLs and, for blocking worker calls,
|
|
16
|
+
SharedArrayBuffer and cross-origin isolation. Static hosting is still needed to
|
|
17
|
+
serve application and runtime assets. It is not a computation backend.
|
|
18
|
+
Python build-on-miss is different: its local builder requires Node/toolchains,
|
|
19
|
+
and its remote builder uses another machine. Prebuilt compatible wheels avoid
|
|
20
|
+
that backend. Not every PyPI package has such a wheel.
|
|
21
|
+
|
|
22
|
+
## Changes in this pass
|
|
23
|
+
|
|
24
|
+
The npm installer now overlaps up to six sibling package downloads while keeping
|
|
25
|
+
filesystem mutations ordered. Installers created through `forCwd` share their
|
|
26
|
+
metadata, archive and platform-detail caches within that installer family.
|
|
27
|
+
Transient metadata/archive failures are evicted so retries can recover.
|
|
28
|
+
An extracted package no longer hides an incomplete dependency tree, and optional
|
|
29
|
+
dependencies override duplicate required entries.
|
|
30
|
+
|
|
31
|
+
Python startup now cleans resources after worker creation or message-send
|
|
32
|
+
failures, and honors cancellation that arrives before or during worker creation.
|
|
33
|
+
|
|
34
|
+
Caches remain in memory; there is no persistent offline package store, lockfile
|
|
35
|
+
implementation, global hoisting, or measured real-project performance claim.
|
|
36
|
+
Browser bundling and the static import guard are checks of build compatibility,
|
|
37
|
+
not substitutes for interactive browser acceptance testing.
|
|
38
|
+
|
|
39
|
+
## Native Linux direction (not implemented)
|
|
40
|
+
|
|
41
|
+
Use a separately loaded browser machine emulator with its own Linux kernel and
|
|
42
|
+
disk image if running unchanged Linux packages is required. Keep it optional:
|
|
43
|
+
ordinary JavaScript/Python projects should not download an entire Linux image.
|
|
44
|
+
Expose the same high-level exec, filesystem transfer and preview operations, but
|
|
45
|
+
report the selected backend and its limitations explicitly. The emulator owns
|
|
46
|
+
its filesystem; transfer files explicitly instead of pretending it shares the
|
|
47
|
+
JavaScript runtime's synchronous in-memory volume.
|
|
48
|
+
|
|
49
|
+
This requires choosing the CPU architecture, emulator, kernel/root filesystem,
|
|
50
|
+
network bridge and image distribution terms. Native addon compatibility then
|
|
51
|
+
comes from running actual Node inside the guest Linux system. It brings larger
|
|
52
|
+
assets and different startup/performance tradeoffs, and does not promise every
|
|
53
|
+
Linux package will work. No such backend is included in this change.
|
|
54
|
+
|
|
55
|
+
MIT licensing of this project can avoid a runtime subscription. It does not
|
|
56
|
+
promise free CDN hosting, registry availability or third-party services forever.
|
|
57
|
+
Audit the separate distribution terms of any kernel, disk image and packages
|
|
58
|
+
before redistributing them.
|
|
59
|
+
|
|
60
|
+
## Extension implementation
|
|
61
|
+
|
|
62
|
+
The subsequent implementation adds ELF dispatch, external WASI command packs,
|
|
63
|
+
three ordered backend tiers, a translator output cache, and optional local Git
|
|
64
|
+
and embedded SQL adapters. See [Developer tool packs](developer-tool-packs.md)
|
|
65
|
+
for the working APIs and explicit limits. Translation and emulator engines
|
|
66
|
+
remain external and are not included.
|
|
67
|
+
|
|
68
|
+
## Verified behaviour, and the gaps that remain
|
|
69
|
+
|
|
70
|
+
A verification harness in the browser test project (`npm run verify`, driving
|
|
71
|
+
real Chrome over the published package) exercises the container boot, Node,
|
|
72
|
+
CPython, an npm install from the registry, both x86-64 ELF tiers, WASI command
|
|
73
|
+
packs, Git, PostgreSQL, an in-container HTTP server, and the frontend
|
|
74
|
+
automation subset. All of it passes in a real browser.
|
|
75
|
+
|
|
76
|
+
One gap is closed and one remains.
|
|
77
|
+
|
|
78
|
+
## Outbound HTTP for Python
|
|
79
|
+
|
|
80
|
+
A browser cannot open a raw TCP socket, so nothing inside the container can
|
|
81
|
+
speak TLS to a real server: outbound traffic has to leave at the HTTP layer,
|
|
82
|
+
through the page's `fetch`. The container therefore serves a loopback HTTP
|
|
83
|
+
endpoint, advertised to the guest as `SBX_HTTP_EGRESS`, which takes an
|
|
84
|
+
*absolute* URL in the request line the way a proxy does and performs the
|
|
85
|
+
transfer on the guest's behalf. `https://` targets are named in that request
|
|
86
|
+
line rather than tunnelled with CONNECT, precisely so the guest never begins a
|
|
87
|
+
TLS handshake it cannot finish; the real request leaves the page over HTTPS.
|
|
88
|
+
|
|
89
|
+
The startup hook points `urllib.request` and `http.client` at it, so the
|
|
90
|
+
standard library reaches the network without changes to a program. The same
|
|
91
|
+
outbound policy every other exit applies is applied here, and because a refusal
|
|
92
|
+
comes back as a real HTTP response it carries the reason -- a guest is told that
|
|
93
|
+
outbound access is disabled and where to enable it, rather than being handed an
|
|
94
|
+
errno it renders as "connection refused".
|
|
95
|
+
|
|
96
|
+
Libraries that manage their own TLS rather than going through `http.client` --
|
|
97
|
+
`urllib3`, and so `requests` -- still cannot egress. Their sockets would need to
|
|
98
|
+
carry a TLS handshake this transport cannot terminate. A page's `fetch` is also
|
|
99
|
+
subject to CORS, so a cross-origin target must send the headers that let the
|
|
100
|
+
page read the response; that is a browser rule, not a container policy.
|
|
101
|
+
|
|
102
|
+
## Blocking syscalls stop every thread
|
|
103
|
+
|
|
104
|
+
Threads in this runtime are real and run in parallel: a worker thread happily
|
|
105
|
+
burns CPU while the main thread sits in `time.sleep`, which releases the GIL.
|
|
106
|
+
What they cannot do is make a syscall while another thread is inside one.
|
|
107
|
+
|
|
108
|
+
The cause is in the interpreter image, not in this package. It is built with
|
|
109
|
+
`-sPROXY_TO_PTHREAD` over a JavaScript-backed filesystem, so every syscall made
|
|
110
|
+
on a thread is forwarded to the single thread that owns that filesystem -- the
|
|
111
|
+
generated glue is full of `if (ENVIRONMENT_IS_PTHREAD) return
|
|
112
|
+
proxyToMainThread(...)`. That thread answers a host call by blocking in
|
|
113
|
+
`Atomics.wait`. While it is blocked it cannot service anybody else's proxied
|
|
114
|
+
syscall, so one thread parked in a blocking read freezes every other thread's
|
|
115
|
+
I/O until it returns.
|
|
116
|
+
|
|
117
|
+
`asyncio.to_thread` is the case that meets this head-on: the loop parks in a
|
|
118
|
+
blocking `select`, and waking it requires the worker thread to make a syscall
|
|
119
|
+
that the parked thread would have to service. Neither completes.
|
|
120
|
+
`to_thread` now raises immediately, naming the limitation, instead of hanging
|
|
121
|
+
until the process is killed. That is a mitigation, not a fix.
|
|
122
|
+
|
|
123
|
+
Three earlier explanations of this were wrong and are recorded here so the next
|
|
124
|
+
person does not re-derive them: it is not a single control slot in the transport
|
|
125
|
+
(threads do run concurrently), it is not the file-scope statics in `sbx_call`
|
|
126
|
+
(they are downstream of the proxying), and it is not a missing per-thread
|
|
127
|
+
channel on its own (a channel per thread does nothing while the syscalls are
|
|
128
|
+
still funnelled to one thread).
|
|
129
|
+
|
|
130
|
+
Fixing it is a change to how the runtime image is built, and there are two
|
|
131
|
+
routes, neither small:
|
|
132
|
+
|
|
133
|
+
- Stack switching. Suspend the wasm stack on a blocking call and return to the
|
|
134
|
+
event loop, instead of holding the thread in `Atomics.wait`. This is what the
|
|
135
|
+
Pyodide backend did through `run_sync`, and it is why that backend did not
|
|
136
|
+
have this problem. It needs JSPI and a transport rewritten around suspension.
|
|
137
|
+
- Unproxied per-thread syscalls. Give each thread its own channel and host
|
|
138
|
+
server and stop routing syscalls through one thread. This fights the
|
|
139
|
+
JavaScript filesystem the image is built on, since that filesystem lives on
|
|
140
|
+
one thread by construction.
|
|
141
|
+
|
|
142
|
+
Both are runtime-image work measured in days, not a patch plus a rebuild.
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Compatibility implementation plan
|
|
2
|
+
|
|
3
|
+
## Decisions and evidence
|
|
4
|
+
|
|
5
|
+
Keep the owned JavaScript runtime and CPython WebAssembly distribution. No
|
|
6
|
+
Nodepod, Pyodide, v86, or replacement VM is required for the first milestones.
|
|
7
|
+
The timeout, missing Node APIs and Python syscall deadlock are separate issues.
|
|
8
|
+
|
|
9
|
+
The extension acceptance suite serves local wheels using node:http. In the
|
|
10
|
+
restricted development environment listen(127.0.0.1) fails with EPERM. Its
|
|
11
|
+
setup promise previously had no error handler, so setup waited for the hook
|
|
12
|
+
timeout. The test now rejects immediately and teardown handles failed startup.
|
|
13
|
+
With loopback listening permitted, all 12 extension acceptance tests passed in
|
|
14
|
+
34.19 seconds of test execution (37.26 seconds total). This includes C, Cython,
|
|
15
|
+
Rust/PyO3, Meson and Pydantic's compiled core. This establishes the Node-hosted
|
|
16
|
+
fixture path, not browser acceptance or arbitrary PyPI compatibility.
|
|
17
|
+
|
|
18
|
+
The Python build uses PROXY_TO_PTHREAD. backend.ts sets SBX_SERIAL_HOST_CALLS
|
|
19
|
+
and refuses asyncio.to_thread because proxied blocking syscalls serialize the
|
|
20
|
+
filesystem-owning thread. Removing the refusal alone is not a fix.
|
|
21
|
+
|
|
22
|
+
## Alternatives
|
|
23
|
+
|
|
24
|
+
| Approach | Useful for | Main constraint | Decision |
|
|
25
|
+
| --- | --- | --- | --- |
|
|
26
|
+
| Owned JavaScript builtins | Node utility and lifecycle APIs | Must reproduce observable semantics | First choice |
|
|
27
|
+
| Compile source to Wasm | C/C++/Rust extensions and tools | ABI, dependencies and host calls must match | Preferred native-code route |
|
|
28
|
+
| JSPI suspension | Freeing the worker event loop during a blocking host call | Runtime support, suspension boundaries, dynamic modules and pthread interaction | Prototype before adopting |
|
|
29
|
+
| Asyncify | Suspension where JSPI is unavailable | Instrumentation cost and dynamic-module compatibility | Compare as fallback |
|
|
30
|
+
| Per-thread syscall channels | Concurrent Python I/O | Must also remove synchronous proxy bottleneck and preserve fd semantics | Prototype alongside suspension |
|
|
31
|
+
| WasmFS integration | Moving filesystem work away from JS-only ownership | Existing virtual FS needs a backend; not a flag-only change | Assess with thread prototype |
|
|
32
|
+
| Host-native executables | Compatible tools on Node servers | Host OS/architecture, process permissions, unavailable in browsers | Optional explicit host capability |
|
|
33
|
+
| Owned instruction interpreter | Selected binaries without source | ISA, loader, syscalls, linking, signals and threads | Defer until a concrete binary needs it |
|
|
34
|
+
| Binary-to-Wasm translation | Repeated execution of supported binaries | Same ABI needs plus translation correctness and cache invalidation | Later, reuse existing backend interface |
|
|
35
|
+
|
|
36
|
+
## Ordered implementation milestones
|
|
37
|
+
|
|
38
|
+
### 1. Make extension acceptance trustworthy
|
|
39
|
+
|
|
40
|
+
Run the complete local wheel suite with working loopback networking. Separate
|
|
41
|
+
setup, download, install, link, import and execution failures. Fix each observed
|
|
42
|
+
failure with a focused regression. Preserve failures; do not raise timeouts to
|
|
43
|
+
make deadlocks disappear. Repeat in the browser using static fixture hosting.
|
|
44
|
+
|
|
45
|
+
Exit: ordinary imports execute compiled C/C++/Rust fixtures; missing/incompatible
|
|
46
|
+
artifacts produce actionable failures, and shutdown releases all resources.
|
|
47
|
+
|
|
48
|
+
### 2. Replace Node stubs in dependency order
|
|
49
|
+
|
|
50
|
+
Inventory actual package imports. Start with diagnostics_channel and utility
|
|
51
|
+
APIs, followed by worker_threads backed by guest workers, then virtual network
|
|
52
|
+
APIs. Separate virtual TCP from external browser networking. cluster/domain,
|
|
53
|
+
http2, tls, inspector, v8 and vm each need an explicit supported surface rather
|
|
54
|
+
than a catch-all function that pretends to implement them. A JS vm substitute
|
|
55
|
+
must not claim a security boundary it does not provide.
|
|
56
|
+
|
|
57
|
+
Exit per module: native-Node differential tests, error/cancellation tests,
|
|
58
|
+
cross-guest isolation, and a real package that consumes that API in Node and
|
|
59
|
+
browser hosts. Mark unsupported operations explicitly.
|
|
60
|
+
|
|
61
|
+
### 3. Prove concurrent Python I/O before rebuilding production
|
|
62
|
+
|
|
63
|
+
Build a minimal C/pthread fixture against the same Emscripten toolchain. One
|
|
64
|
+
thread blocks on a host read while another writes; add cancellation and a
|
|
65
|
+
timer on the owning JS worker. Compare an asynchronous proxy/suspension path
|
|
66
|
+
with unproxied per-thread host calls. Channels alone cannot repair serialized
|
|
67
|
+
proxy dispatch. Feature-detect JSPI; test fallback behavior explicitly.
|
|
68
|
+
|
|
69
|
+
Measure idle CPU, wakeup latency, memory, download size and throughput for each
|
|
70
|
+
variant. Select only after the fixture passes. Then rebuild CPython and all
|
|
71
|
+
affected extension fixtures, preserving one shared descriptor authority.
|
|
72
|
+
|
|
73
|
+
Exit: asyncio.to_thread, run_in_executor, concurrent socket read/write,
|
|
74
|
+
cancellation, subprocess interaction and dynamic extensions work together.
|
|
75
|
+
Remove the current guard only after these pass in supported browsers and Node.
|
|
76
|
+
|
|
77
|
+
### 4. Expand portable binaries
|
|
78
|
+
|
|
79
|
+
Use source builds targeting the existing Wasm ABI first. Track toolchain and
|
|
80
|
+
ABI fingerprints in artifacts. If an essential source-unavailable binary
|
|
81
|
+
remains, define one ISA and a static executable subset for an owned interpreter;
|
|
82
|
+
reject unsupported instructions and syscalls deterministically. Bound memory,
|
|
83
|
+
execution and cancellation. Do not start with dynamic Linux package parity.
|
|
84
|
+
|
|
85
|
+
Exit: selected real binaries have reproducible correctness tests and measured
|
|
86
|
+
startup, memory and throughput. Translation caching follows correctness.
|
|
87
|
+
|
|
88
|
+
## Sources for the suspension prototypes
|
|
89
|
+
|
|
90
|
+
- Emscripten pthread proxying and blocking rules:
|
|
91
|
+
https://emscripten.org/docs/porting/pthreads.html
|
|
92
|
+
- Emscripten Asyncify and JSPI integration:
|
|
93
|
+
https://emscripten.org/docs/porting/asyncify.html
|
|
94
|
+
- V8 JSPI boundary model:
|
|
95
|
+
https://v8.dev/blog/jspi-newapi
|
|
96
|
+
|
|
97
|
+
These sources describe mechanisms; they do not establish that this CPython
|
|
98
|
+
image or its extensions already support them. The prototypes are required.
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
# Optional developer tools and binary backends
|
|
2
|
+
|
|
3
|
+
This is an extension mechanism and two limited developer-tool adapters, not
|
|
4
|
+
universal Linux compatibility. The core has no new third-party dependencies,
|
|
5
|
+
paid endpoint, account requirement, or automatic downloads.
|
|
6
|
+
|
|
7
|
+
## Working developer tools
|
|
8
|
+
|
|
9
|
+
Install engines in the **host application**, then pass them to the adapters:
|
|
10
|
+
|
|
11
|
+
```sh
|
|
12
|
+
npm install sandboxedjs isomorphic-git @electric-sql/pglite
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
```js
|
|
16
|
+
import git from 'isomorphic-git';
|
|
17
|
+
import { PGlite } from '@electric-sql/pglite';
|
|
18
|
+
import { createContainer, createGitCommand, createSqlCommand } from 'sandboxedjs';
|
|
19
|
+
|
|
20
|
+
const box = await createContainer({ cwd: '/app' });
|
|
21
|
+
const db = new PGlite();
|
|
22
|
+
box.kernel.installCommand(createGitCommand(git));
|
|
23
|
+
box.kernel.installCommand(createSqlCommand(db));
|
|
24
|
+
|
|
25
|
+
await box.exec('git init');
|
|
26
|
+
await box.fs.writeFile('/app/hello.txt', 'hello');
|
|
27
|
+
await box.exec('git add hello.txt');
|
|
28
|
+
await box.exec('git commit -m first', {
|
|
29
|
+
env: { GIT_AUTHOR_NAME: 'Your Name', GIT_AUTHOR_EMAIL: 'you@example.com' },
|
|
30
|
+
});
|
|
31
|
+
console.log(await box.exec('git log'));
|
|
32
|
+
console.log(await box.exec('sql -c "SELECT 42 AS answer"'));
|
|
33
|
+
|
|
34
|
+
// The application owns the database lifetime and persistence configuration.
|
|
35
|
+
await db.close();
|
|
36
|
+
box.dispose();
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Git supports `init`, `add PATH...`, `commit -m MESSAGE`, `status`, and `log`.
|
|
40
|
+
These are local operations backed by isomorphic-git and the credential-aware
|
|
41
|
+
container filesystem. Clone, fetch, push, branches, checkout and full Git CLI
|
|
42
|
+
compatibility are not implemented. No CORS proxy is configured implicitly.
|
|
43
|
+
|
|
44
|
+
`sql -c SQL` and `sql -f FILE` print result rows as JSON. This is an embedded
|
|
45
|
+
SQL adapter, not `psql`, a PostgreSQL server, or a TCP endpoint for `pg` clients.
|
|
46
|
+
Database storage belongs to PGlite, separately from the container filesystem.
|
|
47
|
+
Use its persistence options in the host; container snapshots do not capture it.
|
|
48
|
+
The supplied database API does not provide hard query cancellation; a timeout
|
|
49
|
+
is not a transaction rollback. Do not share the same database across tenants.
|
|
50
|
+
|
|
51
|
+
Both adapters are trusted host integrations. This pass validated real Git
|
|
52
|
+
commits with isomorphic-git 1.42.2 and real SQL with PGlite 0.5.8 on Node,
|
|
53
|
+
not a browser UI. Their upstream engines
|
|
54
|
+
support browsers; host asset loading must still be configured and verified.
|
|
55
|
+
|
|
56
|
+
## Precompiled command packs
|
|
57
|
+
|
|
58
|
+
Packs live outside the core. A host can load their manifests and artifact bytes
|
|
59
|
+
from its own storage and call:
|
|
60
|
+
|
|
61
|
+
```js
|
|
62
|
+
import { installWasmCommands } from 'sandboxedjs';
|
|
63
|
+
installWasmCommands(box.kernel, [
|
|
64
|
+
{ name: 'my-tool', bytes: wasmBytes, sha256: trustedManifest.sha256 },
|
|
65
|
+
]);
|
|
66
|
+
await box.exec('my-tool --help');
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Supply a wasm32-wasi command compatible with this runtime's WASI preview1 host.
|
|
70
|
+
Hashes detect changed artifact bytes; trust in the manifest comes from the host.
|
|
71
|
+
Names, collisions, hashes and Wasm validity are checked before commands are
|
|
72
|
+
installed. No network fetch, registry, package archive or license acceptance is
|
|
73
|
+
hidden in this API. WASI imports and behaviors still need compatibility checks;
|
|
74
|
+
valid Wasm alone does not prove that a command can execute here.
|
|
75
|
+
|
|
76
|
+
## ELF execution and fallback
|
|
77
|
+
|
|
78
|
+
The kernel now recognizes executable ELF files instead of treating them as
|
|
79
|
+
shell scripts. It consults `box.kernel.binaries` in this order:
|
|
80
|
+
|
|
81
|
+
1. `compatibility`: exact binary hashes mapped to tested ports.
|
|
82
|
+
2. `translation`: a supplied compiler producing compatible WASI commands.
|
|
83
|
+
3. `emulation`: a supplied emulator backend.
|
|
84
|
+
|
|
85
|
+
`createWasmCompatibilityBackend(id, entries)` registers mappings of
|
|
86
|
+
`{ elfSha256, wasm: { name, bytes, sha256 } }`. Matching content rather than a
|
|
87
|
+
command name prevents silently substituting a port for a different version.
|
|
88
|
+
|
|
89
|
+
`createTranslationBackend(translator, maxCacheBytes?)` wraps a compiler with a
|
|
90
|
+
32 MiB default in-memory LRU output cache. The compiler supplies `id`,
|
|
91
|
+
`supports(info)` and `translate(request, signal)`. Give it a versioned identity.
|
|
92
|
+
Return null only when the input is unsupported. Thrown errors are surfaced.
|
|
93
|
+
This wrapper **does not contain an ELF compiler**. Its checks use a fixture
|
|
94
|
+
translator, not a claim that native instructions were translated in the test.
|
|
95
|
+
Ordinary elfconv/Emscripten output cannot be assumed to match our WASI ABI.
|
|
96
|
+
|
|
97
|
+
An emulator implements `BinaryBackend`, with `tier: 'emulation'` and
|
|
98
|
+
`prepare(request, signal)`. Register it with `box.kernel.binaries.register()`;
|
|
99
|
+
the returned function unregisters it. `list()` reports installed providers.
|
|
100
|
+
An original experimental x86-64 translator and interpreter are now available
|
|
101
|
+
through `createOriginalX64Backends()`. They support a small freestanding subset;
|
|
102
|
+
see [Original engines](original-x64.md). No Linux image is shipped.
|
|
103
|
+
|
|
104
|
+
Preparation returns either `{ supported: false, reason }` or
|
|
105
|
+
`{ supported: true, program: { run(ctx) } }`. Preparation must not execute guest
|
|
106
|
+
code or modify guest files. Backends receive an AbortSignal and must honor it.
|
|
107
|
+
After `run()` starts, its exit code is final: retrying on another backend could
|
|
108
|
+
duplicate writes or other effects. A native program with no supported backend
|
|
109
|
+
exits 126 with a diagnostic. Hard interruption requires a backend worker; the
|
|
110
|
+
existing in-realm WASI runner cannot interrupt a tight compute loop.
|
|
111
|
+
|
|
112
|
+
## Package priorities
|
|
113
|
+
|
|
114
|
+
| Priority | Tools | Approach and current status |
|
|
115
|
+
| --- | --- | --- |
|
|
116
|
+
| 1 | Git | Optional isomorphic-git adapter; local subset implemented |
|
|
117
|
+
| 1 | PostgreSQL SQL | Optional PGlite adapter implemented; embedded SQL only |
|
|
118
|
+
| 2 | jq, ripgrep, SQLite CLI, diff/patch | Candidate separate WASI packs; not bundled or validated here |
|
|
119
|
+
| 3 | C/C++ compiler and build tools | Separate large toolchain pack; compilation and subprocess support need work |
|
|
120
|
+
| Later | Redis-compatible services | Evaluate a separate engine and exact protocol/command coverage |
|
|
121
|
+
| Separate project | Docker Engine | Requires Linux kernel facilities; not a small WASI shim |
|
|
122
|
+
|
|
123
|
+
Broad binary translation and emulation coverage remain substantive follow-up work.
|
|
124
|
+
Do not advertise arbitrary ELF, Docker, complete PostgreSQL service compatibility,
|
|
125
|
+
or WebContainer performance parity based on this extension layer.
|
|
126
|
+
|
|
127
|
+
## Cost and licensing
|
|
128
|
+
|
|
129
|
+
SandboxedJs remains MIT. Upstream PGlite and isomorphic-git publish permissive
|
|
130
|
+
licenses, but retain the notices and check the specific versions you distribute.
|
|
131
|
+
A compatibility layer does not remove a tool's license obligations or make
|
|
132
|
+
enterprise software free. Host-supplied packs let users choose their own tools
|
|
133
|
+
and licenses. Hosting, compilation and network services have real resource costs;
|
|
134
|
+
this API does not require a paid provider or promise free third-party hosting.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Frontend browser automation
|
|
2
|
+
|
|
3
|
+
`createFrontendPlaywright(options?)` provides a Playwright-shaped API that runs
|
|
4
|
+
entirely in the page hosting SandboxedJs. It drives a same-origin `<iframe>`
|
|
5
|
+
using the browser the user already has. It does **not** bundle, download or
|
|
6
|
+
launch Chromium, and it is not Playwright.
|
|
7
|
+
|
|
8
|
+
```js
|
|
9
|
+
import { createFrontendPlaywright } from 'sandboxedjs';
|
|
10
|
+
|
|
11
|
+
const pw = createFrontendPlaywright({ mount: document.querySelector('#tests') });
|
|
12
|
+
const browser = await pw.chromium.launch();
|
|
13
|
+
const page = await browser.newPage();
|
|
14
|
+
|
|
15
|
+
await page.goto('/preview/index.html');
|
|
16
|
+
await page.getByRole('textbox', { name: 'Email' }).fill('a@example.com');
|
|
17
|
+
await page.getByRole('button', { name: 'Submit' }).click();
|
|
18
|
+
await pw.expect(page.getByText('Thanks')).toBeVisible();
|
|
19
|
+
|
|
20
|
+
await pw.dispose();
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Supported
|
|
24
|
+
|
|
25
|
+
`page`: `goto`, `setContent`, `title`, `content`, `evaluate`,
|
|
26
|
+
`setDefaultTimeout`, `close`; locators via `locator`, `getByTestId`,
|
|
27
|
+
`getByText`, and `getByRole` for `button`, `textbox`, `link`, `checkbox`
|
|
28
|
+
(plus any explicit `role=` attribute).
|
|
29
|
+
|
|
30
|
+
`locator`: `first`, `nth`, `locator`, `count`, `isVisible`, `textContent`,
|
|
31
|
+
`innerText`, `inputValue`, `getAttribute`, `click`, `fill`, `waitFor`.
|
|
32
|
+
Locators are strict: more than one match throws unless you narrow with
|
|
33
|
+
`first()`/`nth()`. Actions wait for the element to be visible and enabled.
|
|
34
|
+
|
|
35
|
+
`expect(locator)`: `toBeVisible`, `toHaveText`.
|
|
36
|
+
|
|
37
|
+
## Not supported
|
|
38
|
+
|
|
39
|
+
`screenshot`, `route`/network interception, cross-origin navigation, browser
|
|
40
|
+
launch options, and `firefox`/`webkit` all throw
|
|
41
|
+
`FrontendAutomationUnsupported`. Clicks and typing are dispatched DOM events,
|
|
42
|
+
not OS-level trusted input; `capabilities.trustedEvents` is `false`. `goto`
|
|
43
|
+
returns `null` rather than a Response. `evaluate` uses `eval` inside the frame
|
|
44
|
+
and fails where the page's CSP forbids it.
|
|
45
|
+
|
|
46
|
+
Pass `resolveUrl` to map a container's virtual server URL to the same-origin
|
|
47
|
+
preview URL served by the SandboxedJs service worker. Inspect
|
|
48
|
+
`pw.capabilities` before relying on any behavior; treat this as a labeled
|
|
49
|
+
subset for smoke-testing your own preview output, not a Chromium replacement.
|
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
# Full-stack projects across browser and Node hosts
|
|
2
|
+
|
|
3
|
+
The guest project can keep its frontend, FastAPI backend, and `http://127.0.0.1:8000`
|
|
4
|
+
API URLs. Hosting prerequisites belong to the outer application that embeds SandboxedJS.
|
|
5
|
+
They cannot all be removed by a JavaScript library.
|
|
6
|
+
|
|
7
|
+
| Where SandboxedJS executes | Frontend → guest backend | Backend → external API | Lifetime |
|
|
8
|
+
| --- | --- | --- | --- |
|
|
9
|
+
| Browser, served by a static CDN | `createPreview(box)` rewrites loopback fetch/XHR/EventSource requests into virtual ports | Direct when browser policies permit; otherwise a deployed egress function or an explicitly configured relay | Owner tab stays open |
|
|
10
|
+
| Long-running Node process | `box.request()` or an application HTTP adapter; `box.expose()` for local access | Node performs outbound requests | Process stays alive |
|
|
11
|
+
| Node serverless function | Use `box.request()` within the invocation; return its result through the platform's HTTP response | Node performs outbound requests | Invocation only; do not rely on a background server surviving the response |
|
|
12
|
+
|
|
13
|
+
A Vercel-hosted browser app is still a **browser** runtime. Serving its assets from Vercel
|
|
14
|
+
does not give code in the visitor's browser Node networking permissions. A purely static
|
|
15
|
+
host cannot itself relay an API that rejects cross-origin browser calls. Use a function on
|
|
16
|
+
that host, or provide `network.proxy` pointing to a relay you operate.
|
|
17
|
+
|
|
18
|
+
## Browser host setup
|
|
19
|
+
|
|
20
|
+
1. Serve over HTTPS (localhost is suitable for development).
|
|
21
|
+
2. Send these HTTP response headers on the outer document:
|
|
22
|
+
|
|
23
|
+
```http
|
|
24
|
+
Cross-Origin-Opener-Policy: same-origin
|
|
25
|
+
Cross-Origin-Embedder-Policy: require-corp
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
3. Serve the matching worker and Python runtime assets from the installed package. When
|
|
29
|
+
copying them into `public`, copy them on **every production build**, not just on `dev`:
|
|
30
|
+
|
|
31
|
+
```json
|
|
32
|
+
{
|
|
33
|
+
"scripts": {
|
|
34
|
+
"predev": "npm run stage:runtime",
|
|
35
|
+
"prebuild": "npm run stage:runtime",
|
|
36
|
+
"build": "tsc && vite build"
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
The staging script must copy `dist/service-worker.js` from the same installed
|
|
42
|
+
`sandboxedjs` as the host bundle. Copy the guest worker, Python worker, and Python
|
|
43
|
+
distribution if your host explicitly serves those from `public` too. Do not keep an
|
|
44
|
+
independently maintained old service worker. Alternatively, let your bundler emit the
|
|
45
|
+
default worker asset used by `createPreview()`.
|
|
46
|
+
|
|
47
|
+
4. Register the emitted worker and use its URL for the iframe:
|
|
48
|
+
|
|
49
|
+
```ts
|
|
50
|
+
const preview = await createPreview(box, { scriptUrl: '/service-worker.js' });
|
|
51
|
+
if (!preview) throw new Error('Preview worker could not be registered');
|
|
52
|
+
iframe.src = preview.urlFor(3000);
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Without a copied/custom worker, omit `scriptUrl`. Preserve the same-origin worker asset
|
|
56
|
+
as a file; do not inline it as a data URL. A strict guest CSP that rejects the injected
|
|
57
|
+
scripts, compressed HTML, or `{ websocket: false }` prevents the current loopback shim
|
|
58
|
+
from being installed. Use normal uncompressed preview HTML with injection enabled.
|
|
59
|
+
|
|
60
|
+
5. Enable outbound access for package installation and API calls:
|
|
61
|
+
`network: { allowOutbound: true }`. Start both guest servers, check their exit/output
|
|
62
|
+
and `waitForPort()` results, and keep the owning container alive.
|
|
63
|
+
|
|
64
|
+
## External APIs: create the relay in the right directory
|
|
65
|
+
|
|
66
|
+
Run from the **host application root**, not the guest's `/workspace` shell:
|
|
67
|
+
|
|
68
|
+
```sh
|
|
69
|
+
# Cloudflare Pages
|
|
70
|
+
npx sandboxedjs-egress init --target cloudflare --allow ollama.com,api.openai.com
|
|
71
|
+
# Vercel
|
|
72
|
+
npx sandboxedjs-egress init --target vercel --allow ollama.com,api.openai.com
|
|
73
|
+
# Netlify
|
|
74
|
+
npx sandboxedjs-egress init --target netlify --allow ollama.com,api.openai.com
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Use one command for the platform you deploy to. An explicit project directory may appear
|
|
78
|
+
before or after the flags. Older CLI builds incorrectly treated a flag value as the
|
|
79
|
+
project directory; if you see a folder called `ollama.com,api.openai.com` or `cloudflare`,
|
|
80
|
+
upgrade the CLI and generate the function at the root instead.
|
|
81
|
+
|
|
82
|
+
Expected paths:
|
|
83
|
+
|
|
84
|
+
| Platform | File relative to host project root | Probe URL |
|
|
85
|
+
| --- | --- | --- |
|
|
86
|
+
| Pages | `functions/__sandboxedjs__/egress.ts` | `/__sandboxedjs__/egress` |
|
|
87
|
+
| Vercel | `api/__sandboxedjs__/egress.ts` | `/api/__sandboxedjs__/egress` |
|
|
88
|
+
| Netlify | `netlify/functions/sandboxedjs-egress.ts` | `/.netlify/functions/sandboxedjs-egress` |
|
|
89
|
+
|
|
90
|
+
A GET to the deployed probe must return JSON containing
|
|
91
|
+
`{"sandboxedjs":"egress","protocol":1}`, not the site's HTML. Containers discover these
|
|
92
|
+
paths automatically. Development uses `npx sandboxedjs-egress --allow ollama.com` or
|
|
93
|
+
`npx sandboxedjs-serve dist`; Vite alone does not execute Pages Functions.
|
|
94
|
+
|
|
95
|
+
Deploy the function along with the static build. Cloudflare dashboard drag-and-drop does
|
|
96
|
+
not compile a `functions` directory: use Git integration or Wrangler from the project root.
|
|
97
|
+
See [Cloudflare's deployment documentation](https://developers.cloudflare.com/pages/get-started/direct-upload/).
|
|
98
|
+
|
|
99
|
+
Keep the relay's allowlist narrow and apply your application's authentication and rate limits
|
|
100
|
+
at the host route. Guest `.env` files and browser-delivered keys are visible to the visitor;
|
|
101
|
+
operator-owned credentials belong in server-side secrets. Do not commit actual keys.
|
|
102
|
+
|
|
103
|
+
## FastAPI and a static frontend
|
|
104
|
+
|
|
105
|
+
Inside one container, use the project's normal commands:
|
|
106
|
+
|
|
107
|
+
```sh
|
|
108
|
+
cd /workspace/backend
|
|
109
|
+
pip install -r requirements.txt
|
|
110
|
+
fastapi run main.py --host 0.0.0.0 --port 8000 &
|
|
111
|
+
cd /workspace/frontend
|
|
112
|
+
npx serve . -l 3000
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Preview port 3000 with `preview.urlFor(3000)`. Its fetch to
|
|
116
|
+
`http://127.0.0.1:8000/agent/run/stream` must appear in browser developer tools as a
|
|
117
|
+
same-origin `/…/__sbx__/8000/agent/run/stream` request. No CORS relay is needed for this
|
|
118
|
+
internal hop. An external Ollama request made by FastAPI uses the separate egress path.
|
|
119
|
+
|
|
120
|
+
The bridge currently buffers response bodies. A finite SSE response arrives after the
|
|
121
|
+
backend finishes, preserving its event text, but tokens do not appear progressively.
|
|
122
|
+
An indefinitely open stream will time out. `createPreview(box, { timeoutMs: 600_000 })`
|
|
123
|
+
changes the preview timeout; it cannot change a hosting platform's function limits.
|
|
124
|
+
|
|
125
|
+
## Node and serverless usage
|
|
126
|
+
|
|
127
|
+
The stable transport API is `box.request(port, { method, path, headers, body })`. Start the
|
|
128
|
+
backend and wait for its virtual port before invoking it. Return `response.bytes`, status,
|
|
129
|
+
and headers using your framework's response adapter. Clean up the container when its
|
|
130
|
+
owning request/session ends. `box.expose()` opens a local socket and is useful for local
|
|
131
|
+
Node development; it does not create a public route in Vercel.
|
|
132
|
+
|
|
133
|
+
Serverless workers may be frozen or discarded between invocations. Store files/session
|
|
134
|
+
state externally if needed and reconstruct the container, or use a persistent host for
|
|
135
|
+
interactive sessions. Package support, available workers/WASM, memory, payload sizes and
|
|
136
|
+
[function duration limits](https://vercel.com/docs/functions/limitations) still apply.
|
|
137
|
+
This project does not promise that an arbitrary browser workload runs unchanged inside an
|
|
138
|
+
edge runtime.
|
|
139
|
+
|
|
140
|
+
## Diagnose the actual failed hop
|
|
141
|
+
|
|
142
|
+
- **Frontend still requests `127.0.0.1:8000` in developer tools:** check the deployed worker
|
|
143
|
+
for `installPreviewFetch`, rebuild staged assets, then reload the host and reopen the
|
|
144
|
+
preview. Also check CSP/injection restrictions. Changing FastAPI CORS does not fix an
|
|
145
|
+
address that points at the visitor's machine.
|
|
146
|
+
- **Same-origin `__sbx__/8000` returns 502:** check backend process output and
|
|
147
|
+
`await box.request(8000, { path: '/docs' })`.
|
|
148
|
+
- **Backend receives the request but its AI call fails:** verify the egress probe, allowed
|
|
149
|
+
hostname, upstream credentials and upstream response. Test authenticated and ordinary
|
|
150
|
+
user sessions independently; application permissions may differ.
|
|
151
|
+
- **504 after a long wait:** inspect backend completion and buffering/host time limits.
|
|
152
|
+
- **Works locally but not after build:** inspect actual deployed headers, worker bytes,
|
|
153
|
+
copied runtime versions and function deployment. A successful asset build alone does
|
|
154
|
+
not deploy a function.
|
|
155
|
+
|
|
156
|
+
## Regression evidence
|
|
157
|
+
|
|
158
|
+
`test/preview-worker-build.test.ts` evaluates the distributed service worker, executes its
|
|
159
|
+
injected script, and verifies the loopback POST route and navigation paths.
|
|
160
|
+
`test/egress-cli.test.ts` checks generated paths and flag order for all three platforms.
|
|
161
|
+
`test/fixtures/fullstack-browser` builds a real static browser fixture with a Python
|
|
162
|
+
FastAPI backend, a separate frontend, and a finite SSE POST. It needs no AI credentials;
|
|
163
|
+
see its README for running it.
|