npm - sandboxedjs - Versions diffs - 0.2.10 → 0.2.12 - Mend

sandboxedjs 0.2.10 → 0.2.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (43) hide show

package/README.md +96 -63
package/assets/logo.png +0 -0
package/bin/sandboxedjs-egress.mjs +25 -10
package/dist/index.cjs +1 -1
package/dist/index.js +1 -1
package/dist/service-worker.js +3 -2
package/docs/agent/COMMANDS.md +85 -0
package/docs/agent/DECISION-TREE.md +84 -0
package/docs/agent/INVARIANTS.md +40 -0
package/docs/agent/LAUNCH-PROMPT.md +37 -0
package/docs/agent/LOOP.md +84 -0
package/docs/agent/README.md +77 -0
package/docs/agent/ROADMAP.md +37 -0
package/docs/agent/STATE.md +93 -0
package/docs/agent/tasks/00-verify-inherited-work.md +36 -0
package/docs/agent/tasks/01-authoritative-metadata.md +34 -0
package/docs/agent/tasks/02-native-dependencies.md +35 -0
package/docs/agent/tasks/03-reproducible-inputs.md +27 -0
package/docs/agent/tasks/04-build-frontends.md +27 -0
package/docs/agent/tasks/05-registry-integration.md +27 -0
package/docs/agent/tasks/06-package-cohorts.md +42 -0
package/docs/agent/tasks/07-build-on-miss-boundary.md +31 -0
package/docs/browser-runtime-architecture.md +142 -0
package/docs/compatibility-implementation-plan.md +98 -0
package/docs/developer-tool-packs.md +134 -0
package/docs/frontend-automation.md +49 -0
package/docs/fullstack-deployment.md +163 -0
package/docs/handoff.md +275 -0
package/docs/original-x64.md +39 -0
package/docs/platform-hardening.md +49 -0
package/docs/python/abi.md +97 -0
package/docs/python/architecture.md +94 -0
package/docs/python/baseline-inventory.md +54 -0
package/docs/python/build-on-miss.md +198 -0
package/docs/python/compatibility.md +206 -0
package/docs/python/cross-build.md +354 -0
package/docs/python/extensions.md +282 -0
package/docs/python/release-gates.md +46 -0
package/docs/python/virtual-sockets-plan.md +331 -0
package/docs/runtime-lifecycle-fixes.md +39 -0
package/docs/server-previews.md +268 -0
package/docs/virtual-browser.md +120 -0
package/package.json +5 -3

package/docs/server-previews.md ADDED Viewed

@@ -0,0 +1,268 @@
+# Render a site running inside SandboxedJs
+For deployment recipes and the FastAPI + static frontend case, start with the
+[full-stack deployment guide](fullstack-deployment.md).
+A SandboxedJs server listens on a **virtual port**, not an operating-system socket. A log such as `http://localhost:5173` identifies the server inside the container. Pasting that URL into the host browser does not expose it.
+Choose the bridge for the environment running the container:
+| Goal | API | Result |
+| --- | --- | --- |
+| Render a whole site in a browser-only coding sandbox | `createPreview(box)` | Service-worker URL for an iframe; no Node backend required |
+| Reach a container running in Node | `box.expose(port)` | Real loopback HTTP URL |
+| Inspect an API or obtain HTML/assets without rendering | `box.request(port, { path })` | Status, headers, text and exact bytes |
+| Display one self-contained HTML response | `renderInto(box, element, { port })` | Opaque-origin iframe; not a complete site proxy |
+Use an iframe for a document with its own scripts, styles and navigation. Inserting its HTML into your IDE's DOM does not create a working site environment.
+## Browser-only: a complete small example
+Run this TypeScript in your host application's browser bundle. Supply an element with a visible height, for example `<div id="preview" style="height:500px"></div>`. Configure hosting as described below first.
+```ts
+import { createContainer, createPreview } from 'sandboxedjs';
+export async function mountSandboxSite(element: HTMLElement) {
+  const box = await createContainer({
+    cwd: '/workspace',
+    isolation: 'worker',
+    files: {
+      '/workspace/server.cjs': `
+        require('http').createServer((req, res) => {
+          if (req.url === '/app.js') {
+            res.setHeader('Content-Type', 'text/javascript');
+            res.end('document.querySelector("button").onclick = () => document.querySelector("button").textContent = "It works!";');
+          } else {
+            res.setHeader('Content-Type', 'text/html; charset=utf-8');
+            res.end('<!doctype html><html><body><h1>Inside SandboxedJs</h1><button>Click me</button><script src="/app.js"></script></body></html>');
+          }
+        }).listen(3000, () => console.log('Listening on virtual port 3000'));
+      `,
+    },
+  });
+  const server = box.spawn('node server.cjs', { cwd: '/workspace' });
+  let preview: Awaited<ReturnType<typeof createPreview>> = null;
+  const frame = document.createElement('iframe');
+  frame.title = 'Sandbox site';
+  frame.style.cssText = 'width:100%;height:100%;border:0';
+  const dispose = async () => {
+    frame.remove();
+    server.kill();
+    await server.wait();
+    try { await preview?.dispose(); }
+    finally { box.dispose(); }
+  };
+  try {
+    if (!(await box.waitForPort(3000, { timeoutMs: 30_000 }))) {
+      throw new Error('The sandbox server did not open port 3000. Inspect its stdout/stderr.');
+    }
+    preview = await createPreview(box);
+    if (!preview) {
+      throw new Error('Preview requires a secure context and service-worker support.');
+    }
+    frame.src = preview.urlFor(3000);
+    element.replaceChildren(frame);
+    return {
+      box,
+      server,
+      reload: () => { frame.src = preview!.urlFor(3000); },
+      dispose,
+    };
+  } catch (error) {
+    await dispose();
+    throw error;
+  }
+}
+// Mount after the host DOM exists:
+// const site = await mountSandboxSite(document.querySelector('#preview')!);
+// Later: site.reload();
+// On application/component teardown: await site.dispose();
+```
+This example intentionally renders trusted code on the host origin. The absolute `/app.js` request is routed back into the virtual server. No rewriting of the site's HTML or JavaScript is necessary.
+### What the browser bridge does
+1. `createPreview()` registers the shipped service worker and connects a message channel to the container owner page.
+2. `urlFor(3000)` returns a URL under that worker's scope, such as `/assets/__sbx__/3000/`. Treat the returned URL as opaque; do not hardcode the asset directory.
+3. An iframe navigation claims a virtual port. The worker associates the resulting browser client with that port.
+4. Requests from that client, including absolute asset and API paths, go through `box.request()` and return as browser responses.
+5. A previewed page's calls to another **loopback** address — `http://localhost:8000/api`, `http://127.0.0.1:8000/api` — are answered by that port in the same container, not by the reader's machine. A project split into a frontend and a backend can keep the address its code is written against. Requests to any other host (`https://api.example.com`, a LAN address) still go to the network, and a page that is not previewing anything is never redirected.
+Responses are delivered whole rather than streamed, so a page reading an
+incremental body — a server-sent-event endpoint, a streamed completion — sees
+it arrive in one piece when the response ends rather than progressively. A
+request is therefore outstanding for as long as the whole answer takes, which
+is what `timeoutMs` bounds:
+```ts
+const preview = await createPreview(box, { timeoutMs: 10 * 60_000 });
+```
+The default is five minutes. When it is exceeded the page gets a 504 whose body
+names the path, the port and this setting.
+The container owner page must remain alive. The URL is not a published website, a remote tunnel, or a standalone share link. A virtual port appearing in the terminal does not mean the host machine is listening on that port.
+## Use a real project or a terminal in an IDE
+Create the container with `network: { allowOutbound: true }` when it needs npm downloads. Load project files into its filesystem, then use the normal commands:
+```ts
+const install = await box.exec('npm install', { cwd: '/workspace/my-app' });
+if (install.exitCode !== 0) throw new Error(install.output);
+// Keep this process handle. Do not await completion of a long-running server.
+const dev = box.spawn('npm run dev', { cwd: '/workspace/my-app' });
+if (!(await box.waitForPort(5173, { timeoutMs: 60_000 }))) {
+  dev.kill();
+  throw new Error('Dev server did not start; show its logs.');
+}
+const preview = await createPreview(box);
+if (!preview) throw new Error('Browser preview unavailable');
+iframe.src = preview.urlFor(5173);
+```
+For an interactive terminal, use `Terminal` with `box.session()` and let the user run `npm create vite`, answer the prompts, and choose installation/startup. Worker support is required for the synchronous child-process calls used by scaffolding tools. The preview can attach to that same container; do not create a second container just to render its site.
+For an IDE's port selector, poll `box.net.listening()` and use each entry's `port`. Do not assume every framework uses 5173. For programmatically spawned commands, consume `stdout` and `stderr` using their asynchronous `read()` methods; `Terminal` already forwards terminal output to its configured writer.
+Keep the lifetimes separate:
+- Hide/close the pane: remove or hide the iframe; keep the server and container alive.
+- Reload the page preview: assign `frame.src = preview.urlFor(port)` again.
+- Stop the server: kill the owned process, or send Ctrl+C through the active terminal; wait for its port to disappear and clear the frame.
+- Replace a project: remove its iframe and await the old preview's disposal before attaching the new container.
+- Destroy the IDE session: stop owned processes, dispose the preview, and dispose the container.
+The companion CLI implements an icon toggle, port selection, reload/close, a left-side divider on desktop, and a bottom divider on mobile. The divider supports pointer dragging, arrow keys and double-click reset. These are UI concerns; the core bridge only provides URLs.
+## Hosting the browser application
+The IDE host page—not the Vite server inside the sandbox—needs these response headers for shared memory and threaded WASM:
+```http
+Cross-Origin-Opener-Policy: same-origin
+Cross-Origin-Embedder-Policy: require-corp
+```
+Use HTTPS in deployment, or localhost/loopback for local development. Service workers must be enabled. Check the actual page's `crossOriginIsolated` value; a meta tag cannot enable it.
+For a Vite-based host application:
+```ts
+import { defineConfig } from 'vite';
+const headers = {
+  'Cross-Origin-Opener-Policy': 'same-origin',
+  'Cross-Origin-Embedder-Policy': 'require-corp',
+};
+export default defineConfig({
+  optimizeDeps: { exclude: ['@rolldown/binding-wasm32-wasi'] },
+  server: { headers },
+  preview: { headers },
+});
+```
+### Cloudflare Pages
+Cloudflare Pages can host the outer application as static assets. In a Vite host application, create `public/_headers`, build with
+`npm run build`, set the Pages output directory to `dist`, and verify that Vite copied
+the file to `dist/_headers`:
+```text
+/*
+  Cross-Origin-Opener-Policy: same-origin
+  Cross-Origin-Embedder-Policy: require-corp
+  Cross-Origin-Resource-Policy: same-origin
+```
+Pages treats `_headers` as header configuration. It is not a file the browser needs to download.
+After deployment, verify `crossOriginIsolated` in the deployed page and confirm that the service
+worker script is a JavaScript response under the same HTTPS origin. A Pages static deployment
+hosts the IDE and its service worker; it does not host the virtual sandbox port separately.
+The site running inside SandboxedJs is fetched through the service-worker/message bridge in the
+browser visitor's tab.
+Excluding the binding preserves the URL of its WASI worker. For a built application, the package also includes a local static host:
+```sh
+npm run build
+npx sandboxedjs-serve ./dist 4173
+```
+This starts the **outer host application** with the required headers. It does not replace `npm run dev` inside the container and does not move guest execution to Node. For production, configure your actual host/CDN with HTTPS, these headers, correct JavaScript/WASM MIME types and real asset URLs. Cross-origin assets need compatible CORS/CORP policies.
+The preview service worker must be emitted as a same-origin HTTP(S) file, not an inlined `data:` URL. The default package URL includes Vite's `?no-inline` hint. Other bundlers must preserve/copy the worker asset. If you relocate it, use `createPreview(box, { scriptUrl: '/preview/service-worker.js' })`. A custom `scope` must be permitted by the worker script location or the host's `Service-Worker-Allowed` header.
+## Node-hosted containers
+`box.expose()` opens a real Node HTTP listener. It is not the browser-only API.
+```ts
+import { createContainer } from 'sandboxedjs';
+const box = await createContainer({
+  files: {
+    '/app/server.cjs': "require('http').createServer((q,r)=>r.end('Hello from the sandbox')).listen(3000)",
+  },
+});
+const server = box.spawn('node server.cjs', { cwd: '/app' });
+if (!(await box.waitForPort(3000, { timeoutMs: 30_000 }))) {
+  server.kill();
+  box.dispose();
+  throw new Error('Server startup failed');
+}
+const bridge = await box.expose(3000); // random available loopback port
+console.log(bridge.url);             // open this in a browser or iframe
+// For a fixed host port: box.expose(3000, { hostPort: 8080 })
+// When finished:
+// await bridge.close();
+// server.kill();
+// await server.wait();
+// box.dispose();
+```
+Loopback is reachable only from the host machine. Remote access needs an explicitly designed backend/proxy and authentication. An HTTPS frontend cannot freely embed an HTTP backend; plan TLS and framing policies for deployments.
+This bridge does not make all packages host-independent. In particular, the current Vite 8/Rolldown WASI integration uses the browser binding's mirrored filesystem; the Node binding uses the real host filesystem. See the README's known limits before choosing a Node-hosted toolchain. A plain HTTP server is supported on both hosts.
+## A response without a full site
+```ts
+const response = await box.request(3000, { path: '/api/items' });
+console.log(response.status, response.headers, response.body);
+const exactBytes = response.bytes;
+```
+For self-contained HTML, `renderInto()` uses an iframe with `sandbox="allow-scripts"` and an opaque origin:
+```ts
+import { renderInto } from 'sandboxedjs';
+await renderInto(box, document.querySelector('#preview')!, { port: 3000 });
+```
+This renders one response. It does not route the document's relative/absolute scripts, styles, images or API calls into the container. It is unsuitable as a replacement for `createPreview()` when displaying a normal Vite application. The sandbox attribute also is not a network-access policy.
+## Boundaries to plan for
+- **Trust:** `createPreview()` serves guest scripts on the host origin. They can access parent-page state, cookies and storage. It is not an isolation boundary for untrusted code. Running the preview/runtime on a separate origin requires additional architecture; there is no `previewOrigin` switch that implements this for you.
+- **HMR:** the injected WebSocket shim tunnels supported guest connections through the owner page. It is enabled by default; disabling injection or blocking it with CSP removes this support. External WebSockets and framework-specific behavior need separate testing.
+- **Multiple sessions:** the current service worker has one connected container per registration. Reusing the same scope across containers/tabs can replace that connection, and disposing one registration affects its users. Isolate registrations or use one active session; scopes alone do not provide a security boundary. Concurrent browser Rolldown projects also need distinct absolute working directories because the binding shares a WASI filesystem per page.
+- **Lifetime:** the worker asks live owner pages to reconnect after restart. Its in-memory client bindings can still be lost; explicit preview-prefixed URLs remain routable. This does not persist the container after the owner page closes or provide multi-tenant routing.
+- **HTTP coverage:** the bridge buffers responses; it is not a general TCP socket, streaming transport or complete reverse proxy. Absolute external URLs, redirects, cookies and application-specific framing policies may need additional handling. Check the behavior your framework actually needs.
+## Troubleshooting a blank or refused iframe
+1. Check the current page/build. An older bundle or stale iframe may still contain a previously fixed error. Save/export needed work before refreshing the host page; tab-memory sessions are not persisted.
+2. Confirm the virtual server is listed by `box.net.listening()` and `await box.request(port, { path: '/' })` returns the expected HTML.
+3. Confirm the iframe uses `preview.urlFor(port)`, not the guest's printed `localhost` URL.
+4. Inspect the service-worker registration, script URL, MIME type, scope and browser console. A successful registration is not evidence that every later navigation was routed.
+5. Inspect framing/isolation errors separately from connection errors: CSP, `X-Frame-Options` and COEP can replace a frame with a refusal page even while the virtual server is running.
+6. If HTML/assets work but only a WebSocket fails, that is the current HMR limitation, not proof that the HTTP server failed.
+A browser-native refusal page is not a diagnosis by itself. Compare the virtual HTTP response, bridge state and actual browser error before changing framework commands.

package/docs/virtual-browser.md ADDED Viewed

@@ -0,0 +1,120 @@
+# Virtual browser
+Anything that drives Chromium through the DevTools protocol — Playwright from
+Node.js or Python, Puppeteer — gets the SandboxedJs virtual browser instead of
+a Chromium binary. No browser is downloaded and nothing is patched in the
+driver: the protocol is the seam.
+```js
+// inside a container, after `npm install playwright-core`
+import { chromium } from 'playwright-core';
+const browser = await chromium.launch();
+const page = await browser.newPage();
+await page.goto('https://example.test/');
+console.log(await page.evaluate(() => 6 * 7)); // 42
+await browser.close();
+```
+```python
+# after `pip install playwright`
+import asyncio
+from playwright.async_api import async_playwright
+async def main():
+    async with async_playwright() as p:
+        browser = await p.chromium.launch()
+        page = await browser.new_page()
+        print(await page.evaluate("() => 6 * 7"))
+        await browser.close()
+asyncio.run(main())
+```
+## How a launch is routed
+1. **Installing Playwright writes the browser it will look for.** When
+   `playwright-core` is installed with npm, or `playwright` with pip, the
+   executables named in its `browsers.json` are written under
+   `~/.cache/ms-playwright` as stubs for the `chrome` built-in. Only Chromium
+   (and its headless shell) is provided; Firefox and WebKit stay missing, so
+   asking for them fails with Playwright's own message.
+2. **Playwright spawns it with `--remote-debugging-pipe`.** Its `stdio` is
+   `['ignore', 'pipe', 'pipe', 'pipe', 'pipe']`: commands arrive on fd 3 and
+   replies leave on fd 4, each JSON message terminated by a NUL byte.
+   `child_process` supports pipes above fd 2 for this, in-realm and through
+   the worker.
+3. **`chrome` hands every message to `CdpServer`** (`src/browser/`), which owns
+   targets, sessions, frames and execution contexts.
+`chrome`, `chromium`, `chromium-browser`, `google-chrome` and
+`chrome-headless-shell` are all the same built-in.
+## What a page is today
+A page has a real, isolated JavaScript realm — `node:vm` in Node.js, a detached
+iframe in a browser — and **no DOM yet**. That covers `launch`, contexts,
+`newPage`, `goto` (the navigation commits and fires its lifecycle events; the
+URL is not fetched), `url`, `evaluate` / `evaluateHandle` with arguments,
+promises and thrown errors, and `close`.
+Anything that needs a document — locators, `click`, `fill`, `content`,
+`screenshot`, network interception — does not work yet. Unimplemented protocol
+methods are refused with Chromium's own `'Method' wasn't found` error rather
+than answered with an empty success, so a driver fails where the gap is.
+`*.enable`, `*.disable` and `set*` configuration calls are accepted, because a
+page without the feature can honestly take them.
+The DOM, CSS layout and painting engine plugs in behind `CdpServer`; that is
+the next stage.
+## Python specifics
+`playwright` publishes only platform wheels, because each bundles Node.js. The
+resolver accepts its Linux x86-64 wheel by name (`src/python/substitutions.ts`)
+and the install replaces `driver/node` with a script that runs the container's
+`node`.
+`greenlet` cannot work in WebAssembly — it switches native stacks. A built-in
+stand-in is installed instead: it imports and can be subclassed, and switching
+raises `greenlet.error`. Playwright only switches greenlets in its synchronous
+API, so **`playwright.async_api` works and `playwright.sync_api` fails** with
+that error, which names the asyncio API.
+Playwright's Python driver is the container's Node.js speaking a
+length-prefixed binary protocol over stdin and stdout, which exercised several
+general gaps, now fixed:
+- **Binary stdio.** A `Buffer` written to stdout, or read from a live pipe,
+  used to be decoded as UTF-8 on the way through. Pods still emit text on
+  `output`; bytes as written go on `raw-output`, which `node` forwards into
+  pipes.
+- **Live stdin.** A pipe inherited from Python was reported as non-interactive,
+  so `node` read it to end-of-file before starting the script — and a driver
+  whose stdin never closes never started.
+- **Pipe backpressure.** A kernel pipe holds 64 KiB and accepts a prefix of a
+  larger write; the child-output wrapper ignored the count and dropped the
+  rest. Output is now queued and written as the reader makes room.
+- **Built-in interop.** A partially supported built-in answered `__esModule`,
+  so bundlers' `__toESM` lost its `default` (`net`, read by Playwright at load).
+- **ESM detection.** `new.target` in CommonJS was mistaken for `import.meta`.
+And two in the Python runtime itself:
+- **asyncio subprocesses.** Without `pidfd_open`, CPython waits for children
+  on a helper thread, whose blocking host call fails with EIO. A polling child
+  watcher (installed through the runtime's `sitecustomize`) reaps with
+  `WNOHANG` on the loop's own thread instead.
+- **Kernel pipes in selectors.** A pipe's descriptor number inside the
+  interpreter differs from the kernel's; `poll` now translates it (sockets
+  already use kernel numbers and are tracked so they are never translated),
+  a stream's readiness is asked of the kernel with a zero timeout, and
+  `O_NONBLOCK` reads return `EAGAIN` instead of waiting.
+## Not supported
+- Headed Chromium, `--remote-debugging-port`, and connecting over WebSocket
+  (`connectOverCDP`): only the pipe transport is served.
+- Firefox, WebKit.
+- The Python synchronous API.
+- Anything needing a DOM, layout or pixels (for now).

package/package.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "sandboxedjs",
-  "version": "0.2.10",
+  "version": "0.2.12",
   "description": "A Linux-like container that runs entirely inside Node.js — POSIX shell, ~140 coreutils, Node.js and Python runtimes, virtual filesystem and networking. No Docker, no VM, no native modules.",
   "type": "module",
   "license": "MIT",
@@ -53,7 +53,9 @@
     "!dist/**/*.map",
     "bin",
     "README.md",
-    "LICENSE"
+    "LICENSE",
+    "assets/logo.png",
+    "docs"
   ],
   "scripts": {
     "build": "tsup && node python-runtime/scripts/copy_runtime.mjs",
@@ -63,7 +65,7 @@
     "test:watch": "vitest",
     "check": "npm run typecheck && npm run build && npm test",
     "prepublishOnly": "npm run typecheck && npm run build && npm run test:release",
-    "test:release": "vitest run test/python-runtime/extension-abi.test.ts test/python-runtime/resolver.test.ts test/no-static-node-builtins.test.ts"
+    "test:release": "vitest run test/python-runtime/extension-abi.test.ts test/python-runtime/resolver.test.ts test/no-static-node-builtins.test.ts test/preview-worker-build.test.ts test/egress-cli.test.ts"
   },
   "keywords": [
     "sandbox",