sandboxedjs 0.2.10 → 0.2.12

package/docs/handoff.md

@@ -0,0 +1,275 @@
+# Handoff — SandboxedJS
+
+## Repo state
+
+`npx tsc --noEmit` passes, `npm run build` passes, and **316 tests pass**
+including the network acceptance suite:
+
+```bash
+SANDBOXEDJS_CLEAN_NETWORK_TESTS=1 npx vitest run
+```
+
+(The network tests hit the real npm registry and take ~2 min: Express,
+`npx serve`, `npm install` + `npx`, `npm create vite`, a Vite 7 dev server, and
+an interactive clack prompt driven through `Terminal`.)
+
+## Context
+
+`@scelar/nodepod` was removed entirely (licence: Commons Clause). The Node.js
+runtime is this package's own, under `src/runtime/`. It must stay
+browser-capable: **nothing in `src/runtime/` may import a `node:` builtin**
+except through `nodeOnlyModule`/`nodeBuiltin` (`src/util/binary.ts`) on a path
+that returns `null` off Node. Several npm shims had to be replaced because they
+are not browser-safe — see `util-module.ts`, `assert-module.ts`,
+`zlib-module.ts`, `url-module.ts`, `readline-module.ts`, `readable-from.ts`.
+
+## Verified in a real browser
+
+Chrome, against the Vite host app at
+`/Users/shazi/Practice/sandboxedjs tes/browser-server` (`npm run dev`). That
+project has a `check.html` / `src/browser-check.ts` page that boots a container
+and drives `Terminal` through the whole set; open `/check.html` and it prints a
+pass/fail line per check:
+
+- `readline.question` answered, with the typed text echoed as it is typed
+- `https.get` from inside the sandbox reaching the npm registry
+- clipboard round-trip through `pbcopy`/`pbpaste`
+- `npm create vite@latest` accepting a project name and advancing
+- `ctrl+c` cancelling a prompt and returning to the shell
+- `npx serve --debug` starting with no errors and serving a file
+
+## How isolation is put together
+
+`WorkerRuntimePod` (`src/runtime/worker-runtime-pod.ts`) extends
+`LocalRuntimePod` and overrides `spawn` alone; everything else in the contract
+is identical. Both are held to `test/pod-contract.ts`, which is the thing to
+extend when either changes.
+
+The topology is forced, not chosen. A synchronous call must block its caller
+while the work it waits on still progresses, so the blocking side cannot own
+the shared state — otherwise a child needing the filesystem would call into a
+frozen thread. Hence: guest in the Worker, volume and kernel on the host.
+
+- `sync-channel.ts` — `SharedArrayBuffer` + `Atomics.wait`, chunked both ways.
+  Only the client blocks; the server answers from its ordinary event loop.
+- `remote-volume.ts` — `RuntimeVolume` over that channel. Because the interface
+  was already the seam, `core-modules`, `Vfs` and the kernel are untouched.
+- `sync-syscalls.ts` — the filesystem and `spawnSync` share one channel.
+- `worker-entry.ts` — the guest half, built as its own bundle.
+
+Two packaging traps, both already paid for:
+
+1. The guest bundle must be **fully self-contained** (`noExternal` in
+   `tsup.config.ts`). The main bundle leaves `buffer` and friends external and
+   lets the host's bundler map them onto polyfills; the Worker is fetched as
+   its own module graph and gets no such help, so an external `buffer` resolves
+   to raw CommonJS with no named exports and the Worker dies before it runs.
+2. `new URL("./worker-entry.js", import.meta.url)` **does** survive Vite's
+   dependency pre-bundling — verified in Chrome, it fetches from
+   `node_modules/sandboxedjs/dist/`. Do not "fix" this with a blob URL: a blob
+   worker has no module-resolution context and could no longer `import()` the
+   host's esbuild or Rolldown binding.
+
+## Verifying the preview service worker
+
+The in-app browser pane refuses service worker registration outright — even a
+one-line worker fails — so the preview cannot be checked there and will always
+report itself unavailable. Real Chrome works. The quickest check is
+`/swcheck.html` in the harness, which stands up a tiny server and asserts that
+an absolute-path subresource is routed; `/preview.html` runs the full
+`create-vite` flow and takes about half a minute. Headless Chrome over CDP works
+too, which is how this was verified.
+
+Three things bit during that verification and are easy to re-introduce:
+`navigator.serviceWorker.ready` never resolves for a worker whose scope excludes
+the registering page; the claim path arrives under the worker's scope directory,
+so its pattern must not be anchored to the start; and a framed response needs
+`Cross-Origin-Embedder-Policy` as well as `Cross-Origin-Resource-Policy` or an
+isolated parent refuses it.
+
+## A trap worth remembering: inherited stdin is a terminal
+
+`stdio: "inherit"` hands a child the parent's standard input, and a terminal
+never ends. Give it a pipe and anything that reads to end-of-file first will
+wait forever — and `node` does that in `captureStdin` (`src/runtime/node.ts`)
+before it runs a script. The symptom is remote from the cause: a shell script
+that launches node prints nothing at all and hangs, with no error anywhere.
+
+`KernelChildProcess` (`node-child-process-bridge.ts`) therefore marks an
+inheriting child's stdin `isTTY`/`interactive`. `test/sync-child-process.test.ts`
+pins the reduced case — `spawnSync('sh', ['-c', 'node -e …'], {stdio:"inherit"})`
+— which is much cheaper to run than the create-vite flow it was found in.
+
+## The interactive-input chain, for future work
+
+1. `Terminal.key()` — `src/container/terminal.ts`. While a command runs it
+   writes each keystroke to `this.currentStdin` (a `Pipe`). **Raw mode changes
+   what it does**: no local echo, no CR→LF translation, and `ctrl+c`/`ctrl+d`
+   are passed through as keystrokes rather than becoming a signal and EOF.
+2. `node` command — `src/runtime/node.ts`, `execute()`. Forwards `ctx.stdin`
+   into `proc.write(...)`, propagates EOF via `proc.endInput()`, and mirrors the
+   program's `rawmode` event back onto `ctx.stdin.rawMode`.
+3. `LocalProcess.write` — `src/runtime/local-runtime-pod.ts`. Buffers input
+   until the task starts, then hands it to `core.writeStdin`.
+4. `createCoreModules` — `src/runtime/core-modules.ts`. With
+   `interactiveStdin: true`, `process.stdin` is a `PassThrough` that stays open;
+   `tty: true` sets `isTTY` on all three streams; `setRawMode` calls back into
+   `options.onRawMode`.
+5. `readline-module.ts`. `emitKeypressEvents(stream)` turns incoming data into
+   `keypress` events. `createInterface` defaults `terminal` from `output.isTTY`,
+   as Node does; a terminal `Interface` owns raw mode, keeps `line`/`cursor`
+   current, and echoes. **Prompt libraries read `rl.line` for the answer**, so
+   anything that stops it tracking shows up as an empty answer and a re-prompt,
+   not as an error.
+
+`settle()` in `local-runtime-pod.ts` decides when a process is finished. It
+counts pending timers, `core.readingStdin()` and `core.pendingRequests()` — an
+in-flight HTTP request is event-loop work and schedules no timer of its own.
+Check this first if a process exits before something asynchronous completes.
+
+## Known gaps
+
+- **Vite 8 / Rolldown works in a browser, not under a Node host.** (The old note
+  saying it cannot run at all was wrong.) In a browser it needs two things from
+  the host app, both now in the README: COOP/COEP headers, and
+  `optimizeDeps.exclude: ["@rolldown/binding-wasm32-wasi"]` so the bundler does
+  not pre-bundle away the `import.meta.url` its WASI worker is created from.
+  With those, `npm create vite` installs and the dev server serves transformed
+  modules — verified in Chrome. Under Node the binding's other build makes a
+  `node:wasi` instance preopening the real filesystem root, which cannot be
+  redirected at the sandbox volume, so Rolldown never finds the project and Vite
+  answers with its fallback page. Fixing that means running the *browser* build
+  on Node, which needs a `fetch` that handles `file:` URLs and a global `Worker`
+  over `worker_threads`. Vite 7 works on both and is what the Node test pins.
+
+- **`spawnSync`/`execSync`/`execFileSync` work under the Worker pod**, which is
+  the default. See *Isolation* in the README for the fallback rules. Under the
+  in-realm pod they still throw, naming the command — guest, child and event
+  loop share a thread there, so blocking the caller stops the child.
+- **esbuild in a browser.** `host-esbuild.ts` borrows the host's `esbuild-wasm`
+  on Node and returns `null` in a browser, so browser Vite transforms fail.
+- **No service worker**, so preview iframes have no URL. `box.request()` works
+  everywhere.
+- **`curl` to a non-CORS host cannot work in a browser.** Platform limit, not a
+  bug; the error message says so. The npm registry does send CORS headers, which
+  is why installing packages works.
+- **`child_process.execSync` and friends** cannot exist: they would have to
+  block the JS thread. They throw `ERR_FEATURE_UNAVAILABLE_ON_PLATFORM`.
+- Vite prints two `util` externalization warnings from `readable-stream`, which
+  declares `"util": false` for browsers and falls back on its own. Harmless.
+
+## Python: blocking syscalls, and what they are built on
+
+`src/runtime/python-syscalls.ts` is the layer that makes Pyodide behave like a
+Linux Python rather than a sandboxed evaluator. The constraint it removes is
+that **WebAssembly cannot wait for a JavaScript promise**: Pyodide's `setStdin`
+callback is synchronous, so anything only the host can answer asynchronously
+had to be answered immediately or not at all. Answering "not at all" is what
+made `input()` raise `EOFError`, and it is the same wall behind `socket`,
+`subprocess` and `time.sleep`.
+
+The primitive is WebAssembly stack switching (JSPI), reached through Pyodide's
+`run_sync`. It suspends the whole interpreter stack until a host promise
+settles, so an ordinary `def` — nested arbitrarily deep — can block while the
+event loop keeps running. Verified available unflagged in Node 25 and Chrome
+137+. **Adding a blocking syscall is now a method on the host facade**, not
+another special case; that is the point of the file.
+
+Three are wired:
+
+- **stdin.** `sys.stdin` is rebuilt as a real `TextIOWrapper`, not a patched
+  `input()` — otherwise `csv.reader(sys.stdin)` stays broken. EOF still raises
+  `EOFError` as CPython does.
+- **processes.** Only `subprocess.Popen` is replaced; `run`, `call`,
+  `check_call` and `check_output` are written in terms of it upstream, so the
+  family comes with it. The child is a *live* process on the host's event loop
+  (`kernel.spawn` + pipes), not a finished result: the parent suspends only
+  when it reads, waits or communicates, so `for line in p.stdout` follows a
+  child that is still running and `poll()` can say "running". `os.system` and
+  `os.popen` go to the same place — `os.system` previously returned 0 having
+  run *nothing*, which reads as success and is worse than an error.
+- **HTTP.** `urllib` handlers call `performRequest` (`src/net/commands.ts`),
+  the same path as `curl`. Deliberately not `fetch`: that is what makes Python
+  obey the container's `allowOutbound` policy instead of routing around it.
+
+### Third-party HTTP stacks, and a network-policy escape
+
+A library that brings its own transport does not go through `urllib`, and in
+Pyodide several reach JavaScript's `fetch` directly. **`requests` used to reach
+the internet from a container with `allowOutbound` off, while `curl` in the
+same container was correctly refused** — a real hole in the sandbox, found by
+testing the policy rather than the feature. `test/python-syscalls.test.ts`
+guards it.
+
+Such libraries cannot all be patched at boot, because pip installs them later.
+So adapters are registered by module name and applied when that module is first
+imported, through a hook on `builtins.__import__`. **Supporting another stack
+is a small function registered there, not a change to the machinery.** The
+`requests` adapter replaces `HTTPAdapter.send` — the one seam every call
+crosses, below sessions/redirects/cookies/retries and above the urllib3
+transport that would otherwise reach the network itself.
+
+### Hosts without stack switching
+
+Everything blocking rests on JSPI, and Pyodide 0.28 ships JSPI-only — there is
+no Asyncify build to fall back to. Where it is missing, the degradation is
+deliberate rather than incidental:
+
+- piped and redirected stdin still work, from the pre-drained buffer;
+- anything that must genuinely wait (interactive input, a child process) says
+  so plainly instead of faking end-of-file;
+- `requests` falls back to the library's own transport rather than breaking —
+  **but the network policy is not the part that degrades.** A host the
+  container forbids is still refused. Capability degrades; the sandbox does not.
+
+The only way to get blocking without JSPI is a Worker plus
+`SharedArrayBuffer`/`Atomics.wait` — the machinery `sync-channel.ts` already
+has for the Node runtime. It is a real project: Pyodide would move off the main
+thread and its Emscripten filesystem would have to reach the volume over the
+channel, and in a browser it additionally requires cross-origin isolation
+(COOP/COEP), which is a deployment requirement and not only code.
+
+Five traps, each already paid for and each cheap to reintroduce:
+
+1. **Pyodide maps JS `null` to a truthy `JsNull` proxy; only `undefined`
+   becomes `None`.** `buffered()` returns `undefined` to mean "you must
+   suspend", and a `?? EMPTY` on that path silently turned it back into
+   end-of-file — the exact bug the bridge exists to remove.
+2. **One interpreter is shared per container**, so its streams belong to
+   whichever program is running. A Python child binds over its parent while the
+   parent is suspended, so every host call that can suspend goes through
+   `resuming()`, which puts the parent's binding back.
+3. **`kernel.spawn` does not close a child's output pipes when it exits** —
+   `Container.spawn` does that itself — and a reader waiting on a pipe that
+   never ends waits forever. This deadlocked the whole suite once.
+4. **`Popen` must not run at construction.** `subprocess.run` hands `input=` to
+   `communicate()`, never to `Popen`.
+5. **Pyodide builds urllib without `ssl`, so `urllib.request.HTTPSHandler` does
+   not exist** to subclass. One handler subclassing `HTTPHandler` serves both
+   schemes, which also keeps `build_opener` treating it as a replacement.
+
+`asyncio.run` and `run_until_complete` are patched onto `run_sync` for the same
+reason: programs execute under `eval_code_async`, so Pyodide's loop is always
+already running and `asyncio.run` refuses to start a second one.
+
+Known remaining gaps:
+
+- **Python cannot accept connections.** `loop.create_server` is unimplemented
+  on Pyodide's WebLoop, so uvicorn and aiohttp install, import and start but
+  never bind — outbound requests work, inbound do not. Closing this means
+  widening the `RuntimePod` contract with a serve hook so Python can register
+  with `VirtualHttpRouter` (`src/runtime/virtual-http.ts`), the same router
+  that already routes `curl localhost:3000` to a Node server in the container,
+  plus an asyncio transport that frames HTTP between the router's structured
+  requests and the protocol's raw bytes. `router` is currently `protected` on
+  `LocalRuntimePod`; `WorkerRuntimePod` inherits it, so one implementation
+  covers both. For now the failure at least explains itself.
+
+- A library with its own transport and no registered adapter can still reach
+  the network directly and escape the policy. `requests` and `urllib` are
+  covered; the hook makes the next one cheap, but it is opt-in by design.
+- Package resolution: `loadPackagesFromImports` only sees Pyodide's lockfile
+  and `micropip` only installs pure-Python wheels, so `opencv-python` and
+  everything else needing a C extension built for wasm32-emscripten remains out
+  of reach. The fix is a resolution chain (lockfile → PyPI pure-Python → a wasm
+  binary-wheel index → clear failure), not a special case per library.

package/docs/original-x64.md

@@ -0,0 +1,39 @@
+# Original experimental x86-64 engines
+
+`createOriginalX64Backends()` returns two optional backends written in this
+repository: an instruction-to-WebAssembly translator and a JavaScript instruction
+interpreter. Neither embeds a third-party compiler or CPU emulator. Register the
+returned backends with `box.kernel.binaries.register(backend)`.
+
+The translator emits real Wasm integer operations, using mutable register globals
+and an instruction cache. It translates lazily during execution, rather than
+producing a distributable standalone Wasm file. The interpreter executes the same
+decoded instructions directly. They currently support the same instruction subset;
+register only the `emulation` backend to exercise the interpreter directly.
+
+Supported inputs are freestanding, static, little-endian x86-64 ET_EXEC ELF files.
+Loadable segments must fit a 16 MiB memory window and use addresses below 4 GiB.
+Dynamic linking is rejected. Supported instructions: 32/64-bit immediate and
+register MOV, register ADD/SUB/XOR/CMP, selected immediate ADD/SUB/CMP, RIP-relative
+LEA, NOP, short JMP/JZ/JNZ, relative JMP, and SYSCALL. Only the zero flag is modeled.
+There is no stack/argv ABI, arbitrary memory operand, SSE, libc, file-open syscall,
+networking, threads, or Docker support. Unsupported instructions fail explicitly.
+
+Linux calls: read stdin, write stdout/stderr, exit/exit_group, getpid. Unsupported
+calls return ENOSYS. Segment read/write permissions are checked for I/O buffers;
+self-modifying code is rejected. Each run owns its registers and memory. The
+one-million-instruction default budget and periodic event-loop yields bound loops
+and permit cancellation between batches. This is not a hardened security boundary
+or a claim of native performance. Translation currently has substantial per-
+instruction overhead; performance optimization is future work.
+
+Tests use `test/fixtures/native/hello.s`, assembled by Clang for Linux. The checked
+in ELF is built with the adjacent `package-elf.py` script. Both engines run the
+same binary, including a conditional loop and Linux console output; an infinite
+loop fixture verifies the instruction limit. Clang is only used to produce test
+inputs; it is not part of either engine at runtime.
+
+The local browser project registers both backends and exposes `native-hello`.
+Opening `/?native=emulation` selects the interpreter. Node and Python commands
+continue through their existing runtimes. This is an initial working CPU subset,
+not arbitrary Linux package compatibility.

package/docs/platform-hardening.md

@@ -0,0 +1,49 @@
+# Platform hardening
+
+SandboxedJs is a virtual OS runtime for browser clients and Node hosts. It is
+not a recording application. Continue developing its own JavaScript module
+engine, virtual kernel, filesystem, network and CPython WebAssembly integration;
+do not replace them with Nodepod or Pyodide.
+
+## Release gates
+
+Completion requires evidence in both a real browser and a supported Node host.
+A successful bundle alone does not establish browser runtime compatibility.
+
+1. Runtime lifecycle: repeated boot, cancellation, child-process termination,
+   disposal and startup failure leave no workers, timers, sockets or mounts.
+2. Isolation: concurrent guests cannot observe each other's environment or
+   filesystem. Worker execution and realm fallback must be tested separately.
+3. Package workloads: install, run, edit and rebuild representative frontend
+   and API projects. Test actual package imports rather than lookalike APIs.
+4. Python: test dotenv loading, imports, subprocesses, HTTP servers and async
+   I/O in the shipped interpreter. Resolve extension setup timeouts before
+   claiming the complete Python suite passes.
+5. Performance: measure cold and warm boot, installation, module loading,
+   rebuild latency, memory after disposal and large console output. Record
+   runtime version, host, workload and repetitions with each result.
+6. Distribution: verify published ESM, CommonJS, worker and static browser
+   assets, including missing assets and unsupported host features.
+
+## Current findings
+
+- Corrected environment support to expose `util.parseEnv`; the previous
+  `process.parseEnv` addition did not match native Node. Added
+  `process.loadEnvFile` and native-Node differential parser fixtures.
+- Typed-array inspection now reads at most 100 entries instead of copying the
+  entire array. DataView logging no longer attempts to iterate a non-iterable.
+- The full test run was interrupted after Python extension setup timed out.
+  Follow-up isolated EPERM on the test's loopback listener and fixed missing
+  setup error handling. All 12 extension tests passed when loopback listening
+  was permitted. This is still not a full release pass.
+- Several Node modules still have stubs: cluster, dgram, diagnostics_channel,
+  domain, http2, inspector, tls, v8, vm and worker_threads. Implement and test
+  usable APIs before marking those modules supported.
+- The Python thread/syscall limitation documented in
+  browser-runtime-architecture.md needs interpreter-image work.
+- Browser CORS, static asset hosting, cross-origin isolation requirements,
+  native addons and platform-specific binaries remain constraints. Arbitrary
+  packages cannot be promised to run solely from this JavaScript runtime.
+
+The platform objective remains open. These gates are an implementation and
+verification backlog, not a claim of complete Node, Python or Linux parity.

package/docs/python/abi.md

@@ -0,0 +1,97 @@
+# `sbx_host_v1` — the host ABI
+
+The contract between compiled guest code and the SandboxedJs kernel.
+
+Everything a guest cannot do for itself — files, descriptors, pipes, readiness,
+time, identity, entropy, and later sockets, processes and signals — crosses this
+boundary and nothing else. Emscripten and WASI adapters translate *into* it;
+they are not the source of truth. That distinction is the point: compiling a
+program to WebAssembly does not give it an operating system, and pretending an
+Emscripten build's own filesystem is the container's is how two divergent copies
+of the same state come to exist.
+
+## Source of truth
+
+`python-runtime/abi/host-v1.json` defines the version, the operation codes, the
+errno values and the capability names. `make -C python-runtime abi` regenerates:
+
+- `python-runtime/abi/sbx_host.h` — the C view.
+- `src/python/host-abi.ts` — the TypeScript view.
+
+Both are committed. Neither is edited by hand: a C file and a TypeScript file
+that disagree about an opcode produce a wrong answer, not a link error.
+
+## Framing
+
+```
+request:   u16 version | u16 op | u32 request_id | u32 generation | u32 length | payload
+response:  u16 version | u16 op | u32 request_id | u32 generation | i32 status | u32 length | payload
+```
+
+Little-endian, fixed width, UTF-8 for text, raw bytes for content, 64-bit file
+offsets. `status` is the operation's result: `>= 0` on success, `-errno` on
+failure, with the canonical (Linux/musl) numbers.
+
+Three rules the transport must keep, each of which was a real defect before it
+was written down:
+
+1. **A transport failure is never a successful empty result.** `read` returning
+   zero bytes means end of input. If losing the host can produce that same
+   answer, input silently vanishes instead of raising.
+2. **A stale answer is not an answer.** `request_id` and `generation` are
+   checked on the way back. A completion belonging to a previous incarnation of
+   a PID must not land on the current process's descriptor table.
+3. **A closed transport stays closed.** Once the host has gone, every later call
+   fails immediately. Without that, a killed process republishes a request over
+   the closed marker and hangs on its way out.
+
+The guest writes a zero into `generation`; the transport stamps the real value,
+because the transport belongs to one process and that is where the authority
+lives. Nothing in guest-controlled data may be trusted as identity.
+
+## Blocking
+
+Only the guest blocks, and the guest never owns shared state.
+
+```
+Python calls read(fd)
+  → libc adapter invokes sbx_host_v1
+  → the process worker parks in Atomics.wait
+  → the kernel thread waits for input on its own event loop
+  → the kernel writes the response and notifies
+  → the worker resumes, read() returns
+```
+
+Keyboard input, network responses and storage completions must reach the
+*kernel*, not arrive as messages the blocked worker would have to process. A
+blocked worker processes nothing.
+
+This requires `SharedArrayBuffer` and `Atomics.wait`, so a browser host must be
+cross-origin isolated, and the guest must never run on the main thread.
+
+## Status
+
+Implemented and covered by `test/python-abi/`: files, descriptors, pipes,
+readiness, time, identity, entropy. Reserved but not implemented: sockets,
+processes, signals, storage, services, threads — `handshake` reports which is
+which, so a guest can tell "not present" from "not implemented" rather than
+discovering it at the first call.
+
+## Known limit
+
+`unlink` and `rename` preserve open-file semantics only for names changed
+*through this ABI*. A name removed by the shell or the Node side cannot be
+intercepted, and an open description will then fail with `ENOENT` rather than
+reading stale bytes. Making that hold everywhere means moving the operations
+into the volume itself, which is M3 work.
+
+## Errno translation is not a formality
+
+The canonical values here are Linux's. Emscripten's musl uses different
+numbers — `ENOENT` is 44 there and 2 here, and 2 is `EACCES`. Passing a kernel
+errno through an Emscripten adapter unchanged turns "no such file" into
+"permission denied", and the symptom is a `PermissionError` carrying the errno
+of an entirely different failure, several layers from the cause. Every adapter
+translates at its own edge; `src/runtime/python/sbxfs.ts` does it by name,
+against the build's own `ERRNO_CODES`, so it cannot drift from the interpreter
+it is loaded into.

package/docs/python/architecture.md

@@ -0,0 +1,94 @@
+# Python on SandboxedJs — architecture
+
+The goal is an operating-system compatibility layer with an owned Python
+distribution on top of it, not an interpreter swap. Replacing Pyodide with a
+differently-built CPython in the same lifecycle would inherit the same problems,
+because the problems are not in the interpreter.
+
+## What "our own Python" means here
+
+A **runtime distribution**, not a new language implementation. SandboxedJs owns
+the OS-facing ABI, the process model, the filesystem integration, the build
+pipeline and the tests. Upstream CPython supplies Python semantics, adapted by a
+maintained patch series. Building from source gives ownership and
+reproducibility; stability comes from process isolation, consistent semantics
+and systematic testing.
+
+## Governing rules
+
+1. One active Python process owns one interpreter instance.
+2. One kernel authority owns shared mutable container resources.
+3. Blocking callers never own the services they are waiting for.
+4. All guest I/O crosses an explicit, versioned boundary.
+5. Processes share files and channels — not Python globals.
+6. Every resource has an owner and a cleanup path.
+
+## Shape
+
+```
+browser app / shell                  kernel owner (volume, process table)
+        │                                     │
+        ├── preview service worker ───────────┤
+        │                                     ├── VFS + descriptors + storage
+        │                                     ├── process supervisor + signals
+        │                                     └── virtual sockets + net policy
+        │                                             ▲
+   python worker A ──── sbx_host_v1 ────────────────── ┤
+   python worker B ──── sbx_host_v1 ────────────────── ┤
+   node workers ────────────────────────────────────── ┘
+```
+
+Kernel services stay on the host that owns them today; Python runs in dedicated
+workers and calls back. Moving the kernel into its own worker is a later
+optimisation with its own migration plan, driven by measured UI responsiveness,
+and is not a prerequisite for any of this.
+
+## Three compatibility targets
+
+| Target | Behaviour | Mechanism |
+|---|---|---|
+| Python development | scripts, REPL, venv, pip, subprocesses, local servers | CPython to Wasm plus this kernel |
+| Scientific / ML | supported packages, model inference | recompiled extensions, dedicated compute services |
+| Linux binaries | existing Linux executables and wheels | optional CPU emulation running a real Linux guest |
+
+The first is the product. The third is a separate execution backend and must
+never silently substitute for the first when an install fails: it changes
+architecture, path semantics, performance and persistence.
+
+## What exists today
+
+- `python-runtime/abi/` — the versioned ABI and its generated bindings.
+- `src/kernel/open-file.ts`, `src/kernel/descriptors.ts` — pathnames, inodes,
+  open-file descriptions and per-process descriptor tables.
+- `src/runtime/python/` — framing, the blocking transport, the kernel-side
+  dispatcher, the typed guest client, the Emscripten filesystem bridge, the
+  release manifest, and the per-process supervisor and worker.
+- `python-runtime/native/probe/` — a C program that exercises the whole path.
+- `python-runtime/scripts/` — the pinned build: fetch and verify, host Python,
+  cross-built CPython, and the packaged release with its manifest.
+- `test/python-abi/`, `test/python-runtime/` — the baseline inventory and the
+  M1 and M2 gates.
+
+### The owned interpreter
+
+`make -C python-runtime fetch python package` produces
+`out/sbx-cpython-<version>-<profile>/`: `python.js`, `python.wasm`,
+`python.data`, and a `runtime.json` naming the ABI it was built against and the
+capabilities the profile actually has. A host selects it with
+`configurePython({ backend: "sbx-cpython-wasm", manifest })`; Pyodide remains
+the default until the parity gates pass.
+
+Two things about the build are worth knowing before reading it:
+
+- Upstream's browser target links a *page* — a classic script that assigns a
+  global `Module` and runs `main` on load. A process worker needs a factory it
+  instantiates once per process, so the final link is repeated with
+  `-sMODULARIZE -sEXPORT_ES6 -sINVOKE_RUN=0`. Only the link changes.
+- Some of what a program asks an operating system for is compiled into libc
+  rather than routed through anything a host can serve. musl's `getpid` under
+  Emscripten returns a constant, so every process would report the same
+  identity however many interpreters were running. `-Wl,--wrap=` is what makes
+  those reachable, and `python-runtime/native/js/library_sbx_posix.js` is where
+  the first of them live.
+
+See [release-gates.md](release-gates.md) for what each milestone has to prove.

package/docs/python/baseline-inventory.md

@@ -0,0 +1,54 @@
+# M0 — baseline inventory
+
+What the *current* Python integration does, measured rather than remembered, so
+the owned runtime has something to be compared against. Every entry is pinned by
+a test in `test/python-abi/baseline.test.ts`, which asserts today's behaviour —
+including the wrong answers, on purpose. Turning one of those assertions around
+is what a milestone landing looks like.
+
+Setup: `src/runtime/cpython.ts` caches **one Pyodide interpreter per container**
+(keyed on the VFS) and runs each program with a fresh globals dict.
+
+## Findings
+
+| # | Behaviour | Verified | Cause | Fixed by |
+|---|---|---|---|---|
+| 1 | A module left in `sys.modules` by one program is importable by the next, unrelated one | yes | fresh globals do not reset `sys.modules` | **fixed in M2** — one interpreter per process |
+| 2 | `os.environ` does **not** leak between programs | yes | the environment is explicitly rebound per run | — (the exception, not the rule) |
+| 3 | `asyncio.start_server` fails | yes | `python-syscalls.ts` replaces parts of `asyncio` and refuses server creation | M6 — virtual sockets |
+| 4 | Every program reports the same `os.getpid()` | yes | one interpreter is one process | **fixed in M2** — a process per program, with `getpid` wrapped to ask the kernel |
+| 5 | `subprocess.run` succeeds, via the container's command table | yes | a bridge, not a process: no fresh interpreter, no descriptor inheritance, no process groups | M3 |
+
+Fresh globals reset none of: `sys.modules`, logging handlers, registered
+callbacks, native extension state, or running tasks. Finding 2 is worth naming
+precisely because it is the one piece of state the integration *does* rebind —
+which is why "each program gets a clean slate" feels true until it isn't.
+
+## What this means for the plan
+
+- Findings 1 and 4 are integration failures, not interpreter failures. A
+  differently-built CPython dropped into the same lifecycle would reproduce both.
+  That is the argument for building the kernel first.
+- Finding 3 is a missing subsystem. No amount of interpreter work reaches it.
+- Finding 5 is the one that reads as working and is not, which makes it the most
+  expensive to leave in place.
+
+## Not yet measured
+
+Concurrency and server behaviour under load, native extension state across
+process exits, and interpreter memory growth over repeated runs. These need the
+process model of M3 before the measurement means anything.
+
+## Known gaps in the owned runtime (M2)
+
+Recorded so they are not rediscovered as surprises. None of them is hidden at
+runtime: each either refuses with a reason or is declared absent in
+`runtime.json`.
+
+| Gap | Symptom | Milestone |
+|---|---|---|
+| No packaging | `pip` is micropip and targets the Pyodide interpreter; under this backend it refuses rather than installing into the wrong interpreter | M4 |
+| No `rlcompleter` / `_pyrepl` in the stdlib image | the REPL prints `warning: can't use pyrepl` and falls back to the basic prompt, which works | M2 follow-up — a `wasm_assets` inclusion |
+| No sockets | anything binding a port, Uvicorn included, cannot start | M6 |
+| No spawn from Python | `subprocess` has no backend on this runtime | M3 |
+| Symlinks | `symlink()` returns ENOSYS through the kernel filesystem | M3 |