sandboxedjs 0.2.11 → 0.2.12

package/docs/python/extensions.md

@@ -0,0 +1,282 @@
+# Native extensions for the owned CPython runtime
+
+This describes how a compiled CPython extension — a C module, or a Rust one
+built with PyO3 — is built, tagged, installed and loaded by the SandboxedJs
+Python distribution.
+
+It is a general pipeline. Nothing in the runtime, the installer or the import
+machinery knows about any particular package. Adding support for a package
+means adding a *recipe*; it never means adding a runtime adapter.
+
+## The contract
+
+`python-runtime/abi/extension-abi.json` is the single source of truth. The
+wheel tag, the compiler flags side modules are built with, the identifier the
+wheel cache is keyed on and the checks the builder makes are all generated
+from it by `scripts/generate_extension_abi.py` into:
+
+- `python-runtime/abi/sbx_ext_abi.h` — for C extensions
+- `src/runtime/python/extension-abi.ts` — for the installer and loader
+
+Regenerate with `make -C python-runtime extension-abi` and commit the output.
+
+### What the contract fixes
+
+| Field | Value |
+| --- | --- |
+| Implementation | CPython 3.13.5 |
+| ABI tag | `cp313` |
+| SOABI | `cpython-313-wasm32-emscripten` |
+| Extension suffix | `.cpython-313-wasm32-emscripten.so` |
+| Target triple | `wasm32-unknown-emscripten` |
+| Emscripten | 5.0.6 |
+| Threads | pthreads, shared memory |
+| Memory | 256 MiB initial, growth enabled |
+| Stack | 8 MiB |
+| Exceptions | WebAssembly EH (`-fwasm-exceptions`), longjmp lowered to wasm |
+| Dynamic linking | CPython is `MAIN_MODULE=1`; extensions are `SIDE_MODULE=1` |
+| Wheel tag | `cp313-cp313-emscripten_5_0_6_wasm32` |
+
+The wheel tag is the PEP 425 platform tag CPython itself reports on
+Emscripten, so `pip` and `packaging` accept it without a project-specific
+patch. **Linux and macOS wheels are never accepted.** A `manylinux` wheel
+contains ELF objects this interpreter cannot load; installing one would turn a
+clear resolution failure into an `ImportError` inside the user's program.
+
+### Why there is an ABI id as well as a tag
+
+`abiId` (currently `sbxabi1-…`) is a hash of every field that affects binary
+compatibility. It is deliberately finer-grained than the wheel tag: the
+platform tag names only the Emscripten version, but two builds from the same
+Emscripten with different pthread or memory settings are still incompatible.
+
+The tag is what the packaging ecosystem understands. The id is what this
+project caches and verifies on. Change any field that matters and the id
+changes with it, so every previously built wheel stops matching rather than
+being loaded against an interpreter it no longer fits.
+
+The id is a hash rather than a hand-maintained version number because a
+version number is bumped when someone remembers to bump it, which is not the
+same occasion as the ABI actually changing.
+
+## Build profiles
+
+`--enable-wasm-dynamic-linking` is what makes extension loading possible at
+all, and it is a separate profile rather than a flag on the existing one:
+
+| Profile | Threads | Native extensions |
+| --- | --- | --- |
+| `core` | no | linked in, fixed |
+| `threaded-fixed` | yes | linked in, fixed |
+| `dynamic` | yes | **loaded at import** |
+
+`dynamic` is the profile shipped in the npm package. Build it with:
+
+```bash
+make -C python-runtime python PROFILE=dynamic
+make -C python-runtime package PROFILE=dynamic
+```
+
+`dynamic` has its own OpenSSL sysroot (`out/sysroot-dynamic`) because every
+object linked into a `MAIN_MODULE` must be position independent.
+
+## Adding a package
+
+Write a recipe and build it:
+
+```bash
+python3 python-runtime/scripts/build_extension.py path/to/recipe.json
+```
+
+A recipe names sources or a crate, and nothing else:
+
+Pydantic is version-pinned at the native boundary. The repository carries both
+`pydantic-core==2.23.2` for `pydantic==2.9.x` and `pydantic-core==2.46.5` for
+newer Pydantic releases that require it. Keep each exact pair in the wheel
+index; a newer core cannot satisfy an older Pydantic requirement.
+
+```json
+{
+  "name": "sbx-c-probe",
+  "version": "1.0.0",
+  "kind": "c-extension",
+  "modules": [{ "name": "sbx_c_probe", "sources": ["sbx_c_probe.c"] }]
+}
+```
+
+```json
+{
+  "name": "sbx-rust-probe",
+  "version": "1.0.0",
+  "kind": "pyo3",
+  "modules": [{ "name": "sbx_rust_probe", "crate": ".", "crateName": "sbx_rust_probe" }]
+}
+```
+
+**A recipe cannot set compiler or link flags.** They come from the contract.
+A recipe that could set its own `-pthread` could produce an artifact that
+links and then corrupts memory, and the mismatch would be invisible in the
+wheel it produced.
+
+The builder refuses to produce a wheel unless the artifact is a WebAssembly
+binary, declares a `dylink` section, and exports `PyInit_<module>`. All three
+failures are misleading at import time, so they are caught at build time and
+named.
+
+## Rust and PyO3
+
+Two properties of the Rust toolchain decide how this works, and both were
+found as link errors rather than documented limitations.
+
+**Rust's shipped `std` for `wasm32-unknown-emscripten` has no atomics.** It is
+built without the `atomics` and `bulk-memory` features, so it cannot be linked
+into a shared-memory module — and this runtime's main module has pthreads. The
+error names an `.rcgu.o` file:
+
+```
+wasm-ld: error: --shared-memory is disallowed by …rcgu.o because it was not
+compiled with 'atomics' or 'bulk-memory' features.
+```
+
+So `std` is rebuilt from source with those features, which needs a nightly
+toolchain for `-Z build-std`. This is a property of the threading model, not a
+preference: a runtime built without pthreads would link stable Rust's shipped
+`std` unchanged.
+
+**Rust panics unwind using the WebAssembly exception proposal.** A `cargo`
+built extension imports a `__cpp_exception` tag, and a main module linked
+without exception handling cannot supply one:
+
+```
+LinkError: WebAssembly.Instance(): Import #420 "env" "__cpp_exception":
+tag import requires a WebAssembly.Tag
+```
+
+CPython is therefore linked with `-fwasm-exceptions` (and
+`-sSUPPORT_LONGJMP=wasm`, since the two share a mechanism). Enabling it in the
+main module is preferred over forcing `panic = "abort"` on every Rust
+extension, because PyO3 turns a Rust panic into a Python exception by catching
+the unwind — an aborting build would take the process down instead of raising.
+
+PyO3 is cross-compiled through a generated config file rather than allowed to
+probe, because probing describes the *build machine's* interpreter, which is
+how a cross build silently produces a host artifact.
+`suppress_build_script_link_lines` matters most: without it PyO3 emits
+`-lpython3.13`, and there is no such library here — the CPython symbols come
+from the main module at load time, which is what a side module's undefined
+symbols are for.
+
+## Installation
+
+The installer resolves before it downloads, and downloads before it writes:
+
+1. Already-installed distributions are skipped.
+2. Dependencies are resolved with constraint propagation and conflict
+   learning (`src/runtime/python/resolver.ts`).
+3. A compatible pure-Python wheel is preferred.
+4. Then a wheel tagged for this ABI.
+5. Then, if enabled, a source build.
+6. Every download is verified against the digest the index published. An
+   artifact published without one is refused rather than installed unverified.
+7. The whole solved graph is staged and committed as one transaction.
+
+Resolution reads dependencies from **PEP 658 metadata sidecars**, a few
+kilobytes each, rather than downloading whole wheels to read `METADATA`. That
+is what removed the previous installer's arbitrary 96-candidate cap: the cap
+existed because discovering that forty releases all needed the same
+unavailable extension cost forty multi-megabyte downloads. What bounds the
+search now is a deadline and a cancellation signal, which are honest limits —
+they say the search ran out of time, not out of an arbitrary allowance.
+
+## Security model
+
+- Wheels are fetched only when the container's `allowOutbound` policy permits
+  it. A container that refuses `curl` cannot install packages.
+- Every artifact is verified by SHA-256 against the index's published digest
+  before any of it is written.
+- Extension code runs inside the same WebAssembly sandbox as the interpreter.
+  It has no more access to the host than Python does: the filesystem it sees
+  is the container's, served over the host ABI.
+- The build toolchain is not shipped to consumers. The npm package contains
+  the runtime and the loader.
+
+## "The ASGI callable works" is not "Uvicorn serves"
+
+These are different claims, and only the first is proven.
+
+**Proven.** FastAPI imports, Pydantic 2 validates a request body through the
+compiled `pydantic_core` extension, and the application is driven through the
+public ASGI protocol — an ordinary `await app(scope, receive, send)` — with the
+response status and body asserted. Installing `fastapi[standard]` also installs
+the FastAPI CLI, whose `fastapi dev main.py` and `fastapi run main.py` commands
+are available. A synchronous endpoint is exercised, so Starlette's threadpool
+path runs on real worker threads.
+
+**Not proven.** `uvicorn main:app` binding a port that the container's
+networking can route a request to. This is not merely untested; the layer is
+missing, and the way it is missing is worth stating precisely because it looks
+like success:
+
+```python
+s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+s.bind(("127.0.0.1", 8123)); s.listen(1)   # succeeds
+```
+
+`bind` and `listen` return cleanly, and `asyncio`'s event loop has
+`create_server`. But that socket belongs to **Emscripten's own socket
+emulation**, which is not connected to `VirtualHttpRouter` — the router that
+already routes `curl localhost:3000` to a Node server in the same container. So
+a server would appear to start, report itself listening, and never receive a
+request. A test that asserted "uvicorn started" would pass while proving
+nothing.
+
+Closing this means bridging CPython's socket layer to the container network the
+way `sbxfs.ts` bridges its filesystem: `listen` registers a virtual port with
+the router, and an accepted connection becomes a socket whose reads and writes
+cross the host ABI. It belongs with the socket layer, not with Uvicorn —
+patching Uvicorn would make one server work and leave every other one broken.
+
+Note also that `socket.SO_REUSEADDR` is absent from this build, which some
+servers set unconditionally.
+
+## A resolution behaviour worth knowing
+
+`pip install pydantic` **succeeds** without a wheel index, by backtracking to
+the 1.x line — which is pure Python and needs no compiled core. That is correct
+of the resolver and is what pip does too, but it is unlikely to be what someone
+asking for Pydantic today wants. Constrain the requirement (`pydantic>=2`) when
+the 2.x line is what you mean; with the bound, an unavailable native dependency
+is reported rather than worked around.
+
+## Current limitations
+
+- Building extensions requires Emscripten 5.0.6, a nightly Rust with
+  `rust-src`, and the CPython build tree. Consumers install prebuilt wheels.
+- Source distributions are resolved but not built in-container; a resolution
+  that lands on one reports that no builder is configured, rather than
+  claiming an install. The optional local and remote builders described in the
+  architecture are not implemented.
+- **Uvicorn cannot serve.** See the section above; the ASGI callable works.
+- Emscripten documents dynamic linking with pthreads as experimental.
+- The default package wheel index carries the ABI-matched Pydantic Core wheels
+  beside the interpreter. Consumers can override it with
+  `configurePython({ wheelIndex })` to add privately built extensions.
+- `socket.SO_REUSEADDR` is absent from this build.
+
+## Which tests prove what
+
+| Claim | Test |
+| --- | --- |
+| A C extension imports and runs | `test/python-runtime/extensions.test.ts` |
+| A PyO3 extension imports and runs | `test/python-runtime/extensions.test.ts` |
+| The ABI id is derived, not hand-written | `test/python-runtime/extension-abi.test.ts` |
+| Linux wheels are refused | `test/python-runtime/extension-abi.test.ts` |
+| Resolution has no candidate cap | `test/python-runtime/resolver.test.ts` |
+| Constraints propagate, conflicts explain | `test/python-runtime/resolver.test.ts` |
+| A failed native install leaves nothing | `test/python-runtime/extensions.test.ts` |
+| Real pydantic-core validates data | `test/python-runtime/fastapi.test.ts` |
+| Pydantic 2 + FastAPI + ASGI + console script | `test/python-runtime/fastapi.test.ts` |
+
+The FastAPI test needs `SANDBOXEDJS_CLEAN_NETWORK_TESTS=1`; the extension
+tests need `make -C python-runtime extensions` to have been run, and skip with
+a reason otherwise.

package/docs/python/release-gates.md

@@ -0,0 +1,46 @@
+# Release gates
+
+A milestone is done when its gate passes, not when its code exists. Numbers here
+are proposed targets, not measured results.
+
+| Milestone | Deliverable | Gate | State |
+|---|---|---|---|
+| M0 | behaviour inventory | current failures reproducible and written down, integration failures told apart from interpreter failures | **done** — [baseline-inventory.md](baseline-inventory.md) |
+| M1 | host ABI + C probe | interactive read, exact binary I/O, cancellation, no deadlock | **done** — `test/python-abi/` |
+| M2 | source-built interpreter | scripts, CLI flags, two isolated programs concurrently | **done** — `test/python-runtime/`, and `pythoncheck.html` in the harness |
+| M3 | files and processes | cross-runtime pipelines, correct EOF, cleanup, file consistency | descriptor layer and Emscripten FS bridge done; spawn, pipes into Python, and symlinks not started |
+| M4 | environments | two independent venvs, offline and online wheel installs | **partial** — online wheels install transactionally into the owned interpreter and expose `console_scripts`; the package includes its ABI wheel index, but venvs, offline indexes, source builds, and per-environment site-packages remain |
+| M5 | native packages | Pydantic core and one small extension survive repeated import/exit | **done for the current ABI** — C and PyO3 side-module wheels load through ordinary CPython imports; `pydantic==2.9.0` selects and runs with `pydantic-core==2.23.2` |
+| M6 | local networking | echo server, concurrent connections, timeouts, port reuse | **partial** — the host-owned virtual TCP stack (`src/net/virtual-socket.ts`), its socket ABI family (`1792`), and router integration are done and cover `box.request()` to a non-JS listener plus cross-stack `EADDRINUSE` (`test/virtual-socket.test.ts`, `test/python-socket-abi.test.ts`, `test/universal-server.test.ts`); the CPython `_sbx_socket` adapter and `socket`/`selectors` patches are written but not yet built into the interpreter, so no Python process has bound a port through it |
+| M7 | web serving | FastAPI validation, streaming, WebSocket, shutdown | **partial** — a real FastAPI/Pydantic 2 application validates and answers through ASGI; Uvicorn cannot yet serve through `box.request()` |
+| M8 | threads | sync endpoints, AnyIO operations, thread cleanup | **partial** — pthread CPython, `asyncio.to_thread()`, and a synchronous FastAPI endpoint pass; churn/cleanup soak is outstanding |
+| M9 | persistence | recovery after an interrupted write or install | not started |
+| M10 | ML | correct inference, bounded memory, cancellation | not started |
+| M11 | source builds | compile, install and import a representative C extension in-browser | not started |
+| M12 | release | browser matrix, soak tests, rollback exercise | not started |
+
+M6 still gates any broad FastAPI hosting claim. Importing the framework and
+calling its public ASGI interface is useful evidence, but a publishable server
+runtime must also route a real Uvicorn listener through `box.request()`. M5
+still gates current Pydantic 2 releases because their Rust extension is native.
+
+## Standing gates
+
+Checked at every release, not once:
+
+| Area | Gate |
+|---|---|
+| Process isolation | no module, global or environment leakage across independent processes |
+| Process churn | 10,000 short launches with no growth in live resource counts |
+| I/O | binary correctness, partial reads and writes, EOF, backpressure, Unicode terminal boundaries |
+| Cancellation | kill succeeds during startup, blocked input, network wait, import and inference |
+| Filesystem | rename, unlink and open semantics match the declared contract under concurrent access |
+| Packaging | an interrupted install never reports a complete environment |
+| Threads | repeated creation, blocking, imports and teardown do not deadlock |
+| Storage | reload after an interrupted commit recovers a valid state |
+| Node regression | every supported Node behaviour still passes |
+
+Fault injection belongs in the same suite: worker crash mid-syscall, truncated
+runtime assets, missing extension symbols, quota exhaustion, lost connections,
+GPU device loss, cancellation during extraction, stale responses after PID
+reuse, and teardown while children are blocked.

package/docs/python/virtual-sockets-plan.md

@@ -0,0 +1,331 @@
+# Virtual sockets for Python servers
+
+This is the implementation plan for making ordinary Python servers such as Uvicorn, FastAPI CLI, asyncio servers, and raw TCP servers reachable through the SandboxedJs container network.
+
+## Status (2026-09-08)
+
+- **Phase 0 — contract**: done. Socket ops occupy family `1792` in
+  [host-v1.json](../../python-runtime/abi/host-v1.json) /
+  [sbx_host.h](../../python-runtime/abi/sbx_host.h) /
+  [host-abi.ts](../../src/runtime/python/host-abi.ts); `AF_INET` + `SOCK_STREAM`
+  only; golden-frame coverage in [python-socket-abi.test.ts](../../test/python-socket-abi.test.ts).
+- **Phase 1 — host virtual stream model**: done.
+  [src/net/virtual-socket.ts](../../src/net/virtual-socket.ts) holds the listener
+  backlog, duplex byte queues, EOF/half-close/reset transitions, and a shared
+  `VirtualTcpNetwork` port authority. Tests: [virtual-socket.test.ts](../../test/virtual-socket.test.ts).
+- **Phase 2 — ABI dispatch and descriptors**: done. `SocketDescription`
+  implements `OpenFileDescription`, so `close`, `set_flags`, and `poll` reuse the
+  generic descriptor handlers; socket ops dispatch in
+  [syscall-server.ts](../../src/runtime/python/syscall-server.ts) with blocking
+  `accept`/`recv`/`send` that unpark on `whenReady()` and abort on process kill.
+- **Phase 3 — CPython socket adapter**: written, not yet built.
+  [native/sbx/sbx_socket.c](../../python-runtime/native/sbx/sbx_socket.c), the
+  `socket.py` / `selectors.py` patches
+  ([0003](../../python-runtime/patches/cpython/0003-sandboxedjs-host-sockets.patch),
+  [0004](../../python-runtime/patches/cpython/0004-sandboxedjs-selector-poll.patch)),
+  and the `build_python.py` wiring exist, but the interpreter has not been
+  recompiled with `_sbx_socket`, so no Python process has bound a port.
+- **Phase 5 — router/client integration**: done for the non-JS path.
+  `LocalRuntimePod.request()` falls through to `requestTcp()` for ports held by
+  the socket stack, `proxy.activePorts()` merges both tables, and
+  `VirtualHttpRouter.register()` rejects a port already held by a socket
+  listener. Tests: [universal-server.test.ts](../../test/universal-server.test.ts).
+- **Phases 4 and 6 — asyncio/Uvicorn, hardening**: not started; blocked on
+  Phase 3 build.
+
+Next step: rebuild the dynamic CPython profile
+(`python-runtime/scripts/build_python.py`) and add the raw-TCP echo and
+`asyncio.start_server` acceptance tests from the list below.
+
+## Goal
+
+This must work without changing FastAPI, Starlette, Uvicorn, or application code:
+
+```sh
+pip install "fastapi[standard]"
+fastapi run main.py
+```
+
+Then the host API must reach it through the same port table used by Node:
+
+```ts
+await box.waitForPort(8000)
+const response = await box.request(8000, { path: "/docs" })
+```
+
+The Python process remains CPython compiled to WebAssembly. FastAPI and Uvicorn remain ordinary Python packages. Only the socket implementation and its host bridge are added.
+
+## Current topology
+
+```mermaid
+flowchart LR
+  A[Python Uvicorn] --> B[CPython socket API]
+  B --> C[Emscripten socket emulation]
+  C -. not connected .-> D[VirtualHttpRouter]
+  N[Node http.createServer] --> D
+  D --> E[box.request / curl / preview]
+```
+
+Node works because its HTTP implementation calls `VirtualHttpRouter.register()` directly. Python currently calls Emscripten sockets, which can appear to bind but cannot deliver connections to the SandboxedJs router.
+
+## Target topology
+
+```mermaid
+flowchart LR
+  A[Python Uvicorn] --> B[CPython socket API]
+  B --> C[Emscripten/libc socket adapter]
+  C --> D[sbx_host socket operations]
+  D --> E[syscall-server.ts]
+  E --> F[NetworkStack / VirtualHttpRouter]
+  F --> G[box.request / curl / preview]
+  F --> H[accepted virtual connection]
+  H --> D
+  D --> C
+  C --> B
+  B --> A
+```
+
+The host owns all shared socket state. The WASM process owns only its descriptor numbers and Python-visible socket objects.
+
+## Design decisions
+
+### 1. Implement virtual sockets, not a FastAPI adapter
+
+A FastAPI-specific bridge would make one framework work while leaving every other Python server broken. The correct abstraction is POSIX-like sockets behind the existing host ABI. Uvicorn, asyncio, Flask adapters, Django, WebSockets, and raw TCP programs then use the same path.
+
+### 2. Use the existing descriptor table
+
+Sockets must be normal process handles in `ProcessDescriptorTable`, with `get_flags`, `set_flags`, `dup`, `dup2`, and `poll` behavior. Do not create a second socket-only handle registry that bypasses descriptor lifecycle.
+
+### 3. Keep HTTP parsing in the Python server
+
+The router should transport bytes/connections, not parse HTTP for Python. Uvicorn must receive a real accepted stream and remain responsible for HTTP/1.1, keep-alive, chunking, and WebSocket upgrades. `box.request()` may provide a convenience HTTP client, but it must ultimately feed the same connection abstraction.
+
+### 4. Separate listener registration from connection delivery
+
+`bind()` and `listen()` register a listener with the container port authority. `accept()` consumes pending connections. A listener is not considered complete merely because `bind()` returned successfully.
+
+### 5. Preserve the blocking contract
+
+A Python `accept()`, `recv()`, or `send()` may wait. The Python worker may block on the synchronous transport while the host event loop continues routing requests and resolving readiness. This follows the existing ABI rule documented in [abi.md](abi.md).
+
+## Existing code to wire
+
+### ABI and generated bindings
+
+- [host-v1.json](../../python-runtime/abi/host-v1.json): add socket operation codes in the reserved family `1792`.
+- [generate_abi.py](../../python-runtime/scripts/generate_abi.py): ensure generated C/TypeScript bindings include the new operations and payload definitions if the generator has schema-specific output.
+- [sbx_host.h](../../python-runtime/abi/sbx_host.h): generated C operation constants.
+- [host-abi.ts](../../src/runtime/python/host-abi.ts): generated `Op` constants and capability list.
+- [protocol.ts](../../src/runtime/python/protocol.ts): reuse existing framing; only socket payload readers/writers are needed.
+
+Initial operations:
+
+| Operation | Purpose |
+|---|---|
+| `socket` | Allocate a stream socket handle with family/type/protocol. |
+| `bind` | Bind a socket to an address and port. |
+| `listen` | Mark a bound socket as a listener with a backlog. |
+| `accept` | Return an accepted connection handle and peer address. |
+| `connect` | Connect to an in-container listener or permitted outbound target. |
+| `send` | Send bytes, allowing short writes. |
+| `recv` | Receive bytes, allowing short reads and EOF. |
+| `shutdown` | Close one or both directions. |
+| `getsockname` | Return local address and port. |
+| `getpeername` | Return remote address and port. |
+| `setsockopt` | At minimum support `SO_REUSEADDR`, `SO_KEEPALIVE`, and TCP options Uvicorn touches. |
+| `getsockopt` | Return values for options that libraries inspect. |
+
+Do not add UDP, IPv6, DNS, or arbitrary outbound TCP in the first slice unless a failing acceptance test requires them. Uvicorn HTTP and WebSocket serving need TCP listener/accepted-stream semantics first.
+
+### Host client/server
+
+- [syscall-client.ts](../../src/runtime/python/syscall-client.ts): add typed methods for socket operations.
+- [syscall-server.ts](../../src/runtime/python/syscall-server.ts): dispatch operations, validate process generation, and map errors to canonical errno.
+- [descriptors.ts](../../src/kernel/descriptors.ts): confirm duplicated socket descriptions share state and close behavior.
+- [open-file.ts](../../src/kernel/open-file.ts): add a socket description abstraction or a common readiness/close interface if sockets cannot use the current file description directly.
+- [python/worker-entry.ts](../../src/runtime/python/worker-entry.ts): no Python-specific routing should be added here; it already owns the CPython worker and host transport lifecycle.
+
+### Network authority
+
+- [stack.ts](../../src/net/stack.ts): make listener registration and connection creation use the same authority as Node server ports.
+- [virtual-http.ts](../../src/runtime/virtual-http.ts): extract or reuse a byte-stream connection primitive for request/response and upgrades. Do not force Python into `VirtualHttpServer`; Python needs accepted sockets, not Node EventEmitter semantics.
+- [contracts.ts](../../src/runtime/contracts.ts): extend `RuntimePod`/network contracts only where the host needs a bidirectional accepted connection. Existing `serveExternal` is a possible compatibility seam but is not sufficient for general sockets.
+- [worker-runtime-pod.ts](../../src/runtime/worker-runtime-pod.ts): preserve the existing Node worker proxy and add Python socket events/connection IDs if the Python worker is hosted in a worker that cannot call the host directly.
+- [local-runtime-pod.ts](../../src/runtime/local-runtime-pod.ts): implement the same socket behavior for realm mode or explicitly report that Python server tests require the worker pod.
+
+### CPython/Emscripten side
+
+- [extension-abi.json](../../python-runtime/abi/extension-abi.json): add any required main-module exports and document the socket adapter contract.
+- [library_sbx_posix.js](../../python-runtime/native/js/library_sbx_posix.js): add only libc calls that cannot be redirected through the normal Emscripten syscall path. Do not hide socket behavior in JavaScript globals.
+- [0002-emscripten-pipe-socketpair.patch](../../python-runtime/patches/cpython/0002-emscripten-pipe-socketpair.patch): keep the existing socketpair workaround for local pipes; do not confuse it with network sockets.
+- Add a CPython patch or Emscripten syscall adapter for `socket`, `bind`, `listen`, `accept`, `connect`, `send`, `recv`, `poll`, and socket options. Prefer one libc-level adapter so Python's `socket`, `selectors`, and `asyncio` all see the same semantics.
+- [python-syscalls.ts](../../src/runtime/python-syscalls.ts): the old Pyodide backend's `create_server` refusal remains separate. Do not make the new CPython backend depend on Pyodide's asyncio patches.
+
+## Connection protocol
+
+Use a host-owned `SocketDescription` with these states:
+
+```text
+created -> bound -> listening -> closed
+created -> connected -> open -> half-closed/closed
+```
+
+Each connection needs:
+
+- owning process generation;
+- local and peer address/port;
+- receive queue and byte count;
+- send backpressure state;
+- readable/writable/closed waiters;
+- listener backlog queue;
+- cancellation/teardown behavior;
+- reference count for duplicated descriptors.
+
+Suggested host payloads are fixed-width fields plus length-prefixed byte arrays, matching the existing `Writer`/`Reader` framing. Addresses should initially be normalized to IPv4 strings and `u16` ports. Return the accepted descriptor and peer address from `accept`.
+
+### Incoming request path
+
+1. `box.request(8000, init)` asks the router for port 8000.
+2. The router finds the Python listener registered by `listen()`.
+3. The host creates a virtual TCP connection and queues it on the listener.
+4. A blocked Python `accept()` becomes readable through `poll`.
+5. Python/Uvicorn accepts the descriptor and reads the serialized HTTP request bytes.
+6. Uvicorn writes HTTP response bytes to the accepted descriptor.
+7. The router resolves `box.request()` from the response stream.
+8. Connection close or keep-alive is handled by the same stream state, not by a Python-specific shortcut.
+
+### Outbound local path
+
+For `curl localhost:port` or Python `http.client` to a Node/Python listener:
+
+1. `connect()` resolves the address through the container network authority.
+2. The host locates the listener in the shared port table.
+3. It creates a pair of virtual stream endpoints.
+4. The client descriptor and server listener's accepted descriptor reference opposite endpoints.
+5. `send`/`recv` and `poll` operate on those queues.
+
+### External outbound path
+
+Do not silently map Python TCP to host TCP. Start with the existing HTTP/fetch service for outbound HTTP. Add arbitrary outbound TCP only after capability and security policy are defined. `allowOutbound`, host allowlists, cancellation, and DNS behavior must apply consistently.
+
+## Implementation phases
+
+### Phase 0: freeze the contract
+
+- Define socket operation numbers and payload schemas.
+- Define descriptor ownership, backlog, half-close, timeout, and cancellation semantics.
+- Define supported families: initially `AF_INET` + `SOCK_STREAM`.
+- Define unsupported behavior explicitly: UDP, IPv6, raw sockets, arbitrary outbound TCP.
+- Add ABI version/golden-frame tests before implementation.
+
+### Phase 1: host-side virtual stream model
+
+- Add `SocketDescription` and listener/backlog state.
+- Connect it to the existing port authority and router.
+- Implement in-memory duplex queues with `whenReady()` and EOF/error transitions.
+- Ensure teardown closes listeners and all accepted connections.
+- Add tests for bind conflicts, backlog, short reads/writes, EOF, half-close, reset, and port reuse.
+
+### Phase 2: ABI dispatch and descriptor integration
+
+- Generate operation constants.
+- Implement client methods and server dispatch.
+- Add socket handles to descriptor tables.
+- Implement `poll` readiness for listener-readable, connection-readable, connection-writable, hangup, and error.
+- Test stale process generations and killed workers with blocked socket operations.
+
+### Phase 3: CPython socket adapter
+
+- Patch/build the dynamic CPython profile with the host socket adapter.
+- Make `socket.socket(AF_INET, SOCK_STREAM)` allocate host handles.
+- Implement Python-visible `fileno`, blocking/nonblocking mode, timeout, `accept`, `makefile`, `sendall`, `recv`, `shutdown`, `getsockname`, and `getpeername`.
+- Implement enough socket constants/options for Uvicorn and asyncio, including `SO_REUSEADDR`.
+- Verify `selectors.DefaultSelector` observes the host readiness fd model.
+
+### Phase 4: asyncio and Uvicorn
+
+- Run an ordinary `asyncio.start_server` echo server.
+- Run `uvicorn main:app` and `fastapi run main.py` without adapters.
+- Verify startup, shutdown, keep-alive, request body, response body, and malformed requests.
+- Confirm the process remains alive while the listener is referenced and exits after close.
+
+### Phase 5: router and client integration
+
+- Make `box.waitForPort()` recognize Python listeners.
+- Make `box.request()` send a real HTTP byte stream to the accepted Python socket.
+- Make `curl localhost:port` use the same local virtual connection path.
+- Add `box.expose()` support only after in-container request routing works.
+- Add WebSocket upgrade tests using the same accepted stream abstraction.
+
+### Phase 6: hardening and release
+
+- Stress repeated server start/stop and connection churn.
+- Test concurrent requests and backpressure.
+- Test cancellation while blocked in `accept`, `recv`, `send`, and `poll`.
+- Test browser worker and Node worker modes.
+- Test cross-origin isolation requirements for browser hosts.
+- Update M6/M7 release gates only after real `box.request()` and Uvicorn tests pass.
+
+## Acceptance tests
+
+The first end-to-end test should be a small Python TCP echo server. The second should be HTTP over Uvicorn:
+
+```ts
+const box = await createContainer({
+  workerUrl: WORKER_URL,
+  files: {
+    "/workspace/main.py": `
+from fastapi import FastAPI
+
+app = FastAPI()
+
+@app.get("/health")
+def health():
+    return {"ok": True}
+`,
+  },
+});
+
+await box.exec('pip install "fastapi[standard]"', { timeoutMs: 300_000 });
+const process = box.spawn("fastapi run /workspace/main.py --port 8000");
+try {
+  expect(await box.waitForPort(8000, { timeoutMs: 60_000 })).toBe(true);
+  const response = await box.request(8000, { method: "GET", path: "/health" });
+  expect(response.status).toBe(200);
+  expect(response.json()).toEqual({ ok: true });
+} finally {
+  process.kill();
+  box.dispose();
+}
+```
+
+Required test groups:
+
+- ABI encoding/decoding and capability handshake.
+- Descriptor lifecycle and duplicated descriptors.
+- Listener registration and port conflicts.
+- Readiness and blocking behavior.
+- Raw TCP echo.
+- Node-to-Python and Python-to-Node local connections.
+- Uvicorn/FastAPI GET and POST validation.
+- Keep-alive and concurrent requests.
+- WebSocket upgrade.
+- Cancellation and teardown.
+- Browser worker and Node worker.
+
+## Definition of done
+
+The feature is complete only when all of these are true:
+
+- `fastapi run main.py` is found after installing `fastapi[standard]`.
+- The command binds a port through the SandboxedJs socket ABI, not a fake success path.
+- `ss -ltn` reports the Python listener with the right owner/PID metadata.
+- `box.waitForPort()` returns true because a real request can reach the listener.
+- `box.request()` receives a real Uvicorn response.
+- `curl localhost:8000` reaches the same Python server.
+- Node and Python servers share port conflict and lifecycle behavior.
+- Killing the Python process unblocks and closes all accepted connections.
+- No host socket or listener remains after container disposal.
+- Browser and Node worker modes pass the same core tests.
+- The old Pyodide backend remains unchanged and does not accidentally claim this capability.