sandboxedjs 0.2.10 → 0.2.12

package/docs/python/cross-build.md

@@ -0,0 +1,354 @@
+# Cross-building native extensions
+
+The pipeline that turns a package's own source into a SandboxedJs wheel, and
+the list of things it still assumes about a package — so each of them can be
+removed deliberately rather than discovered by a failure.
+
+## What it does
+
+`python-runtime/scripts/build_extension.py` takes a recipe and produces a
+wheel tagged `cp313-cp313-emscripten_5_0_6_wasm32` with a `Build-ABI` line
+naming the `abiId` computed from `abi/extension-abi.json`. Three recipe kinds
+exist:
+
+| kind | source of truth for *what* to build | covers |
+| --- | --- | --- |
+| `setuptools` | the package's own `setup.py` / `pyproject.toml` | plain C extensions, Cython, anything setuptools builds |
+| `pep517` | the package's declared build backend and hooks | pyproject-only projects, in-tree backends |
+| `meson` | the project's `meson.build`, via meson-python | Meson projects, including NumPy |
+| `pyo3` | the crate's `Cargo.toml` | Rust extensions |
+| `c-extension` | the recipe's own `sources` list | probes only — see the assumptions below |
+
+`kind` selects a backend explicitly; nothing is inferred from which files are
+present. A project with both a `setup.py` and a `pyproject.toml` can be built
+either way, and guessing would silently change how a package is built the day
+upstream adds a file. Recipes are schema-validated before anything is fetched
+or compiled, so a malformed one costs a second rather than a Rust build.
+
+In every case the *how* — compiler, target, PIC, threads, exceptions,
+extension suffix, wheel tag — comes from `abi/extension-abi.json` and only
+from there. A recipe cannot contribute a compile or link flag. This is not
+tidiness: a recipe that could set its own `-pthread` could produce an artifact
+that links against a main module with a different memory model and then
+corrupts memory, and the mismatch would be invisible in the wheel.
+
+## How the setuptools path works
+
+`sysconfig` loads its table of build variables from a module named by the
+`_PYTHON_SYSCONFIGDATA_NAME` environment variable, and
+`distutils.command.build_ext` — which setuptools vendors — takes the compiler,
+its flags and the extension suffix from that table. So the package's own
+`setup.py build` runs on the *build machine's* interpreter with `sysconfig`
+answering for the *target*. This is the mechanism CPython uses for its own
+cross builds and the one `crossenv` automates; nothing about it is specific to
+this project.
+
+The CPython cross build already writes the target's table
+(`_sysconfigdata__emscripten_wasm32-emscripten.py`). `scripts/crossenv.py`
+copies it and overrides exactly three groups of values, all from the ABI
+contract:
+
+* `INCLUDEPY` / `CONFINCLUDEPY` — the shipped table names
+  `$prefix/include/python3.13`, where the headers would be if the interpreter
+  had been installed. It has not been; they are in the source tree.
+* `LDSHARED` / `BLDSHARED` / `LDCXXSHARED` — the shipped link line has no
+  `-sSIDE_MODULE=1`, because CPython adds it in a rule outside `LDSHARED`.
+  Without it the artifact is a complete program and `dlopen` cannot load it.
+* `CCSHARED` — the ABI's `-pthread -fwasm-exceptions -fPIC`, which must match
+  the main module exactly. `CCSHARED` rather than `CFLAGS` so the flags apply
+  to extension objects and not to host tools a package builds during setup.
+
+Cython needs no support at all: `cythonize()` runs inside the package's
+`setup.py` on the build machine and hands generated C to the same `build_ext`.
+The only difference between the plain and Cython fixtures is a pinned
+`buildRequires`.
+
+## Validation
+
+Three checks, in order of how misleading their failure would otherwise be:
+
+1. `verify_toolchain` refuses an Emscripten other than the pinned one. A side
+   module built by a different Emscripten links and then fails at run time.
+2. `verify_side_module` checks the artifact is a WebAssembly binary, has a
+   `dylink` section, and exports `PyInit_<last component of the module name>`.
+   A missing dylink section fails at `dlopen` with a message about the file; a
+   missing init symbol fails with "dynamic module does not define module
+   export function", which reads like a source error and is in fact a link
+   setting.
+3. `test/python-runtime/extensions.test.ts` installs each wheel from an HTTP
+   index with the container's own `pip` and imports it with CPython's ordinary
+   import machinery. This is the only check that proves the ABI, because the
+   first two can pass on a module compiled against the wrong headers.
+
+## Package-specific assumptions, and how to remove each
+
+Each of these is a place the pipeline knows, or requires a recipe to state,
+something a package already states about itself.
+
+**1. Target dependencies are not resolved.** *Resolved.* Native target
+dependencies are declared by `dependency.json` manifests under
+`python-runtime/native-deps/` (real libraries) and
+`python-runtime/fixtures/native/` (test ones), built by
+`scripts/native_deps.py` into the profile's sysroot with the ABI's own flags,
+and reached by extensions through the generated cross configuration — so
+`libraries=["yaml"]` in a package's `setup.py` resolves with no path or flag
+in the recipe. A recipe names what it needs in `nativeRequires`, which is a
+separate namespace from `requires` (Python) and `buildRequires` (build
+machine) so that a build-machine library can never satisfy a target link.
+
+Reuse is decided by a stamp recording the source digest, profile, exact
+compiler flags and the digest of every installed output — not by whether a
+file exists, which is what let an archive built for one profile satisfy
+another. What remains: only `make` and `configure` build systems are
+supported, and CMake and Meson libraries will need a third; and the libraries
+CPython itself links are still built by the older hand-written
+`scripts/build_dependencies.py` rather than through manifests.
+
+**2. Wheel metadata is synthesized from the recipe on the `pyo3` path only.**
+*Resolved for `setuptools`.* That path now runs the package's own `dist_info`
+under the cross environment and ships the `METADATA` it generates, so markers,
+extras and `Requires-Python` are upstream's and a recipe has no `requires`
+field to drift from them. Each wheel records which of the two it used in its
+`WHEEL` file as `Metadata-Source: package|recipe`, so the remaining cases are
+visible in the artifacts rather than only here.
+
+A recipe may still *remove* a requirement through `dependencyOverrides`, which
+must carry a `because`, must name a requirement the package actually has, and
+is recorded in the built wheel as `Build-Dropped-Requirement`. *Remove the
+remaining `recipe` cases by:* reading `[project]` from the crate's
+`pyproject.toml` for maturin-built extensions, or by driving maturin itself
+under the cross environment.
+
+**3. `pythonRoots` exists only because the `pyo3` path has no `build_py`.** It
+is a hand-written statement of which directory holds the package's Python
+half; the pydantic-core recipe encodes maturin's `python/` convention. *Remove
+by:* reading `[tool.maturin] python-source` from the crate's `pyproject.toml`,
+or by driving maturin itself under the cross environment.
+
+**4. The `pyo3` path declares module layout.** `crateName` and `package` state
+where the artifact must sit for `from ._x import ...` to work. The setuptools
+path derives the same thing from the build tree. *Remove by:* same as 3.
+
+**5. The `pyo3` recipe hard-codes an unpacked source path.**
+`recipes/pydantic-core/recipe.json` names
+`../../out/ports/pydantic_core-2.46.5`, duplicating the version that appears
+three other times in the same file. *Remove by:* deriving the crate path from
+`source.unpackTo` and the archive's own stem, which `fetch_source` already
+computes.
+
+**6. `kind: "c-extension"` is a transcription.** The recipe lists sources the
+package's build system already lists, so the two can disagree silently. It is
+retained only for `fixtures/sbx_c_probe`, which deliberately has no build
+system. *Remove by:* giving the fixture a `setup.py` and deleting the kind and
+`compile_c_module` with it.
+
+**7. `setup.py` is assumed to be cross-safe.** *Mitigated, not removable.* A
+`setup.py` that probes the build machine (compiles a test program, runs the
+extension it is building, reads `platform.machine()`) will describe the build
+machine. Nothing detects this; the build succeeds and produces a wrong
+artifact. It is a property of the package, so it cannot be fixed generically —
+but it now has one designated place to be fixed in. A recipe declares
+`patches: {"dir": …}`, and the patch set states the package, the version, the
+digest of the source tree it was written against, and per patch a reason, the
+target fact being substituted, and the test that proves it. Patches apply to a
+staged copy with `git apply` at exact context; a drifted source is refused
+rather than fuzzily patched. `fixtures/sbx_patched_probe` is exactly this
+failure — `platform.machine()` declaring 64-bit pointers for a 32-bit target —
+and its patch is the worked example.
+
+**8. Build tools come from PyPI at first build.** *Resolved.*
+`build-tools.lock` pins every build tool — including `setuptools` itself, which
+was previously taken from whatever the build machine had — and records the
+sha256 of every distribution published for that version, so one lock serves any
+build machine. Installation uses `--require-hashes`, which also refuses a tool
+that grows an unpinned dependency. Regenerate with
+`scripts/lock_build_tools.py`; a build reads the lock and never refreshes it.
+
+**9. One ABI, no matrix.** `abiId` covers exactly one configuration; there is
+no way to build the same recipe for a second profile. *Remove by:* taking the
+profile from the command line into the wheel's local version segment, once a
+second profile exists to want it.
+
+**10. `--build-lib` is assumed to be honoured.** Projects with a custom
+`build` command that ignores it would stage nowhere the collector looks; the
+build "succeeds" and produces no modules. This is caught — the empty-module
+case raises — but only after the fact.
+
+## Native target dependencies
+
+Three kinds of dependency exist and are deliberately kept in three namespaces,
+because a single list would let a build machine's library satisfy a target
+link — producing a module that links cleanly and then faults:
+
+| recipe field | what it is | where it goes |
+| --- | --- | --- |
+| `requires` | Python runtime dependency | installed in the container, imported |
+| `buildRequires` | build-machine tool (Cython, setuptools_scm) | a per-recipe venv on this machine |
+| `nativeRequires` | C library cross compiled for wasm32 | `out/sysroot-<profile>`, linked into the side module |
+
+A native dependency is declared by a `dependency.json`:
+
+```json
+{
+  "name": "yaml", "version": "0.2.5",
+  "license": "MIT", "licenseFiles": ["License"],
+  "provenance": { "url": "…", "sha256": "…", "unpackDir": "yaml-0.2.5" },
+  "profiles": ["dynamic"], "dependsOn": [],
+  "build": { "system": "configure", "configureArgs": ["--disable-shared"] },
+  "outputs": { "libraries": ["libyaml.a"], "headers": ["yaml.h"] }
+}
+```
+
+`name` must be the name a linker is given (`-lyaml`), not the project's title:
+a manifest called `libyaml` would build the right library and leave the
+extension unable to ask for it.
+
+`outputs` is checked after the build and recorded in the stamp. A build system
+that quietly produces nothing — a `configure` that disabled the library, a
+`make` that built only tools — otherwise looks like a success until an
+extension fails to link with a message about a missing symbol.
+
+Provenance is either `local` (source in this repository, still digested so an
+edit rebuilds) or a pinned `url` and `sha256`, verified before anything is
+unpacked. These are kept out of `sources.lock` on purpose: that file pins what
+the *interpreter* links, and a library that exists for packages should not
+become something the runtime build has to download.
+
+`build.host` exists because autotools packages vendor a `config.sub` frozen at
+their release date. libyaml 0.2.5's is from 2018 and rejects
+`wasm32-unknown-emscripten` outright; it accepts `wasm32-unknown-none`, which
+is sufficient because `--host` only tells `configure` that this is a cross
+build — `emconfigure` has already supplied the tools. That is a property of
+the package, so it is declared per dependency rather than worked around for
+everybody.
+
+## Reproducible inputs, patches and provenance
+
+Builds never run in the source tree. The project is copied to a staging
+directory first, then patched, then built. Two failures follow from doing
+otherwise: a patch applied in place leaves the checkout modified, so a second
+build starts from different source and the patch no longer applies; and a build
+leaves generated files behind — Cython's `.c`, `.egg-info` — which then become
+inputs to the next one.
+
+Every input is pinned and verified before use:
+
+| input | pinned by | verified by |
+| --- | --- | --- |
+| interpreter and its libraries | `sources.lock` | sha256 before unpacking |
+| build tools (setuptools, Cython) | `build-tools.lock` | `pip --require-hashes` |
+| native target libraries | `dependency.json` provenance | sha256 before unpacking |
+| package source | recipe `source`, or in-repo | sha256, or tree digest |
+| patches | `patches.json` `sourceDigest` | exact-context `git apply` |
+
+Wheels are byte-reproducible: every zip member is stamped with the zip epoch
+rather than the build time and RECORD is sorted, so two builds from the same
+inputs produce identical files. This is stronger than the "documented timestamp
+normalization" the task allowed, and it is what makes "did anything actually
+change?" answerable by comparing digests. Confirmed by rebuilding every wheel
+and diffing digests, and by `test/python-runtime/reproducible-builds.test.ts`.
+
+Each wheel carries `dist-info/sandboxedjs-provenance.json`, which answers, for
+an artifact held by someone without this repository: which source tree digest
+produced it, which build tools and native libraries went into it, which ABI id
+and recipe revision were used, and which patches were applied. The index
+summarises those fields so they can be compared across wheels without
+downloading any.
+
+## Build backends
+
+`scripts/backends.py` is the only place a package's build system is known
+about. The orchestrator verifies inputs, selects a backend by the name the
+recipe stated, hands it a `BuildRequest`, and packages the `BuildResult` that
+comes back. It decides nothing else — which is what keeps a package-name check
+from appearing in it, since once an orchestrator is already making build
+decisions, one more looks harmless.
+
+A backend receives verified staged source, the locked build-tool interpreter,
+the cross environment, the declared native dependencies, a staging directory
+and a read-only view of the ABI. It returns a wheel-layout tree, the package's
+own `.dist-info`, and evidence recorded in the wheel's provenance. It cannot
+contribute a compiler flag, and the orchestrator digests the ABI contract
+before and after the build — a backend that mutated it would produce a
+correctly-tagged wheel built for a different ABI.
+
+The `pep517` backend calls the package's declared hooks the way a frontend
+does, including resolving an in-tree backend through `backend-path`. It does
+**not** use `python -m build`, whose isolation installs whatever versions
+upstream publishes on the day of the build — the unpinned input the rest of
+this pipeline refuses. The environment is assembled from `build-tools.lock`,
+and a build requirement absent from the lock is refused rather than fetched.
+The wheel the hooks produce is unpacked rather than shipped: it carries
+whatever tag the backend chose, and this pipeline tags a wheel for what it
+actually is.
+
+Build tools run on an interpreter whose feature version matches the target's.
+Most of the cross build does not care, because `sysconfig` answers for the
+target either way — but `bdist_wheel` composes a tag from the *running*
+interpreter's version and the *target's* ABI tag. Building for 3.13 from 3.14
+yields `('cp314', 'cp313', 'emscripten_5_0_6_wasm32')`, which fails an
+assertion inside setuptools rather than anywhere that names the cause.
+
+## Meson
+
+`kind: "meson"` drives a project through meson-python's PEP 517 hooks. It is
+its own backend rather than a `pep517` recipe with extra settings because a
+cross file is backend mechanics, not a package quirk — a recipe that passed its
+own `--cross-file` could describe a different target than the wheel is tagged
+for.
+
+Meson does not read `sysconfig`; it is told about a target by a file. So the
+same facts the cross table states are stated again in Meson's format, generated
+from `abi/extension-abi.json` by `write_meson_cross_file`. Three settings are
+supplied by the backend because they are facts about the target, not choices:
+
+- `needs_exe_wrapper = true`, or Meson believes it can run what it builds and
+  every compile-and-run check silently tests the build machine.
+- `--wrap-mode=nodownload`. Meson otherwise resolves a missing subproject by
+  cloning it mid-build — an unpinned input arriving over the network at the one
+  moment nothing is watching.
+- `-Ddefault_library=static`. There is no such thing as a shared library here;
+  `ld.wasm` refuses, and Meson reports it at configure time as a message about
+  the linker rather than about the default that reached it.
+- `longdouble_format`, derived by asking the pinned compiler for its
+  `__LDBL_MANT_DIG__` and byte order rather than being written down. Meson
+  cannot run a program on the target to find out, and a project that inspects
+  float layouts — NumPy does — will not configure without it. Deriving it
+  matters more than the convenience: a hand-written value that disagreed with
+  the toolchain would produce a library that builds, imports, and computes
+  wrong answers.
+
+A project whose subprojects were previously downloaded supplies them through
+`vendoredSources`: hash-verified archives unpacked at declared paths inside the
+staged tree, recorded in the wheel's provenance because they are compiled into
+it. They cannot escape the staged tree.
+
+### Two things that were failures first
+
+**Split headers.** A cross build leaves `Include/` in the source tree and the
+generated `pyconfig.h` in the build directory; an *installed* interpreter has
+them in one place, and tools assume the installed shape. `Python.h` includes
+`"pyconfig.h"` in quotes, so the compiler looks beside `Python.h` and then walks
+the `-I` list — and finds the *build machine's* `pyconfig.h` from whichever host
+include directory a tool added. The build then fails with `LONG_BIT definition
+appears wrong for platform`, which reads like a broken toolchain and is in fact
+a 64-bit header describing a 32-bit target. Meson adds such a directory;
+setuptools happens not to. Both now compile against one staged directory
+holding every target header.
+
+**Paths in compiled output.** This one recurred three times before the rule was
+clear: *nothing a compiler sees may live at a per-build path.*
+
+First the staged include directory, created inside the temporary build
+directory — the compile line carries `-g`, so the varying path reached debug
+information. Then Meson's own `.mesonpy-<random>` build directory, which also
+has to be pinned together with the cross file, because Meson caches the cross
+file's path inside the build directory and a stale one fails on the next run
+with a `FileNotFoundError` about a file nobody asked for.
+
+Then the staged *source* directory, which was the subtlest: Cython writes the
+`.pyx` path into its generated C so a traceback can name a line. NumPy exposed
+it — 869 of the 874 files in its wheel matched between builds, and the five
+that differed were its Cython-generated modules, all at identical sizes. The
+staged source is now a stable path too, and `reproducible-builds.test.ts`
+checks that the generated cross configuration carries no temporary path.

package/docs/python/extensions.md

@@ -0,0 +1,282 @@
+# Native extensions for the owned CPython runtime
+
+This describes how a compiled CPython extension — a C module, or a Rust one
+built with PyO3 — is built, tagged, installed and loaded by the SandboxedJs
+Python distribution.
+
+It is a general pipeline. Nothing in the runtime, the installer or the import
+machinery knows about any particular package. Adding support for a package
+means adding a *recipe*; it never means adding a runtime adapter.
+
+## The contract
+
+`python-runtime/abi/extension-abi.json` is the single source of truth. The
+wheel tag, the compiler flags side modules are built with, the identifier the
+wheel cache is keyed on and the checks the builder makes are all generated
+from it by `scripts/generate_extension_abi.py` into:
+
+- `python-runtime/abi/sbx_ext_abi.h` — for C extensions
+- `src/runtime/python/extension-abi.ts` — for the installer and loader
+
+Regenerate with `make -C python-runtime extension-abi` and commit the output.
+
+### What the contract fixes
+
+| Field | Value |
+| --- | --- |
+| Implementation | CPython 3.13.5 |
+| ABI tag | `cp313` |
+| SOABI | `cpython-313-wasm32-emscripten` |
+| Extension suffix | `.cpython-313-wasm32-emscripten.so` |
+| Target triple | `wasm32-unknown-emscripten` |
+| Emscripten | 5.0.6 |
+| Threads | pthreads, shared memory |
+| Memory | 256 MiB initial, growth enabled |
+| Stack | 8 MiB |
+| Exceptions | WebAssembly EH (`-fwasm-exceptions`), longjmp lowered to wasm |
+| Dynamic linking | CPython is `MAIN_MODULE=1`; extensions are `SIDE_MODULE=1` |
+| Wheel tag | `cp313-cp313-emscripten_5_0_6_wasm32` |
+
+The wheel tag is the PEP 425 platform tag CPython itself reports on
+Emscripten, so `pip` and `packaging` accept it without a project-specific
+patch. **Linux and macOS wheels are never accepted.** A `manylinux` wheel
+contains ELF objects this interpreter cannot load; installing one would turn a
+clear resolution failure into an `ImportError` inside the user's program.
+
+### Why there is an ABI id as well as a tag
+
+`abiId` (currently `sbxabi1-…`) is a hash of every field that affects binary
+compatibility. It is deliberately finer-grained than the wheel tag: the
+platform tag names only the Emscripten version, but two builds from the same
+Emscripten with different pthread or memory settings are still incompatible.
+
+The tag is what the packaging ecosystem understands. The id is what this
+project caches and verifies on. Change any field that matters and the id
+changes with it, so every previously built wheel stops matching rather than
+being loaded against an interpreter it no longer fits.
+
+The id is a hash rather than a hand-maintained version number because a
+version number is bumped when someone remembers to bump it, which is not the
+same occasion as the ABI actually changing.
+
+## Build profiles
+
+`--enable-wasm-dynamic-linking` is what makes extension loading possible at
+all, and it is a separate profile rather than a flag on the existing one:
+
+| Profile | Threads | Native extensions |
+| --- | --- | --- |
+| `core` | no | linked in, fixed |
+| `threaded-fixed` | yes | linked in, fixed |
+| `dynamic` | yes | **loaded at import** |
+
+`dynamic` is the profile shipped in the npm package. Build it with:
+
+```bash
+make -C python-runtime python PROFILE=dynamic
+make -C python-runtime package PROFILE=dynamic
+```
+
+`dynamic` has its own OpenSSL sysroot (`out/sysroot-dynamic`) because every
+object linked into a `MAIN_MODULE` must be position independent.
+
+## Adding a package
+
+Write a recipe and build it:
+
+```bash
+python3 python-runtime/scripts/build_extension.py path/to/recipe.json
+```
+
+A recipe names sources or a crate, and nothing else:
+
+Pydantic is version-pinned at the native boundary. The repository carries both
+`pydantic-core==2.23.2` for `pydantic==2.9.x` and `pydantic-core==2.46.5` for
+newer Pydantic releases that require it. Keep each exact pair in the wheel
+index; a newer core cannot satisfy an older Pydantic requirement.
+
+```json
+{
+  "name": "sbx-c-probe",
+  "version": "1.0.0",
+  "kind": "c-extension",
+  "modules": [{ "name": "sbx_c_probe", "sources": ["sbx_c_probe.c"] }]
+}
+```
+
+```json
+{
+  "name": "sbx-rust-probe",
+  "version": "1.0.0",
+  "kind": "pyo3",
+  "modules": [{ "name": "sbx_rust_probe", "crate": ".", "crateName": "sbx_rust_probe" }]
+}
+```
+
+**A recipe cannot set compiler or link flags.** They come from the contract.
+A recipe that could set its own `-pthread` could produce an artifact that
+links and then corrupts memory, and the mismatch would be invisible in the
+wheel it produced.
+
+The builder refuses to produce a wheel unless the artifact is a WebAssembly
+binary, declares a `dylink` section, and exports `PyInit_<module>`. All three
+failures are misleading at import time, so they are caught at build time and
+named.
+
+## Rust and PyO3
+
+Two properties of the Rust toolchain decide how this works, and both were
+found as link errors rather than documented limitations.
+
+**Rust's shipped `std` for `wasm32-unknown-emscripten` has no atomics.** It is
+built without the `atomics` and `bulk-memory` features, so it cannot be linked
+into a shared-memory module — and this runtime's main module has pthreads. The
+error names an `.rcgu.o` file:
+
+```
+wasm-ld: error: --shared-memory is disallowed by …rcgu.o because it was not
+compiled with 'atomics' or 'bulk-memory' features.
+```
+
+So `std` is rebuilt from source with those features, which needs a nightly
+toolchain for `-Z build-std`. This is a property of the threading model, not a
+preference: a runtime built without pthreads would link stable Rust's shipped
+`std` unchanged.
+
+**Rust panics unwind using the WebAssembly exception proposal.** A `cargo`
+built extension imports a `__cpp_exception` tag, and a main module linked
+without exception handling cannot supply one:
+
+```
+LinkError: WebAssembly.Instance(): Import #420 "env" "__cpp_exception":
+tag import requires a WebAssembly.Tag
+```
+
+CPython is therefore linked with `-fwasm-exceptions` (and
+`-sSUPPORT_LONGJMP=wasm`, since the two share a mechanism). Enabling it in the
+main module is preferred over forcing `panic = "abort"` on every Rust
+extension, because PyO3 turns a Rust panic into a Python exception by catching
+the unwind — an aborting build would take the process down instead of raising.
+
+PyO3 is cross-compiled through a generated config file rather than allowed to
+probe, because probing describes the *build machine's* interpreter, which is
+how a cross build silently produces a host artifact.
+`suppress_build_script_link_lines` matters most: without it PyO3 emits
+`-lpython3.13`, and there is no such library here — the CPython symbols come
+from the main module at load time, which is what a side module's undefined
+symbols are for.
+
+## Installation
+
+The installer resolves before it downloads, and downloads before it writes:
+
+1. Already-installed distributions are skipped.
+2. Dependencies are resolved with constraint propagation and conflict
+   learning (`src/runtime/python/resolver.ts`).
+3. A compatible pure-Python wheel is preferred.
+4. Then a wheel tagged for this ABI.
+5. Then, if enabled, a source build.
+6. Every download is verified against the digest the index published. An
+   artifact published without one is refused rather than installed unverified.
+7. The whole solved graph is staged and committed as one transaction.
+
+Resolution reads dependencies from **PEP 658 metadata sidecars**, a few
+kilobytes each, rather than downloading whole wheels to read `METADATA`. That
+is what removed the previous installer's arbitrary 96-candidate cap: the cap
+existed because discovering that forty releases all needed the same
+unavailable extension cost forty multi-megabyte downloads. What bounds the
+search now is a deadline and a cancellation signal, which are honest limits —
+they say the search ran out of time, not out of an arbitrary allowance.
+
+## Security model
+
+- Wheels are fetched only when the container's `allowOutbound` policy permits
+  it. A container that refuses `curl` cannot install packages.
+- Every artifact is verified by SHA-256 against the index's published digest
+  before any of it is written.
+- Extension code runs inside the same WebAssembly sandbox as the interpreter.
+  It has no more access to the host than Python does: the filesystem it sees
+  is the container's, served over the host ABI.
+- The build toolchain is not shipped to consumers. The npm package contains
+  the runtime and the loader.
+
+## "The ASGI callable works" is not "Uvicorn serves"
+
+These are different claims, and only the first is proven.
+
+**Proven.** FastAPI imports, Pydantic 2 validates a request body through the
+compiled `pydantic_core` extension, and the application is driven through the
+public ASGI protocol — an ordinary `await app(scope, receive, send)` — with the
+response status and body asserted. Installing `fastapi[standard]` also installs
+the FastAPI CLI, whose `fastapi dev main.py` and `fastapi run main.py` commands
+are available. A synchronous endpoint is exercised, so Starlette's threadpool
+path runs on real worker threads.
+
+**Not proven.** `uvicorn main:app` binding a port that the container's
+networking can route a request to. This is not merely untested; the layer is
+missing, and the way it is missing is worth stating precisely because it looks
+like success:
+
+```python
+s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+s.bind(("127.0.0.1", 8123)); s.listen(1)   # succeeds
+```
+
+`bind` and `listen` return cleanly, and `asyncio`'s event loop has
+`create_server`. But that socket belongs to **Emscripten's own socket
+emulation**, which is not connected to `VirtualHttpRouter` — the router that
+already routes `curl localhost:3000` to a Node server in the same container. So
+a server would appear to start, report itself listening, and never receive a
+request. A test that asserted "uvicorn started" would pass while proving
+nothing.
+
+Closing this means bridging CPython's socket layer to the container network the
+way `sbxfs.ts` bridges its filesystem: `listen` registers a virtual port with
+the router, and an accepted connection becomes a socket whose reads and writes
+cross the host ABI. It belongs with the socket layer, not with Uvicorn —
+patching Uvicorn would make one server work and leave every other one broken.
+
+Note also that `socket.SO_REUSEADDR` is absent from this build, which some
+servers set unconditionally.
+
+## A resolution behaviour worth knowing
+
+`pip install pydantic` **succeeds** without a wheel index, by backtracking to
+the 1.x line — which is pure Python and needs no compiled core. That is correct
+of the resolver and is what pip does too, but it is unlikely to be what someone
+asking for Pydantic today wants. Constrain the requirement (`pydantic>=2`) when
+the 2.x line is what you mean; with the bound, an unavailable native dependency
+is reported rather than worked around.
+
+## Current limitations
+
+- Building extensions requires Emscripten 5.0.6, a nightly Rust with
+  `rust-src`, and the CPython build tree. Consumers install prebuilt wheels.
+- Source distributions are resolved but not built in-container; a resolution
+  that lands on one reports that no builder is configured, rather than
+  claiming an install. The optional local and remote builders described in the
+  architecture are not implemented.
+- **Uvicorn cannot serve.** See the section above; the ASGI callable works.
+- Emscripten documents dynamic linking with pthreads as experimental.
+- The default package wheel index carries the ABI-matched Pydantic Core wheels
+  beside the interpreter. Consumers can override it with
+  `configurePython({ wheelIndex })` to add privately built extensions.
+- `socket.SO_REUSEADDR` is absent from this build.
+
+## Which tests prove what
+
+| Claim | Test |
+| --- | --- |
+| A C extension imports and runs | `test/python-runtime/extensions.test.ts` |
+| A PyO3 extension imports and runs | `test/python-runtime/extensions.test.ts` |
+| The ABI id is derived, not hand-written | `test/python-runtime/extension-abi.test.ts` |
+| Linux wheels are refused | `test/python-runtime/extension-abi.test.ts` |
+| Resolution has no candidate cap | `test/python-runtime/resolver.test.ts` |
+| Constraints propagate, conflicts explain | `test/python-runtime/resolver.test.ts` |
+| A failed native install leaves nothing | `test/python-runtime/extensions.test.ts` |
+| Real pydantic-core validates data | `test/python-runtime/fastapi.test.ts` |
+| Pydantic 2 + FastAPI + ASGI + console script | `test/python-runtime/fastapi.test.ts` |
+
+The FastAPI test needs `SANDBOXEDJS_CLEAN_NETWORK_TESTS=1`; the extension
+tests need `make -C python-runtime extensions` to have been run, and skip with
+a reason otherwise.