sandboxedjs 0.2.20 → 0.2.21

package/dist/index.cjs

@@ -21103,7 +21103,7 @@ var wheels_default = {
 
 // package.json
 var package_default = {
-  version: "0.2.20"};
+  version: "0.2.21"};
 
 // src/python/extension-abi.ts
 var EXTENSION_ABI = {
@@ -30005,7 +30005,7 @@ function applyEdits2(source, edits) {
 function isNode4(value) {
   return typeof value === "object" && value !== null && typeof value.type === "string";
 }
-var EXTENSIONS = [".js", ".mjs", ".cjs", ".json", ".node"];
+var EXTENSIONS = [".js", ".mjs", ".cjs", ".json", ".node", ".ts", ".tsx"];
 var PREFIX_ONLY_BUILTINS = /* @__PURE__ */ new Set(["test", "test/reporters", "sea", "sqlite"]);
 var CONDITION_SETS = {
   import: [["node", "import", "module", "default"], ["node", "require", "default"], ["default"]],

package/dist/index.js

@@ -21086,7 +21086,7 @@ var wheels_default = {
 
 // package.json
 var package_default = {
-  version: "0.2.20"};
+  version: "0.2.21"};
 
 // src/python/extension-abi.ts
 var EXTENSION_ABI = {
@@ -29988,7 +29988,7 @@ function applyEdits2(source, edits) {
 function isNode4(value) {
   return typeof value === "object" && value !== null && typeof value.type === "string";
 }
-var EXTENSIONS = [".js", ".mjs", ".cjs", ".json", ".node"];
+var EXTENSIONS = [".js", ".mjs", ".cjs", ".json", ".node", ".ts", ".tsx"];
 var PREFIX_ONLY_BUILTINS = /* @__PURE__ */ new Set(["test", "test/reporters", "sea", "sqlite"]);
 var CONDITION_SETS = {
   import: [["node", "import", "module", "default"], ["node", "require", "default"], ["default"]],

package/dist/worker-entry.js

@@ -21935,7 +21935,7 @@ function f(e2, r2, t2) {
 }
 
 // src/node/commonjs-engine.ts
-var EXTENSIONS = [".js", ".mjs", ".cjs", ".json", ".node"];
+var EXTENSIONS = [".js", ".mjs", ".cjs", ".json", ".node", ".ts", ".tsx"];
 var PREFIX_ONLY_BUILTINS = /* @__PURE__ */ new Set(["test", "test/reporters", "sea", "sqlite"]);
 var CONDITION_SETS = {
   import: [["node", "import", "module", "default"], ["node", "require", "default"], ["default"]],

package/docs/to_implement/agent.md

@@ -0,0 +1,261 @@
+# Agent feature
+
+I want `agent` command in the sandbox. it is a seperate app like officesuit and vireo, we have to provide all the dependencies for it to run here in Sandboxedjs module but we don't provide the .sbjs package here, it will be downloaded from sandboxedjswheels.pages.dev site, the code of that site is in /Users/shazi/Projects/sandboxedjswheels.
+
+# First Phase
+
+## Objective and scope
+
+Provide the runtime dependencies of the future `agent` application and the
+transitive dependencies needed to install and execute them. The application stays
+an independently downloaded `.sbjs` package from `sandboxedjswheels.pages.dev`;
+SandboxedJs provides its execution environment, dependency resolver, and optional
+runtime packs. Model creation, training, publication, and agent implementation are
+separate work. This section is an implementation plan, not a claim of existing
+package support.
+
+Required targets:
+
+- Transformers.js in the browser using its upstream browser inference backend.
+- Python Transformers and CPU PyTorch using compatible upstream Linux packages.
+- The actual upstream Linux Ollama distribution, including its server and CPU
+  runner. A wllama adapter, recompiled replacement, or implementation of only the
+  Ollama HTTP API does not satisfy this requirement.
+- Owned, reproducible C/C++, Rust, and supporting build-tool distributions for
+  extending package coverage. Ownership means pinned upstream tools, build recipes,
+  patches, and release artifacts; it does not require inventing a compiler.
+
+## Execution architecture: prerequisite before package installation
+
+The current POSIX command layer, CPython-to-Wasm runtime, WASI host, and ELF
+dispatch hooks are useful foundations. They do not execute arbitrary Linux
+packages. The current original x64 engines only handle a small freestanding
+instruction subset and reject dynamic linking; they cannot run Ollama, a normal
+Linux Python interpreter, or PyTorch. See [browser architecture](../browser-runtime-architecture.md),
+[binary backends](../developer-tool-packs.md), and [original x64 limits](../original-x64.md).
+
+Use two explicit execution profiles:
+
+| Profile | Runtime | Package ABI | Initial purpose |
+| --- | --- | --- | --- |
+| Browser/Wasm | Existing JS runtime, owned Wasm CPython, compatible WASI tools | Browser JS, exact SandboxedJs Python extension ABI, supported WASI imports | Transformers.js and individually validated scientific Wasm packages |
+| Linux guest | Optional full-system CPU emulator compiled to Wasm, Linux kernel, persistent root filesystem | One selected Linux CPU architecture, glibc and native Python wheel tags | Unmodified Ollama, Python Transformers, CPU PyTorch, native compilers |
+
+For the Linux profile, select a **64-bit architecture supported by the exact
+Ollama and PyTorch releases**. Start by evaluating x86-64 with a glibc-based Debian
+root filesystem. An emulator must boot that architecture and execute the CPU
+instructions used by those binaries and their runners. Do not select a 32-bit
+emulator just because it can boot a Linux image. Evaluate a browser-capable
+full-system QEMU-derived build or another suitable engine; no working engine is
+selected or supplied by this document. Its browser build, memory model, licensing,
+and performance must be demonstrated before committing to it.
+
+WebAssembly executes the emulator; the emulator executes Linux machine code and
+Linux supplies the kernel ABI. Adding libc, WASI, or a compiler alone cannot make
+an upstream Linux ELF binary execute in the existing Wasm runtime. Extending the
+original translator to that level is a substantial alternative engineering
+project, not a missing dependency that can simply be installed.
+
+## Dependency packs and their dependencies
+
+Pack names below are proposed distribution units, not existing npm packages.
+Heavy assets should download on demand rather than become unconditional npm
+dependencies of `sandboxedjs`.
+
+| Pack | Direct contents | Dependencies and required capabilities |
+| --- | --- | --- |
+| `agent-runtime` | `.sbjs` loader, dependency manifest/resolver, execution-profile selection, process and service supervision | Existing container APIs; runtime-pack registry; version checks; install progress, cancellation, logs and cleanup |
+| `browser-ml` | Pinned `@huggingface/transformers`, its resolved browser dependencies and matching ONNX Runtime Web assets | Correct browser export resolution; module workers; fetch, streams, typed arrays; Wasm assets; persistent model cache; optional SIMD, threads and WebGPU according to selected backend |
+| `linux-machine` | Wasm CPU emulator, boot assets, Linux kernel, initramfs where required, guest control agent | Worker isolation, virtual CPU/MMU/interrupts/timers, disk and network devices, guest RAM, persistent disk adapter, host/guest RPC and termination |
+| `linux-base` | glibc distribution rootfs, ELF loader, shell, coreutils, `apt`/`dpkg`, repository metadata/keyrings | Matching architecture and kernel; distro-resolved shared libraries; process, filesystem, clock, entropy, networking and certificate support |
+| `linux-downloads` | `curl`, CA certificates, `tar`, `zstd`, gzip/xz/unzip as required | DNS and working TCP/TLS transport, writable temporary storage, trusted artifact metadata and extraction limits |
+| `linux-python-ml` | Native CPython, `venv`, pip, CPU `torch`, `transformers` | Native wheels matching Python, architecture and glibc; full Python dependency closure and ELF library closure described below |
+| `linux-ollama` | Pinned upstream Linux Ollama release, bundled libraries and CPU runners | `linux-machine`, `linux-base`, `linux-downloads`; runner-compatible CPU features; writable model store; local HTTP server access |
+| `linux-build-tools` | Native C/C++ compiler, linker, Rust toolchain and build drivers | Linux profile, development headers, language standard libraries, build-system dependencies and package-specific source recipes |
+| `wasm-build-sdk` | Emscripten/LLVM, supported WASI SDK, Rust Wasm target, owned Python extension SDK | Separate target sysroots and ABI locks; reproducible external build environment initially; compatible Wasm outputs tested in SandboxedJs |
+
+Transformers.js uses ONNX Runtime in the browser; provide the assets matching the
+locked package release and resolve browser exports instead of accidentally loading
+native Node addons. WebGPU is optional for this path and does not accelerate the
+Linux guest automatically. [Upstream Transformers.js documentation](https://huggingface.co/docs/transformers.js/en/index)
+and [asset configuration](https://huggingface.co/docs/transformers.js/custom_usage).
+
+## Linux facilities required underneath the packages
+
+The Linux machine must supply more than command names:
+
+- **CPU and memory:** 64-bit guest execution, virtual memory and page permissions,
+  floating point, required SIMD, atomics, and coherent thread behavior. Verify
+  CPUID/feature detection against the actual Ollama runner and PyTorch build.
+  Guest 64-bit addressing does not remove browser or emulator memory limits.
+- **Processes:** executable loading, dynamic linking, `fork`/`exec`/`wait`, threads,
+  futexes, signals, pipes, polling/epoll, timers and service termination. A full
+  guest kernel provides these, but the emulator must support their foundations.
+- **Filesystem:** permissions, symlinks, executable bits, atomic rename, locks,
+  mmap, large files, `/tmp`, `/proc`, `/sys`, `/dev`, entropy devices and shared
+  memory. Mount a persistent writable disk with capacity checks and recovery.
+- **Networking:** guest loopback, DNS, TCP, TLS certificates and correct time;
+  downloads, package repositories, Hugging Face and the Ollama registry must work
+  through the configured outbound policy.
+- **Host integration:** execute/upload/download APIs, streaming stdout/stderr,
+  cancellation, service readiness, and a bridge to guest HTTP ports, including
+  Ollama's default port 11434. Keep the guest disk separate from the current VFS
+  and transfer files explicitly with documented ownership and consistency.
+
+Browsers cannot directly supply arbitrary raw TCP sockets. General Linux package
+managers and native TLS clients require a guest network bridge to an explicitly
+configured WebSocket/TCP relay or equivalent host transport. The current
+fetch-based HTTP egress is not transparent networking for unmodified Linux
+programs. A relay forwards network traffic; model computation still runs locally.
+Document this infrastructure dependency. Offline operation is possible after
+required packages and models are cached; arbitrary live registry access cannot be
+promised on a static host alone.
+
+## Python and scientific/ML dependency closure
+
+Use native Python inside the Linux guest for the initial full Transformers/PyTorch
+target. Choose the Python version together with the available CPU wheels, rather
+than inheriting the Wasm interpreter version automatically. Resolve the exact
+dependency tree from the selected releases and extras; the following is the
+coverage inventory, not a substitute for a lockfile:
+
+| Layer | Dependencies to resolve and test |
+| --- | --- |
+| Transformers | `transformers`, `huggingface-hub`, `tokenizers`, `safetensors`, NumPy, packaging, filelock, PyYAML, regex, tqdm, and the network/filesystem dependencies declared by the selected versions |
+| PyTorch CPU | CPU-only `torch` wheel and its declared Python dependencies; included or required C/C++ runtime, OpenMP and math libraries; inspect ELF dependencies recursively |
+| Model-dependent extras | SentencePiece/protobuf, Pillow, audio libraries, torchvision/torchaudio, or other processors only when required by the supported model and task |
+| Additional science coverage | NumPy first; SciPy, pandas and scikit-learn as separately validated additions, with matching BLAS/LAPACK, OpenMP and Fortran runtimes where required |
+| Download/cache stack | The selected hub client's HTTP/TLS and filesystem packages, cache locks, resumable downloads, credentials and offline-cache behavior |
+
+Rust-based `tokenizers` and `safetensors`, C/C++ extensions, and PyTorch's native
+libraries are transitive runtime artifacts even when a user only installs a
+Python package. Prebuilt compatible wheels avoid compiling them at installation
+time. Pin CPU wheels explicitly so resolution does not pull an unwanted CUDA
+stack. The supported Python/torch combination must follow the chosen releases'
+[Transformers installation requirements](https://huggingface.co/docs/transformers/installation)
+and [PyTorch CPU installation instructions](https://pytorch.org/get-started/locally/).
+
+For the existing Wasm Python profile, pure Python packages still need their native
+dependencies ported. Build every compiled extension against this project's exact
+Python/Emscripten/extension ABI; never relabel a manylinux wheel as Wasm. A full
+PyTorch Wasm port is separate work and is not implied by providing Clang or Rust.
+See the [owned Python architecture](../python/architecture.md) and
+`python-runtime/abi/extension-abi.json` for the current ABI contract.
+
+## Real Linux Ollama installation contract
+
+Download a pinned upstream archive for the selected guest architecture, verify
+its recorded digest, and install its binary **and required bundled libraries and
+runners** inside the guest. Preserve upstream layout and resolve all additional
+ELF shared-library requirements against the chosen rootfs. Maintain a release
+inventory of loader paths, `DT_NEEDED` dependencies, minimum libc requirements
+and runner CPU features; determine these from the selected artifact rather than
+guessing a permanent list.
+
+Support the upstream manual installation path and launch `ollama serve` through
+the guest supervisor. Systemd is not required for this path. Running the upstream
+installer script is a separate compatibility test because its service/user setup
+can require additional distro tools. [Official Linux installation instructions](https://docs.ollama.com/linux).
+
+Validate version reporting, server readiness, API access, model download/import,
+persistence, restart and cancellation. These must execute the real upstream
+Linux processes, with backend and version reported to the user. Browser WebGPU
+does not provide CUDA or ROCm to these processes; CPU execution is the first
+target. GPU passthrough/translation is outside this phase.
+
+## Owned compiler and build dependencies
+
+Separate the machine **running a compiler** from the architecture it **produces**.
+A Linux `clang` executable needs the Linux guest; emitting Wasm does not make the
+compiler itself browser-executable.
+
+- **Native Linux toolchain:** GCC/G++ or Clang/LLVM, linker (GNU ld or LLD),
+  binutils/LLVM inspection tools, libc development headers, C++ headers/runtime,
+  `make`, CMake, Ninja, pkg-config, Git, patch and archive tools. Add Autoconf,
+  Automake and libtool when recipes require them.
+- **Rust:** pinned `rustc`, Cargo, matching standard libraries and target support,
+  a working C linker, cached crates and locked dependency resolution. Use maturin
+  or setuptools-rust for relevant Python extension recipes. `rustup` may provision
+  a toolchain but is not itself the compiler or an inference dependency.
+- **Python builds:** Python development headers, pip/build/setuptools/wheel,
+  Cython, meson-python/Meson or scikit-build-core as declared by each package.
+  Add OpenSSL, libffi, zlib, other compression headers, BLAS/LAPACK, and Fortran
+  compiler/runtime only for recipes that require them.
+- **Wasm builds:** pinned Emscripten, LLVM/LLD, separate WASI sysroot, compatible
+  Rust target and the owned CPython extension headers/ABI. Keep Emscripten Python
+  side modules distinct from WASI command modules. Begin with reproducible builds
+  outside the browser; compiling these tools themselves to browser-compatible
+  Wasm is a separately tested capability.
+- **Ollama source builds, if later needed:** derive the Go, C/C++ and build-system
+  requirements from the pinned upstream release. Building from source is optional;
+  installing its official Linux artifact must work without a compiler.
+
+Publish recipes, patches, source hashes and target manifests for these toolchains.
+Prebuilt runtime packs should remain sufficient for normal users; downloading an
+entire compiler stack must not be required merely to load a model.
+
+## Distribution and dependency resolution
+
+Define a versioned manifest for each pack and the future agent package containing:
+
+- Pack ID/version, execution profile, CPU architecture, OS/libc, Python ABI or
+  Wasm ABI as applicable, and minimum runtime capabilities.
+- Direct dependencies plus a resolved transitive lock with versions, artifact
+  URLs, hashes, installed paths, compressed/expanded sizes and licenses/notices.
+- Required CPU features, RAM/disk estimates, network endpoints, install recipe,
+  readiness checks and supported test matrix.
+- Build-only versus runtime dependencies, optional extras and model assets.
+
+The agent manifest requests these capabilities; the resolver chooses only packs
+compatible with its declared profile. Use separate Linux and Wasm package stores,
+and never silently switch execution profiles after an installation failure.
+Provide atomic installs, retry/resume, cache reuse, clean uninstall and rollback
+without deleting user models. Browser asset hosting must support the required
+CORS, worker URLs and cross-origin isolation for SharedArrayBuffer-based paths.
+Record third-party distribution obligations, including kernel/rootfs source and
+notice requirements, before publishing packs through the wheels site.
+
+## Model responsibility and documented limits
+
+The user will supply approximately 120M–200M parameter models; creating and
+uploading them is outside this phase. Validate a compatible small model once one
+is available. Parameter count alone does not establish compatibility: model
+architecture, file format, operators, quantization and runner support also matter.
+
+There must be no arbitrary parameter-count block. Let users select models and
+show estimated memory, available resources and measured performance. Billion-
+parameter models are outside the initial validation target and may be impractical
+in the emulated Linux profile; this is not a universal claim that every such model
+cannot run. Even 120M–200M models are not guaranteed to run smoothly until measured.
+Weights alone require roughly parameters × bits-per-weight / 8 bytes, with
+additional memory for the kernel, emulator, runtime, activations, KV cache and
+temporary copies. Resource exhaustion and cancellation must fail cleanly.
+
+## Implementation order and acceptance gates
+
+1. **Lock the architecture:** select candidate emulator, guest architecture,
+   kernel/rootfs and package versions. Inventory the actual binary dependency
+   closure. Stop claiming Linux compatibility until the following gates pass.
+2. **Prove the Linux foundation:** boot in a real browser, run dynamically linked
+   native programs, exercise processes/threads/mmap, install a distro package and
+   preserve files across restart. Verify networking through the declared transport.
+3. **Prove upstream Ollama:** install the official pinned artifact, start its
+   server and exercise its API. Separately validate its CPU runner with a small
+   compatible model; report memory and performance without making model creation
+   part of this phase.
+4. **Prove native Python ML:** install the locked CPU wheels, import torch and
+   Transformers, perform tensor operations and run one small supported inference
+   task. Test hub downloads, offline cache and process cleanup.
+5. **Prove browser ML independently:** load Transformers.js through the browser
+   export, load matching backend assets, and run a small ONNX model. Exercise the
+   Wasm path and optional WebGPU path separately.
+6. **Prove toolchains:** compile and run a C/C++ program and Rust program in the
+   Linux guest; build/import representative C and Rust Python extensions. Verify
+   separate Wasm outputs against the existing runtime ABI.
+7. **Publish dependency packs:** lock transitive artifacts and document browser,
+   CPU, memory, storage, networking and package limitations. Test interrupted
+   installs, restart, cancellation, version conflicts and missing capabilities.
+
+Phase one is complete when these dependency paths are reproducible and their
+tests pass. The later `.sbjs` agent application can then depend on demonstrated
+capabilities instead of assumed Linux command compatibility.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sandboxedjs",
-  "version": "0.2.20",
+  "version": "0.2.21",
   "description": "A zero-infra JavaScript and Python sandbox for AI agents and browser IDEs. A lightweight, secure alternative to WebContainers and NodePod for running untrusted code with a full POSIX shell.",
   "type": "module",
   "license": "MIT",
@@ -108,6 +108,7 @@
     "set-cookie-parser": "^3.1.2",
     "stream-browserify": "^3.0.0",
     "string_decoder": "^1.3.0",
+    "sucrase": "^3.35.1",
     "timers-browserify": "^2.0.12",
     "url": "^0.11.4",
     "wa-sqlite": "^1.0.0"