semantifold 0.1.1 → 0.3.0

package/docs/testing.md

@@ -1,6 +1,110 @@
 # Testing
 
-Specs use the released `@velocious/testing@0.0.0` framework and standalone `velocious-test` runner with one top-level `describe` per file. Direct value checks use the framework's `expect` API where it improves clarity; predicate-rich diagnostic assertions retain `node:assert/strict`. Frontend tests load real fixtures through Babel, php-parser, Prism, and Lezer and compare modeled meaning after removing source locations and the separately tested provenance index. They separately assert normalized JSDoc and TypeScript types and verify locations exist.
+## Task 018 C operational qualification
+
+The canonical Ubuntu 26.04 development image installs `clang=1:21.1.6-71`, `clang-21=1:21.1.8-6ubuntu1`, and `libclang-rt-21-dev=1:21.1.8-6ubuntu1`. These exact installed versions were read with `dpkg-query -W clang clang-21 libclang-rt-21-dev`. The `clang` metapackage version differs from the compiler version; the discovered executable is Ubuntu Clang 21.1.8 at `/usr/lib/llvm-21/bin/clang`, target `x86_64-pc-linux-gnu`. Compiler-rt is required for ASan/UBSan and leak detection. Missing tools or runtime libraries fail the C lane; sanitizer coverage is mandatory on this qualified Linux platform.
+
+The source-independent root `Dockerfile` remains the canonical image definition and `compose.yml` service `dev` remains the canonical lane. An operator can reproduce the package prerequisite on Ubuntu 26.04 with:
+
+```sh
+sudo apt-get update
+sudo apt-get install --yes --no-install-recommends clang=1:21.1.6-71 clang-21=1:21.1.8-6ubuntu1 libclang-rt-21-dev=1:21.1.8-6ubuntu1
+dpkg-query -W clang clang-21 libclang-rt-21-dev
+clang --version
+clang -dumpmachine
+clang -print-resource-dir
+test "$(clang -dumpversion)" = "21.1.8"
+test "$(clang -dumpmachine)" = "x86_64-pc-linux-gnu"
+test -r "$(clang -print-resource-dir)/lib/linux/libclang_rt.asan-x86_64.a"
+test -r "$(clang -print-resource-dir)/lib/linux/libclang_rt.ubsan_standalone-x86_64.a"
+```
+
+TensorBuzz uses the standard Peakflow Ubuntu 24.04 (`noble`) base, not the development image. Its package repositories cannot resolve the Ubuntu 26.04 pins above. CI instead installs the exact Clang 21.1.8 and compiler-rt builds from the official signed LLVM Noble repository and selects `/usr/bin/clang-21` through `SEMANTIFOLD_CLANG`. The signing key is checksum-pinned before APT receives it, and the repository is restricted to that key through `signed-by`; there is no unsigned package or unpinned compiler fallback. The checked-in `tensorbuzz.yml` owns the executable CI setup:
+
+```sh
+sudo apt-get install --yes --no-install-recommends php-cli python3 ruby default-jdk-headless dotnet-sdk-10.0 ca-certificates curl gnupg
+curl --fail --silent --show-error --location https://apt.llvm.org/llvm-snapshot.gpg.key --output /tmp/semantifold-llvm.asc
+printf '%s  %s\n' '8b2a587ffd672c4687e7581dad4b2f6c1bb2ad6b480cd9771ba2ff48e0b8c75d' '/tmp/semantifold-llvm.asc' | sha256sum --check -
+sudo install -D -m 0644 /tmp/semantifold-llvm.asc /etc/apt/keyrings/llvm.asc
+printf '%s\n' 'deb [arch=amd64 signed-by=/etc/apt/keyrings/llvm.asc] https://apt.llvm.org/noble/ llvm-toolchain-noble-21 main' | sudo tee /etc/apt/sources.list.d/semantifold-llvm21.list
+sudo apt-get update
+sudo apt-get install --yes --no-install-recommends clang-21=1:21.1.8~++20251221032922+2078da43e25a-1~exp1~20251221153059.70 libclang-rt-21-dev=1:21.1.8~++20251221032922+2078da43e25a-1~exp1~20251221153059.70
+clang-21 --version
+clang-21 -dumpmachine
+clang-21 -print-resource-dir
+test "$(clang-21 -dumpversion)" = "21.1.8"
+test "$(clang-21 -dumpmachine)" = "x86_64-pc-linux-gnu"
+test -r "$(clang-21 -print-resource-dir)/lib/linux/libclang_rt.asan-x86_64.a"
+test -r "$(clang-21 -print-resource-dir)/lib/linux/libclang_rt.ubsan_standalone-x86_64.a"
+```
+
+The standard CI base was qualified at `peakflow/base-ubuntu-24-04@sha256:9b23d3d5f82a8babb312b7ce549e02d60a6f32344b0088b90d3f486a9d8207a5`: signed APT installation plus UID1000 compile/link/run passed at O0/O2, ordinary and ASan/UBSan/leak profiles. The development image retains its working Ubuntu 26.04 package pins and prints the same compiler probes. Provider versions/authentication, container replacement and host preparation remain external. Package pins can become unavailable if Ubuntu repositories retire them; that is an operational failure requiring deliberate requalification, never a skip or unpinned fallback.
+
+For an already materialized generated `program.c` and `semantifold_runtime.h`, use separate compile, link and native execution commands in a fresh directory:
+
+```sh
+export LANG=C.UTF-8 LC_ALL=C.UTF-8 TZ=UTC
+clang --no-default-config -std=c17 -Wall -Wextra -Werror -pedantic-errors -Wconversion -Wsign-conversion -Wshadow -Wstrict-prototypes -Wmissing-prototypes -Wformat=2 -ftrapv -finput-charset=UTF-8 -fexec-charset=UTF-8 -fno-color-diagnostics -O0 -c program.c -o program.o
+clang --no-default-config -O0 program.o -o program
+./program
+```
+
+Repeat with `-O2`. For both optimizations, repeat compilation and linking with `-g -fsanitize=address,undefined -fno-sanitize-recover=all -fno-omit-frame-pointer`, then execute with `ASAN_OPTIONS=detect_leaks=1:halt_on_error=1:allocator_may_return_null=1` and `UBSAN_OPTIONS=halt_on_error=1`. The initial UID1000 image qualification passed all four profiles with exact `OK\n`, status zero and empty stderr. Repository C specs independently compile/link/run generated artifacts under all four profiles and compare raw stdout bytes, decoded stdout, stderr and status. Their native byte capture is private test infrastructure; the public acceptance API/lifecycle is unchanged.
+
+The C focused files are `c-registry-toolchain.spec.js`, `c-frontend-validation.spec.js`, `c-backend-validation.spec.js`, `c-ordered-expressions.spec.js`, `c-provenance.spec.js`, `c-native-execution.spec.js`, `c-runtime-ownership.spec.js`, and `c-cross-language-acceptance.spec.js`, all under `spec/`. Run explicit files sequentially, for example:
+
+```sh
+npm run verify:legacy-runtime
+npx velocious-test spec/c-frontend-validation.spec.js
+npx velocious-test spec/c-ordered-expressions.spec.js
+npx velocious-test spec/c-native-execution.spec.js
+npx velocious-test spec/c-runtime-ownership.spec.js
+npx velocious-test spec/c-cross-language-acceptance.spec.js
+```
+
+The ownership spec separately compiles native linker instrumentation for `--wrap=malloc` and `--wrap=free`: exact allocation/free counts are checked at normal/fatal exit, the second allocation can fail deterministically, and duplicate/foreign frees are fatal. It never patches a dependency or adds a runtime test hook. Direct native header probes cover invalid lengths, metadata overflow and cumulative arena limits before any borrowed memory access. Signed-64-bit minimum/maximum output and all four arithmetic failures execute at O0/O2 with and without sanitizers. Compile, link and native status failures are tested independently, including removal of their temporary directories. Unicode/NUL copies, returned slices, every operator/consumer, eager ordering, both short-circuit paths, branch fallthrough, forward calls, recursion and unused bindings execute through real Clang.
+
+The unchanged legacy parser has a measured 32,767-UTF-16-code-unit source limit (including comments and whitespace). Direct parsing succeeds at that boundary, including astral comment text, and fails at 32,768. Both C source validation and exact generated-size preflight enforce it with located diagnostics before artifacts; no parser payload or consumer setting changes. Clang itself accepts larger generated programs, so native compilation alone cannot establish round-trip support. Strict C17 additionally rejects decoded literals above 4095 bytes; runtime concatenations can exceed that literal limit. The focused C specs execute the accepted literal boundary and nested-region round trips, and reject larger unrepresentable input/output before artifacts.
+
+The five C Tasks 001–004 fixtures generate/reparse/execute all original-five targets using real PHP, Ruby, Node, TypeScript/Node and Java compiler/runtime commands. One original-five source rotates through each C profile. Rich/v3 tests verify token origins, synthetic context, shared occurrences, stale metadata and UTF-16 coordinates. Negative corpora cover parser recovery, preprocessing/declarators, ownership/pointers, collisions, malformed IR and every ordered-region boundary. [The C profile](c.md) specifies the exact source, memory and overflow contracts.
+
+For this delivery, local testing is restricted to explicitly named changed/affected files. Aggregate `npm test`, directories and shards run only in TensorBuzz, whose end-to-end label includes ten languages and mandatory C/CPP O0/O2 sanitizers. Existing generic aggregate examples below describe that CI gate. Coordinator-owned independent review, exact-head CI and merge remain separate from local implementation delivery; publication/version/tag/release is not part of Tasks 018–019.
+
+Specs use the released `@velocious/testing@0.0.0` framework and standalone `velocious-test` runner with one top-level `describe` per file. Direct value checks use the framework's `expect` API where it improves clarity; predicate-rich diagnostic assertions retain `node:assert/strict`. Frontend tests load real fixtures through Babel, php-parser, Prism, Lezer, and the qualified official Tree-sitter Python/C#/Go routes and compare modeled meaning after removing source locations and the separately tested provenance index. They separately assert normalized annotations, complete named/anonymous/comment/directive traversal, parser recovery rejection, and exact UTF-16 locations.
+
+The private `packages/tree-sitter-legacy` workspace owns source checking and parser-neutral declaration generation for the internal legacy runtime; neither package is independently publishable. Its direct spec asserts the versioned frozen plain-data C CST, ordered field-bearing children, error/missing/extra state, UTF-16 indices and positions, and a declaration surface without native types. Its distribution spec runs the root package's real `prepack`, checks that the root tarball physically contains root `tree-sitter@0.25.1` and the internal package's isolated `tree-sitter@0.21.1` plus `tree-sitter-c@0.23.2` subtree, including licenses, platform prebuilds, and source fallbacks. It installs only that root tarball into a credential-free consumer with a fresh npm cache, verifies the installed lock and full npm tree, compiles a root API type consumer, and runs modern Go beside legacy C in one Node 24 process. The temporary tarball, cache, lockfile, dependencies, and consumer are always removed; no packed archive or dependency build is committed.
+
+`tree-sitter-legacy-install-consistency.spec.js` reproduces source edits after `npm ci` in unique isolated npm fixtures, using the real pretest/prepack hooks and a semantic change to the serializer. Each shipped file must cause a stale-payload rejection until ordinary `npm ci` refreshes the installed copy; real tar extraction proves that subsequent packs contain the edited bytes. The shared gate uses npm's source pack inventory, checks the complete installed payload except its dependency subtree, and compares bytes before runtime specs, lint, and packing. The consumer spec independently compares every bundled internal file with source. Its configuration regression uses only synthetic registry and credential settings, removes all inherited `npm_config_*` inputs and token/module-path variables case-insensitively, isolates both user and global configuration files, and checks npm's effective public registry/config/cache settings before the real fresh-cache installation.
+
+The consumer environment must assert npm's ordinary `install-links=false` default without setting it. Root `acceptDependencies` accepts the exact private package already supplied by the bundle while retaining the local `file:` installation source. The spec runs ordinary `npm install`, full `npm ls --all`, and then clean `npm ci` from the generated consumer lock, repeating dependency, simultaneous Go/C runtime, frozen-data, and TypeScript consumer checks. Bundled lock entries have neither links nor external resolutions; real paths must remain inside the consumer. The fresh cache must contain public dependency requests but no internal or retired adapter lookup. No consumer flags, project configuration, workspace fallback, or registry release of the private name are required.
+
+Task 015 has separate focused specs for immutable registry-derived discovery, role-specific diagnostics, safe transactional artifact sets, byte-coordinate provenance, exact tool discovery, and staged acceptance. Every caller-supplied ordered array in these contracts must be dense before element validation or iteration; each field independently retains its documented empty/non-empty rule. Error-path discovery uses controlled local fake executables. Positive runner coverage discovers and invokes real `php`, `ruby`, `node`, the lockfile-installed `tsc`, `javac`, `java`, and `go`; a missing or unsupported tool is a failure, never a skip. Runner assertions cover isolated directory cleanup, fixed `C.UTF-8` locale and UTC timezone, absence of undeclared ambient variables, exact argument arrays, and distinct invalid-input, missing, ambiguous, version, setup/materialization, launch, output-limit, nonzero-stage, signal, and timeout diagnostics. `ACCEPTANCE_LAUNCH_FAILURE` is restricted to explicit operating-system spawn errors. `ACCEPTANCE_OUTPUT_LIMIT` identifies a child that launched but exceeded the fixed 16 MiB capture limit; its diagnostic retains stage/tool/version and the deterministic limit evidence while omitting the oversized streams and child error object.
+
+`runAcceptanceStages` sets `LANG=C.UTF-8`, `LC_ALL=C.UTF-8`, and `TZ=UTC` for every child it owns, independently of the ambient locale. The aggregate repository suite also includes older direct native execution specs that invoke real tools with `execFile` and inherit the outer test process locale. Their Unicode round trips require a UTF-8 locale, so run the aggregate suite as `LANG=C.UTF-8 LC_ALL=C.UTF-8 npm test`.
+
+Task 016 extends the same fail-loud discovery and staged runner to configured Python 3. Native Python acceptance first invokes `python3 -m py_compile program.py`, then executes `python3 program.py` with deterministic `C.UTF-8`, `PYTHONUTF8=1`, and `PYTHONIOENCODING=utf-8`. Cross-language coverage generates and reparses Python from every original-five Tasks 001–004 fixture, then generates every original target from Python-derived IR, compiling where applicable and comparing exact runtime output. Missing Python is a test failure.
+
+Task 017 adds the distinct ordered `restore` stage and configured .NET SDK 10 discovery. C# acceptance generates exactly `Program.cs` and `Semantifold.csproj`, then invokes `dotnet restore Semantifold.csproj --source . --no-cache --force --disable-parallel --nologo`, `dotnet build Semantifold.csproj --configuration Release --no-restore --nologo --warnaserror`, and `dotnet exec bin/Release/net10.0/Semantifold.dll` as separate stages. Tests set the .NET telemetry/first-run/no-logo/invariant variables and an isolated absolute `NUGET_PACKAGES` path. They require exact exit/stdout and no warning output. Two fresh directories must produce identical ordered artifact text/mappings/provenance and identical `Semantifold.dll`, `.deps.json`, and `.runtimeconfig.json`; path-bearing restore bookkeeping under `obj/` is intentionally not compared. Missing .NET, restore dependency/network demand, warnings, SDK major mismatch, or nondeterministic compiler output fails without a skip.
+
+Task 024 adds configured Go 1.26.x discovery restricted to Linux/amd64; the patch release is read from the discovered toolchain rather than fixed by runtime tests. Acceptance obtains `GOROOT` from that exact `go`, requires the companion `gofmt` beside the configured or discovered executable to resolve to the same `GOROOT/bin/gofmt`, and checks `gofmt -d` before native execution. Each dependency-free generated module is run with `GOTOOLCHAIN=local`, `GOPROXY=off`, `GOSUMDB=off`, `GOVCS=off`, `CGO_ENABLED=0`, `GOENV=off`, `GOWORK=off`, `GOOS=linux`, `GOARCH=amd64`, and isolated HOME/GOPATH/build/module caches. The short-lived discovery fixture also writes exact `off` mode into its isolated Go telemetry directory before invoking the tool, then asserts that the mode file is the only telemetry state: Go 1.26 otherwise starts an unawaited uploader sidecar even in default `local` mode, which can recreate `upload/` while the fixture root is being removed. Cleanup errors remain fatal. Stages run offline `go build`, mandatory `go vet`, and `go run`, while compiler-only differential cases use the matching `go tool compile`. Tests require exact status/stdout, unchanged `go.mod`/`main.go`, no `go.sum`/`go.work`/vendor tree, and byte-identical binaries and output in two fresh directories. Missing, wrong-minor, wrong-architecture, mismatched-gofmt, download, cgo, workspace, vet, formatting, nondeterminism, telemetry leakage, or cleanup evidence fails without a skip.
+
+| Tool ID | Canonical command | Absolute-path override | Accepted canonical version |
+| --- | --- | --- | --- |
+| `php` | `php` | `SEMANTIFOLD_PHP` | PHP 8.x |
+| `ruby` | `ruby` | `SEMANTIFOLD_RUBY` | Ruby 3.x |
+| `node` | `node` | `SEMANTIFOLD_NODE` | Node 24.x |
+| `tsc` | `tsc` | `SEMANTIFOLD_TSC` | TypeScript 7.x |
+| `javac` | `javac` | `SEMANTIFOLD_JAVAC` | JDK 17–29 |
+| `java` | `java` | `SEMANTIFOLD_JAVA` | JRE 17–29 |
+| `python` | `python3` | `SEMANTIFOLD_PYTHON` | Python 3.x |
+| `dotnet` | `dotnet` | `SEMANTIFOLD_DOTNET` | .NET SDK 10.x |
+| `go` | `go` | `SEMANTIFOLD_GO` | Go 1.26.x, Linux/amd64 |
+| `clangpp` | `clang++` | `SEMANTIFOLD_CLANGPP` | Ubuntu Clang 21.1.8, x86_64-pc-linux-gnu, C++20/libstdc++ ABI1 |
+| `clang` | `clang` | `SEMANTIFOLD_CLANG` | Ubuntu Clang 21.1.8, x86_64-pc-linux-gnu |
+
+Canonical toolchain IDs must be non-empty primitive strings and are validated before property lookup or diagnostic formatting; malformed values normalize to `INVALID_TOOLCHAIN`, while an unknown valid string retains that string as its diagnostic identity. Discovery records the complete version output and first version line. Optional discovery and runner configuration defaults only when a field is `undefined`; explicit `null`, including in an environment entry, is invalid before lookup, setup, or launch. Explicit timeouts must be positive integers no greater than Node's exact timer ceiling of 2,147,483,647 milliseconds; larger values are rejected before lookup, setup, or launch rather than being clamped to one millisecond. Explicit environments must be ordinary or null-prototype records containing only enumerable own data properties with string or `undefined` values; inherited fields, accessors, symbol keys, collection/class instances, and other object shapes are rejected. Accepted environment values are synchronously detached before override lookup, setup, or launch. An override must be an executable absolute path resolving to a regular file; directories and other invalid file types report `TOOL_NOT_FOUND`. Canonical lookup examines the explicit PATH, accepts only executable regular files, resolves symlinks to exact paths, deduplicates aliases of the same executable, and rejects zero or multiple distinct matches. Version arguments and the accepted non-global/non-sticky version-policy expression are validated and detached before asynchronous executable lookup; the immutable discovery result retains the exact argument array that was executed, and caller mutation cannot change the pending version decision. TensorBuzz installs its pinned Node archive at `/usr/local/bin/node` and binds that exact executable through `SEMANTIFOLD_NODE`; a distinct host `/usr/bin/node` therefore cannot make the real-runtime lane ambiguous. This does not weaken canonical ambiguity detection when no override is configured. The runner inherits no ambient variables other than PATH unless the caller explicitly supplies them; locale and timezone are always normalized. Stage executable and version fields are copied and frozen before setup yields, embedded NUL is rejected before temporary-directory setup, and each completed stage returns a frozen copy of its executed argument array, so caller mutation cannot change execution or its audit record. On POSIX, version probes and acceptance stages spawn into one distinct owned process group: the configured deadline signals that exact group with `SIGTERM`, a fixed 250ms grace escalates the still-open group to `SIGKILL`, and a second fixed 250ms grace closes only the owned capture-pipe readers if an escaped descendant still retains their writers. The API then settles on the direct child's `close` event without claiming that an escaped process was terminated; captured output and timer-owned timeout classification remain unchanged. Platforms without this process-group contract fail with normalized unsupported launch/version diagnostics before spawning rather than claiming descendant cleanup. Timeout diagnostics derive from timer ownership rather than a subprocess signal, so spontaneous signals remain `TOOL_VERSION_FAILURE` or `ACCEPTANCE_SIGNAL` with their exact evidence. If a stage and isolated-directory cleanup both fail, the stage diagnostic remains primary with its exact stage, exit, signal, and bounded output fields; a bounded `ACCEPTANCE_CLEANUP_FAILURE` diagnostic is retained in its cause chain. Cleanup failure after successful stages remains `ACCEPTANCE_CLEANUP_FAILURE`.
+
+TERM-to-KILL escalation coverage does not assume a spawned fixture has already executed its first instruction. A focused spec at the internal subprocess owner creates an executor with instance-owned spawn observation that begins its test deadline only after the fixture emits process-boundary readiness; the production executor continues to register directly on the child's `spawn` event. The internal factory is absent from the package export map, and no Node builtin binding or shared test dependency is replaced. The fixture records readiness only after installing its TERM handler and capturing its Linux PID/start-time/process-group/session identity. Assertions retain the exact `SIGKILL` result and `SIGTERM` evidence and require both that identity and its complete owned group to disappear, while high-level discovery and acceptance coverage retains timeout-diagnostic translation without timing delays, retries, or relaxed signal expectations.
 
 Backend tests start from one parsed semantic module, generate every target, parse each generated program back to equivalent modeled meaning, and work in isolated `mkdtemp` directories. They invoke real commands and assert exact `5\n` output:
 
@@ -9,10 +113,15 @@ Backend tests start from one parsed semantic module, generate every target, pars
 - `node program.js`
 - local lockfile-installed `tsc program.ts`, then `node program.js`
 - `javac Main.java`, then `java -cp … Main`
+- `python3 -m py_compile program.py`, then `python3 program.py` with `PYTHONUTF8=1` and `PYTHONIOENCODING=utf-8`
+- C#'s separate local-only `dotnet restore`, `dotnet build --no-restore`, and `dotnet exec` commands above
+- matching-GOROOT `gofmt -d`, then isolated offline `go build`, `go vet`, and `go run`
 
 Cleanup runs in `finally`. A missing tool raises the process error and fails the spec; there are no skip branches or mocked compilers. Diagnostic specs cover unsupported syntax, missing JSDoc types, and backend capability rejection.
 
-Mapping specs assert all 25 frontend/backend combinations, exact/anchor/synthetic coverage without gaps, semantic-token roles, deterministic source/node/symbol IDs, exact `sourcesContent`, independently decoded v3 maps, rich/v3 composition, malformed caller metadata repair, legacy modules, caller-assembled multi-source IR, and deterministic JSON round trips. Transformation regressions cover same-kind function reorder, insertion, deletion, cloning, ID regeneration, source ownership, shared parameter, expression, and statement occurrences, and their distinct JSON-pointer node identities. UTF-16 cases cover BMP/astral text plus LF, canonical CRLF, and lone CR; escaped Babel identifiers cover declarations, references, and calls. Imported maps are checked against TraceMap on unmapped prefixes and lines; content-backed EOF anchors, including empty sources, round-trip as exact points, while mapped imports without source content are rejected before offset construction. Composition coverage includes derived intermediate provenance that resolves only to unmapped synthetic content, untraced intermediate EOF points whose registry source must remain available, and ordering where a later range remains traceable. Identity-index coverage checks direct, derived, and synthetic-related node/symbol associations for ordered deduplication. External directive coverage includes mixed parent-relative depths and resolution to the advertised map path. A bounded 50-to-200-function scaling test guards linear validation and indexed lookups; deterministic operation-count coverage guards Babel token indexing without wall-clock thresholds. A real `tsc` map is composed through generated TypeScript to original Ruby source. A separate mapped-artifact acceptance executes PHP, Ruby, Node, TypeScript through real `tsc`/Node, and Java through real `javac`/`java`; unavailable commands fail. SHA-256 characterization checks preserve every existing base/local generated byte across all source languages and targets.
+Mapping specs assert all 36 frontend/backend combinations, exact/anchor/synthetic coverage without gaps, semantic-token roles, deterministic source/node/symbol IDs, exact `sourcesContent`, independently decoded v3 maps, rich/v3 composition, malformed caller metadata repair, legacy modules, caller-assembled multi-source IR, and deterministic JSON round trips. Transformation regressions cover same-kind function reorder, insertion, deletion, cloning, ID regeneration, source ownership, shared parameter, expression, and statement occurrences, and their distinct JSON-pointer node identities. UTF-16 cases cover BMP/astral text plus LF, canonical CRLF, and lone CR; escaped Babel identifiers cover declarations, references, and calls. Imported maps are checked against TraceMap on unmapped prefixes and lines; content-backed EOF anchors, including empty sources, round-trip as exact points, while mapped imports without source content are rejected before offset construction. Composition coverage includes derived intermediate provenance that resolves only to unmapped synthetic content, untraced intermediate EOF points whose registry source must remain available, and ordering where a later range remains traceable. Identity-index coverage checks direct, derived, and synthetic-related node/symbol associations for ordered deduplication. External directive coverage includes mixed parent-relative depths and resolution to the advertised map path. A bounded 50-to-200-function scaling test guards linear validation and indexed lookups; deterministic operation-count coverage guards Babel token indexing without wall-clock thresholds. A real `tsc` map is composed through generated TypeScript to original Ruby source. Mapped-artifact acceptance executes PHP, Ruby, Node, TypeScript through real `tsc`/Node, Java through real `javac`/`java`, and Python through its configured interpreter after `py_compile`; unavailable commands fail. SHA-256 characterization checks preserve every existing base/local generated byte across all original source languages and targets.
+
+Task 016 adds mapped Python execution through the configured interpreter after `py_compile`; Task 017 adds the five C# profiles, C# generation into every original-five target, a rotated original-five source into C# per profile, and real deterministic .NET project execution. Task 024 adds five Go profiles into every original-five target/runtime, rotates one original-five source per profile into Go, and verifies native evaluation order, signed-64-bit boundaries, reparsing, and deterministic binary/output behavior. The original-six SHA-256 characterizations remain unchanged.
 
 Local-binding fixtures cover initialized mutable and immutable declarations, plain assignment, function/branch/entry sequencing, exact Ruby/PHP profile metadata, and source locations in all five frontends. Focused negatives cover every declaration/assignment exclusion, declaration-before-use, unresolved and duplicate bindings, callable-name and lexical no-shadowing, immutable parameters/locals, exact type preservation, and RHS-before-target validation. Ruby coverage rejects profile-like tag misspellings, places multibyte text before later function and local carrier comments, asserts their UTF-16 locations, and reparses generated Ruby with the same offset shift. Backend shape coverage applies missing, null, primitive, array, sparse, and unsupported-object values to recursive blocks, statement elements, declaration/assignment fields, and recursively nested call expressions, proving caller-supplied malformed IR receives the nearest located capability diagnostic before emitter dispatch. Backend runtime coverage regenerates and reparses the same binding/mutability semantics, then executes the result through all five real toolchains with exact `yes\n` output. Target-focused coverage rejects JavaScript/TypeScript `console` and Java `args`/`System` entry captures, JavaScript/TypeScript `console` and Java `System` function parameter/local print-receiver captures, and JavaScript/TypeScript `console` and Ruby `puts` function captures before emission. Real `tsc`, PHP, and Ruby coverage demonstrates their strict or reserved binding failures and requires located target-backend rejection for each proven declaration/assignment role. The PHP probe matrix separately covers automatic-global parameters, the narrower `GLOBALS`/`this` assignment restriction, legal function names, assignable non-`GLOBALS` superglobals, ordinary predefined variables, and exact-case near-collisions; other runtime coverage executes safe JavaScript behavior, Ruby `_10`, and TypeScript near-collisions.
 
@@ -22,4 +131,23 @@ Typed-operation fixtures cover all 18 closed unary/binary operations and explici
 
 Focused correction specs prove that unsafe numeric literals never enter the semantic tree; async/generator flags are rejected; Java receivers, unsupported arguments, and extra entry statements cannot disappear; Ruby annotations and parameter forms stay function-local; and PHP accepts only its precise strict-types declaration. Backend specs cover target reserved identifiers, Java's signed 32-bit range, and exact one-return branch cardinality. Repository-contract specs parse package metadata, Compose/TensorBuzz YAML, and the Dockerfile AST; they prove the custom ESLint plugin uses an immutable commit and TensorBuzz is the sole CI topology.
 
-Run `npx velocious-test spec/repository-contract.spec.js` for the focused repository contract and `npm test` for framework-native discovery of `.spec.js` files under `spec`. An empty selection is a framework failure and exits nonzero. Run `npm run lint`, `npm run typecheck`, and `npm run build` for source quality and declaration generation. Packaging validation is documented in the root README and `AGENTS.md`.
+Run `npx velocious-test spec/repository-contract.spec.js` for the focused repository contract and `npm test` for framework-native discovery of `.spec.js` files under `spec`. An empty selection is a framework failure and exits nonzero. Run `npm run lint`, `npm run typecheck`, and `npm run build` for source quality and declaration generation. Inspect `npm pack --dry-run --json`; the focused packed-consumer spec performs the stronger real root-tarball installation proof. Packaging validation is also documented in the root README and `AGENTS.md`.
+
+## Task019 CPP acceptance
+
+The CPP lane selects `/usr/bin/clang++-21` through `SEMANTIFOLD_CLANGPP`; C retains `SEMANTIFOLD_CLANG`. The development image pins libstdc++-15-dev 15.2.0-16ubuntu1 and already obtains clang++ from the exact clang-21 package. Noble TensorBuzz reuses the signed LLVM21 setup above, explicitly installs g++-13/libstdc++ headers, probes clang++ and libstdc++, and runs framework-native discovery of all ten languages. [The CPP profile](cpp.md) lists every compile/link, encoding, warning, exception/RTTI and sanitizer flag. Local validation uses explicit named spec files only; full-suite discovery belongs to TensorBuzz.
+
+```sh
+export SEMANTIFOLD_CLANG=/usr/bin/clang-21
+export SEMANTIFOLD_CLANGPP=/usr/bin/clang++-21
+npx velocious-test spec/cpp-native-execution.spec.js
+npx velocious-test spec/cpp-runtime-values.spec.js
+npx velocious-test spec/cpp-cross-language-acceptance.spec.js
+npx velocious-test spec/tree-sitter-legacy-packed-consumer.spec.js
+```
+
+Each native spec executes O0/O2, ordinary and ASan/UBSan/leak profiles, with byte/stdout/stderr/status assertions and fresh-directory cleanup. The original-five crossing spec invokes real PHP, Ruby, Node, tsc, javac and java; none may silently skip. The packed consumer proves ordinary install and clean npm ci with fresh cache, empty configs, default install-links=false, both bundled runtime paths, CPP and C frozen data, modern Go loading, and public CPP parse/generate/reparse/types.
+
+## Release-version package verification
+
+Repository delivery checks validate a stable SemVer root version and require both lockfile version entries to match it. Packed-consumer checks compare the packed and installed package versions with the source manifest, rather than pinning a historical release. When validating a release candidate, run `npx velocious-test spec/repository-contract.spec.js spec/tree-sitter-legacy-packed-consumer.spec.js` on the actual candidate version; all existing runtime isolation, ordinary install/ci, and strict TypeScript checks remain mandatory.

package/index.js

@@ -2,8 +2,18 @@
 
 import {parseSource} from "./src/frontends/index.js"
 import {generateArtifactSource, generateSource} from "./src/backends/index.js"
+import {createGeneratedArtifactSet as constructArtifactSet} from "./src/artifacts.js"
+import {SemantifoldDiagnostic} from "./src/diagnostic.js"
+import {languageRegistry} from "./src/language-registry.js"
 
-export {SemantifoldDiagnostic} from "./src/diagnostic.js"
+const artifactBackendRoles = new Set(["text", "binary", "application"])
+
+export {SemantifoldDiagnostic}
+export {languageCapabilities, supportedLanguages} from "./src/language-registry.js"
+export {createGeneratedArtifactSet} from "./src/artifacts.js"
+export {createByteMapping, parseByteMapping, stringifyByteMapping} from "./src/binary-mapping.js"
+export {canonicalToolchains, discoverCanonicalToolchain, discoverToolchain} from "./src/toolchains.js"
+export {runAcceptanceStages} from "./src/acceptance.js"
 export {getNodeProvenance, getSymbolProvenance} from "./src/semantic/provenance.js"
 export {primaryLocation} from "./src/semantic/provenance.js"
 export {
@@ -21,9 +31,6 @@ export {
   toSourceMapV3
 } from "./src/mapping.js"
 
-/** @type {readonly import("./src/semantic/types.js").SemanticLanguage[]} */
-export const supportedLanguages = Object.freeze(["php", "ruby", "javascript", "typescript", "java"])
-
 /**
  * Parses source into Semantifold's shared semantic representation.
  * @param {object} input - Parse request.
@@ -61,3 +68,101 @@ export function generate(input) {
 export function generateArtifact(input) {
   return generateArtifactSource(input)
 }
+
+/**
+ * Generates a deterministic artifact set for one semantic module.
+ * @param {object} input - Artifact-set generation request.
+ * @param {string} input.language - Registered target language or platform ID.
+ * @param {import("./src/semantic/types.js").SemanticModule} input.module - Semantic module.
+ * @param {"text" | "binary" | "application"} [input.role] - Requested backend artifact role.
+ * @param {string} [input.filename] - Safe output path for a text target.
+ * @param {"none" | "external" | "inline"} [input.mapDirective] - JavaScript-family map directive.
+ * @param {string} [input.sourceMapFilename] - External map filename.
+ * @param {{filename: string, content: string, language?: import("./src/semantic/types.js").SemanticLanguage}[]} [input.sources] - Original sources.
+ * @returns {import("./src/semantic/types.js").GeneratedArtifactSet} Complete generated set.
+ */
+export function generateArtifactSet(input) {
+  if (!input || typeof input != "object" || Array.isArray(input) || Object.getPrototypeOf(input) != Object.prototype) {
+    throw new SemantifoldDiagnostic({
+      code: "INVALID_ARTIFACT_SET",
+      language: "artifact",
+      message: "Artifact-set generation requires a request object."
+    })
+  }
+  const {filename, language, mapDirective, module, role = "text", sourceMapFilename, sources} = input
+
+  if (typeof language != "string" || language.length == 0) {
+    throw new SemantifoldDiagnostic({
+      code: "INVALID_ARTIFACT_SET",
+      language: "artifact",
+      message: "Artifact-set generation requires a non-empty string language or target ID."
+    })
+  }
+  if (typeof role != "string" || !artifactBackendRoles.has(role)) {
+    throw new SemantifoldDiagnostic({
+      code: "INVALID_ARTIFACT_SET",
+      language,
+      message: "Artifact backend role must be 'text', 'binary', or 'application'."
+    })
+  }
+
+  if (role == "text") {
+    const record = languageRegistry.record(language)
+
+    if (record.artifactMultiplicity == "multiple") {
+      const backend = languageRegistry.resolve(language, "textBackend", module?.location)
+
+      return constructArtifactSet(backend({filename, language, mapDirective, module, sourceMapFilename, sources}))
+    }
+    const artifact = generateArtifactSource({
+      filename,
+      language: /** @type {import("./src/semantic/types.js").SemanticLanguage} */ (language),
+      mapDirective,
+      module,
+      sourceMapFilename,
+      sources
+    })
+
+    /** @type {import("./src/semantic/types.js").GeneratedSetArtifact[]} */
+    const artifacts = [{
+      content: artifact.code,
+      contentKind: "text",
+      mediaType: /** @type {string} */ (record.mediaType),
+      ownership: "generated",
+      path: artifact.filename,
+      provenance: {
+        kind: "text",
+        mapping: artifact.mapping,
+        sourceMap: artifact.sourceMap,
+        sourceMapFilename: artifact.sourceMapFilename
+      },
+      role: "entry"
+    }]
+
+    if (mapDirective == "external") {
+      artifacts.push({
+        content: `${JSON.stringify(artifact.sourceMap)}\n`,
+        contentKind: "text",
+        mediaType: "application/json",
+        ownership: "generated",
+        path: artifact.sourceMapFilename,
+        provenance: {
+          kind: "synthetic",
+          reason: "Serialized Source Map v3 projection for the generated entry artifact.",
+          relatedOrigins: []
+        },
+        role: "mapping"
+      })
+    }
+
+    return constructArtifactSet({
+      artifacts,
+      target: language
+    })
+  }
+
+  const registryRole = role == "binary" ? "binaryBackend" : "applicationBackend"
+  const backend = languageRegistry.resolve(language, registryRole, module?.location)
+
+  return constructArtifactSet(backend({filename, language, module, sources}))
+}

package/node_modules/node-addon-api/LICENSE.md

@@ -0,0 +1,9 @@
+The MIT License (MIT)
+
+Copyright (c) 2017 [Node.js API collaborators](https://github.com/nodejs/node-addon-api#collaborators)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/node_modules/node-addon-api/README.md

@@ -0,0 +1,95 @@
+# **node-addon-api module**
+
+[![codecov](https://codecov.io/gh/nodejs/node-addon-api/branch/main/graph/badge.svg)](https://app.codecov.io/gh/nodejs/node-addon-api/tree/main)
+
+[![NPM](https://nodei.co/npm/node-addon-api.png?downloads=true&downloadRank=true)](https://nodei.co/npm/node-addon-api/) [![NPM](https://nodei.co/npm-dl/node-addon-api.png?months=6&height=1)](https://nodei.co/npm/node-addon-api/)
+
+This module contains  **header-only C++ wrapper classes** which simplify
+the use of the C based [Node-API](https://nodejs.org/dist/latest/docs/api/n-api.html)
+provided by Node.js when using C++. It provides a C++ object model
+and exception handling semantics with low overhead.
+
+- [API References](doc/README.md)
+- [Badges](#badges)
+- [Contributing](#contributing)
+- [License](#license)
+
+## API References
+
+API references are available in the [doc](doc/README.md) directory.
+
+<!-- x-release-please-start-version -->
+## Current version: 8.9.2
+<!-- x-release-please-end -->
+
+(See [CHANGELOG.md](CHANGELOG.md) for complete Changelog)
+
+node-addon-api is based on [Node-API](https://nodejs.org/api/n-api.html) and supports using different Node-API versions.
+This allows addons built with it to run with Node.js versions which support the targeted Node-API version.
+**However** the node-addon-api support model is to support only the active LTS Node.js versions. This means that
+every year there will be a new major which drops support for the Node.js LTS version which has gone out of service.
+
+The oldest Node.js version supported by the current version of node-addon-api is Node.js 18.x.
+
+## Badges
+
+The use of badges is recommended to indicate the minimum version of Node-API
+required for the module. This helps to determine which Node.js major versions are
+supported. Addon maintainers can consult the [Node-API support matrix][] to determine
+which Node.js versions provide a given Node-API version. The following badges are
+available:
+
+![Node-API v1 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v1%20Badge.svg)
+![Node-API v2 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v2%20Badge.svg)
+![Node-API v3 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v3%20Badge.svg)
+![Node-API v4 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v4%20Badge.svg)
+![Node-API v5 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v5%20Badge.svg)
+![Node-API v6 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v6%20Badge.svg)
+![Node-API v7 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v7%20Badge.svg)
+![Node-API v8 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v8%20Badge.svg)
+![Node-API v9 Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20v9%20Badge.svg)
+![Node-API Experimental Version Badge](https://github.com/nodejs/abi-stable-node/blob/doc/assets/Node-API%20Experimental%20Version%20Badge.svg)
+
+## Contributing
+
+We love contributions from the community to **node-addon-api**!
+See [CONTRIBUTING.md](CONTRIBUTING.md) for more details on our philosophy around extending this module.
+
+## Team members
+
+### Active
+
+| Name                | GitHub Link                                           |
+| ------------------- | ----------------------------------------------------- |
+| Anna Henningsen     | [addaleax](https://github.com/addaleax)               |
+| Chengzhong Wu       | [legendecas](https://github.com/legendecas)           |
+| Jack Xia            | [JckXia](https://github.com/JckXia)                   |
+| Kevin Eady          | [KevinEady](https://github.com/KevinEady)             |
+| Michael Dawson      | [mhdawson](https://github.com/mhdawson)               |
+| Nicola Del Gobbo    | [NickNaso](https://github.com/NickNaso)               |
+| Vladimir Morozov    | [vmoroz](https://github.com/vmoroz)                   |
+
+<details>
+
+<summary>Emeritus</summary>
+
+### Emeritus
+
+| Name                | GitHub Link                                           |
+| ------------------- | ----------------------------------------------------- |
+| Arunesh Chandra     | [aruneshchandra](https://github.com/aruneshchandra)   |
+| Benjamin Byholm     | [kkoopa](https://github.com/kkoopa)                   |
+| Gabriel Schulhof    | [gabrielschulhof](https://github.com/gabrielschulhof) |
+| Hitesh Kanwathirtha | [digitalinfinity](https://github.com/digitalinfinity) |
+| Jason Ginchereau    | [jasongin](https://github.com/jasongin)               |
+| Jim Schlight        | [jschlight](https://github.com/jschlight)             |
+| Sampson Gao         | [sampsongao](https://github.com/sampsongao)           |
+| Taylor Woll         | [boingoing](https://github.com/boingoing)             |
+
+</details>
+
+## License
+
+Licensed under [MIT](./LICENSE.md)
+
+[Node-API support matrix]: https://nodejs.org/dist/latest/docs/api/n-api.html#node-api-version-matrix

package/node_modules/node-addon-api/common.gypi

@@ -0,0 +1,21 @@
+{
+  'variables': {
+    'NAPI_VERSION%': "<!(node -p \"process.env.NAPI_VERSION || process.versions.napi\")",
+    'disable_deprecated': "<!(node -p \"process.env['npm_config_disable_deprecated']\")"
+  },
+  'conditions': [
+    ['NAPI_VERSION!=""', { 'defines': ['NAPI_VERSION=<@(NAPI_VERSION)'] } ],
+    ['NAPI_VERSION==2147483647', { 'defines': ['NAPI_EXPERIMENTAL', 'NODE_API_EXPERIMENTAL_NO_WARNING'] } ],
+    ['disable_deprecated=="true"', {
+      'defines': ['NODE_ADDON_API_DISABLE_DEPRECATED']
+    }],
+    ['OS=="mac"', {
+      'cflags+': ['-fvisibility=hidden'],
+      'xcode_settings': {
+        'OTHER_CFLAGS': ['-fvisibility=hidden']
+      }
+    }]
+  ],
+  'cflags': [ '-Werror', '-Wall', '-Wextra', '-Wpedantic', '-Wunused-parameter' ],
+  'cflags_cc': [ '-Werror', '-Wall', '-Wextra', '-Wpedantic', '-Wunused-parameter' ]
+}

package/node_modules/node-addon-api/except.gypi

@@ -0,0 +1,25 @@
+{
+  'defines': [ 'NAPI_CPP_EXCEPTIONS' ],
+  'cflags!': [ '-fno-exceptions' ],
+  'cflags_cc!': [ '-fno-exceptions' ],
+  'conditions': [
+    ["OS=='win'", {
+      "defines": [
+        "_HAS_EXCEPTIONS=1"
+      ],
+      "msvs_settings": {
+        "VCCLCompilerTool": {
+          "ExceptionHandling": 1,
+          'EnablePREfast': 'true',
+        },
+      },
+    }],
+    ["OS=='mac'", {
+      'xcode_settings': {
+        'GCC_ENABLE_CPP_EXCEPTIONS': 'YES',
+        'CLANG_CXX_LIBRARY': 'libc++',
+        'MACOSX_DEPLOYMENT_TARGET': '10.7',
+      },
+    }],
+  ],
+}

package/node_modules/node-addon-api/index.js

@@ -0,0 +1,14 @@
+const path = require('path');
+const { version } = require('./package.json');
+
+const includeDir = path.relative('.', __dirname);
+
+module.exports = {
+  include: `"${__dirname}"`, // deprecated, can be removed as part of 4.0.0
+  include_dir: includeDir,
+  gyp: path.join(includeDir, 'node_api.gyp:nothing'), // deprecated.
+  targets: path.join(includeDir, 'node_addon_api.gyp'),
+  version,
+  isNodeApiBuiltin: true,
+  needsFlag: false
+};