@greenpandastudios/aug-cli 0.20.1 → 0.21.0

package/docs/releasing.md

@@ -5,15 +5,20 @@ All first-party packages and the extension use one compiler-compatible version.
 ## Verify and create artifacts
 
 ```sh
-node scripts/version.mjs 0.20.1
+node scripts/version.mjs 0.21.0
 npm ci
 npm --prefix vscode ci
 node scripts/bootstrap-native.mjs
+node scripts/prepare-llvm-tools.mjs
+node scripts/prepare-llvm-maintainer.mjs
+node scripts/build-runtime-pack.mjs
+export AUG_LLVM_HOME="$PWD/.aug-build/llvm-tools"
 npm run version:check
 npm run check
 npm test
 npm run docs:check
 npm run docs:build
+node scripts/merge-compiler-packs.mjs .aug-build/release-packs
 npm run package:packages
 npm run test:packages -- --native
 npm run package:extension
@@ -22,15 +27,15 @@ node scripts/publish-release.mjs dist/release --verify-only
 node scripts/publish-extension.mjs dist/release --verify-only
 ```
 
-Update both changelogs and relevant guides, and commit regenerated docs. The final artifact step combines four installable npm tarballs, a VSIX, offline documentation, package metadata and SHA-256 checksums under `dist/release`. It excludes native caches, private credentials and application build output.
+The merge step requires the exact qualified producer archives and manifests for all three hosts under `.aug-build/release-packs`; `release.yml` obtains them before packaging. Update both changelogs and relevant guides, and commit regenerated docs. The final artifact step combines four installable npm tarballs, a VSIX, compiler packs, offline documentation, package metadata and SHA-256 checksums under `dist/release`. It excludes native caches, private credentials and application build output.
 
 ## GitHub release
 
 After verification and committing, create and push the version tag:
 
 ```sh
-git tag v0.20.1
-git push origin main v0.20.1
+git tag v0.21.0
+git push origin main v0.21.0
 ```
 
 `release.yml` validates the tag against every manifest, runs compiler/native/docs/package gates, and uploads artifacts to a **draft prerelease**. Review the draft and publish it in GitHub Releases. Publishing starts **Publish npm packages** and **Publish VS Code extension** automatically. Each workflow deploys the archives attached to that release. Changing an asset after review invalidates its checksum.
@@ -43,6 +48,22 @@ The manually published Marketplace `0.19.0` contains files that differ from the
 
 ## npm publication
 
+The LLVM preview release builds official pinned LLVM tools and the August runtime
+on macOS ARM64, Linux x86-64 and Linux ARM64 before packaging. Linux producers
+use the pinned Debian 12 maintainer image. `scripts/merge-compiler-packs.mjs`
+rejects missing, duplicate, stale or modified platform inputs, then records all
+exact compiler archive hashes in the CLI and bundled editor compiler. The GitHub
+release includes every selected archive.
+Runtime compilation remains a maintainer operation. Application installation
+downloads the reviewed pack and does not build LLVM or invoke Clang. CI runs the
+LLVM execution tests with its prepared toolchain; unsupported platforms retain
+explicit preview diagnostics. See [native packages](native-packages.md).
+Producer jobs require source breakpoint/variable inspection, actual LLVM ASan
+instrumentation with a failing negative control, runtime UBSan, and the frozen
+paired C/LLVM performance limits. Consumer jobs exercise an ordinary default
+LLVM starter and all four public native repositories without native tools,
+including frozen/offline locks, cleanup and relocated deployment bundles.
+
 The packages use the `@greenpandastudios` npm scope. Verify ownership and each package's trusted publisher before a release. GitHub tarballs can also be installed directly.
 
 Configure a trusted publisher for each of the four npm packages:
@@ -57,7 +78,9 @@ Use npm CLI 11.5.1+ and GitHub-hosted runners. The workflow grants `id-token: wr
 
 The job downloads the four reviewed tarballs, `packages.json` and `SHA256SUMS`. It checks every archive's SHA-256 and SHA-512 integrity, exact version and complete manifest against the tagged source before publishing anything. It checks all existing registry versions, then publishes standard library, web, crypto and CLI in that order with public access and the `next` dist tag. Lifecycle scripts are disabled. No rebuild or dependency installation runs in the npm deployment job.
 
-Retries skip a version only when its registry integrity matches the release archive. A registry failure or a different published archive stops deployment. Each new publication is checked against the registry before proceeding. The CLI is published last because its dependencies use exact matching versions. An interrupted run can leave some libraries published; retry the same release to finish. Retries leave already-published versions and their dist tags alone. This pipeline publishes preview packages to `next`; promoting a release to `latest` remains a separate maintainer decision.
+Retries skip a version only when its registry integrity matches the release archive. A registry failure or a different published archive stops deployment. Each new publication is checked against the registry before proceeding. npm may accept an upload several minutes before its public metadata becomes available. The publisher on main checks visibility at five-second intervals for about five minutes per package; it retries only missing-version responses and uploads each archive once. If that wait expires, let npm finish processing before retrying the same release. The `v0.20.1` publisher checks visibility immediately and may need a retry after each accepted upload; the bounded wait applies to future tags.
+
+The CLI is published last because its dependencies use exact matching versions. An interrupted run can leave some libraries published; retry the same release to finish. Retries leave already-published versions and their dist tags alone. This pipeline publishes preview packages to `next`; promoting a release to `latest` remains a separate maintainer decision.
 
 ## VS Code Marketplace
 
@@ -90,4 +113,56 @@ Enable GitHub Pages with **GitHub Actions** as its publishing source. `docs.yml`
 
 ## Current limits
 
-August is experimental. Native web/crypto bootstrap supports macOS and Linux; other platforms are unverified. npm and Marketplace deployment require owner-configured trust; a live upload has not yet verified the new Marketplace workflow. User libraries can use public Git repositories, local folders, or npm archives. Prebuilt native dependency releases and a stable external native adapter ABI remain future work. See [the gap ledger](web-library-gaps.md) and [performance assessment](performance.md).
+August is experimental. The LLVM/native candidate targets macOS 14+ ARM64 and GNU/Linux x86-64/ARM64 with glibc 2.36+; other platforms are unverified. npm and Marketplace deployment require owner-configured trust. The first `v0.20.1` Marketplace attempt failed during the VSCE 4.0.0 OIDC token exchange with an API-version error. The publisher now pins VSCE 4.0.1-1, whose [upstream fix](https://github.com/microsoft/vscode-vsce/blob/main/src/oidc.ts) supplies the API version and federated authorization scheme. Automated Marketplace publication remains unverified until a real release passes; the checked VSIX is available from GitHub Releases. User libraries use ordinary public Git repositories, local folders, or npm archives. The four native library repositories publish prebuilt artifacts; the 0.21.0 compiler release remains pending. Stabilizing the external adapter ABI is a 1.0 gate. See [the gap ledger](web-library-gaps.md) and [performance assessment](performance.md).
+## Native preview qualification
+
+Before publishing a compiler with native package support, build its LLVM pack
+before the npm archives and extension. `scripts/release-artifacts.mjs` checks
+that both shipped manifests pin that exact compiler archive. The release job
+runs `scripts/qualify-native-consumers.mjs --local-compiler`: it installs the npm
+archives, fetches the four native libraries from their public repositories and
+release URLs, then runs LLVM programs through URL imports and named aliases.
+Git, native compilers, and SDK paths are unavailable to those CLI processes.
+Frozen offline runs must preserve the locks and produce the same results.
+
+The release consumer jobs repeat this check on macOS 14 ARM64 and each Linux
+architecture. The macOS runner removes Xcode and Command Line Tools. Linux uses
+the pinned Node/Debian slim image with no compiler, Git or development headers.
+A draft is created only after all consumer jobs pass. A local result on a
+newer OS does not qualify the minimum OS. After release publication, omit
+`--local-compiler` to verify the compiler download too. Keep the resulting JSON
+report with release evidence; never commit artifact caches or generated binaries.
+
+Before library artifacts are public, contributors can pass
+`--candidate-libraries DIRECTORY --local-compiler` to the qualification script.
+`DIRECTORY` contains the four `aug-*` repository folders and their measured native
+archives. This mode installs the same CLI archives and checks each native file
+through the installed verifier, but reports local transport and does not claim
+repository URL/download acceptance. Public release gates omit this option.
+On a small test VM, `--discard-builds` removes verified deployment copies after
+their checks while keeping the source, locks, compiler outputs and JSON evidence.
+
+Each library candidate records the build commit and a complete input fingerprint:
+August source, the ABI descriptor, headers, native code, dependency locks and
+build recipes. Before adding release artifact pins, run
+`node native/verify-candidate.mjs PATH_TO_CANDIDATE_JSON` in the library repository.
+Run it again after updating the manifest. Only artifact metadata may change;
+changed binding or build inputs require a new candidate. Publish the exact tested
+archives without rebuilding them. `native/library-qualification.json` records
+the reviewed package tag, source commit and archive hash for each consumer host.
+A platform without reviewed pins fails qualification before any library download.
+
+The four library repositories keep a `release-candidates.json` record for the
+reviewed build run and source commit. Their tag workflow uses the shared release
+template under `native/templates` and the canonical assembly/publication scripts
+under `scripts`. It downloads that successful run, checks all three platform
+archives against the tagged manifest and source identity, then verifies the
+uploaded bytes before publishing. Retries accept an existing file only when its
+bytes match; they do not overwrite release assets. Keep the scripts in those
+repositories aligned when this maintainer protocol changes.
+
+To retry a library release after a publishing-tool correction, run its workflow
+on main and enter the existing version tag. The job checks out that immutable tag
+for source and manifest verification and uses the current maintainer publisher.
+It validates the tagged commit again before creating or changing a release.
+Partial uploads remain draft until the complete archive set passes byte checks.

package/docs/research/libtorch-native-qualification.md

@@ -0,0 +1,115 @@
+# CPU LibTorch native qualification
+
+This investigation tested the proposed August tensor adapter against the real official **LibTorch 2.14.1 macOS ARM64 distribution** on October 1, 2026. It establishes that the selected native library can implement the first package's bounded C interface. It does not establish August imports, LLVM lowering, automatic installation, macOS 14 execution, or clean-machine compatibility. Those remain integration gates in the [native/LLVM plan](../native-interop-llvm-plan.md).
+
+The probe ran on macOS 26.6.2 ARM64 with Apple Clang 21.0.0 from Xcode. Only ignored files under `.aug-build/native-libtorch-probe` were created for the experiment; no package repository or artifact was published. The adapter's error structure and counter exports are probe interfaces, not a frozen shipping ABI.
+
+## Verified upstream input
+
+The [official CPU archive index](https://download.pytorch.org/libtorch/cpu/) links to [libtorch-macos-arm64-2.14.1.zip](https://download.pytorch.org/libtorch/cpu/libtorch-macos-arm64-2.14.1.zip). An ordinary HTTPS GET with curl succeeded. A previous metadata request returning 403 therefore did not prevent downloading this distribution.
+
+| Input | Observed value |
+| --- | --- |
+| Archive length | 103,834,077 bytes |
+| Archive SHA-256 | `6ab4e92bed813981cb26434db0ea12aaae7ce7c548d031a5b25032a286de1b58` |
+| Embedded `build-version` | `2.14.1` |
+| Embedded `build-hash` | `5c4886908584029761b579af026dcfb627c84070` |
+| Header version | Major 2, minor 14, patch 1, ABI tag 0 |
+| Unpacked regular files | 10,172 |
+| Unpacked regular-file bytes | 427,709,993 |
+
+The [official tag API](https://api.github.com/repos/pytorch/pytorch/git/ref/tags/v2.14.1) resolves `v2.14.1` to the same complete commit as the embedded build hash. The archive digest was computed locally from bytes obtained through the official HTTPS endpoint. This is an observed content pin, not an independently verified upstream signature or publisher-supplied checksum.
+
+The installed Torch CMake configuration sets C++20. Its macOS configuration did not add a `_GLIBCXX_USE_CXX11_ABI` definition; that GNU libstdc++ choice does not describe this Apple libc++ build. Upstream recommends CMake but does not require it for consuming LibTorch. [Official installation guide](https://docs.pytorch.org/cppdocs/installing.html).
+
+## Adapter and independent native client
+
+The ignored `adapter.cpp` uses real `at::Tensor` objects. `from_f64` explicitly chooses CPU and float64, clones `at::from_blob` storage, and retains no caller memory. `add` calls `at::add`; `sum` calls `at::sum(...).item<double>()`. `values` makes contiguous storage and copies its doubles into an adapter-owned allocation. Tensor and array releases use their matching C++/C allocation families.
+
+Every fallible C export is `noexcept` and contains C++ exceptions. The probe distinguishes invalid pointer/length arguments, `c10::Error`, standard exceptions, and unknown exceptions with status codes. It clears outputs before calling the library. The actual mismatched-shape test produced a `c10::Error` converted to status 2, a nonempty bounded error message, and a null output handle. Exception text never crosses the C ABI as a borrowed C++ string.
+
+The initial adapter intentionally uses ATen inside its private implementation and is paired with the exact distribution above. The installed stable headers provide creation, clone, pointer access, and dispatcher support, but do not provide named `add` and `sum` helpers in `torch/csrc/stable/ops.h`. This experiment does not qualify a stable-dispatcher variant or claim C++ binary compatibility across Torch upgrades. August's exported C symbols can remain stable while maintainers rebuild the adapter for another qualified Torch version. [Pinned stable headers](https://github.com/pytorch/pytorch/blob/5c4886908584029761b579af026dcfb627c84070/torch/csrc/stable/ops.h), [upstream stable API scope](https://docs.pytorch.org/docs/main/notes/libtorch_stable_abi.html).
+
+`probe.c` is an independent C11 client of the header. It checks results from fixed inputs and uses no Torch headers or replacement implementation. Each of 1,000 cycles checks `[1, 2, 3] + [4, 5, 6] = [5, 7, 9]` and sum 21; mutation of the original input after creation does not affect the tensor. The output remains readable after its source tensor is destroyed. Empty tensors copy to a null/zero-length output and sum to zero. Null inputs, overflowing lengths, and incompatible shapes return failures. Live adapter tensor and buffer counters return to zero after each cycle, including the failed addition.
+
+The ordinary client and an AddressSanitizer/UndefinedBehaviorSanitizer build both exited successfully with this output:
+
+```text
+LibTorch 2.14.1: [5, 7, 9], sum 21; 1000 independent copy/empty/error/cleanup cycles passed
+```
+
+The sanitizer build instrumented the adapter and client; upstream LibTorch remained its prebuilt uninstrumented binary. Sanitizer stderr was empty. LeakSanitizer was disabled because this macOS experiment does not establish support for it. The counters measure adapter-owned objects and copies, not all upstream process allocations. They also do not establish that August's future ownership lowering invokes release correctly; that requires separate compiler integration tests on normal, return, and failure paths.
+
+### Maintainer commands actually used
+
+Run these from the August worktree after placing the prototype and official archive in the ignored probe directory. These are contributor experiment commands, not consumer installation instructions:
+
+```sh
+clang++ -std=c++20 -arch arm64 -mmacosx-version-min=14.0 \
+  -O2 -g -fvisibility=hidden -fPIC -dynamiclib \
+  -I .aug-build/native-libtorch-probe/libtorch/include \
+  -I .aug-build/native-libtorch-probe/libtorch/include/torch/csrc/api/include \
+  .aug-build/native-libtorch-probe/adapter.cpp \
+  -L .aug-build/native-libtorch-probe/libtorch/lib -ltorch_cpu -lc10 \
+  -Wl,-install_name,@rpath/libaug_torch.1.dylib \
+  -Wl,-rpath,@loader_path \
+  -o .aug-build/native-libtorch-probe/libaug_torch.1.dylib
+
+clang -std=c11 -arch arm64 -mmacosx-version-min=14.0 -O2 -g \
+  .aug-build/native-libtorch-probe/probe.c \
+  -L .aug-build/native-libtorch-probe/bundle/lib -laug_torch.1 \
+  -Wl,-rpath,@executable_path/lib \
+  -o .aug-build/native-libtorch-probe/bundle/probe
+```
+
+The bundle's `lib` directory contains the adapter and the three upstream libraries listed below. The sanitizer variation replaces `-O2` with `-O1`, adds `-fsanitize=address,undefined -fno-omit-frame-pointer` to both commands, and runs with `ASAN_OPTIONS=detect_leaks=0:halt_on_error=1` and `UBSAN_OPTIONS=halt_on_error=1`. These builds use the installed Apple SDK and are maintainer-toolchain evidence only.
+
+## Actual runtime closure and deployment floor
+
+All six dylibs shipped in the archive are thin ARM64 Mach-O images. `libc10`, `libshm`, `libtorch`, `libtorch_cpu`, and `libtorch_global_deps` declare macOS 14.0 and SDK 26.5. `libomp` declares macOS 11.0 and SDK 11.0. The experimental adapter declares macOS 14.0 and SDK 27.0. SDK values identify build inputs, not the declared deployment minimum. Actual execution on macOS 14 remains required.
+
+The ATen adapter links only `libtorch_cpu` and `libc10`. Loader inspection and execution with a copied deployment directory confirmed the following upstream closure:
+
+| Required redistributed file | Bytes | Unmodified archive SHA-256 |
+| --- | ---: | --- |
+| `libtorch_cpu.dylib` | 386,016,816 | `886a21732227bea91a64dcc355bdea98808fab5014a84891de2fe3a081744079` |
+| `libc10.dylib` | 1,121,632 | `332fdb431f20a9690f38be0768cab415584480c2ce31744153347778f85a04d8` |
+| `libomp.dylib` | 856,096 | `6256bee09e93c28d71c65711cc69224d69994c6965648b628b70a22772fe98d4` |
+
+For this narrow adapter, the tested loader does not load `libtorch`, `libshm`, or `libtorch_global_deps`. They need not be copied merely because they are in the upstream archive. Recompute the closure when expanding API coverage, switching implementation APIs, or upgrading Torch. The three required upstream files total 387,994,544 bytes before the adapter, notices, and archive compression; the source-package reader must not be used to install this binary payload.
+
+`libtorch_cpu` loads sibling `libc10` and `libomp` with `@loader_path`. Their rpaths are loader-relative. The adapter uses `@rpath/libaug_torch.1.dylib` as its install name, links Torch libraries by `@rpath`, and provides `@loader_path` as its own rpath. The application provides `@executable_path/lib`. The archive's `libomp` install-name identity is `/opt/llvm-openmp/lib/libomp.dylib`; the Torch dependency that actually loads it is `@loader_path/libomp.dylib`. The copied closure ran without `/opt/llvm-openmp`. If a shipping recipe normalizes this identity, it must record changed file hashes and restore valid ARM64 signing.
+
+System dependencies are supplied by macOS, rather than copied from an SDK. Strong load commands include Accelerate, CoreFoundation, libSystem, and libc++; Foundation, MetalPerformanceShaders, MetalPerformanceShadersGraph, Metal, IOKit, and libobjc appear as weak load commands in the inspected Torch images. Thus a CPU August API does not mean the upstream distribution contains no Metal dependencies. This adapter never requests a GPU device.
+
+The experiment copied the executable and closure to `relocated bundle`, started it from `/`, and cleared its environment except `PATH` and loader tracing. It passed all 1,000 cycles. `DYLD_PRINT_LIBRARIES` showed every non-system Torch/adapter library loading from the relocated directory, with no original LibTorch path, Xcode path, or `/opt` dependency. This tests relocation on the same machine, not another OS version or a clean VM.
+
+## Licenses and publication work still required
+
+The inspected ZIP did **not** include a license or notice file. August's release cannot treat the upstream archive as a complete redistributable notice bundle. The pinned [PyTorch LICENSE](https://github.com/pytorch/pytorch/blob/5c4886908584029761b579af026dcfb627c84070/LICENSE) permits binary redistribution with its stated conditions, including reproducing its notices. The pinned [NOTICE](https://github.com/pytorch/pytorch/blob/5c4886908584029761b579af026dcfb627c84070/NOTICE) and [package license inventory](https://github.com/pytorch/pytorch/blob/5c4886908584029761b579af026dcfb627c84070/pyproject.toml) show that the distribution contains multiple licensed components; the top-level BSD license is insufficient as a complete component inventory.
+
+The ignored probe retains the exact main LICENSE and NOTICE, with SHA-256 `bd018feef8825e88181c84eb7e3aa4eafb8f08a20d9fd6ef948569610c4a3e43` and `c2cc7bf0caec7652c2b460a8a470bea1677f241e4ab8e431df34cf17f5a9fec0`. The follow-up investigation below collected the pinned component/submodule texts and resolved OpenMP provenance. The upstream wheel metadata uses a broad third-party license glob; it cannot alone establish the complete compiled component graph.
+
+Publication and acceptance still require the shipping native error ABI, descriptor checks, August resource ownership tests, an independently checked archive manifest, release source/toolchain provenance, generated notices/SBOM, real repository imports, LLVM execution, and macOS 14 clean-machine/deployment tests. This native client is one prerequisite for those checks.
+
+### Follow-up: component notice collection
+
+The package workspace `aug-native-packages/aug-pytorch/native/licenses` now contains **43 primary license/notice files**, their hashes and source provenance in `provenance.json`, and a maintainer README. This collection is source material for the upcoming release archive; no binary assets were published during this investigation.
+
+The official [CPU wheel index](https://download.pytorch.org/whl/cpu/torch/) publishes SHA-256 `9cf3082d25560efb1eef921871595227c0cf305abb7761ca45b6b988ef6cdd45` for `torch-2.14.1-cp312-cp312-macosx_14_0_arm64.whl`. The downloaded wheel matched that digest. Its three required dylibs match the LibTorch archive **byte for byte**, and 22 collected notice files match its supplied texts. The pinned [LibTorch extraction program](https://github.com/pytorch/pytorch/blob/5c4886908584029761b579af026dcfb627c84070/.ci/libtorch/extract_libtorch_from_wheel.py) copies the wheel's libraries, headers, and CMake configuration, explaining the shared inputs and omission of distribution-level notices. The wheel's license glob itself misses filenames such as `LICENSE.md`, `LICENSE.MIT`, and SPDX license directories; those additional texts were retrieved from exact component revisions.
+
+Actual build configuration and symbol inspection confirmed SLEEF, NNPACK/QNNPACK, pthreadpool, KleidiAI, protobuf, ONNX, fmt, Kineto, Gloo, TensorPipe, FlatBuffers, PocketFFT, miniz, libuv, and libnop in `libtorch_cpu`, with CPUinfo and fmt in `libc10`. The collection also preserves their relevant kernel/header dependencies: FP16, FXdiv, PSIMD, gemmlowp, clog, uvw, JSON/Hedley, dynolog headers, protobuf UTF-8 validation, moodycamel, and conservative Perfetto/nested-fmt notices. Main component revisions come from PyTorch's exact gitlinks; TensorPipe's pinned gitlinks identify libuv 1.51.0 and libnop. Raw root/component notice bytes were checked against their Git blob identities where supplied, and all 43 saved file lengths/SHA-256 values were validated.
+
+OpenMP is **LLVM 21.1.8**, conda-forge package build `h4a912ad_0`. The pinned [macOS selection recipe](https://github.com/pytorch/pytorch/blob/5c4886908584029761b579af026dcfb627c84070/.ci/macwheel/install_libomp.sh) downloads that exact package and changes its install name before ad-hoc signing. The downloaded [conda package](https://conda.anaconda.org/conda-forge/osx-arm64/llvm-openmp-21.1.8-h4a912ad_0.conda) has SHA-256 `56bcd20a0a44ddd143b6ce605700fdf876bcf5c509adc50bf27e76673407a070`. Repeating those two documented normalization commands on an ignored copy reproduced the shipped `libomp` digest exactly. Its package metadata supplies the full LLVM Apache 2.0 license with LLVM exceptions, source archive digest `7ba3f2a8d8fda88be18a31d011e8195d3b7f87f9fa92b20c94cba2d7f65b0e3f`, and [feedstock revision](https://github.com/conda-forge/openmp-feedstock/tree/9a0e9859495a5a8c247cae11d866a0aac58d22f9). The `5.0.20140926` binary string is not its LLVM release version.
+
+The OpenMP recipe includes an ARM64 compiler-rt builtins linkage patch. Its pinned metadata declares compiler-rt 19.1.7 among build inputs; that [compiler-rt license](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.7/compiler-rt/LICENSE.TXT), the actual OpenMP package license, and recipe attribution are retained. The metadata and matched OpenMP binary establish the selected package and transformation; they do not identify every extracted builtins object without an upstream build link map.
+
+The nested libuv notices include its MIT text and complete FreeBSD tree BSD and ISC inet notices. This matters because the wheel's aggregate license expression omits ISC. JSON's actual Hedley header has MIT SPDX statements for Niels Lohmann and Evan Nemerson; those copyrights are retained separately. The repository's old `.reuse/dep5` mentions a different Hedley path and unrelated GPL test data, so it was not treated as the compiled header's license. GPU-only libraries, test frameworks, documentation assets, amalgamation tooling, and GPL-only JSON test notices were excluded from the runtime collection. The selected build defines `AT_USE_EIGEN_SPARSE()` as zero and uses system Accelerate BLAS/LAPACK; no Eigen template implementation symbols were observed.
+
+The remaining redistribution work is concrete: reconcile the **final** adapter and binary closure with a versioned SBOM/component manifest, resolve any extra embedded-header or compiler-runtime attribution revealed by that review, and verify that release and deployment archives preserve the collected texts and final modified-binary hashes. The supplied upstream archives lack an exhaustive native link map/SBOM. The collection therefore remains marked `redistribution-review-incomplete` and does not establish legal clearance. System frameworks and libc++/libSystem are declared OS requirements, rather than redistributed SDK or OS binaries.
+
+## Local evidence locations
+
+The experiment directory contains `aug_torch_probe.h`, `adapter.cpp`, `probe.c`, `inspect.mjs`, the original ZIP, extracted headers/libraries, the adapter, `bundle/probe`, `relocated bundle/probe`, `sanitized/probe`, upstream tag/license metadata, `runtime-inspection.json`, loader tracing in `relocation.dyld.txt`, and sanitizer output files. These files remain ignored and are not release artifacts.
+
+The adapter source SHA-256 is `ae4d6d18a3936d0ddbcaa99e6f0233552b4a6783a5a3d892e9e50cf81bbce7a3`; the C client is `8a76f252795dd57ae87d349b01bfc7a43d37527a74c6ce264ea7f98c8771fd55`. The local optimized adapter binary is `4dabf83b8bd7d2f52f15ef44dbe9c9ed7d0917ec777bb68b837fd64101a78650`; it includes local build/debug provenance and is not an immutable public release artifact.

package/docs/research/native-interop-llvm.md

@@ -0,0 +1,197 @@
+# Native interoperability and an LLVM backend
+
+## Debugger compatibility during implementation
+
+Platform qualification found that LLDB 14 and the macOS runner's LLDB can stop at August source lines but cannot import variables when a compile unit uses an unregistered vendor language code. The implemented metadata uses `DW_LANG_C99` for the real tagged C-compatible storage, with August filenames, symbols and compiler producer. Actual breakpoint tests inspect a parameter and a local, then resume execution. This compatibility choice enables the existing C type reader; it does not provide August expression evaluation or collection formatters. LLVM defines the compile-unit language field, and LLDB documents the additional language/type/runtime plugins needed for a distinct language. [LLVM metadata](https://llvm.org/docs/LangRef.html#dicompileunit), [LLDB language support](https://lldb.llvm.org/resources/addinglanguagesupport.html).
+
+Research checked October 1, 2026. This note supports August implementation work. It does not describe shipping native package support or an implemented LLVM backend. Upstream evidence, recommendations, and the limited local AOT experiment below are distinct. Selected LLVM binaries and SDK-input-free linkage were measured; a fresh macOS 14 consumer machine and the full August/native dependency closure remain unqualified.
+
+The recommended first target is Apple Silicon, macOS 14 or later. Ordinary August consumers receive prebuilt compiler tools, runtime, and native adapters. Maintainers build those artifacts with C, C++, Rust, CMake, and an appropriately licensed Apple SDK. Consumer compilation must produce a native executable without invoking a system compiler or requiring Xcode, Command Line Tools, or a separately installed LLVM.
+
+## Verified release candidates
+
+These exact versions are available in first party release sources. They are qualification candidates, rather than a claim that August has built, tested, or approved their artifacts. Record immutable source revisions, dependency locks, build configuration, and hashes of the actual output before publishing an August package.
+
+| Component | Candidate pin | Primary evidence and qualification boundary |
+| --- | --- | --- |
+| LLVM and LLD | `llvmorg-23.1.2` | The [release page](https://github.com/llvm/llvm-project/releases/tag/llvmorg-23.1.2) lists an Apple Silicon archive and verification instructions. The local experiment below verifies archive bytes, selected tool deployment metadata, and narrow linkage mechanics; the clean machine/full runtime gate remains open. |
+| CPU LibTorch | `2.14.1` | The [release](https://github.com/pytorch/pytorch/releases/tag/v2.14.1) and [official CPU archive index](https://download.pytorch.org/libtorch/cpu/) list `libtorch-macos-arm64-2.14.1.zip`. The archive bytes, dependency closure, and checksum still require inspection. |
+| SQLite | `3.53.4` | The [download page](https://www.sqlite.org/download.html) supplies `sqlite-amalgamation-3530400.zip` and its SHA3-256 digest. Its macOS downloads are command line tools, rather than the reusable August adapter library. |
+| zlib | `1.3.2` | The [upstream release page](https://zlib.net/) supplies source archives, digests, and signatures. Build the adapter and library in maintainer CI. |
+| Rust BLAKE3 crate | `blake3 = "=1.8.7"` | The [release](https://github.com/BLAKE3-team/BLAKE3/releases/tag/1.8.7) and [tagged Cargo manifest](https://github.com/BLAKE3-team/BLAKE3/blob/1.8.7/Cargo.toml) identify this version. Release 1.8.7 removes the `arrayref` dependency after the reported owner compromise. Preserve `Cargo.lock` and verify the dependency graph. |
+
+PyTorch 2.12 release notes document raising `MACOSX_DEPLOYMENT_TARGET` to 14.0, validating dylib minimum versions, and requiring C++20 in its build configuration. These released changes support a macOS 14 baseline for the newer LibTorch candidate. The final supported floor must be the highest requirement found in the compiler, runtime, adapter, and every dependency, then demonstrated on that OS. This research did not inspect the 2.14.1 Mach-O files: an archive metadata request returned HTTP 403. [PyTorch 2.12 release notes](https://github.com/pytorch/pytorch/releases/tag/v2.12.0)
+
+## LLVM integration and the platform ABI
+
+LLVM's object emission tutorial uses a target machine, target triple, and target data layout before emitting an object file. `llc` also supports direct object output. Emitting an object is distinct from linking an executable. [LLVM object emission tutorial](https://llvm.org/docs/tutorial/MyFirstLanguageFrontend/LangImpl08.html), [llc reference](https://llvm.org/docs/CommandGuide/llc.html)
+
+LLVM does not supply a common C++ or Rust ABI. LLVM's own FAQ states that frontends must emit platform specific IR to satisfy C ABIs. Apple's ARM64 ABI has differences from the generic AArch64 ABI, including caller extension of arguments smaller than 32 bits. [LLVM ABI FAQ](https://www.llvm.org/docs/FAQ.html#can-i-compile-c-or-c-code-to-platform-independent-llvm-bitcode), [Apple ARM64 ABI](https://developer.apple.com/documentation/xcode/writing-arm64-code-for-apple-platforms)
+
+**Recommendation:** begin with a small target specific C ABI: `int32_t` status values, fixed width integers, `float`/`double` where needed, pointer parameters, explicit lengths, caller supplied output buffers, and opaque owning handles. Exclude varargs, C++ classes, Rust layouts, aggregate arguments or returns by value, callbacks, and borrowed views that survive a call from the first supported surface. Compile ABI probe functions with the maintainer C/C++ toolchain and compare August calls against them; spelling an LLVM calling convention `ccc` is insufficient validation.
+
+For this target, distinguish the consumer package key `macos-arm64` from the LLVM target `arm64-apple-macosx14.0.0` and Rust target `aarch64-apple-darwin`. Rust documents Mach-O for this target and a default ARM64 minimum of macOS 11, with `MACOSX_DEPLOYMENT_TARGET` able to raise it. Set the maintainer Rust deployment target to the August baseline. Rust's lower default cannot lower a LibTorch dependency's requirement. [Rust Apple Darwin target](https://doc.rust-lang.org/rustc/platform-support/apple-darwin.html)
+
+Keep native objects and libraries as the distribution boundary. Do not require Rust's LLVM version to match August's LLVM merely to link ordinary native objects through the tested C ABI. Cross language LLVM bitcode or LTO introduces another compatibility contract and should be a later, independently tested feature.
+
+**Recommendation:** retain the TypeScript parser and checker, introduce one explicit typed lowering representation, and initially emit textual LLVM IR. Package `llvm-as`/verification, optimization, object generation, and Mach-O LLD behind one versioned toolchain interface. A small prebuilt LLVM helper can later consolidate those operations; avoid coupling Node.js to LLVM's C++ layout through an unmaintained binding. LLVM's C API stability is best effort, and its release policy preserves patch branch stability within stated limits. [LLVM API policy](https://llvm.org/docs/DeveloperPolicy.html#c-api-changes)
+
+Derive data layout from the selected target machine. Preserve August's existing evaluation order, integer behavior, checked error branches, ownership cleanup, and runtime semantics in explicit lowering. The LLVM language reference defines poison and undefined behavior, so overflow flags, `inbounds`, alignment, and alias attributes must follow proven August contracts. Start with conservative attributes and add optimizations only after semantic regression evidence. [LLVM language reference](https://llvm.org/docs/LangRef.html)
+
+LLVM documents selecting distribution components and target backends through CMake. Use that workflow to ship the tools or helper actually needed, their runtime dependencies, and notices, rather than the complete maintainer installation. [Building an LLVM distribution](https://llvm.org/docs/BuildingADistribution.html)
+
+## SDK independent AOT on a clean Mac
+
+Apple's Command Line Tools package includes the macOS SDK and compiler tools. Installing it is a workable maintainer path, but fails the consumer requirement in this plan. The current Xcode and Apple SDKs agreement restricts redistribution of Apple Software without express permission (§2.7), restricts permitted copies and separated SDK use (§2.5), and treats open source components according to their governing licenses. Do not assume that distributing LLVM authorizes copying Apple SDK `.tbd` files, SDK headers, or Apple's proprietary tools. These are license terms and a provenance constraint, not a legal opinion about independently authored metadata. [Apple command line tools FAQ](https://developer.apple.com/library/archive/technotes/tn2339/_index.html), [Xcode and SDK agreement](https://www.apple.com/legal/sla/docs/xcode.pdf)
+
+LLVM's Mach-O linker is an available independent linker implementation. Upstream LLD source selects `_main` as the default executable entry and emits `LC_MAIN`, `LC_LOAD_DYLINKER` for `/usr/lib/dyld`, and platform/version load commands. Its driver describes `crt1.o` as a support file for macOS 10.7 and older. This supports a modern entry path without copying legacy CRT objects; it does not prove the full August link recipe. The source inspected here is upstream `main`; repeat these checks against the pinned release. [Mach-O LLD](https://lld.llvm.org/MachO/index.html), [LLD driver](https://raw.githubusercontent.com/llvm/llvm-project/main/lld/MachO/Driver.cpp), [LLD writer](https://raw.githubusercontent.com/llvm/llvm-project/main/lld/MachO/Writer.cpp)
+
+LLD's ARM64 tests check `LC_CODE_SIGNATURE` both with default settings and with `-adhoc_codesign`. The linker can write the signature itself, without executing a consumer `/usr/bin/codesign`. Treat local generated executable signing separately from publisher signing/notarization of downloaded toolchain artifacts. [LLD ad hoc signing tests](https://raw.githubusercontent.com/llvm/llvm-project/main/lld/test/MachO/adhoc-codesign.s)
+
+Ordinary lazy bindings need `dyld_stub_binder`, as shown in LLD's stub helper implementation. Chained fixups follow another path; pin and test one initial recipe rather than relying on changing linker defaults. LLVM's minimal test SDK illustrates a TAPI stub format and binder spelling, but is a synthetic fixture with test install names and UUIDs. It establishes linker mechanics, not production runtime compatibility or the provenance of a distributable August platform file. [LLD binding implementation](https://raw.githubusercontent.com/llvm/llvm-project/main/lld/MachO/SyntheticSections.cpp), [LLVM test stub](https://raw.githubusercontent.com/llvm/llvm-project/main/lld/test/MachO/Inputs/MacOSX.sdk/usr/lib/libSystem.tbd)
+
+**Recommended AOT feasibility experiment:** build an August runtime dylib and each adapter on maintainer machines. Keep OS calls inside those prebuilt libraries. Generate August object code that references their small exported C surfaces. Supply a separately authored, provenance documented platform import description for the real system install name, with only the binder and any audited system symbols introduced by LLVM lowering. Apple documents `/usr/lib/libSystem.B.dylib` in real Mach-O import load commands. Verify the exact install name and exports on the oldest supported OS. Do not copy the synthetic LLVM fixture's `/usr/lib/libSystem.dylib` install name into a release assumption. [Apple dynamic library identification](https://developer.apple.com/forums/thread/736719)
+
+The experiment must inventory undefined object symbols after optimization and object generation. LLVM may introduce `memcpy`, `memmove`, `memset`, stack probes, or compiler builtins even when August source only calls its runtime. Resolve them through a measured platform import surface or prebuilt August implementations. A missing symbol must fail the build. Broad `-undefined dynamic_lookup`, an ambient SDK search, or extracting files from the user's dyld cache is not the proposed production recipe.
+
+Link with the bundled Mach-O linker, explicit architecture and platform minimum, explicit runtime/adapter paths, and controlled library search paths. The linker SDK version metadata must be defined by the tested artifact recipe; it must not imply that a consumer has an SDK installed. Retain two level namespace binding. Use relocatable install names and rpaths for bundled dependencies; Apple documents `@rpath`, `@loader_path`, and `@executable_path` relationships. [Apple run path libraries](https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html)
+
+This is a realistic bounded approach because all source requiring Apple headers is compiled by maintainers and consumers link only generated August objects against published artifacts. It remains a required feasibility gate: the research does not establish rights to every possible import description or prove that the proposed runtime dependency graph will link without SDK inputs. Require provenance review of the actual platform files and a fresh macOS 14 ARM64 VM before claiming a self sufficient AOT toolchain.
+
+Alternative approaches have different outcomes. User installed Apple Command Line Tools provide an SDK but violate the clean machine prerequisite. ORC can resolve symbols from prebuilt dynamic libraries for JIT execution, but a JIT run is not the requested native executable output. Remote compilation requires transmitting source and a service contract. Neither alternative should silently become the fallback for a failed local AOT gate. [LLVM ORC dynamic library resolution](https://llvm.org/docs/ORCv2.html#process-and-library-symbols)
+
+## CPU LibTorch adapter
+
+LibTorch distributions contain headers, libraries, and CMake configuration. The documented build uses `find_package(Torch)`, upstream compile flags, and upstream link libraries. For the selected modern release, use a C++20 maintainer build and the macOS libc++ environment used to qualify the archive. The documented GNU/Linux cxx11 configuration currently requires glibc 2.29 and GCC 9 or newer; those GNU requirements do not describe macOS. Do not relabel that distribution as musl or infer a Linux ARM64 artifact from an Apple Silicon archive. [LibTorch installation](https://docs.pytorch.org/cppdocs/installing.html)
+
+PyTorch now documents a limited stable ABI with low level C shims and `torch::stable` wrappers. Its stable C shim guarantee has a bounded compatibility window; C++ API policy and operator behavior are separate. `TORCH_TARGET_VERSION` limits which shim versions an extension can use. The stable API's tensor wrapper manages an opaque tensor handle, and its constructor from an existing handle takes ownership. Prefer this supported surface where it covers the initial operations; verify each operation in the pinned headers instead of treating all ATen or neural network APIs as stable. [LibTorch stable ABI](https://docs.pytorch.org/docs/main/notes/libtorch_stable_abi.html), [stable tensor and operators](https://docs.pytorch.org/cppdocs/api/stable/operators.html)
+
+**Recommendation:** publish a C++ adapter exporting August's own small C ABI. The accompanying implementation plan selects owned CPU float64 creation from copied arrays, addition, sum, copied result extraction, and release. Keep tensor classes, reference counting, and C++ allocators inside the adapter. Pair the adapter with its exact qualified LibTorch archive even if it internally uses the limited stable ABI. Shape inspection, matrix multiplication, and additional types can follow with their own error and ownership tests.
+
+Every export that calls C++ must catch library exceptions, `std::exception`, and unknown exceptions before returning to August. Mark the exported boundary `noexcept`, initialize output handles to null, return status plus operation local error text, and make the catch path avoid further allocation. No C++ exception may cross August frames. A fixed caller supplied error buffer and an explicit required/truncated length are preferable to a global last error string.
+
+Copy input data into library owned storage and copy output bytes into August owned storage initially. PyTorch exposes creation from a blob and APIs returning new tensor references; those are lifecycle obligations, not permission to retain an August buffer without a lifetime agreement. Defer zero copy views and custom deleter callbacks. CPU scope means the adapter explicitly selects a CPU device; the actual macOS archive may still depend on system frameworks or additional libraries, so inspect and package its complete dependency closure. [PyTorch C shim declarations](https://github.com/pytorch/pytorch/blob/main/torch/csrc/inductor/aoti_torch/c/shim.h)
+
+## SQLite and zlib adapters
+
+SQLite provides opaque connections and prepared statements through a mature C interface. Opening can produce a connection even on failure, which still needs closing. Binding with `SQLITE_TRANSIENT` copies bytes before the bind returns; `SQLITE_STATIC` imposes a longer caller lifetime. Returned column pointers can be invalidated by conversions, stepping, resetting, or finalizing. Copy text/blob column values before advancing. [SQLite open](https://www.sqlite.org/c3ref/open.html), [binding lifetime](https://www.sqlite.org/c3ref/bind_blob.html), [column lifetime](https://www.sqlite.org/c3ref/column_blob.html)
+
+`sqlite3_finalize` destroys a statement and can report the last execution error. `sqlite3_close` can return `SQLITE_BUSY` while objects remain open; `sqlite3_close_v2` instead defers actual disposal through its zombie connection behavior. **Recommendation:** model a connection owner and child statement scopes explicitly, finalize children before closing the connection, and report checked close/finalize errors. Choose a named close contract and test it; do not call the deferred disposal API and imply immediate closure. Start with parameter binding, iteration, copied values, transactions, and explicit error results. [SQLite finalize](https://www.sqlite.org/c3ref/finalize.html), [connection close](https://www.sqlite.org/c3ref/close.html)
+
+Build SQLite from its pinned amalgamation into the prebuilt adapter so the selected release and compile options do not depend on the host macOS SQLite. Record the thread mode and enabled features. Do not expose extension loading or permit connection/statement transfer between workers in the first package scope.
+
+zlib's pinned header specifies caller buffers for compression/decompression and the `deflateInit`/`deflateEnd`, `inflateInit`/`inflateEnd` lifecycle for streaming. `compressBound` gives an output bound for compression; decompression needs a caller limit. **Recommendation:** start with copied byte input and bounded one shot output, translate status codes, and reject length overflow. If streaming is later needed, wrap its state in an opaque owning handle and guarantee End on every cleanup path. Keep `z_stream` layout inside the C adapter. Reject malformed/truncated data and output exceeding the caller's cap. [zlib 1.3.2 header](https://github.com/madler/zlib/blob/v1.3.2/zlib.h)
+
+## Rust crate adapter
+
+Use the Rust `blake3` crate through a new Rust adapter crate, built with Cargo; using upstream BLAKE3's separate C implementation would not exercise Rust interoperability. `Hasher` supplies incremental update and finalization. Its tagged build script enables C NEON on ordinary little endian AArch64 unless disabled, so a default crate build is not a purely Rust implementation. The `pure` feature prevents that C path, but the tagged manifest labels it among unstable testing features. Pin 1.8.7 exactly, keep `std` plus `pure`, avoid explicitly enabling `neon`, and inspect the build log and object provenance. [BLAKE3 Hasher](https://docs.rs/blake3/1.8.7/blake3/struct.Hasher.html), [tagged build script](https://raw.githubusercontent.com/BLAKE3-team/BLAKE3/1.8.7/build.rs), [feature contract](https://github.com/BLAKE3-team/BLAKE3/blob/1.8.7/Cargo.toml)
+
+Rust's `cdylib` is intended for dynamic libraries loaded from other languages; `staticlib` is another foreign linking output. Export named `extern "C"` functions and use an opaque handle. `repr(C)` controls layout of deliberately shared data; ordinary Rust structs, references, `Vec`, `String`, and trait objects must stay behind the boundary. [Rust linkage](https://doc.rust-lang.org/reference/linkage.html), [Rust FFI](https://doc.rust-lang.org/nomicon/ffi.html), [Rust type layout](https://doc.rust-lang.org/reference/type-layout.html)
+
+**Recommendation:** begin with the one-shot digest API chosen by the accompanying implementation plan, including copied Rust-owned output and Rust's matching deallocator. Later expose create, update with borrowed bytes valid only during the call, finalize, and release if an incremental API is needed. Allocate a hasher in Rust and destroy it in Rust exactly once; define whether finalize preserves or consumes it. Reject null/nonzero length inputs, invalid output capacity, and lengths outside the target's representable slice range before constructing a Rust slice. Handle zero length input without constructing a slice from null. Keep any handles confined to their owning August scope and initial runtime thread.
+
+Prevent Rust unwinding across the boundary. Build the adapter with an explicit unwind panic policy and catch panics inside exported operations, translating caught panics to a checked adapter failure with a defined unusable-handle policy. `catch_unwind` only catches unwinding panics; aborting panics and process allocation failure are not recoverable guarantees. Do not catch foreign C++ exceptions through Rust. Test a deliberate adapter panic and continued August error handling in a subprocess. [Rust unwind contract](https://doc.rust-lang.org/nomicon/ffi.html#ffi-and-unwinding), [catch_unwind limits](https://doc.rust-lang.org/std/panic/fn.catch_unwind.html)
+
+## Artifact and license obligations
+
+LLVM's license is Apache 2.0 with LLVM exceptions; inspect included components and preserve required notices. PyTorch's license permits source and binary redistribution subject to its stated conditions, including binary notices. Its top level license alone does not enumerate every dependency in an archive. SQLite states that its core code is dedicated to the public domain, with separate practical considerations on its copyright page. zlib's license permits use and redistribution subject to origin, alteration, and notice conditions. BLAKE3's tagged Cargo manifest lists a choice among CC0, Apache 2.0, and Apache 2.0 with LLVM exception; choose and retain the actual applicable license text. [LLVM license](https://llvm.org/LICENSE.txt), [PyTorch license](https://github.com/pytorch/pytorch/blob/v2.14.1/LICENSE), [SQLite copyright](https://www.sqlite.org/copyright.html), [zlib license](https://zlib.net/zlib_license.html), [BLAKE3 manifest](https://github.com/BLAKE3-team/BLAKE3/blob/1.8.7/Cargo.toml)
+
+**Recommendation:** each independently released August native package supplies its August sources, public C header, ABI revision, per target artifacts, build recipe, upstream source identities, dependency license inventory, notices, and checksums. The toolchain artifact supplies the native compiler tools/runtime and its independently authored platform metadata. Consumers download only qualified artifacts, verify before extraction, and restore exact selections from their lock. Source builds are an explicit maintainer/developer workflow. Preserve the repository's existing security and package source boundaries; native code is a real additional execution capability and cannot acquire new implicit trust through a Git import.
+
+Begin with macOS ARM64 only. Later GNU/Linux packages require separate `x86_64-unknown-linux-gnu` and `aarch64-unknown-linux-gnu` qualification, glibc floors, C++ runtime ABI/dependency checks, and a distributable sysroot/startup recipe. Musl and Windows need separate artifacts and ABI decisions. A target triple by itself does not encode all these requirements.
+
+## Required implementation evidence
+
+The feasibility work must first produce and execute a small Mach-O binary using only packaged LLVM/LLD, the prebuilt runtime, and reviewed platform metadata on a fresh macOS 14 ARM64 machine. Check entry behavior, argv/env access, process exit, runtime initialization/finalization, symbol bindings, relocation paths, and signatures. Inspect the actual pinned linker output with bundled `llvm-otool` or `llvm-objdump`; maintainers may additionally use Apple tools for qualification. [LLVM Mach-O inspection](https://llvm.org/docs/CommandGuide/llvm-otool.html)
+
+Then qualify each adapter with ABI probes and adversarial resource tests: large lengths, embedded null bytes, failure after partial construction, use after close rejection, double close rejection, output limits, native errors, C++ throws, Rust panic, and scope cleanup. Inspect every distributed dylib's minimum OS and dependency load commands. Run the installed CLI's check/run/build/test/spec workflow and all four library examples with compiler and SDK tools absent, then repeat from the verified offline cache. Build output must remain executable after copying its supported deployment bundle to another clean machine.
+
+Only after those gates and the existing August semantic suite agree should LLVM become the default backend. Remove the C generation path after parity, diagnostic/debug information, package tests, and clean machine AOT are complete. The required public guides must then explain the actual consumer and maintainer prerequisites and tested platform floor. Until implementation and qualification succeed, these capabilities remain proposed scope.
+
+## Measured SDK-input-free AOT experiment
+
+On October 1, 2026, an ignored prototype under `.aug-build/native-llvm-probe` generated and ran narrow Mach-O programs on this development host: **macOS 26.6.2 ARM64**, with Xcode installed. The compiler/linker subprocess environment set `PATH`, `DEVELOPER_DIR`, and `SDKROOT` to `/nonexistent`. Programs were handwritten LLVM IR; every linker input was an explicit object, a prototype dylib, or the independently authored stub below. No Clang, Apple `ld`, `xcrun`, `codesign`, SDK headers, SDK `.tbd` files, or startup objects participated. This demonstrates SDK-input-free linkage mechanics on the present host; it does **not** satisfy the fresh macOS 14 VM, complete August runtime, package distribution, or four-library acceptance gate.
+
+### Verified official tools
+
+The downloaded [official ARM64 zstd archive](https://github.com/llvm/llvm-project/releases/download/llvmorg-23.1.2/LLVM-23.1.2-macOS-ARM64.tar.zst) was **873,761,429 bytes** with SHA-256 **`3da0e91b5dfe3a5ec795ad2be79b3f5e6f28c8b23edcd3847fad7742b25e0507`**, matching the [official release API](https://api.github.com/repos/llvm/llvm-project/releases/tags/llvmorg-23.1.2) and [expanded asset list](https://github.com/llvm/llvm-project/releases/expanded_assets/llvmorg-23.1.2). The alternative [xz archive](https://github.com/llvm/llvm-project/releases/download/llvmorg-23.1.2/LLVM-23.1.2-macOS-ARM64.tar.xz) is 1,569,989,604 bytes, SHA-256 `d7c26fc6177e42842e2d1ffaad31aec057c56a924392b1a23d830abe2c5d53b1`. The release lists no smaller official macOS tool archive. Hash verification was performed before selective extraction and execution; GPG/attestation signature verification was not performed in this experiment.
+
+The release also supplies `.sig` and `.jsonl` sidecars. Metadata inspection of the small attestation identified source commit `85ac560262434c9ccfc0c183ec22d4138ed647fb` and the release workflow. The upstream workflow uses a 1 GiB zstd compression window; extraction needs a compatible decompressor. The probe used Node 24.18.0's zstd stream with maximum window log 30 and a bounded selected-file tar reader. [Release verification instructions](https://github.com/llvm/llvm-project/releases/tag/llvmorg-23.1.2), [release builder](https://github.com/llvm/llvm-project/blob/llvmorg-23.1.2/.github/workflows/release-binaries.yml)
+
+Usable local tools are at `/Users/august/.codex/worktrees/release-publishing/augscript/.aug-build/native-llvm-probe/llvm23/bin/`. `ld64.lld` is a symlink to `lld`. The binaries report LLVM/LLD 23.1.2, and LLD reports the source revision above. Actual Mach-O inspection found:
+
+| Selected tool | File bytes | SHA-256 | Imported OS libraries |
+| --- | ---: | --- | --- |
+| `llc` | 135,003,744 | `7a9ff3ffea3ed5f3e4c2e6603b5792446702cfcb46d978e80bc6cd1678f192eb` | `/usr/lib/libSystem.B.dylib`, `/usr/lib/libz.1.dylib`, `/usr/lib/libc++.1.dylib` |
+| `lld` | 146,098,192 | `87de299f2482f07991579207d3673694f5e152cfcd41e9c8c7c864fc91d1398e` | Same, plus `/usr/lib/libxml2.2.dylib` |
+| `llvm-objdump` | 40,147,456 | `2ecce60ac491cb4840abe75f64cff3a56c80daf81f4aca2b2dcab859471b453f` | Same as `llc` |
+
+All three have `LC_BUILD_VERSION` **minos 14.0, sdk 14.5** and `LC_RPATH @loader_path/../lib`. Their import load commands name only OS libraries; none names a bundled LLVM dylib. Therefore the smallest measured stock compile/link selection is `llc` plus `lld`/`ld64.lld`: **281,101,936 regular-file bytes**, before compression and notices. `llvm-objdump` is useful for inspection but is not required by the compile/link recipe. This does not establish a compressed August artifact size or prove the imported OS symbol versions on macOS 14.
+
+For a smaller owned distribution, LLVM documents `LLVM_TARGETS_TO_BUILD=AArch64`, project `lld`, selected distribution components, and optional compression dependencies. Stock LLD includes multiple linker flavors. A custom driver can instead link the necessary LLVM code-generation components with `lldMachO`/`lldCommon`; the public driver API permits selecting only the Mach-O driver. Set an explicit maintainer deployment target of 14.0 and qualify the resulting host binary. Its size is not measured here. [LLVM distribution guide](https://llvm.org/docs/BuildingADistribution.html), [CMake options](https://llvm.org/docs/CMake.html), [pinned LLD tool build](https://github.com/llvm/llvm-project/blob/llvmorg-23.1.2/lld/tools/lld/CMakeLists.txt), [pinned driver API](https://github.com/llvm/llvm-project/blob/llvmorg-23.1.2/lld/include/lld/Common/Driver.h)
+
+### Inputs and exact successful linkage
+
+The independently typed prototype `libSystem.tbd` was:
+
+```yaml
+--- !tapi-tbd-v3
+archs: [ arm64 ]
+platform: macosx
+install-name: /usr/lib/libSystem.B.dylib
+current-version: 1.0.0
+compatibility-version: 1.0.0
+exports:
+  - archs: [ arm64 ]
+    symbols: [ _puts, dyld_stub_binder ]
+...
+```
+
+The 1.0.0 fields are prototype link requirements, not a measurement of the actual OS library's current version. This file was not copied from an Apple SDK. It is neither a general SDK nor the final August platform surface. Its provenance, symbol availability, and version requirements still need review for a distributable artifact. The binder's spelling deliberately has no leading underscore.
+
+The direct program's entire IR was:
+
+```text
+target triple = "arm64-apple-macosx14.0.0"
+@message = private unnamed_addr constant [27 x i8] c"August LLVM SDK-free probe\00"
+declare i32 @puts(ptr)
+define i32 @main(i32 %argc, ptr %argv, ptr %envp) {
+entry:
+  %written = call i32 @puts(ptr @message)
+  %ok = icmp sge i32 %written, 0
+  %status = select i1 %ok, i32 0, i32 1
+  ret i32 %status
+}
+```
+
+The following are the successful commands, with paths shortened to the probe directory and tool directory. `-Z` removes standard library/framework search directories; `-t` recorded only the explicit inputs. `-fixup_chains` selects the tested binding form, while `-adhoc_codesign` makes signing explicit. The SDK metadata argument is an explicit recipe value; no SDK is read.
+
+```sh
+llc -mtriple=arm64-apple-macosx14.0.0 -filetype=obj -O0 \
+  -relocation-model=pic direct.ll -o direct.o
+ld64.lld -arch arm64 -platform_version macos 14.0 14.0 \
+  -Z -fixup_chains -adhoc_codesign -t -e _main \
+  direct.o libSystem.tbd -o direct
+```
+
+The program printed `August LLVM SDK-free probe` and exited 0. A second IR module implemented a stand-in runtime function `aug_probe_runtime_v1(i32, ptr, ptr)`, a `puts` call, and an `llvm.global_ctors` initializer. The initializer stored 42; the runtime function checked that state, `argc >= 1`, and nonnull `argv`/`envp` before printing. A main module only called that runtime function and returned its status. Both objects used the same `llc` options. The successful dynamic link commands were:
+
+```sh
+ld64.lld -arch arm64 -platform_version macos 14.0 14.0 \
+  -Z -fixup_chains -adhoc_codesign -t -dylib \
+  -install_name @rpath/libaug_probe.1.dylib \
+  -exported_symbol _aug_probe_runtime_v1 \
+  runtime.o libSystem.tbd -o bundle/lib/libaug_probe.1.dylib
+ld64.lld -arch arm64 -platform_version macos 14.0 14.0 \
+  -Z -fixup_chains -adhoc_codesign -t -e _main \
+  -rpath @executable_path/lib main.o \
+  bundle/lib/libaug_probe.1.dylib libSystem.tbd -o bundle/program
+```
+
+Both `bundle/program entry-argument` and a copied `relocated/program moved-entry-argument` printed `August LLVM prebuilt runtime probe` and exited 0. Inspection with the bundled `llvm-objdump --macho --private-headers` found `LC_MAIN`, `/usr/lib/dyld`, the intended relative runtime import/rpath, `/usr/lib/libSystem.B.dylib`, chained fixups, `LC_CODE_SIGNATURE`, two-level namespace flags, and minos 14.0 in generated binaries. The dylib's initializer appeared in `__init_offsets`. No `crt1.o` or application C bridge was used. The same narrow experiment initially passed with installed Rust LLVM/LLD 22.1.8; the results above supersede that preliminary tool choice.
+
+### What the result enables and what remains
+
+The result supports proceeding with the approved maintainer-built dynamic runtime design: compile the real C runtime/private thunks and C/C++/Rust adapters with licensed maintainer tools, give distributed libraries relative install names, and link consumer-generated LLVM objects with the bundled driver/LLD and the reviewed import surface. The consumer-facing `_main` must call explicit runtime initialize/run/shutdown services; the probe verifies loader constructors, not August runtime initialization or finalization. Preserve matching C callback/private pointer ABI work from the approved plan.
+
+Remaining release blockers are a **fresh macOS 14 ARM64 VM with developer tools absent**, actual August runtime and all four native dependency closures, introduced symbol/builtin inventory under release optimization, moved full deployment bundles, publisher signing/notarization, platform file provenance/rights review, and the complete frontend/IR/error/ownership/regression gates. A metadata minimum of 14.0 and execution on macOS 26 do not close the oldest-OS runtime gate. The ignored prototype is a reproducible local experiment, not an artifact published by August.

package/docs/research/qualification-methods.md

@@ -0,0 +1,15 @@
+# Performance and safety qualification methods
+
+Reviewed October 2, 2026. These sources inform the suite's design; August's programs and oracles are original.
+
+LLVM's test suite checks reference outputs and collects execution, compilation and code-size measurements. It distinguishes correctness tests from programs useful for benchmarking. August adopts that separation: every timed sample must produce the required result, while the safety gyms also exercise rejection, cleanup and deliberate faults. This is an architectural choice based on the [LLVM test-suite guide](https://llvm.org/docs/TestSuiteGuide.html#structure), not a claim that August runs LLVM's external benchmark corpus.
+
+The [AddressSanitizer documentation](https://clang.llvm.org/docs/AddressSanitizer.html#introduction) describes memory instrumentation and its supported checks. August instruments its emitted LLVM program as well as the core runtime and requires a failing out-of-bounds-store control. An uninstrumented foreign library is outside that check's internal coverage. The current common gate disables leak detection; native resource counters and drop traces are separate observations, not an equivalent whole-process leak proof.
+
+[UBSan's documentation](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#available-checks) lists checks and exclusions. August applies address/undefined instrumentation to the C runtime. This does not constitute UBSan coverage of every LLVM operation, third-party binary or intentional wrapping integer operation. Floating-point infinity and NaN behavior is checked by result oracles without fast-math.
+
+Measurements use rotated implementation order, fresh executable processes, warmups, raw samples and independent output checks. Batch clients now run outside the compiler process, so the measurement client does not retain compiler allocations. More samples improve the basis for the median but cannot remove host scheduling, thermal or deployment differences. The frozen LLVM migration thresholds remain unchanged.
+
+The extended C references expose their differences: concrete C values, explicit cleanup, a dense ordered map with linear searches, and sequential calls for the task case. They establish useful scoped baselines rather than identical runtime machinery. August's C backend is measured separately on the same August source to isolate migration regressions. No aggregate “as fast as C” or production-safety score follows from these workloads.
+
+The generated gyms save seeds, input bounds, source units, expected results and deliberate behavioral mutants. Invalid domains and infrastructure failures cannot count as passing tests or detected mutations. Independent behavioral requirements remain necessary for application correctness; neither the generated spec nor compiler acceptance is an acceptance oracle.

package/docs/roadmap.md

@@ -7,6 +7,7 @@ August's goal is a language that stays understandable as a codebase grows: local
 | 0.19: public preview | Runnable CLI, editor, source packages, wiki, same-file tests, specs, measured benchmarks, and starter projects. | Current CI and [example projects](examples/index.md); experimental label remains. |
 | 0.x: language and ownership | Define and check mutable captures, shared object lifetime, cleanup, delayed errors, and task cancellation for the documented cooperative task model; keep both block styles and specs in sync. | **Open:** the [ownership and task conformance suite](language-conformance.md) covers moves, aliases, branch and nested-scope joins, initialization, return/error exits, collection pressure, and public `Task<T>` errors. [Generated robustness checks](production-readiness.md#what-is-measured-and-verified) now exercise parser recovery, native results, and the core runtime under sanitizers. Adversarial review found additional capture and lifetime gaps after the first green CI run; independent review and wider runtime coverage remain before this gate can close. |
 | 0.x: portable native stack | Reproducible Linux and macOS web/crypto builds; build/run container images for full apps; platform-specific CI. | Versioned dependency manifests, license notices, native integration tests, and matching package builds on each target. |
+| 0.x: LLVM and native packages | Compile checked August execution IR through LLVM and make C, C++ and Rust libraries ordinary repository dependencies. | [Four separate libraries](native-packages.md) publish real CPU LibTorch, SQLite, zlib and Rust BLAKE3 archives. macOS 14 and clean Debian 12 consumers pass public imports, locking, offline execution, relocation and cleanup. The 0.21.0 candidate selects LLVM by default; repeat debugger, sanitizer, performance and installed-package gates on all three hosts before publication. See [current evidence](native-implementation.md). |
 | 0.x: developer distribution | Publish version-matched npm packages and VS Code extension; project bootstrap with `npx`; package compatibility and reproducible releases. | The [0.19.0 npm packages and core `npx` workflow](packages.md#npm-registry) are verified. The gate also requires clean installed-package tests, signed/versioned artifacts, release instructions, and a supported upgrade path. |
 | 1.0.0 | Freeze the supported language and package/native ABI; publish migration policy and production support matrix. | All preceding gates pass in CI, examples and specs regenerate deterministically, dependency audit and security review pass, and known gaps are classified explicitly. |
 

package/docs/safety-gyms.md

@@ -0,0 +1,41 @@
+# Safety gyms
+
+A gym is a repeatable exercise for the compiler and runtime. It generates programs, checks their results against an independently written oracle, and saves enough information to replay a failure. The [performance suite](performance.md) measures how quickly a correct program runs. These gyms check whether the program behaves correctly in the first place.
+
+The current suite exercises LLVM output in development and optimized builds. It also checks operations the compiler must reject, stresses allocation and cleanup, and runs native memory instrumentation. The [recorded qualification](qualification-results.md) lists the actual host, source fingerprint, cases, mutations and skipped checks. Passing these exercises is finite evidence for the paths tested; it is not a proof that every August program or native library is safe.
+
+## What gets exercised
+
+Each generated exercise uses a fixed seed and a bounded input domain. The default is 256 generated vectors per exercise, plus fixed edge cases. Changing the seed explores another reproducible set of inputs.
+
+| Exercise | What the program does | How its result is checked |
+| --- | --- | --- |
+| Integer arithmetic | Combine signed 64-bit values, including extremes, large exact integers, division and short circuiting. | A JavaScript BigInt oracle applies August's wrapping rules after each operation. |
+| Floating-point arithmetic | Call operations with integer and float values, including widened integers, signed zero, infinities and NaN comparisons. | Separately calculated integer or IEEE results become checked equality cases; zero division must raise the checked error. |
+| Collections | Insert, read, replace and remove map entries; add duplicate set values and check membership. | Independent JavaScript Map and Set states supply every expected result. |
+| Control flow | Call a function with varied values and branch boundaries. | A separate conditional calculation supplies the expected return. |
+| Checked bounds | Read valid, negative and out-of-range list indices. | Valid reads return the expected element; invalid reads must reach the IndexError catch. |
+| Cleanup | Create an owned resource, throw inside its scope and catch the failure. | Every iteration must report exactly one Resource drop before the program finishes. |
+| Tasks | Start a child operation, join it and accumulate its return value. | An independently calculated sum checks that each result was delivered. |
+
+The larger qualification also runs the existing ownership and cancellation cases, source mutations, native ABI boundary tests, package-integrity tests, HTTP policies and transport cleanup, and sanitizer stress programs. Public native-library qualification separately exercises real LibTorch, SQLite, zlib and BLAKE3 resources. The [native implementation report](native-implementation.md) describes those release checks and their remaining limits.
+
+## Why the suite breaks its own programs
+
+A test that accepts every program is useless. Each generated exercise therefore has a deliberately faulty version that remains valid August and runs to completion. The suite reverses a comparison, substitutes lookup for removal, changes an arithmetic operation, alters an index, removes resource construction, or changes a task result. Each fault must change the independently expected behavior.
+
+A surviving fault fails qualification. A mutant that cannot compile, crashes or times out also fails this bounded exercise: it cannot stand in for a successfully detected wrong result. Reports retain the original and faulty source units, expected output, actual output and cleanup counts. These mutations cover selected faults; they do not measure detection of every possible compiler or application defect.
+
+The compiler rejection cases are a separate gate. They attempt a second ownership move, an alias read during a mutable borrow, an unhandled error, an unlabeled input, a native call without unsafe, a task outside its required scope, a record write and a private export. Each must fail for the relevant contract, rather than an unrelated parse error.
+
+## Memory checks and their limits
+
+The sanitizer circuit instruments the actual August LLVM IR with AddressSanitizer and compiles the core C runtime with AddressSanitizer and UBSan. It runs retained text under collection pressure, Map/Set growth and deletion, task joining, and owned Shared transfer. An intentionally out-of-bounds LLVM store must be detected as a negative control.
+
+AddressSanitizer checks instrumented memory accesses; UBSan checks selected undefined operations in the runtime C. Neither replaces ownership checking or a behavior oracle. Prebuilt third-party library internals are not covered by these core sanitizer runs. Leak detection is disabled in this shared gate; explicit native allocation counters and owned-drop cases provide separate cleanup evidence. This is not a coverage-guided fuzzer or a universal race, leak or protocol proof. The [research note](research/qualification-methods.md) explains the choice of checks.
+
+## Use the evidence for your project
+
+Choose a [downloadable measured program](examples/index.md#measured-programs) that resembles your workload. Read its code and compiled spec, then run it with the published CLI. Add your application's expected results and failure cases as [same-file tests](testing.md). Use [project benchmarks](performance.md#benchmark-your-own-project) to measure that tested behavior under your own deployment conditions.
+
+Language contributors can reproduce the full reports, change seeds, replay saved cases and add exercises using the [qualification workflow](contributing-benchmarks.md#run-the-safety-gyms). That workflow requires maintainer tools for the C references and instrumented runtime; ordinary package consumers do not need them.