@holoscript/holosystem 0.2.1 → 0.2.3

package/README.md

@@ -55,6 +55,553 @@ returns one highest-priority candidate, and carries authority, validation, lease
 and spend stop conditions. It selects work; it does not claim a board task,
 publish a package, spend funds, or bypass caller authority.
 
+## Substrate Closure
+
+The substrate closure replaces an implicit operating-system dependency tower
+with a deterministic, caller-owned infrastructure graph. Every component names
+an exact version, portable source revision, content digest, custody owner,
+dependency edges, installation-script state, and a cryptographically
+authenticated rebuild. External
+components can remain during migration, but they are visible sovereignty
+boundaries rather than hidden transitive dependencies.
+
+```bash
+npx holosystem substrate \
+  --input substrate.json \
+  --output substrate-lock.json \
+  --json
+```
+
+```js
+import { buildSubstrateClosure, createRebuildAttestationPayload } from '@holoscript/holosystem';
+
+const component = {
+  id: 'holosystem',
+  kind: 'runtime',
+  version: '1.0.0',
+  custody: {
+    mode: 'owned',
+    owner: 'holoscript',
+    trustDomain: 'holoscript-release',
+  },
+  source: { uri: 'holorepo://holoscript', revision: 'abc123' },
+  artifact: { digest: 'sha256:<64 lowercase hex characters>' },
+  execution: { installScripts: 'none' },
+  requires: [],
+  verification: {
+    rebuilds: [
+      {
+        verifier: 'independent-builder',
+        digest: 'sha256:<same 64 lowercase hex characters>',
+        signature: '<Ed25519 signature in canonical base64>',
+      },
+    ],
+  },
+};
+
+// The independent builder signs this exact UTF-8 payload with its private key.
+const payload = createRebuildAttestationPayload({
+  verifier: 'independent-builder',
+  component,
+});
+
+const receipt = buildSubstrateClosure({
+  root: 'holosystem',
+  coverage: {
+    includedLayers: ['kernel', 'runtime', 'toolchain'],
+    missingLayers: [],
+  },
+  verificationPolicy: {
+    minimumIndependentRebuilds: 1,
+    trustRoots: [
+      {
+        verifier: 'independent-builder',
+        trustDomain: 'independent-builders',
+        publicKey: '<Ed25519 public key in PEM format>',
+      },
+    ],
+  },
+  components: [component],
+});
+```
+
+The command exits with code `2` and still emits the blocked receipt when it
+finds a missing or unreachable dependency, a cycle, a floating version, a local
+source path, an invalid digest, an untrusted verifier, a forged signature, a
+same-domain rebuild, a present installation script, or too few matching
+rebuilds. The receipt exposes only public-key fingerprints, not the supplied
+PEM text. Installation scripts remain blocked until a later execution layer can
+prove they were forcibly disabled or sandboxed.
+
+`coverage.includedLayers` declares what the graph represents, while any known
+gap remains in `coverage.missingLayers` and blocks readiness. Coverage is still
+a caller assertion rather than discovery proof; importers must carry their
+known blind spots into this field.
+
+Trust domains are assertions in the caller-owned policy. The signature proves
+that the named key attested to the exact component tuple; it does not prove that
+the builder is organizationally independent or that the component is safe.
+Production trust roots therefore belong to independently governed rebuilders,
+not keys created by the component custodian for the same release.
+
+### Import an npm dependency graph
+
+`substrate-import` derives the graph from an npm `package-lock.json` instead of
+asking an operator to maintain the dependency edges manually. It supports lock
+formats [v2 and v3](https://docs.npmjs.com/cli/v11/configuring-npm/package-lock-json/),
+preserves nested Node resolution, converts canonical npm SRI
+integrity into content digests, and marks every registry package as external
+custody.
+
+```json
+{
+  "root": {
+    "id": "demo-app",
+    "custody": {
+      "mode": "owned",
+      "owner": "demo",
+      "trustDomain": "demo-release"
+    },
+    "source": {
+      "uri": "https://github.com/example/demo",
+      "revision": "demo-revision-1"
+    }
+  },
+  "verificationPolicy": {
+    "minimumIndependentRebuilds": 1,
+    "trustRoots": [
+      {
+        "verifier": "rebuild-farm",
+        "trustDomain": "independent-rebuild-farm",
+        "publicKey": "<Ed25519 public key in PEM format>"
+      }
+    ]
+  }
+}
+```
+
+```bash
+npx holosystem substrate-import \
+  --lock package-lock.json \
+  --config substrate-import.json \
+  --output substrate-input.json \
+  --json
+```
+
+The command writes the generated closure input to `--output` and emits an
+import receipt on stdout. A valid import has status
+`coverage-and-attestation-required`, or `execution-policy-required` when a
+package declares an installation script; neither means `ready`. Lockfiles prove
+resolution and registry integrity, but not a rebuild, safe lifecycle execution,
+or complete operating-system coverage. After independent builders attach signed
+attestations, run the existing
+`holosystem substrate --input substrate-input.json --json` gate.
+
+Production dependencies are imported by default. Set `"includeDev": true` in
+the import configuration to include direct development dependencies. Optional
+branches present in the lock are retained as a conservative superset; missing
+optional dependencies are counted but do not block import. No package or
+install script is executed. `registry.npmjs.org` lockfile indirection is
+preserved as `npm://configured-registry/...` instead of pretending that the
+configured registry endpoint is known. The root component's artifact digest
+represents the canonical lockfile, not a compiled application artifact. Native
+libraries and operating-system packages are not derived yet, so those gaps are
+carried into the generated input and block the closure. Local links,
+missing runtime entries, malformed or weak integrity, non-portable sources, and
+private keys in the trust policy block import without echoing their values.
+
+```js
+import { importNpmPackageLock } from '@holoscript/holosystem';
+
+const imported = importNpmPackageLock({
+  lockfile,
+  root,
+  verificationPolicy,
+});
+```
+
+### Import an installed Debian package graph
+
+`substrate-import-debian` converts offline Debian evidence into an
+operating-system substrate graph. It accepts one or more independently anchored
+repository indexes, parses the standard control-stanza format,
+resolves `Depends`, `Pre-Depends`, alternatives, exact version relations, and
+installed virtual-package providers according to the
+[Debian relationship rules](https://www.debian.org/doc/debian-policy/ch-relationships.html),
+and binds each installed package to the SHA-256 or SHA-512 and `Filename` in a
+caller-anchored `Packages` index.
+
+The command does not run `dpkg`, `apt`, or any package script. Supply:
+
+- the installed `dpkg` status text;
+- each uncompressed repository `Packages` index and its expected digest;
+- a JSON map for every installed `package:architecture`, containing hashes for
+  any `preinst`, `postinst`, `prerm`, `postrm`, or `config` scripts;
+- a config containing the repository base, system root, custody, and optional
+  rebuild policy.
+
+```json
+{
+  "repository": {
+    "uri": "https://deb.debian.org/debian/",
+    "packagesIndexDigest": "sha256:<digest of the exact Packages input>"
+  },
+  "root": {
+    "id": "demo-linux",
+    "version": "12.1-1",
+    "custody": {
+      "mode": "owned",
+      "owner": "demo",
+      "trustDomain": "demo-os-release"
+    },
+    "source": {
+      "uri": "https://images.example.com/demo-linux",
+      "revision": "snapshot-20260716"
+    }
+  }
+}
+```
+
+```json
+{
+  "base-files:amd64": {},
+  "demo-app:amd64": {
+    "postinst": "sha256:<digest of the installed postinst file>"
+  }
+}
+```
+
+```bash
+npx holosystem substrate-import-debian \
+  --status status \
+  --packages Packages \
+  --maintainer-scripts maintainer-scripts.json \
+  --config debian-substrate-import.json \
+  --output debian-substrate-input.json \
+  --json
+```
+
+Real installations usually span base, security, update, or third-party
+archives. Use `--sources` instead of `--packages` for those systems. The source
+manifest is a JSON array; local `packages` paths are read by the CLI but are not
+copied into the receipt.
+
+```json
+[
+  {
+    "packages": "Packages.main",
+    "uri": "https://deb.debian.org/debian/",
+    "packagesIndexDigest": "sha256:<digest>",
+    "authentication": {
+      "inReleasePath": "InRelease",
+      "verifier": {
+        "path": "/usr/bin/gpgv",
+        "digest": "sha256:<gpgv-binary-digest>"
+      },
+      "keyrings": [
+        {
+          "path": "/usr/share/keyrings/debian-archive-keyring.gpg",
+          "digest": "sha256:<keyring-digest>"
+        }
+      ],
+      "trustedFingerprints": ["<exact-primary-or-signing-key-fingerprint>"],
+      "expected": {
+        "suite": "stable",
+        "codename": "trixie",
+        "architecture": "amd64",
+        "component": "main"
+      },
+      "packagesIndexPath": "main/binary-amd64/Packages",
+      "maxReleaseAgeSeconds": 604800
+    },
+    "custody": {
+      "owner": "debian-main",
+      "trustDomain": "debian-main-archive"
+    }
+  },
+  {
+    "packages": "Packages.security",
+    "uri": "https://security.debian.org/debian-security/",
+    "packagesIndexDigest": "sha256:<digest>",
+    "custody": {
+      "owner": "debian-security",
+      "trustDomain": "debian-security-archive"
+    }
+  }
+]
+```
+
+```bash
+npx holosystem substrate-import-debian \
+  --status status \
+  --sources debian-sources.json \
+  --maintainer-scripts maintainer-scripts.json \
+  --config debian-substrate-import.json \
+  --output debian-substrate-input.json \
+  --json
+```
+
+Authentication paths are operational inputs resolved from the current working
+directory and are not copied into receipts. Use `releasePath` plus
+`releaseSignaturePath` instead of `inReleasePath` for detached `Release.gpg`
+signatures. A Git-for-Windows verifier can set `verifier.pathStyle` to `msys`;
+other verifiers use the default `native` path style. The package never fetches
+or imports keys, so establishing the verifier digest, keyring digest, and exact
+fingerprint allowlist remains a caller-owned trust-bootstrap operation.
+On the first import, omit `minimumReleaseDate`; on later imports, copy the
+accepted `release.date` from the last receipt to reject rollback within the
+freshness window. Multi-signed archives require every signature visible to
+`gpgv` to resolve through the pinned keyring and fingerprint allowlist. Treat
+archive key transitions as explicit configuration updates rather than weakening
+verification. Signing-key compromise, key lifecycle, and the trusted clock
+remain caller-owned boundaries.
+
+Every fully installed package becomes an external-custody component and the
+system root depends on all of them, including packages no longer reachable from
+another package. The importer implements Debian's version ordering, including
+epochs and the special tilde order. It fails closed on an index-digest mismatch,
+missing package artifact, unsatisfied or ambiguous dependency, path traversal,
+incomplete maintainer-script evidence, untrusted signature, stale Release,
+distribution identity change, or signed index mismatch.
+
+A source without authentication still uses its caller digest only; it does not
+claim that a Debian `Release` or `InRelease` signature was verified and retains
+the blocking `repository-authentication` coverage gap. When every source passes,
+the receipt includes that layer and records the signed Release-to-Packages hash
+chain, pinned verifier and keyring digests, exact signer fingerprints, and replay
+window. Likewise, Debian
+[maintainer scripts execute during package transitions](https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html),
+so any present script still blocks substrate readiness. The generated input
+always marks operating-system coverage as included. Signed rebuild attestations,
+native-build provenance, and the existing `substrate` gate remain required.
+
+```js
+import { importDebianPackageSnapshot } from '@holoscript/holosystem';
+
+const imported = importDebianPackageSnapshot({
+  status,
+  sources: [{ packagesIndex, repository, custody }],
+  maintainerScripts,
+  root,
+  verificationPolicy,
+});
+```
+
+For a verification-only workflow, call `verifyDebianRepositoryRelease` with the
+same authentication fields plus `packagesIndex` and `packagesIndexDigest`. It
+returns `holoscript.holosystem.debian-release-auth.v1`; `verified` is true only
+when OpenPGP verification, signer pinning, release identity and freshness, and
+the signed Packages checksum and size all pass.
+
+### Build a native artifact in a closed executor
+
+`native-build` is the first executable bridge from the substrate graph to a
+reproducible native artifact. The initial vocabulary deliberately supports one
+small target: a C11 translation unit compiled by GCC into a Linux AMD64 GNU ELF
+executable. It does not accept a shell command, arbitrary compiler executable,
+free-form flags, lifecycle script, network mode, user id, or weaker isolation
+policy from the plan.
+
+Hash the caller-owned source tree first:
+
+```bash
+npx holosystem native-build-source --source ./native-source --json
+```
+
+Then create a plan using that source digest and a digest-pinned local Docker
+binary plus an immutable OCI manifest reference:
+
+```json
+{
+  "schema": "holoscript.holosystem.native-build-plan.v1",
+  "id": "demo-native-build",
+  "source": {
+    "digest": "sha256:<source-manifest-digest>",
+    "sourceDateEpoch": 1700000000
+  },
+  "target": {
+    "os": "linux",
+    "architecture": "amd64",
+    "abi": "gnu"
+  },
+  "executor": {
+    "kind": "docker",
+    "digest": "sha256:<docker-binary-digest>",
+    "image": "docker.io/library/gcc@sha256:<amd64-manifest-digest>"
+  },
+  "compiler": {
+    "family": "gcc",
+    "language": "c11",
+    "source": "main.c",
+    "optimization": "speed"
+  },
+  "output": {
+    "path": "demo",
+    "format": "elf-executable"
+  },
+  "limits": {
+    "timeoutSeconds": 60,
+    "memoryMiB": 512,
+    "cpus": 1,
+    "pids": 64,
+    "tmpfsMiB": 64
+  },
+  "rebuilds": 2
+}
+```
+
+```bash
+npx holosystem native-build \
+  --plan native-build.json \
+  --source ./native-source \
+  --executor /absolute/operational/path/to/docker \
+  --artifact-dir ./native-output \
+  --output native-build-receipt.json \
+  --json
+```
+
+The first successful run is intentionally `artifact-pin-required` and exits
+with code `2`. It performs two clean builds and reports their common digest but
+does not claim `native-build` coverage. Copy that digest into
+`expectedArtifactDigest`, use a new empty artifact directory, and rerun. Only a
+matching reproducible result returns `verified`, exits `0`, and includes the
+`native-build` layer.
+
+The runner verifies an immutable source snapshot before mounting it, checks the
+Docker executable digest, prohibits image pulls and networking, mounts both the
+root filesystem and source read-only, drops all Linux capabilities, enables
+`no-new-privileges`, runs as uid/gid 65534, bounds time, memory, CPU, PIDs and
+temporary storage, and invokes `/usr/local/bin/gcc` directly with generated
+deterministic arguments. It rejects undeclared files, symlink outputs,
+non-AMD64 ELF results, mismatched artifact pins, and differing rebuild digests.
+Compiler logs are represented only by byte counts and hashes; operational host
+paths are not copied into the receipt.
+
+```js
+import {
+  createNativeRebuildAttestationPayload,
+  inspectNativeBuildPlan,
+  inspectNativeBuildSource,
+  runNativeBuild,
+} from '@holoscript/holosystem';
+```
+
+`createNativeRebuildAttestationPayload` joins a verified native receipt to the
+existing `holoscript.holosystem.rebuild-attestation.v1` payload. The independent
+builder still signs that payload outside this package, and the substrate gate
+still verifies the signature against caller-owned trust roots. The receipt
+does not prove the Docker host, compiler-image supply chain, host kernel, CPU,
+or organizational independence; those remain named boundaries.
+
+This tracer commences the build/execution layer. It is not a claim that
+HoloSystem already replaces measured boot, UEFI/device firmware, an ISA backend,
+or physical hardware verification. Those require separate target contracts and
+receipts rather than broader process execution in this API.
+The adversary classes, attack specifications, test mapping, and residual trust
+boundary are recorded in the [native-build threat model](./docs/native-build-threat-model.md).
+
+### Launch a measured machine VM
+
+`vm-launch` extends the executable tracer below the container boundary. It boots
+an AMD64 Linux kernel and initramfs as a full-system q35 guest with QEMU TCG.
+`vm-launch-whpx` is a separately named adapter for QEMU's Windows Hypervisor
+Platform backend. The adapters do not share an accelerator switch and never
+fall back to one another. Both vocabularies are deliberately closed: Windows
+AMD64 host, `qemu-system-x86_64.exe`, q35, 128 MiB, one CPU, and exactly two
+launches. Plans cannot provide a command, argument, environment variable,
+device, network, firmware, monitor, display, or fallback setting.
+
+First measure the complete caller-owned QEMU runtime closure and both guest
+artifacts:
+
+```bash
+npx holosystem vm-executor --runtime C:/absolute/path/to/qemu-runtime --json
+npx holosystem vm-asset --kind kernel --file C:/absolute/path/to/vmlinuz --json
+npx holosystem vm-asset --kind initrd --file C:/absolute/path/to/initramfs --json
+```
+
+Create a plan with those digests and the SHA-256 digest of the exact expected
+serial success signal:
+
+```json
+{
+  "schema": "holoscript.holosystem.vm-launch-plan.v1",
+  "id": "demo-vm-proof",
+  "host": { "os": "windows", "architecture": "amd64" },
+  "executor": {
+    "kind": "qemu-system",
+    "binary": "qemu-system-x86_64.exe",
+    "binaryDigest": "sha256:<qemu-executable-digest>",
+    "runtimeDigest": "sha256:<complete-runtime-manifest-digest>"
+  },
+  "target": { "architecture": "amd64", "machine": "q35", "accelerator": "tcg" },
+  "guest": {
+    "kernelDigest": "sha256:<kernel-digest>",
+    "initrdDigest": "sha256:<initrd-digest>",
+    "expectedConsoleDigest": "sha256:<exact-serial-output-digest>"
+  },
+  "resources": { "memoryMiB": 128, "cpus": 1, "timeoutSeconds": 30 },
+  "launches": 2
+}
+```
+
+For WHPX, use schema `holoscript.holosystem.whpx-vm-launch-plan.v1`, set the
+target accelerator to `whpx`, and add `guest.expectedDiagnosticsDigest`. That
+digest pins the exact WHPX diagnostic bytes after removal of only the generated
+private executor-path prefix. The WHPX adapter requires successful explicit
+WHPX execution; feature labels or QEMU's compiled backend list are not treated
+as proof that acceleration is usable.
+
+```bash
+npx holosystem vm-launch \
+  --plan vm-launch.json \
+  --runtime C:/absolute/path/to/qemu-runtime \
+  --kernel C:/absolute/path/to/vmlinuz \
+  --initrd C:/absolute/path/to/initramfs \
+  --output vm-launch-receipt.json \
+  --json
+```
+
+Use `npx holosystem vm-launch-whpx` with the same four caller-owned artifact
+arguments for the WHPX schema. WHPX requires the Windows Hypervisor Platform
+API exposed to QEMU; see the [QEMU WHPX documentation](https://www.qemu.org/docs/master/system/whpx.html)
+and [Microsoft's Windows Hypervisor Platform API overview](https://learn.microsoft.com/en-us/virtualization/api/).
+
+The runner creates a private snapshot and remeasures the complete QEMU closure,
+kernel, and initramfs before and after each launch. It then generates QEMU
+arguments that disable user configuration, default devices, networking, USB,
+display, monitor, and reboot;
+passes only a minimal environment; bounds output and time; and requires both
+launches to produce the pinned serial digest and fixed debug-exit code. TCG
+requires no emulator diagnostics. WHPX requires the normalized diagnostic
+digest pinned by its plan; unexpected bytes block the receipt. Receipts contain
+only digests and byte counts, never raw console output or operational host
+paths.
+
+A verified TCG receipt includes `machine-vm-launch`,
+`guest-artifact-measurement`, and `virtual-device-minimization`, reports
+`hardwareBacked: false`, and leaves hardware acceleration missing. A verified
+WHPX receipt additionally includes `hardware-hypervisor-acceleration` and sets
+`hardwareBacked: true` only after two explicit WHPX launches succeed. Both
+adapters leave `host-process-isolation` missing: QEMU still has the ambient
+rights of the Windows process. Receipts therefore expose
+`isolation.hostProcess: "ambient-windows-process"` and
+`isolation.verified: false` even when WHPX acceleration is verified. Neither
+receipt proves an IOMMU, measured boot, firmware authenticity, confidential
+memory, or side-channel resistance. The
+exact claims are in the [VM launch threat model](./docs/vm-launch-threat-model.md).
+
+```js
+import {
+  inspectVmExecutor,
+  inspectVmLaunchAsset,
+  inspectVmLaunchPlan,
+  inspectWhpxVmLaunchPlan,
+  runVmLaunch,
+  runWhpxVmLaunch,
+} from '@holoscript/holosystem';
+```
+
 ## What It Creates
 
 The default configuration pins public contracts for: