as-test 1.4.1 → 1.5.1

package/CHANGELOG.md

@@ -1,5 +1,69 @@
 # Change Log
 
+## 2026-05-30 - v1.5.1
+
+### An early-exiting runtime now fails instead of warning
+
+- fix: when the runtime never delivers its final report payload, the CLI reconstructs a result from the streamed lifecycle events (`synthesizeReportFromRuntimeEvents`). Previously this always emitted a `WARN` and returned the reconstruction, only escalating to a `FAIL` if the child exited non-zero or wrote to stderr — so a spec that trapped/exited early with exit code `0` and no stderr came back as a passing reconstruction (the `runtime report payload missing; reconstructed result from streamed lifecycle events` warning storm). Both `runProcess` (WASI/bindings) and `runWebSessionProcess` (web) now treat `!runtimeEvents.sawFileEnd` — the runtime never emitted `event:file-end`, i.e. it exited before the file finished — as a failure: `appendRuntimeFailureReport` with a persisted crash record and a `test runtime exited before completing the test file` message, no misleading `WARN`. A run that _did_ reach `file-end` but simply failed to flush the final report frame is still the recoverable case (`WARN` + reconstructed result). This also closes a gap in the web path, which previously never escalated to a failure in the synthesized branch — it only warned, even on a non-zero exit.
+
+### esm bindings now run
+
+- fix: `instantiateEsmInstance` (`lib/src/index.ts`) now calls `patchNodeIo()` before importing the bindings helper. An esm helper auto-instantiates at import time and writes the WIPC report by calling the global `process.stdout.write(ArrayBuffer)` directly. `patchNodeIo()` — which teaches `process.stdout.write`/`process.stdin.read` to accept a raw `ArrayBuffer` and route it through `fs.writeSync` — was only wired into the raw path (via `withNodeIo`), never the esm path. So under esm bindings Node threw `ERR_INVALID_ARG_TYPE` ("chunk must be of type string or Buffer…, received an instance of ArrayBuffer") before any report was emitted, and the run crashed with `missing report payload from test runtime`. The patch is now in place by the time the helper instantiates.
+
+### `--bindings` is respected instead of overridden
+
+- feat: as-test no longer forces `--bindings raw` when you've already declared bindings yourself. `getDefaultBuildArgs` (`cli/commands/build-core.ts`) now takes a `bindingsAlreadyConfigured` flag and only appends `--bindings raw` when neither `buildOptions.args` nor a referenced asconfig declares `--bindings`. The other bindings flags (`AS_TEST_BINDINGS=1`, `--exportRuntime`, `--exportStart _start`) are still always injected. Two new detectors back this: `argsDeclareBindings(args)` (scans for `--bindings`/`--bindings=`) and `asconfigDeclaresBindings(configPath)` (reads `options.bindings`, follows `extends`), mirroring the existing try-as detection. Previously `--bindings esm` in `buildOptions.args` was combined with the forced `--bindings raw`, so `asc` emitted glue for **both** styles into one file; the runtime then mis-detected the kind and crashed. With this, `--bindings esm` produces esm-only glue and runs.
+
+### Mocking works on every runtime
+
+- change: `mock.spec.ts` is split into `mock.spec.ts` (mocking + `unmockFn` only) and `unmock.spec.ts` (the `unmockImport` cases). The split tracks the real esm/standalone-WASI boundary: the transform removes a `@external` import from the wasm when it is **only ever mocked**, but keeps it (for fall-back) when it is `unmockImport`'d anywhere. A pure-mock spec therefore imports nothing virtual and runs on **every** runtime — verified via `WebAssembly.Module.imports()`: the pure-mock wasi build imports only `wasi_snapshot_preview1`, while the unmock build imports `mock.foo`. `unmockFn` (function mocks) does not retain an import; only `unmockImport` does.
+- feat: pure `mockImport` specs now run under **esm bindings** and the standalone WASI runtimes (`wasmtime`, `wasmer`, `wazero`) — `mock.spec.ts` is no longer excluded from those modes (it was in v1.5.0). Only `unmock.spec.ts`, which retains a real host binding the host can't supply under those runtimes, is excluded (`!**/unmock.spec.ts`).
+- feat: two new modes in `as-test.config.json` — `node:bindings:raw` and `node:bindings:esm` (both `default: false`) — exercise each bindings style explicitly, and both are added to the `test:modes` matrix so `test:all` covers them.
+
+### Watch mode exits with the last verdict
+
+- feat: quitting `--watch` (ctrl+c, both the raw-mode `0x03` path and the `SIGINT` handler in `runWatchLoop`) now exits `1` when the most recent run left any spec failing **or a run is still in flight**, instead of always exiting `0`. The watch loop already tracks currently-failing `(spec, mode)` pairs in its sticky `failingSpecs` map and an `isRunning` flag, so the exit code is `isRunning || failingSpecs.size ? 1 : 0` — an interrupted run counts as a failure. This lets a red watch session fail CI and shell pipelines (`ast test --watch && deploy`) instead of masking the failure on quit.
+
+### CI uses the main config
+
+- chore: removed `as-test.ci.config.json`. `test:ci` now runs against the main `as-test.config.json` (`npm run test -- --mode node:bindings,node:wasi,wasmtime`), so CI uses the same modes, `features` (`try-as`), and per-mode spec exclusions as everything else — `try-as` no longer needs an explicit `--enable`, and the stale CI-only `wasmtime` exclusion that still ran `unmock.spec.ts` (and failed on the missing `mock::foo` host import) is gone.
+
+## 2026-05-28 - v1.5.0
+
+### `mockFn` and `mockImport` now work anywhere
+
+- feat: `mockFn(target, callback)` and `unmockFn(target)` can be called from **anywhere** in the source — including inside a `test()` / `it()` callback — not just module top-level. The MockTransform now reads the visitor's enclosing-statement `ref` so it locates and removes the directive wherever it lives, defers list mutations to the end of `visitSource` so traversal isn't disturbed, and `unshift`s the generated mock fn to module scope so every rewritten call site can reach it. Previously, a nested `mockFn` silently no-op'd because the transform's top-level-statement scan never found the directive — calls remained un-rewritten and the test passed only because trivial assertions passed too.
+- feat: `mockImport(path, callback)` + `unmockImport(path)` now retains the real import as a fallback (call this "route 2"). When a `mockImport`'d path is **never unmocked**, the `@external` import is removed and the wrapper always dispatches through `__mock_import` — same as before. When a path **is** unmocked somewhere, the transform keeps the real import (renamed `__as_test_real_<name>`, de-exported), and the wrapper becomes `if (__mock_import.has(path)) call_indirect(...) else __as_test_real_<name>(...)`. So a call after `unmockImport` falls back to the host binding instead of trapping on a missing `__mock_import` entry.
+- fix: the runtime import-stub now covers any module the wasm declares (not just `env`), so a retained `mockImport` target with no host implementation (e.g. an artificial test import) gets a `() => 0` no-op stub before `WebAssembly.instantiate`. Without this, the wasm `LinkError`s on instantiation. `env` is no longer skipped — asc raw bindings put their `abort`/`trace`/`seed` defaults on the inner object and user env imports on the prototype, so the stub fills missing `env.*` user imports without clobbering asc's defaults.
+- fix: in raw bindings the helper reads `imports.<module>` while building its import object, so the stub had to be applied **before** `helper.instantiate(module, imports)` is called — adding it inside the `WebAssembly.instantiate` wrapper alone was too late and surfaced as `instantiate@.../<spec>.js:...` failures (this was the chromium-headful + watch + full-suite crash where one bad job poisoned the persistent session).
+
+### Class serializer rename — `__AS_TEST_TO_JSON()`
+
+- change: the transform-injected class serializer is now emitted under an internal name `__AS_TEST_TO_JSON(): string` instead of `toJSON(): string`. Users can't accidentally call the generated method, and a hand-written `toJSON()` no longer collides with the generated one.
+- feat: at runtime, `stringify<T>` for a managed class calls a user `toJSON()` only when its return type is a string (probed via a generic-parameter helper so the dead branch is pruned at compile time); a non-string `toJSON` cleanly falls back to the generated `__AS_TEST_TO_JSON()` structural serializer instead of being a compile error. The transform now generates `__AS_TEST_TO_JSON` for every eligible class so the fallback always exists.
+
+### Chromium fix — fragmented WebSocket frames
+
+- fix: chromium fragments outbound WebSocket binary messages at ~128KB. The two web server frame parsers (standalone runner in `lib/src/index.ts` and managed `PersistentWebSessionHost` in `cli/commands/web-session.ts`) read the opcode but **never checked the FIN bit or handled continuation frames (`0x0`)** — so chromium's continuation frame was silently dropped, the wipc byte stream desynced, and frame-header bytes bled into report payloads as `Bad control character in JSON`. Both parsers now reassemble fragments (buffer `0x1`/`0x2` with FIN=0, append `0x0` until FIN=1). firefox and webkit don't fragment at that size, which is why they always worked.
+- fix: the CLI reassembled chunked report payloads by `data.toString("utf8")` per-chunk then joining. The producer (`assembly/util/wipc.ts`) chunks raw UTF-8 bytes, so a multibyte character straddling a chunk boundary was getting mis-decoded into replacement chars (or worse) on any transport, not just chromium. Chunks are now buffered as `Buffer`s and decoded UTF-8 once after `Buffer.concat`.
+
+### Web mode — repro command + browser test infrastructure
+
+- fix: when a managed web run (`ast test --mode firefox/chromium/webkit`) fails, the printed repro now points at the **managed CLI command** (`node ./bin/index.js run <spec> --mode <browser>`) instead of `node .as-test/runners/default.web.js <wasm>`. The managed path uses `PersistentWebSessionHost`'s panel UI; the standalone runner uses `lib/src/web-runner/*`'s terminal UI — completely different stacks. So the old repro never actually reproduced the failing run, and showed a different UI to boot. Top-level negations in `runCommandForLog` are now mode-aware via `webSession`.
+- change: removed `tests/fixtures/fake-browser.mjs`. The integration suite's web tests now use real headless chromium (auto-detected from PATH or the Playwright cache), and `AS_TEST_SKIP_WEB=1` cleanly skips them in CI. CI workflows (`.github/workflows/as-test.yml`, `release.yml`) already set `AS_TEST_SKIP_WEB=1`. Without this, the shim's flow hung indefinitely in some local environments, eating ~hours of integration time.
+- feat: a small "browser-closes-early" test now ships a `process.exit(0)` shebanged executable as `--browser` to verify the framework reports a clean disconnect error rather than hanging.
+
+### Per-mode spec exclusion
+
+- feat: mode-level `input` overrides can use `!`-prefixed patterns to exclude specific specs from that mode. The orchestrator's per-mode `run()` call now checks the **mode-only additions** (negations in the mode's `input` array that aren't in the top-level `input`) against the selector and returns an empty result for excluded files. Top-level negations remain ignored when a file is explicitly selected, so `test __tmp_foo` still works for paths the top-level config would otherwise filter.
+- chore: `mock.spec` is excluded from `wasmtime`, `wasmer`, and `wazero` modes in `as-test.config.json`. Those CLI-only WASI runtimes can't accept arbitrary imports at instantiation, so the retained `env.*` `mockImport` target has no JS host to satisfy it.
+
+### Scripts and CI standardization
+
+- feat: new `build` script aliases `build:cli && build:lib && build:transform`. `test:all`, `release:check`, and `prepublishOnly` now compose with it instead of repeating the three sub-builds. `test:all` runs `build && test:modes && test:integration && test:examples`.
+- chore: `test:modes` drops the duplicate `--parallel` (the base `test` script already has it), and `test:ci` composes with `npm run test --` instead of hard-coding the bin path.
+- chore: `.github/workflows/as-test.yml` and `release.yml` now share identical test-job setup (checkout@v4, `actions/setup-node@v4`, `oven-sh/setup-bun@v1`, install Wasmtime, lint, typecheck, build, `test:ci`, `test:integration`). `as-test.yml` keeps the Test Summary action for PR feedback; the manual `curl`/`tar` Node bootstrap and the inline Bun installer are gone. `examples.yml` uses the `build` alias.
+
 ## 2026-05-27 - v1.4.1
 
 ### Dependency hygiene

package/assembly/src/expectation.ts

@@ -20,8 +20,8 @@ function safeStringify<T>(value: T): string {
     !isString<T>() &&
     !isArray<T>()
   ) {
-    // @ts-expect-error: optional user-supplied serializer
-    if (!isDefined(value.toJSON)) {
+    // @ts-expect-error: warn only when nothing serializes it — no user `toJSON` and no transform-generated `__AS_TEST_TO_JSON` → renders as a `<TypeName>` placeholder.
+    if (!isDefined(value.toJSON) && !isDefined(value.__AS_TEST_TO_JSON)) {
       sendWarning(
         "Class " +
           nameof<T>() +

package/assembly/src/stringify.ts

@@ -15,8 +15,11 @@
 //   * Array / StaticArray               → element-wise recursion
 //   * Set                               → array of values
 //   * Map                               → object with stringified keys
-//   * managed with `toJSON(): string`   → call it
-//   * managed without `toJSON()`        → "<TypeName>" placeholder
+//   * managed, user `toJSON()` returns string → call it
+//   * managed otherwise                       → `__AS_TEST_TO_JSON()`
+//       (transform-generated structural serializer; the fallback when a
+//        user `toJSON` returns a non-string, or when there's no `toJSON`)
+//   * managed with neither                    → "<TypeName>" placeholder
 //
 // The Date/ArrayBuffer/typed-array/StaticArray/Set/Map branches use
 // `value instanceof X` guards. In a generic function AssemblyScript resolves
@@ -134,8 +137,16 @@ export function stringify<T>(value: T): string {
   }
 
   if (isManaged<T>()) {
-    // @ts-ignore: hand-written or transform-generated serializer
-    if (isDefined(value.toJSON)) return value.toJSON();
+    // A user-supplied `toJSON` wins, but only when it returns a string.
+    // `preferToJSONString` decides that on the return type and otherwise
+    // falls back to the transform-generated structural serializer.
+    // @ts-ignore: optional user-supplied serializer
+    if (isDefined(value.toJSON)) {
+      // @ts-ignore: optional user-supplied serializer
+      return preferToJSONString(value, value.toJSON());
+    }
+    // @ts-ignore: transform-generated structural serializer
+    if (isDefined(value.__AS_TEST_TO_JSON)) return value.__AS_TEST_TO_JSON();
     return escape("<" + nameof<T>() + ">");
   }
 
@@ -144,6 +155,20 @@ export function stringify<T>(value: T): string {
   return escape("<" + nameof<T>() + ">");
 }
 
+// Given a managed value and the result of calling its `toJSON()`, return
+// that result when it's a string, otherwise the transform-generated
+// `__AS_TEST_TO_JSON` structural serializer (or a `<TypeName>` placeholder
+// if neither applies). `R` is the `toJSON` return type, so the
+// `result`-returning branch is pruned — and never type-checked — for any
+// non-string return type. That's what lets a `toJSON` returning, say,
+// `i32` fall back here instead of being a hard compile error.
+function preferToJSONString<T, R>(value: T, result: R): string {
+  if (isString<R>()) return result as string;
+  // @ts-ignore: transform-generated structural serializer
+  if (isDefined(value.__AS_TEST_TO_JSON)) return value.__AS_TEST_TO_JSON();
+  return escape("<" + nameof<T>() + ">");
+}
+
 // JSON string escape per RFC 8259, with explicit handling for UTF-16
 // surrogates (matches json-as's serializeString behaviour).
 //

package/bin/commands/build-core.js

@@ -68,9 +68,13 @@ export async function build(
   const sourceInputPatterns =
     overrides.kind === "fuzz" ? config.fuzz.input : config.input;
   const inputPatterns = resolveInputPatterns(sourceInputPatterns, selectors);
-  const inputFiles = (await glob(inputPatterns)).sort((a, b) =>
-    a.localeCompare(b),
-  );
+  const includePatterns = inputPatterns.filter((p) => !p.startsWith("!"));
+  const ignorePatterns = inputPatterns
+    .filter((p) => p.startsWith("!"))
+    .map((p) => p.slice(1));
+  const inputFiles = (
+    await glob(includePatterns, { ignore: ignorePatterns })
+  ).sort((a, b) => a.localeCompare(b));
   await assertNoArtifactCollisions(sourceInputPatterns);
   warnOnUnknownModeReferences(inputFiles, loadedConfig.modes ?? {});
   const coverageEnabled = resolveCoverageEnabled(
@@ -434,10 +438,13 @@ function getBuildCommand(
   }
   const tryAsAlreadyConfigured =
     argsDeclareTryAs(userArgs) || asconfigDeclaresTryAs(config.config);
+  const bindingsAlreadyConfigured =
+    argsDeclareBindings(userArgs) || asconfigDeclaresBindings(config.config);
   const defaultArgs = getDefaultBuildArgs(
     config,
     featureToggles,
     tryAsAlreadyConfigured,
+    bindingsAlreadyConfigured,
   );
   const ascInvocation = resolveAscInvocation(pkgRunner);
   // as-test's own transform goes first so CoverageTransform sees the
@@ -765,6 +772,7 @@ function getDefaultBuildArgs(
   config,
   featureToggles,
   tryAsAlreadyConfigured = false,
+  bindingsAlreadyConfigured = false,
 ) {
   const buildArgs = [];
   const effectiveFeatures = resolveEffectiveFeatures(config, featureToggles);
@@ -791,7 +799,6 @@ function getDefaultBuildArgs(
     if (INTERNAL_FEATURE_NAMES.has(feature)) continue;
     buildArgs.push("--enable", feature);
   }
-  // Should also strip any bindings-enabling from asconfig
   if (
     config.buildOptions.target == "bindings" ||
     config.buildOptions.target == "web"
@@ -799,12 +806,18 @@ function getDefaultBuildArgs(
     buildArgs.push(
       "--use",
       "AS_TEST_BINDINGS=1",
-      "--bindings",
-      "raw",
       "--exportRuntime",
       "--exportStart",
       "_start",
     );
+    // `raw` bindings are the default the runtime host knows how to drive.
+    // If the user already declared `--bindings` (via buildOptions.args or
+    // an asconfig), respect their choice — the runtime supports both `raw`
+    // and `esm` — and don't force `raw` on top of it (asc would otherwise
+    // emit glue for both styles, confusing kind detection).
+    if (!bindingsAlreadyConfigured) {
+      buildArgs.push("--bindings", "raw");
+    }
   } else if (config.buildOptions.target == "wasi") {
     const wasiShim = resolveWasiShim();
     if (!wasiShim) {
@@ -875,6 +888,49 @@ function argsDeclareTryAs(args) {
   }
   return false;
 }
+function argsDeclareBindings(args) {
+  for (const arg of args) {
+    if (arg === "--bindings" || arg.startsWith("--bindings=")) return true;
+  }
+  return false;
+}
+function asconfigDeclaresBindings(configPath, seen = new Set()) {
+  if (!configPath || configPath === "none") return false;
+  const resolved = path.isAbsolute(configPath)
+    ? configPath
+    : path.resolve(process.cwd(), configPath);
+  if (seen.has(resolved)) return false;
+  seen.add(resolved);
+  if (!existsSync(resolved)) return false;
+  let parsed;
+  try {
+    parsed = JSON.parse(readFileSync(resolved, "utf8"));
+  } catch {
+    return false;
+  }
+  if (!parsed || typeof parsed !== "object") return false;
+  const obj = parsed;
+  const options = obj.options;
+  if (options && typeof options === "object") {
+    const bindings = options.bindings;
+    if (typeof bindings === "string" && bindings.length) return true;
+    if (Array.isArray(bindings) && bindings.length) return true;
+  }
+  const extendsField = obj.extends;
+  const extendsList = Array.isArray(extendsField)
+    ? extendsField
+    : typeof extendsField === "string"
+      ? [extendsField]
+      : [];
+  for (const ext of extendsList) {
+    if (typeof ext !== "string") continue;
+    const extPath = path.isAbsolute(ext)
+      ? ext
+      : path.resolve(path.dirname(resolved), ext);
+    if (asconfigDeclaresBindings(extPath, seen)) return true;
+  }
+  return false;
+}
 function asconfigDeclaresTryAs(configPath, seen = new Set()) {
   if (!configPath || configPath === "none") return false;
   const resolved = path.isAbsolute(configPath)

package/bin/commands/fuzz-core.js

@@ -13,6 +13,10 @@ import { persistCrashRecord } from "../crash-store.js";
 const DEFAULT_CONFIG_PATH = path.join(process.cwd(), "./as-test.config.json");
 const MAGIC = Buffer.from("WIPC");
 const HEADER_SIZE = 9;
+// See cli/wipc.ts: the magic can occur by chance in passthrough output, so a
+// declared length above this bound means the match is coincidental.
+const MAX_FRAME_SIZE = 16 * 1024 * 1024;
+const KNOWN_FRAME_TYPES = new Set([0x00, 0x01, 0x02, 0x03]);
 const MAX_DEFAULT_SEED = 0x7fffffff;
 export async function fuzz(
   configPath = DEFAULT_CONFIG_PATH,
@@ -135,7 +139,14 @@ async function runFuzzTarget(
   };
   const captured = captureFrames((type, payload, respond) => {
     if (type == 0x02) {
-      const event = JSON.parse(payload.toString("utf8"));
+      // A coincidental magic + CALL match whose payload is not JSON must not
+      // crash the run (it would be misreported as a fuzz crash); drop it.
+      let event;
+      try {
+        event = JSON.parse(payload.toString("utf8"));
+      } catch {
+        return;
+      }
       const kind = String(event.kind ?? "");
       if (kind == "fuzz:config") {
         const resolved = config;
@@ -419,6 +430,16 @@ function captureFrames(onFrame) {
       if (buffer.length < HEADER_SIZE) return true;
       const type = buffer.readUInt8(4);
       const length = buffer.readUInt32LE(5);
+      // A coincidental magic match in passthrough output: an unknown type or
+      // an implausible length means these 4 bytes are data, not a frame.
+      // Emit them and resync past the magic so we don't stall or misparse.
+      if (!KNOWN_FRAME_TYPES.has(type) || length > MAX_FRAME_SIZE) {
+        const raw = buffer.subarray(0, MAGIC.length);
+        passthrough = Buffer.concat([passthrough, raw]);
+        originalWrite(raw);
+        buffer = buffer.subarray(MAGIC.length);
+        continue;
+      }
       const frameSize = HEADER_SIZE + length;
       if (buffer.length < frameSize) return true;
       const payload = buffer.subarray(HEADER_SIZE, frameSize);