as-test 0.5.2 → 0.5.4

package/CHANGELOG.md

@@ -1,5 +1,59 @@
 # Change Log
 
+## 2026-03-04
+
+### CLI & Config Validation
+
+- feat: validate config shape and field types before applying defaults for all CLI entry points.
+- feat: reject unknown config keys with nearest-key suggestions.
+- feat: show structured validation diagnostics with JSON paths and fix hints.
+- feat: fail fast on invalid config JSON with parser error details.
+
+### Docs
+
+- docs: add README guidance for strict config validation behavior and example error output.
+
+## 2026-02-25 - v0.5.3
+
+### CLI, Modes & Matrix
+
+- feat: support mode fan-out behavior consistently across `ast build`, `ast run`, and `ast test`.
+- feat: when no `--mode` is provided and modes are configured, run using configured modes.
+- feat: add matrix-style per-file output with mode-aware timing:
+  - non-verbose: average time
+  - verbose: per-mode times.
+- feat: add real-time matrix line updates in the default reporter and normalize timing precision to one decimal.
+- feat: support comma-separated bare selectors (for example `ast test box,custom,generics,string`) across build/run/test selectors.
+
+### Config Merge & Env Behavior
+
+- fix: apply mode config as field-level merge over base config instead of replacing entire sections.
+- fix: merge `buildOptions.args` between base config and mode config.
+- fix: pass config env variables to both build and run processes for mode execution.
+
+### Build Pipeline & Feature Flags
+
+- feat: include the exact build command in build failure output.
+- feat: allow `buildOptions.cmd` to override default command generation while still appending user build args.
+- feat: support CLI feature toggles:
+  - `--enable coverage` / `--disable coverage`
+  - `--enable try-as` / `--disable try-as`.
+
+### Reporter & Summaries
+
+- fix: move mode summary rendering into the default reporter (via run-complete event payload).
+- feat: include `modeSummary` for single-mode runs.
+- feat: include mode and snapshot totals in final summary output.
+- fix: `--clean` output now behaves as non-TTY in default reporter:
+  - no in-place line editing,
+  - no suite expand/collapse logs,
+  - final per-file verdict lines only.
+
+### Coverage & Transform
+
+- fix: ignore AssemblyScript builtin/compiler helper calls during coverage instrumentation (including `isString`, `changetype<T>`, `idof<T>`, `sizeof<T>`).
+- fix: mock transform now collects mocked import targets across sources so WASI mock imports are rewritten reliably (resolves import-shape runtime failures).
+
 ## 2026-02-24 - v0.5.2
 
 ### Runtime & Serialization

package/README.md

@@ -9,11 +9,13 @@
 - [Installation](#installation)
 - [Examples](#examples)
 - [Writing Tests](#writing-tests)
+- [Setup Diagnostics](#setup-diagnostics)
 - [Mocking](#mocking)
 - [Snapshots](#snapshots)
 - [Coverage](#coverage)
 - [Custom Reporters](#custom-reporters)
 - [Assertions](#assertions)
+- [CLI Style Guide](#cli-style-guide)
 - [License](#license)
 - [Contact](#contact)
 
@@ -40,6 +42,11 @@ The installation script will set everything up for you:
 npx as-test init --dir ./path-to-install
 ```
 
+To scaffold and install dependencies in one step:
+```bash
+npx as-test init --dir ./path-to-install --install
+```
+
 Alternatively, you can install it manually:
 ```bash
 npm install as-test --save-dev
@@ -74,7 +81,7 @@ describe("math", () => {
 });
 ```
 
-### Test file selection (`ast test`)
+### File selection (`ast run`, `ast build`, `ast test`)
 
 No selectors:
 
@@ -105,6 +112,14 @@ Multiple selectors:
 ast test sleep array ./assembly/__tests__/snapshot.spec.ts
 ```
 
+Comma-separated bare suite names:
+
+```bash
+ast test box,custom,generics,string
+ast run box,custom,generics,string
+ast build box,custom,generics,string
+```
+
 If nothing matches, `ast test` exits non-zero with:
 
 ```text
@@ -114,11 +129,57 @@ No test files matched: ...
 ### Useful flags
 
 - `--config <path>`: use another config file
-- `--mode <name[,name...]>`: run one or multiple named config modes
+- `--mode <name[,name...]>`: run one or multiple named config modes (if omitted and `modes` is configured, as-test runs all configured modes)
 - `--update-snapshots`: write snapshot updates
 - `--no-snapshot`: disable snapshot assertions for the run
 - `--show-coverage`: print uncovered coverage points
+- `--enable <feature>`: enable as-test feature (`coverage`, `try-as`)
+- `--disable <feature>`: disable as-test feature (`coverage`, `try-as`)
 - `--verbose`: keep expanded suite/test lines and update running `....` statuses in place
+- `--clean`: disable in-place TTY updates and print only final per-file verdict lines. Useful for CI/CD.
+- `--list`: show resolved files, per-mode artifacts, and runtime command without executing
+- `--list-modes`: show configured and selected modes without executing
+- `--help` / `-h`: show command-specific help (`ast test --help`, `ast init --help`, etc.)
+
+Example:
+
+```bash
+ast build --enable try-as
+ast test --disable coverage
+```
+
+Preview execution plan:
+
+```bash
+ast test --list
+ast test --list-modes
+ast run sleep --list --mode wasi
+ast build --list --mode wasi,bindings
+```
+
+## Setup Diagnostics
+
+Use `ast doctor` to validate local setup before running tests.
+
+```bash
+ast doctor
+```
+
+You can also target specific modes and config files:
+
+```bash
+ast doctor --config ./as-test.config.json --mode wasi,bindings
+```
+
+`doctor` checks:
+
+- config file loading and mode resolution
+- required dependencies (for example `assemblyscript`, `@assemblyscript/wasi-shim` for WASI targets)
+- runtime command parsing and executable availability
+- runtime script path existence (for script-host runtimes)
+- test spec file discovery from configured input patterns
+
+If any `ERROR` checks are found, `ast doctor` exits non-zero.
 
 ## Mocking
 
@@ -230,13 +291,12 @@ Example:
 {
   "$schema": "./as-test.config.schema.json",
   "input": ["./assembly/__tests__/*.spec.ts"],
-  "outDir": "./.as-test/build",
-  "logs": "./.as-test/logs",
-  "coverageDir": "./.as-test/coverage",
-  "snapshotDir": "./.as-test/snapshots",
+  "output": "./.as-test/",
   "config": "none",
   "coverage": true,
+  "env": {},
   "buildOptions": {
+    "cmd": "",
     "args": [],
     "target": "wasi"
   },
@@ -253,15 +313,38 @@ Example:
 Key fields:
 
 - `input`: glob list of spec files
+- `output`: output alias. Use a root string (`"./.as-test/"`) or object (`{ "build": "...", "logs": "...", "coverage": "...", "snapshots": "..." }`)
 - `outDir`: compiled wasm output dir
 - `logs`: log output dir or `"none"`
 - `coverageDir`: coverage output dir or `"none"`
 - `snapshotDir`: snapshot storage dir
+- `outDir`, `logs`, `coverageDir`, and `snapshotDir` still work; when both are set, these explicit fields override `output`
+- `env`: environment variables injected into build and runtime processes
+- `buildOptions.cmd`: optional custom build command template; when set it replaces default build command and flags. Supports `<file>`, `<name>`, `<outFile>`, `<target>`, `<mode>`
 - `buildOptions.target`: `wasi` or `bindings`
-- `modes`: named overrides for target/args/runtime/env/artifact directories (selected via `--mode`)
+- `modes`: named overrides for command/target/args/runtime/env/artifact directories (selected via `--mode`); `mode.env` overrides top-level `env`
 - `runOptions.runtime.cmd`: runtime command, supports `<file>` and `<name>`; if its script path is missing, as-test falls back to the default runner for the selected target
 - `runOptions.reporter`: reporter selection as a string or object
 
+Validation behavior:
+
+- Config parsing is strict for `ast build`, `ast run`, `ast test`, and `ast doctor`.
+- Invalid JSON fails early with parser details (`line`/`column` when provided by Node).
+- Unknown properties are rejected and include a nearest-key suggestion when possible.
+- Invalid property types are reported with their JSON path and a short fix hint.
+- On validation failure, the command exits non-zero and prints `run "ast doctor" to check your setup.`
+
+Example validation error:
+
+```text
+invalid config at ./as-test.config.json
+1. $.inpoot: unknown property
+     fix: use "input" if that was intended, otherwise remove this property
+2. $.runOptions.runtime.cmd: must be a string
+     fix: set to a runtime command including "<file>"
+run "ast doctor" to check your setup.
+```
+
 Example multi-runtime matrix:
 
 ```json
@@ -309,6 +392,13 @@ Run all modes:
 ast test --mode wasi-simd,wasi-nosimd,bindings-node-simd
 ```
 
+Summary totals:
+
+- `Modes` in the default reporter is config-scoped (`total` is all configured modes)
+- when selecting fewer modes with `--mode`, unselected modes are counted as `skipped`
+- `Files` in the default reporter is also config-scoped (`total` is all files from configured input patterns)
+- when selecting fewer files, unselected files are counted as `skipped`
+
 When using `--mode`, compiled artifacts are emitted as:
 
 ```text
@@ -472,7 +562,7 @@ Available matchers:
 - `toStartWith(prefix)`
 - `toEndWith(suffix)`
 - `toHaveLength(length)`
-- `toContain(item)`
+- `toContain(itemOrSubstring)` (`toContains` alias supported)
 - `toThrow()` (with `try-as`)
 - `toMatchSnapshot(name?)`
 

package/as-test.config.schema.json

@@ -17,6 +17,37 @@
       },
       "default": ["./assembly/__tests__/*.spec.ts"]
     },
+    "output": {
+      "description": "Output alias. Use a root string or a per-artifact object. Legacy fields (outDir/logs/coverageDir/snapshotDir) still work and override this alias when both are set.",
+      "oneOf": [
+        {
+          "type": "string",
+          "description": "Root output directory. Expands to build/logs/coverage/snapshots subdirectories."
+        },
+        {
+          "type": "object",
+          "additionalProperties": false,
+          "properties": {
+            "build": {
+              "type": "string",
+              "description": "Build artifact output directory (maps to outDir)."
+            },
+            "logs": {
+              "type": "string",
+              "description": "Log artifact output directory (maps to logs). Use \"none\" to disable."
+            },
+            "coverage": {
+              "type": "string",
+              "description": "Coverage artifact output directory (maps to coverageDir). Use \"none\" to disable."
+            },
+            "snapshots": {
+              "type": "string",
+              "description": "Snapshot output directory (maps to snapshotDir)."
+            }
+          }
+        }
+      ]
+    },
     "outDir": {
       "type": "string",
       "description": "Directory where compiled artifacts are written.",
@@ -80,10 +111,23 @@
       ],
       "default": true
     },
+    "env": {
+      "type": "object",
+      "description": "Environment variables injected when building/running.",
+      "additionalProperties": {
+        "type": "string"
+      },
+      "default": {}
+    },
     "buildOptions": {
       "type": "object",
       "additionalProperties": false,
       "properties": {
+        "cmd": {
+          "type": "string",
+          "description": "Custom build command template. When set, this replaces the default asc command/flags/output handling. Supports placeholders: <file>, <name>, <outFile>, <target>, <mode>.",
+          "default": ""
+        },
         "args": {
           "type": "array",
           "items": {
@@ -98,6 +142,7 @@
         }
       },
       "default": {
+        "cmd": "",
         "args": [],
         "target": "wasi"
       }
@@ -153,6 +198,10 @@
             "type": "object",
             "additionalProperties": false,
             "properties": {
+              "cmd": {
+                "type": "string",
+                "description": "Mode-specific custom build command template."
+              },
               "args": {
                 "type": "array",
                 "items": {

package/assembly/coverage.ts

@@ -23,6 +23,26 @@ export function __REGISTER(point: CoverPoint): void {
   Coverage.SN.hashes.set(point.hash, point);
 }
 
+export function __REGISTER_RAW(
+  file: string,
+  hash: string,
+  line: i32,
+  column: i32,
+  type: string,
+): void {
+  if (Coverage.SN.allIndex.has(hash)) return;
+  const point = new CoverPoint();
+  point.file = file;
+  point.hash = hash;
+  point.line = line;
+  point.column = column;
+  point.type = type;
+  Coverage.SN.points++;
+  Coverage.SN.allIndex.set(hash, Coverage.SN.all.length);
+  Coverage.SN.all.push(point);
+  Coverage.SN.hashes.set(hash, point);
+}
+
 export function __COVER(hash: string): void {
   if (Coverage.SN.allIndex.has(hash)) {
     const index = Coverage.SN.allIndex.get(hash);

package/assembly/src/expectation.ts

@@ -334,18 +334,41 @@ export class Expectation<T> extends Tests {
   }
 
   /**
-   * Tests if an array contains an element
+   * Tests if an array or string contains a value
    */
   // @ts-ignore
   toContain(value: valueof<T>): void {
-    // @ts-ignore
-    const passed = isArray<T>() && this._left.includes(value);
-    this._resolve(
-      passed,
-      "toContain",
-      q("includes value"),
-      q("does not include value"),
-    );
+    if (isString<T>()) {
+      // @ts-ignore
+      const left = this._left as string;
+      // @ts-ignore
+      const needle = value as string;
+      const passed = left.indexOf(needle) >= 0;
+      this._resolve(passed, "toContain", q(left), q(needle));
+      return;
+    }
+
+    if (isArray<T>()) {
+      // @ts-ignore
+      const passed = this._left.includes(value);
+      this._resolve(
+        passed,
+        "toContain",
+        stringifyValue<T>(this._left),
+        stringifyValue<valueof<T>>(value),
+      );
+      return;
+    }
+
+    ERROR("toContain() can only be used on string and array types!");
+  }
+
+  /**
+   * Alias for toContain().
+   */
+  // @ts-ignore
+  toContains(value: valueof<T>): void {
+    this.toContain(value);
   }
 
   /**

package/assembly/util/json.ts

@@ -18,17 +18,16 @@ export function stringifyValue<T>(value: T): string {
   }
 
   if (isInteger<T>() || isFloat<T>()) {
-    // @ts-ignore
+    // @ts-expect-error: type
     return value.toString();
   }
 
   if (isString<T>()) {
-    // @ts-ignore
     return quote(value as string);
   }
 
   if (isArray<T>()) {
-    // @ts-ignore
+    // @ts-expect-error: type
     return stringifyArray<valueof<T>>(value as valueof<T>[]);
   }
 

package/bin/commands/build-core.js

@@ -0,0 +1,268 @@
+import { existsSync } from "fs";
+import { glob } from "glob";
+import chalk from "chalk";
+import { spawnSync } from "child_process";
+import * as path from "path";
+import { applyMode, getPkgRunner, loadConfig, tokenizeCommand, } from "../util.js";
+const DEFAULT_CONFIG_PATH = path.join(process.cwd(), "./as-test.config.json");
+export async function build(configPath = DEFAULT_CONFIG_PATH, selectors = [], modeName, featureToggles = {}) {
+    const loadedConfig = loadConfig(configPath, false);
+    const mode = applyMode(loadedConfig, modeName);
+    const config = mode.config;
+    if (!hasCustomBuildCommand(config)) {
+        ensureDeps(config);
+    }
+    const pkgRunner = getPkgRunner();
+    const inputPatterns = resolveInputPatterns(config.input, selectors);
+    const inputFiles = (await glob(inputPatterns)).sort((a, b) => a.localeCompare(b));
+    const duplicateSpecBasenames = await resolveDuplicateSpecBasenames(config.input);
+    const coverageEnabled = resolveCoverageEnabled(config.coverage, featureToggles.coverage);
+    const buildEnv = {
+        ...mode.env,
+        AS_TEST_COVERAGE_ENABLED: coverageEnabled ? "1" : "0",
+    };
+    for (const file of inputFiles) {
+        const outFile = `${config.outDir}/${resolveArtifactFileName(file, config.buildOptions.target, modeName, duplicateSpecBasenames)}`;
+        const invocation = getBuildCommand(config, pkgRunner, file, outFile, modeName, featureToggles);
+        try {
+            buildFile(invocation, buildEnv);
+        }
+        catch (error) {
+            const modeLabel = modeName ?? "default";
+            throw new Error(`Failed to build ${path.basename(file)} in mode ${modeLabel} with ${getBuildStderr(error)}\nBuild command: ${formatInvocation(invocation)}`);
+        }
+    }
+}
+function hasCustomBuildCommand(config) {
+    return !!config.buildOptions.cmd.trim().length;
+}
+function getBuildCommand(config, pkgRunner, file, outFile, modeName, featureToggles = {}) {
+    const userArgs = getUserBuildArgs(config);
+    if (hasCustomBuildCommand(config)) {
+        const tokens = tokenizeCommand(expandBuildCommand(config.buildOptions.cmd, file, outFile, config.buildOptions.target, modeName));
+        if (!tokens.length) {
+            throw new Error("custom build command is empty");
+        }
+        return {
+            command: tokens[0],
+            args: [...tokens.slice(1), ...userArgs],
+        };
+    }
+    const defaultArgs = getDefaultBuildArgs(config, featureToggles);
+    const args = ["asc", file, ...userArgs, ...defaultArgs];
+    if (config.outDir.length) {
+        args.push("-o", outFile);
+    }
+    return {
+        command: pkgRunner,
+        args,
+    };
+}
+function getUserBuildArgs(config) {
+    return config.buildOptions.args.filter((value) => value.length > 0);
+}
+function expandBuildCommand(template, file, outFile, target, modeName) {
+    const name = path
+        .basename(file)
+        .replace(/\.spec\.ts$/, "")
+        .replace(/\.ts$/, "");
+    return template
+        .replace(/<file>/g, file)
+        .replace(/<name>/g, name)
+        .replace(/<outFile>/g, outFile)
+        .replace(/<target>/g, target)
+        .replace(/<mode>/g, modeName ?? "");
+}
+function resolveArtifactFileName(file, target, modeName, duplicateSpecBasenames = new Set()) {
+    const base = path
+        .basename(file)
+        .replace(/\.spec\.ts$/, "")
+        .replace(/\.ts$/, "");
+    const legacy = !modeName
+        ? `${path.basename(file).replace(".ts", ".wasm")}`
+        : `${base}.${modeName}.${target}.wasm`;
+    if (!duplicateSpecBasenames.has(path.basename(file))) {
+        return legacy;
+    }
+    const disambiguator = resolveDisambiguator(file);
+    if (!disambiguator.length) {
+        return legacy;
+    }
+    const ext = path.extname(legacy);
+    const stem = ext.length ? legacy.slice(0, -ext.length) : legacy;
+    return `${stem}.${disambiguator}${ext}`;
+}
+async function resolveDuplicateSpecBasenames(configured) {
+    const patterns = Array.isArray(configured) ? configured : [configured];
+    const files = await glob(patterns);
+    const counts = new Map();
+    for (const file of files) {
+        const base = path.basename(file);
+        counts.set(base, (counts.get(base) ?? 0) + 1);
+    }
+    const duplicates = new Set();
+    for (const [base, count] of counts) {
+        if (count > 1)
+            duplicates.add(base);
+    }
+    return duplicates;
+}
+function resolveDisambiguator(file) {
+    const relDir = path.dirname(path.relative(process.cwd(), file));
+    if (!relDir.length || relDir == ".")
+        return "";
+    return relDir
+        .replace(/[\\/]+/g, "__")
+        .replace(/[^A-Za-z0-9._-]/g, "_")
+        .replace(/^_+|_+$/g, "");
+}
+function resolveInputPatterns(configured, selectors) {
+    const configuredInputs = Array.isArray(configured)
+        ? configured
+        : [configured];
+    if (!selectors.length)
+        return configuredInputs;
+    const patterns = new Set();
+    for (const selector of expandSelectors(selectors)) {
+        if (!selector)
+            continue;
+        if (isBareSuiteSelector(selector)) {
+            const base = stripSuiteSuffix(selector);
+            for (const configuredInput of configuredInputs) {
+                patterns.add(path.join(path.dirname(configuredInput), `${base}.spec.ts`));
+            }
+            continue;
+        }
+        patterns.add(selector);
+    }
+    return [...patterns];
+}
+function expandSelectors(selectors) {
+    const expanded = [];
+    for (const selector of selectors) {
+        if (!selector)
+            continue;
+        if (!shouldSplitSelector(selector)) {
+            expanded.push(selector);
+            continue;
+        }
+        for (const token of selector.split(",")) {
+            const trimmed = token.trim();
+            if (!trimmed.length)
+                continue;
+            expanded.push(trimmed);
+        }
+    }
+    return expanded;
+}
+function shouldSplitSelector(selector) {
+    return (selector.includes(",") &&
+        !selector.includes("/") &&
+        !selector.includes("\\") &&
+        !/[*?[\]{}]/.test(selector));
+}
+function isBareSuiteSelector(selector) {
+    return (!selector.includes("/") &&
+        !selector.includes("\\") &&
+        !/[*?[\]{}]/.test(selector));
+}
+function stripSuiteSuffix(selector) {
+    return selector.replace(/\.spec\.ts$/, "").replace(/\.ts$/, "");
+}
+function ensureDeps(config) {
+    if (config.buildOptions.target == "wasi") {
+        if (!existsSync("./node_modules/@assemblyscript/wasi-shim/asconfig.json")) {
+            console.log(`${chalk.bgRed(" ERROR ")}${chalk.dim(":")} could not find @assemblyscript/wasi-shim! Add it to your dependencies to run with WASI!`);
+            process.exit(1);
+        }
+    }
+}
+function buildFile(invocation, env) {
+    const result = spawnSync(invocation.command, invocation.args, {
+        stdio: ["ignore", "pipe", "pipe"],
+        encoding: "utf8",
+        env,
+        shell: false,
+    });
+    if (result.error)
+        throw result.error;
+    if (result.status !== 0) {
+        const error = new Error(result.stderr?.trim() ||
+            result.stdout?.trim() ||
+            `command exited with code ${result.status}`);
+        error.stderr = result.stderr ?? "";
+        throw error;
+    }
+}
+function formatInvocation(invocation) {
+    return [invocation.command, ...invocation.args]
+        .map((token) => (/\s/.test(token) ? JSON.stringify(token) : token))
+        .join(" ");
+}
+function getBuildStderr(error) {
+    const err = error;
+    const stderr = err?.stderr;
+    if (typeof stderr == "string") {
+        const trimmed = stderr.trim();
+        if (trimmed.length)
+            return trimmed;
+    }
+    else if (stderr instanceof Buffer) {
+        const trimmed = stderr.toString("utf8").trim();
+        if (trimmed.length)
+            return trimmed;
+    }
+    const message = typeof err?.message == "string" ? err.message.trim() : "";
+    return message || "unknown error";
+}
+function getDefaultBuildArgs(config, featureToggles) {
+    const buildArgs = [];
+    const tryAsEnabled = resolveTryAsEnabled(featureToggles.tryAs);
+    buildArgs.push("--transform", "as-test/transform");
+    if (tryAsEnabled) {
+        buildArgs.push("--transform", "try-as/transform");
+    }
+    if (config.config && config.config !== "none") {
+        buildArgs.push("--config", config.config);
+    }
+    if (tryAsEnabled) {
+        buildArgs.push("--use", "AS_TEST_TRY_AS=1");
+    }
+    // Should also strip any bindings-enabling from asconfig
+    if (config.buildOptions.target == "bindings") {
+        buildArgs.push("--use", "AS_TEST_BINDINGS=1", "--bindings", "raw", "--exportRuntime", "--exportStart", "_start");
+    }
+    else if (config.buildOptions.target == "wasi") {
+        buildArgs.push("--use", "AS_TEST_WASI=1", "--config", "./node_modules/@assemblyscript/wasi-shim/asconfig.json");
+    }
+    else {
+        console.log(`${chalk.bgRed(" ERROR ")}${chalk.dim(":")} could not determine target in config! Set target to 'bindings' or 'wasi'`);
+        process.exit(1);
+    }
+    return buildArgs;
+}
+function resolveTryAsEnabled(override) {
+    const installed = hasTryAsRuntime();
+    if (override === false)
+        return false;
+    if (override === true && !installed) {
+        throw new Error('try-as feature was enabled, but package "try-as" is not installed');
+    }
+    return installed;
+}
+function resolveCoverageEnabled(rawCoverage, override) {
+    if (override != undefined)
+        return override;
+    if (typeof rawCoverage == "boolean")
+        return rawCoverage;
+    if (rawCoverage && typeof rawCoverage == "object") {
+        const enabled = rawCoverage.enabled;
+        if (typeof enabled == "boolean")
+            return enabled;
+    }
+    return true;
+}
+function hasTryAsRuntime() {
+    return (existsSync(path.join(process.cwd(), "node_modules/try-as")) ||
+        existsSync(path.join(process.cwd(), "node_modules/try-as/package.json")));
+}