@exodus/test 1.0.0-rc.51 → 1.0.0-rc.52

package/README.md

@@ -2,91 +2,94 @@
 
 A runner for `node:test`, `jest`, and `tape` test suites on top of `node:test` (and any runtime)
 
-Most likely it will just work on your simple jest tests as as drop-in replacement
-
-Comes with typescript support, optional esm/cjs interop, and also loading babel transforms!
-
-Use `--coverage` to generate coverage output
-
-Default `NODE_ENV` value is "test", use `NODE_ENV=` to override (e.g. to empty)
-
-## Why?
-
-- Can run your tests on Node.js, Bun, Deno, JavaScriptCore and Hermes without extra churn
-
-- Unlike `jest`, it is fast
-
-- Unlike `node:test`, it is a drop-in replacement for `jest`
-
-  - With `expect`, support for snapshots, mocks and matchers
-
-  - `jest-when` and `jest-extended` are fully compatible and can just be used
-
-  - Snapshots are compatible with Jest and can just be used both ways
-
-  - Also compatible to `node:test`
-
-- Unlike `bun:test`, it runs all test files in isolated contexts
-
-  Bun leaks globals / side effects between test files and has incompatible `test()` lifecycle / order
-
-- Can use Jest config
-
-- Native coverage support (enable via `--coverage`)
-
-- Can record / replay `fetch` and `WebSocket` sessions. And run them on all runtimes (including Hermes)
-
-- Automatic polyfills for JavaScriptCore / Hermes, including crypto
-
-- Hanging tests error by default (unlike `jest`)
-
-- Native ESM out of the box
+## Features
 
+- Native ESM, including in Jest tests
 - Esbuild on the fly for babelified ESM interop (enable via `--esbuild`)
-
-- TypeScript support in both transform (enable via `--esbuild`) and typestrip (via `--typescript`) modes
-
-- Babel support, picks up your Babel config (enable via `--babel`)
-
+- TypeScript support in both transform (through [tsx](https://tsx.is/), enable via `--esbuild`)
+  and typestrip (via `--typescript`) modes
+- Runs on [node:test](https://nodejs.org/api/test.html), and (experimental) on bun, deno, d8, JSC and
+  [Hermes](https://hermesengine.dev)
+- Testsuite-agnostic -- can run any file as long as it sets exit code based on test results
+- Built-in [Jest](https://jestjs.io) compatibility (with `--jest`), including `jest.*` global
+  - Up to ~10x faster depending on the original setup
+  - Actual `expect` module, also `jest-extended` and `jest-when` just work on top
+  - Snapshots, including snapshot matchers
+  - Function and timer mocks
+  - [test.concurrent]()
+  - Module mocks (on top of Node.js runtime only), including for ESM modules
+  - Loads Jest configuration
+  - It works on Hermes too!
+- Built-in network record/replay for offline tests, mocking `fetch` and `WebSocket` sessions
 - `--drop-network` support for guaranteed offline testing
-
-## Library
-
-### Using with `node:test` natively
-
-You can just use pure [`node:test`](https://nodejs.org/api/test.html) in your tests,
-this runner is fully compatible with that (and will set version-specific options for you)!
-
-### Moving from jest
-
-```js
-import {
-  jest,
-  expect,
-  describe,
-  it,
-  beforeEach,
-  afterEach,
-  beforeAll,
-  afterAll,
-} from '@exodus/test/jest'
-```
-
-Or, run with [`--jest` option](#options) to register jest globals
-
-### Moving from tap/tape
-
-```js
-import test from '@exodus/test/tape'
+- Native code coverage via v8 (Node.js or [c8](https://github.com/bcoe/c8)), with istanbul reporters
+- GitHub reporter (auto-enabled by default)
+- JSDOM env support
+- Hanging tests error by default (unlike `jest`)
+- Babel support, picks up your Babel config (enable via `--babel`)
+- Unlike `bun:test`, it runs test files in isolated contexts \
+  Bun leaks globals / side effects between test files ([ref](https://github.com/oven-sh/bun/issues/6024)),
+  and has incompatible `test()` lifecycle / order
+- Also features a tape API for drop-in replacement
+
+## Reporter samples
+
+#### CLI (but uses colors when output supports them, e.g. in terminal):
+
+```console
+# tests/jest/expect.mock.test.js
+✔ PASS drinkAll > drinks something lemon-flavoured (1.300417ms)
+✔ PASS drinkAll > does not drink something octopus-flavoured (0.191791ms)
+✔ PASS drinkAll (1.842959ms)
+✔ PASS drinkEach > drinkEach drinks each drink (0.360625ms)
+✔ PASS drinkEach (0.463416ms)
+✔ PASS toHaveBeenCalledWith > registration applies correctly to orange La Croix (0.53325ms)
+✔ PASS toHaveBeenCalledWith (0.564166ms)
+✔ PASS toHaveBeenLastCalledWith > applying to all flavors does mango last (0.380375ms)
+✔ PASS toHaveBeenLastCalledWith (0.473417ms)
+# tests/jest/fn.invocationCallOrder.test.js
+✔ PASS mock.invocationCallOrder (4.221042ms)
 ```
 
-### Running tests asynchronously
+#### GitHub Actions collapses test results per-file, like this:
+
+<details>
+ <summary>✅ <strong>tests/jest/lifecycle.test.js</strong></summary>
+ <pre>
+  ✔ PASS A > B > C (3.26166ms)
+  ✔ PASS A > B > D (1.699463ms)
+  ✔ PASS A > B (6.72719ms)
+  ✔ PASS A > E > F (1.117997ms)
+  ✔ PASS A > E > G > H (1.330904ms)
+  ✔ PASS A > E > G (1.94971ms)
+  ✔ PASS A > E (3.821825ms)
+  ✔ PASS A > I (0.533096ms)
+  ✔ PASS A (13.887889ms)
+  ✔ PASS J (0.373187ms)
+  ✔ PASS K > L (0.659852ms)
+  ✔ PASS K (1.143195ms)
+ </pre>
+</details><details>
+ <summary>✅ <strong>tests/jest/timers.async.test.js</strong></summary>
+ <pre>
+  ✔ PASS advanceTimersByTime() does not let microtasks to pass (5.326604ms)
+  ✔ PASS advanceTimersByTime() does not let microtasks to pass even with await (1.336064ms)
+  ✔ PASS advanceTimersByTimeAsync() lets microtasks to pass (6.99526ms)
+  ✔ PASS advanceTimersByTimeAsync() lets microtasks to pass, chained (10.131664ms)
+  ✔ PASS advanceTimersByTimeAsync() lets microtasks to pass, longer chained (8.635472ms)
+  ✔ PASS advanceTimersByTimeAsync() lets microtasks to pass, async chain (56.937983ms)
+ </pre>
+</details>
+
+See live output in [CI](https://github.com/ExodusMovement/test/actions/workflows/checks.yaml)
 
-Add `{ concurrency: true }`, like this: `describe('my testsuite', { concurrency: true }, () => {`
+## Library
 
 ### List of exports
 
-- `@exodus/test/jest` -- `jest` mock
+- `@exodus/test/node` -- `node:test` API, working under non-Node.js platforms
+
+- `@exodus/test/jest` -- `jest` implementation
 
 - `@exodus/test/tape` -- `tape` mock (can also be helpful when moving from `tap`)
 

package/bin/index.js

@@ -16,9 +16,7 @@ import glob from 'fast-glob'
 import { haveModuleMocks, haveSnapshots, haveForceExit } from '../src/version.js'
 
 const bindir = dirname(fileURLToPath(import.meta.url))
-
-const EXTS = `.?([cm])[jt]s?(x)` // we differ from jest, allowing [cm] before everything
-const DEFAULT_PATTERNS = [`**/__tests__/**/*${EXTS}`, `**/?(*.)+(spec|test)${EXTS}`]
+const DEFAULT_PATTERNS = [`**/?(*.)+(spec|test).?([cm])[jt]s?(x)`] // do not trust magic dirs by default
 
 const bundleOptions = { pure: true, bundle: true, esbuild: true, ts: 'auto' }
 const hermesAv = ['-Og', '-Xmicrotask-queue']
@@ -191,10 +189,18 @@ const { options, patterns } = parseOptions()
 const warnHuman = isTTY && !isCI ? (...args) => console.warn(...args) : () => {}
 if (isCI) process.env.FORCE_COLOR = '1' // should support colored output even though not a TTY, overridable with --no-color
 
+const setEnv = (name, value) => {
+  const env = process.env[name]
+  if (env && env !== value) throw new Error(`env conflict: ${name}="${env}", effective: "${value}"`)
+  process.env[name] = value
+}
+
 const engineOptions = ENGINES.get(options.engine)
 assert(engineOptions, `Unknown engine: ${options.engine}`)
 Object.assign(options, engineOptions)
 options.platform = options.binary // binary can be overriden by c8
+setEnv('EXODUS_TEST_ENGINE', options.engine) // e.g. 'hermes:bundle', 'node:bundle', 'node:test', 'node:pure'
+setEnv('EXODUS_TEST_PLATFORM', options.binary) // e.g. 'hermes', 'node'
 
 const require = createRequire(import.meta.url)
 const resolveRequire = (query) => require.resolve(query)
@@ -406,15 +412,6 @@ if (!options.bundle) {
 }
 
 if (!Object.hasOwn(process.env, 'NODE_ENV')) process.env.NODE_ENV = 'test'
-
-const setEnv = (name, value) => {
-  const env = process.env[name]
-  if (env && env !== value) throw new Error(`env conflict: ${name}="${env}", effective: "${value}"`)
-  process.env[name] = value
-}
-
-setEnv('EXODUS_TEST_PLATFORM', options.binary) // e.g. 'hermes', 'node'
-setEnv('EXODUS_TEST_ENGINE', options.engine) // e.g. 'hermes:bundle', 'node:bundle', 'node:test', 'node:pure'
 setEnv('EXODUS_TEST_ONLY', options.only ? '1' : '')
 
 const c8 = resolveRequire('c8/bin/c8.js')

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@exodus/test",
-  "version": "1.0.0-rc.51",
+  "version": "1.0.0-rc.52",
   "author": "Exodus Movement, Inc.",
   "description": "A test suite runner",
   "homepage": "https://github.com/ExodusMovement/test",
@@ -64,6 +64,7 @@
     "src/engine.js",
     "src/engine.node.cjs",
     "src/engine.pure.cjs",
+    "src/engine.pure.snapshot.cjs",
     "src/engine.select.cjs",
     "src/jest.js",
     "src/jest.config.js",
@@ -87,8 +88,9 @@
     "test:typescript": "./bin/index.js --jest --typescript tests/typescript.test.ts",
     "test:jest": "./bin/index.js --jest --esbuild",
     "test:tape": "./bin/index.js --esbuild 'tests/tape/tests/*.js' tests/tape.test.js",
-    "test:pure": "EXODUS_TEST_ENGINE=node:pure npm run test",
-    "test:bun:pure": "EXODUS_TEST_ENGINE=bun:pure npm run test",
+    "test:pure": "EXODUS_TEST_ENGINE=node:pure EXODUS_TEST_IGNORE='tests/jest-extended/**' npm run test",
+    "test:bun": "EXODUS_TEST_ENGINE=bun:pure EXODUS_TEST_IGNORE='tests/jest-extended/**' npm run test",
+    "test:hermes": "EXODUS_TEST_ENGINE=hermes:bundle EXODUS_TEST_IGNORE='tests/{{jest-extended,inband}/**,jest-when/when.test.*,jest/jest.{mock,resetModules}.*}' npm run test",
     "test:fetch": "./bin/index.js --jest --drop-network --engine node:pure tests/fetch.test.js tests/websocket.test.js",
     "coverage": "./bin/index.js --jest --esbuild --coverage",
     "lint": "prettier --list-different . && eslint .",
@@ -137,6 +139,7 @@
     "@types/jest-when": "^3.5.2",
     "@typescript-eslint/eslint-plugin": "^7.15.0",
     "eslint": "^8.44.0",
+    "hermes-engine-cli": "^0.12.0",
     "jest": "^29.7.0",
     "jest-matcher-utils": "^29.7.0",
     "jest-serializer-ansi-escapes": "^3.0.0",

package/src/engine.pure.cjs

@@ -1,5 +1,6 @@
 const assert = require('node:assert/strict')
 const assertLoose = require('node:assert')
+const { matchSnapshot } = require('./engine.pure.snapshot.cjs')
 
 const { setTimeout, setInterval, setImmediate, Date } = globalThis
 const { clearTimeout, clearInterval, clearImmediate } = globalThis
@@ -28,17 +29,21 @@ class Context {
   children = []
   assert = { ...assertLoose, snapshot: undefined }
   hooks = { __proto__: null, before: [], after: [], beforeEach: [], afterEach: [] }
+  #fullName
 
   constructor(parent, name, options = {}) {
     Object.assign(this, { root: parent?.root, parent, name, options })
-    this.fullName = parent && parent !== parent.root ? `${parent.fullName} > ${name}` : name
-    if (this.fullName === name) this.fullName = this.fullName.replace(INBAND_PREFIX_REGEX, '')
+    this.#fullName = parent && parent !== parent.root ? `${parent.fullName} > ${name}` : name
+    if (this.#fullName === name) this.#fullName = this.#fullName.replace(INBAND_PREFIX_REGEX, '')
     if (this.root) {
       this.parent.children.push(this)
     } else {
       assert(this.name === '<root>' && !this.parent)
       this.root = this
     }
+
+    this.assert.snapshot = (obj) =>
+      matchSnapshot(readSnapshot, assert, this.fullName, serializeSnapshot(obj))
   }
 
   get onlySomewhere() {
@@ -48,6 +53,10 @@ class Context {
   get only() {
     return (this.options.only && !this.children.some((x) => x.onlySomewhere)) || this.parent?.only
   }
+
+  get fullName() {
+    return this.#fullName
+  }
 }
 
 function enterContext(name, options) {
@@ -466,7 +475,17 @@ if (process.env.EXODUS_TEST_ENVIRONMENT === 'bundle') {
 
 // eslint-disable-next-line no-undef
 let snapshotResolver = (dir, name) => [dir, `${name}.snapshot`] // default per Node.js docs
-const setSnapshotSerializers = () => {}
+let snapshotSerializers = [(obj) => JSON.stringify(obj, null, 2)]
+const serializeSnapshot = (obj) => {
+  let val = obj
+  for (const fn of snapshotSerializers) val = fn(val)
+  return val
+}
+
+const setSnapshotSerializers = ([...arr]) => {
+  snapshotSerializers = arr
+}
+
 const setSnapshotResolver = (fn) => {
   snapshotResolver = fn
 }

package/src/engine.pure.snapshot.cjs

@@ -0,0 +1,35 @@
+const nameCounts = new Map()
+let snapshotText
+
+const escapeSnapshot = (str) => str.replaceAll(/([\\`])/gu, '\\$1')
+
+function matchSnapshot(readSnapshot, assert, name, serialized) {
+  // We don't have native snapshots, polyfill reading
+  if (snapshotText !== null) {
+    try {
+      const snapshotRaw = readSnapshot()
+      snapshotText = snapshotRaw ? `\n${snapshotRaw}\n` : null // we'll search wrapped in \n
+    } catch {
+      snapshotText = null
+    }
+  }
+
+  const addFail = `Adding new snapshots requires Node.js >=22.3.0`
+
+  // We don't support polyfilled snapshot generation here, only parsing
+  // Also be careful with assertion plan counters
+  if (!snapshotText) assert.fail(`Could not find snapshot file. ${addFail}`)
+
+  const count = (nameCounts.get(name) || 0) + 1
+  nameCounts.set(name, count)
+  const escaped = escapeSnapshot(serialized)
+  const key = `${name} ${count}`
+  const makeEntry = (x) => `\nexports[\`${escapeSnapshot(key)}\`] = \`${x}\`;\n`
+  const final = escaped.includes('\n') ? `\n${escaped}\n` : escaped
+  if (snapshotText.includes(makeEntry(final))) return
+  // Perhaps wrapped with newlines from Node.js snapshots?
+  if (!final.includes('\n') && snapshotText.includes(makeEntry(`\n${final}\n`))) return
+  return assert.fail(`Could not match "${key}" in snapshot. ${addFail}`)
+}
+
+module.exports = { escapeSnapshot, matchSnapshot }

package/src/jest.config.js

@@ -2,7 +2,9 @@
 import assert from 'node:assert/strict'
 import { specialEnvironments } from './jest.environment.js'
 
+const EXTS = `.?([cm])[jt]s?(x)` // we differ from jest, allowing [cm] before everything
 const normalizeJestConfig = (config) => ({
+  testMatch: [`**/__tests__/**/*${EXTS}`, `**/?(*.)+(spec|test)${EXTS}`],
   testEnvironment: 'node',
   testTimeout: 5000,
   testPathIgnorePatterns: [],

package/src/jest.snapshot.js

@@ -11,6 +11,7 @@ import { format, plugins as builtinPlugins } from 'pretty-format'
 import { jestConfig } from './jest.config.js'
 import { getTestNamePath } from './dark.cjs'
 import { haveSnapshotsReportUnescaped } from './version.js'
+import { matchSnapshot, escapeSnapshot } from './engine.pure.snapshot.cjs'
 
 const { snapshotFormat, snapshotSerializers } = jestConfig()
 const plugins = Object.values(builtinPlugins)
@@ -19,10 +20,6 @@ const serialize = (val) => format(val, { ...snapshotFormat, plugins }).replaceAl
 let serializersAreSetup = false
 let snapshotsAreJest = false
 
-// For manually loading the snapshot
-const nameCounts = new Map()
-let snapshotText
-
 function maybeSetupSerializers() {
   if (serializersAreSetup) return
   // empty require and serializers should not let this fail, non-empty serializers and empty require should
@@ -48,6 +45,7 @@ const wrap = (check) => {
   }
 }
 
+// Older Node.js versions do not have context.assert, which we want to prefer for counting
 let context
 beforeEach((t) => (context = t))
 const getAssert = () => context?.assert ?? assert // do not use non-strict comparisons on this!
@@ -123,47 +121,20 @@ const snapOnDisk = (orig, matcher) => {
   }
 
   const obj = matcher ? deepMerge(orig, matcher) : orig
-  const escape = (str) => str.replaceAll(/([\\`])/gu, '\\$1')
 
   maybeSetupJestSnapshots()
 
   if (!context?.assert?.snapshot) {
-    // We don't have native snapshots, polyfill reading
-    if (snapshotText !== null) {
-      try {
-        const snapshotRaw = readSnapshot()
-        snapshotText = snapshotRaw ? `\n${snapshotRaw}\n` : null // we'll search wrapped in \n
-      } catch {
-        snapshotText = null
-      }
-    }
-
-    const addFail = `Adding new snapshots requires Node.js >=22.3.0`
-
-    // We don't support polyfilled snapshot generation here, only parsing
-    // Also be careful with assertion plan counters
-    if (!snapshotText) getAssert().fail(`Could not find snapshot file. ${addFail}`)
-
     const namePath = getTestNamePath(context).map((x) => (x === '<anonymous>' ? '' : x))
-    const name = namePath.join(' ')
-    const count = (nameCounts.get(name) || 0) + 1
-    nameCounts.set(name, count)
-    const escaped = escape(serialize(obj))
-    const key = `${name} ${count}`
-    const makeEntry = (x) => `\nexports[\`${escape(key)}\`] = \`${x}\`;\n`
-    const final = escaped.includes('\n') ? `\n${escaped}\n` : escaped
-    if (snapshotText.includes(makeEntry(final))) return
-    // Perhaps wrapped with newlines from Node.js snapshots?
-    if (!final.includes('\n') && snapshotText.includes(makeEntry(`\n${final}\n`))) return
-    return getAssert().fail(`Could not match "${key}" in snapshot. ${addFail}`)
+    return matchSnapshot(readSnapshot, getAssert(), namePath.join(' '), serialize(obj))
   }
 
   // Node.js always wraps with newlines, while jest wraps only those that are already multiline
   try {
-    wrapContextName(() => getAssert().snapshot(obj))
+    wrapContextName(() => context.assert.snapshot(obj))
   } catch (e) {
     if (typeof e.expected === 'string') {
-      const escaped = haveSnapshotsReportUnescaped ? e.expected : escape(e.expected)
+      const escaped = haveSnapshotsReportUnescaped ? e.expected : escapeSnapshot(e.expected)
       const final = escaped.includes('\n') ? escaped : `\n${escaped}\n`
       if (final === e.actual) return
     }