test-inventory 0.2.1 → 0.4.0

package/CHANGELOG.md

@@ -1,5 +1,33 @@
 # test-inventory
 
+## 0.4.0
+
+### Minor Changes
+
+- [#42](https://github.com/Huldoser/test-inventory/pull/42) [`6849f55`](https://github.com/Huldoser/test-inventory/commit/6849f555d8b214ebbc8fdb9aa9cbac8cfe0913db) - Tests say what their body does: `fixtures` (the names the test and the `beforeEach`, `afterEach` and `aroundEach` hooks around it take from the context, without Vitest's own keys), `expectCount` (`expect()`, `expect.soft()` and `expect.poll()` calls in the body), Vitest's `expect.assertions(n)` and `expect.hasAssertions()` as `expectedAssertions` and `expectsAssertions`, `hasEmptyBody`, and Playwright's `test.step()` titles in `steps`. Vitest's `annotate()` from the test context adds annotations. Vitest 5 benchmarks, in `*.bench.ts` files or using the `bench` fixture, have `isBenchmark` and a `benchmarkCount` in the summary. New warnings: `empty-title` for a test or describe titled `''`, and `bench-outside-benchmark-file`.
+
+- [#40](https://github.com/Huldoser/test-inventory/pull/40) [`f462ee0`](https://github.com/Huldoser/test-inventory/commit/f462ee015fea8ee802f567a26dbaa4db94ada1d5) - The output has a record for each scanned file in `files`, and counts hooks: `hooks` on a file holds the hooks at its top level, on a describe the hooks registered in it, and on a test every hook that runs around it, by kind (`beforeAll`, `beforeEach`, `afterEach`, `afterAll`, and Vitest's `aroundEach` and `aroundAll`). Hooks a helper function registers count where it is called. JSON Lines output has a `file` line for each file, and a filtered inventory keeps the files of the matched tests and the files without tests.
+
+- [#41](https://github.com/Huldoser/test-inventory/pull/41) [`1ab9cb4`](https://github.com/Huldoser/test-inventory/commit/1ab9cb4002dd971ddb62570dc55ac564504e3a83) - Tests and describes have their run settings: `timeout`, `isSlow` and `slowCondition`, `retries`, `retryDelay`, `retryCondition` and `repeats`. Playwright's come from `test.describe.configure({ timeout, retries })`, `test.setTimeout()` and `test.slow()` in a describe, a `beforeEach` or the test body; Vitest's from the `timeout`, `retry` and `repeats` options, the timeout after the body, and the options a describe passes down. A test gets the value from the nearest place that sets it, and `null` when the file sets none, so the config decides. A timeout that can't be worked out, or that is set only under a condition, gets an `unresolved-option` warning.
+
+## 0.3.0
+
+### Minor Changes
+
+- [#37](https://github.com/Huldoser/test-inventory/pull/37) [`5aeb139`](https://github.com/Huldoser/test-inventory/commit/5aeb13959daaeb0fdf4488765944fe42ef7754bc) - `--fields` keeps only the named keys of each test, and `id`, as in `--fields fullName,reason`, so a short question gets a short answer. Output written with it leaves keys out, so it no longer matches the JSON Schema. The library exports the same as `pickTestFields()`.
+
+- [#37](https://github.com/Huldoser/test-inventory/pull/37) [`5aeb139`](https://github.com/Huldoser/test-inventory/commit/5aeb13959daaeb0fdf4488765944fe42ef7754bc) - Filters keep only the tests a question is about: `--state`, `--tag`, `--grep` (a regular expression on the full name, ignoring case), `--only` and `--without-reason`, as in `--state skip --state fixme` for the tests that don't run. The output then describes only those tests: the describes around them, their diagnostics and a summary counted over them. The library exports the same as `filterInventory()`.
+
+- [#37](https://github.com/Huldoser/test-inventory/pull/37) [`5aeb139`](https://github.com/Huldoser/test-inventory/commit/5aeb13959daaeb0fdf4488765944fe42ef7754bc) - `--format jsonl` writes the inventory as JSON Lines: an `inventory` line, one line per suite, test and diagnostic, and a `summary` line, each marked with a `type` key. Tools that read one record per line, such as `grep` or DuckDB, no longer need the whole document in memory. The `InventoryLine` type describes the lines.
+
+### Patch Changes
+
+- [#34](https://github.com/Huldoser/test-inventory/pull/34) [`0f96258`](https://github.com/Huldoser/test-inventory/commit/0f96258e65085fa1b7c667b5fa7219a8b19230df) - A fixtures file that builds its test object with a function, written there or imported from another file, as in `export const test = defineDeskTest('equities')`, is followed: MSW's test files import such a `test`. Their tests were missing before, with no diagnostic when the fixtures file didn't import the framework itself. A test object built with a call the scan can't follow now gets an `unresolved-test-import` error that names the call, as in "builds it with withWarmup(), which is not followed". A function that returns what another function returns, as `() => onFeed('NYSE')`, is followed too.
+
+- [#34](https://github.com/Huldoser/test-inventory/pull/34) [`0f96258`](https://github.com/Huldoser/test-inventory/commit/0f96258e65085fa1b7c667b5fa7219a8b19230df) - A call the runner refuses that is written like a declaration, as `describe.sequential('title', fn)` in Vitest 5, `test('title', fn, { retry: 2 })` from Vitest 4 or `test.only.skip('title', fn)`, is listed as a plain test or describe without the part that fails. Its tests were missing before; now they are `notLoaded`, like every other test in the file.
+
+- [#34](https://github.com/Huldoser/test-inventory/pull/34) [`0f96258`](https://github.com/Huldoser/test-inventory/commit/0f96258e65085fa1b7c667b5fa7219a8b19230df) - States follow the runners' precedence more closely, as checked against Vitest 5.0.4 running generated files. A skip in a test or hook body loses to a fixme, skip or todo set while the file loads, as on a describe around the test, since the body never runs. In Vitest, a test's own options win over its chain, so `test.skip('title', { skip: false }, fn)` is active, but the chain still sets its mode: `test.concurrent('title', { concurrent: false }, fn)` runs in parallel. A describe passes its `skip` option down to the tests and describes inside, where it wins over todo, also when the describe's own `.only` keeps the describe itself from being skipped.
+
 ## 0.2.1
 
 ### Patch Changes

package/README.md

@@ -70,7 +70,9 @@ the first test's record includes:
 ```
 
 The output also has a `summary` with counts, a record for each describe, and `diagnostics` that point at code the
-scan could not read and values it could not work out.
+scan could not read and values it could not work out. Filters such as `--state skip`, `--tag @smoke` and
+`--grep "limit order"` keep only some tests, `--fields fullName,reason` keeps only some keys, and `--format jsonl`
+writes one record per line.
 
 ## Library
 
@@ -84,6 +86,12 @@ const todo = inventory.tests.filter((test) => test.isTodo).map((test) => test.fu
 `scanSource()` scans the code of one file without touching the disk. The JSON Schema of the output is exported as
 `test-inventory/schema/v1.json`, so a reader can pin the version of the format it understands.
 
+## AI agents
+
+[test-inventory-mcp](https://github.com/Huldoser/test-inventory-mcp) is an MCP server built on test-inventory: agents
+in Claude Code, Cursor, VS Code and other MCP clients ask it which tests a project has without running them. The
+[AI agents](https://huldoser.github.io/test-inventory/guide/agents) page has the setup and the other ways in.
+
 ## Limits
 
 - Config files are not read, so `grep`, projects and Vitest tag options in them do not change the output.

package/dist/ast.d.ts

@@ -4,6 +4,17 @@ export declare function childNodes(node: Node): Node[];
 /** Strips TypeScript-only wrappers such as `test!`, `test as any` and `<any>test`. */
 export declare function unwrap(node: Expression): Expression;
 export declare function isFunctionNode(node: Node | null | undefined): node is FunctionNode;
+/** A value a function returns, with the nodes inside the function around its `return`, outermost first. */
+export interface ReturnValue {
+    expression: Expression;
+    around: Node[];
+}
+/**
+ * The values a function returns; null when it may return no value, as with `return;` or no `return` at all, and for
+ * an async function or a generator, which returns a promise or an iterator whatever its `return` says. A `return` in a
+ * function inside it is that function's.
+ */
+export declare function returnValues(fn: FunctionNode): ReturnValue[] | null;
 export declare function isStringLiteral(node: Node): node is StringLiteral;
 /** `import.meta.vitest`, which in-source Vitest tests read their functions from. */
 export declare function isImportMetaVitest(expression: Expression): boolean;

package/dist/ast.js

@@ -32,6 +32,36 @@ export function unwrap(node) {
 export function isFunctionNode(node) {
     return node?.type === 'ArrowFunctionExpression' || node?.type === 'FunctionExpression';
 }
+/**
+ * The values a function returns; null when it may return no value, as with `return;` or no `return` at all, and for
+ * an async function or a generator, which returns a promise or an iterator whatever its `return` says. A `return` in a
+ * function inside it is that function's.
+ */
+export function returnValues(fn) {
+    if (fn.async || fn.generator)
+        return null;
+    // Only declarations of overloads have no body.
+    const body = fn.body;
+    if (body.type !== 'BlockStatement')
+        return [{ expression: body, around: [] }];
+    const values = [];
+    // Copied only at a `return`, so that a long function costs no more than its size.
+    const around = [];
+    const visit = (node) => {
+        if (node.type === 'ReturnStatement') {
+            values.push(node.argument ? { expression: node.argument, around: [...around] } : null);
+            return;
+        }
+        if (node.type === 'FunctionDeclaration' || isFunctionNode(node))
+            return;
+        around.push(node);
+        for (const child of childNodes(node))
+            visit(child);
+        around.pop();
+    };
+    visit(body);
+    return values.length > 0 && !values.includes(null) ? values : null;
+}
 export function isStringLiteral(node) {
     return node.type === 'Literal' && typeof node.value === 'string';
 }

package/dist/cli.d.ts

@@ -4,6 +4,6 @@ export interface Io {
     stderr: Writable;
     cwd: string;
 }
-export declare const HELP = "Usage: test-inventory <glob...> --framework <playwright|vitest> [options]\n\nLists the tests in the matching files as JSON, read from the source code without running them.\n\nOptions:\n  --framework <name>           playwright or vitest (required)\n  --root <dir>                 project root; paths in the output are relative to it (default: current directory)\n  --ignore <glob>              leave out matching files; can be repeated (node_modules is always left out)\n  --framework-version <x.y.z>  apply the rules of this framework version instead of the one in the project\n  --comments <mode>            comments above tests: all, non-active or none (default: non-active)\n  -o, --output <file>          write to a file instead of standard output\n  --pretty                     indent the JSON\n  -h, --help                   show this help\n  -v, --version                show the version\n\nExit codes: 0 when the scan finishes, even with diagnostics; 2 for invalid arguments, an output that cannot be\nwritten, or a crash. 1 is reserved for a future release.\n\nExample:\n  test-inventory \"tests/**/*.spec.ts\" --framework playwright --output inventory.json\n";
+export declare const HELP = "Usage: test-inventory <glob...> --framework <playwright|vitest> [options]\n\nLists the tests in the matching files as JSON, read from the source code without running them.\n\nOptions:\n  --framework <name>           playwright or vitest (required)\n  --root <dir>                 project root; paths in the output are relative to it (default: current directory)\n  --ignore <glob>              leave out matching files; can be repeated (node_modules is always left out)\n  --framework-version <x.y.z>  apply the rules of this framework version instead of the one in the project\n  --comments <mode>            comments above tests: all, non-active or none (default: non-active)\n  --state <state>              keep tests in this state, such as skip or fixme; can be repeated\n  --tag <tag>                  keep tests with this tag, such as @smoke; can be repeated\n  --grep <regex>               keep tests whose full name matches, ignoring case\n  --only                       keep tests that use .only, or are inside a describe that does\n  --without-reason             keep tests that don't run and give no reason\n  --fields <keys>              write only these keys of each test, separated by commas; id is always kept\n  --format <format>            json or jsonl, one record per line (default: json)\n  -o, --output <file>          write to a file instead of standard output\n  --pretty                     indent the JSON\n  -h, --help                   show this help\n  -v, --version                show the version\n\nExit codes: 0 when the scan finishes, even with diagnostics; 2 for invalid arguments, an output that cannot be\nwritten, or a crash. 1 is reserved for a future release.\n\nExamples:\n  test-inventory \"tests/**/*.spec.ts\" --framework playwright --output inventory.json\n  test-inventory \"tests/**/*.spec.ts\" --framework playwright --state skip --fields fullName,reason\n";
 /** Runs the command line with the given arguments and streams. Resolves to the exit code. */
 export declare function run(argv: string[], io: Io): Promise<number>;

package/dist/cli.js

@@ -3,7 +3,8 @@ import { once } from 'node:events';
 import path from 'node:path';
 import { parseArgs } from 'node:util';
 import { readFailure } from "./files.js";
-import { writeInventory } from "./json.js";
+import { checkFields, filterInventory, matcher, pickTestFields } from "./filter.js";
+import { writeInventory, writeInventoryLines } from "./json.js";
 import { OptionsError } from "./options.js";
 import { scan } from "./scan.js";
 import { toolVersion } from "./version.js";
@@ -17,6 +18,13 @@ Options:
   --ignore <glob>              leave out matching files; can be repeated (node_modules is always left out)
   --framework-version <x.y.z>  apply the rules of this framework version instead of the one in the project
   --comments <mode>            comments above tests: all, non-active or none (default: non-active)
+  --state <state>              keep tests in this state, such as skip or fixme; can be repeated
+  --tag <tag>                  keep tests with this tag, such as @smoke; can be repeated
+  --grep <regex>               keep tests whose full name matches, ignoring case
+  --only                       keep tests that use .only, or are inside a describe that does
+  --without-reason             keep tests that don't run and give no reason
+  --fields <keys>              write only these keys of each test, separated by commas; id is always kept
+  --format <format>            json or jsonl, one record per line (default: json)
   -o, --output <file>          write to a file instead of standard output
   --pretty                     indent the JSON
   -h, --help                   show this help
@@ -25,18 +33,24 @@ Options:
 Exit codes: 0 when the scan finishes, even with diagnostics; 2 for invalid arguments, an output that cannot be
 written, or a crash. 1 is reserved for a future release.
 
-Example:
+Examples:
   test-inventory "tests/**/*.spec.ts" --framework playwright --output inventory.json
+  test-inventory "tests/**/*.spec.ts" --framework playwright --state skip --fields fullName,reason
 `;
-/** CLI names for the option names in `scan()`'s errors. */
+/** CLI names for the option names in the library's errors. */
 const FLAGS = {
-    patterns: 'the glob patterns',
-    framework: '--framework',
-    root: '--root',
-    ignore: '--ignore',
-    frameworkVersion: '--framework-version',
-    comments: '--comments',
+    'options.patterns': 'the glob patterns',
+    'options.framework': '--framework',
+    'options.root': '--root',
+    'options.ignore': '--ignore',
+    'options.frameworkVersion': '--framework-version',
+    'options.comments': '--comments',
+    'filter.states': '--state',
+    'filter.tags': '--tag',
+    'filter.grep': '--grep',
+    fields: '--fields',
 };
+const FORMATS = ['json', 'jsonl'];
 async function writeTo(stream, chunk) {
     if (stream.errored)
         throw stream.errored;
@@ -89,6 +103,13 @@ export async function run(argv, io) {
                 ignore: { type: 'string', multiple: true },
                 'framework-version': { type: 'string' },
                 comments: { type: 'string' },
+                state: { type: 'string', multiple: true },
+                tag: { type: 'string', multiple: true },
+                grep: { type: 'string' },
+                only: { type: 'boolean' },
+                'without-reason': { type: 'boolean' },
+                fields: { type: 'string' },
+                format: { type: 'string' },
                 output: { type: 'string', short: 'o' },
                 pretty: { type: 'boolean' },
                 help: { type: 'boolean', short: 'h' },
@@ -112,8 +133,26 @@ export async function run(argv, io) {
         return fail('give at least one glob pattern for the test files.');
     if (values.framework === undefined)
         return fail('--framework is required: playwright or vitest.');
+    const format = values.format ?? 'json';
+    if (!FORMATS.includes(format))
+        return fail('--format must be json or jsonl.');
+    if (format === 'jsonl' && values.pretty)
+        return fail('--pretty indents JSON, so it does not go with --format jsonl.');
+    const filter = {
+        states: values.state,
+        tags: values.tag,
+        grep: values.grep,
+        only: values.only,
+        withoutReason: values['without-reason'],
+    };
+    const fields = values.fields?.split(',').map((field) => field.trim());
     let inventory;
+    let result;
     try {
+        // Checked before the scan, so a mistyped state or key doesn't wait for every file to be read.
+        matcher(filter);
+        if (fields !== undefined)
+            checkFields(fields);
         inventory = await scan({
             patterns: positionals,
             framework: values.framework,
@@ -122,10 +161,12 @@ export async function run(argv, io) {
             frameworkVersion: values['framework-version'],
             comments: values.comments,
         });
+        const filtered = filterInventory(inventory, filter);
+        result = fields === undefined ? filtered : pickTestFields(filtered, fields);
     }
     catch (error) {
         if (error instanceof OptionsError) {
-            return fail(error.message.replace(/^options\.(\w+)/, (_, name) => FLAGS[name]));
+            return fail(error.message.replace(/^[\w.]+/, (name) => FLAGS[name]));
         }
         io.stderr.write(`test-inventory: unexpected error: ${error instanceof Error ? String(error.stack) : String(error)}\n`);
         return FAILED;
@@ -139,8 +180,14 @@ export async function run(argv, io) {
     io.stdout.on('error', quiet);
     try {
         await writeOutput(io, values.output, async (output) => {
-            await writeInventory(inventory, values.pretty === true, (chunk) => writeTo(output, chunk));
-            await writeTo(output, '\n');
+            const write = (chunk) => writeTo(output, chunk);
+            if (format === 'jsonl') {
+                await writeInventoryLines(result, write);
+            }
+            else {
+                await writeInventory(result, values.pretty === true, write);
+                await write('\n');
+            }
         });
     }
     catch (error) {

package/dist/diagnostics.js

@@ -34,6 +34,8 @@ export const LEVELS = {
     'no-files-matched': 'warning',
     'unsupported-file': 'warning',
     'approximate-framework-version': 'warning',
+    'empty-title': 'warning',
+    'bench-outside-benchmark-file': 'warning',
     'test-in-function': 'warning',
 };
 /**

package/dist/extract.d.ts

@@ -1,8 +1,8 @@
 import type { CallExpression } from 'oxc-parser';
-import type { Adapter, Details, Mode, ModuleReader, ModuleSource, Problem, Span, StateSetting, SuiteDeclaration, TestDeclaration } from './model.ts';
+import type { Adapter, Details, Mode, ModuleReader, ModuleSource, Problem, RunSettings, Span, StateSetting, SuiteDeclaration, TestDeclaration } from './model.ts';
 import type { ParsedFile } from './parse.ts';
 import { type SourceText } from './source.ts';
-import type { Annotation } from './types.ts';
+import type { Annotation, HookCounts, Step } from './types.ts';
 import { type Values } from './values.ts';
 /** The file or a describe, with what is set on it. Tests inherit from every scope around them. */
 export interface Scope {
@@ -12,6 +12,12 @@ export interface Scope {
     mode: Mode | null;
     isOnly: boolean;
     details: Details;
+    /** Hooks registered directly in the scope. */
+    hooks: HookCounts;
+    /** Run settings from `beforeEach` hooks in the scope, which run before each test in it and win over the scope's own. */
+    hookRun: RunSettings;
+    /** Fixtures of the hooks in the scope that run around each test. */
+    hookFixtures: string[];
     suite: PendingSuite | null;
 }
 /** Where a test or describe registered through an imported helper is written, in the helper's module. */
@@ -48,6 +54,12 @@ export interface PendingTest {
     isInCondition: boolean;
     isInFunction: boolean;
     comments: string[];
+    /** Run settings from calls in the body, which win over everything around the test. */
+    run: RunSettings;
+    expectCount: number;
+    expectedAssertions: number | null;
+    expectsAssertions: boolean;
+    steps: Step[];
     problems: Problem[];
 }
 /** A file the walk reads code from: the scanned file, or a module a helper function is imported from. */
@@ -65,7 +77,19 @@ export interface Extraction {
     tests: PendingTest[];
     /** Problems not tied to one test. */
     problems: Problem[];
+    /** Hooks at the top level of the file. */
+    hooks: HookCounts;
     usesGlobals: boolean;
 }
+/**
+ * Applies settings written later in the same place: the last value of each wins, and a test is slow when any
+ * `test.slow()` applies.
+ */
+export declare function mergeRun(target: RunSettings, later: RunSettings): void;
+/** Any of several `test.slow()` calls: always when one always applies, otherwise under any of their conditions. */
+export declare function anySlow(slows: NonNullable<RunSettings['slow']>[]): NonNullable<RunSettings['slow']>;
+/** Conditions joined with `||`, adding parentheses where a part could otherwise be read differently. */
+export declare function eitherCondition(parts: string[]): string;
+export declare function noHooks(): HookCounts;
 /** Walks the scanned file; `openModule` reads the modules of imported helpers that the file calls. */
 export declare function extract(main: Frame, openModule: OpenModule): Extraction;