@t09tanaka/test-gates 0.4.0 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +47 -0
- package/README.md +15 -15
- package/dist/cli.js +1253 -371
- package/dist/index.cjs +61 -44
- package/dist/index.js +61 -44
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,52 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.4.2
|
|
4
|
+
|
|
5
|
+
Most of this release closes cases where `test-gates mutation` took over the results of the last run although a file the run depends on had changed, with `--affected` and without it. The rest spares full runs that were not needed. Two things to know when updating: the first run after it is one of every mutant, since the record holds the version of test-gates; and in a project whose runner config maps modules in a way that `settings.importAliases` does not name, `--affected` no longer tells the gates apart until the aliases are added (`test-gates doctor` names them).
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
These made a run take over results that a change could have turned into survivors.
|
|
10
|
+
|
|
11
|
+
- `mutation --affected` kept the results of a gate after a change of a file it depends on:
|
|
12
|
+
- An import put together from strings (`import('./locales/' + lang + '.ts')`, `require('./rules/' + lang)`) was not followed at all. Any `require()` / `import()` whose argument is not one string literal is now one whose path is not written out, and what the gates depend on is unknown. Allowlist mode of `test-gates check` now rejects such an import too.
|
|
13
|
+
- With `x.ts` and `x.js` next to each other, `./x` was taken for `x.ts` alone while the runner may load `x.js`. Every script that exists for a path is now followed (also by the comparison and by `selfcheck --jobs`).
|
|
14
|
+
- Module maps of the runner (`resolve.alias` / `test.alias` of Vite, `moduleNameMapper` of Jest, also in package.json, `jsc.paths` of `.swcrc`, tsconfig `paths` for Vitest, Babel `module-resolver`) can send an import somewhere else than the path says. Such a map now makes what the gates depend on unknown, unless it does what an alias of `settings.importAliases` does, is a `moduleNameMapper` entry to one fixed module (a style stub), or is a `paths` entry of one name that leads to an installed package. The reason names every such map and the alias to add; `test-gates doctor` shows it in its note.
|
|
15
|
+
- Where the run drops imports of types, a spec with `import type { Rate } from './rate'` and `jest.createMockFromModule<Rate>('./rate')` left `./rate` out of what the gate loads. `createMockFromModule` and `genMockFromModule` are now followed, also with a type argument.
|
|
16
|
+
- `test-gates mutation` (without `--affected` too) reused the results after a change in files the comparison did not reach:
|
|
17
|
+
- A directory whose name holds glob characters (`shared/[rates]`, a `[locale]` of Next.js) was listed as empty by git and dropped out of the comparison. The path handed to git is now escaped.
|
|
18
|
+
- A plain `..` path that exists only from the subproject, where the run starts (`readFileSync('../shared-data/x.json')` in a spec, `path.resolve('../…')` in a setup file), was resolved from the directory of the file only. It is now resolved from the subproject as well. One from the subproject that lies outside of what can be listed is refused, and one that ends in a directory holding the repository makes every file of the repository part of the comparison.
|
|
19
|
+
- A file git ignores that the run loads (generated code, downloaded fixtures a spec imports) was not compared. With the helper's runner config it is now hashed and its paths followed.
|
|
20
|
+
- `patches/` of patch-package, in the subproject and in the directories above it, is compared.
|
|
21
|
+
- The node on PATH, which the shebang of `node_modules/.bin/stryker` starts, is recorded as `Node.js on PATH`; a change of it runs every mutant.
|
|
22
|
+
- A path in a file above the subproject that the comparison reaches (`"extends": "./configs/strict.json"` in a base tsconfig at the top of the repository) is followed by the same rules as a path in the subproject.
|
|
23
|
+
|
|
24
|
+
### Changed
|
|
25
|
+
|
|
26
|
+
These spare full runs, or runs of more gates, that were not needed.
|
|
27
|
+
|
|
28
|
+
- `mutation --affected` gives up telling the gates apart in fewer cases:
|
|
29
|
+
- An import of a module that only has a declaration file (`@/types/schema` for `src/types/schema.d.ts`, `index.d.ts` of a directory) leads to that file. A script of the same name still comes first.
|
|
30
|
+
- An import of types only (`import type`, `export type … from`, an import whose names are all marked `type`) is left out of what a gate depends on, but only where the run is known not to check types: no checkers in the Stryker config, and Vitest without `typecheck`, or Jest with ts-jest and `isolatedModules: true` or `diagnostics: false`, or with `@swc/jest`, `babel-jest`, `esbuild-jest` or `@sucrase/jest` and neither ts-jest nor a preset. `import { type A } from './a'` is left out only where no compiler config keeps imports as written (`verbatimModuleSyntax`, `preserveValueImports`, `importsNotUsedAsValues`, `onlyRemoveTypeImports`). Anything else keeps the import.
|
|
31
|
+
- A file other than a script that a runner config names (`coverage.exclude: ['src/components/Foo.vue']`, a JSON stub of `moduleNameMapper`) is taken as a config by itself: a change of it runs every gate again, but a component named there no longer makes `--affected` refuse.
|
|
32
|
+
- A file that uses `__dirname`, `__filename` or `import.meta.url` / `dirname` / `filename` only to find where it lies keeps the gates apart. It is refused only when it may also read by that path: a call that reads or looks at a file or a directory, a mock named by a computed path, a `Worker`, or an import of a package or built-in module other than `path`, `url`, `vitest` and `@jest/globals`.
|
|
33
|
+
- `require('./rates')` of `rates.json` leads to the JSON file (also `<dir>/index.json`), after the scripts, as Node, Jest and Vite do.
|
|
34
|
+
- An import or a word such as `readFileSync` in a comment no longer counts. A relative path in quotes is still followed wherever it is written.
|
|
35
|
+
- A mock in a `__mocks__` next to a module (`src/__mocks__/dep.ts`) is loaded with that module, so a change of it runs again only the gates that load the module. The `__mocks__` at the top of the subproject, and a mock named as an installed package or a built-in module, are still configs.
|
|
36
|
+
- A run without `--affected` writes what each gate depends on too, whenever it writes a record. Before, the next `--affected` run after a run without it, or after `-- --force`, had nothing to go on and ran every mutant. Runs with `--gate` and runs that fail still write none.
|
|
37
|
+
- A mutant that timed out in the last run and that an `expectedTimeouts` entry of its gate points at is taken over, with and without `--affected`. Before, such mutants that never end were run again on every run. Any other timeout is still run again.
|
|
38
|
+
- A plain path to a file directly in a directory above the subproject (`"extends": "../../tsconfig.base.json"`) no longer makes every file of the repository part of the comparison: that file is compared anyway. A path to a directory above, or to a missing file there, still does.
|
|
39
|
+
- The files of Stryker's reporters (`htmlReporter`, `jsonReporter`, or their defaults) and the files that `ignorePatterns` leaves out (`.eslintcache`, `tsconfig.tsbuildinfo`) are no longer taken for files the run reads. A new report or cache no longer runs every mutant, and the paths in an HTML report no longer make every file of the repository part of the comparison. Where a pattern of `ignorePatterns` starts with `!`, nothing is left out by it. The paths in `.html` files are not followed.
|
|
40
|
+
- `test-gates check` keeps a `..` of an import that leaves the subproject (`resolveSpecifier`): `../../shared/x` from `src/a.ts` is `../shared/x`, not `shared/x`. An import outside is judged by the path it points at, a mock of a module outside is no longer taken for a mock of the gate, and `selfcheck` no longer lists a file that imports a module outside as an importer of a gate of the same path inside. A root-anchored `impurePaths` rule (`^shared/`) no longer matches an import outside.
|
|
41
|
+
- The comparison and the work of `--affected` before a run are faster: the files of each path are looked up once in a run (on this repository the snapshot went from about 140 to 90 ms).
|
|
42
|
+
|
|
43
|
+
## 0.4.1
|
|
44
|
+
|
|
45
|
+
### Fixed
|
|
46
|
+
|
|
47
|
+
- `test-gates mutation --affected` reuses every result after no change also where it cannot tell what a gate depends on (a file a gate or a spec loads reads files at run time, for one). Before, such a subproject ran every mutant on every run with the option, while a run without it reused the results after no change. The first line says so; a change still runs every mutant.
|
|
48
|
+
- `test-gates doctor` reports what keeps `--affected` from telling the gates apart: as a problem when a hook relies on `--affected`, else as a note. In such a subproject it no longer asks for `--affected` in a hook. Before, it said OK although a run with `--affected` was always one of every mutant.
|
|
49
|
+
|
|
3
50
|
## 0.4.0
|
|
4
51
|
|
|
5
52
|
Most of this release is about runs that were full ones although little had changed. `test-gates doctor` tells why a project's runs are full ones. `mutation --affected` runs again only the gates a change can affect. The comparison before a run covers only what the run loads. Two changes of behaviour: Stryker's `concurrency` now defaults to half of the cores, and the record of a run has a new format, so the first run after the update is one of every mutant.
|
package/README.md
CHANGED
|
@@ -256,30 +256,30 @@ What is left of incremental mode is the run after no change: a push that does no
|
|
|
256
256
|
What is compared:
|
|
257
257
|
|
|
258
258
|
- Every file of the subproject: gates, specs, other sources, helpers of the specs, configs, `test-gates.json`, the lock file.
|
|
259
|
-
- The files directly in the directories above the subproject, up to the root of the repository: the lock file and the shared configs of a monorepo.
|
|
260
|
-
- Packages of the same repository that are linked into `node_modules`, and directories outside the subproject that a file leads to, with a link or with a relative path (`../../pricing-kit/rounding/half-up`). A path that starts at the directory of its own file is followed as well, whether it is written as a template (`` `${__dirname}/../pricing-kit` ``) or as parts handed to `path.join` or `path.resolve` (`__dirname, '..', 'pricing-kit'`, also with `import.meta.dirname`). A path that leads back into the subproject adds nothing.
|
|
261
|
-
- When a path cannot be followed, every file of the repository is compared: when its start is not known (`<rootDir>/../pricing-kit`, an alias such as `@/../shared`, `'..'` after anything but the directory of the file), when a part after it is not known (`join(__dirname, '..', name)`, `join(resolve(__dirname, '..'), name)`, a way up after a placeholder), and when a plain relative path leads to a
|
|
262
|
-
- Files written for people to read (`.md`, `.markdown`, `.txt`, `.rst`, `.adoc
|
|
263
|
-
- When the config of the test runner is made with this package's helper (`createJestGatesConfig` or `createVitestGatesConfig`, which run the specs of the gates and no others), only the files of the subproject that the run loads are searched: the Stryker config, the runner config and the files it names (a setup file), the gates and their specs, `test-gates.json`, `tsconfig*.json`, `jsconfig*.json`, the Babel and SWC configs, the files below a root `__mocks__/` and the mock next to a loaded file, what those configs name (in a script, any path as text, such as a setup file; in `tsconfig.json` or `.babelrc`, what it extends and the plugins and presets it names, and what those name in turn), and everything those import, also what a component (`.vue`, `.svelte`, `.astro`, `.mdx`) imports. The other files of the subproject are compared, but a path in them is not followed: `package.json` (its scripts, and a runner config in it that the helper's config replaces), a lint config, the settings of an editor, the config of another test run, a component no gate loads. An import of an alias that `settings.importAliases` does not name (one that the runner config maps) loads every file of the subproject whose path ends as the import does (`@shared/money/fee` loads `src/money/fee.ts`); when none does, it leads out of the subproject, to where the config that maps it points. An import of an alias with the extension of the compiled file (`@shared/fee.js`) leads to `fee.ts` as well, and one of a single part (`@helper`, `#config`) to the files of that name. Every file is searched when a loaded file imports a module whose path is not written out, when the runner config is not made with the helper, when what the runner config exports is not what the helper returns (merged with another config after it, changed after the call), and when the runner config or a file it is put together from names `projects` or `workspace` outside of a comment: then it may run more than the specs of the gates. Another config spread into the helper's options is fine: the helper sets the specs after them.
|
|
259
|
+
- The files directly in the directories above the subproject, up to the root of the repository: the lock file and the shared configs of a monorepo. Also the files in a `patches/` directory of the subproject or of a directory above it (patch-package).
|
|
260
|
+
- Packages of the same repository that are linked into `node_modules`, and directories outside the subproject that a file leads to, with a link or with a relative path (`../../pricing-kit/rounding/half-up`). A path that starts at the directory of its own file is followed as well, whether it is written as a template (`` `${__dirname}/../pricing-kit` ``) or as parts handed to `path.join` or `path.resolve` (`__dirname, '..', 'pricing-kit'`, also with `import.meta.dirname`). A plain relative path is taken both from the directory of its file (an import) and from the subproject, where the run starts (`readFileSync`, `path.resolve`), wherever something is there: `readFileSync('../shared-data/rates.json')` in `src/fee.spec.ts` reads the `shared-data` next to the subproject. Where what is there from the subproject lies outside the repository, reuse is off, as for any path there; where it is a directory that holds the repository (a way up to the top in a nested file), every file of the repository is compared. When nothing is there from either, a file that appears later at one of them makes the run a full one. A path that leads back into the subproject adds nothing, and so does a plain path to a file directly in a directory above the subproject (`"extends": "../../tsconfig.base.json"`): that file is compared anyway.
|
|
261
|
+
- When a path cannot be followed, every file of the repository is compared: when its start is not known (`<rootDir>/../pricing-kit`, an alias such as `@/../shared`, `'..'` after anything but the directory of the file), when a part after it is not known (`join(__dirname, '..', name)`, `join(resolve(__dirname, '..'), name)`, a way up after a placeholder), and when a plain relative path leads to a directory above the subproject (`'..'`, `'../..'`), or to a file in one that is not there (`'../tsconfig.missing.json'`).
|
|
262
|
+
- Files written for people to read (`.md`, `.markdown`, `.txt`, `.rst`, `.adoc`, `.html`, such as the report of Stryker) are compared, but the paths in them are not followed. Every other file is searched, whatever its extension.
|
|
263
|
+
- When the config of the test runner is made with this package's helper (`createJestGatesConfig` or `createVitestGatesConfig`, which run the specs of the gates and no others), only the files of the subproject that the run loads are searched: the Stryker config, the runner config and the files it names (a setup file), the gates and their specs, `test-gates.json`, `tsconfig*.json`, `jsconfig*.json`, the Babel and SWC configs, the files below a root `__mocks__/` and the mock next to a loaded file, what those configs name (in a script, any path as text, such as a setup file; in `tsconfig.json` or `.babelrc`, what it extends and the plugins and presets it names, and what those name in turn), and everything those import, also what a component (`.vue`, `.svelte`, `.astro`, `.mdx`) imports. The other files of the subproject are compared, but a path in them is not followed: `package.json` (its scripts, and a runner config in it that the helper's config replaces), a lint config, the settings of an editor, the config of another test run, a component no gate loads. An import of an alias that `settings.importAliases` does not name (one that the runner config maps) loads every file of the subproject whose path ends as the import does (`@shared/money/fee` loads `src/money/fee.ts`); when none does, it leads out of the subproject, to where the config that maps it points. An import of an alias with the extension of the compiled file (`@shared/fee.js`) leads to `fee.ts` as well, and one of a single part (`@helper`, `#config`) to the files of that name. Every file is searched when a loaded file imports a module whose path is not written out, when the runner config is not made with the helper, when what the runner config exports is not what the helper returns (merged with another config after it, changed after the call), and when the runner config or a file it is put together from names `projects` or `workspace` outside of a comment: then it may run more than the specs of the gates. Another config spread into the helper's options is fine: the helper sets the specs after them. A file the run loads in this way is compared also when git ignores it (generated code, fixtures that are downloaded), and the paths in it are followed. The reports of Stryker (the files of its HTML and JSON reporters, `reports/mutation/mutation.html` and `reports/mutation/mutation.json` unless the config names others) and what its `ignorePatterns` leave out (`.eslintcache`, `tsconfig.tsbuildinfo`; nothing, where one of them takes files back with `!`) are not taken for files the run loads, though the Stryker config names them; the reports are not compared at all.
|
|
264
264
|
- A path that leads outside of the repository to nothing is compared as missing: a path from a config that is meant for the sandbox of Stryker (`path.resolve(__dirname, '../../../shared')`, which from the subproject itself leads above the repository). Once something is there, reuse is off, as for any path outside the repository.
|
|
265
|
-
- What is installed (`node_modules/.package-lock.json`, or what yarn and pnpm write in its place), the versions of Node.js, of this package, of Stryker and of the test runners, and the arguments handed to Stryker.
|
|
265
|
+
- What is installed (`node_modules/.package-lock.json`, or what yarn and pnpm write in its place), the versions of Node.js (the one this tool runs on, and the one on `PATH`, which starts `node_modules/.bin/stryker`), of this package, of Stryker and of the test runners, and the arguments handed to Stryker.
|
|
266
266
|
- The incremental file itself: it has to be the one the recorded run wrote.
|
|
267
267
|
|
|
268
|
-
With git, "every file" is what git tracks plus the untracked files it does not ignore. In a git hook of a linked worktree, git sets `GIT_DIR` and not `GIT_WORK_TREE`; then git would take the subproject for the top of the repository, so test-gates calls git without `GIT_DIR` in that case and lets it find the repository from the subproject. Without git, or when git ignores the subproject, every file is read except `node_modules`, `.git` and what the run writes itself, up to the nearest directory that has a `.git` entry, else the topmost one that declares workspaces, else only the subproject. Such a listing also contains what git would ignore (coverage output, an HTML report, logs), so a file that changes between two runs makes every run a full one; reuse is meant for a checkout with git.
|
|
268
|
+
With git, "every file" is what git tracks plus the untracked files it does not ignore, and, with the helper's runner config, the files git ignores that the run loads. In a git hook of a linked worktree, git sets `GIT_DIR` and not `GIT_WORK_TREE`; then git would take the subproject for the top of the repository, so test-gates calls git without `GIT_DIR` in that case and lets it find the repository from the subproject. Without git, or when git ignores the subproject, every file is read except `node_modules`, `.git` and what the run writes itself, up to the nearest directory that has a `.git` entry, else the topmost one that declares workspaces, else only the subproject. Such a listing also contains what git would ignore (coverage output, an HTML report, logs), so a file that changes between two runs makes every run a full one; reuse is meant for a checkout with git.
|
|
269
269
|
|
|
270
270
|
The record is removed before each run and written only after a run that Stryker ended without an error and in which it wrote the incremental file, so a run that failed, was interrupted or was a dry run is followed by a full one. A run that is given `--force` is not changed. When Stryker does not run in incremental mode, nothing is recorded or compared.
|
|
271
271
|
|
|
272
|
-
The paths that leave the subproject are found by text, in every file that is compared but those written for people and, with the helper's runner config, those of the subproject that the run does not load. The search errs towards comparing too much: text that merely looks like such a path widens the comparison, and a path or a link to a directory that exists outside the repository turns reuse off, with the file and the path as it is written named in the message. There
|
|
272
|
+
The paths that leave the subproject are found by text, in every file that is compared but those written for people and, with the helper's runner config, those of the subproject that the run does not load. The search errs towards comparing too much: text that merely looks like such a path widens the comparison, and a path or a link to a directory that exists outside the repository turns reuse off, with the file and the path as it is written named in the message. There are two exceptions, where the path is known: a path from the directory of the file that ends in a directory above the subproject (`path.resolve(__dirname, '..')` in a file at the top of the subproject) adds only the files directly in that directory, and a plain path to a file that is directly in such a directory (`'../../tsconfig.base.json'`) adds only that file, and what that file points at with a path from its own directory (`"extends": "./configs/strict.json"`) or with `..`, followed in the same way. Both are compared anyway (see the last two points of the list below).
|
|
273
273
|
|
|
274
274
|
The comparison loads the Stryker config in a process of its own, to see where the incremental file is, and reads and hashes those files before each run. Timed with a stand-in for Stryker, so that only this tool is measured, in a project of 5,200 files written for the measurement: 0.13 s without the comparison, 0.44 to 0.55 s with it (medians of three rounds; other jobs were running on the machine).
|
|
275
275
|
|
|
276
276
|
What it does not see, and what therefore still needs `-- --force` by hand, or CI, where the run should always be a full one:
|
|
277
277
|
|
|
278
|
-
- files git ignores (build output that a spec or a config loads
|
|
278
|
+
- files git ignores that the run reads without importing them (`.env` read by `dotenv`, a JSON file read with `readFileSync`), and every file git ignores when the runner config is not made with the helper (build output that a spec or a config loads),
|
|
279
279
|
- environment variables, the clock, the network, and a Stryker config that reads them or the command line to decide about incremental mode,
|
|
280
280
|
- files outside the subproject that are reached without a link and without `..` in a path: an absolute path, an alias defined in a config above the subproject (in that case every file of the repository is compared only if the subproject points at that config with a relative path),
|
|
281
281
|
- changes inside `node_modules` that leave the files of the package manager as they were,
|
|
282
|
-
- paths in files written for people (`.md`, `.txt` and the like): a script that reads a path out of a README,
|
|
282
|
+
- paths in files written for people (`.md`, `.txt`, `.html` and the like): a script that reads a path out of a README,
|
|
283
283
|
- with the helper's runner config, paths in a file of the subproject that the run does not load as a module: a file that a loaded one reads as data (`readFileSync`) and takes a path out of, and a config that a package reads by its name and that is not among those listed above (`next.config.js` read by `next/jest`, for example, or a Babel config kept in the `babel` key of `package.json`),
|
|
284
284
|
- with the helper's runner config, what a component uses without importing it (a component that a plugin registers), and the paths in it,
|
|
285
285
|
- a path outside the repository to nothing, to which something is added that starts with the same text and continues the last part of its name (`'../../../shared' + '-kit'`),
|
|
@@ -311,16 +311,16 @@ The results of a gate depend on
|
|
|
311
311
|
- its own spec, every spec that imports it (directly or through other files), and every spec whose tests covered or killed one of its mutants in the last run, with every file those specs import. A spec that does not load the gate cannot see its mutants: Jest and Vitest give each spec file a module registry of its own. This is the way a change reaches another gate: when the spec of `next` checks its result through the gate `same`, a change of `same` runs `next` again, and so does a weakened spec of `next` for `same`,
|
|
312
312
|
- snapshots next to a spec (`__snapshots__/<spec>.snap`) and data a spec or a gate imports.
|
|
313
313
|
|
|
314
|
-
Imports are followed when they are relative, absolute, or start with a prefix of `settings.importAliases
|
|
314
|
+
Imports are followed when they are relative, absolute, or start with a prefix of `settings.importAliases`. An import of types only (`import type`, and `import { type Rate }` where a `tsconfig.json` tells that such imports are dropped: no `verbatimModuleSyntax` or `preserveValueImports` in it or in what it extends) counts where the run may check types: with a checker in the Stryker config, with the typecheck of Vitest, with ts-jest that reports type errors, and where that cannot be told. Where the run only strips types (Vitest, `@swc/jest`, `babel-jest`, ts-jest with `isolatedModules: true` or `diagnostics: false`), it does not count. Paths written as text for the runner (`vi.mock('./rates')`, `jest.createMockFromModule('./rates')`) are followed as well. A path without an extension leads to the scripts of that name, and to a JSON file of that name (`require('./rates')` to `rates.json`); where there is no script of that name, to its declaration file (`schema.d.ts` for `./schema`), which counts as a config (see below). An import or a name such as `readFileSync` in a comment does not count, but a relative path in quotes is followed wherever it is written, except in an import of types only that does not count. A package is taken to be what the lock file and the versions of the tools say it is, unless it is a link to a package of the same repository: see below.
|
|
315
315
|
|
|
316
316
|
A change elsewhere decides for every gate:
|
|
317
317
|
|
|
318
|
-
- **Every mutant is run** after a change of a config (the Stryker config and the config of the test runner, what they import, the setup files and other files they name, `package.json`, `test-gates.json`, `tsconfig*.json`, `*.config.*`, Babel configs, dotfiles, declaration files `.d.ts`, anything below `__mocks__`), of a lock file or of what is installed, of the tools or of the arguments; after a change of a file at the top of the subproject or outside of it that no gate depends on; and after a change of a file whose kind is not known (`.yaml`, `.html` and the like) that no gate depends on. Such a file that a gate does depend on runs that gate again, as any other file does.
|
|
318
|
+
- **Every mutant is run** after a change of a config (the Stryker config and the config of the test runner, what they import, the setup files and other files they name, `package.json`, `test-gates.json`, `tsconfig*.json`, `*.config.*`, Babel configs, dotfiles, declaration files `.d.ts`, anything below the `__mocks__` at the top of the subproject, and a mock below another `__mocks__` that is named as a package or a builtin module, such as `src/__mocks__/axios.ts`, which Jest can take for the package in every spec), of a lock file or of what is installed, of the tools or of the arguments; after a change of a file at the top of the subproject or outside of it that no gate depends on; and after a change of a file whose kind is not known (`.yaml`, `.html` and the like) that no gate depends on. Such a file that a gate does depend on runs that gate again, as any other file does. A mock next to a module (`src/__mocks__/rates.ts` for `src/rates.ts`, which `jest.mock('./rates')` and `vi.mock('./rates')` without a factory take) is loaded with that module: a change of it runs again the gates that load the module. One that no gate loads, or one that is gone, runs every mutant.
|
|
319
319
|
- **Nothing is run again** for a change of a source, data, a snapshot, a style or an image below the subproject that no gate depends on, or of a document (`.md`, `.txt` and the like).
|
|
320
320
|
|
|
321
|
-
When what a gate depends on cannot be told from the text, every mutant
|
|
321
|
+
When what a gate depends on cannot be told from the text, a change runs every mutant, and the first line says why. After no change at all the results are reused as they are without the option, so `--affected` is never slower than a run without it. Gate by gate it cannot tell when it meets an import whose path is not written out (`import(name)`, `require(name)`, strings put together as in `import('./locales/' + lang)`, a template with a value in it, a second argument), an import that leads to no file, a specifier that is neither a file nor an installed package (an alias defined only in a config: name it in `settings.importAliases`), a file read at run time (`node:fs`, `readFileSync`, `process.cwd`, `toMatchFileSnapshot`), a file that finds where it lies (`__dirname`, `import.meta.url`) and then may read by that path (it calls a function such as `existsSync` or `readFile`, names a mock by a computed path, or imports a package other than `path`, `url` and the test runner's, such as `dotenv`: a package is not looked into; finding where it lies alone is fine), a file that is loaded but not compared (git ignores it, and the comparison does not find that the run loads it: see above), a gate loaded for every spec by a setup file, a package of the same repository linked into `node_modules` (`@acme/shared`, also when it lies in the subproject), a component (`.vue`, `.svelte`, `.astro`, `.mdx`), which can use others that it does not import (registered by a plugin), a map of modules that `settings.importAliases` does not name (`resolve.alias` or `test.alias` of Vite, `moduleNameMapper` of Jest, `jsc.paths` of `.swcrc`, and with Vitest tsconfig `paths` other than those of a prefix the runner config maps itself; a map may lead an installed package to a file of the project or give an import more than one target. One that maps the same prefix to the same directory as an alias of the settings, or a part of it, is fine: `'@': path.resolve(__dirname, 'src')`, `{ find: /^@\//, replacement: resolve(__dirname, 'src') + '/' }`, `'^@/(.*)$': '<rootDir>/src/$1'`, `"@/*": ["./src/*"]`. So is a `moduleNameMapper` entry to one fixed module such as a style stub, and a `paths` entry to an installed package. `test-gates doctor` names each map that is not fine, and the alias to add where there is one), a Babel config with `module-resolver`, and a test runner other than Jest and Vitest. Where one name stands for both a script and a TypeScript file (`rate.js` and `rate.ts` for `./rate`), both are followed: runners do not agree on which comes first. A run without `--affected` writes down what each gate depends on as well, so a full run by hand (`-- --force`) does not make the next run with it a full one. Every mutant is also run when there is no record of the last run (a run with `--gate` leaves none), when the record cannot be read or does not say what each gate depends on (a run in CI writes no such part), with `--force`, and **when `CI` is set**: CI keeps running every mutant, and is the final check.
|
|
322
322
|
|
|
323
|
-
A gate with a survivor or a timeout that the allow lists do not cover is not reused: it runs again until it is clean. A mutant that timed out in the last run is
|
|
323
|
+
A gate with a survivor or a timeout that the allow lists do not cover is not reused: it runs again until it is clean. A mutant that timed out in the last run is not taken over either, also not in a gate whose other results are, and also not in a run without `--affected` after no change: on a loaded machine a mutant that would survive can time out instead, and a reused timeout would keep that. Those mutants alone are run again (`--mutate` with their ranges); the first lines count them. Only a timeout that an `expectedTimeouts` entry of its gate points at is taken over (matched as the verdict matches it): such a mutant times out on every run, and running it again would only cost the wait for its timeout. An entry says the mutant cannot end, so a listed mutant that would survive is kept as well: list only mutants that never end. The first lines count these apart (`2 mutant(s) that timed out in the last run are taken over: expectedTimeouts lists them`). The run is judged as one of every gate, and its first lines say what was reused:
|
|
324
324
|
|
|
325
325
|
```
|
|
326
326
|
test-gates mutation --affected: 1 of 3 gate(s) reused, 2 run again
|
|
@@ -347,7 +347,7 @@ problem: .husky/pre-push runs test-gates mutation without --affected: a change o
|
|
|
347
347
|
test-gates doctor: 1 problem(s) that make runs full ones
|
|
348
348
|
```
|
|
349
349
|
|
|
350
|
-
A **problem** makes runs full ones that need not be: Stryker not in incremental mode, a path that turns reuse off (one that leads outside of the repository to something, or a Stryker config that cannot be loaded), a path that makes every file of the repository part of the comparison (it names the file and the path), a Stryker config that sets `force`,
|
|
350
|
+
A **problem** makes runs full ones that need not be: Stryker not in incremental mode, a path that turns reuse off (one that leads outside of the repository to something, or a Stryker config that cannot be loaded), a path that makes every file of the repository part of the comparison (it names the file and the path), a Stryker config that sets `force`, a hook that runs `test-gates mutation` of the subproject without `--affected`, also through its `test:gates:mutation` script, and, when a hook relies on `--affected`, what keeps it from telling the gates apart (a file a gate or a spec loads that reads files at run time, for one). Where it cannot tell them apart, a hook without `--affected` is not taken for a problem, and what keeps it from doing so is a note. A run in a hook is taken to be one of the subproject that its line names (`--dir`, `--prefix`, `-w`, `cd`), else the nearest `cd` before it; a name with a variable in it (`cd "$project"` in a loop over the subprojects), or none, may be one of any subproject. Each run is judged on its own line, and comments are left out. A **note** is worth knowing: that every file of the subproject is searched for paths, and why (a runner config that is not made with the helper, an import whose path is not written out), and that no hook runs a mutation run. The exit code is 1 when there is a problem, so that the doctor can run in CI next to the other checks.
|
|
351
351
|
|
|
352
352
|
### `test-gates mutation-result [--report <path>]`
|
|
353
353
|
|