@blzsky/variant 0.2.1 → 0.3.0-canary.0

package/README.md

@@ -8,56 +8,68 @@
 
 ## What is variant?
 
-variant tells you which tests a change in a TypeScript monorepo actually needs.
-It reads the diff at the level of the syntax tree, not the line, classifies
-each changed file by what it does to the file's exported surface, and follows
-every test file's imports across workspace packages to see which ones reach the
-change.
+variant answers one question about a change to a TypeScript repository: what
+has to be verified before this is safe to merge?
+
+It reads the diff at the level of the syntax tree, not the line, follows every
+import across workspace packages, and says, on the pull request:
+
+- how risky each changed file is, and why;
+- which test files reach the change, with the import chain from each test to
+  the file that changed;
+- which typecheck, build and lint commands apply to the packages it affects;
+- which changed files no test reaches at all.
+
+You set it up once. After that nobody types a variant command: your own test
+command and your pull requests are the interface.
 
 ```bash
-npx @blzsky/variant impact --base origin/main
+npx @blzsky/variant init
 ```
 
-<p align="center">
-  <img alt="Running `variant impact --base main` after changing formatPrice's signature and documenting slugify: price.ts is breaking, slug.ts is non-impacting, 2 of 4 test files run" src="./docs/assets/impact.gif" width="840">
-</p>
+`init` finds out how the repository installs, tests and runs CI, shows every
+change it would make as a diff, and makes them when you agree. It adds one
+line to your Vitest config and one workflow that keeps a comment on each pull
+request.
+
+It does not skip tests. Your full suite runs as it always did; in CI variant
+predicts beside it which test files the change needs, then checks that
+prediction against what failed and prints one line:
+
+```
+variant: predicted 8 of 55 test files (high). 2 failed, all predicted.
+```
+
+Leaving tests out is something variant has to earn in each repository, from
+that record, and is not built yet.
 
-In that repository, `shop` and `blog` both depend on `utils`. A tool that works
-from the package graph would run all four test files. variant follows the
-imports: only `shop` reaches the changed `price.ts`, and a doc comment on
-`slug.ts` selects nothing.
-
-It does not skip anything. Every prediction is logged, and
-`variant impact verify` checks it against the test run you did anyway,
-counting each failure the prediction would have missed. That count, the
-false-skip rate, is what has to be measured before skipping is worth turning
-on, and it is the point of the project right now.
-
-The same analysis powers `pr check`, which rolls a branch's changes into one
-build verdict for a pull request comment, and `workspace check`, which fails CI
-when a package imports something it never declared.
-
-> **Status**: `0.x`, report-only. `impact`, `impact verify`, `diff`,
-> `pr check`, `pr report` and `workspace check` work today and are where
-> development happens. Test skipping is **not implemented**, deliberately, and
-> will not be until the false-skip rate is measured on real repositories; see
-> [Help measure it](#help-measure-it). The task runner underneath (`build`,
-> `run`, `insight`) works but is frozen. Any minor release can break, and every
-> break is in the [CHANGELOG](./CHANGELOG.md).
+> **Status**: `0.x`. What is described here is published as a canary,
+> `0.3.0-canary.0`, under the `canary` tag: write `@blzsky/variant@canary`
+> wherever this page says `@blzsky/variant`. Without the tag npm serves
+> 0.2.1, which has `impact`, `pr check`, `pr report` and `workspace check`,
+> an `init` that only writes a task-runner config, and no adapter. The
+> adapter is for Vitest; a Jest repository gets the pull request report
+> only. Test skipping is **not implemented**, deliberately.
+> The task runner underneath (`build`, `run`, `insight`) works but is frozen.
+> Any minor release can break, and every break is in the
+> [CHANGELOG](./CHANGELOG.md).
 
 ## Install
 
-variant needs Node 20 or newer and a git repository with at least one commit
-before the one you are checking. There is nothing to configure.
+variant needs Node 20 or newer and a git repository.
 
 ```bash
-# run it once
-npx @blzsky/variant impact --base origin/main
-
-# keep it in a project
-npm install -D @blzsky/variant    # or pnpm add -D / yarn add -D
+npx @blzsky/variant init            # shows the changes, asks, then applies
+npx @blzsky/variant init --dry-run  # only look
 ```
 
+`init` installs `@blzsky/variant` as a dev dependency with your package
+manager (npm, pnpm, Yarn or Bun), adds the
+[Vitest adapter](./docs/vitest-adapter.md) to your test config, ignores
+`.variant/`, gives your test workflow's checkout the history a prediction
+needs, and writes `.github/workflows/variant.yml`. Anything it cannot edit
+with confidence it leaves alone and tells you the lines to add.
+
 The package is `@blzsky/variant`; the binary it installs is `variant`.
 `npx variant` fetches an unrelated package, so always use the scope with `npx`.
 
@@ -67,26 +79,43 @@ The package is `@blzsky/variant`; the binary it installs is `variant`.
 npm install -D @blzsky/variant@latest
 ```
 
-Because variant is `0.x`, pin a version in CI (`npx --yes @blzsky/variant@0.2.1`)
-and read the CHANGELOG before moving to a new minor version.
+Because variant is `0.x`, read the CHANGELOG before moving to a new minor
+version.
 
 ## Usage
 
+After `init` there is nothing to run. What you see:
+
+- **On every pull request**, one comment, kept up to date: the risk of each
+  changed file, the tests to run and why, the checks, and what is not
+  verified. [An example](./docs/api.md#pr-report).
+- **In every CI test run**, one line saying what was predicted and whether a
+  failure fell outside it.
+
+The first runs after setup select every test, because the setup itself changes
+the lockfile and the test config. Narrowed predictions start with the next
+pull request.
+
+The commands underneath are still there when you want them:
+
 ```bash
-variant impact --base origin/main          # which tests does this branch need?
-npx vitest run --reporter=json --outputFile=report.json
-variant impact verify report.json          # did the prediction miss a failure?
-variant pr check --base origin/main        # one build verdict for the branch
+variant pr report --markdown               # the pull request comment, locally
+variant impact                             # which tests does this change need?
 variant workspace check                    # undeclared dependencies (exits 1)
 ```
 
+<p align="center">
+  <img alt="Running `variant impact --base main` after changing formatPrice's signature and documenting slugify: price.ts is breaking, slug.ts is non-impacting, 2 of 4 test files run" src="./docs/assets/impact.gif" width="840">
+</p>
+
 | Command | What it does |
 |---|---|
 | `impact` | Predict which test files a change requires, and how much of the import graph resolved. Report-only. |
-| `impact verify <report>` | Reconcile the last prediction against a Vitest or Jest JSON report and report false skips |
+| `impact verify <report...>` | Reconcile the last prediction against one or more Vitest or Jest JSON reports and report false skips |
 | `diff <file>` | Classify one file's change and list the exported symbols that changed |
+| `init` | Set a repository up: the dependency, the adapter, the workflow. Once. |
 | `pr check` | Classify every TypeScript change on the branch and roll them into one build verdict |
-| `pr report` | The `pr check` verdict as JSON, or as markdown for a PR comment |
+| `pr report` | What the branch changes and what has to be verified, as JSON or as markdown for a PR comment |
 | `workspace check` | Fail when a package imports a dependency it does not declare |
 
 Run `variant <command> --help` for its flags, or see the
@@ -103,7 +132,9 @@ setup file selects every test.
 
 ### In CI
 
-Check out with `fetch-depth: 0` and pass `--base origin/<branch>`. A shallow
+`init` writes what CI needs. To wire the commands in by hand: check out
+with `fetch-depth: 0`; the base is detected on GitHub Actions, and elsewhere
+pass `--base origin/<branch>`. A shallow
 clone has no base commit to compare against, and `actions/checkout` creates no
 local branch for the target; variant stops with `GIT_REF_ERROR` rather than
 guess.
@@ -117,6 +148,9 @@ Ready-to-copy GitHub Actions workflows are in
   the build on an undeclared dependency.
 - [`impact-shadow.yml`](./examples/github-actions/impact-shadow.yml) logs a
   prediction, runs the full suite anyway, and reconciles the two.
+- [`impact-shadow-workspace.yml`](./examples/github-actions/impact-shadow-workspace.yml)
+  does the same for a pnpm workspace that tests each package on its own. It
+  needs a release newer than 0.2.1.
 
 ### Help measure it
 
@@ -128,7 +162,8 @@ If you add `impact-shadow.yml` to your CI, it changes nothing about your build.
 It uploads the counts as a `variant-reconciliation` artifact: commit SHAs,
 branch names and numbers, no file paths and no source.
 [Share them in an issue](https://github.com/saintparish4/variant/issues/new?template=3.shadow_results.yml),
-and they go into the measurement that decides whether skipping ever ships.
+and they go into the measurement that decides whether skipping ever ships. The
+[measurement log](./docs/measurement.md) has every number collected so far.
 
 ## Benchmarks
 
@@ -182,24 +217,41 @@ confidence wherever it can tell it is missing something, but some edges it
 cannot see at all:
 
 - **Runtime-only wiring is invisible.** Dependency injection by token,
-  string-keyed registries, computed `require()` and `eval` never appear as
-  imports. A test that reaches code only this way is not selected, and nothing
-  in the output says so. This is the largest blind spot.
+  string-keyed registries and `eval` never appear as imports. A test that
+  reaches code only this way is not selected, and nothing in the output says
+  so. This is the largest blind spot.
 - **Service boundaries end the graph.** An end-to-end test that calls a running
   server imports none of the code it exercises.
 - **Setup files are recognized by name.** `vitest.setup.ts`, `setupTests.ts`,
   `global-setup.ts` and the like select every test; a setup file named
   anything else selects none. variant does not read the runner config.
-- **Fixtures, snapshots and non-TypeScript assets** a test reads at runtime are
-  outside its import closure. Unresolved imports lower the confidence score and
-  add a note, but the missing edges stay missing.
+- **Files read at runtime are outside the closure.** A changed stylesheet or
+  JSON file that a TypeScript file imports selects that file's tests, but a
+  fixture, snapshot or asset a test reads with `fs` is invisible. A changed
+  file that reaches no test is listed as `Unreached`.
+- **Only the directory it runs in is indexed.** A file changed elsewhere in
+  the repository selects the tests of the files that import it by relative
+  path, and is listed as `Unreached` otherwise.
+- **Unreached is not the same as untested.** A changed file no test reaches
+  is listed as `Unreached` and the prediction reads `low`. variant cannot tell
+  an untested file from one a runtime lookup uses.
+- **JavaScript is indexed when git says it is yours.** Tracked or new
+  `.js`, `.jsx`, `.mjs` and `.cjs` files are read like TypeScript; build
+  output and bundles are not. A changed CommonJS module reaches everything
+  that requires it.
+- **Computed imports are followed only as far as their literal start.**
+  `` import(`./locales/${lang}.js`) `` reaches everything under `./locales/`;
+  `import(name)` could load anything, and is a standing note on every run.
 - **Dynamic `import()` is assumed to use everything.** The change propagates
   and a note is printed; the run gets wider, not narrower.
+- **Workspace discovery can miss a package**, and a bare import of a package it
+  missed counts as external. `impact` prints how many packages it found and
+  notes imports no discovered package or manifest accounts for.
 - **Type-only changes are not narrowed.** Changing an interface selects every
   test that imports the file, though no runtime behavior changed.
 - **The confidence score is not a safety number.** It says how much of the
-  import graph resolved. Only the false-skip rate says how safe a skip would
-  have been.
+  import graph resolved, as `high`, `medium` or `low` and a percentage. Only
+  the false-skip rate says how safe a skip would have been.
 
 The full list, with how each case is handled, is in
 [Impact & workspace](./docs/impact-and-workspace.md#limitations).
@@ -212,11 +264,13 @@ The full list, with how each case is handled, is in
 | [Getting started](./docs/getting-started.md) | Run it on your own repository, then set up the task runner |
 | [Impact & workspace](./docs/impact-and-workspace.md) | How `impact` and `workspace check` work, and what static analysis cannot see |
 | [PR commands](./docs/pr-commands.md) | `pr check` and `pr report` |
+| [Vitest adapter](./docs/vitest-adapter.md) | One config line in place of the commands: predict and check on every test run |
 | [CLI reference](./docs/cli-reference.md) | Every command, flag, exit code and file |
 | [API reference](./docs/api.md) | `defineConfig`, the config types, and every JSON output |
 | [Monorepo setup](./docs/monorepo.md) | Workspace tasks and `--affected` |
 | [Config reference](./docs/config-reference.md) | Every `variant.config.ts` key |
 | [Troubleshooting](./docs/troubleshooting.md) | Common problems and what they mean |
+| [Measurement log](./docs/measurement.md) | Every prediction checked against a full run so far, and the gate skipping has to pass |
 
 ## Contributing
 

package/dist/{build-3DSBMMI3.js → build-GKWNLU4Q.js}

@@ -2,23 +2,24 @@ import {
   executeTarget,
   renderDryRunPlan,
   reportRunInsights
-} from "./chunk-FA64KWJ2.js";
-import "./chunk-43VLELTL.js";
+} from "./chunk-7NC7AO75.js";
+import "./chunk-IHXJ72XA.js";
+import "./chunk-CWF6QXEW.js";
 import "./chunk-QRPI6TJE.js";
 import {
   createContext,
   toRunOptions
-} from "./chunk-ZH2PQUNL.js";
+} from "./chunk-CN25SRKD.js";
 import "./chunk-V5CTGXMM.js";
-import "./chunk-26DHEXGS.js";
-import "./chunk-YYGCSKIQ.js";
+import "./chunk-RJBFCURK.js";
 import "./chunk-GDSZFIHM.js";
-import "./chunk-7OPTWU6E.js";
+import "./chunk-DMDAJPFJ.js";
 import "./chunk-3E5LBU3O.js";
-import "./chunk-6BXSZPHC.js";
+import "./chunk-YYGCSKIQ.js";
 import "./chunk-6J4XDG44.js";
 import "./chunk-EISTWXDW.js";
-import "./chunk-VYWPQKMS.js";
+import "./chunk-RSIOYQOQ.js";
+import "./chunk-ACTY42RE.js";
 
 // src/core/scope/task-filter.ts
 function taskPackage(taskName) {

package/dist/{check-SIQJU6Z2.js → check-KO5VY67Z.js}

@@ -1,21 +1,21 @@
 import {
   createContext
-} from "./chunk-ZH2PQUNL.js";
+} from "./chunk-CN25SRKD.js";
 import "./chunk-V5CTGXMM.js";
-import "./chunk-26DHEXGS.js";
-import "./chunk-YYGCSKIQ.js";
-import "./chunk-7OPTWU6E.js";
+import "./chunk-RJBFCURK.js";
+import "./chunk-DMDAJPFJ.js";
 import "./chunk-3E5LBU3O.js";
-import "./chunk-6BXSZPHC.js";
+import "./chunk-YYGCSKIQ.js";
 import {
   lines
 } from "./chunk-6J4XDG44.js";
 import {
   getPrinter
 } from "./chunk-EISTWXDW.js";
+import "./chunk-RSIOYQOQ.js";
 import {
   ConfigError
-} from "./chunk-VYWPQKMS.js";
+} from "./chunk-ACTY42RE.js";
 
 // src/core/graph/validation.ts
 function validateTaskGraph(tasks, graph) {

package/dist/{chunk-FA64KWJ2.js → chunk-7NC7AO75.js}

@@ -1,23 +1,32 @@
+import {
+  CLEAR_LINE,
+  HIDE_CURSOR,
+  SHOW_CURSOR,
+  SYNC_END,
+  SYNC_START,
+  cursorUp,
+  fitWidth
+} from "./chunk-IHXJ72XA.js";
 import {
   computeInsights,
   evictStaleEntries,
   readCache,
   writeCache,
   writeCacheSync
-} from "./chunk-43VLELTL.js";
+} from "./chunk-CWF6QXEW.js";
 import {
   renderInsights
 } from "./chunk-QRPI6TJE.js";
 import {
   recordCacheMiss,
   runScheduled
-} from "./chunk-ZH2PQUNL.js";
-import {
-  getColors
-} from "./chunk-YYGCSKIQ.js";
+} from "./chunk-CN25SRKD.js";
 import {
   mapLimit
 } from "./chunk-GDSZFIHM.js";
+import {
+  getColors
+} from "./chunk-YYGCSKIQ.js";
 import {
   lines
 } from "./chunk-6J4XDG44.js";
@@ -27,7 +36,7 @@ import {
 import {
   TaskExecutionError,
   VariantError
-} from "./chunk-VYWPQKMS.js";
+} from "./chunk-ACTY42RE.js";
 
 // src/core/execution/runner.ts
 import { createHash as createHash2 } from "crypto";
@@ -283,28 +292,6 @@ async function runTasksWithDeps(target, graph, options, executor = executeTask)
   return results;
 }
 
-// src/cli/visuals/ansi.ts
-var ESC = "\x1B";
-var HIDE_CURSOR = `${ESC}[?25l`;
-var SHOW_CURSOR = `${ESC}[?25h`;
-var CLEAR_LINE = `${ESC}[2K`;
-var SYNC_START = `${ESC}[?2026h`;
-var SYNC_END = `${ESC}[?2026l`;
-function cursorUp(lines2) {
-  return lines2 > 0 ? `${ESC}[${lines2}A` : "";
-}
-var STYLE_PATTERN = new RegExp(`${ESC}\\[[0-9;]*m`, "g");
-function stripStyles(text) {
-  return text.replace(STYLE_PATTERN, "");
-}
-function visibleLength(line) {
-  return stripStyles(line).length;
-}
-function fitWidth(line, columns) {
-  if (visibleLength(line) <= columns) return line;
-  return stripStyles(line).slice(0, Math.max(0, columns - 1));
-}
-
 // src/cli/visuals/progress.ts
 var SPINNER_FRAMES = ["\u280B", "\u2819", "\u2839", "\u2838", "\u283C", "\u2834", "\u2826", "\u2827", "\u2807", "\u280F"];
 var TICK_MS = 80;

package/dist/{chunk-VYWPQKMS.js → chunk-ACTY42RE.js}

@@ -62,12 +62,30 @@ var GitRefError = class extends VariantError {
     this.hint = "Check the ref with `git rev-parse --verify <ref>`. In CI, check out with `fetch-depth: 0` and pass a remote-tracking ref such as `origin/main`.";
   }
 };
+var NoBaseCommitError = class extends VariantError {
+  constructor() {
+    super(
+      "NO_BASE_COMMIT",
+      "This push has no previous commit to compare against"
+    );
+    this.hint = "The first push of a branch, or a force-push whose old head is gone. Pass --base to compare against a commit of your choice.";
+  }
+};
 var ImpactReportError = class extends VariantError {
   constructor(message) {
     super("IMPACT_REPORT_ERROR", `Could not read the test report: ${message}`);
     this.hint = "Pass a Vitest `--reporter=json --outputFile=<path>` or Jest `--json --outputFile=<path>` report.";
   }
 };
+var NoTestFilesError = class extends VariantError {
+  constructor() {
+    super(
+      "NO_TEST_FILES",
+      "No test files found, so there is nothing to predict"
+    );
+    this.hint = "variant finds TypeScript tests named `*.test.ts` or `*.spec.ts` (also `.tsx`, `.mts`, `.cts`) or placed under `__tests__/`. JavaScript files are not indexed.";
+  }
+};
 var CliUsageError = class extends VariantError {
   constructor(message) {
     super("CLI_USAGE", message);
@@ -83,6 +101,8 @@ export {
   CacheError,
   GraphError,
   GitRefError,
+  NoBaseCommitError,
   ImpactReportError,
+  NoTestFilesError,
   CliUsageError
 };

package/dist/{chunk-ZH2PQUNL.js → chunk-CN25SRKD.js}

@@ -7,19 +7,19 @@ import {
 } from "./chunk-V5CTGXMM.js";
 import {
   loadConfig
-} from "./chunk-26DHEXGS.js";
-import {
-  getColors
-} from "./chunk-YYGCSKIQ.js";
+} from "./chunk-RJBFCURK.js";
 import {
   changedFilesToPackages,
   getChangedFiles
-} from "./chunk-7OPTWU6E.js";
+} from "./chunk-DMDAJPFJ.js";
 import {
   computeAffectedPackages,
   loadPackageGraph,
   tasksFromPackageGraph
 } from "./chunk-3E5LBU3O.js";
+import {
+  getColors
+} from "./chunk-YYGCSKIQ.js";
 import {
   errorLines
 } from "./chunk-6J4XDG44.js";
@@ -29,7 +29,7 @@ import {
 import {
   ConfigError,
   CycleError
-} from "./chunk-VYWPQKMS.js";
+} from "./chunk-ACTY42RE.js";
 
 // src/adapters/frameworks/plugin.ts
 var wrapFrameworkAsPlugin = (a) => ({

package/dist/{chunk-43VLELTL.js → chunk-CWF6QXEW.js}

@@ -1,6 +1,6 @@
 import {
   CacheError
-} from "./chunk-VYWPQKMS.js";
+} from "./chunk-ACTY42RE.js";
 
 // src/core/cache/store.ts
 import { mkdirSync, renameSync, unlinkSync, writeFileSync } from "fs";

package/dist/{chunk-7OPTWU6E.js → chunk-DMDAJPFJ.js}

@@ -3,7 +3,7 @@ import {
 } from "./chunk-3E5LBU3O.js";
 import {
   listChangedFiles
-} from "./chunk-6BXSZPHC.js";
+} from "./chunk-RSIOYQOQ.js";
 
 // src/core/cache/git-diff.ts
 import path from "path";

package/dist/{chunk-RC23JQKI.js → chunk-DSQDMFQ6.js}

@@ -1,4 +1,11 @@
 // src/core/semantic/surface.ts
+var STAR_REEXPORT_PREFIX = "* from ";
+function starReexportKey(moduleSpec) {
+  return `${STAR_REEXPORT_PREFIX}${moduleSpec}`;
+}
+function isStarReexportKey(name) {
+  return name.startsWith(STAR_REEXPORT_PREFIX);
+}
 function collectExportedSurface(file, tsm) {
   const symbols = /* @__PURE__ */ new Map();
   const notes = [];
@@ -42,7 +49,7 @@ function collectExportedSurface(file, tsm) {
       });
       continue;
     }
-    symbols.set(`* from ${moduleSpec}`, {
+    symbols.set(starReexportKey(moduleSpec), {
       signature: reexportSignature("*", moduleSpec, declTypeOnly),
       body: "",
       typeSpace: declTypeOnly,
@@ -345,6 +352,7 @@ function normalize(text) {
 }
 
 export {
+  isStarReexportKey,
   collectExportedSurface,
   significantText
 };