flakelab 0.1.0

package/README.md

@@ -0,0 +1,341 @@
+# FlakeLab
+
+FlakeLab is an AI debugging scientist for flaky Playwright tests. The product architecture,
+scope, and milestone-based development plan live in
+[`../../FLAKELAB.md`](../../FLAKELAB.md).
+
+## Quick start
+
+Run a bounded, local, credit-free stability scan from any Playwright project:
+
+```bash
+npx flakelab@latest .
+```
+
+Pass a file when you already know which test is suspicious, and increase repetitions when the
+failure is rare:
+
+```bash
+npx flakelab@latest tests/checkout.spec.ts --runs 20
+npx flakelab@latest tests/checkout.spec.ts --runs 20 --verbose
+```
+
+The default command repeatedly executes the selected target in isolated Playwright processes,
+distinguishes intermittent failures from consistent test failures, prints a concise result, and
+writes machine-readable evidence to `.flakelab/runs/scan.json`. It does not call Groq or Solari.
+
+For the complete causal workflow, use FlakeLab's Playwright fixture in tests that should accept
+controlled browser faults:
+
+```ts
+import { expect, test } from "flakelab/playwright"
+```
+
+Then explicitly opt into AI investigation and isolated proof:
+
+```bash
+npx flakelab@latest tests/checkout.spec.ts \
+  --prove \
+  --pattern "**/api/checkout" \
+  --open
+```
+
+`--prove` requires `GROQ_API_KEY` and `SOLARI_API_KEY`. It chains trigger discovery, reproducer
+verification, bounded AI investigation, candidate generation, proof in a disposable Solari
+microVM, and the offline evidence report. The candidate remains a reviewable diff and is never
+applied automatically.
+
+See every command and default without running tests:
+
+```bash
+npx flakelab@latest --help
+```
+
+Check whether the current project is ready without running tests or consuming provider credits:
+
+```bash
+npx flakelab@latest doctor
+```
+
+The doctor checks Node.js, Playwright configuration, Chromium, ignored evidence output,
+credential availability, and test-process credential isolation. It reports only whether a
+credential is configured and where it came from; it never prints the value or a fingerprint.
+
+The local diagnostic core can already run controlled baseline and fault-injected trials:
+
+```bash
+pnpm flakelab diagnose tests/fixtures/flaky-checkout.spec.ts \
+  --runs 4 \
+  --seed 42 \
+  --delay-ms 250
+```
+
+The command runs alternating control and network-delay trials, prints their failure rates,
+and writes a validated NDJSON event stream under `.flakelab/runs/`. The included checkout
+fixture passes normally and fails reliably when its checkout request is delayed.
+
+## Discover and replay a minimal reproducer
+
+Run repeated baseline and fault trials, then minimize the delay that crosses the configured
+failure-rate and confidence thresholds:
+
+```bash
+pnpm flakelab discover tests/fixtures/flaky-checkout.spec.ts \
+  --trials 4 \
+  --concurrency 4 \
+  --seed 42 \
+  --max-delay 125 \
+  --min-rate 0.7 \
+  --output flakelab.repro.yaml
+```
+
+The command writes a strict portable YAML reproducer and a JSON discovery sidecar containing
+the evidence for every candidate. It normalizes Playwright failures into stable signatures,
+uses an 80% Wilson lower confidence bound, and accepts a trigger only when both the observed
+rate and lower bound meet the requested minimum. Replay the result with:
+
+```bash
+pnpm flakelab replay flakelab.repro.yaml --concurrency 4
+```
+
+The current browser-fault matrix supports deterministic network delay and injected HTTP
+failures. Fault routes are removed after every trial, including failed tests.
+
+## AI investigator
+
+Run the investigator directly. If `GROQ_API_KEY` is not already configured, an interactive
+terminal requests it through hidden input and keeps it in memory for this run only:
+
+```bash
+pnpm flakelab investigate tests/fixtures/flaky-checkout.spec.ts \
+  --trials 4 \
+  --concurrency 4 \
+  --max-delay 125
+```
+
+The investigator uses the provider-neutral [Vercel AI SDK](https://ai-sdk.dev/docs/agents/overview)
+with Groq's [`qwen/qwen3.8-27b`](https://console.groq.com/docs/model/qwen/qwen3.8-27b)
+model. It makes two bounded model calls: one to propose competing hypotheses and a three-part
+experiment batch, and one to assess the resulting evidence. FlakeLab—not the model—executes
+trials, calculates confidence, enforces budgets, and decides whether a causal claim is valid.
+
+The model receives only the selected test and at most eight local imported source files through
+a 64 KiB, path-confined, credential-blocking reader. The resulting evidence-backed report is
+written to `flakelab.investigation.json`. If no model key is configured, deterministic
+`diagnose`, `discover`, and `replay` commands continue to work.
+
+## Bring your own credentials
+
+FlakeLab connects directly from the developer's machine or CI runner to Groq and Solari. It does
+not operate a credential relay or store provider keys in evidence. Credential lookup follows this
+order:
+
+1. the current process environment, including protected CI environment secrets;
+2. a local `.env` file for compatibility with existing workflows;
+3. hidden interactive input, retained in memory only until the FlakeLab process exits.
+
+Do not put credentials in command arguments: shell history and process inspection can expose
+them. FlakeLab deliberately has no `--api-key` option. Local `.env` files and `.flakelab` evidence
+are ignored by this repository, but protected CI secrets or the hidden run-once prompt are the
+preferred choices.
+
+Groq and Solari credentials are removed from the environment passed to Playwright subprocesses.
+Application-specific environment variables remain available so existing test suites continue to
+work. Provider keys are used only by the FlakeLab process at the API boundary and are never
+injected into disposable proof sandboxes.
+
+To ignore an outdated shell or `.env` credential and enter replacements securely for one run:
+
+```bash
+npx flakelab@latest . --prove --prompt-credentials
+```
+
+FlakeLab requests each required key once, hides terminal input, and reuses it only inside that
+process. It does not modify the shell, `.env`, or CI configuration.
+
+Provider failures are normalized from typed SDK errors, HTTP status, and machine-readable error
+codes. Authentication, permissions, billing, rate limits, concurrency caps, capacity, network,
+timeouts, and invalid requests receive separate next actions. FlakeLab does not branch on provider
+prose or print raw response bodies. Groq rate and capacity failures may be retried within the
+configured bound; Solari concurrency `429` responses are not retried because a session must be
+released before another can start.
+
+## Isolated candidate repair
+
+After reviewing an investigation, generate and prove a candidate without changing the working
+tree:
+
+```bash
+pnpm flakelab repair flakelab.investigation.json \
+  --reproducer flakelab.repro.yaml \
+  --patch candidate.diff \
+  --proof flakelab.proof.json
+```
+
+This command requires both `GROQ_API_KEY` and `SOLARI_API_KEY`. The model can propose only exact,
+bounded edits to application source it previously received. FlakeLab rejects test changes,
+assertion weakening, lint suppressions, credential-like additions, path escapes, and numeric-only
+timeout increases before execution.
+
+The candidate is copied into a disposable Solari microVM. Typecheck, lint, hostile trials, clean
+controls, and nearby regression tests all run there. The machine is destroyed afterward, and the
+candidate is returned as a reviewable diff; FlakeLab never applies it to the local checkout.
+Cold validation installs the pinned Node/pnpm toolchain and Chromium, so it is intentionally
+slower than the future snapshot-backed warm path.
+
+## Evidence report
+
+Turn the validated investigation, reproducer, candidate, and proof into one portable report:
+
+```bash
+pnpm flakelab report flakelab.investigation.json \
+  --reproducer flakelab.repro.yaml \
+  --proof flakelab.proof.json \
+  --patch candidate.diff \
+  --html flakelab.report.html \
+  --open
+```
+
+The React/Recharts interface is bundled by Vite into a single HTML file. It explains the root
+cause, deterministic ownership classification, experiment timeline, competing hypotheses,
+minimal trigger, before/after proof matrix, static checks, model usage, and reviewable artifacts.
+The evidence is schema-validated and credential-redacted before rendering. A restrictive content
+security policy blocks runtime network access, so the report remains usable offline.
+
+`--open` launches only the local file. `--publish` is optional and always asks for interactive
+confirmation immediately before creating a public Solari preview. Published reports expire and
+their hosting sandbox is automatically killed after at most 60 minutes.
+
+## Statistical Git bisect
+
+After producing a deterministic reproducer, locate the introducing commit without executing
+historical code on the developer machine:
+
+```bash
+pnpm flakelab bisect \
+  --good v1.4.0 \
+  --bad HEAD \
+  --reproducer flakelab.repro.yaml \
+  --bisect-report flakelab.bisect.json
+```
+
+The selected good revision must be an ancestor of the bad revision, and both endpoints are
+measured rather than trusted by name. FlakeLab archives each candidate without Git credentials,
+prepares up to two midpoint revisions concurrently in disposable Solari sandboxes, and runs each
+probabilistic trial in an independent fork of that revision's snapshot.
+
+Classification uses both sides of an 80% Wilson interval. A revision is bad only when the lower
+bound reaches `--min-rate`; it is good only when the upper bound stays below that threshold.
+Ambiguous evidence receives another trial batch up to `--max-trials`. Dependency installation or
+test-discovery failures are recorded as incompatible instead of being counted as test failures.
+
+The JSON report contains every evaluated commit, pass/fail/error counts, confidence bounds,
+snapshot reuse, duration, and decision reason. `firstFailingCommit` is populated only when the
+good/bad boundary is exact. If an incompatible or inconclusive commit hides the boundary, the CLI
+reports `earliestKnownBadCommit`, exits with code 2, and does not overclaim an exact answer.
+
+Run the explicit credit-consuming end-to-end demonstration with:
+
+```bash
+pnpm verify:bisect
+```
+
+It creates a temporary four-commit repository with one intentional hydration regression, bisects
+it in Solari, writes `.flakelab/bisect-demo.json`, and removes its sandboxes, snapshots, and local
+temporary history. It is separate from the default test suite.
+
+## GitHub Actions
+
+The repository includes a reusable composite action and a pull-request workflow:
+
+- `../../.github/actions/flakelab/action.yml` installs the pinned Node/pnpm/Chromium toolchain,
+  selects changed tests, diagnoses one bounded target, uploads all evidence, and writes the job
+  summary.
+- `../../.github/workflows/flakelab.yml` runs the credit-free quality gate on every matching pull
+  request and gates provider-backed diagnosis behind the protected `flakelab` environment.
+
+Configure `GROQ_API_KEY` and `SOLARI_API_KEY` as GitHub environment secrets, then add required
+reviewers to the `flakelab` environment. Fork pull requests never receive those secrets, checkout
+credentials are not persisted, and the workflow deliberately avoids `pull_request_target`.
+
+Changed-test selection prefers directly modified Playwright specs. When application, support,
+Playwright configuration, manifest, or lockfile behavior changes, it falls back to the bounded
+`tests/e2e` and `tests/fixtures` suite. The current action deeply diagnoses the first selected test
+to keep time and cost predictable; `.flakelab/changed-tests.json` preserves the complete selection.
+
+The uploaded artifact is retained for seven days and includes the offline HTML report,
+investigation, reproducer, discovery evidence, proof, candidate diff, and selection manifest. A
+rejected repair still uploads its evidence before the job reports failure.
+
+## Parallel Solari runner
+
+Run the live eight-worker verification explicitly so routine tests remain local and
+credit-free:
+
+```bash
+pnpm verify:solari-parallel
+```
+
+For a sequential comparison:
+
+```bash
+pnpm verify:solari-parallel -- --concurrency 1 --runs 8
+```
+
+The runner prepares and snapshots the application once, reuses snapshots by a cache key
+derived from the Git commit, pnpm lockfile, and fixture configuration, and shares one
+application sandbox for browser-only faults. Each trial receives an independent Solari
+browser session. Metrics include wall time, cumulative trial time, peak concurrency,
+infrastructure retries, cache status, and created/released resource counts.
+
+## Live Solari verification
+
+The Solari verification script proves the remote execution path:
+
+1. create a Solari sandbox;
+2. write and start a tiny test application;
+3. expose it through a preview URL;
+4. snapshot the running sandbox and fork a second sandbox from it;
+5. launch a recorded Solari browser against the fork;
+6. inject network latency and verify the application still reaches its ready state;
+7. save a Playwright trace as the guaranteed diagnostic artifact;
+8. retrieve the optional Solari rrweb replay when available;
+9. destroy both sandboxes, even when a step fails.
+
+## Verify Solari
+
+Set `SOLARI_API_KEY` in your shell, or add it to this directory's `.env` file:
+
+```text
+SOLARI_API_KEY=...
+```
+
+Then run the explicit live check:
+
+```bash
+pnpm install
+pnpm verify:solari
+```
+
+The command exits non-zero if any required capability is not proven. It never prints the API key.
+
+## Development quality gates
+
+```bash
+pnpm typecheck
+pnpm lint
+pnpm test
+```
+
+Run `pnpm quality` before handing back a completed behavior change. The default Playwright suite
+is local and credit-free; `repair`, `pnpm verify:solari`, and `pnpm verify:solari-parallel` are
+explicit live integrations that can consume Solari credits. `pnpm verify:bisect` is also an
+explicit live integration.
+
+## Solari documentation
+
+- [Sandboxes](https://docs.getsolari.com/sandboxes)
+- [Snapshots](https://docs.getsolari.com/snapshots)
+- [Browser sessions](https://docs.getsolari.com/sessions)
+- [Session recording](https://docs.getsolari.com/recording)
+- [Browser API reference](https://docs.getsolari.com/api-reference/browser)

package/bin/flakelab.mjs

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+
+await import("../dist/cli.js")

package/dist/artifacts/ndjson.d.ts

@@ -0,0 +1,3 @@
+import type { RunEvent } from "../domain/schema.js";
+export type EventWriter = (event: RunEvent) => Promise<void>;
+export declare function createNdjsonWriter(filePath: string): EventWriter;

package/dist/artifacts/ndjson.js

@@ -0,0 +1,16 @@
+import { appendFile, mkdir } from "node:fs/promises";
+import { dirname } from "node:path";
+import { runEventSchema } from "../domain/schema.js";
+export function createNdjsonWriter(filePath) {
+    let pendingWrite = Promise.resolve();
+    return (event) => {
+        const write = pendingWrite.then(async () => {
+            const validatedEvent = runEventSchema.parse(event);
+            await mkdir(dirname(filePath), { recursive: true });
+            await appendFile(filePath, `${JSON.stringify(validatedEvent)}\n`, { encoding: "utf8" });
+        });
+        pendingWrite = write.catch(() => undefined);
+        return write;
+    };
+}
+//# sourceMappingURL=ndjson.js.map

package/dist/artifacts/ndjson.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"ndjson.js","sourceRoot":"","sources":["../../src/artifacts/ndjson.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,KAAK,EAAE,MAAM,kBAAkB,CAAA;AACpD,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAA;AAEnC,OAAO,EAAE,cAAc,EAAE,MAAM,qBAAqB,CAAA;AAKpD,MAAM,UAAU,kBAAkB,CAAC,QAAgB;IACjD,IAAI,YAAY,GAAG,OAAO,CAAC,OAAO,EAAE,CAAA;IACpC,OAAO,CAAC,KAAK,EAAE,EAAE;QACf,MAAM,KAAK,GAAG,YAAY,CAAC,IAAI,CAAC,KAAK,IAAI,EAAE;YACzC,MAAM,cAAc,GAAG,cAAc,CAAC,KAAK,CAAC,KAAK,CAAC,CAAA;YAClD,MAAM,KAAK,CAAC,OAAO,CAAC,QAAQ,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAA;YACnD,MAAM,UAAU,CAAC,QAAQ,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,cAAc,CAAC,IAAI,EAAE,EAAE,QAAQ,EAAE,MAAM,EAAE,CAAC,CAAA;QACzF,CAAC,CAAC,CAAA;QACF,YAAY,GAAG,KAAK,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAA;QAC3C,OAAO,KAAK,CAAA;IACd,CAAC,CAAA;AACH,CAAC"}

package/dist/bisect/archive.d.ts

@@ -0,0 +1 @@
+export declare function createRevisionArchive(repositoryRoot: string, revision: string): Promise<Buffer>;

package/dist/bisect/archive.js

@@ -0,0 +1,14 @@
+import { execFile } from "node:child_process";
+import { promisify } from "node:util";
+const MAX_ARCHIVE_BYTES = 64 * 1024 * 1024;
+const execFileAsync = promisify(execFile);
+export async function createRevisionArchive(repositoryRoot, revision) {
+    const result = await execFileAsync("git", ["archive", "--format=tar", revision], {
+        cwd: repositoryRoot,
+        encoding: "buffer",
+        maxBuffer: MAX_ARCHIVE_BYTES,
+        windowsHide: true,
+    });
+    return result.stdout;
+}
+//# sourceMappingURL=archive.js.map

package/dist/bisect/archive.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"archive.js","sourceRoot":"","sources":["../../src/bisect/archive.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,oBAAoB,CAAA;AAC7C,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAA;AAErC,MAAM,iBAAiB,GAAG,EAAE,GAAG,IAAI,GAAG,IAAI,CAAA;AAC1C,MAAM,aAAa,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAA;AAEzC,MAAM,CAAC,KAAK,UAAU,qBAAqB,CACzC,cAAsB,EACtB,QAAgB;IAEhB,MAAM,MAAM,GAAG,MAAM,aAAa,CAAC,KAAK,EAAE,CAAC,SAAS,EAAE,cAAc,EAAE,QAAQ,CAAC,EAAE;QAC/E,GAAG,EAAE,cAAc;QACnB,QAAQ,EAAE,QAAQ;QAClB,SAAS,EAAE,iBAAiB;QAC5B,WAAW,EAAE,IAAI;KAClB,CAAC,CAAA;IACF,OAAO,MAAM,CAAC,MAAM,CAAA;AACtB,CAAC"}

package/dist/bisect/confidence.d.ts

@@ -0,0 +1,7 @@
+export interface ConfidenceInterval {
+    lower: number;
+    upper: number;
+}
+export declare function wilsonInterval80(failures: number, trials: number): ConfidenceInterval;
+export type ConfidenceClassification = "bad" | "good" | "inconclusive";
+export declare function classifyFailureProbability(failures: number, trials: number, errors: number, minimumFailureRate: number): ConfidenceClassification;

package/dist/bisect/confidence.js

@@ -0,0 +1,27 @@
+const WILSON_Z_80 = 1.281_551_565_545;
+export function wilsonInterval80(failures, trials) {
+    if (trials === 0) {
+        return { lower: 0, upper: 1 };
+    }
+    const probability = failures / trials;
+    const squaredZ = WILSON_Z_80 ** 2;
+    const denominator = 1 + squaredZ / trials;
+    const center = probability + squaredZ / (2 * trials);
+    const margin = WILSON_Z_80 * Math.sqrt((probability * (1 - probability) + squaredZ / (4 * trials)) / trials);
+    return {
+        lower: Math.max(0, (center - margin) / denominator),
+        upper: Math.min(1, (center + margin) / denominator),
+    };
+}
+export function classifyFailureProbability(failures, trials, errors, minimumFailureRate) {
+    const validTrials = trials - errors;
+    if (validTrials <= 0 || failures > validTrials) {
+        return "inconclusive";
+    }
+    const interval = wilsonInterval80(failures, validTrials);
+    if (interval.lower >= minimumFailureRate) {
+        return "bad";
+    }
+    return interval.upper < minimumFailureRate ? "good" : "inconclusive";
+}
+//# sourceMappingURL=confidence.js.map

package/dist/bisect/confidence.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"confidence.js","sourceRoot":"","sources":["../../src/bisect/confidence.ts"],"names":[],"mappings":"AAAA,MAAM,WAAW,GAAG,iBAAiB,CAAA;AAOrC,MAAM,UAAU,gBAAgB,CAAC,QAAgB,EAAE,MAAc;IAC/D,IAAI,MAAM,KAAK,CAAC,EAAE,CAAC;QACjB,OAAO,EAAE,KAAK,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,CAAA;IAC/B,CAAC;IACD,MAAM,WAAW,GAAG,QAAQ,GAAG,MAAM,CAAA;IACrC,MAAM,QAAQ,GAAG,WAAW,IAAI,CAAC,CAAA;IACjC,MAAM,WAAW,GAAG,CAAC,GAAG,QAAQ,GAAG,MAAM,CAAA;IACzC,MAAM,MAAM,GAAG,WAAW,GAAG,QAAQ,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAA;IACpD,MAAM,MAAM,GAAG,WAAW,GAAG,IAAI,CAAC,IAAI,CACpC,CAAC,WAAW,GAAG,CAAC,CAAC,GAAG,WAAW,CAAC,GAAG,QAAQ,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,GAAG,MAAM,CACrE,CAAA;IACD,OAAO;QACL,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,MAAM,GAAG,MAAM,CAAC,GAAG,WAAW,CAAC;QACnD,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,MAAM,GAAG,MAAM,CAAC,GAAG,WAAW,CAAC;KACpD,CAAA;AACH,CAAC;AAID,MAAM,UAAU,0BAA0B,CACxC,QAAgB,EAChB,MAAc,EACd,MAAc,EACd,kBAA0B;IAE1B,MAAM,WAAW,GAAG,MAAM,GAAG,MAAM,CAAA;IACnC,IAAI,WAAW,IAAI,CAAC,IAAI,QAAQ,GAAG,WAAW,EAAE,CAAC;QAC/C,OAAO,cAAc,CAAA;IACvB,CAAC;IACD,MAAM,QAAQ,GAAG,gBAAgB,CAAC,QAAQ,EAAE,WAAW,CAAC,CAAA;IACxD,IAAI,QAAQ,CAAC,KAAK,IAAI,kBAAkB,EAAE,CAAC;QACzC,OAAO,KAAK,CAAA;IACd,CAAC;IACD,OAAO,QAAQ,CAAC,KAAK,GAAG,kBAAkB,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,cAAc,CAAA;AACtE,CAAC"}

package/dist/bisect/engine.d.ts

@@ -0,0 +1,10 @@
+import type { BisectReport, Revision, RevisionEvidence } from "./schema.js";
+export type RevisionEvaluator = (revision: Revision) => Promise<RevisionEvidence>;
+interface BisectOptions {
+    evaluate: RevisionEvaluator;
+    minimumFailureRate: number;
+    parallelism?: number;
+    revisions: Revision[];
+}
+export declare function statisticalBisect(options: BisectOptions): Promise<BisectReport>;
+export {};

package/dist/bisect/engine.js

@@ -0,0 +1,110 @@
+import { bisectReportSchema } from "./schema.js";
+function selectCandidates(lower, upper, evaluated, parallelism) {
+    const available = Array.from({ length: Math.max(0, upper - lower - 1) }, (_, offset) => lower + offset + 1).filter((index) => !evaluated.has(index));
+    if (available.length <= parallelism) {
+        return available;
+    }
+    return Array.from({ length: parallelism }, (_, slot) => {
+        const position = Math.floor(((slot + 1) * available.length) / (parallelism + 1));
+        return available[Math.min(position, available.length - 1)];
+    }).filter((value, index, values) => values.indexOf(value) === index);
+}
+function assertMonotonic(evaluated) {
+    const goodIndexes = [...evaluated.entries()]
+        .filter((entry) => entry[1].classification === "good")
+        .map((entry) => entry[0]);
+    const badIndexes = [...evaluated.entries()]
+        .filter((entry) => entry[1].classification === "bad")
+        .map((entry) => entry[0]);
+    if (goodIndexes.length > 0 && badIndexes.length > 0
+        && Math.max(...goodIndexes) >= Math.min(...badIndexes)) {
+        throw new Error("revision evidence is non-monotonic; a good revision follows a bad revision");
+    }
+}
+function boundaries(evaluated) {
+    const good = [...evaluated.entries()]
+        .filter((entry) => entry[1].classification === "good")
+        .map((entry) => entry[0]);
+    const bad = [...evaluated.entries()]
+        .filter((entry) => entry[1].classification === "bad")
+        .map((entry) => entry[0]);
+    return [Math.max(...good), Math.min(...bad)];
+}
+async function evaluateIndexes(indexes, revisions, evaluate, evaluated) {
+    const results = await Promise.all(indexes.map(async (index) => {
+        try {
+            return {
+                status: "fulfilled",
+                index,
+                evidence: await evaluate(revisions[index]),
+            };
+        }
+        catch (error) {
+            return {
+                status: "rejected",
+                error: error instanceof Error ? error : new Error("revision evaluation failed"),
+            };
+        }
+    }));
+    const failure = results.find((result) => result.status === "rejected");
+    if (failure?.status === "rejected") {
+        throw failure.error;
+    }
+    for (const result of results) {
+        if (result.status === "fulfilled") {
+            evaluated.set(result.index, result.evidence);
+        }
+    }
+}
+function validateOptions(options) {
+    if (options.revisions.length < 2) {
+        throw new Error("bisect requires at least two revisions");
+    }
+    if (options.minimumFailureRate <= 0 || options.minimumFailureRate > 1) {
+        throw new Error("minimum failure rate must be greater than 0 and at most 1");
+    }
+    const parallelism = options.parallelism ?? 2;
+    if (!Number.isInteger(parallelism) || parallelism < 1 || parallelism > 4) {
+        throw new Error("bisect parallelism must be an integer between 1 and 4");
+    }
+    return parallelism;
+}
+export async function statisticalBisect(options) {
+    const parallelism = validateOptions(options);
+    const evaluated = new Map();
+    const lastIndex = options.revisions.length - 1;
+    await evaluateIndexes([0, lastIndex], options.revisions, options.evaluate, evaluated);
+    if (evaluated.get(0)?.classification !== "good") {
+        throw new Error("the selected good revision was not classified as good");
+    }
+    if (evaluated.get(lastIndex)?.classification !== "bad") {
+        throw new Error("the selected bad revision was not classified as bad");
+    }
+    let [goodIndex, badIndex] = boundaries(evaluated);
+    while (badIndex - goodIndex > 1) {
+        const candidates = selectCandidates(goodIndex, badIndex, evaluated, parallelism);
+        if (candidates.length === 0) {
+            break;
+        }
+        await evaluateIndexes(candidates, options.revisions, options.evaluate, evaluated);
+        assertMonotonic(evaluated);
+        [goodIndex, badIndex] = boundaries(evaluated);
+    }
+    const exact = badIndex - goodIndex === 1;
+    const evidence = [...evaluated.entries()]
+        .sort((left, right) => left[0] - right[0])
+        .map((entry) => entry[1]);
+    return bisectReportSchema.parse({
+        version: 1,
+        goodRevision: options.revisions[0],
+        badRevision: options.revisions[lastIndex],
+        firstFailingCommit: exact ? options.revisions[badIndex] : null,
+        earliestKnownBadCommit: options.revisions[badIndex],
+        exact,
+        minimumFailureRate: options.minimumFailureRate,
+        evaluatedRevisionCount: evaluated.size,
+        totalRevisionCount: options.revisions.length,
+        evidence,
+    });
+}
+//# sourceMappingURL=engine.js.map

package/dist/bisect/engine.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"engine.js","sourceRoot":"","sources":["../../src/bisect/engine.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,kBAAkB,EAAE,MAAM,aAAa,CAAA;AAWhD,SAAS,gBAAgB,CACvB,KAAa,EACb,KAAa,EACb,SAAgD,EAChD,WAAmB;IAEnB,MAAM,SAAS,GAAG,KAAK,CAAC,IAAI,CAC1B,EAAE,MAAM,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,GAAG,KAAK,GAAG,CAAC,CAAC,EAAE,EAC1C,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE,CAAC,KAAK,GAAG,MAAM,GAAG,CAAC,CAClC,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,SAAS,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,CAAA;IAC1C,IAAI,SAAS,CAAC,MAAM,IAAI,WAAW,EAAE,CAAC;QACpC,OAAO,SAAS,CAAA;IAClB,CAAC;IACD,OAAO,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,WAAW,EAAE,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,EAAE;QACrD,MAAM,QAAQ,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC,GAAG,SAAS,CAAC,MAAM,CAAC,GAAG,CAAC,WAAW,GAAG,CAAC,CAAC,CAAC,CAAA;QAChF,OAAO,SAAS,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,EAAE,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAA;IAC5D,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,KAAK,KAAK,CAAC,CAAA;AACtE,CAAC;AAED,SAAS,eAAe,CAAC,SAAgD;IACvE,MAAM,WAAW,GAAG,CAAC,GAAG,SAAS,CAAC,OAAO,EAAE,CAAC;SACzC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,cAAc,KAAK,MAAM,CAAC;SACrD,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAA;IAC3B,MAAM,UAAU,GAAG,CAAC,GAAG,SAAS,CAAC,OAAO,EAAE,CAAC;SACxC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,cAAc,KAAK,KAAK,CAAC;SACpD,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAA;IAC3B,IAAI,WAAW,CAAC,MAAM,GAAG,CAAC,IAAI,UAAU,CAAC,MAAM,GAAG,CAAC;WAC9C,IAAI,CAAC,GAAG,CAAC,GAAG,WAAW,CAAC,IAAI,IAAI,CAAC,GAAG,CAAC,GAAG,UAAU,CAAC,EAAE,CAAC;QACzD,MAAM,IAAI,KAAK,CAAC,4EAA4E,CAAC,CAAA;IAC/F,CAAC;AACH,CAAC;AAED,SAAS,UAAU,CAAC,SAAgD;IAClE,MAAM,IAAI,GAAG,CAAC,GAAG,SAAS,CAAC,OAAO,EAAE,CAAC;SAClC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,cAAc,KAAK,MAAM,CAAC;SACrD,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAA;IAC3B,MAAM,GAAG,GAAG,CAAC,GAAG,SAAS,CAAC,OAAO,EAAE,CAAC;SACjC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,cAAc,KAAK,KAAK,CAAC;SACpD,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAA;IAC3B,OAAO,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,GAAG,CAAC,CAAC,CAAA;AAC9C,CAAC;AAED,KAAK,UAAU,eAAe,CAC5B,OAAiB,EACjB,SAAqB,EACrB,QAA2B,EAC3B,SAAwC;IAExC,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,EAAE,EAAE;QAC5D,IAAI,CAAC;YACH,OAAO;gBACL,MAAM,EAAE,WAAoB;gBAC5B,KAAK;gBACL,QAAQ,EAAE,MAAM,QAAQ,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC;aAC3C,CAAA;QACH,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,OAAO;gBACL,MAAM,EAAE,UAAmB;gBAC3B,KAAK,EAAE,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,4BAA4B,CAAC;aAChF,CAAA;QACH,CAAC;IACH,CAAC,CAAC,CAAC,CAAA;IACH,MAAM,OAAO,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,MAAM,KAAK,UAAU,CAAC,CAAA;IACtE,IAAI,OAAO,EAAE,MAAM,KAAK,UAAU,EAAE,CAAC;QACnC,MAAM,OAAO,CAAC,KAAK,CAAA;IACrB,CAAC;IACD,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;QAC7B,IAAI,MAAM,CAAC,MAAM,KAAK,WAAW,EAAE,CAAC;YAClC,SAAS,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK,EAAE,MAAM,CAAC,QAAQ,CAAC,CAAA;QAC9C,CAAC;IACH,CAAC;AACH,CAAC;AAED,SAAS,eAAe,CAAC,OAAsB;IAC7C,IAAI,OAAO,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACjC,MAAM,IAAI,KAAK,CAAC,wCAAwC,CAAC,CAAA;IAC3D,CAAC;IACD,IAAI,OAAO,CAAC,kBAAkB,IAAI,CAAC,IAAI,OAAO,CAAC,kBAAkB,GAAG,CAAC,EAAE,CAAC;QACtE,MAAM,IAAI,KAAK,CAAC,2DAA2D,CAAC,CAAA;IAC9E,CAAC;IACD,MAAM,WAAW,GAAG,OAAO,CAAC,WAAW,IAAI,CAAC,CAAA;IAC5C,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,WAAW,CAAC,IAAI,WAAW,GAAG,CAAC,IAAI,WAAW,GAAG,CAAC,EAAE,CAAC;QACzE,MAAM,IAAI,KAAK,CAAC,uDAAuD,CAAC,CAAA;IAC1E,CAAC;IACD,OAAO,WAAW,CAAA;AACpB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,iBAAiB,CAAC,OAAsB;IAC5D,MAAM,WAAW,GAAG,eAAe,CAAC,OAAO,CAAC,CAAA;IAC5C,MAAM,SAAS,GAAG,IAAI,GAAG,EAA4B,CAAA;IACrD,MAAM,SAAS,GAAG,OAAO,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAA;IAC9C,MAAM,eAAe,CAAC,CAAC,CAAC,EAAE,SAAS,CAAC,EAAE,OAAO,CAAC,SAAS,EAAE,OAAO,CAAC,QAAQ,EAAE,SAAS,CAAC,CAAA;IACrF,IAAI,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,cAAc,KAAK,MAAM,EAAE,CAAC;QAChD,MAAM,IAAI,KAAK,CAAC,uDAAuD,CAAC,CAAA;IAC1E,CAAC;IACD,IAAI,SAAS,CAAC,GAAG,CAAC,SAAS,CAAC,EAAE,cAAc,KAAK,KAAK,EAAE,CAAC;QACvD,MAAM,IAAI,KAAK,CAAC,qDAAqD,CAAC,CAAA;IACxE,CAAC;IACD,IAAI,CAAC,SAAS,EAAE,QAAQ,CAAC,GAAG,UAAU,CAAC,SAAS,CAAC,CAAA;IACjD,OAAO,QAAQ,GAAG,SAAS,GAAG,CAAC,EAAE,CAAC;QAChC,MAAM,UAAU,GAAG,gBAAgB,CAAC,SAAS,EAAE,QAAQ,EAAE,SAAS,EAAE,WAAW,CAAC,CAAA;QAChF,IAAI,UAAU,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC5B,MAAK;QACP,CAAC;QACD,MAAM,eAAe,CAAC,UAAU,EAAE,OAAO,CAAC,SAAS,EAAE,OAAO,CAAC,QAAQ,EAAE,SAAS,CAAC,CAAA;QACjF,eAAe,CAAC,SAAS,CAAC,CACzB;QAAA,CAAC,SAAS,EAAE,QAAQ,CAAC,GAAG,UAAU,CAAC,SAAS,CAAC,CAAA;IAChD,CAAC;IACD,MAAM,KAAK,GAAG,QAAQ,GAAG,SAAS,KAAK,CAAC,CAAA;IACxC,MAAM,QAAQ,GAAG,CAAC,GAAG,SAAS,CAAC,OAAO,EAAE,CAAC;SACtC,IAAI,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;SACzC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAA;IAC3B,OAAO,kBAAkB,CAAC,KAAK,CAAC;QAC9B,OAAO,EAAE,CAAC;QACV,YAAY,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC;QAClC,WAAW,EAAE,OAAO,CAAC,SAAS,CAAC,SAAS,CAAC;QACzC,kBAAkB,EAAE,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI;QAC9D,sBAAsB,EAAE,OAAO,CAAC,SAAS,CAAC,QAAQ,CAAC;QACnD,KAAK;QACL,kBAAkB,EAAE,OAAO,CAAC,kBAAkB;QAC9C,sBAAsB,EAAE,SAAS,CAAC,IAAI;QACtC,kBAAkB,EAAE,OAAO,CAAC,SAAS,CAAC,MAAM;QAC5C,QAAQ;KACT,CAAC,CAAA;AACJ,CAAC"}

package/dist/bisect/git.d.ts

@@ -0,0 +1,7 @@
+import type { Revision } from "./schema.js";
+export interface GitHistory {
+    projectPath: string;
+    repositoryRoot: string;
+    revisions: Revision[];
+}
+export declare function resolveGitHistory(projectRoot: string, goodReference: string, badReference: string): Promise<GitHistory>;

package/dist/bisect/git.js

@@ -0,0 +1,56 @@
+import { execFile } from "node:child_process";
+import { relative, resolve } from "node:path";
+import { promisify } from "node:util";
+import { revisionSchema } from "./schema.js";
+const execFileAsync = promisify(execFile);
+const MAX_REVISIONS = 1_000;
+async function git(cwd, args) {
+    const result = await execFileAsync("git", args, {
+        cwd,
+        encoding: "utf8",
+        maxBuffer: 4 * 1024 * 1024,
+        windowsHide: true,
+    });
+    return result.stdout.trim();
+}
+async function resolveCommit(repositoryRoot, reference) {
+    if (reference.length > 200 || /[\r\n\0]/u.test(reference)) {
+        throw new Error("Git revision must be a single value no longer than 200 characters");
+    }
+    return git(repositoryRoot, ["rev-parse", "--verify", `${reference}^{commit}`]);
+}
+async function revisionDetails(repositoryRoot, hash) {
+    const subject = await git(repositoryRoot, ["show", "-s", "--format=%s", hash]);
+    return revisionSchema.parse({ hash, shortHash: hash.slice(0, 12), subject });
+}
+export async function resolveGitHistory(projectRoot, goodReference, badReference) {
+    const repositoryRoot = resolve(await git(projectRoot, ["rev-parse", "--show-toplevel"]));
+    const goodHash = await resolveCommit(repositoryRoot, goodReference);
+    const badHash = await resolveCommit(repositoryRoot, badReference);
+    try {
+        await git(repositoryRoot, ["merge-base", "--is-ancestor", goodHash, badHash]);
+    }
+    catch {
+        throw new Error("the selected good revision must be an ancestor of the bad revision");
+    }
+    const descendants = await git(repositoryRoot, [
+        "rev-list",
+        "--ancestry-path",
+        "--reverse",
+        `${goodHash}..${badHash}`,
+    ]);
+    const hashes = [goodHash, ...descendants.split(/\r?\n/u).filter(Boolean)];
+    if (hashes.length > MAX_REVISIONS) {
+        throw new Error(`bisect history exceeds the ${MAX_REVISIONS}-revision safety limit`);
+    }
+    const projectPath = relative(repositoryRoot, resolve(projectRoot));
+    if (projectPath.startsWith("..")) {
+        throw new Error("the project must be inside its Git repository");
+    }
+    return {
+        projectPath,
+        repositoryRoot,
+        revisions: await Promise.all(hashes.map((hash) => revisionDetails(repositoryRoot, hash))),
+    };
+}
+//# sourceMappingURL=git.js.map

package/dist/bisect/git.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"git.js","sourceRoot":"","sources":["../../src/bisect/git.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,oBAAoB,CAAA;AAC7C,OAAO,EAAE,QAAQ,EAAE,OAAO,EAAE,MAAM,WAAW,CAAA;AAC7C,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAA;AAGrC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AAE5C,MAAM,aAAa,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAA;AACzC,MAAM,aAAa,GAAG,KAAK,CAAA;AAE3B,KAAK,UAAU,GAAG,CAAC,GAAW,EAAE,IAAc;IAC5C,MAAM,MAAM,GAAG,MAAM,aAAa,CAAC,KAAK,EAAE,IAAI,EAAE;QAC9C,GAAG;QACH,QAAQ,EAAE,MAAM;QAChB,SAAS,EAAE,CAAC,GAAG,IAAI,GAAG,IAAI;QAC1B,WAAW,EAAE,IAAI;KAClB,CAAC,CAAA;IACF,OAAO,MAAM,CAAC,MAAM,CAAC,IAAI,EAAE,CAAA;AAC7B,CAAC;AAED,KAAK,UAAU,aAAa,CAAC,cAAsB,EAAE,SAAiB;IACpE,IAAI,SAAS,CAAC,MAAM,GAAG,GAAG,IAAI,WAAW,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC;QAC1D,MAAM,IAAI,KAAK,CAAC,mEAAmE,CAAC,CAAA;IACtF,CAAC;IACD,OAAO,GAAG,CAAC,cAAc,EAAE,CAAC,WAAW,EAAE,UAAU,EAAE,GAAG,SAAS,WAAW,CAAC,CAAC,CAAA;AAChF,CAAC;AAED,KAAK,UAAU,eAAe,CAAC,cAAsB,EAAE,IAAY;IACjE,MAAM,OAAO,GAAG,MAAM,GAAG,CAAC,cAAc,EAAE,CAAC,MAAM,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,CAAC,CAAC,CAAA;IAC9E,OAAO,cAAc,CAAC,KAAK,CAAC,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,CAAA;AAC9E,CAAC;AAQD,MAAM,CAAC,KAAK,UAAU,iBAAiB,CACrC,WAAmB,EACnB,aAAqB,EACrB,YAAoB;IAEpB,MAAM,cAAc,GAAG,OAAO,CAAC,MAAM,GAAG,CAAC,WAAW,EAAE,CAAC,WAAW,EAAE,iBAAiB,CAAC,CAAC,CAAC,CAAA;IACxF,MAAM,QAAQ,GAAG,MAAM,aAAa,CAAC,cAAc,EAAE,aAAa,CAAC,CAAA;IACnE,MAAM,OAAO,GAAG,MAAM,aAAa,CAAC,cAAc,EAAE,YAAY,CAAC,CAAA;IACjE,IAAI,CAAC;QACH,MAAM,GAAG,CAAC,cAAc,EAAE,CAAC,YAAY,EAAE,eAAe,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC,CAAA;IAC/E,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,KAAK,CAAC,oEAAoE,CAAC,CAAA;IACvF,CAAC;IACD,MAAM,WAAW,GAAG,MAAM,GAAG,CAAC,cAAc,EAAE;QAC5C,UAAU;QACV,iBAAiB;QACjB,WAAW;QACX,GAAG,QAAQ,KAAK,OAAO,EAAE;KAC1B,CAAC,CAAA;IACF,MAAM,MAAM,GAAG,CAAC,QAAQ,EAAE,GAAG,WAAW,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,CAAA;IACzE,IAAI,MAAM,CAAC,MAAM,GAAG,aAAa,EAAE,CAAC;QAClC,MAAM,IAAI,KAAK,CAAC,8BAA8B,aAAa,wBAAwB,CAAC,CAAA;IACtF,CAAC;IACD,MAAM,WAAW,GAAG,QAAQ,CAAC,cAAc,EAAE,OAAO,CAAC,WAAW,CAAC,CAAC,CAAA;IAClE,IAAI,WAAW,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;QACjC,MAAM,IAAI,KAAK,CAAC,+CAA+C,CAAC,CAAA;IAClE,CAAC;IACD,OAAO;QACL,WAAW;QACX,cAAc;QACd,SAAS,EAAE,MAAM,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,eAAe,CAAC,cAAc,EAAE,IAAI,CAAC,CAAC,CAAC;KAC1F,CAAA;AACH,CAAC"}