@gfdelarue/testkit 0.6.4 → 0.6.5

package/README.md

@@ -1,9 +1,17 @@
 <p align="center">
-  <img src="https://turnstone.cloud/assets/social-card.png" alt="AI Testkit logo" width="600">
+  <a href="https://turnstone.cloud"><img src="https://turnstone.cloud/assets/media/poster.jpg" alt="Your codebase is hiding something." width="720"></a>
 </p>
 
 # @gfdelarue/testkit
 
+Give your coding agent a target. Testkit files every bug it finds as a GitHub or
+Gitea issue, with a failing test that reproduces it. Connect
+[Turnstone](https://turnstone.cloud) to see where your bugs come from and how
+long they went unnoticed.
+
+Testkit is free. [Get started](https://turnstone.cloud/quickstart) or read the
+[docs](https://turnstone.cloud/docs).
+
 `@gfdelarue/testkit` is a test runner and CLI for coding agents and CI. It runs
 a repository's ordinary Vitest test files, such as `src/sum.test.ts`, as they
 are, and typed `*.testkit.ts` suites for Playwright and Node. It assigns
@@ -133,8 +141,9 @@ own issue and stops, naming the older one. GitHub can list a new issue a few
 seconds late, so Testkit waits up to 7.5 seconds for its own and says so when
 it doesn't appear. The token needs permission to create issues and labels.
 
-`testkit bug record` saves the `--vitest-config`, `--vitest-project`, and
-`--vitest-runner` it was given in the draft, and asks for `--vitest-project`
+`testkit bug record` saves the `--vitest-config`, `--vitest-project`,
+`--vitest-runner`, `--vitest-shuffle`, `--timezone`, and `--retries` it was given in the
+draft, and asks for `--vitest-project`
 when two Vitest projects run the test's file. `testkit bug file` reruns with
 them, and the issue's reproduce command carries them. With
 `TESTKIT_REQUIRE_COMPLETE_SUITE=1`, it reruns every file the selection lists,
@@ -182,6 +191,81 @@ Then run the batch again with a token that may label issues. The batch creates
 at most one issue a second. When the tracker limits the rate, it stops and
 says when to run it again.
 
+### Find the commit that broke it
+
+`testkit bug file` names the commit that introduced a bug from `git blame` of
+the fault lines, which only says who last touched them. `testkit bug bisect`
+runs the regression test at older commits and finds where it starts to fail:
+
+```bash
+npx testkit bug bisect <id>
+```
+
+- It runs the test at the checked-out commit first, which must fail as the
+  draft records. It then runs it at the commit that `git blame` names and at
+  that commit's parent. When the test fails at the one and passes at the
+  other, two runs confirm the blame.
+- Otherwise it looks further back along the first-parent line, a doubling
+  number of commits at a time, until the test passes, and then halves the
+  range between the last passing and the first failing commit.
+- Each commit runs in a temporary directory that holds that commit's tracked
+  files, its dependencies, and the regression test and its setup module from
+  your checkout, so the test can run at commits that came before it. It also
+  holds the ignored `.env` files and package manager settings (`.npmrc`,
+  `.yarnrc`, `.yarnrc.yml`, `.pnpmfile.cjs`, `bunfig.toml`) from anywhere in
+  your checkout, except where the commit tracks its own. Your installed
+  `node_modules` isn't used, not even at the checked-out commit, since install
+  scripts and local packages can make it differ from a fresh install. Testkit
+  installs each lockfile the commit has in the product directory or any
+  directory above it, root first, with `npm ci`, pnpm or bun's
+  `install --frozen-lockfile`, or `yarn install` with `--immutable` or
+  `--frozen-lockfile`, so every install runs that commit's own scripts with
+  those files in place. A directory counts as covered by a lockfile above it
+  when that directory's `workspaces` include it, with no `!` pattern excluding
+  it, or when it is an importer in that directory's `pnpm-lock.yaml`. Any
+  other directory without a lockfile has to declare nothing but test tooling,
+  or the commit is skipped, because Testkit can't install the packages it had.
+  From your checkout, and from any `node_modules` above it, Testkit then links
+  only the test tooling the commit doesn't have, Testkit, Vitest and
+  Playwright, such as Testkit itself before the commit that added it. Every
+  other package comes from the commit's own install, and a commit whose
+  install fails is skipped. The directory starts the services and databases
+  that commit's config declares. Build output that the commit's config doesn't
+  build isn't there.
+- A commit counts as failing when the recorded test fails in a project where
+  it fails at the checked-out commit, and as passing when the engine's report
+  shows that test, in its own file, passed in each of those projects. A
+  project where the test skips itself or is expected to fail, such as with
+  Playwright's `test.fail()`, doesn't count either way, and Vitest's `fails`
+  never counts as passing. When none of the projects that ran at the
+  checked-out commit run at an older one, as after a rename, every project
+  there counts. Other tests in its file can fail either way. A commit is
+  skipped when the config doesn't load, the recorded test doesn't run or pass,
+  a failing test shares its title, or something fails outside a named test in
+  its file.
+- It runs the test at no more than 20 commits besides the checked-out one.
+  `--max-probes` changes that. When it skips a commit while looking back, it
+  tries the commit before that one. When that is skipped too, it takes turns
+  jumping further back, twice as far each time, and searching between the last
+  failing commit and the first skipped one.
+- Each commit runs with the services its config starts, even when
+  `testkit bug file` runs the test against [supplied values](#run-against-a-supplied-environment).
+
+When the test passes at a commit and fails at its child, the child is the
+commit that broke it. Otherwise the result is a range, and Testkit says why:
+it skipped the commits between, or it ran out of probes. Either way it saves
+the two commits, whether they are parent and child, and the commits it ran the
+test at in the draft, and keeps each run's results under
+`.testkit/bisect/<id>/`. A bisect that saves no result keeps its runs under
+`.testkit/bisect/<id>.attempt/` and leaves the saved result and its runs as
+they were. `testkit bug file` adds them to the issue and its
+facts block, next to what `git blame` found, as long as the test, its title,
+and its flags, including the Vitest selectors given to `bug file`, are the
+same as when it was bisected and the failing commit is
+still in the history being filed. When the test fails at every commit Testkit
+could judge, the draft is unchanged and the command exits 1, and Testkit says
+whether it reached the first commit or the end of a shallow clone's history. A filed draft can't be bisected.
+
 ### Name the product's areas
 
 Add `areas` to `testkit.config.ts` so that each issue names the part of the
@@ -248,6 +332,209 @@ claims, how to work, and the bug record format. Give that output to the agent.
 summary for the pull request that adds the tests to
 `.testkit/campaigns/<id>/summary.md` and prints it.
 
+## Prove the tests catch a change
+
+A test can pass without checking anything. `testkit prove` checks whether the
+selected tests notice a change to the code. It makes small changes, called
+mutants, to the product code changed since a ref. It runs the tests against
+each mutant, the way `testkit run` does, and reports which ones no test
+noticed.
+
+```bash
+npx testkit prove --since origin/main
+npx testkit prove --since origin/main --file src/refunds/refunds.test.ts --json
+```
+
+Testkit starts the product's services for every run, so a mutant counts as
+caught when an HTTP test of a service fails, or a `ui` test fails on code the
+service sends to the browser. Each mutant changes one place: a comparison
+(`>` to `>=`, `===` to `!==`), `&&` to `||`, `+` to `-`, `*` to `/`, a
+`true` or `false`, or a `!` it removes. Testkit mutates the changed lines of
+JavaScript and TypeScript files in the product directory. It leaves test
+files, setup modules, config files, and files under `test`, `tests`, `e2e`,
+`fixtures`, and `__mocks__` alone, and mutates at most 25 places. Use
+`--max-mutants` to change that.
+
+Testkit works in a temporary copy of the checkout. The copy holds uncommitted,
+untracked, and ignored files, and links to the installed `node_modules`.
+Testkit removes the copy when it finishes, or when Ctrl-C stops it. Your
+source files never change, though tools that cache inside `node_modules`, such
+as Vite, write their caches through the links. The selected tests must pass in
+the copy before any mutant runs.
+
+Each mutated place also records when a process runs it, whether that is a
+test process, a service, or a browser page. The unmutated run records which
+places any process ran, and only those mutants run. That record gives each
+mutant one of these results:
+
+| Result        | Meaning                                                                                                                          |
+| ------------- | -------------------------------------------------------------------------------------------------------------------------------- |
+| `killed`      | A test failed, and the mutated code ran. The result names the test.                                                              |
+| `survived`    | The mutated code ran, and every test passed. No test checks an input that tells the two versions apart, or they behave the same. |
+| `not-reached` | No process ran the code in the unmutated run. No test covers it, or a service runs it from a build that doesn't include it.      |
+| `errored`     | The run failed outside its tests, or it ran other code than the mutant, such as a cached build.                                  |
+
+A service's build reruns for a mutant only when a file in the build's `inputs`
+changed. When a run doesn't run the mutated code, or runs the code of the
+unmutated run or of an earlier mutant, a build reused its output. The mutant
+ends as `errored`, and the reason names the file to add to `inputs`.
+
+The mutation score is the share of killed mutants among killed, survived, and
+not-reached ones. `--fail-under 80` exits 1 when the score is below 80%, or
+when no mutant could be judged. Testkit writes `prove.json` and `prove.md`
+under `.testkit/prove/<id>/`, and appends `prove.md` to `$GITHUB_STEP_SUMMARY`
+when the variable is set. In GitHub Actions, check out with `fetch-depth: 0`
+and pass `--since origin/main`, because `actions/checkout` fetches one commit
+and sets no `origin/HEAD`.
+
+A coding agent can write mutants of its own, for example a refund that skips
+its idempotency check, and pass them with `--mutants`:
+
+```json
+[
+  {
+    "file": "src/refunds.ts",
+    "line": 42,
+    "find": "await assertNotRefunded(order.id);",
+    "replace": "",
+    "note": "refund twice"
+  }
+]
+```
+
+`file` is relative to the product directory, and `find` must be a whole
+expression or statement on that line. An empty `replace` removes a statement.
+Testkit refuses an expression that is called, assigned, deleted, or optionally
+chained, such as `order.refund` in `order.refund()`; replace the whole call.
+`--mutants` runs those mutants instead of mutating the changed lines.
+
+- A run that `bugs.knownFailures: "allow"` would pass still counts its
+  failures, because a mutant's failing test is the result being measured.
+- Testkit rebuilds only what `testkit.config.ts` declares. A service that runs
+  output you built yourself, such as a copied `dist/`, runs the code from
+  before the mutation. Its mutants read as `not-reached`, unless a test also
+  runs the source in its own process, when they can survive.
+- A flaky test can fail during a mutant's run and count as a kill. Run
+  [`testkit shake`](#shake-out-flaky-tests) on a test that fails now and then
+  before you trust its kills.
+- Changed files outside the product directory, such as a workspace package
+  elsewhere in a monorepo, aren't mutated.
+- A browser page reports that it ran a mutant with a request to a listener on
+  `127.0.0.1`. A Content Security Policy that blocks the request makes the
+  mutant read as `not-reached`.
+- Each run gets a file time limit of three times the unmutated run, and at
+  least 30 seconds, so a mutant that loops forever ends as a failure.
+
+## Shake out flaky tests
+
+A test that passes once can still fail in CI: when another test runs first,
+beside a copy of itself, in another time zone, or on a busy machine. `testkit
+shake` reruns test files under each of those conditions and reports which one
+fails them.
+
+```bash
+npx testkit shake --file src/orders/orders.test.ts
+npx testkit shake --since origin/main --runs 10
+```
+
+`--file` names the test files, and `--since` takes the test files added or
+changed since the merge base with a ref, so a job can shake new tests before
+they merge. Each condition runs `--runs` times, 5 by default:
+
+| Condition       | What changes                                                                                                   |
+| --------------- | -------------------------------------------------------------------------------------------------------------- |
+| `control`       | Nothing. A failure here means the test fails on its own.                                                       |
+| `shuffle`       | Plain Vitest files, and the tests in them, run in a random order. Each run records its seed.                   |
+| `copies`        | Each file runs at the same time as copies of itself, 3 by default (`--copies`), on the same services and data. |
+| `timezone-east` | Tests and the services they use run with `TZ=Pacific/Kiritimati`, 14 hours ahead of UTC.                       |
+| `timezone-west` | Tests and services run with `TZ=Pacific/Pago_Pago`, 11 hours behind UTC.                                       |
+| `load`          | A busy loop runs on every CPU core during the run.                                                             |
+
+The two time zones are always on different calendar dates, so whatever the
+time, one of them is on a different date from UTC. `--only shuffle copies`
+runs some conditions. Testkit runs the conditions in turn, one run of each at
+a time, so a change on the machine during the shake doesn't fall on one
+condition.
+
+Testkit runs every condition in a temporary copy of the checkout, as
+[`testkit prove`](#prove-the-tests-catch-a-change) does, so the copies and the
+runs stay out of your checkout and its `.testkit/results`. It turns off the
+retries that the Vitest or Playwright config sets, so a test that passes only
+on a second try fails here. A `retry` option on a single test still applies.
+For each test that failed under a condition, it compares how often the test
+failed there with how often the same test failed in the control runs, using
+Fisher's exact test, and gives the condition a verdict:
+
+| Verdict           | Meaning                                                                                                |
+| ----------------- | ------------------------------------------------------------------------------------------------------ |
+| `steady`          | The control never failed.                                                                              |
+| `fails`           | Every test that failed in the control failed in every control run that reached it.                     |
+| `flaky`           | The control failed, but some of its failing tests passed in some runs.                                 |
+| `reproduces`      | A test failed in every run that reached it, at least two, and never in the control, with p below 0.05. |
+| `raises failures` | A test failed more often than in the control, with p below 0.05.                                       |
+| `inconclusive`    | A test failed more often than in the control, but not often enough to tell from chance.                |
+| `no effect`       | No test failed more often than in the control.                                                         |
+| `errored`         | Every run failed outside its tests, so there is nothing to compare.                                    |
+
+A run that fails outside its tests counts as errored and leaves the comparison.
+A condition is blamed only for the tests that fail more often under it than in
+the control. Testkit multiplies each test's p by K, the smallest number such
+that at most K tests failed often enough, under the condition and in the
+control together, to reach p at or below 0.05/K (Tarone's correction). One
+test failing by chance among many doesn't count, and a test that fails in every
+run of both doesn't make the others harder to judge. The correction is within
+a condition: each condition is compared with the control on its own. With 5 runs, a test must fail in at least
+4 of them, and more when other tests could reach significance too, to get past
+`inconclusive`, so pass `--runs 10` or more for a test that fails now and
+then.
+
+For each condition that failed, the report names the tests and how often they
+failed there and in the control, each with a 95% Wilson interval for the
+failure rate. It gives a command that reproduces the failure, and each
+`testkit run` command ends with `--retries 0`, as the shake's runs do:
+
+- A time zone replays with `testkit run --timezone <zone>`, and a shuffle with
+  `testkit run --vitest-shuffle <seed>` for up to three seeds whose runs failed
+  a test the shuffle is blamed for.
+- A control that `fails` replays with `testkit run`.
+- Copies, load, and a `flaky` control replay with
+  `testkit shake --only <condition>`.
+
+After the shake, Testkit runs the first `testkit run` command once more and
+says whether a test the condition is blamed for failed again, the failure
+didn't come back, or the run failed outside its tests. A condition that fails
+only some of the time can pass that run, and when the blamed test also fails in
+some control runs, a failure that comes back shows less.
+
+Testkit writes `shake.json` and `shake.md` under `.testkit/shake/<id>/`, and
+keeps the run bundle of every run that didn't pass under `runs/` there, named
+by condition and run, such as `copies-3` or `timezone-east-replay`.
+`testkit shake` exits 1 when any run failed. When `--since` finds no changed
+test file, it shakes nothing and exits 0.
+
+`testkit bug record` takes `--timezone`, `--vitest-shuffle`, and `--retries`
+too. The draft keeps them, `testkit bug file` reruns the test with them, and
+the issue's reproduce command carries them, so a bug that shows only in another
+time zone can be filed with a command that fails every time.
+
+- `--timezone` sets `TZ` for the test processes and the services Testkit
+  starts, and the time zone of a `ui` test's browser, unless the Playwright
+  config sets `use.timezoneId`. A `TZ` that a service's `env` sets in
+  `testkit.config.ts` wins over it, and a database server keeps its own time
+  zone.
+- A test that reads the current time fails in one of the two zones only at
+  some hours of the day, so its reproduce command fails only at those hours.
+- Each copy runs as a task of its own, so the copies overlap, and the
+  original files run at the same time as their copies too. A test that
+  creates a fixed row, counts a whole table, or holds a shared resource fails
+  beside its copies. Give each test its own data, as
+  [Give each test its own data](#give-each-test-its-own-data) describes. A
+  copy also runs the test more often, so a test that fails at random fails
+  more often under `copies` too.
+- A seed gives the same order of files and tests every time, because Testkit
+  sorts the files before Vitest shuffles them. `shuffle` changes the order
+  only of plain Vitest files, not of `ui` files.
+
 ## Prepare and publish a report bundle
 
 Prepare a new directory from your own template, recorded bug IDs, and selected
@@ -419,6 +706,13 @@ npx @gfdelarue/testkit --type ui -s ui:checkout
 npx @gfdelarue/testkit --type ui --file __testkit__/checkout.ui.testkit.ts
 npx @gfdelarue/testkit run --file src/sum.test.ts
 
+# Shuffle plain test files and their tests with a seed, or run in a time zone
+npx @gfdelarue/testkit run --file src/cart.test.ts --vitest-shuffle 3
+npx @gfdelarue/testkit run --file src/invoice.test.ts --timezone Pacific/Kiritimati
+
+# Override the retries the Vitest and Playwright configs set, 0 to 99
+npx @gfdelarue/testkit run --file src/cart.test.ts --retries 0
+
 # Temporarily ignore repo-declared skip rules
 npx @gfdelarue/testkit --ignore-skip-rules --file src/billing.test.ts
 

package/dist/app/bugs/batch.js

@@ -1 +1 @@
-import I from"node:path";import{setTimeout as S}from"node:timers/promises";import{loadTestkitConfig as F}from"../../config/loader/index.js";import{stableJson as v}from"../../evidence/storage.js";import{listCommitFiles as R}from"../../repo/tree-listing.js";import{TrackerError as E}from"../../tracker/http.js";import{namesSameTest as A}from"./draft.js";import{readFactsBlock as C,renderFactsBlock as y}from"./facts.js";import{bugFiling as D,bugRerun as $,readOpenIssues as N,readyBugFiling as q,recordFiledBug as k,renderReadableBugIssue as x,requireCleanPushedCheckout as L,requireCommittedDraftFiles as O,requireFileableDraft as _,requireNoOpenIssue as B,requireRecordedFailure as j,requireReplayableSuppliedInput as M,resolveBugTracker as J,shortCommit as K}from"./file.js";import{bugFacts as P,bugLabels as V,TESTKIT_LABEL as G}from"./issue.js";import{readKnownTests as U}from"./known-failures.js";import{bugDraftFile as W,displayPath as Y,readStoredDrafts as z,resolveBugCheckout as H}from"./store.js";import{suppliedReproduceArgs as Q}from"./supplied-reproduce.js";const X=1e3;export async function prepareBugBatch(e){const d=Q(e.supplied??{}),t=await H(e),{candidates:n,order:i,settled:a}=Z(t,e.ids);if(n.length===0)return{checkout:t,order:i,ready:[],settled:a};e.supplied!==void 0&&M(e.supplied,t,e.env);const o=await R(t.repoRoot,t.headCommit);await L(t,e.env);const{config:s}=await F(t.productDir),r=await J(t,s,e.env),l=await r.listOpenIssues(G),u=U(l).tests,p=[],f=[...a];for(const{draft:c,file:m}of n)try{_(c),O(c,o,t);const g=await D(t,s,o,c),b=q({checkout:t,draft:c,draftFile:m,filing:g,reproduceArgs:d,tracker:r}),w=ee(l,b);if(w!==null){f.push({id:c.id,issue:k(b,w,new Date).issue,status:"already-filed"});continue}B(u,c.test);const h=p.find(T=>A(T.draft,c));if(h!==void 0)throw new Error(`${h.draft.id} in this batch files the same test. Testkit keeps one open bug for each test.`);p.push(b)}catch(g){f.push(rejectedBugBatchDraft(c.id,g))}return{checkout:t,order:i,ready:p,settled:f}}export async function planBugBatchRuns(e,d,t={},n=process.env){const i=[],a=new Map;for(const o of e.ready)try{const s=await $(o,d,t,n),{vitest:r}=s.ready.draft,l=v(r??{}),u=a.get(l)??{drafts:[],files:new Set,vitest:r};u.drafts.push(s.ready);for(const p of s.files)u.files.add(p);a.set(l,u)}catch(s){i.push(rejectedBugBatchDraft(o.draft.id,s))}return{rejected:i,runs:[...a.values()].map(o=>({drafts:o.drafts,files:[...o.files].sort(),vitest:o.vitest}))}}export async function fileBugBatch(e,d={}){const t=e.at(0)?.ready.tracker;if(t===void 0)return[];let n;try{n=[...await N(t)]}catch(r){if(!(r instanceof Error))throw r;const l=`Testkit could not read the open issues again before filing: ${r.message}`;return e.map(({ready:u})=>({id:u.draft.id,reason:l,status:"not-filed"}))}const i=new Set,a=[];let o=null,s=Number.NEGATIVE_INFINITY;for(const{ready:r,run:l}of e){const{id:u}=r.draft;if(o!==null){a.push({id:u,reason:o,status:"not-filed"});continue}let p;try{const f=j(r.draft,l,K(r.checkout.headCommit),r.reproduceArgs);B(n,f),p=x(r.draft,r.filing,r.reproduceArgs)}catch(f){a.push(rejectedBugBatchDraft(u,f));continue}try{const f=V(r.draft.severity,r.filing.area).filter(({name:h})=>!i.has(h));await t.ensureLabels(f);for(const{name:h}of f)i.add(h);await S(Math.max(0,s+X-performance.now()));const c=await t.createIssue(p);s=performance.now();const m=k(r,c,d.at??new Date);n.push({file:r.draft.test.file,issue:m.issue.number,title:r.draft.test.title,url:m.issue.url});const{area:g,introducedBy:b,path:w}=m;a.push({area:g,id:u,introducedBy:b,issue:m.issue,path:w,status:"filed"})}catch(f){if(!(f instanceof Error))throw f;const c=te(f);a.push({id:u,reason:`${f.message}${c}`,status:"not-filed"}),o=`Testkit stopped filing when ${u} failed.${c}`}}return a}export function rejectedBugBatchDraft(e,d){if(!(d instanceof Error))throw d;return{id:e,reason:d.message,status:"rejected"}}export function bugBatchReport(e,d){const t=[...d].sort((n,i)=>e.order.indexOf(n.id)-e.order.indexOf(i.id));return{drafts:t,ok:t.every(({status:n})=>n==="filed"||n==="already-filed")}}function Z(e,d){const t=z(e.productDir),n=new Map(t.unreadable.map(s=>[s.file,s.problem])),i=d.length>0?[...new Set(d)]:[...t.drafts.filter(({draft:s})=>s.filed===void 0).map(({draft:s})=>s.id),...t.unreadable.map(({file:s})=>I.basename(s,".json"))].sort(),a=[],o=[];for(const s of i){const r=t.drafts.find(({draft:p})=>p.id===s),l=W(e.productDir,s),u=n.get(l);r?.draft.filed!==void 0?o.push({id:s,issue:{number:r.draft.filed.number,url:r.draft.filed.url},status:"already-filed"}):r!==void 0?a.push(r):o.push({id:s,reason:u===void 0?`No bug draft has the id ${JSON.stringify(s)}.`:`Testkit cannot read ${Y(e,l)}: ${u}. Record the bug again with \`testkit bug record\`.`,status:"rejected"})}return{candidates:a,order:i,settled:o}}function ee(e,{draft:d,filing:t}){const n=y(P(d,t));for(const i of e){const a=C(i.body);if(a.status==="ok"&&y(a.facts)===n)return{number:i.number,url:i.url}}return null}function te(e){return!(e instanceof E)||e.kind!=="rate-limited"?"":e.retryAfterSeconds===null?" The tracker is limiting requests. Run the batch again later; it skips the drafts it filed.":` The tracker is limiting requests. Run the batch again in ${String(e.retryAfterSeconds)} seconds; it skips the drafts it filed.`}
+import I from"node:path";import{setTimeout as S}from"node:timers/promises";import{loadTestkitConfig as v}from"../../config/loader/index.js";import{stableJson as F}from"../../evidence/storage.js";import{listCommitFiles as R}from"../../repo/tree-listing.js";import{TrackerError as E}from"../../tracker/http.js";import{namesSameTest as A}from"./draft.js";import{readFactsBlock as C,renderFactsBlock as y}from"./facts.js";import{bugFiling as D,bugRerun as $,readOpenIssues as N,readyBugFiling as q,recordFiledBug as k,renderReadableBugIssue as x,requireCleanPushedCheckout as L,requireCommittedDraftFiles as O,requireFileableDraft as _,requireNoOpenIssue as B,requireRecordedFailure as j,requireReplayableSuppliedInput as z,resolveBugTracker as M,shortCommit as J}from"./file.js";import{bugFacts as K,bugLabels as P,TESTKIT_LABEL as V}from"./issue.js";import{readKnownTests as G}from"./known-failures.js";import{bugDraftFile as U,displayPath as W,readStoredDrafts as Y,resolveBugCheckout as H}from"./store.js";import{suppliedReproduceArgs as Q}from"./supplied-reproduce.js";const X=1e3;export async function prepareBugBatch(e){const u=Q(e.supplied??{}),t=await H(e),{candidates:o,order:d,settled:a}=Z(t,e.ids);if(o.length===0)return{checkout:t,order:d,ready:[],settled:a};e.supplied!==void 0&&z(e.supplied,t,e.env);const i=await R(t.repoRoot,t.headCommit);await L(t,e.env);const{config:s}=await v(t.productDir),r=await M(t,s,e.env),l=await r.listOpenIssues(V),f=G(l).tests,p=[],n=[...a];for(const{draft:c,file:m}of o)try{_(c),O(c,i,t);const g=await D(t,s,i,c),b=q({checkout:t,draft:c,draftFile:m,filing:g,reproduceArgs:u,tracker:r}),w=ee(l,b);if(w!==null){n.push({id:c.id,issue:k(b,w,new Date).issue,status:"already-filed"});continue}B(f,c.test);const h=p.find(T=>A(T.draft,c));if(h!==void 0)throw new Error(`${h.draft.id} in this batch files the same test. Testkit keeps one open bug for each test.`);p.push(b)}catch(g){n.push(rejectedBugBatchDraft(c.id,g))}return{checkout:t,order:d,ready:p,settled:n}}export async function planBugBatchRuns(e,u,t={},o=process.env){const d=[],a=new Map;for(const i of e.ready)try{const s=await $(i,u,t,o),{retries:r,timezone:l,vitest:f}=s.ready.draft,p=F({retries:r??null,timezone:l??null,vitest:f??{}}),n=a.get(p)??{drafts:[],files:new Set,retries:r,timezone:l,vitest:f};n.drafts.push(s.ready);for(const c of s.files)n.files.add(c);a.set(p,n)}catch(s){d.push(rejectedBugBatchDraft(i.draft.id,s))}return{rejected:d,runs:[...a.values()].map(i=>({drafts:i.drafts,files:[...i.files].sort(),retries:i.retries,timezone:i.timezone,vitest:i.vitest}))}}export async function fileBugBatch(e,u={}){const t=e.at(0)?.ready.tracker;if(t===void 0)return[];let o;try{o=[...await N(t)]}catch(r){if(!(r instanceof Error))throw r;const l=`Testkit could not read the open issues again before filing: ${r.message}`;return e.map(({ready:f})=>({id:f.draft.id,reason:l,status:"not-filed"}))}const d=new Set,a=[];let i=null,s=Number.NEGATIVE_INFINITY;for(const{ready:r,run:l}of e){const{id:f}=r.draft;if(i!==null){a.push({id:f,reason:i,status:"not-filed"});continue}let p;try{const n=j(r.draft,l,J(r.checkout.headCommit),r.reproduceArgs);B(o,n),p=x(r.draft,r.filing,r.reproduceArgs)}catch(n){a.push(rejectedBugBatchDraft(f,n));continue}try{const n=P(r.draft.severity,r.filing.area).filter(({name:h})=>!d.has(h));await t.ensureLabels(n);for(const{name:h}of n)d.add(h);await S(Math.max(0,s+X-performance.now()));const c=await t.createIssue(p);s=performance.now();const m=k(r,c,u.at??new Date);o.push({file:r.draft.test.file,issue:m.issue.number,title:r.draft.test.title,url:m.issue.url});const{area:g,introducedBy:b,path:w}=m;a.push({area:g,id:f,introducedBy:b,issue:m.issue,path:w,status:"filed"})}catch(n){if(!(n instanceof Error))throw n;const c=te(n);a.push({id:f,reason:`${n.message}${c}`,status:"not-filed"}),i=`Testkit stopped filing when ${f} failed.${c}`}}return a}export function rejectedBugBatchDraft(e,u){if(!(u instanceof Error))throw u;return{id:e,reason:u.message,status:"rejected"}}export function bugBatchReport(e,u){const t=[...u].sort((o,d)=>e.order.indexOf(o.id)-e.order.indexOf(d.id));return{drafts:t,ok:t.every(({status:o})=>o==="filed"||o==="already-filed")}}function Z(e,u){const t=Y(e.productDir),o=new Map(t.unreadable.map(s=>[s.file,s.problem])),d=u.length>0?[...new Set(u)]:[...t.drafts.filter(({draft:s})=>s.filed===void 0).map(({draft:s})=>s.id),...t.unreadable.map(({file:s})=>I.basename(s,".json"))].sort(),a=[],i=[];for(const s of d){const r=t.drafts.find(({draft:p})=>p.id===s),l=U(e.productDir,s),f=o.get(l);r?.draft.filed!==void 0?i.push({id:s,issue:{number:r.draft.filed.number,url:r.draft.filed.url},status:"already-filed"}):r!==void 0?a.push(r):i.push({id:s,reason:f===void 0?`No bug draft has the id ${JSON.stringify(s)}.`:`Testkit cannot read ${W(e,l)}: ${f}. Record the bug again with \`testkit bug record\`.`,status:"rejected"})}return{candidates:a,order:d,settled:i}}function ee(e,{draft:u,filing:t}){const o=y(K(u,t));for(const d of e){const a=C(d.body);if(a.status==="ok"&&y(a.facts)===o)return{number:d.number,url:d.url}}return null}function te(e){return!(e instanceof E)||e.kind!=="rate-limited"?"":e.retryAfterSeconds===null?" The tracker is limiting requests. Run the batch again later; it skips the drafts it filed.":` The tracker is limiting requests. Run the batch again in ${String(e.retryAfterSeconds)} seconds; it skips the drafts it filed.`}

package/dist/app/bugs/bisect-dependencies.js

@@ -0,0 +1 @@
+import l from"node:path";import{Minimatch as g}from"minimatch";import{isTestTooling as x}from"../sandbox/index.js";const d={"bun.lock":()=>({args:["install","--frozen-lockfile"],command:"bun"}),"bun.lockb":()=>({args:["install","--frozen-lockfile"],command:"bun"}),"npm-shrinkwrap.json":()=>({args:["ci","--no-audit","--no-fund"],command:"npm"}),"package-lock.json":()=>({args:["ci","--no-audit","--no-fund"],command:"npm"}),"pnpm-lock.yaml":()=>({args:["install","--frozen-lockfile"],command:"pnpm"}),"yarn.lock":o=>({args:["install",o?"--immutable":"--frozen-lockfile"],command:"yarn"})},h=["dependencies","devDependencies","optionalDependencies","peerDependencies"],k="(workspaces)",m="pnpm-lock.yaml";export function dependencyPaths(o){return p(o).flatMap(s=>[...Object.keys(d),".yarnrc.yml"].map(n=>l.posix.join(s,n)))}export function manifestPaths(o){return p(o).flatMap(s=>["package.json",m].map(n=>l.posix.join(s,n)))}export function readCommitDependencies(o,s){const n=new Map;let c=!1;for(const e of o){const i=/^\d+ blob ([0-9a-f]+)\t(.+)$/u.exec(e);i!==null&&(l.posix.basename(i[2])===".yarnrc.yml"?c=!0:n.set(i[2],i[1]))}const r=new Map,t=new Map;for(const{path:e,text:i}of s){if(l.posix.basename(e)===m){t.set($(e),j(i));continue}let f;try{const a=JSON.parse(i.replace(/^\uFEFF/u,""));f=typeof a=="object"&&a!==null?a:{}}catch{r.set(`${e}\0(unreadable)`,i);continue}f.workspaces!==void 0&&r.set(`${e}\0${k}`,JSON.stringify(f.workspaces));for(const a of h){const u=f[a];if(!(typeof u!="object"||u===null))for(const[y,b]of Object.entries(u))r.set(`${e}\0${y}`,String(b))}}return{berry:c,declared:r,importers:t,lockfiles:n}}export function dependencyPlan(o,s){const n=t=>Object.keys(d).map(e=>l.posix.join(t,e)).find(e=>o.lockfiles.has(e)),c=t=>p(t).slice(1).some(e=>{const i=n(e);if(i===void 0)return!1;const f=l.posix.relative(e,t);if(l.posix.basename(i)===m)return(o.importers.get(e)??[]).includes(f);const a=o.declared.get(`${l.posix.join(e,"package.json")}\0${k}`);return a!==void 0&&O(a,f)}),r=[];for(const t of[...p(s)].reverse()){const e=n(t);if(e!==void 0){r.push({...d[l.posix.basename(e)](o.berry),directory:t});continue}if(c(t))continue;const i=`${l.posix.join(t,"package.json")}\0`;if(![...o.declared.keys()].filter(a=>a.startsWith(i)).map(a=>a.slice(i.length)).every(x))return{directory:t,kind:"unknown"}}return r.length===0?{kind:"tooling"}:{installs:r,kind:"install"}}function j(o){const s=o.split(/\r?\n/u),n=s.indexOf("importers:");if(n===-1)return[];const c=[];for(const r of s.slice(n+1)){if(r!==""&&!r.startsWith(" "))break;const t=/^ {2}(['"]?)([^'"\s].*?)\1:(?: |$)/u.exec(r);t!==null&&c.push(t[2])}return c}function O(o,s){const n=JSON.parse(o),c=typeof n=="object"&&n!==null&&!Array.isArray(n)?n.packages:n;let r=!1,t=!1;for(const e of Array.isArray(c)?c:[]){if(typeof e!="string")continue;const i=e.startsWith("!"),f=(i?e.slice(1):e).replace(/^\.\//u,"").replace(/\/$/u,"");new g(f).match(s)&&(i?t=!0:r=!0)}return r&&!t}function $(o){const s=l.posix.dirname(o);return s==="."?"":s}function p(o){const s=[];for(let n=o;;n=l.posix.dirname(n)){const c=n==="."?"":n;if(s.push(c),c==="")return s}}

package/dist/app/bugs/bisect-identity.js

@@ -0,0 +1 @@
+import{createHash as c}from"node:crypto";import r from"node:fs";import n from"node:path";export function bisectOverlay(i,e){const t=e.replace(/\.[^./]+$/u,""),s=n.posix.dirname(e),o=n.posix.basename(t),p=[`${t}.setup.mjs`,`${t}.setup.js`,`${s}/__setup__/${o}.mjs`,`${s}/__setup__/${o}.js`];return[e,...p.filter(u=>r.existsSync(n.join(i,u)))]}export function bisectIdentity(i,e){const t=c("sha256");for(const s of bisectOverlay(i,e.test.file))t.update(`${s}\0`).update(r.readFileSync(n.join(i,s))).update("\0");return t.update(JSON.stringify({retries:e.retries??null,timezone:e.timezone??null,title:e.test.title,vitest:e.vitest??{}})),t.digest("hex")}

package/dist/app/bugs/bisect-report.js

@@ -0,0 +1 @@
+import p from"node:path";import{isUnknownRecord as i}from"../../kernel/unknown-values.js";export function recordedEntries(t,s,a,n){if(!i(t)||!i(t.result))return[];const{result:o}=t,r=p.resolve(s,a),c=e=>typeof e=="string"&&p.relative(r,p.resolve(s,e))==="",d=e=>typeof e=="string"?e:"",f=(Array.isArray(o.cases)?o.cases:[]).filter(i),u=(Array.isArray(o.specs)?o.specs:[]).filter(i);return[...f.filter(e=>e.title===n&&c(e.file)).map(e=>({project:d(e.project),state:e.state==="skipped"||e.state==="pending"?"skipped":e.state==="passed"&&e.fails!==!0?"passed":"failed"})),...u.filter(e=>j(e)===n&&c(e.sourceFile)).map(e=>({project:d(e.projectName),state:e.status==="skipped"||e.expectedStatus==="skipped"||e.status==="expected"&&e.expectedStatus==="failed"?"skipped":(e.status==="expected"||e.status==="flaky")&&e.expectedStatus==="passed"?"passed":"failed"}))]}export const UNKNOWN_HEAD={failed:new Set,ran:new Set};export function headProjects(t){return{failed:new Set(t.filter(s=>s.state==="failed").map(s=>s.project)),ran:new Set(t.map(s=>s.project))}}export function passedWhereItFailed(t,s){const a=l(t,s),n=t.filter(r=>r.state!=="skipped"&&(a.size===0||a.has(r.project))),o=new Set(n.filter(r=>r.state==="passed").map(r=>r.project));return n.length>0&&n.every(r=>r.state==="passed")&&[...a].every(r=>o.has(r))}export function failedWhereItFails(t,s){const a=l(t,s);return a.size===0||t.some(n=>n.state==="failed"&&a.has(n.project))}function l(t,s){return t.some(a=>s.ran.has(a.project))?s.failed:new Set}function j(t){return[...Array.isArray(t.parents)?t.parents.slice(1):[],t.title].filter(Boolean).join(" ")}

package/dist/app/bugs/bisect-search.js

@@ -0,0 +1 @@
+export async function searchFirstParentLine(n,k,x,L){const M=[],l=new Map,e={bad:0,good:null},g=()=>M.length<L,f=async t=>{const a=l.get(t);if(a!==void 0)return a;const o=await x(n[t]);return M.push({commit:n[t],...o}),l.set(t,o.verdict),o.verdict==="bad"&&(e.bad=Math.max(e.bad,t)),o.verdict==="good"&&(e.good=e.good===null?t:Math.min(e.good,t)),o.verdict},b=k===null?-1:n.indexOf(k);b>0&&g()&&await f(b)==="bad"&&b+1<n.length&&g()&&await f(b+1);let w=1,h=e.bad+1,i=n.length,p=-1,v=!1;const F=t=>t[Math.floor((t.length-1)/2)];for(;e.good===null&&g();){const t=[];for(let d=e.bad+1;d<n.length;d+=1)l.has(d)||t.push(d);if(t.length===0)break;const a=t.filter(d=>d<i),o=t.find(d=>d>=h),s=i<n.length&&a.length>0,r=t.includes(p)?"neighbour":o!==void 0&&!(s&&!v)?"outward":s?"narrow":"middle";v=r==="narrow";const c=r==="neighbour"?p:r==="outward"?o??0:F(r==="narrow"?a:t);p=-1,await f(c)==="skip"?(r!=="neighbour"&&(i=Math.min(i,c)),r==="outward"&&(p=c+1),h=Math.max(h,c+1),r==="neighbour"&&(w*=2,h=Math.min(c+w,n.length))):(r==="outward"||r==="neighbour")&&(i=n.length,w*=2,h=Math.min(e.bad+w,n.length-1))}for(;e.good!==null&&g();){const t=e.bad,a=e.good,o=[];for(let s=t+1;s<a;s+=1)l.has(s)||o.push(s);if(o.length===0)break;await f(o[Math.floor((o.length-1)/2)])}const{bad:m,good:u}=e,N=(t,a)=>{for(let o=t;o<a;o+=1)if(!l.has(o))return!0;return!1},O=[...l].some(([t,a])=>a==="skip"&&t>m),y=u===m+1?"found":N(m+1,u??n.length)?"budget":u!==null||O?"skipped":"history";return{bad:n[m],exact:y==="found",good:u===null?null:n[u],probes:M,stop:y}}

package/dist/app/bugs/bisect.js

@@ -0,0 +1,2 @@
+import a from"node:fs";import k from"node:path";import{runModel as I}from"../../evidence/index.js";import{errorMessage as F}from"../../kernel/error-serialization.js";import{requireManagedHostCommandSuccess as T}from"../../process/index.js";import{findIntroducingCommit as _}from"../../repo/history.js";import{REPOSITORY_COMMAND_GUARD_ARGS as j,runManagedGitCommand as A}from"../../repo/managed-git.js";import{readRunArtifact as G}from"./run-reading.js";import{useTimeZone as H}from"../run/time-zone.js";import{createProductSandbox as J}from"../sandbox/index.js";import{runSandboxArtifact as L}from"../sandbox/run.js";import{dependencyPaths as W,dependencyPlan as B,manifestPaths as U,readCommitDependencies as Z}from"./bisect-dependencies.js";import{bisectIdentity as q,bisectOverlay as z}from"./bisect-identity.js";import{failedWhereItFails as K,passedWhereItFailed as Y,recordedEntries as Q,headProjects as V,UNKNOWN_HEAD as X}from"./bisect-report.js";import{searchFirstParentLine as tt}from"./bisect-search.js";import{toProductPath as et}from"./draft.js";import{readStoredDrafts as rt,resolveBugCheckout as ot,writeJsonFile as it}from"./store.js";export async function bisectBug(o){const e=await ot(o),n=rt(e.productDir).drafts.find(r=>r.draft.id===o.id);if(n===void 0)throw new Error(`No readable bug draft has the id ${JSON.stringify(o.id)}`);const{draft:t}=n;if(t.filed!==void 0)throw new Error(`${o.id} is filed already, as ${t.filed.url}.`);const c=e.headCommit,d=et(e.productPath,t.test.file),i=z(e.repoRoot,t.test.file),s=k.join(e.productDir,".testkit","bisect",t.id),f=`${s}.attempt`;a.rmSync(f,{force:!0,recursive:!0});let l=null;const w=async r=>Z(await b(e.repoRoot,["ls-tree","-r",r,"--",...W(e.productPath)]),await at(e.repoRoot,r,U(e.productPath))),S=async r=>{const p=H(t.timezone);let y=null,m=null;const D=()=>m?.kind==="install"?{dependencies:"installed"}:m?.kind==="tooling"?{dependencies:"tooling"}:{};try{if(m=B(await w(r),e.productPath),m.kind==="unknown")return{reason:`${m.directory===""?"the repository root":m.directory} declares packages other than test tooling, and this commit has no lockfile to install them from`,verdict:"skip"};y=await J(e.repoRoot,e.productPath,{commit:r,overlay:i,...m.kind==="install"?{installs:m.installs}:{toolingOnly:!0}});const v=await L(y,{file:[d],...t.vitest?{vitest:t.vitest}:{}},t.retries===void 0?{}:{retries:t.retries}),O=k.join(y.productDir,".testkit","results","runs",v.archive.runId);a.existsSync(O)&&a.cpSync(O,k.join(f,r.slice(0,12)),{recursive:!0});const M=st(v,y,d,t.test.title),E=nt(v,e.productPath,t,M,l??X);return l??=V(M),{...E,...D()}}catch(v){return{...D(),reason:F(v),verdict:"skip"}}finally{p(),await y?.dispose()}},h=await S(c);if(o.onProbe?.({commit:c,...h}),h.verdict!=="bad")throw new Error(`${t.test.file} must fail at ${c.slice(0,12)} for Testkit to look for the commit that broke it, but ${h.verdict==="good"?"it passed":`Testkit could not judge it: ${h.reason??""}`}.`);const P=await _(e.repoRoot,c,t.fault.path,t.fault.lines),g=P.status==="found"?P.commit:null,$=await b(e.repoRoot,["rev-list","--first-parent",c]),u=await tt($,g,async r=>{const p=await S(r);return o.onProbe?.({commit:r,...p}),p},o.maxProbes),x=[{commit:c,...h},...u.probes],R=u.good===null?t:{...t,bisect:{bad:u.bad,exact:u.exact,good:u.good,probes:x.map(({commit:r,verdict:p})=>({commit:r,verdict:p})),test:q(e.repoRoot,t)}};if(u.good!==null){const r=`${s}.previous`;a.rmSync(r,{force:!0,recursive:!0}),a.existsSync(s)&&a.renameSync(s,r);try{a.renameSync(f,s),it(n.file,R)}catch(p){throw a.existsSync(s)&&!a.existsSync(f)&&a.renameSync(s,f),a.existsSync(r)&&a.renameSync(r,s),p}a.rmSync(r,{force:!0,recursive:!0})}const C=(await b(e.repoRoot,["rev-parse","--is-shallow-repository"])).at(0)==="true",N=u.stop==="history"&&C?"shallow":u.stop;return{...u,blame:g,blameOnLine:g!==null&&$.includes(g),draft:R,evidenceDir:u.good===null?f:s,probes:x,stop:N}}function nt(o,e,n,t,c){const d=G(o,e);if(d.kind==="run-failed")return{reason:d.reason,verdict:"skip"};const i=d.kind==="tests-failed"?d.failures:[],s=i.find(l=>l.title===null);if(s!==void 0)return{reason:`${s.file} failed outside any named test, for example on an import this commit doesn't have`,verdict:"skip"};if(!i.every(l=>l.file===n.test.file))return{reason:`it failed as ${i.map(w=>`${w.file} \u203A ${w.title??""}`).join(", ")}, outside the tests of ${n.test.file}`,verdict:"skip"};const f=i.filter(l=>l.title===n.test.title);return f.some(l=>l.ambiguous===!0)?{reason:"more than one test has the recorded title",verdict:"skip"}:f.length>0?K(t,c)?{verdict:"bad"}:{reason:"the recorded test failed only in projects where it passes now",verdict:"skip"}:Y(t,c)?{verdict:"good"}:{reason:"the run never passed the recorded test where it fails now",verdict:"skip"}}function st(o,e,n,t){return(I(o).groups.flatMap(i=>i.suites.flatMap(s=>s.files)).find(i=>i.path===n)?.artifacts??[]).filter(i=>i.kind==="engine.testkit-vitest-report"||i.kind==="engine.testkit-playwright-report").flatMap(i=>Q(JSON.parse(a.readFileSync(k.join(e.productDir,i.path),"utf8")),e.productDir,n,t))}async function at(o,e,n){const t=[];for(const c of n){const d=await A(o,[...j,"show",`${e}:${c}`]);d.exitCode===0&&t.push({path:c,text:d.stdout})}return t}async function b(o,e){const n=await A(o,[...j,...e]);return T(`git ${e[0]}`,n).stdout.split(`
+`).filter(t=>t!=="")}

package/dist/app/bugs/draft.js

@@ -1,4 +1,4 @@
-import G from"node:crypto";import A from"node:path";import{isPublicTestType as j}from"../../domain/identifiers.js";import{isUnknownRecord as g}from"../../kernel/unknown-values.js";import{BUG_CATCHERS as O,BUG_CAUSES as P,BUG_CONFIDENCES as C,BUG_ESCAPES as U,BUG_IMPACTS as F,BUG_SEVERITIES as R,categoryIds as _,isOneOf as E}from"./categories.js";import{BUG_FACTS_TEST_TYPES as z,decodeFaultLines as D,isCommitId as $}from"./facts.js";import{redactBugText as B}from"../../tracker/redaction.js";import{isTimestamp as w}from"../campaigns/campaign.js";export const BUG_DRAFT_SCHEMA_VERSION=1,BUG_TITLE_MAX_LENGTH=200,BUG_SECTION_MAX_LENGTH=12e3;const V=48,L=/^[a-z0-9]+(?:-[a-z0-9]+)*$/u;export const CAMPAIGN_ID_MAX_LENGTH=60;export class BugRecordError extends Error{problems;constructor(e,n){super(`${e}:
+import z from"node:crypto";import S from"node:path";import{isPublicTestType as G}from"../../domain/identifiers.js";import{isUnknownRecord as l}from"../../kernel/unknown-values.js";import{BUG_CATCHERS as O,BUG_CAUSES as j,BUG_CONFIDENCES as $,BUG_ESCAPES as P,BUG_IMPACTS as R,BUG_SEVERITIES as U,categoryIds as _,isOneOf as B}from"./categories.js";import{BUG_FACTS_TEST_TYPES as F,decodeFaultLines as V,isCommitId as b}from"./facts.js";import{redactBugText as w}from"../../tracker/redaction.js";import{isTimestamp as I}from"../campaigns/campaign.js";export const BUG_DRAFT_SCHEMA_VERSION=1,BUG_TITLE_MAX_LENGTH=200,BUG_SECTION_MAX_LENGTH=12e3;const D=48,L=/^[a-z0-9]+(?:-[a-z0-9]+)*$/u;export const CAMPAIGN_ID_MAX_LENGTH=60;export class BugRecordError extends Error{problems;constructor(e,n){super(`${e}:
 ${n.map(s=>`- ${s}`).join(`
-`)}`),this.name="BugRecordError",this.problems=n}}export function isCampaignId(t){return typeof t=="string"&&t.length<=CAMPAIGN_ID_MAX_LENGTH&&L.test(t)}export function testTypeOfFile(t){const e=/\.([a-z0-9]+)\.testkit\.ts$/u.exec(t)?.[1];return j(e)?e:null}function M(t){const e=/\.([a-z0-9]+)\.testkit\.ts$/u.exec(t)?.[1];return z.find(n=>n===e)??null}export function toRepositoryPath(t,e,n=process.platform){const s=n==="win32"?e.replaceAll("\\","/"):e;if(s.startsWith("/")||/^[a-z]:/iu.test(s))return null;const i=A.posix.normalize(A.posix.join(t,s));return i==="."||i===".."||i.startsWith("../")?null:i}export function toProductPath(t,e){return t===""?e:A.posix.relative(t,e)}export function readBugReport(t,e){if(!g(t))throw new BugRecordError("The bug record is invalid",["it must be a JSON object"]);const n=[],s=[],i=I(t,n,{path:r=>toRepositoryPath(e,r),text:(r,d)=>{const a=B(r);return a!==r&&s.push(d),a}});if(n.length>0)throw new BugRecordError("The bug record is invalid",n);return{redactedFields:s,report:i}}export function createBugDraft(t,e){const n=testTypeOfFile(t.test.file);return{schemaVersion:BUG_DRAFT_SCHEMA_VERSION,id:bugDraftId(t),recordedAt:e.recordedAt.toISOString(),recordedCommit:e.recordedCommit,productPath:e.productPath,caughtBy:e.campaign===null?"manual":"campaign",...e.campaign===null?{}:{campaign:e.campaign},...n===null?{}:{testType:n},...e.vitest===void 0?{}:{vitest:e.vitest},...t}}export function requireOneVitestProject(t,e,n){if(e.length<2||n.project!==null)return;const s=e.map(({project:i})=>JSON.stringify(i)).join(", ");throw new Error(`test.file ${t} runs in the Vitest projects ${s}. Name the one whose run fails with --vitest-project.`)}export function readBugDraft(t){const e="The bug draft is unreadable";if(!g(t))throw new BugRecordError(e,["it must be a JSON object"]);if(t.schemaVersion!==BUG_DRAFT_SCHEMA_VERSION)throw new BugRecordError(e,[`unknown schemaVersion ${JSON.stringify(t.schemaVersion)}`]);const n=[],s=(p,y,b,x)=>y(p)?p:(n.push(x),b),i=I(t,n,{path:p=>toRepositoryPath("",p),text:p=>p}),r=t.testType===void 0&&testTypeOfFile(i.test.file)===null?null:M(i.test.file);t.testType!==(r??void 0)&&n.push(r===null?"testType must be absent for a test file without a type":"testType does not match test.file");const d=s(t.caughtBy,J,"manual","caughtBy must be campaign or manual"),a=s(t.campaign,k,void 0,"campaign must be a campaign id");d==="campaign"!=(a!==void 0)&&n.push("campaign must be present when caughtBy is campaign and absent otherwise");const T={schemaVersion:BUG_DRAFT_SCHEMA_VERSION,id:s(t.id,isBugDraftId,"","id is not a draft id"),recordedAt:s(t.recordedAt,w,"","recordedAt must be a timestamp"),recordedCommit:s(t.recordedCommit,$,"","recordedCommit must be a full commit id"),productPath:s(t.productPath,N,"","productPath must be a string"),caughtBy:d,...a===void 0?{}:{campaign:a},...r===null?{}:{testType:r},...t.vitest===void 0?{}:{vitest:s(t.vitest,X,{},"vitest must hold runner, config and project text")},...i,...t.filed===void 0?{}:{filed:H(t.filed,n)}};if(n.length>0)throw new BugRecordError(e,n);return T}export function bugDraftId(t){const e=t.title.normalize("NFKD").replace(/\p{M}+/gu,"").toLowerCase().replace(/[^a-z0-9]+/gu,"-").slice(0,V).replace(/^-+|-+$/gu,""),n=G.createHash("sha256").update(`${t.test.file}
-${t.test.title}`).digest("hex");return`${e||"bug"}-${n.slice(0,8)}`}export function isBugDraftId(t){return typeof t=="string"&&/^[a-z0-9]+(?:-[a-z0-9]+)*-[0-9a-f]{8}$/u.test(t)}export function namesSameTest(t,e){return t.test.file===e.test.file&&t.test.title===e.test.title}function I(t,e,n){const s=(o,c,u,f)=>{const m=o[c];if(typeof m!="string"||m.trim().length===0)return e.push(`${u} must be a non-empty string`),"";const h=m.trim(),l=h.length>f?h:n.text(h,u);return l.length>f&&e.push(`${u} must be at most ${String(f)} characters`),l},i=(o,c,u)=>{const f=o[c],m=typeof f=="string"?f.trim():"",h=m.length>0?n.path(m):null;return h===null?e.push(`${u} must be a path inside the repository`):[m,h].some(l=>B(l)!==l)&&e.push(`${u} is a path that holds text Testkit redacts as a secret`),h??""},r=(o,c,u)=>E(o,c)?c:(e.push(`${u} must be one of ${o.join(", ")}`),o[0]),d=o=>{const c=t[o];return g(c)?c:(e.push(`${o} must be an object`),{})},a=(o,c,u)=>{const f=s(o,c,u,BUG_TITLE_MAX_LENGTH);return/[\r\n]/u.test(f)&&e.push(`${u} must be one line`),f},T=a(t,"title","title"),p=d("test"),y=d("fault"),b=d("confidence"),x=i(p,"file","test.file");let S=[1,1];try{S=D(y.lines)}catch(o){e.push(o instanceof Error?o.message:"fault.lines is invalid")}return{title:T,severity:r(R,t.severity,"severity"),expected:s(t,"expected","expected",BUG_SECTION_MAX_LENGTH),authority:s(t,"authority","authority",BUG_SECTION_MAX_LENGTH),observed:s(t,"observed","observed",BUG_SECTION_MAX_LENGTH),rootCause:s(t,"rootCause","rootCause",BUG_SECTION_MAX_LENGTH),test:{file:x,title:a(p,"title","test.title")},fault:{lines:S,path:i(y,"path","fault.path")},cause:r(_(P),t.cause,"cause"),impact:r(_(F),t.impact,"impact"),confidence:{cause:r(C,b.cause,"confidence.cause"),impact:r(C,b.impact,"confidence.impact")},escape:r(_(U),t.escape,"escape")}}function H(t,e){return g(t)&&w(t.filedAt)&&$(t.foundAt)&&typeof t.number=="number"&&Number.isSafeInteger(t.number)&&t.number>=1&&typeof t.url=="string"?{filedAt:t.filedAt,foundAt:t.foundAt,number:t.number,url:t.url}:(e.push("filed must hold filedAt, foundAt, number, and url"),{filedAt:"",foundAt:"",number:0,url:""})}function k(t){return t===void 0||isCampaignId(t)}function J(t){return E(O,t)}function X(t){return g(t)&&Object.values(t).every(N)}function N(t){return typeof t=="string"}
+`)}`),this.name="BugRecordError",this.problems=n}}export function isCampaignId(t){return typeof t=="string"&&t.length<=CAMPAIGN_ID_MAX_LENGTH&&L.test(t)}export function testTypeOfFile(t){const e=/\.([a-z0-9]+)\.testkit\.ts$/u.exec(t)?.[1];return G(e)?e:null}function M(t){const e=/\.([a-z0-9]+)\.testkit\.ts$/u.exec(t)?.[1];return F.find(n=>n===e)??null}export function toRepositoryPath(t,e,n=process.platform){const s=n==="win32"?e.replaceAll("\\","/"):e;if(s.startsWith("/")||/^[a-z]:/iu.test(s))return null;const i=S.posix.normalize(S.posix.join(t,s));return i==="."||i===".."||i.startsWith("../")?null:i}export function toProductPath(t,e){return t===""?e:S.posix.relative(t,e)}export function readBugReport(t,e){if(!l(t))throw new BugRecordError("The bug record is invalid",["it must be a JSON object"]);const n=[],s=[],i=N(t,n,{path:r=>toRepositoryPath(e,r),text:(r,p)=>{const a=w(r);return a!==r&&s.push(p),a}});if(n.length>0)throw new BugRecordError("The bug record is invalid",n);return{redactedFields:s,report:i}}export function createBugDraft(t,e){const n=testTypeOfFile(t.test.file);return{schemaVersion:BUG_DRAFT_SCHEMA_VERSION,id:bugDraftId(t),recordedAt:e.recordedAt.toISOString(),recordedCommit:e.recordedCommit,productPath:e.productPath,caughtBy:e.campaign===null?"manual":"campaign",...e.campaign===null?{}:{campaign:e.campaign},...n===null?{}:{testType:n},...e.retries===void 0?{}:{retries:e.retries},...e.timezone===void 0?{}:{timezone:e.timezone},...e.vitest===void 0?{}:{vitest:e.vitest},...t}}export function requireOneVitestProject(t,e,n){if(e.length<2||n.project!==null)return;const s=e.map(({project:i})=>JSON.stringify(i)).join(", ");throw new Error(`test.file ${t} runs in the Vitest projects ${s}. Name the one whose run fails with --vitest-project.`)}export function readBugDraft(t){const e="The bug draft is unreadable";if(!l(t))throw new BugRecordError(e,["it must be a JSON object"]);if(t.schemaVersion!==BUG_DRAFT_SCHEMA_VERSION)throw new BugRecordError(e,[`unknown schemaVersion ${JSON.stringify(t.schemaVersion)}`]);const n=[],s=(d,y,T,A)=>y(d)?d:(n.push(A),T),i=N(t,n,{path:d=>toRepositoryPath("",d),text:d=>d}),r=t.testType===void 0&&testTypeOfFile(i.test.file)===null?null:M(i.test.file);t.testType!==(r??void 0)&&n.push(r===null?"testType must be absent for a test file without a type":"testType does not match test.file");const p=s(t.caughtBy,J,"manual","caughtBy must be campaign or manual"),a=s(t.campaign,k,void 0,"campaign must be a campaign id");p==="campaign"!=(a!==void 0)&&n.push("campaign must be present when caughtBy is campaign and absent otherwise");const x={schemaVersion:BUG_DRAFT_SCHEMA_VERSION,id:s(t.id,isBugDraftId,"","id is not a draft id"),recordedAt:s(t.recordedAt,I,"","recordedAt must be a timestamp"),recordedCommit:s(t.recordedCommit,b,"","recordedCommit must be a full commit id"),productPath:s(t.productPath,C,"","productPath must be a string"),caughtBy:p,...a===void 0?{}:{campaign:a},...r===null?{}:{testType:r},...t.retries===void 0?{}:{retries:s(t.retries,X,0,"retries must be a whole number from 0 to 99")},...t.timezone===void 0?{}:{timezone:s(t.timezone,C,"","timezone must be a time zone name")},...t.vitest===void 0?{}:{vitest:s(t.vitest,q,{},"vitest must hold Vitest selector text")},...t.bisect===void 0?{}:{bisect:s(t.bisect,W,{bad:"",exact:!1,good:"",probes:[],test:""},"bisect must hold the good and bad commits, the test it ran, and its probes")},...i,...t.filed===void 0?{}:{filed:H(t.filed,n)}};if(n.length>0)throw new BugRecordError(e,n);return x}export function bugDraftId(t){const e=t.title.normalize("NFKD").replace(/\p{M}+/gu,"").toLowerCase().replace(/[^a-z0-9]+/gu,"-").slice(0,D).replace(/^-+|-+$/gu,""),n=z.createHash("sha256").update(`${t.test.file}
+${t.test.title}`).digest("hex");return`${e||"bug"}-${n.slice(0,8)}`}export function isBugDraftId(t){return typeof t=="string"&&/^[a-z0-9]+(?:-[a-z0-9]+)*-[0-9a-f]{8}$/u.test(t)}export function namesSameTest(t,e){return t.test.file===e.test.file&&t.test.title===e.test.title}function N(t,e,n){const s=(o,c,u,f)=>{const m=o[c];if(typeof m!="string"||m.trim().length===0)return e.push(`${u} must be a non-empty string`),"";const h=m.trim(),g=h.length>f?h:n.text(h,u);return g.length>f&&e.push(`${u} must be at most ${String(f)} characters`),g},i=(o,c,u)=>{const f=o[c],m=typeof f=="string"?f.trim():"",h=m.length>0?n.path(m):null;return h===null?e.push(`${u} must be a path inside the repository`):[m,h].some(g=>w(g)!==g)&&e.push(`${u} is a path that holds text Testkit redacts as a secret`),h??""},r=(o,c,u)=>B(o,c)?c:(e.push(`${u} must be one of ${o.join(", ")}`),o[0]),p=o=>{const c=t[o];return l(c)?c:(e.push(`${o} must be an object`),{})},a=(o,c,u)=>{const f=s(o,c,u,BUG_TITLE_MAX_LENGTH);return/[\r\n]/u.test(f)&&e.push(`${u} must be one line`),f},x=a(t,"title","title"),d=p("test"),y=p("fault"),T=p("confidence"),A=i(d,"file","test.file");let E=[1,1];try{E=V(y.lines)}catch(o){e.push(o instanceof Error?o.message:"fault.lines is invalid")}return{title:x,severity:r(U,t.severity,"severity"),expected:s(t,"expected","expected",BUG_SECTION_MAX_LENGTH),authority:s(t,"authority","authority",BUG_SECTION_MAX_LENGTH),observed:s(t,"observed","observed",BUG_SECTION_MAX_LENGTH),rootCause:s(t,"rootCause","rootCause",BUG_SECTION_MAX_LENGTH),test:{file:A,title:a(d,"title","test.title")},fault:{lines:E,path:i(y,"path","fault.path")},cause:r(_(j),t.cause,"cause"),impact:r(_(R),t.impact,"impact"),confidence:{cause:r($,T.cause,"confidence.cause"),impact:r($,T.impact,"confidence.impact")},escape:r(_(P),t.escape,"escape")}}function H(t,e){return l(t)&&I(t.filedAt)&&b(t.foundAt)&&typeof t.number=="number"&&Number.isSafeInteger(t.number)&&t.number>=1&&typeof t.url=="string"?{filedAt:t.filedAt,foundAt:t.foundAt,number:t.number,url:t.url}:(e.push("filed must hold filedAt, foundAt, number, and url"),{filedAt:"",foundAt:"",number:0,url:""})}function k(t){return t===void 0||isCampaignId(t)}function J(t){return B(O,t)}function X(t){return typeof t=="number"&&Number.isInteger(t)&&t>=0&&t<=99}function W(t){return l(t)&&b(t.good)&&b(t.bad)&&t.good!==t.bad&&typeof t.exact=="boolean"&&typeof t.test=="string"&&/^[0-9a-f]{64}$/u.test(t.test)&&Array.isArray(t.probes)&&t.probes.every(e=>l(e)&&b(e.commit)&&["bad","good","skip"].includes(String(e.verdict)))}function q(t){return l(t)&&Object.values(t).every(C)}function C(t){return typeof t=="string"}

package/dist/app/bugs/facts.js

@@ -1,2 +1,2 @@
-import{isAreaId as w}from"../../config/loader/bugs.js";import{isUnknownRecord as m}from"../../kernel/unknown-values.js";import{BUG_CATCHERS as E,BUG_CAUSES as S,BUG_CONFIDENCES as p,BUG_ESCAPES as b,BUG_FACTS_VERSION as h,BUG_IMPACTS as T,categoryIds as s,isOneOf as A}from"./categories.js";export const FACTS_START_MARKER="<!-- testkit-facts:start -->",FACTS_END_MARKER="<!-- testkit-facts:end -->";const B=/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/u;export const BUG_FACTS_TEST_TYPES=Object.freeze(["ui","e2e","int","contract","data","dal","load"]);export function renderFactsBlock(e){return["<details>","<summary>Testkit facts</summary>","",FACTS_START_MARKER,"","```json",JSON.stringify(l(e),null,2),"```","",FACTS_END_MARKER,"","</details>"].join(`
-`)}export function readFactsBlock(e){const t=e.lastIndexOf(FACTS_START_MARKER);if(t===-1)return{status:"missing"};const r=t+FACTS_START_MARKER.length,n=e.indexOf(FACTS_END_MARKER,r);if(n===-1)return{status:"invalid",reason:"the facts block has no end marker"};const d=/```[^\n]*\r?\n([\s\S]*?)\r?\n[ \t]*```/u.exec(e.slice(r,n));if(d===null)return{status:"invalid",reason:"the facts block has no fenced code block"};let u;try{u=JSON.parse(d[1])}catch{return{status:"invalid",reason:"the facts block is not valid JSON"}}try{return{facts:_(u),status:"ok"}}catch(f){return{status:"invalid",reason:f instanceof Error?f.message:"the facts block is invalid"}}}export function isCommitId(e){return typeof e=="string"&&B.test(e)}function _(e){if(!m(e))throw new Error("the facts block is not a JSON object");if(e.v!==h)throw new Error(`unknown facts version ${JSON.stringify(e.v)}`);const t=c(e.confidence,"confidence"),r=c(e.test,"test");return l({v:h,cause:o(s(S),e.cause,"cause"),impact:o(s(T),e.impact,"impact"),confidence:{cause:o(p,t.cause,"confidence.cause"),impact:o(p,t.impact,"confidence.impact")},escape:i(e.escape,n=>o(s(b),n,"escape")),caughtBy:o(E,e.caughtBy,"caughtBy"),campaign:i(e.campaign,n=>a(n,"campaign")),testType:i(e.testType,n=>o(BUG_FACTS_TEST_TYPES,n,"testType")),test:{file:a(r.file,"test.file"),title:i(r.title,n=>a(n,"test.title"))},foundAt:y(e.foundAt,"foundAt"),fault:i(e.fault,k),area:i(e.area,O),introducedBy:i(e.introducedBy,n=>y(n,"introducedBy")),introducedMethod:i(e.introducedMethod,n=>o(["blame"],n,"introducedMethod"))})}function k(e){const t=c(e,"fault");return{lines:i(t.lines,decodeFaultLines),path:a(t.path,"fault.path")}}export function decodeFaultLines(e){if(!Array.isArray(e)||e.length!==2)throw new Error("fault.lines must be [start, end]");const t=e,[r,n]=t;if(!g(r)||!g(n)||r>n)throw new Error("fault.lines must be two line numbers counted from 1, with start no greater than end");return[r,n]}function O(e){const t=c(e,"area");if(!w(t.id))throw new Error("area.id is not an area id");const r=t.share;if(typeof r!="number"||!Number.isFinite(r)||r<0||r>1)throw new Error("area.share must be a number from 0 to 1");return{id:t.id,name:a(t.name,"area.name"),share:r}}function l(e){return{v:e.v,cause:e.cause,impact:e.impact,confidence:{cause:e.confidence.cause,impact:e.confidence.impact},...e.escape===void 0?{}:{escape:e.escape},caughtBy:e.caughtBy,...e.campaign===void 0?{}:{campaign:e.campaign},...e.testType===void 0?{}:{testType:e.testType},test:{file:e.test.file,...e.test.title===void 0?{}:{title:e.test.title}},foundAt:e.foundAt,...e.fault===void 0?{}:{fault:{path:e.fault.path,...e.fault.lines===void 0?{}:{lines:e.fault.lines}}},...e.area===void 0?{}:{area:{id:e.area.id,name:e.area.name,share:e.area.share}},...e.introducedBy===void 0?{}:{introducedBy:e.introducedBy},...e.introducedMethod===void 0?{}:{introducedMethod:e.introducedMethod}}}function i(e,t){return e===void 0?void 0:t(e)}function c(e,t){if(!m(e))throw new Error(`${t} must be an object`);return e}function a(e,t){if(typeof e!="string"||e.trim().length===0)throw new Error(`${t} must be a non-empty string`);return e}function o(e,t,r){if(!A(e,t))throw new Error(`${r} must be one of ${e.join(", ")}`);return t}function y(e,t){if(!isCommitId(e))throw new Error(`${t} must be a full lowercase commit id`);return e}function g(e){return typeof e=="number"&&Number.isSafeInteger(e)&&e>=1}
+import{isAreaId as y}from"../../config/loader/bugs.js";import{isUnknownRecord as p}from"../../kernel/unknown-values.js";import{BUG_CATCHERS as w,BUG_CAUSES as E,BUG_CONFIDENCES as h,BUG_ESCAPES as S,BUG_FACTS_VERSION as b,BUG_IMPACTS as B,categoryIds as d,isOneOf as T}from"./categories.js";export const FACTS_START_MARKER="<!-- testkit-facts:start -->",FACTS_END_MARKER="<!-- testkit-facts:end -->";const A=/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/u;export const BUG_FACTS_TEST_TYPES=Object.freeze(["ui","e2e","int","contract","data","dal","load"]);export function renderFactsBlock(e){return["<details>","<summary>Testkit facts</summary>","",FACTS_START_MARKER,"","```json",JSON.stringify(l(e),null,2),"```","",FACTS_END_MARKER,"","</details>"].join(`
+`)}export function readFactsBlock(e){const t=e.lastIndexOf(FACTS_START_MARKER);if(t===-1)return{status:"missing"};const r=t+FACTS_START_MARKER.length,n=e.indexOf(FACTS_END_MARKER,r);if(n===-1)return{status:"invalid",reason:"the facts block has no end marker"};const u=/```[^\n]*\r?\n([\s\S]*?)\r?\n[ \t]*```/u.exec(e.slice(r,n));if(u===null)return{status:"invalid",reason:"the facts block has no fenced code block"};let f;try{f=JSON.parse(u[1])}catch{return{status:"invalid",reason:"the facts block is not valid JSON"}}try{return{facts:x(f),status:"ok"}}catch(m){return{status:"invalid",reason:m instanceof Error?m.message:"the facts block is invalid"}}}export function isCommitId(e){return typeof e=="string"&&A.test(e)}function x(e){if(!p(e))throw new Error("the facts block is not a JSON object");if(e.v!==b)throw new Error(`unknown facts version ${JSON.stringify(e.v)}`);const t=c(e.confidence,"confidence"),r=c(e.test,"test");return l({v:b,cause:o(d(E),e.cause,"cause"),impact:o(d(B),e.impact,"impact"),confidence:{cause:o(h,t.cause,"confidence.cause"),impact:o(h,t.impact,"confidence.impact")},escape:i(e.escape,n=>o(d(S),n,"escape")),caughtBy:o(w,e.caughtBy,"caughtBy"),campaign:i(e.campaign,n=>s(n,"campaign")),testType:i(e.testType,n=>o(BUG_FACTS_TEST_TYPES,n,"testType")),test:{file:s(r.file,"test.file"),title:i(r.title,n=>s(n,"test.title"))},foundAt:a(e.foundAt,"foundAt"),fault:i(e.fault,k),area:i(e.area,O),introducedBy:i(e.introducedBy,n=>a(n,"introducedBy")),introducedMethod:i(e.introducedMethod,n=>o(["blame"],n,"introducedMethod")),bisect:i(e.bisect,_)})}function _(e){const t=c(e,"bisect");if(typeof t.exact!="boolean")throw new Error("bisect.exact must be true or false");return{bad:a(t.bad,"bisect.bad"),exact:t.exact,good:a(t.good,"bisect.good")}}function k(e){const t=c(e,"fault");return{lines:i(t.lines,decodeFaultLines),path:s(t.path,"fault.path")}}export function decodeFaultLines(e){if(!Array.isArray(e)||e.length!==2)throw new Error("fault.lines must be [start, end]");const t=e,[r,n]=t;if(!g(r)||!g(n)||r>n)throw new Error("fault.lines must be two line numbers counted from 1, with start no greater than end");return[r,n]}function O(e){const t=c(e,"area");if(!y(t.id))throw new Error("area.id is not an area id");const r=t.share;if(typeof r!="number"||!Number.isFinite(r)||r<0||r>1)throw new Error("area.share must be a number from 0 to 1");return{id:t.id,name:s(t.name,"area.name"),share:r}}function l(e){return{v:e.v,cause:e.cause,impact:e.impact,confidence:{cause:e.confidence.cause,impact:e.confidence.impact},...e.escape===void 0?{}:{escape:e.escape},caughtBy:e.caughtBy,...e.campaign===void 0?{}:{campaign:e.campaign},...e.testType===void 0?{}:{testType:e.testType},test:{file:e.test.file,...e.test.title===void 0?{}:{title:e.test.title}},foundAt:e.foundAt,...e.fault===void 0?{}:{fault:{path:e.fault.path,...e.fault.lines===void 0?{}:{lines:e.fault.lines}}},...e.area===void 0?{}:{area:{id:e.area.id,name:e.area.name,share:e.area.share}},...e.introducedBy===void 0?{}:{introducedBy:e.introducedBy},...e.introducedMethod===void 0?{}:{introducedMethod:e.introducedMethod},...e.bisect===void 0?{}:{bisect:{good:e.bisect.good,bad:e.bisect.bad,exact:e.bisect.exact}}}}function i(e,t){return e===void 0?void 0:t(e)}function c(e,t){if(!p(e))throw new Error(`${t} must be an object`);return e}function s(e,t){if(typeof e!="string"||e.trim().length===0)throw new Error(`${t} must be a non-empty string`);return e}function o(e,t,r){if(!T(e,t))throw new Error(`${r} must be one of ${e.join(", ")}`);return t}function a(e,t){if(!isCommitId(e))throw new Error(`${t} must be a full lowercase commit id`);return e}function g(e){return typeof e=="number"&&Number.isSafeInteger(e)&&e>=1}

package/dist/app/bugs/file.js

@@ -1 +1 @@
-import h from"node:fs";import{setTimeout as T}from"node:timers/promises";import m from"node:path";import{readAreaMap as R,readBugPolicy as x}from"../../config/loader/bugs.js";import{loadTestkitConfig as A}from"../../config/loader/index.js";import{findIntroducingCommit as C,findRemoteBranchHolding as _,listUncommittedTrackedFiles as D}from"../../repo/history.js";import{listCommitFiles as O}from"../../repo/tree-listing.js";import{resolveTracker as B}from"../tracker/index.js";import{isTypedTestFile as j}from"../../domain/identifiers.js";import{rejectRetiredTestFile as L}from"../../domain/retired-test-types.js";import{listPlainVitestFiles as M,resolveVitestSelection as N}from"../../engines/catalog.js";import{resolveFaultArea as P}from"./areas.js";import{requireOneVitestProject as q,toProductPath as p}from"./draft.js";import{FACTS_END_MARKER as J,FACTS_START_MARKER as K,readFactsBlock as U,renderFactsBlock as k}from"./facts.js";import{bugFacts as G,bugLabels as H,renderBugIssue as V,reproduceCommand as E,TESTKIT_LABEL as W}from"./issue.js";import{findKnownTest as Q,readKnownTests as Y}from"./known-failures.js";import{holdsRedaction as z,redactBugText as g}from"../../tracker/redaction.js";import{errorMessage as X}from"../../kernel/error-serialization.js";import{readEnvironmentFile as S}from"../environments/environment-file.js";import{resolveSuppliedInput as v,suppliedBindings as Z}from"../environments/supplied-input.js";import{suppliedReproduceArgs as tt}from"./supplied-reproduce.js";import{bugDraftFile as et,displayPath as F,readStoredDrafts as rt,resolveBugCheckout as it,writeJsonFile as st,writeTextFile as ot}from"./store.js";const nt=12,y=10,at=[500,1e3,2e3,4e3],lt=2e3;export async function prepareBugFiling(t){const i=tt(t.supplied??{}),e=await it(t),{draft:r,file:s}=ft(e,t.id);if(r.filed!==void 0)return{draft:r,issue:{number:r.filed.number,url:r.filed.url},status:"already-filed"};requireFileableDraft(r),t.supplied!==void 0&&requireReplayableSuppliedInput(t.supplied,e,t.env);const o=await O(e.repoRoot,e.headCommit);requireCommittedDraftFiles(r,o,e),await requireCleanPushedCheckout(e,t.env);const{config:n}=await A(e.productDir),c=await bugFiling(e,n,o,r),u=await resolveBugTracker(e,n,t.env);return requireNoOpenIssue(await readOpenIssues(u),r.test),readyBugFiling({checkout:e,draft:r,draftFile:s,filing:c,reproduceArgs:i,tracker:u})}export function requireFileableDraft(t){L(t.test.file),requireNoSecretInCommand(t)}export function requireCommittedDraftFiles(t,i,e){for(const r of[t.test.file,t.fault.path])if(!i.some(s=>s.path===r))throw new Error(`${r} is not in commit ${shortCommit(e.headCommit)}. Commit it and push the commit, then run this again.`)}export async function requireCleanPushedCheckout(t,i){const e=shortCommit(t.headCommit),r=(await D(t.repoRoot)).filter(o=>!mt(t,o));if(r.length>0)throw new Error(`Tracked files have uncommitted changes: ${pt(r)}. The issue says that the test fails at ${e}, so commit or discard the changes first.`);const s=await _(t.repoRoot,t.headCommit,i);if(s.status==="missing")throw new Error(`No remote branch contains commit ${e}. Push it, then run this again. The issue points readers to the test at that commit.`);if(s.status==="unknown")throw new Error([`Testkit could not tell whether a remote branch contains commit ${e}, and the issue points readers to the test at that commit.`,...s.problems.map(o=>o.replace(/\.?$/u,".")),...s.problems.length===0?[]:["Make sure Git can read the remote from this checkout, for example by giving the checkout credentials that can read the repository, then run this again."],...s.shallow?["This checkout is a shallow clone, so its history can stop before a branch tip reaches the commit. Fetch the full history with `git fetch --unshallow`, then run this again."]:[]].join(" "))}export async function bugFiling(t,i,e,r){const s=ht(P(R(i),e,r.fault.path),r.fault.path),o=await C(t.repoRoot,t.headCommit,r.fault.path,r.fault.lines);return{area:s,foundAt:t.headCommit,introducedBy:o.status==="found"?o.commit:null}}export function resolveBugTracker(t,i,e){return B({env:e,productDir:t.productDir,repository:x(i).repository??void 0})}export function readyBugFiling(t){return{...t,status:"ready",testFile:m.join(t.checkout.productDir,p(t.checkout.productPath,t.draft.test.file))}}export async function bugRerun(t,i,e={},r=process.env){const{checkout:s,draft:o}=t,n=Object.entries(i).filter(([f,I])=>f.startsWith("vitest-")&&typeof I=="string"),c={...o.vitest,...Object.fromEntries(n)},u=Object.keys(c).length===0?o:{...o,vitest:c};requireNoSecretInCommand(u);const l=e.environmentFile,a=v(e,l===void 0?null:S(l,s.cwd),r),d=N(c,s.productDir,r),w=p(s.productPath,o.test.file),b=j(w)?[]:await M(s.productDir,{...d,host:{...d.host,...Z(a)}}),$=b.filter(f=>f.file===w);return q(o.test.file,$,d),{files:{...r,...a.env}.TESTKIT_REQUIRE_COMPLETE_SUITE==="1"&&$.length>0?[...new Set(b.map(({file:f})=>m.join(s.productDir,f)))]:[t.testFile],ready:{...t,draft:u}}}export async function fileBug(t,i,e={}){const r=requireRecordedFailure(t.draft,i,shortCommit(t.checkout.headCommit),t.reproduceArgs);requireNoOpenIssue(await readOpenIssues(t.tracker),r);const s=renderReadableBugIssue(t.draft,t.filing,t.reproduceArgs);await t.tracker.ensureLabels(H(t.draft.severity,t.filing.area));const o=h.readFileSync(t.draftFile,"utf8"),n=await t.tracker.createIssue(s),c=recordFiledBug(t,n,e.at??new Date),u=await ut(t.tracker,n.number).catch(a=>{throw new Error(`Testkit filed #${String(n.number)}: ${n.url}, and its draft records that, but Testkit could not read the open issues again to check whether another checkout filed this test at the same time: ${X(a).replace(/\.?$/u,".")} Look for a second open issue for this test.`)}),l=dt(u.issues,r,n.number);if(l!==null){const a=await t.tracker.closeDuplicateIssue(n,{number:l.issue,url:l.url});throw ot(t.draftFile,o),ct(l,n,a)}return{...c,listed:u.listed}}export function recordFiledBug({checkout:t,draft:i,draftFile:e,filing:r},s,o){const n={...i,filed:{filedAt:o.toISOString(),foundAt:r.foundAt,number:s.number,url:s.url}};return st(e,n),{area:r.area,covers:"test",draft:n,introducedBy:r.introducedBy,issue:{number:s.number,url:s.url},path:F(t,e)}}export function renderReadableBugIssue(t,i,e=[]){const r=V(t,i,e),s=U(r.body),o=k(G(t,i));if(s.status==="ok"&&k(s.facts)===o)return r;throw new Error(`Testkit cannot read back the facts of the issue it would file, so it files nothing. This happens when test.title, test.file, fault.path, or the area's name holds ${K} or ${J}. Rename the test, the file, or the area, then record the bug again.`)}export function requireNoSecretInCommand(t){const i=E(t);if(g(i)!==i)throw new Error("The command that reproduces this bug holds text that Testkit redacts as a secret, and an issue carries that command. The text is in the path of the product directory, in the test's path from that directory, or in a --vitest-config or --vitest-project selector. Rename what holds it, commit and push the change, then record the bug again.")}export function requireReplayableSuppliedInput(t,i,e){const r=t.environmentFile;if(r!==void 0&&!m.isAbsolute(r)&&h.realpathSync.native(i.cwd)!==h.realpathSync.native(i.repoRoot))throw new Error(`The issue's reproduce command runs from the repository root, but --environment-file ${r} is relative to ${i.cwd}. Run testkit bug file from ${i.repoRoot}, or pass the file's absolute path.`);v(t,r===void 0?null:S(r,i.cwd),e)}export function requireRecordedFailure(t,i,e,r=[]){const{file:s,title:o}=t.test,n=`\`${E(t,r)}\``;if(i.kind==="passed")throw new Error(`${s} passed at ${e}, so Testkit has no failure to file. ${n} must fail.`);if(i.kind==="run-failed")throw new Error(`${n} failed outside its tests at ${e}: ${i.reason.replace(/\.?$/u,".")} Testkit files a bug when the test itself fails.`);const c=i.failures.filter(a=>a.file===s);if(c.length===0)throw new Error(`${n} failed at ${e}, but ${s} is not among the files that failed.`);const u=c.flatMap(a=>a.title===null?[]:[a.title]),l=u.filter(a=>!/[\r\n]/u.test(a)&&!z(a)&&g(a)===a);if(l.includes(o)){if(c.some(a=>a.title===o&&a.ambiguous===!0))throw new Error(`${s} has more than one test titled ${JSON.stringify(o)} at ${e}. An issue names its test by file and title, so it could not say which of them failed. Give each test in the file its own title, commit and push the change, then record the bug again.`);return{file:s,title:o}}if(u.length>0){const a=u.length-l.length;throw new Error([`${s} failed at ${e}, but not under the title ${JSON.stringify(o)}.`,...l.length===0?[]:[`Testkit reported ${l.map(d=>JSON.stringify(d)).join(", ")}. Set test.title to the title Testkit reports, then record the bug again.`],...a===0?[]:[`Testkit ${l.length===0?"":"also "}reported ${a===1?"1 title that spans lines or holds":`${String(a)} titles that span lines or hold`} text it redacts as a secret. It files no bug under such a title. Rename the test, or label the assertion, so that its title is one line and holds no such text.`]].join(" "))}throw new Error(`${s} failed at ${e} without reaching a result, so Testkit cannot tell that ${JSON.stringify(o)} failed. A file fails this way when it crashes, times out, or does not load. Make the test fail on its assertion, then run this again.`)}export async function readOpenIssues(t){return Y(await t.listOpenIssues(W)).tests}async function ut(t,i){let e=await readOpenIssues(t);for(const r of at){if(e.some(s=>s.issue===i))break;await T(r),e=await readOpenIssues(t)}return e.some(r=>r.issue===i)?t.target.provider!=="github"?{issues:e,listed:!0}:(await T(lt),{issues:await readOpenIssues(t),listed:!0}):{issues:e,listed:!1}}function ct(t,i,e){const r=`#${String(i.number)}`;return new Error(`Open issue #${String(t.issue)} already covers this test: ${t.url}. Another checkout filed it while this one filed ${r}, ${e?`so Testkit closed ${r}`:`and Testkit could not close ${r}; close it by hand: ${i.url}`}.`)}function dt(t,i,e){return t.filter(s=>s.issue<e&&s.file===i.file&&(s.title===null||s.title===i.title)).sort((s,o)=>s.issue-o.issue).at(0)??null}export function requireNoOpenIssue(t,i){const e=Q(t,i);if(e===null)return;const r=`Open issue #${String(e.issue)}`;throw new Error(e.title===null?`${r} already covers the whole file ${i.file}: ${e.url}. While it is open, Testkit files no other bug for that file.`:`${r} already covers this test: ${e.url}. Testkit keeps one open bug for each test.`)}function ft(t,i){const e=rt(t.productDir),r=e.drafts.find(n=>n.draft.id===i);if(r!==void 0)return r;const s=e.unreadable.find(n=>n.file===et(t.productDir,i));if(s!==void 0)throw new Error(`Testkit cannot read ${F(t,s.file)}: ${s.problem}. Record the bug again with \`testkit bug record\`.`);const o=e.drafts.filter(n=>n.draft.filed===void 0).map(n=>n.draft.id);throw new Error(o.length===0?`No bug draft has the id ${JSON.stringify(i)}, and this checkout has no unfiled draft. Record the bug with \`testkit bug record\` first.`:`No bug draft has the id ${JSON.stringify(i)}. Unfiled drafts: ${o.join(", ")}.`)}function ht(t,i){if(t.status==="no-map")return null;const e="Fix `areas` in testkit.config.ts, commit and push the change, then run this again.";if(t.status==="resolved"){const{area:r}=t;if(g(r.name)===r.name)return r;throw new Error(`areas.${r.id}.name holds text that Testkit redacts as a secret, and an issue carries its area's name. ${e}`)}throw new Error(t.status==="unclaimed"?`No area claims ${i}. Every bug needs its fault in one area. ${e}`:`The areas ${t.areas.join(" and ")} both claim ${i}. A file belongs to one area. ${e}`)}function mt(t,i){return p(t.productPath,i).startsWith(".testkit/")}function pt(t){const i=t.length-y;return`${t.slice(0,y).join(", ")}${i>0?`, and ${String(i)} more`:""}`}export function shortCommit(t){return t.slice(0,nt)}
+import h from"node:fs";import{setTimeout as E}from"node:timers/promises";import m from"node:path";import{readAreaMap as _,readBugPolicy as D}from"../../config/loader/bugs.js";import{loadTestkitConfig as O}from"../../config/loader/index.js";import{findIntroducingCommit as B,findRemoteBranchHolding as j,isAncestor as L,listUncommittedTrackedFiles as M}from"../../repo/history.js";import{bisectIdentity as k}from"./bisect-identity.js";import{listCommitFiles as N}from"../../repo/tree-listing.js";import{resolveTracker as P}from"../tracker/index.js";import{isTypedTestFile as q}from"../../domain/identifiers.js";import{rejectRetiredTestFile as J}from"../../domain/retired-test-types.js";import{listPlainVitestFiles as K,resolveVitestSelection as U}from"../../engines/catalog.js";import{resolveFaultArea as G}from"./areas.js";import{requireOneVitestProject as H,toProductPath as p}from"./draft.js";import{FACTS_END_MARKER as V,FACTS_START_MARKER as W,readFactsBlock as Q,renderFactsBlock as S}from"./facts.js";import{bugFacts as Y,bugLabels as z,renderBugIssue as X,reproduceCommand as v,TESTKIT_LABEL as Z}from"./issue.js";import{findKnownTest as tt,readKnownTests as et}from"./known-failures.js";import{holdsRedaction as it,redactBugText as g}from"../../tracker/redaction.js";import{errorMessage as rt}from"../../kernel/error-serialization.js";import{readEnvironmentFile as F}from"../environments/environment-file.js";import{resolveSuppliedInput as y,suppliedBindings as st}from"../environments/supplied-input.js";import{suppliedReproduceArgs as ot}from"./supplied-reproduce.js";import{bugDraftFile as nt,displayPath as I,readStoredDrafts as at,resolveBugCheckout as lt,writeJsonFile as ut,writeTextFile as ct}from"./store.js";const dt=12,R=10,ft=[500,1e3,2e3,4e3],ht=2e3;export async function prepareBugFiling(t){const r=ot(t.supplied??{}),i=await lt(t),{draft:e,file:s}=wt(i,t.id);if(e.filed!==void 0)return{draft:e,issue:{number:e.filed.number,url:e.filed.url},status:"already-filed"};requireFileableDraft(e),t.supplied!==void 0&&requireReplayableSuppliedInput(t.supplied,i,t.env);const o=await N(i.repoRoot,i.headCommit);requireCommittedDraftFiles(e,o,i),await requireCleanPushedCheckout(i,t.env);const{config:n}=await O(i.productDir),u=await bugFiling(i,n,o,e),l=await resolveBugTracker(i,n,t.env);return requireNoOpenIssue(await readOpenIssues(l),e.test),readyBugFiling({checkout:i,draft:e,draftFile:s,filing:u,reproduceArgs:r,tracker:l})}export function requireFileableDraft(t){J(t.test.file),requireNoSecretInCommand(t)}export function requireCommittedDraftFiles(t,r,i){for(const e of[t.test.file,t.fault.path])if(!r.some(s=>s.path===e))throw new Error(`${e} is not in commit ${shortCommit(i.headCommit)}. Commit it and push the commit, then run this again.`)}export async function requireCleanPushedCheckout(t,r){const i=shortCommit(t.headCommit),e=(await M(t.repoRoot)).filter(o=>!$t(t,o));if(e.length>0)throw new Error(`Tracked files have uncommitted changes: ${Tt(e)}. The issue says that the test fails at ${i}, so commit or discard the changes first.`);const s=await j(t.repoRoot,t.headCommit,r);if(s.status==="missing")throw new Error(`No remote branch contains commit ${i}. Push it, then run this again. The issue points readers to the test at that commit.`);if(s.status==="unknown")throw new Error([`Testkit could not tell whether a remote branch contains commit ${i}, and the issue points readers to the test at that commit.`,...s.problems.map(o=>o.replace(/\.?$/u,".")),...s.problems.length===0?[]:["Make sure Git can read the remote from this checkout, for example by giving the checkout credentials that can read the repository, then run this again."],...s.shallow?["This checkout is a shallow clone, so its history can stop before a branch tip reaches the commit. Fetch the full history with `git fetch --unshallow`, then run this again."]:[]].join(" "))}export async function bugFiling(t,r,i,e){const s=bt(G(_(r),i,e.fault.path),e.fault.path),o=await B(t.repoRoot,t.headCommit,e.fault.path,e.fault.lines),{bisect:n}=e,u=n!==void 0&&k(t.repoRoot,e)===n.test&&await L(t.repoRoot,n.bad,t.headCommit);return{area:s,bisect:u?n:null,foundAt:t.headCommit,introducedBy:o.status==="found"?o.commit:null}}export function resolveBugTracker(t,r,i){return P({env:i,productDir:t.productDir,repository:D(r).repository??void 0})}export function readyBugFiling(t){return{...t,status:"ready",testFile:m.join(t.checkout.productDir,p(t.checkout.productPath,t.draft.test.file))}}export async function bugRerun(t,r,i={},e=process.env){const{checkout:s,draft:o}=t,n=Object.entries(r).filter(([f,C])=>f.startsWith("vitest-")&&typeof C=="string"),u={...o.vitest,...Object.fromEntries(n)},l=Object.keys(u).length===0?o:{...o,vitest:u};requireNoSecretInCommand(l);const c=i.environmentFile,a=y(i,c===void 0?null:F(c,s.cwd),e),d=U(u,s.productDir,e),w=p(s.productPath,o.test.file),b=q(w)?[]:await K(s.productDir,{...d,host:{...d.host,...st(a)}}),$=b.filter(f=>f.file===w);H(o.test.file,$,d);const x={...e,...a.env}.TESTKIT_REQUIRE_COMPLETE_SUITE==="1"&&$.length>0?[...new Set(b.map(({file:f})=>m.join(s.productDir,f)))]:[t.testFile],{bisect:T}=t.filing,A=l===o||!T||k(s.repoRoot,l)===T.test?t.filing:{...t.filing,bisect:null};return{files:x,ready:{...t,draft:l,filing:A}}}export async function fileBug(t,r,i={}){const e=requireRecordedFailure(t.draft,r,shortCommit(t.checkout.headCommit),t.reproduceArgs);requireNoOpenIssue(await readOpenIssues(t.tracker),e);const s=renderReadableBugIssue(t.draft,t.filing,t.reproduceArgs);await t.tracker.ensureLabels(z(t.draft.severity,t.filing.area));const o=h.readFileSync(t.draftFile,"utf8"),n=await t.tracker.createIssue(s),u=recordFiledBug(t,n,i.at??new Date),l=await mt(t.tracker,n.number).catch(a=>{throw new Error(`Testkit filed #${String(n.number)}: ${n.url}, and its draft records that, but Testkit could not read the open issues again to check whether another checkout filed this test at the same time: ${rt(a).replace(/\.?$/u,".")} Look for a second open issue for this test.`)}),c=gt(l.issues,e,n.number);if(c!==null){const a=await t.tracker.closeDuplicateIssue(n,{number:c.issue,url:c.url});throw ct(t.draftFile,o),pt(c,n,a)}return{...u,listed:l.listed}}export function recordFiledBug({checkout:t,draft:r,draftFile:i,filing:e},s,o){const n={...r,filed:{filedAt:o.toISOString(),foundAt:e.foundAt,number:s.number,url:s.url}};return ut(i,n),{area:e.area,covers:"test",draft:n,introducedBy:e.introducedBy,issue:{number:s.number,url:s.url},path:I(t,i)}}export function renderReadableBugIssue(t,r,i=[]){const e=X(t,r,i),s=Q(e.body),o=S(Y(t,r));if(s.status==="ok"&&S(s.facts)===o)return e;throw new Error(`Testkit cannot read back the facts of the issue it would file, so it files nothing. This happens when test.title, test.file, fault.path, or the area's name holds ${W} or ${V}. Rename the test, the file, or the area, then record the bug again.`)}export function requireNoSecretInCommand(t){const r=v(t);if(g(r)!==r)throw new Error("The command that reproduces this bug holds text that Testkit redacts as a secret, and an issue carries that command. The text is in the path of the product directory, in the test's path from that directory, or in a --vitest-config or --vitest-project selector. Rename what holds it, commit and push the change, then record the bug again.")}export function requireReplayableSuppliedInput(t,r,i){const e=t.environmentFile;if(e!==void 0&&!m.isAbsolute(e)&&h.realpathSync.native(r.cwd)!==h.realpathSync.native(r.repoRoot))throw new Error(`The issue's reproduce command runs from the repository root, but --environment-file ${e} is relative to ${r.cwd}. Run testkit bug file from ${r.repoRoot}, or pass the file's absolute path.`);y(t,e===void 0?null:F(e,r.cwd),i)}export function requireRecordedFailure(t,r,i,e=[]){const{file:s,title:o}=t.test,n=`\`${v(t,e)}\``;if(r.kind==="passed")throw new Error(`${s} passed at ${i}, so Testkit has no failure to file. ${n} must fail.`);if(r.kind==="run-failed")throw new Error(`${n} failed outside its tests at ${i}: ${r.reason.replace(/\.?$/u,".")} Testkit files a bug when the test itself fails.`);const u=r.failures.filter(a=>a.file===s);if(u.length===0)throw new Error(`${n} failed at ${i}, but ${s} is not among the files that failed.`);const l=u.flatMap(a=>a.title===null?[]:[a.title]),c=l.filter(a=>!/[\r\n]/u.test(a)&&!it(a)&&g(a)===a);if(c.includes(o)){if(u.some(a=>a.title===o&&a.ambiguous===!0))throw new Error(`${s} has more than one test titled ${JSON.stringify(o)} at ${i}. An issue names its test by file and title, so it could not say which of them failed. Give each test in the file its own title, commit and push the change, then record the bug again.`);return{file:s,title:o}}if(l.length>0){const a=l.length-c.length;throw new Error([`${s} failed at ${i}, but not under the title ${JSON.stringify(o)}.`,...c.length===0?[]:[`Testkit reported ${c.map(d=>JSON.stringify(d)).join(", ")}. Set test.title to the title Testkit reports, then record the bug again.`],...a===0?[]:[`Testkit ${c.length===0?"":"also "}reported ${a===1?"1 title that spans lines or holds":`${String(a)} titles that span lines or hold`} text it redacts as a secret. It files no bug under such a title. Rename the test, or label the assertion, so that its title is one line and holds no such text.`]].join(" "))}throw new Error(`${s} failed at ${i} without reaching a result, so Testkit cannot tell that ${JSON.stringify(o)} failed. A file fails this way when it crashes, times out, or does not load. Make the test fail on its assertion, then run this again.`)}export async function readOpenIssues(t){return et(await t.listOpenIssues(Z)).tests}async function mt(t,r){let i=await readOpenIssues(t);for(const e of ft){if(i.some(s=>s.issue===r))break;await E(e),i=await readOpenIssues(t)}return i.some(e=>e.issue===r)?t.target.provider!=="github"?{issues:i,listed:!0}:(await E(ht),{issues:await readOpenIssues(t),listed:!0}):{issues:i,listed:!1}}function pt(t,r,i){const e=`#${String(r.number)}`;return new Error(`Open issue #${String(t.issue)} already covers this test: ${t.url}. Another checkout filed it while this one filed ${e}, ${i?`so Testkit closed ${e}`:`and Testkit could not close ${e}; close it by hand: ${r.url}`}.`)}function gt(t,r,i){return t.filter(s=>s.issue<i&&s.file===r.file&&(s.title===null||s.title===r.title)).sort((s,o)=>s.issue-o.issue).at(0)??null}export function requireNoOpenIssue(t,r){const i=tt(t,r);if(i===null)return;const e=`Open issue #${String(i.issue)}`;throw new Error(i.title===null?`${e} already covers the whole file ${r.file}: ${i.url}. While it is open, Testkit files no other bug for that file.`:`${e} already covers this test: ${i.url}. Testkit keeps one open bug for each test.`)}function wt(t,r){const i=at(t.productDir),e=i.drafts.find(n=>n.draft.id===r);if(e!==void 0)return e;const s=i.unreadable.find(n=>n.file===nt(t.productDir,r));if(s!==void 0)throw new Error(`Testkit cannot read ${I(t,s.file)}: ${s.problem}. Record the bug again with \`testkit bug record\`.`);const o=i.drafts.filter(n=>n.draft.filed===void 0).map(n=>n.draft.id);throw new Error(o.length===0?`No bug draft has the id ${JSON.stringify(r)}, and this checkout has no unfiled draft. Record the bug with \`testkit bug record\` first.`:`No bug draft has the id ${JSON.stringify(r)}. Unfiled drafts: ${o.join(", ")}.`)}function bt(t,r){if(t.status==="no-map")return null;const i="Fix `areas` in testkit.config.ts, commit and push the change, then run this again.";if(t.status==="resolved"){const{area:e}=t;if(g(e.name)===e.name)return e;throw new Error(`areas.${e.id}.name holds text that Testkit redacts as a secret, and an issue carries its area's name. ${i}`)}throw new Error(t.status==="unclaimed"?`No area claims ${r}. Every bug needs its fault in one area. ${i}`:`The areas ${t.areas.join(" and ")} both claim ${r}. A file belongs to one area. ${i}`)}function $t(t,r){return p(t.productPath,r).startsWith(".testkit/")}function Tt(t){const r=t.length-R;return`${t.slice(0,R).join(", ")}${r>0?`, and ${String(r)} more`:""}`}export function shortCommit(t){return t.slice(0,dt)}

package/dist/app/bugs/issue.js

@@ -1,6 +1,6 @@
-import{toProductPath as l}from"./draft.js";import{renderFactsBlock as p}from"./facts.js";import{redactBugText as n}from"../../tracker/redaction.js";export const TESTKIT_LABEL="testkit",SEVERITY_LABEL_PREFIX="severity:",AREA_LABEL_PREFIX="area:";const d={color:"0f766e",description:"Filed by Testkit with a regression test.",name:TESTKIT_LABEL};export const SEVERITY_LABELS={critical:{color:"9f1239",description:"The product is unsafe to run or to trust."},high:{color:"e11d48",description:"A core behaviour is broken with no acceptable workaround."},medium:{color:"d97706",description:"A behaviour is broken and a workaround exists, or the reach is limited."},low:{color:"4d7c0f",description:"A minor defect with little consequence."}};const m="475569";export function bugLabels(e,t){return[d,{...SEVERITY_LABELS[e],name:`${SEVERITY_LABEL_PREFIX}${e}`},...t===null?[]:[{color:m,description:t.name,name:`${AREA_LABEL_PREFIX}${t.id}`}]]}export function bugFacts(e,t){return{v:1,cause:e.cause,impact:e.impact,confidence:e.confidence,escape:e.escape,caughtBy:e.caughtBy,...e.campaign===void 0?{}:{campaign:e.campaign},...e.testType===void 0?{}:{testType:e.testType},test:e.test,foundAt:t.foundAt,fault:e.fault,...t.area===null?{}:{area:t.area},...t.introducedBy===null?{}:{introducedBy:t.introducedBy,introducedMethod:"blame"}}}export function renderBugIssue(e,t,r=[]){const o=h(e),[i,c]=o.fault.lines,u=i===c?`line ${String(i)}`:`lines ${String(i)} to ${String(c)}`;return{body:`${["## Expected behaviour",o.expected,"## Authority",o.authority,"## Observed behaviour",o.observed,"## Root cause",o.rootCause,"## Reproduce",["```sh",reproduceCommand(o,r),"```"].join(`
-`),[`Test: \`${o.test.file}\`, "${o.test.title}".`,`Found at \`${t.foundAt}\`.`,`Fault: \`${o.fault.path}\`, ${u}.`].join(`
-`),p(bugFacts(o,t))].join(`
+import{toProductPath as a}from"./draft.js";import{renderFactsBlock as d}from"./facts.js";import{redactBugText as s}from"../../tracker/redaction.js";export const TESTKIT_LABEL="testkit",SEVERITY_LABEL_PREFIX="severity:",AREA_LABEL_PREFIX="area:";const p={color:"0f766e",description:"Filed by Testkit with a regression test.",name:TESTKIT_LABEL};export const SEVERITY_LABELS={critical:{color:"9f1239",description:"The product is unsafe to run or to trust."},high:{color:"e11d48",description:"A core behaviour is broken with no acceptable workaround."},medium:{color:"d97706",description:"A behaviour is broken and a workaround exists, or the reach is limited."},low:{color:"4d7c0f",description:"A minor defect with little consequence."}};const l="475569";export function bugLabels(e,t){return[p,{...SEVERITY_LABELS[e],name:`${SEVERITY_LABEL_PREFIX}${e}`},...t===null?[]:[{color:l,description:t.name,name:`${AREA_LABEL_PREFIX}${t.id}`}]]}export function bugFacts(e,t){return{v:1,cause:e.cause,impact:e.impact,confidence:e.confidence,escape:e.escape,caughtBy:e.caughtBy,...e.campaign===void 0?{}:{campaign:e.campaign},...e.testType===void 0?{}:{testType:e.testType},test:e.test,foundAt:t.foundAt,fault:e.fault,...t.area===null?{}:{area:t.area},...t.introducedBy===null?{}:{introducedBy:t.introducedBy,introducedMethod:"blame"},...t.bisect?{bisect:{bad:t.bisect.bad,exact:t.bisect.exact,good:t.bisect.good}}:{}}}export function renderBugIssue(e,t,n=[]){const o=m(e),[i,c]=o.fault.lines,r=i===c?`line ${String(i)}`:`lines ${String(i)} to ${String(c)}`;return{body:`${["## Expected behaviour",o.expected,"## Authority",o.authority,"## Observed behaviour",o.observed,"## Root cause",o.rootCause,"## Reproduce",["```sh",reproduceCommand(o,n),"```"].join(`
+`),[`Test: \`${o.test.file}\`, "${o.test.title}".`,`Found at \`${t.foundAt}\`.`,`Fault: \`${o.fault.path}\`, ${r}.`,...t.bisect?h(t.bisect):[]].join(`
+`),d(bugFacts(o,t))].join(`
 
 `)}
-`,labels:bugLabels(o.severity,t.area).map(a=>a.name),title:o.title}}function h(e){return{...e,title:n(e.title),expected:n(e.expected),authority:n(e.authority),observed:n(e.observed),rootCause:n(e.rootCause),test:{file:e.test.file,title:n(e.test.title)}}}export function reproduceCommand(e,t=[]){const r=l(e.productPath,e.test.file),o=e.productPath===""?`npx testkit run --file ${s(r)}`:`npx testkit run --dir ${s(e.productPath)} --file ${s(r)}`,i=Object.entries(e.vitest??{}).flatMap(([c,u])=>[`--${c}`,u]);return[o,...[...i,...t].map(s)].join(" ")}function s(e){return/^[\w./@:+=-]+$/u.test(e)?e:`'${e.replaceAll("'","'\\''")}'`}
+`,labels:bugLabels(o.severity,t.area).map(u=>u.name),title:o.title}}function m(e){return{...e,title:s(e.title),expected:s(e.expected),authority:s(e.authority),observed:s(e.observed),rootCause:s(e.rootCause),test:{file:e.test.file,title:s(e.test.title)}}}export function reproduceCommand(e,t=[]){const n=a(e.productPath,e.test.file),o=e.productPath===""?`npx testkit run --file ${shellWord(n)}`:`npx testkit run --dir ${shellWord(e.productPath)} --file ${shellWord(n)}`,i=[...Object.entries(e.vitest??{}).flatMap(([c,r])=>[`--${c}`,r]),...e.timezone===void 0?[]:["--timezone",e.timezone],...e.retries===void 0?[]:["--retries",String(e.retries)]];return[o,...[...i,...t].map(shellWord)].join(" ")}export function shellWord(e){return/^[\w./@:+=-]+$/u.test(e)?e:`'${e.replaceAll("'","'\\''")}'`}function h(e){const t={bad:"fails",good:"passes",skip:"gave no verdict"},n=e.probes.map(o=>`\`${o.commit.slice(0,12)}\` ${t[o.verdict]}`);return[e.exact?`The test passes at \`${e.good}\` and fails from its child \`${e.bad}\`.`:`The test passes at \`${e.good}\` and fails at \`${e.bad}\`. The fault came in between them.`,`Testkit ran it at: ${n.join(", ")}.`]}

package/dist/app/bugs/record.js

@@ -1,2 +1,2 @@
-import c from"node:fs";import h from"node:path";import{text as b}from"node:stream/consumers";import{loadTestkitConfig as D}from"../../config/loader/index.js";import{discoverOwnedTests as k}from"../../config/loader/owned-discovery.js";import{isTypedTestFile as S}from"../../domain/identifiers.js";import{resolveVitestSelection as T}from"../../engines/catalog.js";import{posixPath as E}from"../../process/files/paths.js";import{findActiveCampaign as P,readCampaigns as F}from"../campaigns/store.js";import{BugRecordError as $,createBugDraft as x,namesSameTest as C,readBugReport as R,requireOneVitestProject as j,toProductPath as B}from"./draft.js";import{requireNoSecretInCommand as O}from"./file.js";import{bugDraftFile as N,displayPath as y,displayUnreadableDrafts as I,readStoredDrafts as J,resolveBugCheckout as q,writeJsonFile as A}from"./store.js";export async function recordBug(o){const t=await q(o),{redactedFields:i,report:e}=W(await U(t.cwd,o.file),t.productPath);for(const[r,g]of[["test.file",e.test.file],["fault.path",e.fault.path]])if(c.statSync(h.join(t.repoRoot,g),{throwIfNoEntry:!1})?.isFile()!==!0)throw new Error(`${r} resolves to ${g} from the repository root, and no file is there. Write the path relative to the product directory.`);const{config:n}=await D(t.productDir),f=T(o.vitest??{},t.productDir),l=B(t.productPath,e.test.file),d=await k(t.productDir,n,{plain:!S(l),selection:f}),m=d.problems.at(0)??d.migration;if(m!==null)throw new Error(m);if(!d.project.files.some(r=>r.filePath===l))throw new Error(`test.file resolves to ${e.test.file}, and Testkit does not discover a test there. Run \`testkit discover\` to list the tests it runs.`);j(e.test.file,d.plainFiles.filter(({file:r})=>r===l),f);const a=x(e,{campaign:P(F(t.productDir))?.id??null,productPath:t.productPath,recordedAt:o.at??new Date,recordedCommit:t.headCommit,vitest:V(o.vitest??{},t.productDir,f)});O(a);const u=J(t.productDir),p=u.drafts.filter(r=>C(r.draft,a)),w=p.find(r=>r.draft.filed!==void 0)?.draft.filed;if(w!==void 0)throw new Error(`This test already has a filed bug: ${w.url}. Testkit keeps one bug for each test.`);const s=N(t.productDir,a.id),v=u.unreadable.find(r=>r.file===s);if(v!==void 0)throw new Error(`${y(t,s)} exists and Testkit cannot read it: ${v.problem}. Fix or delete the file, then run this again.`);A(s,a);for(const r of p)r.file!==s&&c.rmSync(r.file,{force:!0});return{draft:a,path:y(t,s),redactedFields:i,replaced:p.length>0,unreadableDrafts:I(t,u.unreadable)}}function V({"vitest-project":o,"vitest-runner":t},i,{config:e}){const n={...t===void 0?{}:{"vitest-runner":t},...e===null?{}:{"vitest-config":E(h.relative(c.realpathSync(i),e))},...o===void 0?{}:{"vitest-project":o}};return Object.keys(n).length===0?void 0:n}async function U(o,t){if(t==="-")return{source:"standard input",text:await b(process.stdin)};try{return{source:t,text:c.readFileSync(h.resolve(o,t),"utf8")}}catch(i){throw new Error(`Testkit cannot read the bug record file ${t}.`,{cause:i})}}function W(o,t){let i;try{i=JSON.parse(o.text)}catch(e){throw new Error(`The bug record in ${o.source} is not valid JSON.`,{cause:e})}try{return R(i,t)}catch(e){throw e instanceof $?new Error(`${e.message}
-Run \`testkit bug guide\` to see the record format.`,{cause:e}):e}}
+import c from"node:fs";import h from"node:path";import{text as b}from"node:stream/consumers";import{loadTestkitConfig as k}from"../../config/loader/index.js";import{discoverOwnedTests as D}from"../../config/loader/owned-discovery.js";import{isTypedTestFile as S}from"../../domain/identifiers.js";import{resolveVitestSelection as T}from"../../engines/catalog.js";import{posixPath as E}from"../../process/files/paths.js";import{findActiveCampaign as $,readCampaigns as P}from"../campaigns/store.js";import{BugRecordError as F,createBugDraft as x,namesSameTest as C,readBugReport as R,requireOneVitestProject as N,toProductPath as O}from"./draft.js";import{requireTimeZone as j}from"../run/time-zone.js";import{requireNoSecretInCommand as B}from"./file.js";import{bugDraftFile as q,displayPath as y,displayUnreadableDrafts as J,readStoredDrafts as z,resolveBugCheckout as I,writeJsonFile as A}from"./store.js";export async function recordBug(t){const e=await I(t),{redactedFields:i,report:r}=W(await U(e.cwd,t.file),e.productPath);for(const[o,v]of[["test.file",r.test.file],["fault.path",r.fault.path]])if(c.statSync(h.join(e.repoRoot,v),{throwIfNoEntry:!1})?.isFile()!==!0)throw new Error(`${o} resolves to ${v} from the repository root, and no file is there. Write the path relative to the product directory.`);const{config:d}=await k(e.productDir),n=T(t.vitest??{},e.productDir),u=O(e.productPath,r.test.file),f=await D(e.productDir,d,{plain:!S(u),selection:n}),p=f.problems.at(0)??f.migration;if(p!==null)throw new Error(p);if(!f.project.files.some(o=>o.filePath===u))throw new Error(`test.file resolves to ${r.test.file}, and Testkit does not discover a test there. Run \`testkit discover\` to list the tests it runs.`);N(r.test.file,f.plainFiles.filter(({file:o})=>o===u),n);const s=x(r,{campaign:$(P(e.productDir))?.id??null,productPath:e.productPath,recordedAt:t.at??new Date,recordedCommit:e.headCommit,...t.retries===void 0?{}:{retries:Z(t.retries)},...t.timezone===void 0?{}:{timezone:j(t.timezone)},vitest:V(t.vitest??{},e.productDir,n)});B(s);const l=z(e.productDir),m=l.drafts.filter(o=>C(o.draft,s)),w=m.find(o=>o.draft.filed!==void 0)?.draft.filed;if(w!==void 0)throw new Error(`This test already has a filed bug: ${w.url}. Testkit keeps one bug for each test.`);const a=q(e.productDir,s.id),g=l.unreadable.find(o=>o.file===a);if(g!==void 0)throw new Error(`${y(e,a)} exists and Testkit cannot read it: ${g.problem}. Fix or delete the file, then run this again.`);A(a,s);for(const o of m)o.file!==a&&c.rmSync(o.file,{force:!0});return{draft:s,path:y(e,a),redactedFields:i,replaced:m.length>0,unreadableDrafts:J(e,l.unreadable)}}function V({"vitest-project":t,"vitest-runner":e,"vitest-shuffle":i},r,{config:d}){const n={...e===void 0?{}:{"vitest-runner":e},...d===null?{}:{"vitest-config":E(h.relative(c.realpathSync(r),d))},...t===void 0?{}:{"vitest-project":t},...i===void 0?{}:{"vitest-shuffle":i}};return Object.keys(n).length===0?void 0:n}async function U(t,e){if(e==="-")return{source:"standard input",text:await b(process.stdin)};try{return{source:e,text:c.readFileSync(h.resolve(t,e),"utf8")}}catch(i){throw new Error(`Testkit cannot read the bug record file ${e}.`,{cause:i})}}function W(t,e){let i;try{i=JSON.parse(t.text)}catch(r){throw new Error(`The bug record in ${t.source} is not valid JSON.`,{cause:r})}try{return R(i,e)}catch(r){throw r instanceof F?new Error(`${r.message}
+Run \`testkit bug guide\` to see the record format.`,{cause:r}):r}}function Z(t){if(!/^\d{1,2}$/u.test(t))throw new Error(`--retries takes a whole number from 0 to 99, not ${JSON.stringify(t)}`);return Number(t)}