kwikqa 0.1.0 → 0.2.0

package/LICENSE

@@ -1,6 +1,6 @@
 KwikQA End User License Agreement
 
-Effective 21 October 2026
+Effective 23 October 2026
 Copyright (c) 2026 KwikQA. All rights reserved.
 
 KwikQA is proprietary software. It is not open source, and this agreement does not grant any
@@ -47,8 +47,9 @@ you do not accept, do not install or use the Software.
    https://kwikqa.com/pricing/. Usage is counted per AI request as described there. Paid plans are
    sold by Paddle.com, our merchant of record, renew monthly or yearly until cancelled, and are
    refunded as described at https://kwikqa.com/refunds/, including a full refund of your first
-   payment if you cancel within 14 days of it. We may change plans or prices with at least 30
-   days' notice; a change applies from your next billing period after the notice ends.
+   payment if you cancel within 14 days of it, once in any 12 months. We may change plans or
+   prices with at least 30 days' notice; a change applies from your next billing period after the
+   notice ends.
 
 5. Your data. You keep all rights in your code, test reports, evidence, and the changes the
    Software makes to your code. The Software builds a redacted, size-limited evidence packet on

package/README.md

@@ -13,6 +13,8 @@ npx kwikqa fix
 npx kwikqa validate
 ```
 
+Using pnpm, Yarn, or Bun? See [Package managers and monorepos](#package-managers-and-monorepos).
+
 ## What it will and won't do
 
 - **Diagnoses with evidence.** Every cause it reports cites the report fields, trace events, or source
@@ -34,10 +36,13 @@ npx kwikqa validate
 ## Requirements
 
 - Node.js 20.11 or later
-- Playwright Test, Cypress, or WebdriverIO (Mocha or Jasmine) installed in your project. KwikQA
-  runs your project's own copy and never installs packages or browsers.
+- Playwright Test, Cypress, or WebdriverIO (Mocha or Jasmine) installed in your project with npm,
+  pnpm, Yarn, or Bun. KwikQA runs your project's own copy and never installs packages or browsers.
 - A git repository
 
+For repairs, KwikQA needs one of those three frameworks. It also reads and diagnoses runs from
+[other test frameworks](#other-test-frameworks).
+
 ## Setup
 
 ```sh
@@ -92,12 +97,92 @@ failing specs and records each failure's page (see below).
 
 `login` signs you in with GitHub; the free plan needs no payment details.
 
+### Other test frameworks
+
+KwikQA also reads the reports these write, and diagnoses the run:
+
+- Vitest (`--reporter=junit`);
+- Jest (`jest-junit`);
+- Mocha (`mocha-junit-reporter`);
+- pytest (`--junitxml`);
+- Playwright's own `junit` reporter;
+- JUnit and TestNG through Maven Surefire or Failsafe (`target/surefire-reports`), including
+  Selenium tests;
+- JUnit and TestNG through Gradle (`build/test-results/test`), including the test-retry plugin's
+  retries;
+- TestNG's own `testng-results.xml`;
+- Allure results (`allure-results`) from TestNG, JUnit, Playwright, and pytest;
+- CTRF reports, from Playwright, Jest, pytest, and other tools that write them;
+- Cucumber's JSON report, from cucumber-js (`--format json:<file>`) and Cucumber-JVM (the `json`
+  plugin).
+
+Point `--report` at the file or its directory:
+
+```sh
+npx kwikqa analyze --report reports/junit.xml --html
+```
+
+You get failure groups, a diagnosis, and the [failure report](#failure-report). KwikQA doesn't
+capture, repair, or validate these runs, and `fix` says so before it sends anything. For Playwright
+repairs, use its JSON reporter as shown above.
+
+Some details depend on the reporter:
+
+- For TestNG, `testng-results.xml` keeps retried attempts and failed configuration methods, which
+  Surefire's XML of the same run doesn't correlate. Under Maven, add TestNG's XML reporter to
+  Surefire's configuration, then point `--report` at `test-output`:
+
+  ```xml
+  <properties>
+    <property>
+      <name>listener</name>
+      <value>org.testng.reporters.XMLReporter</value>
+    </property>
+  </properties>
+  ```
+- Allure adds each run's results to its directory, and KwikQA reads the directory as one run, so
+  clean it between runs: `mvn clean`, or pytest's `--clean-alluredir`. Otherwise an earlier run's
+  results read as retries.
+- Cucumber's JSON keeps only each scenario's final attempt, so a scenario that passed on a retry
+  reads as passed. Cucumber-JVM 8 needs a JSON library on the test class path to write it, such as
+  `tools.jackson.core:jackson-databind`.
+- Gradle names a JUnit parameterized test by its display name, which includes its arguments. To
+  keep test data out of KwikQA's titles, set
+  `junit.jupiter.params.displayname.default = {displayName}[{index}]` in
+  `junit-platform.properties`.
+- jest-junit records no test file unless you set `addFileAttribute: true`.
+- pytest records each test's line only with `-o junit_family=xunit1`.
+- JUnit XML from a tool KwikQA doesn't recognize is still read and diagnosed, as an unidentified
+  framework.
+
+### Package managers and monorepos
+
+This page uses npm's commands. With another package manager, install and run KwikQA the same way:
+
+| | Install | Run |
+| --- | --- | --- |
+| npm | `npm i -D kwikqa` | `npx kwikqa analyze` |
+| pnpm | `pnpm add -D kwikqa` | `pnpm exec kwikqa analyze` |
+| Yarn | `yarn add -D kwikqa` | `yarn kwikqa analyze` |
+| Bun | `bun add -d kwikqa` | `bunx kwikqa analyze` |
+
+The commands KwikQA suggests use the same form as the one you ran.
+
+In a monorepo, install KwikQA in the package that holds your tests, and run it from that package's
+folder, where its `kwikqa.config.json` goes. KwikQA finds that package's Playwright, Cypress, or
+WebdriverIO wherever your package manager installed it inside the repository, including the root's
+`node_modules`. It never runs one from outside the repository, so pnpm's global virtual store
+(`enableGlobalVirtualStore`) isn't supported.
+
 ## Commands
 
 | Command | What it does |
 | --- | --- |
 | `npx kwikqa analyze` | Diagnose the failures in the latest report. `--report <path>` picks a report file or a directory of one run's reports; `--framework` names the framework of a JUnit report; `--local` builds the evidence summary without sending anything; `--no-changes` leaves out recent code changes (see below). |
 | `npx kwikqa analyze --capture` | First re-run the failing tests so every failure has full evidence. Playwright runs with full tracing; with Playwright 1.59+ and [Playwright CLI](https://www.npmjs.com/package/@playwright/cli) installed, it also pauses locator failures at the failing line and asks Playwright for locators of the closest elements (`--no-probe` skips this). Cypress and WebdriverIO re-run the failing specs with your own config plus KwikQA's reporter and failure capture, which records each failed attempt's DOM, command log, console, and network. Asks before running anything; `--yes` approves in CI. |
+| `npx kwikqa analyze --html` | Also write the [failure report](#failure-report), `kwikqa-report.html`; `--output <path>` writes it elsewhere. |
+| `npx kwikqa report` | Write the failure report of the latest analysis without analyzing again or sending anything. `--run <id>` picks an earlier run under `.kwikqa/runs/`; `--output <path>` picks the file. |
+| `npx kwikqa ci` | In a CI job: analyze as `analyze --yes --html` does, then publish a summary. On GitHub Actions that is the job summary and one pull-request comment; with your webhooks, also a Slack or Teams message. See [CI](#ci). Without a KwikQA token it analyzes locally. It takes `--report`, `--framework`, `--local`, `--no-changes`, and `--output` as `analyze` does, and `--notify always` to post to chat after every run, not only failed ones. |
 | `npx kwikqa fix` | Propose a repair for a test-side cause, show the diff and validation commands, and apply it if you approve. First it offers to re-run the failing tests once: a test that passes now is flaky, so KwikQA reports it and requests no repair. It starts with the failure group whose test-side diagnosis is strongest; if the model declines, or its proposal fails KwikQA's checks, it asks about the next such group, at most three, each with its own preview. One `fix` counts as one fix however many groups it tries. `--group <n>` repairs only group n, as `analyze` numbered it. `--propose-only` stops after the proposal. |
 | `npx kwikqa validate` | Run the approved validation. The repaired test must pass three times in a row, with retries off. Passing keeps the patch; failing rolls it back. When other tests failed with the same cause, it then offers to re-run them once and names any that still fail, so you can run `fix` again for them; the patch stays either way. |
 | `npx kwikqa fix --retry` | After a failed validation, re-diagnose from the new evidence and propose one revised patch. |
@@ -109,6 +194,27 @@ failing specs and records each failure's page (see below).
 Exit codes: `0` success or an informative result, `1` the repair or validation did not succeed, `2`
 usage error, `3` sign-in, plan, or usage-limit problem.
 
+## Failure report
+
+`npx kwikqa analyze --html`, or `npx kwikqa report` after an analysis, writes one HTML file,
+`kwikqa-report.html`. It shows:
+
+- the run's counts;
+- each failure group, numbered as `analyze` numbered it, with its tests, its owner (test code,
+  application, or environment), the diagnosis, and the evidence the diagnosis cites;
+- flaky tests and failures no rule could classify;
+- what KwikQA could see for each evidence channel, and every request that was sent.
+
+A local analysis (`--local`) gets the same report without owners. Writing a report is free on
+every plan and needs no sign-in. `kwikqa report` rebuilds the evidence from the same test report and
+artifacts. If those have changed since the analysis, it says so instead of showing evidence the
+diagnosis did not see.
+
+The file is self-contained: it loads nothing from the network, and it stays readable when a viewer
+blocks its script and styles, as Jenkins does for archived HTML by default. It holds the same
+redacted evidence a request preview shows, so share it the way you share CI logs. KwikQA only ever
+replaces a file it wrote itself, and `init` offers to add `kwikqa-report.html` to `.gitignore`.
+
 ## Plans
 
 | Plan | Monthly | Annual | Analyses / month | Fixes / month |
@@ -127,7 +233,7 @@ produced for the same run, so it doesn't use another analysis.
 The paid plans are coming soon: until they open, every account uses Free.
 
 Annual billing gives two months free. Cancel within 14 days of your first payment, monthly or
-annual, and it is refunded in full. After that, cancelling an annual plan from the dashboard refunds
+annual, and it is refunded in full, once in any 12 months. After that, cancelling an annual plan from the dashboard refunds
 its unused months; monthly plans stay active until the end of the paid month. Payments go through
 Paddle, the merchant of record.
 
@@ -196,14 +302,81 @@ Run records, transaction journals, captures, and validation output stay on your
 
 ## CI
 
-`analyze` can run unattended; `fix` cannot, because applying a patch always needs a person.
+`analyze` and `ci` run unattended; `fix` doesn't, because applying a patch always needs a person.
+Create a CI token at https://kwikqa.com/dashboard/account/ and store it as the secret
+`KWIKQA_TOKEN`. Running KwikQA with it approves its diagnosis requests, and each request's preview
+is printed to the job's log.
+
+### GitHub Actions
+
+`npx kwikqa ci` analyzes the failed run, writes `kwikqa-report.html`, and publishes a summary: one
+row per failure group with its owner and evidence grade, the flaky tests, and a pointer to the
+report. It holds no error text or evidence; the report does.
+
+```yaml
+permissions:
+  contents: read
+  pull-requests: write # only for the comment
+steps:
+  - uses: actions/checkout@v7
+  - run: npx playwright test
+  - if: failure()
+    run: npx kwikqa ci
+    env:
+      KWIKQA_TOKEN: ${{ secrets.KWIKQA_TOKEN }}
+      GITHUB_TOKEN: ${{ github.token }}
+      # Optional, see "Slack and Microsoft Teams" below
+      KWIKQA_SLACK_WEBHOOK_URL: ${{ secrets.KWIKQA_SLACK_WEBHOOK_URL }}
+      KWIKQA_TEAMS_WEBHOOK_URL: ${{ secrets.KWIKQA_TEAMS_WEBHOOK_URL }}
+  - if: failure()
+    uses: actions/upload-artifact@v7
+    with:
+      name: kwikqa-report
+      path: kwikqa-report.html
+```
+
+- **Job summary.** The summary is shown on the run's page.
+- **Pull-request comment.** On a pull request, the summary is also one comment. A re-run of the
+  same workflow job updates that comment instead of adding another, and a later run with no
+  failures updates it to say so. Each job of a workflow keeps its own comment; jobs of one matrix
+  share one, so analyze a sharded run once, after merging its reports.
+- **Pull requests from forks.** GitHub gives a fork's workflows no secrets and a read-only token.
+  KwikQA analyzes such a run locally, writes the report and the summary, and skips the comment.
+  Don't switch to `pull_request_target` to get the token: it runs the fork's code with your
+  repository's secrets.
+- **Never the cause of a failed build.** A comment or summary that can't be published is reported
+  in the log and skipped; the exit code is the analysis's. Without a KwikQA token, or past your
+  plan's limit, the run is analyzed locally and the summary says why it wasn't diagnosed.
+
+### Slack and Microsoft Teams
+
+`kwikqa ci` posts the run to the incoming webhooks you store as CI secrets. KwikQA keeps no webhook
+anywhere, and never prints one.
+
+- **Slack.** Create an incoming webhook for the channel, and store its URL as
+  `KWIKQA_SLACK_WEBHOOK_URL`.
+- **Teams.** In the channel, add the Workflows template "Post to a channel when a webhook request is
+  received", and store its URL as `KWIKQA_TEAMS_WEBHOOK_URL`. (Office 365 connector webhooks were
+  retired in May 2026.)
+
+The message names the repository and branch, links the pull request or commit and the run, and gives
+the counts. Then it has one line per failure group, with the group's category, where it fails, its
+size, and its owner, and the flaky count. It holds no error text, evidence, or source: a channel is
+a wider audience than a build log. KwikQA posts only when tests failed, unless you pass
+`--notify always`. A failed post is retried once, printed, and never fails the build.
+
+### Other CI systems
+
+`npx kwikqa ci` writes the report there too, and posts to Slack and Teams, but writes no summary or
+comment yet. Or run `analyze` directly:
 
 ```sh
-export KWIKQA_TOKEN=...   # create a CI token at https://kwikqa.com/dashboard/account/
-npx playwright test || npx kwikqa analyze --yes
-npx cypress run || npx kwikqa analyze --yes
+npx playwright test || npx kwikqa analyze --yes --html
+npx cypress run || npx kwikqa analyze --yes --html
 ```
 
+Then keep `kwikqa-report.html` as a build artifact.
+
 ## Account and billing
 
 The dashboard at https://kwikqa.com/dashboard/ shows your plan, usage, and recent requests. There