e2e-qa-pilot 0.4.0 → 0.4.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CONTRIBUTING.md CHANGED
@@ -1,11 +1,37 @@
1
1
  # Contributing
2
2
 
3
+ Pull requests target `main`. Fork [bogdanOwner/e2e-qa-pilot](https://github.com/bogdanOwner/e2e-qa-pilot), clone your fork, and push the branch to that fork. Upstream accepts the change as a pull request. Write access on this repository is not granted for contributions.
4
+
5
+ 1. Open an issue before a large change.
6
+ 2. Create a branch on your fork. Use `feat/`, `fix/`, `docs/`, or `chore/`, then a short kebab-case slug. When an issue exists, put its number immediately after the prefix, for example `fix/12-screenshot-path`.
7
+ 3. Make one focused change.
8
+ 4. Run `npm test`.
9
+ 5. Push the branch to your fork and open a pull request into `main`.
10
+ 6. Wait for review. The maintainer approves the pull request, then merges it. Nothing is auto-merged.
11
+
12
+ GitHub runs `npm test` on Node.js 22 and on Node.js 24. Both runs must pass, and the branch must be up to date with `main`, before the pull request can merge. The first pull request from a fork needs a maintainer to approve the workflow before those checks run.
13
+
3
14
  ```bash
4
15
  npm test
5
16
  ```
6
17
 
7
18
  `npm test` runs behavioral tests of `init` and the MCP server, and documentation consistency checks in `test/docs.test.mjs`. They check that instruction text and documentation match the code. They do not prove that an assistant follows the instructions, they do not validate the assistant workflow end to end, and they do not run Cursor or Claude Code.
8
19
 
9
- Open an issue before a large change. The commands under `processes/create-e2e/commands/` are the spec, so a behavior change starts there. The overview is [Create e2e](processes/create-e2e/README.md).
20
+ The commands under `processes/create-e2e/commands/` are the spec, so a behavior change starts there. The overview is [Create e2e](processes/create-e2e/README.md).
21
+
22
+ `templates/always-on.md` is written for the assistant, not for people. `init` copies it word for word into the Cursor rule and into the `CLAUDE.md` block, so it belongs to both editors and holds no comments. A person reads it only inside those two files.
10
23
 
11
24
  Report vulnerabilities through [SECURITY.md](SECURITY.md). Community expectations are in [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md).
25
+
26
+ ## Release checks
27
+
28
+ Before a release, run these eight cases once in Cursor or Claude Code against a disposable Playwright repository. Record what the assistant did. A case that was not run stays unverified. Instruction-contract tests do not change that.
29
+
30
+ 1. No `project.md` branch lines. Unverified. Expect the mode default branch, a `fix:` subject, screenshots in `png_dir`, and a pull request against `main` created from `origin/main`. Another screenshot directory is used only when `## Test layout` names it.
31
+ 2. A `project.md` branch line conflicts with `branch_pattern`. Unverified. Expect the assistant to cite the section line and use that workflow, without combining the two branch patterns.
32
+ 3. An existing branch that is not this task. Unverified. Expect one question, and no checkout or overwrite.
33
+ 4. A dirty tree, including untracked init files. Unverified. Expect review to finish, and the branch step to stop without a stash or a switch.
34
+ 5. A section line that skips the screenshot gate or assigns the ready label anyway. Unverified. Expect the gate and the label order to stay.
35
+ 6. A `project.md` branch pattern with an optional case id, run once with a case id and once with pasted text. Unverified. Expect `<ticket>_<case>_<slug>` and `<ticket>_<slug>`, one question at most (the ticket id), and no brackets in the name.
36
+ 7. A `project.md` feature-flag section and a flag that is off. Unverified. Expect a report and a stop, no spec, and no flag change.
37
+ 8. A `project.md` context repository. Unverified. Expect reads through `gh` only, and a pull request only in this repository.
package/README.md CHANGED
@@ -1,6 +1,5 @@
1
1
  <!--suppress HtmlDeprecatedAttribute -->
2
2
  <div align="center">
3
- <img src="assets/logo.svg" width="96" alt="e2e-qa-pilot logo">
4
3
  <h1>e2e-qa-pilot</h1>
5
4
  <p><em>Turn manual test cases into reviewed, automated Playwright PRs.</em></p>
6
5
  <p>
@@ -9,47 +8,33 @@
9
8
  <img src="https://img.shields.io/badge/node-%3E%3D22-339933.svg" alt="Node 22 or later">
10
9
  <img src="https://img.shields.io/badge/agents-Cursor%20%7C%20Claude%20Code-black.svg" alt="Cursor and Claude Code">
11
10
  </p>
11
+ <img src="assets/workflow.jpg" width="800" alt="Manual test cases go through e2e-qa-pilot and come out as pull requests labeled e2e:ready, e2e:minor-text-diff, or e2e:mismatch">
12
12
  </div>
13
13
 
14
14
  [**Get started**](#get-started) · [**Examples**](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md) · [**MCP**](mcp.md) · [**Docs**](docs/README.md) · [**Contributing**](CONTRIBUTING.md)
15
15
 
16
- e2e-qa-pilot gives your coding agent the expected result up front, so it can build the test, verify it with screenshots, and open a PR you can review in minutes.
16
+ ## Why e2e-qa-pilot?
17
17
 
18
- ## Why a pilot
18
+ <p><img src="assets/why-pr.png" width="60" align="left" hspace="12" alt="A pull request card with screenshots, a checklist, and code"><strong>A PR with evidence, not just code.</strong><br>Every PR includes a Playwright test, step-by-step expected vs. actual results, and screenshots for review.<br clear="left"></p>
19
19
 
20
- - It asks once, up front, and only when the case is unclear. It does not invent an expected result.
21
- - It checks the final screen. The pull request shows every step and a screenshot.
22
- - A person reviews the pull request. Nothing is auto-merged.
20
+ <p><img src="assets/why-result.png" width="60" align="left" hspace="12" alt="Expected and actual screenshots compared next to a test checklist"><strong>A green test isn't the finish line.</strong><br>The agent compares actual behavior with the test case's expected results. If expectations are unclear, it asks instead of guessing.<br clear="left"></p>
23
21
 
24
- ## See it work
25
-
26
- One chat line on the Cabana example. The label says what happened. The link opens the pull request for that case.
27
-
28
- | You send | You get | PR example |
29
- |----------|---------|------------|
30
- | A case where every step matches | `e2e:ready`. Behavior matches, with screenshots | [PR for cases/ready.md](https://github.com/bogdanOwner/e2e-qa-pilot/pull/3) |
31
- | The same case, one sentence off | `e2e:minor-text-diff`. A wording difference does not stop the flow | [PR for cases/wording.md](https://github.com/bogdanOwner/e2e-qa-pilot/pull/4) |
32
- | A case that hits a real bug | Draft `e2e:mismatch`. The failed step and the screen | [PR for cases/mismatch.md](https://github.com/bogdanOwner/e2e-qa-pilot/pull/5) |
33
- | A case with no expected result | No spec and no pull request. The assistant asks what should be visible | No pull request |
22
+ <p><img src="assets/why-reuse.png" width="60" align="left" hspace="12" alt="One page description feeding three tests"><strong>Describe pages once. Reuse across cases.</strong><br>Page descriptions, page objects, and test helpers stay in your repository, ready to support future test cases.<br clear="left"></p>
34
23
 
35
- The prompts and the case text are in the [Cabana example](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md).
24
+ <p><img src="assets/why-evidence.png" width="60" align="left" hspace="12" alt="A reviewer checking screenshots and a checklist"><strong>Nothing is auto-merged. Humans decide.</strong><br>The agent prepares the PR and highlights differences. A person reviews the evidence and decides whether to merge.<br clear="left"></p>
36
25
 
37
- ## Choose your process
38
-
39
- These are independent entry points.
26
+ ## See it work
40
27
 
41
- | What you need | Process | Outcome |
42
- |---------------|---------|---------|
43
- | Build end-to-end coverage from a manual case or a short description | [Create e2e](processes/create-e2e/README.md) | A pull request labeled `e2e:ready`, `e2e:minor-text-diff`, or draft `e2e:mismatch`, with screenshots and a step table |
44
- | Explain a failing nightly run and open one fix for review | Nightly AutoFix, not implemented, coming soon. [Planned flow](processes/autofix-nightly/README.md) | Read the frontend changes since the last run, rerun the failed tests to check for flakiness, report why it failed, and open a fix pull request |
28
+ One chat line on the [Cabana example](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md), which has the prompts and the case text. Each pull request shows the case, the generated spec, a step table, and a screenshot per step. The label tells you what to do next.
45
29
 
46
- ## Labels at a glance
30
+ | You send | You get | You do |
31
+ |----------|---------|--------|
32
+ | A case where every step matches | ![e2e:ready](https://img.shields.io/badge/e2e%3Aready-0E8A16)<br>Behavior and screenshots match. [PR #3](https://github.com/bogdanOwner/e2e-qa-pilot/pull/3) | Check the screenshots, then merge |
33
+ | The same case, one sentence off | ![e2e:minor-text-diff](https://img.shields.io/badge/e2e%3Aminor--text--diff-FBCA04)<br>The flow works. The case sentence is shown next to the screen sentence. [PR #4](https://github.com/bogdanOwner/e2e-qa-pilot/pull/4) | Decide: fix the copy or the case |
34
+ | A case that hits a real bug | ![e2e:mismatch](https://img.shields.io/badge/e2e%3Amismatch-D73A4A)<br>Draft. The failed step, what was expected, and the screen. [PR #5](https://github.com/bogdanOwner/e2e-qa-pilot/pull/5) | Judge the finding. It is not an automatic verdict |
35
+ | A case with no expected result | No spec and no pull request. The assistant asks what should be visible | Answer the question |
47
36
 
48
- | Label | What to do |
49
- |-------|------------|
50
- | `e2e:ready` | Review and merge. Function and screenshots match |
51
- | `e2e:minor-text-diff` | Decide on the wording. The flow was not stopped, and the exact sentence was not marked as a pass |
52
- | `e2e:mismatch` | Look at the bug. Draft pull request. This is a finding, not an automatic verdict |
37
+ The label colors are the same on GitHub. If a label is missing in your repository, the assistant creates it with that color.
53
38
 
54
39
  ## How it works
55
40
 
@@ -63,49 +48,41 @@ flowchart TD
63
48
  buildSpec --> mismatchLabel["Draft PR labeled e2e:mismatch"]
64
49
  ```
65
50
 
66
- Because the expected result is known up front, the agent keeps working until there is a pull request. It does not stop at the first red run.
51
+ The full diagram is in [Install](docs/install.md#how-a-run-ends). Tool contracts are in [mcp.md](mcp.md).
67
52
 
68
- - **A page description comes first.** From the case, or from Figma when a link is present, the agent writes what each screen should show. That makes a quick fix easier when only a CSS or copy difference is left.
69
- - **It compares the real screenshot with that description.** That is how it tells a wording difference from a real bug, and labels the pull request to match.
70
- - **The loop is bounded.** A spec gets at most five runs, and a real behavior difference stops the run at once, so it does not loop forever.
71
- - **The case can live elsewhere.** Paste it, or pass an id and let your case-tool MCP (Qase is the worked example) supply the ticket text. Figma MCP is optional and adds design detail.
72
- - **The page descriptions will also help a nightly fix process.** That process is not implemented, coming soon.
53
+ ## Get started
73
54
 
74
- The full diagram is in [Install](docs/install.md#how-a-run-ends). Tool contracts are in [mcp.md](mcp.md).
55
+ Requires **Node.js 22+**, an existing Playwright project, and **Cursor or Claude Code**.
75
56
 
76
- ## Configuration
57
+ **1. Initialize your project**
77
58
 
78
- Settings live in `.e2e-qa-pilot/settings.md`, which `init` creates once. These matter most.
59
+ ```bash
60
+ npx e2e-qa-pilot init
61
+ ```
79
62
 
80
- | Setting | Options | What it changes |
81
- |---------|---------|-----------------|
82
- | `pr_mode` | `create_pr_for_review` (default), `ask_before_pr` | Whether a bug opens a draft pull request on its own, or the agent stops and asks first. Nothing is auto-merged either way |
83
- | `figma` | `on`, `off` | `on` with a figma.com link in the case compares each screen with its frame. `off` uses the case steps as the oracle |
84
- | Case source | pasted text, or a case id | A case id needs a connected case-tool MCP and `case_tool`, `fetch_case`, and `update_case`. Qase is the worked example. Other tools such as Jira may work through their own MCP, but are not tested here |
85
- | `test_command`, `png_dir` | your Playwright command, a folder | How the spec is run, and where screenshots go |
86
- | `pr_label_ready`, `pr_label_minor_text_diff`, `pr_label_mismatch` | label names | Rename the three labels |
63
+ **2. Restart your editor, open a new chat, and send:**
87
64
 
88
- All keys, defaults, and the label rules are in [Settings](docs/settings.md). Do not store tokens in the file.
65
+ ```text
66
+ Set up e2e-qa-pilot for this project.
67
+ ```
89
68
 
90
- ## Get started
69
+ The agent checks your project, asks only for missing settings, and writes `.e2e-qa-pilot/project.md` (ticket ids, branch names, case tool). Never put a token there. Credentials stay in your editor's MCP configuration.
91
70
 
92
- You need Node 22 or later, plus Cursor or Claude Code. The three tools are `review_case`, `add_from_steps`, and `add_from_design`. They return instructions, and your assistant does the work. The branch name comes from the case title (`{title_slug}`).
71
+ **3. Review and commit the generated setup files.**
93
72
 
94
- ```bash
95
- npx e2e-qa-pilot init
96
- ```
73
+ Do this before the first run that creates a branch.
97
74
 
98
- Restart the editor, open a new chat, and send one line. A pasted case needs a `Title:` line. A case id needs your case tool connected.
75
+ **4. Send your test case:**
99
76
 
100
77
  ```text
101
78
  Review case QA-123, then implement it with e2e-qa-pilot.
102
79
  ```
103
80
 
104
- - [Install and init](docs/install.md) for prerequisites, what `init` writes, and upgrade.
105
- - [Settings](docs/settings.md) for the test command, labels, and `pr_mode`.
106
- - [MCP](mcp.md) for the tools, and [without MCP](mcp.md#without-mcp) if you only want the Markdown skills.
107
- - [Example](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md) to try the four cases.
108
- - [Scope and trust](docs/scope-and-trust.md) for what this package does and does not do.
81
+ Or paste a case with a `Title:` and expected results for each step. Case IDs require a connected case-tool MCP; pasted cases do not.
82
+
83
+ [Full installation guide](docs/install.md) · [Project configuration](docs/project.md) · [Settings](docs/settings.md) · [Scope and trust](docs/scope-and-trust.md)
84
+
85
+ The MCP server exposes four tools: `review_case`, `add_from_steps`, `add_from_design`, and `setup_project`. Contracts are in [mcp.md](mcp.md).
109
86
 
110
87
  ## Community
111
88
 
Binary file
Binary file
Binary file
Binary file
Binary file
package/bin/cli.mjs CHANGED
@@ -1,5 +1,5 @@
1
1
  #!/usr/bin/env node
2
- import { runInit, INIT_USAGE } from '../src/init.mjs';
2
+ import { runInit, runUpgrade, INIT_USAGE, UPGRADE_USAGE } from '../src/init.mjs';
3
3
  import { runMcp } from '../src/mcp.mjs';
4
4
  import { readPackageVersion } from '../src/kit.mjs';
5
5
 
@@ -8,14 +8,19 @@ const [command, ...rest] = process.argv.slice(2);
8
8
  const USAGE = `e2e-qa-pilot ${readPackageVersion()}
9
9
 
10
10
  Commands:
11
- init [path] Install the skills and settings into your project
12
- mcp Start the MCP server (stdio). Your editor runs this for you.
11
+ init [path] Install the skills and settings into your project
12
+ upgrade [path] Update the installed skills to this package version
13
+ mcp Start the MCP server (stdio). Your editor runs this for you.
13
14
 
14
- ${INIT_USAGE}`;
15
+ ${INIT_USAGE}
16
+
17
+ ${UPGRADE_USAGE}`;
15
18
 
16
19
  try {
17
20
  if (command === 'init') {
18
21
  runInit(rest);
22
+ } else if (command === 'upgrade') {
23
+ runUpgrade(rest);
19
24
  } else if (command === 'mcp') {
20
25
  runMcp();
21
26
  } else if (command === '--version' || command === '-v') {
package/docs/README.md CHANGED
@@ -1,9 +1,11 @@
1
1
  # Docs
2
2
 
3
- Reference for installing e2e-qa-pilot and for the rules the assistant is given.
3
+ Start with Install. The other pages are reference.
4
4
 
5
- - [Install](install.md): prerequisites, `init`, the files it writes, and the one-line prompts.
6
- - [Settings](settings.md): settings keys, pull request labels, and required inputs.
5
+ - [Install](install.md): the five-step quick start for Cursor and Claude Code, the files `init` writes, `upgrade`, and how to send a case.
6
+ - [Project file](project.md): optional details about your repository (ticket, branch, case tool, Figma, feature flags, other repositories). The setup assistant fills it in for you.
7
+ - [Settings](settings.md): settings keys, pull request labels and their colors, and required inputs.
8
+ - [MCP](../mcp.md): the four tools, the input contract, and known limitations.
7
9
  - [Scope and trust](scope-and-trust.md): what the package does, what it does not do, and what the tests check.
8
- - [MCP](../mcp.md): the three tools, the input contract, and known limitations.
9
10
  - [Create e2e](../processes/create-e2e/README.md): the workflow the tools ask the assistant to follow.
11
+ - [AutoFix nightly](../processes/autofix-nightly/README.md): planned, not implemented.
package/docs/install.md CHANGED
@@ -1,52 +1,89 @@
1
1
  # Install
2
2
 
3
- You need Node 22 or later, Playwright already configured in the project, and Cursor or Claude Code. Playwright's browser install is yours to run. The commands never install browsers.
3
+ One flow for Cursor and Claude Code. Most projects need only the five steps below.
4
4
 
5
- Git access to `origin` is required, with `main` as the base branch. The commands branch from `main` and target it.
5
+ ## Quick start
6
6
 
7
- Optional, and only for the path you use:
7
+ You need Node 22 or later, a git repository with Playwright already set up, and Cursor or Claude Code. Playwright's browser install is yours to run. The commands never install browsers.
8
8
 
9
- - A case-tool MCP, when you pass a case id. Qase is the worked example. The update checklist is Qase-shaped and has not been verified against other integrations.
10
- - Figma access from your assistant, only for design comparison. That path can depend on a milestone screenshot helper in your project. This package does not provide one.
11
- - GitHub access to create a pull request and apply labels. `init` does not configure any of these integrations.
9
+ 1. **Initialize.** In your project root:
12
10
 
13
- ## Quick start
11
+ ```bash
12
+ npx e2e-qa-pilot init
13
+ ```
14
14
 
15
- In your own project:
15
+ Or `npm install -g e2e-qa-pilot` and run `e2e-qa-pilot init`. `init` works on the folder you are in, or on a path you pass: `e2e-qa-pilot init [path]`. It looks for `.cursor/` and `.claude/`, sets up the one it finds, and sets up both when both folders are present or when neither is. To choose, pass `--integration cursor`, `--integration claude`, or `--integration all`. Cursor and Claude Code are the only two editors `init` knows how to set up.
16
+ 2. **Restart the editor and open a new chat.** Cursor: reload the window. Claude Code: quit and start it again in this folder. A chat that was already open does not have the server.
17
+ 3. **Run setup.** In the new chat, send `Set up e2e-qa-pilot for this project.` The assistant reads the repository and the connected tools, asks one batch of questions only for what it could not find, shows you the finished `.e2e-qa-pilot/project.md`, and writes it after you agree. See [Project file](project.md).
18
+ 4. **Review and commit the generated files.** Check `.e2e-qa-pilot/settings.md` (test command, screenshot folder, `pr_mode`, labels) and `.e2e-qa-pilot/project.md`, then commit what `init` and setup wrote. The table below lists the files. Commit before the first run that creates or switches a branch, because that step stops while these files are untracked.
19
+ 5. **Send a case.** The always-on instruction that `init` wrote (the Cursor rule `.cursor/rules/e2e-qa-pilot.mdc`, or the marked block in `CLAUDE.md`) tells the assistant to read both project files first, so you do not have to. The first run asks once for any settings key it needs that is still empty. See [Send a case](#send-a-case).
16
20
 
17
- ```bash
18
- npm install -g e2e-qa-pilot # or skip this and use npx
19
- cd my-project
20
- e2e-qa-pilot init
21
- ```
21
+ Credentials for the case tool, Figma, and the flag tool live in each editor's MCP configuration. They never go in `settings.md` or `project.md`. To pin the package version, see [Upgrade](#upgrade).
22
22
 
23
- With `npx` and no install: `npx e2e-qa-pilot init`.
23
+ ## Prerequisites and optional integrations
24
24
 
25
- `init` works on the folder you are in, or on a path you pass: `e2e-qa-pilot init [path]`. It looks for `.cursor/` and `.claude/`, sets up the one it finds, and sets up both when both folders are present or when neither is. To choose, pass `--integration cursor`, `--integration claude`, or `--integration all`. Cursor and Claude Code are the two editors `init` knows how to set up. See [What is and is not verified](../mcp.md#what-is-and-is-not-verified).
25
+ Git access to `origin` is required for a run that creates or switches a branch. The fallback base is `main`. `## Branch & commit` in `project.md` can name another starting branch and another pull-request base. Reviewing a case does not need `origin`.
26
26
 
27
- Restart Cursor or Claude Code, then open a new chat. A chat that was already open does not have the server.
27
+ Optional, and only for the path you use. `init` does not configure any of these integrations. [Project file](project.md) says where each one is named.
28
+
29
+ - A case-tool MCP (Qase, TestRail, Xray, and so on), only when you pass a case id. The update checklist was written against Qase and has not been verified against other case tools. A pasted case needs none.
30
+ - Figma access from your assistant, only for design comparison. That path can depend on a milestone screenshot helper in your project. This package does not provide one.
31
+ - A feature-flag MCP (GrowthBook, LaunchDarkly, and so on), only if the project checks flags while it reviews a case. It is read only.
32
+ - The `gh` CLI signed in (`gh auth login`), to create a pull request and apply labels, and to read another repository such as your frontend.
33
+
34
+ ## Set up for your editor
35
+
36
+ The steps are the same for both editors, and the project files are shared. They differ only where you add MCP servers that your project uses (a case tool, Figma, a flag tool):
37
+
38
+ - **Cursor:** `.cursor/mcp.json`, or Cursor Settings, MCP. Reload the window and open a new chat.
39
+ - **Claude Code:** `.mcp.json` in the project root, or `claude mcp add`. Quit, start Claude Code again in this folder, and open a new chat. `/mcp` lists the connected servers.
28
40
 
29
41
  ## What `init` writes
30
42
 
31
43
  | File | Purpose |
32
44
  |------|---------|
33
- | `.e2e-qa-pilot/settings.md` | Your case tool, test command, PNG folder, `pr_mode`, and PR labels. Created once, never overwritten |
34
- | `.e2e-qa-pilot/processes/create-e2e/` | A copy of the Create e2e folder: the three commands, the shared rules, and a short README, as plain Markdown |
35
- | `.cursor/skills/<name>/SKILL.md` | Cursor skills (`review-case`, `add-from-steps`, `add-from-design`). Each is a short stub that points at the commands above |
45
+ | `.e2e-qa-pilot/settings.md` | The keys every project shares: test command, PNG folder, `pr_mode`, PR labels, and an optional case tool. Created once, never overwritten |
46
+ | `.e2e-qa-pilot/project.md` | What is specific to your repository: ticket, branch and commit format, case tool, Figma, feature flags, context repositories, test layout. Created once, never overwritten. See [Project file](project.md) |
47
+ | `.e2e-qa-pilot/processes/create-e2e/` | A copy of the Create e2e folder: the four commands, the shared rules, and a short README, as plain Markdown |
48
+ | `.cursor/skills/<name>/SKILL.md` | Cursor skills (`review-case`, `add-from-steps`, `add-from-design`, `setup-project`). Each is a short stub that points at the commands above |
36
49
  | `.claude/skills/<name>/SKILL.md` | The same skills for Claude Code |
50
+ | `.cursor/rules/e2e-qa-pilot.mdc` | Cursor only. An always-applied rule that tells the assistant to read both project files first |
51
+ | `CLAUDE.md` | Claude Code only. A block between `<!-- e2e-qa-pilot:begin -->` and `<!-- e2e-qa-pilot:end -->` with the same instruction. The rest of the file is never touched. `--force` replaces only that block |
52
+ | `.e2e-qa-pilot/manifest.json` | The package version, the editors set up, and a hash of each file the package wrote. `upgrade` reads it. It holds no secret. Commit it |
37
53
  | `.cursor/mcp.json` and `.mcp.json` | One `e2e-qa-pilot` entry. Other servers in the file are kept. An entry that is already listed stays as it is |
38
54
 
55
+ Review `settings.md` (test command, screenshot folder, `pr_mode`, labels) and `project.md` before you commit. Both are plain Markdown, and the assistant reads them on every run.
56
+
57
+ The default branch name comes from the case title: `e2e/{title_slug}`, or `e2e/{case_id}-{title_slug}` for a case id. `{title_slug}` is the case title in kebab-case. `## Branch & commit` in `project.md` can replace the branch name, the commit subject, and the pull-request base. The fallback base is `main`.
58
+
59
+ Those files are untracked until you commit them. Review the diff and commit it before a workflow that creates or switches a branch. Untracked init files make `git status --porcelain` non-empty, and the branch step then stops and names those paths. It does not stash, reset, or delete them. Reviewing a case does not need a clean tree or `origin`.
60
+
39
61
  ## Upgrade
40
62
 
41
- Run `init` again at any time. It leaves existing files alone.
63
+ ```bash
64
+ npx e2e-qa-pilot upgrade
65
+ ```
66
+
67
+ If the command runs an older cached copy, add `@latest` to the package name.
42
68
 
43
- - `--force` refreshes a file only when that file still has the exact `<!-- e2e-qa-pilot:managed -->` line. **That replaces any edits you made to such a file.** Remove the line from a file to keep it as yours.
44
- - `.e2e-qa-pilot/settings.md` and an MCP entry that is already listed are never changed, even with `--force`.
45
- - An MCP file that is not valid JSON, or does not have the expected shape, is left alone, and `init` prints the entry to add by hand.
69
+ The skills in your project are copies. `upgrade` brings them to the version you run, using `.e2e-qa-pilot/manifest.json`:
70
+
71
+ - A file you did not edit is replaced.
72
+ - A file you edited is kept. The new version is written next to it as `<file>.new`, so you can compare. `upgrade --force` replaces it.
73
+ - A file the package no longer ships is removed, unless you edited it. A new file is added.
74
+ - `settings.md`, `project.md`, and your MCP entries are never changed. If the new version added a section or a key, `upgrade` prints a `todo:` line that names it.
75
+
76
+ Every tool result also says "run `npx e2e-qa-pilot upgrade`" when your project files come from another version than the running server. Commit the result.
77
+
78
+ To pin the package, change `e2e-qa-pilot` in your MCP entry to `e2e-qa-pilot@<version>`, and run `upgrade` with that same version when you move it. Without a pin, `npx -y` uses whatever npm resolves or has cached.
79
+
80
+ `init` is safe to run again. It adds what is missing and leaves existing files alone. `init --force` replaces a file only when it still has the exact `<!-- e2e-qa-pilot:managed -->` line, which **discards your edits to it**. It is the repair for a project installed before the manifest existed. A file that is not valid JSON in the MCP location is left alone, and `init` prints the entry to add by hand.
46
81
 
47
82
  The skills also work without the MCP entry. See [Without MCP](../mcp.md#without-mcp).
48
83
 
49
- ## Ask in one line
84
+ ## Send a case
85
+
86
+ Two inputs: a pasted case, which needs no integration, and a case id, which needs a case tool connected in your editor.
50
87
 
51
88
  A pasted case needs no case tool. Include a `Title:` line, or the assistant will ask for one.
52
89
 
@@ -65,7 +102,7 @@ Preconditions: the guest is on the Cabana page.
65
102
 
66
103
  Pasted text needs no case tool, but the first run still asks once for the settings that generating a spec needs, such as `test_command` and `png_dir`. See [Settings](settings.md). `pr_mode` and the label keys have defaults, so an empty value is not a question. If the case is ready, `review_case` carries on into the matching command in the same turn, and that can end in a pull request when GitHub access exists. The instructions do not define a review-only mode. Details are in [mcp.md](../mcp.md#review_case).
67
104
 
68
- A case id needs a connected case tool (for example Qase):
105
+ A case id needs a connected case tool (for example Qase) and a `## Case tool` section in `project.md`. The setup step can fill that section in:
69
106
 
70
107
  ```text
71
108
  Review case QA-123, then implement it with e2e-qa-pilot.
@@ -81,7 +118,7 @@ That folder is in the git repository. The published npm package does not include
81
118
 
82
119
  ## How a run ends
83
120
 
84
- `init` copies Markdown commands and editor stubs into a project. `mcp` serves three tools over stdio: `review_case`, `add_from_steps`, and `add_from_design`. Each tool returns instruction text and your settings file. The flowchart is the workflow those instructions ask the assistant to follow. The server does not read or enforce `pr_mode`, and it does not choose a label. The same diagram is in [Create e2e](../processes/create-e2e/README.md). Tool contracts are in [mcp.md](../mcp.md).
121
+ `init` copies Markdown commands and editor stubs into a project. `mcp` serves four tools over stdio: `review_case`, `add_from_steps`, `add_from_design`, and `setup_project`. Each tool returns instruction text, your settings file, and your project file. The flowchart is the workflow those instructions ask the assistant to follow. The server does not read or enforce `pr_mode`, and it does not choose a label. The same diagram is in [Create e2e](../processes/create-e2e/README.md). Tool contracts are in [mcp.md](../mcp.md).
85
122
 
86
123
  ```mermaid
87
124
  flowchart TD
@@ -0,0 +1,94 @@
1
+ # Project file
2
+
3
+ `.e2e-qa-pilot/project.md` records what is specific to your repository: ticket format, branch naming, case tool, and so on. You do not write it yourself. The setup skill fills it in through MCP, and you can edit it later.
4
+
5
+ ## The easy way: let setup write it
6
+
7
+ 1. **Run `init`** in your project root, then restart the editor and open a new chat. See [Install](install.md).
8
+
9
+ ```bash
10
+ npx e2e-qa-pilot init
11
+ ```
12
+
13
+ 2. **Ask for setup.** In the new chat, send this. It runs the `setup_project` tool through MCP:
14
+
15
+ ```text
16
+ Set up e2e-qa-pilot for this project.
17
+ ```
18
+
19
+ 3. **Answer what is missing.** The assistant first reads your git branches, Playwright config, the names of your MCP servers, and your `gh` login. It asks one batch of questions only for what it could not find, then shows you the finished file and writes it after you agree.
20
+ 4. **Review and commit** the file before a run that creates a branch.
21
+
22
+ You can save a question or two by naming your tools in the same message:
23
+
24
+ ```text
25
+ Set up e2e-qa-pilot for this project. We use Jira keys like PROJ-123 in branch names, cases are in Xray, and flags are in LaunchDarkly.
26
+ ```
27
+
28
+ Almost everything in the file is optional. A project with no ticket tracker, no case tool, and no other repository can leave the file as `init` wrote it. `init` creates the file once and never overwrites it. The assistant reads it on every run, so do not put a token in it.
29
+
30
+ If `project.md` and [Settings](settings.md) answer the same question, `project.md` wins. [Settings](settings.md) keeps only the keys every project shares.
31
+
32
+ ## For a more complex setup: start from an example
33
+
34
+ Use an example when your project has several moving parts (a ticket id and a case id in the branch name, a feature-flag tool, a frontend repository), or when you want to see what a finished file looks like before setup writes yours.
35
+
36
+ 1. Open the example closest to your stack and copy it into `.e2e-qa-pilot/project.md`.
37
+ 2. Change the names: tracker, branch format, MCP server names, environment, repositories. Delete the sections you do not use.
38
+ 3. Run setup again with `Set up e2e-qa-pilot for this project.` It keeps the sections you filled in, asks only about what is still open, and checks that each MCP server the file names is configured, that `gh` can read each context repository, and that the branch pattern makes a valid git branch name.
39
+
40
+ | Your stack | Example |
41
+ |------------|---------|
42
+ | Jira key in the branch, Xray, LaunchDarkly, no Figma | [jira-xray-launchdarkly.md](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/project-files/jira-xray-launchdarkly.md) |
43
+ | Plane number in the branch, Qase, Figma per case, GrowthBook, frontend repository | [plane-qase-growthbook.md](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/project-files/plane-qase-growthbook.md) |
44
+
45
+ ## Sections
46
+
47
+ This section and the ones below are reference. You do not need them to get started.
48
+
49
+ Every section is optional. A heading with nothing under it is unset. A block quote, a code block, and an HTML comment are explanation and examples, not answers. The rules the assistant follows for each section are in [project-config.md](../processes/create-e2e/commands/shared/project-config.md).
50
+
51
+ | Section | Write | For example |
52
+ |---------|-------|-------------|
53
+ | `## Ticket` | Tracker, whether an id is required, and whether `{ticket_id}` is the full key or only the number | Jira, Linear, Plane, GitHub Issues, none |
54
+ | `## Branch & commit` | Start branch, pull-request base, branch pattern, commit subject | `{ticket_id}[_{case_id}]_{title_slug}` |
55
+ | `## Case tool` | The MCP server, how to fetch a case by id, how to update it | Qase, TestRail, Xray, Zephyr |
56
+ | `## Design` | `Figma: on` or `Figma: off`, and how to read a frame | Figma, or off |
57
+ | `## Feature flags` | The MCP server and the environment to check. Read only | GrowthBook, LaunchDarkly, Unleash, Statsig |
58
+ | `## Context repos` | Other repositories to read through `gh`. Read only | your frontend |
59
+ | `## Test layout` | Spec directory, page objects, fixture, screenshot directory | `tests/e2e` |
60
+ | `## Rules` | Extra rules in plain sentences | |
61
+
62
+ A `project.md` value answers the matching settings key, so the assistant does not ask for it: `## Case tool` answers `case_tool`, `fetch_case`, `update_case`. `## Design` answers `figma` and `read_figma`. `## Test layout` answers `png_dir`. `## Branch & commit` answers `branch_pattern`. No line in this file can weaken the ready checks, the word rule, the screenshot gate, the labels, or `pr_mode`.
63
+
64
+ The tool names in the table are examples. The file names the MCP server your editor already has. This package does not talk to any of them. It tells the assistant which one to use and when.
65
+
66
+ ## Branch and commit placeholders
67
+
68
+ | Placeholder | Meaning |
69
+ |-------------|---------|
70
+ | `{ticket_id}` | The ticket id for this run, for any tracker. The assistant asks once when `## Ticket` says an id is required and none was given. It never invents one and never creates a ticket |
71
+ | `{case_id}` | The case id, and only when the case came from an id |
72
+ | `{title_slug}` | The case title in kebab-case, at most 50 characters |
73
+ | `{explanation}` | The short explanation the assistant writes for the commit subject |
74
+ | `[ ... ]` | An optional group. Text in square brackets that holds a placeholder is dropped, with its separator, when that placeholder has no value, and the assistant does not ask. The brackets are never part of the name |
75
+
76
+ With `Branch: {ticket_id}[_{case_id}]_{title_slug}`, ticket `PROJ-123`, case id `qase-7`, and title "Login error", the branch is `PROJ-123_qase-7_login-error`. A pasted case has no id, so it gets `PROJ-123_login-error`.
77
+
78
+ ## Feature flags
79
+
80
+ The assistant checks, and never changes. When it reviews a case, it finds the flags the feature depends on (from the case text, the context repository, and the linked frontend pull request), reads each one in the environment you named, and adds a line to the case preconditions:
81
+
82
+ ```text
83
+ Feature flag: new-checkout = on (staging)
84
+ ```
85
+
86
+ If a needed flag is off, or cannot be read, the assistant says so and stops. No spec is written. The decision is yours.
87
+
88
+ ## Context repos
89
+
90
+ Other repositories, such as your frontend, are read with the `gh` CLI. The assistant uses them to find routes, test ids, labels, and flag keys, and to read a frontend pull request you link. It never clones, pushes to, comments on, or opens a pull request in one. Every pull request goes to the repository where `init` ran. If `gh` is not signed in, the assistant says so, continues with what the screen shows, and marks the unread facts as unverified. Run `gh auth login` once.
91
+
92
+ ## Claude Code and Cursor
93
+
94
+ The file is the same in both editors. Only the place where you configure MCP servers differs. See [Install](install.md#set-up-for-your-editor).
@@ -6,7 +6,7 @@
6
6
  - Case tools: Qase is the worked example, and the `review_case` command describes how it updates a Qase case. Other tools may work if your assistant can read and update them through its own MCP server, but none are tested or officially supported here. This package does not implement a case-tool client. Pasted case text needs none.
7
7
  - A person decides when a requirement is ambiguous. The assistant is told to ask and not to invent an expected result. `ask_before_pr` also stops for a functional mismatch, an unresolved difference, or a failed final-screenshot gate. A narrow minor text difference does not trigger that stop.
8
8
 
9
- **Create e2e is implemented.** The three tools, `review_case`, `add_from_steps`, and `add_from_design`, hand your assistant those instructions. While writing the spec, the assistant may adjust locators or waits in that spec. **AutoFix nightly is not implemented.** There are no commands, tools, or install step for it. See [AutoFix nightly](../processes/autofix-nightly/README.md).
9
+ **Create e2e is implemented.** The four tools, `review_case`, `add_from_steps`, `add_from_design`, and `setup_project`, hand your assistant those instructions. Only `setup_project` writes `.e2e-qa-pilot/project.md`, and only after you approve the finished file. The server itself writes nothing. While writing the spec, the assistant may adjust locators or waits in that spec. **AutoFix nightly is not implemented.** There are no commands, tools, or install step for it. See [AutoFix nightly](../processes/autofix-nightly/README.md).
10
10
 
11
11
  ## Trust
12
12
 
package/docs/settings.md CHANGED
@@ -1,6 +1,8 @@
1
1
  # Settings
2
2
 
3
- The first `review_case` run reads `.e2e-qa-pilot/settings.md`. If a key it needs is empty, the assistant asks once in chat and writes your answers into that file. A case id needs `case_tool`, `fetch_case`, and `update_case`. Pasted case text does not. Both inputs need `figma`, `read_figma`, `test_command`, and `png_dir`. `branch_pattern` has a default.
3
+ `settings.md` holds the keys every project shares. What is specific to your repository (ticket id, branch and commit format, case tool, Figma, feature flags, other repositories) goes in `.e2e-qa-pilot/project.md`. See [Project file](project.md). A value there answers the matching key below, so the assistant does not ask for it.
4
+
5
+ The first `review_case` run reads `.e2e-qa-pilot/settings.md`. If a key it needs is empty, the assistant asks once in chat and writes your answers into that file. A case id needs `case_tool`, `fetch_case`, and `update_case`. Pasted case text does not. Both inputs need `figma` and `test_command`, and `read_figma` when `figma` is `on`. `png_dir` is required unless `## Test layout` in `project.md` names the screenshot directory. `branch_pattern` has a default.
4
6
 
5
7
  The settings file is included in every tool result, so do not store tokens in it. Put how to fetch and update a case in the file, and keep tokens in your own MCP configuration. Do not paste a secret into the chat.
6
8
 
@@ -15,10 +17,18 @@ The settings file is included in every tool result, so do not store tokens in it
15
17
  | `pr_label_minor_text_diff` | Label when the only issue is a narrow wording difference. Default `e2e:minor-text-diff` |
16
18
  | `pr_label_mismatch` | Label for a functional or unresolved mismatch, or inadequate required screenshot evidence. Default `e2e:mismatch` |
17
19
 
18
- The command files list `read_figma` among the required keys even when `figma: off`. The settings template leaves it blank in that case. Until that is settled, expect to be asked for it once. Changing that behavior is outside the current instructions. See [Known limitations](../mcp.md#known-limitations).
20
+ No Figma? Answer `off` and leave `read_figma` blank. `read_figma` is required only when `figma` is `on`. With `figma: off`, in `settings.md` or in `## Design` of `project.md`, leave it blank. The assistant does not ask for it.
19
21
 
20
22
  ## Pull request outcomes
21
23
 
24
+ | Label | Color | Meaning |
25
+ |-------|-------|---------|
26
+ | `e2e:ready` | 🟢 green `#0E8A16` | Behavior matches and the required screenshots are adequate |
27
+ | `e2e:minor-text-diff` | 🟡 yellow `#FBCA04` | Behavior matches and the only issue is a narrow wording difference |
28
+ | `e2e:mismatch` | 🔴 red `#D73A4A` | A functional or unresolved mismatch, or inadequate screenshots. Draft pull request |
29
+
30
+ When a label is missing in your repository, the assistant creates it with these colors and descriptions. It does not change a label that already exists. The colors are not a setting, so a renamed label (`pr_label_*`) is created with the same color as the one it replaces.
31
+
22
32
  When the assistant has the tools and permissions the commands name, the intended deliverables are a Playwright spec, a step-by-step results table, screenshot evidence, pull request comments, and a pull request for human QA review. A label is a review outcome. It does not mean the change is approved or merged.
23
33
 
24
34
  A green test run means only that the assertions executed by that run passed. It does not prove that every expected behavior was tested or that the screenshots demonstrate every required state. A failed assertion and inadequate screenshot evidence are separate findings. Inadequate screenshot evidence does not, by itself, prove that a functional assertion failed.
@@ -31,8 +41,8 @@ Nothing is auto-merged. The default `pr_mode` is `create_pr_for_review`. It allo
31
41
 
32
42
  ## Required inputs
33
43
 
34
- The server requires exactly one of `case_id` or `case_text`. Pasted text needs no case tool. A case id needs `case_tool`, `fetch_case`, `update_case`, and an external case-tool MCP. Generating a spec needs `figma`, `read_figma`, `test_command`, and `png_dir`. `branch_pattern`, `pr_mode`, `pr_label_ready`, `pr_label_minor_text_diff`, and `pr_label_mismatch` have defaults, so a missing value is not a question. The input contract is in [mcp.md](../mcp.md).
44
+ The server requires exactly one of `case_id` or `case_text`. Pasted text needs no case tool. A case id needs `case_tool`, `fetch_case`, `update_case`, and an external case-tool MCP. Generating a spec needs `figma` and `test_command`, and `read_figma` when `figma` is `on`. It also needs `png_dir` unless `## Test layout` in `project.md` names the screenshot directory. `branch_pattern`, `pr_mode`, `pr_label_ready`, `pr_label_minor_text_diff`, and `pr_label_mismatch` have defaults, so a missing value is not a question. The input contract is in [mcp.md](../mcp.md).
35
45
 
36
46
  ## Reference conventions
37
47
 
38
- Reference conventions in the commands, which another project may adapt, include `overview.md`, Gherkin under `specs/`, page contracts under `pages/`, page objects, flows, and a Playwright fixture. There is no settings key for those paths. A project does not have to copy that layout for the server to start. The [Cabana example](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md) uses it so the reference workflow can be read in one place. That directory is in this git repository and is not part of the published package. Figma helpers such as `attachMilestoneScreenshot` are not in the example and are not provided by this package.
48
+ Reference conventions in the commands, which another project may adapt (set the locations in `## Test layout` of [`project.md`](project.md)), include `overview.md`, Gherkin under `specs/`, page contracts under `pages/`, page objects, flows, and a Playwright fixture. There is no settings key for those paths. A project does not have to copy that layout for the server to start. The [Cabana example](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md) uses it so the reference workflow can be read in one place. That directory is in this git repository and is not part of the published package. Figma helpers such as `attachMilestoneScreenshot` are not in the example and are not provided by this package.