e2e-qa-pilot 0.0.0-stage → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CODE_OF_CONDUCT.md +126 -0
- package/CONTRIBUTING.md +11 -0
- package/LICENSE +21 -0
- package/README.md +111 -2
- package/SECURITY.md +13 -0
- package/assets/logo.svg +12 -0
- package/bin/cli.mjs +32 -0
- package/docs/README.md +9 -0
- package/docs/install.md +111 -0
- package/docs/scope-and-trust.md +19 -0
- package/docs/settings.md +38 -0
- package/mcp.md +166 -0
- package/package.json +47 -4
- package/processes/autofix-nightly/README.md +18 -0
- package/processes/create-e2e/README.md +42 -0
- package/processes/create-e2e/commands/add-from-design/SKILL.md +173 -0
- package/processes/create-e2e/commands/add-from-steps/SKILL.md +178 -0
- package/processes/create-e2e/commands/review-case/SKILL.md +130 -0
- package/processes/create-e2e/commands/shared/pr-evidence.md +211 -0
- package/processes/create-e2e/commands/shared/ready-rules.md +42 -0
- package/src/init.mjs +267 -0
- package/src/kit.mjs +89 -0
- package/src/mcp.mjs +227 -0
- package/templates/settings.md +57 -0
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
# Contributor Covenant Code of Conduct
|
|
2
|
+
|
|
3
|
+
## Our Pledge
|
|
4
|
+
|
|
5
|
+
We as members, contributors, and leaders pledge to make participation in our
|
|
6
|
+
community a harassment-free experience for everyone, regardless of age, body
|
|
7
|
+
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
|
8
|
+
identity and expression, level of experience, education, socio-economic status,
|
|
9
|
+
nationality, personal appearance, race, caste, color, religion, or sexual
|
|
10
|
+
identity and orientation.
|
|
11
|
+
|
|
12
|
+
We pledge to act and interact in ways that contribute to an open, welcoming,
|
|
13
|
+
diverse, inclusive, and healthy community.
|
|
14
|
+
|
|
15
|
+
## Our Standards
|
|
16
|
+
|
|
17
|
+
Examples of behavior that contributes to a positive environment for our
|
|
18
|
+
community include:
|
|
19
|
+
|
|
20
|
+
* Demonstrating empathy and kindness toward other people
|
|
21
|
+
* Being respectful of differing opinions, viewpoints, and experiences
|
|
22
|
+
* Giving and gracefully accepting constructive feedback
|
|
23
|
+
* Accepting responsibility and apologizing to those affected by our mistakes,
|
|
24
|
+
and learning from the experience
|
|
25
|
+
* Focusing on what is best not just for us as individuals, but for the overall
|
|
26
|
+
community
|
|
27
|
+
|
|
28
|
+
Examples of unacceptable behavior include:
|
|
29
|
+
|
|
30
|
+
* The use of sexualized language or imagery, and sexual attention or advances of
|
|
31
|
+
any kind
|
|
32
|
+
* Trolling, insulting or derogatory comments, and personal or political attacks
|
|
33
|
+
* Public or private harassment
|
|
34
|
+
* Publishing others' private information, such as a physical or email address,
|
|
35
|
+
without their explicit permission
|
|
36
|
+
* Other conduct which could reasonably be considered inappropriate in a
|
|
37
|
+
professional setting
|
|
38
|
+
|
|
39
|
+
## Enforcement Responsibilities
|
|
40
|
+
|
|
41
|
+
Community leaders are responsible for clarifying and enforcing our standards of
|
|
42
|
+
acceptable behavior and will take appropriate and fair corrective action in
|
|
43
|
+
response to any behavior that they deem inappropriate, threatening, offensive,
|
|
44
|
+
or harmful.
|
|
45
|
+
|
|
46
|
+
Community leaders have the right and responsibility to remove, edit, or reject
|
|
47
|
+
comments, commits, code, wiki edits, issues, and other contributions that are
|
|
48
|
+
not aligned to this Code of Conduct, and will communicate reasons for moderation
|
|
49
|
+
decisions when appropriate.
|
|
50
|
+
|
|
51
|
+
## Scope
|
|
52
|
+
|
|
53
|
+
This Code of Conduct applies within all community spaces, and also applies when
|
|
54
|
+
an individual is officially representing the community in public spaces.
|
|
55
|
+
Examples of representing our community include using an official email address,
|
|
56
|
+
posting via an official social media account, or acting as an appointed
|
|
57
|
+
representative at an online or offline event.
|
|
58
|
+
|
|
59
|
+
## Enforcement
|
|
60
|
+
|
|
61
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
|
62
|
+
reported privately to the repository owner through
|
|
63
|
+
[GitHub Security Advisories](https://github.com/bogdanOwner/e2e-qa-pilot/security/advisories/new).
|
|
64
|
+
All complaints will be reviewed and investigated promptly and fairly.
|
|
65
|
+
|
|
66
|
+
All community leaders are obligated to respect the privacy and security of the
|
|
67
|
+
reporter of any incident.
|
|
68
|
+
|
|
69
|
+
## Enforcement Guidelines
|
|
70
|
+
|
|
71
|
+
Community leaders will follow these Community Impact Guidelines in determining
|
|
72
|
+
the consequences for any action they deem in violation of this Code of Conduct:
|
|
73
|
+
|
|
74
|
+
### 1. Correction
|
|
75
|
+
|
|
76
|
+
**Community Impact**: Use of inappropriate language or other behavior deemed
|
|
77
|
+
unprofessional or unwelcome in the community.
|
|
78
|
+
|
|
79
|
+
**Consequence**: A private, written warning from community leaders, providing
|
|
80
|
+
clarity around the nature of the violation and an explanation of why the
|
|
81
|
+
behavior was inappropriate. A public apology may be requested.
|
|
82
|
+
|
|
83
|
+
### 2. Warning
|
|
84
|
+
|
|
85
|
+
**Community Impact**: A violation through a single incident or series of
|
|
86
|
+
actions.
|
|
87
|
+
|
|
88
|
+
**Consequence**: A warning with consequences for continued behavior. No
|
|
89
|
+
interaction with the people involved, including unsolicited interaction with
|
|
90
|
+
those enforcing the Code of Conduct, for a specified period of time. This
|
|
91
|
+
includes avoiding interactions in community spaces as well as external channels
|
|
92
|
+
like social media. Violating these terms may lead to a temporary or permanent
|
|
93
|
+
ban.
|
|
94
|
+
|
|
95
|
+
### 3. Temporary Ban
|
|
96
|
+
|
|
97
|
+
**Community Impact**: A serious violation of community standards, including
|
|
98
|
+
sustained inappropriate behavior.
|
|
99
|
+
|
|
100
|
+
**Consequence**: A temporary ban from any sort of interaction or public
|
|
101
|
+
communication with the community for a specified period of time. No public or
|
|
102
|
+
private interaction with the people involved, including unsolicited interaction
|
|
103
|
+
with those enforcing the Code of Conduct, is allowed during this period.
|
|
104
|
+
Violating these terms may lead to a permanent ban.
|
|
105
|
+
|
|
106
|
+
### 4. Permanent Ban
|
|
107
|
+
|
|
108
|
+
**Community Impact**: Demonstrating a pattern of violation of community
|
|
109
|
+
standards, including sustained inappropriate behavior, harassment of an
|
|
110
|
+
individual, or aggression toward or disparagement of classes of individuals.
|
|
111
|
+
|
|
112
|
+
**Consequence**: A permanent ban from any sort of public interaction within the
|
|
113
|
+
community.
|
|
114
|
+
|
|
115
|
+
## Attribution
|
|
116
|
+
|
|
117
|
+
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org),
|
|
118
|
+
version 2.1, available at
|
|
119
|
+
<https://www.contributor-covenant.org/version/2/1/code_of_conduct.html>.
|
|
120
|
+
|
|
121
|
+
Community Impact Guidelines were inspired by
|
|
122
|
+
[Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/diversity).
|
|
123
|
+
|
|
124
|
+
For answers to common questions about this code of conduct, see the FAQ at
|
|
125
|
+
<https://www.contributor-covenant.org/faq>. Translations are available at
|
|
126
|
+
<https://www.contributor-covenant.org/translations>.
|
package/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
```bash
|
|
4
|
+
npm test
|
|
5
|
+
```
|
|
6
|
+
|
|
7
|
+
`npm test` runs behavioral tests of `init` and the MCP server, and documentation consistency checks in `test/docs.test.mjs`. They check that instruction text and documentation match the code. They do not prove that an assistant follows the instructions, they do not validate the assistant workflow end to end, and they do not run Cursor or Claude Code.
|
|
8
|
+
|
|
9
|
+
Open an issue before a large change. The commands under `processes/create-e2e/commands/` are the spec, so a behavior change starts there. The overview is [Create e2e](processes/create-e2e/README.md).
|
|
10
|
+
|
|
11
|
+
Report vulnerabilities through [SECURITY.md](SECURITY.md). Community expectations are in [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md).
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 e2e-qa-pilot contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,112 @@
|
|
|
1
|
-
|
|
1
|
+
<!--suppress HtmlDeprecatedAttribute -->
|
|
2
|
+
<div align="center">
|
|
3
|
+
<img src="assets/logo.svg" width="96" alt="e2e-qa-pilot logo">
|
|
4
|
+
<h1>e2e-qa-pilot</h1>
|
|
5
|
+
<p><em>Turn manual test cases into reviewed, automated Playwright PRs.</em></p>
|
|
6
|
+
<p>
|
|
7
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License: MIT"></a>
|
|
8
|
+
<img src="https://img.shields.io/badge/tests-Playwright-45ba4b.svg" alt="Playwright">
|
|
9
|
+
<img src="https://img.shields.io/badge/node-%3E%3D22-339933.svg" alt="Node 22 or later">
|
|
10
|
+
<img src="https://img.shields.io/badge/agents-Cursor%20%7C%20Claude%20Code-black.svg" alt="Cursor and Claude Code">
|
|
11
|
+
</p>
|
|
12
|
+
</div>
|
|
2
13
|
|
|
3
|
-
|
|
14
|
+
[**Get started**](#get-started) · [**Examples**](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md) · [**MCP**](mcp.md) · [**Docs**](docs/README.md) · [**Contributing**](CONTRIBUTING.md)
|
|
15
|
+
|
|
16
|
+
e2e-qa-pilot gives your coding agent the expected result up front, so it can build the test, verify it with screenshots, and open a PR you can review in minutes.
|
|
17
|
+
|
|
18
|
+
## Why a pilot
|
|
19
|
+
|
|
20
|
+
- It asks once, up front, and only when the case is unclear. It does not invent an expected result.
|
|
21
|
+
- It checks the final screen. The pull request shows every step and a screenshot.
|
|
22
|
+
- A person reviews the pull request. Nothing is auto-merged.
|
|
23
|
+
|
|
24
|
+
## See it work
|
|
25
|
+
|
|
26
|
+
One chat line on the Cabana example. The label says what happened. The link opens the pull request for that case.
|
|
27
|
+
|
|
28
|
+
| You send | You get | PR example |
|
|
29
|
+
|----------|---------|------------|
|
|
30
|
+
| A case where every step matches | `e2e:ready`. Behavior matches, with screenshots | [PR for cases/ready.md](https://github.com/bogdanOwner/e2e-qa-pilot/pull/3) |
|
|
31
|
+
| The same case, one sentence off | `e2e:minor-text-diff`. A wording difference does not stop the flow | [PR for cases/wording.md](https://github.com/bogdanOwner/e2e-qa-pilot/pull/4) |
|
|
32
|
+
| A case that hits a real bug | Draft `e2e:mismatch`. The failed step and the screen | [PR for cases/mismatch.md](https://github.com/bogdanOwner/e2e-qa-pilot/pull/5) |
|
|
33
|
+
| A case with no expected result | No spec and no pull request. The assistant asks what should be visible | No pull request |
|
|
34
|
+
|
|
35
|
+
The prompts and the case text are in the [Cabana example](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md).
|
|
36
|
+
|
|
37
|
+
## Choose your process
|
|
38
|
+
|
|
39
|
+
These are independent entry points.
|
|
40
|
+
|
|
41
|
+
| What you need | Process | Outcome |
|
|
42
|
+
|---------------|---------|---------|
|
|
43
|
+
| Build end-to-end coverage from a manual case or a short description | [Create e2e](processes/create-e2e/README.md) | A pull request labeled `e2e:ready`, `e2e:minor-text-diff`, or draft `e2e:mismatch`, with screenshots and a step table |
|
|
44
|
+
| Explain a failing nightly run and open one fix for review | Nightly AutoFix, not implemented, coming soon. [Planned flow](processes/autofix-nightly/README.md) | Read the frontend changes since the last run, rerun the failed tests to check for flakiness, report why it failed, and open a fix pull request |
|
|
45
|
+
|
|
46
|
+
## Labels at a glance
|
|
47
|
+
|
|
48
|
+
| Label | What to do |
|
|
49
|
+
|-------|------------|
|
|
50
|
+
| `e2e:ready` | Review and merge. Function and screenshots match |
|
|
51
|
+
| `e2e:minor-text-diff` | Decide on the wording. The flow was not stopped, and the exact sentence was not marked as a pass |
|
|
52
|
+
| `e2e:mismatch` | Look at the bug. Draft pull request. This is a finding, not an automatic verdict |
|
|
53
|
+
|
|
54
|
+
## How it works
|
|
55
|
+
|
|
56
|
+
```mermaid
|
|
57
|
+
flowchart TD
|
|
58
|
+
caseIn[Manual case] --> reviewCase[review_case]
|
|
59
|
+
reviewCase -->|unclear| askStop[Ask and stop]
|
|
60
|
+
reviewCase -->|ready| buildSpec[Playwright spec and screenshots]
|
|
61
|
+
buildSpec --> readyLabel["PR labeled e2e:ready"]
|
|
62
|
+
buildSpec --> minorLabel["PR labeled e2e:minor-text-diff"]
|
|
63
|
+
buildSpec --> mismatchLabel["Draft PR labeled e2e:mismatch"]
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Because the expected result is known up front, the agent keeps working until there is a pull request. It does not stop at the first red run.
|
|
67
|
+
|
|
68
|
+
- **A page description comes first.** From the case, or from Figma when a link is present, the agent writes what each screen should show. That makes a quick fix easier when only a CSS or copy difference is left.
|
|
69
|
+
- **It compares the real screenshot with that description.** That is how it tells a wording difference from a real bug, and labels the pull request to match.
|
|
70
|
+
- **The loop is bounded.** A spec gets at most five runs, and a real behavior difference stops the run at once, so it does not loop forever.
|
|
71
|
+
- **The case can live elsewhere.** Paste it, or pass an id and let your case-tool MCP (Qase is the worked example) supply the ticket text. Figma MCP is optional and adds design detail.
|
|
72
|
+
- **The page descriptions will also help a nightly fix process.** That process is not implemented, coming soon.
|
|
73
|
+
|
|
74
|
+
The full diagram is in [Install](docs/install.md#how-a-run-ends). Tool contracts are in [mcp.md](mcp.md).
|
|
75
|
+
|
|
76
|
+
## Configuration
|
|
77
|
+
|
|
78
|
+
Settings live in `.e2e-qa-pilot/settings.md`, which `init` creates once. These matter most.
|
|
79
|
+
|
|
80
|
+
| Setting | Options | What it changes |
|
|
81
|
+
|---------|---------|-----------------|
|
|
82
|
+
| `pr_mode` | `create_pr_for_review` (default), `ask_before_pr` | Whether a bug opens a draft pull request on its own, or the agent stops and asks first. Nothing is auto-merged either way |
|
|
83
|
+
| `figma` | `on`, `off` | `on` with a figma.com link in the case compares each screen with its frame. `off` uses the case steps as the oracle |
|
|
84
|
+
| Case source | pasted text, or a case id | A case id needs a connected case-tool MCP and `case_tool`, `fetch_case`, and `update_case`. Qase is the worked example. Other tools such as Jira may work through their own MCP, but are not tested here |
|
|
85
|
+
| `test_command`, `png_dir` | your Playwright command, a folder | How the spec is run, and where screenshots go |
|
|
86
|
+
| `pr_label_ready`, `pr_label_minor_text_diff`, `pr_label_mismatch` | label names | Rename the three labels |
|
|
87
|
+
|
|
88
|
+
All keys, defaults, and the label rules are in [Settings](docs/settings.md). Do not store tokens in the file.
|
|
89
|
+
|
|
90
|
+
## Get started
|
|
91
|
+
|
|
92
|
+
You need Node 22 or later, plus Cursor or Claude Code. The three tools are `review_case`, `add_from_steps`, and `add_from_design`. They return instructions, and your assistant does the work. The branch name comes from the case title (`{title_slug}`).
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
npx e2e-qa-pilot init
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Restart the editor, open a new chat, and send one line. A pasted case needs a `Title:` line. A case id needs your case tool connected.
|
|
99
|
+
|
|
100
|
+
```text
|
|
101
|
+
Review case QA-123, then implement it with e2e-qa-pilot.
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
- [Install and init](docs/install.md) for prerequisites, what `init` writes, and upgrade.
|
|
105
|
+
- [Settings](docs/settings.md) for the test command, labels, and `pr_mode`.
|
|
106
|
+
- [MCP](mcp.md) for the tools, and [without MCP](mcp.md#without-mcp) if you only want the Markdown skills.
|
|
107
|
+
- [Example](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md) to try the four cases.
|
|
108
|
+
- [Scope and trust](docs/scope-and-trust.md) for what this package does and does not do.
|
|
109
|
+
|
|
110
|
+
## Community
|
|
111
|
+
|
|
112
|
+
[Contributing](CONTRIBUTING.md) · [Security](SECURITY.md) · [Code of conduct](CODE_OF_CONDUCT.md) · [MIT](LICENSE)
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Security policy
|
|
2
|
+
|
|
3
|
+
## Supported versions
|
|
4
|
+
|
|
5
|
+
Security fixes land on the latest release published to npm and on the default branch.
|
|
6
|
+
|
|
7
|
+
## Reporting a vulnerability
|
|
8
|
+
|
|
9
|
+
Report a vulnerability privately through [GitHub Security Advisories](https://github.com/bogdanOwner/e2e-qa-pilot/security/advisories/new).
|
|
10
|
+
|
|
11
|
+
Please do not open a public issue for a security report.
|
|
12
|
+
|
|
13
|
+
This project does not store credentials. The MCP server makes no network requests. It only returns instruction text. Review the diff and the pull request your assistant opens, because that part is outside this package.
|
package/assets/logo.svg
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128" role="img" aria-label="e2e-qa-pilot">
|
|
2
|
+
<defs>
|
|
3
|
+
<linearGradient id="pilot-bg" x1="16" y1="8" x2="112" y2="120" gradientUnits="userSpaceOnUse">
|
|
4
|
+
<stop offset="0" stop-color="#4f46e5"/>
|
|
5
|
+
<stop offset="0.55" stop-color="#7c3aed"/>
|
|
6
|
+
<stop offset="1" stop-color="#06b6d4"/>
|
|
7
|
+
</linearGradient>
|
|
8
|
+
</defs>
|
|
9
|
+
<rect x="4" y="4" width="120" height="120" rx="28" fill="url(#pilot-bg)"/>
|
|
10
|
+
<path d="M34 66.5 L54.5 87 L96 40" fill="none" stroke="#ffffff" stroke-width="12" stroke-linecap="round" stroke-linejoin="round"/>
|
|
11
|
+
<circle cx="96" cy="32" r="6" fill="#ffffff"/>
|
|
12
|
+
</svg>
|
package/bin/cli.mjs
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
import { runInit, INIT_USAGE } from '../src/init.mjs';
|
|
3
|
+
import { runMcp } from '../src/mcp.mjs';
|
|
4
|
+
import { readPackageVersion } from '../src/kit.mjs';
|
|
5
|
+
|
|
6
|
+
const [command, ...rest] = process.argv.slice(2);
|
|
7
|
+
|
|
8
|
+
const USAGE = `e2e-qa-pilot ${readPackageVersion()}
|
|
9
|
+
|
|
10
|
+
Commands:
|
|
11
|
+
init [path] Install the skills and settings into your project
|
|
12
|
+
mcp Start the MCP server (stdio). Your editor runs this for you.
|
|
13
|
+
|
|
14
|
+
${INIT_USAGE}`;
|
|
15
|
+
|
|
16
|
+
try {
|
|
17
|
+
if (command === 'init') {
|
|
18
|
+
runInit(rest);
|
|
19
|
+
} else if (command === 'mcp') {
|
|
20
|
+
runMcp();
|
|
21
|
+
} else if (command === '--version' || command === '-v') {
|
|
22
|
+
console.log(readPackageVersion());
|
|
23
|
+
} else if (!command || command === '--help' || command === '-h' || command === 'help') {
|
|
24
|
+
console.log(USAGE);
|
|
25
|
+
} else {
|
|
26
|
+
console.error(`Unknown command: ${command}\n\n${USAGE}`);
|
|
27
|
+
process.exit(1);
|
|
28
|
+
}
|
|
29
|
+
} catch (error) {
|
|
30
|
+
console.error(error instanceof Error ? error.message : String(error));
|
|
31
|
+
process.exit(1);
|
|
32
|
+
}
|
package/docs/README.md
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# Docs
|
|
2
|
+
|
|
3
|
+
Reference for installing e2e-qa-pilot and for the rules the assistant is given.
|
|
4
|
+
|
|
5
|
+
- [Install](install.md): prerequisites, `init`, the files it writes, and the one-line prompts.
|
|
6
|
+
- [Settings](settings.md): settings keys, pull request labels, and required inputs.
|
|
7
|
+
- [Scope and trust](scope-and-trust.md): what the package does, what it does not do, and what the tests check.
|
|
8
|
+
- [MCP](../mcp.md): the three tools, the input contract, and known limitations.
|
|
9
|
+
- [Create e2e](../processes/create-e2e/README.md): the workflow the tools ask the assistant to follow.
|
package/docs/install.md
ADDED
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Install
|
|
2
|
+
|
|
3
|
+
You need Node 22 or later, Playwright already configured in the project, and Cursor or Claude Code. Playwright's browser install is yours to run. The commands never install browsers.
|
|
4
|
+
|
|
5
|
+
Git access to `origin` is required, with `main` as the base branch. The commands branch from `main` and target it.
|
|
6
|
+
|
|
7
|
+
Optional, and only for the path you use:
|
|
8
|
+
|
|
9
|
+
- A case-tool MCP, when you pass a case id. Qase is the worked example. The update checklist is Qase-shaped and has not been verified against other integrations.
|
|
10
|
+
- Figma access from your assistant, only for design comparison. That path can depend on a milestone screenshot helper in your project. This package does not provide one.
|
|
11
|
+
- GitHub access to create a pull request and apply labels. `init` does not configure any of these integrations.
|
|
12
|
+
|
|
13
|
+
## Quick start
|
|
14
|
+
|
|
15
|
+
In your own project:
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
npm install -g e2e-qa-pilot # or skip this and use npx
|
|
19
|
+
cd my-project
|
|
20
|
+
e2e-qa-pilot init
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
With `npx` and no install: `npx e2e-qa-pilot init`.
|
|
24
|
+
|
|
25
|
+
`init` works on the folder you are in, or on a path you pass: `e2e-qa-pilot init [path]`. It looks for `.cursor/` and `.claude/`, sets up the one it finds, and sets up both when both folders are present or when neither is. To choose, pass `--integration cursor`, `--integration claude`, or `--integration all`. Cursor and Claude Code are the two editors `init` knows how to set up. See [What is and is not verified](../mcp.md#what-is-and-is-not-verified).
|
|
26
|
+
|
|
27
|
+
Restart Cursor or Claude Code, then open a new chat. A chat that was already open does not have the server.
|
|
28
|
+
|
|
29
|
+
## What `init` writes
|
|
30
|
+
|
|
31
|
+
| File | Purpose |
|
|
32
|
+
|------|---------|
|
|
33
|
+
| `.e2e-qa-pilot/settings.md` | Your case tool, test command, PNG folder, `pr_mode`, and PR labels. Created once, never overwritten |
|
|
34
|
+
| `.e2e-qa-pilot/processes/create-e2e/` | A copy of the Create e2e folder: the three commands, the shared rules, and a short README, as plain Markdown |
|
|
35
|
+
| `.cursor/skills/<name>/SKILL.md` | Cursor skills (`review-case`, `add-from-steps`, `add-from-design`). Each is a short stub that points at the commands above |
|
|
36
|
+
| `.claude/skills/<name>/SKILL.md` | The same skills for Claude Code |
|
|
37
|
+
| `.cursor/mcp.json` and `.mcp.json` | One `e2e-qa-pilot` entry. Other servers in the file are kept. An entry that is already listed stays as it is |
|
|
38
|
+
|
|
39
|
+
## Upgrade
|
|
40
|
+
|
|
41
|
+
Run `init` again at any time. It leaves existing files alone.
|
|
42
|
+
|
|
43
|
+
- `--force` refreshes a file only when that file still has the exact `<!-- e2e-qa-pilot:managed -->` line. **That replaces any edits you made to such a file.** Remove the line from a file to keep it as yours.
|
|
44
|
+
- `.e2e-qa-pilot/settings.md` and an MCP entry that is already listed are never changed, even with `--force`.
|
|
45
|
+
- An MCP file that is not valid JSON, or does not have the expected shape, is left alone, and `init` prints the entry to add by hand.
|
|
46
|
+
|
|
47
|
+
The skills also work without the MCP entry. See [Without MCP](../mcp.md#without-mcp).
|
|
48
|
+
|
|
49
|
+
## Ask in one line
|
|
50
|
+
|
|
51
|
+
A pasted case needs no case tool. Include a `Title:` line, or the assistant will ask for one.
|
|
52
|
+
|
|
53
|
+
```text
|
|
54
|
+
Review this case with e2e-qa-pilot and implement it if it is ready:
|
|
55
|
+
|
|
56
|
+
Title: Book Chalet 4 at Cabana
|
|
57
|
+
Preconditions: the guest is on the Cabana page.
|
|
58
|
+
1. Set check-in to 4 Aug 2026 and check-out to 5 Aug 2026, then click Search.
|
|
59
|
+
Expected: Chalet 4 is listed at 497 EUR.
|
|
60
|
+
2. Press Increase quantity on the 497 EUR rate, then click Reserve Now.
|
|
61
|
+
Expected: the guest form opens on the confirm page.
|
|
62
|
+
3. Enter First Name "Ana", Last Name "Pop", and Email Address "ana.pop@example.com", accept the terms, and click Confirm Booking.
|
|
63
|
+
Expected: the confirmation says "Your request was successfully sent." and a booking reference is shown.
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Pasted text needs no case tool, but the first run still asks once for the settings that generating a spec needs, such as `test_command` and `png_dir`. See [Settings](settings.md). `pr_mode` and the label keys have defaults, so an empty value is not a question. If the case is ready, `review_case` carries on into the matching command in the same turn, and that can end in a pull request when GitHub access exists. The instructions do not define a review-only mode. Details are in [mcp.md](../mcp.md#review_case).
|
|
67
|
+
|
|
68
|
+
A case id needs a connected case tool (for example Qase):
|
|
69
|
+
|
|
70
|
+
```text
|
|
71
|
+
Review case QA-123, then implement it with e2e-qa-pilot.
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
## Worked example
|
|
75
|
+
|
|
76
|
+
This repository includes an example you can use to see e2e-qa-pilot run. The pull requests from that run are linked there, and the same page explains how to run it locally.
|
|
77
|
+
|
|
78
|
+
[examples/boutique-cabin/README.md](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md)
|
|
79
|
+
|
|
80
|
+
That folder is in the git repository. The published npm package does not include it.
|
|
81
|
+
|
|
82
|
+
## How a run ends
|
|
83
|
+
|
|
84
|
+
`init` copies Markdown commands and editor stubs into a project. `mcp` serves three tools over stdio: `review_case`, `add_from_steps`, and `add_from_design`. Each tool returns instruction text and your settings file. The flowchart is the workflow those instructions ask the assistant to follow. The server does not read or enforce `pr_mode`, and it does not choose a label. The same diagram is in [Create e2e](../processes/create-e2e/README.md). Tool contracts are in [mcp.md](../mcp.md).
|
|
85
|
+
|
|
86
|
+
```mermaid
|
|
87
|
+
flowchart TD
|
|
88
|
+
reviewNode[review_case]
|
|
89
|
+
implNode[implementation command]
|
|
90
|
+
readyNode[Behavior matches and evidence is adequate]
|
|
91
|
+
minorNode[Minor text difference only]
|
|
92
|
+
assertNode[Functional or unresolved assertion]
|
|
93
|
+
shotNode[Required screenshot evidence fails]
|
|
94
|
+
modeNode{pr_mode}
|
|
95
|
+
readyPr["PR labeled e2e:ready"]
|
|
96
|
+
minorPr["PR labeled e2e:minor-text-diff"]
|
|
97
|
+
draftPr["Draft PR labeled e2e:mismatch"]
|
|
98
|
+
askNode[Ask then continue only if authorized]
|
|
99
|
+
reviewNode -->|case not ready| askClarify[Ask and do not invent]
|
|
100
|
+
reviewNode -->|case ready same turn| implNode
|
|
101
|
+
implNode --> readyNode
|
|
102
|
+
implNode --> minorNode
|
|
103
|
+
implNode --> assertNode
|
|
104
|
+
implNode --> shotNode
|
|
105
|
+
readyNode --> readyPr
|
|
106
|
+
minorNode --> minorPr
|
|
107
|
+
assertNode --> modeNode
|
|
108
|
+
shotNode --> modeNode
|
|
109
|
+
modeNode -->|create_pr_for_review| draftPr
|
|
110
|
+
modeNode -->|ask_before_pr| askNode
|
|
111
|
+
```
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Scope and trust
|
|
2
|
+
|
|
3
|
+
## Scope
|
|
4
|
+
|
|
5
|
+
- Playwright only. The commands describe Playwright specs, runs, and screenshots. Selenium is not supported.
|
|
6
|
+
- Case tools: Qase is the worked example, and the `review_case` command describes how it updates a Qase case. Other tools may work if your assistant can read and update them through its own MCP server, but none are tested or officially supported here. This package does not implement a case-tool client. Pasted case text needs none.
|
|
7
|
+
- A person decides when a requirement is ambiguous. The assistant is told to ask and not to invent an expected result. `ask_before_pr` also stops for a functional mismatch, an unresolved difference, or a failed final-screenshot gate. A narrow minor text difference does not trigger that stop.
|
|
8
|
+
|
|
9
|
+
**Create e2e is implemented.** The three tools, `review_case`, `add_from_steps`, and `add_from_design`, hand your assistant those instructions. While writing the spec, the assistant may adjust locators or waits in that spec. **AutoFix nightly is not implemented.** There are no commands, tools, or install step for it. See [AutoFix nightly](../processes/autofix-nightly/README.md).
|
|
10
|
+
|
|
11
|
+
## Trust
|
|
12
|
+
|
|
13
|
+
This tool is MIT licensed and small on purpose. The MCP server is [src/mcp.mjs](../src/mcp.mjs) with shared constants in [src/kit.mjs](../src/kit.mjs). It has no dependencies, makes no network requests, and only reads the packaged command files and your `.e2e-qa-pilot/settings.md`. It writes only JSON-RPC to stdout. Case text in a tool call is the text the client sent. `init` is [src/init.mjs](../src/init.mjs) and uses the same constants.
|
|
14
|
+
|
|
15
|
+
This project does not store credentials. What your assistant commits, reports, or posts to a pull request is outside this project's control, so review the diff and the pull request yourself. The `npx` entry that `init` writes downloads the package from npm when it starts, which is separate from the server's own behavior.
|
|
16
|
+
|
|
17
|
+
## What the tests check
|
|
18
|
+
|
|
19
|
+
`npm test` runs behavioral tests of `init` and the MCP server, and documentation consistency checks in `test/docs.test.mjs`. They check that instruction text and documentation match the code. They do not prove that an assistant follows the instructions, they do not validate the assistant workflow end to end, and they do not run Cursor or Claude Code. What is and is not verified is also summarized in [mcp.md](../mcp.md#what-is-and-is-not-verified).
|
package/docs/settings.md
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# Settings
|
|
2
|
+
|
|
3
|
+
The first `review_case` run reads `.e2e-qa-pilot/settings.md`. If a key it needs is empty, the assistant asks once in chat and writes your answers into that file. A case id needs `case_tool`, `fetch_case`, and `update_case`. Pasted case text does not. Both inputs need `figma`, `read_figma`, `test_command`, and `png_dir`. `branch_pattern` has a default.
|
|
4
|
+
|
|
5
|
+
The settings file is included in every tool result, so do not store tokens in it. Put how to fetch and update a case in the file, and keep tokens in your own MCP configuration. Do not paste a secret into the chat.
|
|
6
|
+
|
|
7
|
+
| Key | Meaning |
|
|
8
|
+
|-----|---------|
|
|
9
|
+
| `case_tool`, `fetch_case`, `update_case` | Where cases live and how to read and update one. Used only for a case id |
|
|
10
|
+
| `figma`, `read_figma` | `on` or `off`, and how to read a frame |
|
|
11
|
+
| `branch_pattern` | Branch name. `{title_slug}` is the case title in kebab-case and `{case_id}` is the case id, for a case id only. Default `e2e/{title_slug}`, or `e2e/{case_id}-{title_slug}` for a case id. A name with no placeholder is used as it is |
|
|
12
|
+
| `test_command`, `png_dir` | How to run the spec, and where PNGs go |
|
|
13
|
+
| `pr_mode` | `create_pr_for_review` (default) or `ask_before_pr`. A missing value means `create_pr_for_review` |
|
|
14
|
+
| `pr_label_ready` | Label when behavior matches and required screenshot evidence is adequate. Default `e2e:ready` |
|
|
15
|
+
| `pr_label_minor_text_diff` | Label when the only issue is a narrow wording difference. Default `e2e:minor-text-diff` |
|
|
16
|
+
| `pr_label_mismatch` | Label for a functional or unresolved mismatch, or inadequate required screenshot evidence. Default `e2e:mismatch` |
|
|
17
|
+
|
|
18
|
+
The command files list `read_figma` among the required keys even when `figma: off`. The settings template leaves it blank in that case. Until that is settled, expect to be asked for it once. Changing that behavior is outside the current instructions. See [Known limitations](../mcp.md#known-limitations).
|
|
19
|
+
|
|
20
|
+
## Pull request outcomes
|
|
21
|
+
|
|
22
|
+
When the assistant has the tools and permissions the commands name, the intended deliverables are a Playwright spec, a step-by-step results table, screenshot evidence, pull request comments, and a pull request for human QA review. A label is a review outcome. It does not mean the change is approved or merged.
|
|
23
|
+
|
|
24
|
+
A green test run means only that the assertions executed by that run passed. It does not prove that every expected behavior was tested or that the screenshots demonstrate every required state. A failed assertion and inadequate screenshot evidence are separate findings. Inadequate screenshot evidence does not, by itself, prove that a functional assertion failed.
|
|
25
|
+
|
|
26
|
+
- `e2e:ready` — functional assertions passed, the expected behavior matches, and all required screenshot evidence meets the acceptance criteria.
|
|
27
|
+
- `e2e:minor-text-diff` — functional behavior matches, and the only discrepancy is a minor wording difference that meets the documented classification rule. The difference is explicitly recorded. It is not reported as a passed exact-text assertion.
|
|
28
|
+
- `e2e:mismatch` — a functional mismatch or unresolved discrepancy exists, or required screenshot evidence fails to demonstrate the expected end state. The pull request is a draft. It is not a confirmed product regression.
|
|
29
|
+
|
|
30
|
+
Nothing is auto-merged. The default `pr_mode` is `create_pr_for_review`. It allows the assistant to create a pull request for review without auto-merging it. The optional `ask_before_pr` mode requires the assistant to stop and ask before creating a pull request when a functional mismatch, unresolved discrepancy, or failed final-screenshot gate remains. If the discrepancy remains after clarification, the assistant must obtain explicit authorization before creating the pull request.
|
|
31
|
+
|
|
32
|
+
## Required inputs
|
|
33
|
+
|
|
34
|
+
The server requires exactly one of `case_id` or `case_text`. Pasted text needs no case tool. A case id needs `case_tool`, `fetch_case`, `update_case`, and an external case-tool MCP. Generating a spec needs `figma`, `read_figma`, `test_command`, and `png_dir`. `branch_pattern`, `pr_mode`, `pr_label_ready`, `pr_label_minor_text_diff`, and `pr_label_mismatch` have defaults, so a missing value is not a question. The input contract is in [mcp.md](../mcp.md).
|
|
35
|
+
|
|
36
|
+
## Reference conventions
|
|
37
|
+
|
|
38
|
+
Reference conventions in the commands, which another project may adapt, include `overview.md`, Gherkin under `specs/`, page contracts under `pages/`, page objects, flows, and a Playwright fixture. There is no settings key for those paths. A project does not have to copy that layout for the server to start. The [Cabana example](https://github.com/bogdanOwner/e2e-qa-pilot/blob/main/examples/boutique-cabin/README.md) uses it so the reference workflow can be read in one place. That directory is in this git repository and is not part of the published package. Figma helpers such as `attachMilestoneScreenshot` are not in the example and are not provided by this package.
|