@ai-support-agent/cli 0.1.32-beta.1 → 0.1.33-beta.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/commands/claude-code-args.d.ts +1 -0
- package/dist/commands/claude-code-args.d.ts.map +1 -1
- package/dist/commands/claude-code-args.js +3 -0
- package/dist/commands/claude-code-args.js.map +1 -1
- package/dist/commands/claude-code-runner.d.ts.map +1 -1
- package/dist/commands/claude-code-runner.js +2 -1
- package/dist/commands/claude-code-runner.js.map +1 -1
- package/dist/commands/plugin-dir.d.ts +20 -0
- package/dist/commands/plugin-dir.d.ts.map +1 -0
- package/dist/commands/plugin-dir.js +71 -0
- package/dist/commands/plugin-dir.js.map +1 -0
- package/dist/plugin/.claude-plugin/plugin.json +9 -0
- package/dist/plugin/LICENSE +26 -0
- package/dist/plugin/README.md +86 -0
- package/dist/plugin/SYNC.md +113 -0
- package/dist/plugin/agents/build-error-resolver.md +159 -0
- package/dist/plugin/agents/code-reviewer.md +277 -0
- package/dist/plugin/agents/django-reviewer.md +159 -0
- package/dist/plugin/agents/infra-reviewer.md +172 -0
- package/dist/plugin/agents/investigator.md +75 -0
- package/dist/plugin/agents/nextjs-reviewer.md +191 -0
- package/dist/plugin/agents/php-reviewer.md +167 -0
- package/dist/plugin/agents/planner.md +184 -0
- package/dist/plugin/agents/python-reviewer.md +161 -0
- package/dist/plugin/agents/react-reviewer.md +150 -0
- package/dist/plugin/agents/silent-failure-hunter.md +158 -0
- package/dist/plugin/agents/typescript-reviewer.md +179 -0
- package/dist/plugin/agents/ui-reviewer.md +203 -0
- package/dist/plugin/commands/add-feature.md +301 -0
- package/dist/plugin/commands/build-fix.md +47 -0
- package/dist/plugin/commands/code-review.md +228 -0
- package/dist/plugin/commands/fix-defect.md +393 -0
- package/dist/plugin/commands/learn-eval.md +94 -0
- package/dist/plugin/commands/learn.md +84 -0
- package/dist/plugin/commands/plan.md +211 -0
- package/dist/plugin/commands/test-coverage.md +64 -0
- package/dist/plugin/commands/update-docs.md +98 -0
- package/dist/plugin/hooks/hooks.json +59 -0
- package/dist/plugin/hooks/scripts/auto-format.sh +63 -0
- package/dist/plugin/hooks/scripts/check-secrets-before-commit.sh +50 -0
- package/dist/plugin/hooks/scripts/guard-dangerous-commands.sh +55 -0
- package/dist/plugin/hooks/scripts/on-command-resume.sh +112 -0
- package/dist/plugin/hooks/scripts/on-command-stop.sh +200 -0
- package/dist/plugin/hooks/scripts/protect-sensitive-files.sh +58 -0
- package/dist/plugin/rules/common/coding-guidelines.md +73 -0
- package/dist/plugin/rules/documentation/api-docs.md +46 -0
- package/dist/plugin/rules/documentation/docs-site.md +60 -0
- package/dist/plugin/rules/documentation/source-docs.md +89 -0
- package/dist/plugin/rules/documentation/test-docs.md +39 -0
- package/dist/plugin/rules/logging/logging-rules.md +83 -0
- package/dist/plugin/rules/php/coding-rules.md +40 -0
- package/dist/plugin/rules/python/coding-rules.md +40 -0
- package/dist/plugin/rules/typescript/coding-rules.md +45 -0
- package/dist/plugin/skills/api-design/SKILL.md +269 -0
- package/dist/plugin/skills/backend-patterns/SKILL.md +312 -0
- package/dist/plugin/skills/database-migrations/SKILL.md +323 -0
- package/dist/plugin/skills/docker-patterns/SKILL.md +308 -0
- package/dist/plugin/skills/docs-site/SKILL.md +341 -0
- package/dist/plugin/skills/e2e-testing/SKILL.md +334 -0
- package/dist/plugin/skills/frontend-patterns/SKILL.md +318 -0
- package/dist/plugin/skills/integration-testing/SKILL.md +273 -0
- package/package.json +2 -2
|
@@ -0,0 +1,341 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: docs-site
|
|
3
|
+
description: A pattern library for building and operating a documentation site with Docusaurus. Covers i18n (ja/en) structure, automatic OpenAPI generation from NestJS, wiring OpenAPI/TypeDoc output into Docusaurus, type and client generation with openapi-typescript / orval, templates for hand-written reference pages, and CI freshness checks. Use this when setting up a new documentation site, adding pages, updating API references, syncing translations, or building a type-generation pipeline.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# docs-site: Docusaurus documentation site patterns
|
|
7
|
+
|
|
8
|
+
This skill is a pattern library focused on "how to build it." The authoritative source for the conventions themselves lives in `rules/documentation/` (aimed at repository maintainers); this skill assumes those conventions are already settled and provides the implementation steps.
|
|
9
|
+
|
|
10
|
+
## Baseline conventions assumed here
|
|
11
|
+
|
|
12
|
+
- Locales are `ja` (default) + `en`. Hand-written guides must exist in both languages. Auto-generated reference pages exist only in the default locale.
|
|
13
|
+
- NestJS APIs are documented by auto-generating from OpenAPI (hand-transcribing is not allowed). For TS libraries, prefer TypeDoc auto-generation. Python / PHP are handled with hand-written templates plus a consistency check.
|
|
14
|
+
- Hand-written reference pages must include a `source: src/path/file.ts#functionName` key in the frontmatter.
|
|
15
|
+
- Links to tests use a repository-absolute URL anchored to the default branch (never pin to a commit SHA).
|
|
16
|
+
|
|
17
|
+
## 1. Basic Docusaurus i18n setup
|
|
18
|
+
|
|
19
|
+
### docusaurus.config.js
|
|
20
|
+
|
|
21
|
+
```js
|
|
22
|
+
// docusaurus.config.js (excerpt)
|
|
23
|
+
module.exports = {
|
|
24
|
+
title: 'Docs',
|
|
25
|
+
url: 'https://docs.example.com',
|
|
26
|
+
baseUrl: '/',
|
|
27
|
+
i18n: {
|
|
28
|
+
defaultLocale: 'ja', // default locale is ja
|
|
29
|
+
locales: ['ja', 'en'],
|
|
30
|
+
},
|
|
31
|
+
themeConfig: {
|
|
32
|
+
navbar: {
|
|
33
|
+
// always include a locale-switcher dropdown
|
|
34
|
+
items: [{ type: 'localeDropdown', position: 'right' }],
|
|
35
|
+
},
|
|
36
|
+
},
|
|
37
|
+
};
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### Directory structure
|
|
41
|
+
|
|
42
|
+
The source-of-truth content for the default locale (ja) lives under `docs/`, and the en translation mirrors the same structure under `i18n/en/`.
|
|
43
|
+
|
|
44
|
+
```text
|
|
45
|
+
docs/
|
|
46
|
+
guides/getting-started.md # ja original (source of truth)
|
|
47
|
+
i18n/
|
|
48
|
+
en/
|
|
49
|
+
docusaurus-plugin-content-docs/
|
|
50
|
+
current/
|
|
51
|
+
guides/getting-started.md # en translation (same relative path as docs/)
|
|
52
|
+
code.json # translations for UI strings
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
### Translation workflow
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
# 1. Generate translation files for UI strings / sidebar labels (for en)
|
|
59
|
+
npm run write-translations -- --locale en
|
|
60
|
+
|
|
61
|
+
# 2. Place the en translation of a hand-written guide under i18n/en/.../current/
|
|
62
|
+
# at the same relative path, then translate it
|
|
63
|
+
cp docs/guides/getting-started.md \
|
|
64
|
+
i18n/en/docusaurus-plugin-content-docs/current/guides/getting-started.md
|
|
65
|
+
|
|
66
|
+
# 3. Run the dev server per locale to check it (dev server serves one locale at a time)
|
|
67
|
+
npm run start -- --locale en
|
|
68
|
+
|
|
69
|
+
# 4. A production build verifies all locales together
|
|
70
|
+
npm run build
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Whenever the original (ja) content is updated, update the en translation in the same PR. Auto-generated reference pages (from OpenAPI / TypeDoc) exist only in the default locale — do not copy them into `i18n/en/`.
|
|
74
|
+
|
|
75
|
+
## 2. Generating OpenAPI output from NestJS
|
|
76
|
+
|
|
77
|
+
Prepare a generation-only script that doesn't depend on the server actually listening. It's important that this can run in CI without opening a port.
|
|
78
|
+
|
|
79
|
+
```ts
|
|
80
|
+
// scripts/generate-openapi.ts
|
|
81
|
+
import { NestFactory } from '@nestjs/core';
|
|
82
|
+
import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';
|
|
83
|
+
import { writeFileSync } from 'fs';
|
|
84
|
+
import { AppModule } from '../src/app.module';
|
|
85
|
+
|
|
86
|
+
async function generate() {
|
|
87
|
+
// Build only the application context, without calling listen()
|
|
88
|
+
const app = await NestFactory.create(AppModule, { logger: false });
|
|
89
|
+
|
|
90
|
+
const config = new DocumentBuilder()
|
|
91
|
+
.setTitle('API').setVersion('1.0.0').addBearerAuth()
|
|
92
|
+
.build();
|
|
93
|
+
|
|
94
|
+
const document = SwaggerModule.createDocument(app, config);
|
|
95
|
+
writeFileSync('openapi.json', JSON.stringify(document, null, 2));
|
|
96
|
+
|
|
97
|
+
await app.close();
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
generate();
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
```jsonc
|
|
104
|
+
// package.json (excerpt)
|
|
105
|
+
{ "scripts": { "openapi:generate": "ts-node scripts/generate-openapi.ts" } }
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Enabling the `@nestjs/swagger` CLI plugin lets it auto-infer `@ApiProperty` from DTO properties, which reduces missing-decorator mistakes.
|
|
109
|
+
|
|
110
|
+
```jsonc
|
|
111
|
+
// nest-cli.json
|
|
112
|
+
{
|
|
113
|
+
"sourceRoot": "src",
|
|
114
|
+
"compilerOptions": {
|
|
115
|
+
"plugins": [
|
|
116
|
+
{ "name": "@nestjs/swagger", "options": { "introspectComments": true } }
|
|
117
|
+
]
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
## 3. Pulling OpenAPI into Docusaurus
|
|
123
|
+
|
|
124
|
+
Use `docusaurus-plugin-openapi-docs` to generate MDX pages from `openapi.json`.
|
|
125
|
+
|
|
126
|
+
```js
|
|
127
|
+
// docusaurus.config.js (excerpt)
|
|
128
|
+
module.exports = {
|
|
129
|
+
plugins: [
|
|
130
|
+
[
|
|
131
|
+
'docusaurus-plugin-openapi-docs',
|
|
132
|
+
{
|
|
133
|
+
id: 'api',
|
|
134
|
+
docsPluginId: 'classic',
|
|
135
|
+
config: {
|
|
136
|
+
myApi: {
|
|
137
|
+
specPath: 'openapi.json', // the generated spec
|
|
138
|
+
outputDir: 'docs/api', // output location (default locale only)
|
|
139
|
+
sidebarOptions: {
|
|
140
|
+
groupPathsBy: 'tag', // group by tag
|
|
141
|
+
categoryLinkSource: 'tag',
|
|
142
|
+
},
|
|
143
|
+
},
|
|
144
|
+
},
|
|
145
|
+
},
|
|
146
|
+
],
|
|
147
|
+
],
|
|
148
|
+
themes: ['docusaurus-theme-openapi-docs'],
|
|
149
|
+
};
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
```js
|
|
153
|
+
// sidebars.js (excerpt): pull in the generated sidebar
|
|
154
|
+
module.exports = {
|
|
155
|
+
docs: [
|
|
156
|
+
{ type: 'category', label: 'Guides', items: [{ type: 'autogenerated', dirName: 'guides' }] },
|
|
157
|
+
{ type: 'category', label: 'API Reference', items: require('./docs/api/sidebar.js') },
|
|
158
|
+
],
|
|
159
|
+
};
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
# Regenerate (run this every time the API changes; clear old pages before rebuilding)
|
|
164
|
+
npm run openapi:generate # in the API repository
|
|
165
|
+
npx docusaurus clean-api-docs myApi # remove existing generated output
|
|
166
|
+
npx docusaurus gen-api-docs myApi # regenerate
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
The generated `docs/api/` content is served only in the default locale (the ja site) — do not create a translated copy under `i18n/en/`, and do not hand-edit it.
|
|
170
|
+
|
|
171
|
+
## 4. Generating Next.js types and clients from OpenAPI
|
|
172
|
+
|
|
173
|
+
Use `openapi-typescript` if you only need types, or `orval` if you also want a fetch client. In both cases, standardize the output location as `src/generated/`.
|
|
174
|
+
|
|
175
|
+
```jsonc
|
|
176
|
+
// package.json (excerpt): generate types only
|
|
177
|
+
{ "scripts": { "codegen:types": "openapi-typescript ../api/openapi.json -o src/generated/api-types.ts" } }
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
```ts
|
|
181
|
+
// orval.config.ts: generate both types and client
|
|
182
|
+
import { defineConfig } from 'orval';
|
|
183
|
+
|
|
184
|
+
export default defineConfig({
|
|
185
|
+
myApi: {
|
|
186
|
+
input: '../api/openapi.json',
|
|
187
|
+
output: {
|
|
188
|
+
target: 'src/generated/api-client.ts',
|
|
189
|
+
client: 'fetch', // use 'react-query' if you're using React Query
|
|
190
|
+
clean: true, // remove stale files on regeneration
|
|
191
|
+
},
|
|
192
|
+
},
|
|
193
|
+
});
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Enforce the "no hand-editing" rule with a two-pronged approach: exclude generated files from lint, and detect regeneration drift in CI (see section 7).
|
|
197
|
+
|
|
198
|
+
```jsonc
|
|
199
|
+
// .eslintrc.json (excerpt): don't lint generated output — this signals it isn't meant to be hand-edited
|
|
200
|
+
{
|
|
201
|
+
"ignorePatterns": ["src/generated/**"]
|
|
202
|
+
}
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
`src/generated/` should still be committed (so reviewers can see the diff), but a PR containing manual edits to it will fail the CI freshness check.
|
|
206
|
+
|
|
207
|
+
## 5. Pulling TypeDoc into Docusaurus
|
|
208
|
+
|
|
209
|
+
```js
|
|
210
|
+
// docusaurus.config.js (excerpt)
|
|
211
|
+
module.exports = {
|
|
212
|
+
plugins: [
|
|
213
|
+
[
|
|
214
|
+
'docusaurus-plugin-typedoc',
|
|
215
|
+
{
|
|
216
|
+
entryPoints: ['../packages/sdk/src/index.ts'],
|
|
217
|
+
tsconfig: '../packages/sdk/tsconfig.json',
|
|
218
|
+
out: 'docs/sdk', // output location (default locale only)
|
|
219
|
+
sidebar: { autoConfiguration: true },
|
|
220
|
+
},
|
|
221
|
+
],
|
|
222
|
+
],
|
|
223
|
+
};
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
```js
|
|
227
|
+
// sidebars.js (excerpt): pull in the sidebar generated by TypeDoc
|
|
228
|
+
module.exports = {
|
|
229
|
+
docs: [
|
|
230
|
+
{ type: 'category', label: 'SDK Reference', items: require('./docs/sdk/typedoc-sidebar.cjs') },
|
|
231
|
+
],
|
|
232
|
+
};
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Embed links to tests via `@see` in TSDoc comments. Links should use an absolute URL anchored to the default branch (never pin to a SHA).
|
|
236
|
+
|
|
237
|
+
```ts
|
|
238
|
+
/**
|
|
239
|
+
* Calculates the order total including tax.
|
|
240
|
+
* @param items - array of order line items
|
|
241
|
+
* @see {@link https://github.com/example-org/sdk/blob/main/test/order.spec.ts | order.spec.ts test cases}
|
|
242
|
+
*/
|
|
243
|
+
export function calcOrderTotal(items: OrderItem[]): number { /* ... */ }
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
## 6. Full template for a hand-written function reference
|
|
247
|
+
|
|
248
|
+
For languages such as Python / PHP where auto-generation isn't available. The `source` frontmatter key is required so the consistency check (`/update-docs`) can trace back to the implementation.
|
|
249
|
+
|
|
250
|
+
```markdown
|
|
251
|
+
---
|
|
252
|
+
id: calc-order-total
|
|
253
|
+
title: calc_order_total
|
|
254
|
+
source: src/services/order.py#calc_order_total
|
|
255
|
+
---
|
|
256
|
+
|
|
257
|
+
## Overview
|
|
258
|
+
|
|
259
|
+
Calculates the tax-inclusive total from an array of order line items.
|
|
260
|
+
|
|
261
|
+
## Signature
|
|
262
|
+
|
|
263
|
+
\`\`\`python
|
|
264
|
+
def calc_order_total(items: list[OrderItem], tax_rate: Decimal = Decimal("0.10")) -> int
|
|
265
|
+
\`\`\`
|
|
266
|
+
|
|
267
|
+
## Parameters
|
|
268
|
+
|
|
269
|
+
| Name | Type | Required | Description |
|
|
270
|
+
| --- | --- | --- | --- |
|
|
271
|
+
| items | list[OrderItem] | Yes | Array of order line items. An empty array is allowed |
|
|
272
|
+
| tax_rate | Decimal | No | Tax rate. Defaults to 0.10 |
|
|
273
|
+
|
|
274
|
+
## Output
|
|
275
|
+
|
|
276
|
+
The tax-inclusive total (in yen, as an int). Returns 0 for an empty array.
|
|
277
|
+
|
|
278
|
+
## Errors
|
|
279
|
+
|
|
280
|
+
| Exception | Code | Condition |
|
|
281
|
+
| --- | --- | --- |
|
|
282
|
+
| ValueError | E_NEGATIVE_PRICE | A line item has a negative unit price |
|
|
283
|
+
| ValueError | E_INVALID_TAX_RATE | tax_rate is less than 0 or greater than or equal to 1 |
|
|
284
|
+
|
|
285
|
+
## Example
|
|
286
|
+
|
|
287
|
+
\`\`\`python
|
|
288
|
+
total = calc_order_total(items, tax_rate=Decimal("0.10")) # => 1100
|
|
289
|
+
\`\`\`
|
|
290
|
+
|
|
291
|
+
## Tests
|
|
292
|
+
|
|
293
|
+
- [tests/services/test_order.py](https://github.com/example-org/shop-api/blob/main/tests/services/test_order.py)
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
As with hand-written guides, this kind of reference page should also be prepared in both ja and en. When the implementation changes, use the `source` key as the anchor for finding and updating the corresponding page.
|
|
297
|
+
|
|
298
|
+
## 7. Example CI freshness check implementation
|
|
299
|
+
|
|
300
|
+
The principle is: "regenerate it, and fail if there's a diff." Apply this to both `openapi.json` and the generated client.
|
|
301
|
+
|
|
302
|
+
```yaml
|
|
303
|
+
# .github/workflows/docs-freshness.yml
|
|
304
|
+
name: docs-freshness
|
|
305
|
+
on: [pull_request]
|
|
306
|
+
|
|
307
|
+
jobs:
|
|
308
|
+
freshness:
|
|
309
|
+
runs-on: ubuntu-latest
|
|
310
|
+
steps:
|
|
311
|
+
- uses: actions/checkout@v4
|
|
312
|
+
- uses: actions/setup-node@v4
|
|
313
|
+
with: { node-version: 20, cache: npm }
|
|
314
|
+
- run: npm ci
|
|
315
|
+
# Regenerate openapi.json and check it matches what's committed
|
|
316
|
+
- run: npm run openapi:generate
|
|
317
|
+
- run: git diff --exit-code openapi.json
|
|
318
|
+
# Regenerate the client/types and check src/generated/ has no diff
|
|
319
|
+
- run: npm run codegen:types && npx orval
|
|
320
|
+
- run: git diff --exit-code src/generated/
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
The same idea applies in GitLab CI — just line up "regenerate → `git diff --exit-code`" as script steps in the job.
|
|
324
|
+
|
|
325
|
+
This check catches both failure modes: (a) a PR that changed the API but forgot to update `openapi.json`, and (b) a PR that hand-edited generated output.
|
|
326
|
+
|
|
327
|
+
## 8. Anti-patterns
|
|
328
|
+
|
|
329
|
+
| Anti-pattern | Problem | Do this instead |
|
|
330
|
+
| --- | --- | --- |
|
|
331
|
+
| Hand-writing API response types | Falls out of sync with API changes; types and reality drift apart | Generate into `src/generated/` with openapi-typescript / orval |
|
|
332
|
+
| Hand-editing generated output (`docs/api`, `src/generated/`) | Gets wiped out on the next regeneration, and fails the CI freshness check anyway | Edit the source of truth (decorators, TSDoc, the spec) and regenerate |
|
|
333
|
+
| Test links pinned to a commit SHA | Keeps pointing at old code and drifts from the current state | Use an absolute URL anchored to the default branch |
|
|
334
|
+
| Leaving translations behind (only updating the ja original) | English readers keep seeing stale information | Update ja and en together in the same PR for hand-written guides |
|
|
335
|
+
| Orphan pages missing from the sidebar | Unreachable except via search; nobody knows they exist | Always register pages in sidebars.js (either autogenerated or explicit) |
|
|
336
|
+
|
|
337
|
+
## Related
|
|
338
|
+
|
|
339
|
+
- Source of truth for conventions: `rules/documentation/` (for repository maintainers)
|
|
340
|
+
- API design guidance itself: the api-design skill
|
|
341
|
+
- Running the implementation/documentation consistency check: `/update-docs`
|
|
@@ -0,0 +1,334 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: e2e-testing
|
|
3
|
+
description: A pattern collection for Playwright-based E2E testing. Covers how to choose what to test in Next.js/Vue apps, selector strategy, wait strategy, test structure, test data management, flaky test countermeasures, and CI integration. Use when writing new E2E tests, reviewing E2E tests, investigating flaky tests, or analyzing test failures in CI.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# E2E Testing Patterns (Playwright / Next.js / Vue)
|
|
7
|
+
|
|
8
|
+
E2E tests should be "few and elite." This skill assumes Playwright and
|
|
9
|
+
collects patterns — and anti-patterns — for writing E2E tests that stay
|
|
10
|
+
robust and easy to maintain.
|
|
11
|
+
|
|
12
|
+
## 1. What should you test with E2E?
|
|
13
|
+
|
|
14
|
+
### Scope it to critical business flows
|
|
15
|
+
|
|
16
|
+
E2E tests are expensive to run and expensive to maintain. Don't aim for
|
|
17
|
+
exhaustive coverage — target only "the flows that would directly hurt the
|
|
18
|
+
business if they broke."
|
|
19
|
+
|
|
20
|
+
Things worth covering:
|
|
21
|
+
|
|
22
|
+
- Login/logout and screen transitions involving authentication
|
|
23
|
+
- Flows that finalize money or data — applications, orders, payments
|
|
24
|
+
- End-to-end paths for major forms: submission through list/detail confirmation
|
|
25
|
+
- Permission-based differences in what's shown or actionable (e.g. admin vs. regular user)
|
|
26
|
+
|
|
27
|
+
Things that should NOT be E2E tests (leave these to unit/component tests):
|
|
28
|
+
|
|
29
|
+
- Every variation of a validation message
|
|
30
|
+
- Pure logic such as date formatting or amount calculations
|
|
31
|
+
- Individual component rendering variations (combinations of props)
|
|
32
|
+
- Fine-grained branches of error handling
|
|
33
|
+
|
|
34
|
+
### Where to draw the line with unit tests
|
|
35
|
+
|
|
36
|
+
| What you want to verify | Appropriate layer |
|
|
37
|
+
|---|---|
|
|
38
|
+
| Function/class logic | Unit test |
|
|
39
|
+
| Component rendering/events | Component test |
|
|
40
|
+
| API input/output contract | API test |
|
|
41
|
+
| Business flow spanning multiple screens | E2E test |
|
|
42
|
+
|
|
43
|
+
Always ask yourself "can this only be verified through E2E?" Don't push
|
|
44
|
+
something into E2E that could be written at a lower layer.
|
|
45
|
+
|
|
46
|
+
## 2. Selector strategy
|
|
47
|
+
|
|
48
|
+
### Priority order
|
|
49
|
+
|
|
50
|
+
Prefer attributes that are visible to the user. Work through this list top to bottom:
|
|
51
|
+
|
|
52
|
+
1. Role: `page.getByRole('button', { name: 'Save' })`
|
|
53
|
+
2. Label: `page.getByLabel('Email address')`
|
|
54
|
+
3. Placeholder/text: `page.getByPlaceholder(...)` / `page.getByText(...)`
|
|
55
|
+
4. Test ID: `page.getByTestId('order-summary')`
|
|
56
|
+
|
|
57
|
+
Always use role and label when an element can be identified that way —
|
|
58
|
+
it doubles as an accessibility check. Reserve test IDs for elements with
|
|
59
|
+
no visual cue (containers, rows in a dynamic list, etc.).
|
|
60
|
+
|
|
61
|
+
```typescript
|
|
62
|
+
// Good: identify by the name the user perceives
|
|
63
|
+
await page.getByRole('textbox', { name: 'Email address' }).fill('a@example.com');
|
|
64
|
+
await page.getByRole('button', { name: 'Log in' }).click();
|
|
65
|
+
|
|
66
|
+
// Acceptable: use a test ID for a row with no visual cue
|
|
67
|
+
const row = page.getByTestId('user-row-123');
|
|
68
|
+
await row.getByRole('button', { name: 'Edit' }).click();
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
### No selectors that depend on CSS structure
|
|
72
|
+
|
|
73
|
+
Selectors that depend on DOM structure or framework-generated class names
|
|
74
|
+
are forbidden. They break easily during refactors, and when they break
|
|
75
|
+
it's not obvious why.
|
|
76
|
+
|
|
77
|
+
```typescript
|
|
78
|
+
// Forbidden: depends on structure / auto-generated classes
|
|
79
|
+
await page.locator('div.container > div:nth-child(2) > button').click();
|
|
80
|
+
await page.locator('.css-1q2w3e4').click(); // CSS-in-JS generated class
|
|
81
|
+
await page.locator('#app > main span').first().click(); // depends on ordering
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
This also applies to attributes generated by Vue's scoped CSS
|
|
85
|
+
(`data-v-xxxxx`) and to Next.js/Tailwind utility classes — don't depend on
|
|
86
|
+
those either.
|
|
87
|
+
|
|
88
|
+
## 3. Wait strategy
|
|
89
|
+
|
|
90
|
+
### Trust auto-waiting and web-first assertions
|
|
91
|
+
|
|
92
|
+
Playwright's locator actions and `expect(locator)`-style assertions
|
|
93
|
+
automatically retry until the element is actionable or the condition
|
|
94
|
+
holds. Rely on this by default.
|
|
95
|
+
|
|
96
|
+
```typescript
|
|
97
|
+
// Good: an assertion that retries automatically
|
|
98
|
+
await expect(page.getByRole('heading', { name: 'Order complete' })).toBeVisible();
|
|
99
|
+
await expect(page.getByTestId('cart-count')).toHaveText('3');
|
|
100
|
+
|
|
101
|
+
// Forbidden: a fixed sleep. Flaky on slow environments, wasteful on fast ones
|
|
102
|
+
await page.waitForTimeout(3000);
|
|
103
|
+
|
|
104
|
+
// Forbidden: an assertion evaluated immediately, with no retry — vulnerable to races
|
|
105
|
+
expect(await page.getByTestId('cart-count').textContent()).toBe('3');
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Also avoid depending on `networkidle`. In Next.js/Vue apps that use
|
|
109
|
+
polling or SSE, the network may never go idle — or idle may trigger too
|
|
110
|
+
early. What you should wait for isn't network silence, but the result
|
|
111
|
+
that's visible to the user.
|
|
112
|
+
|
|
113
|
+
### Arm response waits before the action, not after
|
|
114
|
+
|
|
115
|
+
When you need to wait for a specific response, start `waitForResponse`
|
|
116
|
+
*before* the click that triggers it. If you start waiting afterward, you
|
|
117
|
+
can lose the race and miss it.
|
|
118
|
+
|
|
119
|
+
```typescript
|
|
120
|
+
// Good: arm the wait first, then perform the action
|
|
121
|
+
const responsePromise = page.waitForResponse(
|
|
122
|
+
(res) => res.url().includes('/api/orders') && res.status() === 201
|
|
123
|
+
);
|
|
124
|
+
await page.getByRole('button', { name: 'Confirm order' }).click();
|
|
125
|
+
await responsePromise;
|
|
126
|
+
await expect(page.getByText('Your order has been received')).toBeVisible();
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
## 4. Test structure
|
|
130
|
+
|
|
131
|
+
### Test independence
|
|
132
|
+
|
|
133
|
+
Every test must be able to run alone, in any order.
|
|
134
|
+
|
|
135
|
+
- Don't carry variables, created data, or login state across tests
|
|
136
|
+
- Set up prerequisite data at the start of each test (or in `beforeEach`) — don't rely on shared fixtures
|
|
137
|
+
- Never let one test's "output" become another test's "input"
|
|
138
|
+
- Avoid `test.describe.serial` as a rule; if a sequence is truly required, fold it into a single test
|
|
139
|
+
|
|
140
|
+
```typescript
|
|
141
|
+
test.beforeEach(async ({ page }) => {
|
|
142
|
+
// Each test sets up its own prerequisites, independent of other tests' results
|
|
143
|
+
await page.goto('/projects');
|
|
144
|
+
});
|
|
145
|
+
|
|
146
|
+
test('can create a project', async ({ page }) => { /* ... */ });
|
|
147
|
+
test('can search for a project', async ({ page }) => {
|
|
148
|
+
// The data to search for is created within this test (or via API setup)
|
|
149
|
+
});
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
### Use the Page Object Model in moderation
|
|
153
|
+
|
|
154
|
+
Use a Page Object to consolidate "a sequence of operations repeated
|
|
155
|
+
across multiple tests." Don't build a giant class that wraps every
|
|
156
|
+
element on the screen.
|
|
157
|
+
|
|
158
|
+
```typescript
|
|
159
|
+
// Good: a thin Page Object that exposes only business-level operations
|
|
160
|
+
export class LoginPage {
|
|
161
|
+
constructor(private page: Page) {}
|
|
162
|
+
|
|
163
|
+
async login(email: string, password: string) {
|
|
164
|
+
await this.page.goto('/login');
|
|
165
|
+
await this.page.getByLabel('Email address').fill(email);
|
|
166
|
+
await this.page.getByLabel('Password').fill(password);
|
|
167
|
+
await this.page.getByRole('button', { name: 'Log in' }).click();
|
|
168
|
+
}
|
|
169
|
+
}
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Signs of over-abstraction (avoid these):
|
|
173
|
+
|
|
174
|
+
- A pile of methods that are each called from only one test
|
|
175
|
+
- Assertions buried inside the Page Object, making failures hard to trace
|
|
176
|
+
- Three layers of wrapper around what could be a single `getByRole` call
|
|
177
|
+
|
|
178
|
+
Only extract a shared helper after you've repeated something two or three
|
|
179
|
+
times. Don't abstract preemptively.
|
|
180
|
+
|
|
181
|
+
## 5. Test data management
|
|
182
|
+
|
|
183
|
+
### Generate unique data
|
|
184
|
+
|
|
185
|
+
Fixed-value data collides under parallel execution or retries. Generate a
|
|
186
|
+
value that's unique per run.
|
|
187
|
+
|
|
188
|
+
```typescript
|
|
189
|
+
// Include a unique identifier for each run
|
|
190
|
+
const unique = `${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
|
191
|
+
const email = `e2e-${unique}@example.com`;
|
|
192
|
+
const projectName = `E2E Test Project-${unique}`;
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Prioritize "a design that can't collide" over cleanup after the fact.
|
|
196
|
+
Make sure that even if cleanup fails, it doesn't drag other tests down
|
|
197
|
+
with it.
|
|
198
|
+
|
|
199
|
+
### Reuse authentication state (storageState)
|
|
200
|
+
|
|
201
|
+
Don't repeat a UI login in every test. Log in once in a setup project,
|
|
202
|
+
save the resulting state, and reuse it across all tests.
|
|
203
|
+
|
|
204
|
+
```typescript
|
|
205
|
+
// auth.setup.ts: create and save authentication state once
|
|
206
|
+
import { test as setup } from '@playwright/test';
|
|
207
|
+
|
|
208
|
+
setup('save authentication state', async ({ page }) => {
|
|
209
|
+
await page.goto('/login');
|
|
210
|
+
await page.getByLabel('Email address').fill(process.env.E2E_USER!);
|
|
211
|
+
await page.getByLabel('Password').fill(process.env.E2E_PASS!);
|
|
212
|
+
await page.getByRole('button', { name: 'Log in' }).click();
|
|
213
|
+
await page.waitForURL('/dashboard');
|
|
214
|
+
await page.context().storageState({ path: 'playwright/.auth/user.json' });
|
|
215
|
+
});
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
```typescript
|
|
219
|
+
// playwright.config.ts: make setup a dependency and reuse the storageState
|
|
220
|
+
projects: [
|
|
221
|
+
{ name: 'setup', testMatch: /auth\.setup\.ts/ },
|
|
222
|
+
{
|
|
223
|
+
name: 'chromium',
|
|
224
|
+
use: { storageState: 'playwright/.auth/user.json' },
|
|
225
|
+
dependencies: ['setup'],
|
|
226
|
+
},
|
|
227
|
+
],
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
The one exception is testing the login feature itself — that should still
|
|
231
|
+
go through the UI rather than reusing `storageState`.
|
|
232
|
+
|
|
233
|
+
### Mock external APIs (route)
|
|
234
|
+
|
|
235
|
+
Mock external APIs you don't control (payment providers, maps, other
|
|
236
|
+
external SaaS) with `page.route` to make the test deterministic.
|
|
237
|
+
|
|
238
|
+
```typescript
|
|
239
|
+
// Mock the external payment API and pin down a success response
|
|
240
|
+
await page.route('**/api/external/payment', async (route) => {
|
|
241
|
+
await route.fulfill({
|
|
242
|
+
status: 200,
|
|
243
|
+
contentType: 'application/json',
|
|
244
|
+
body: JSON.stringify({ status: 'succeeded', id: 'pay_test_001' }),
|
|
245
|
+
});
|
|
246
|
+
});
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
Don't mock your own backend as a rule. The moment you do, everything
|
|
250
|
+
downstream stops being an E2E test. Only mock things outside your own
|
|
251
|
+
system boundary.
|
|
252
|
+
|
|
253
|
+
## 6. Flaky test countermeasures
|
|
254
|
+
|
|
255
|
+
### Symptom-to-cause-to-fix table
|
|
256
|
+
|
|
257
|
+
| Symptom | Typical cause | Fix |
|
|
258
|
+
|---|---|---|
|
|
259
|
+
| Passes locally but fails in CI | Fixed sleeps, dependence on machine speed | Replace with web-first assertions |
|
|
260
|
+
| Element occasionally not found | Evaluated immediately, before rendering completes | Rely on `expect(locator)`'s auto-retry |
|
|
261
|
+
| Fails only under parallel execution | Data collisions between tests | Generate unique data; restore independence |
|
|
262
|
+
| Fails when order changes | Depends on a previous test's result | Have each test set up its own prerequisites |
|
|
263
|
+
| Click doesn't register | Competing with an animation or overlay | Assert the target is stably visible before acting |
|
|
264
|
+
| Can't capture the response you're waiting for | `waitForResponse` armed after the action | Arm the wait before the action |
|
|
265
|
+
|
|
266
|
+
### Where retries fit in
|
|
267
|
+
|
|
268
|
+
Retries (`retries`) are a safety net for absorbing transient
|
|
269
|
+
infrastructure failures — they are not a "cure" for flakiness. It's fine
|
|
270
|
+
to set something like `retries: 2` in CI, but any test that only passes
|
|
271
|
+
after a retry should be logged as flaky and investigated. "It has
|
|
272
|
+
retries, so we don't need to fix it" is not an acceptable conclusion.
|
|
273
|
+
|
|
274
|
+
### Quarantining requires a ticket number
|
|
275
|
+
|
|
276
|
+
If root-causing takes time, you may temporarily quarantine a test — but
|
|
277
|
+
only with a ticket number attached. A `skip`/`fixme` with no ticket
|
|
278
|
+
number should be sent back in review.
|
|
279
|
+
|
|
280
|
+
```typescript
|
|
281
|
+
// Always leave a ticket number and reason. skip without one is forbidden
|
|
282
|
+
test.fixme('can export order history as CSV', async ({ page }) => {
|
|
283
|
+
// TICKET-1234: investigating an issue where the download event isn't captured in CI only
|
|
284
|
+
});
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
## 7. CI integration
|
|
288
|
+
|
|
289
|
+
### Traces and screenshots on failure
|
|
290
|
+
|
|
291
|
+
Always retain the evidence needed to reproduce a failure. Recording
|
|
292
|
+
everything all the time wastes storage and run time, so limit it to
|
|
293
|
+
failures and retries.
|
|
294
|
+
|
|
295
|
+
```typescript
|
|
296
|
+
// playwright.config.ts: keep evidence only on failure
|
|
297
|
+
export default defineConfig({
|
|
298
|
+
retries: process.env.CI ? 2 : 0,
|
|
299
|
+
use: {
|
|
300
|
+
trace: 'on-first-retry', // record a trace on retry
|
|
301
|
+
screenshot: 'only-on-failure', // screenshot only on failure
|
|
302
|
+
video: 'retain-on-failure', // keep video only on failure
|
|
303
|
+
},
|
|
304
|
+
reporter: process.env.CI ? [['html', { open: 'never' }], ['github']] : 'list',
|
|
305
|
+
});
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
In CI, save the trace and HTML report as artifacts, and investigate
|
|
309
|
+
failures with `npx playwright show-trace` rather than guessing from logs
|
|
310
|
+
alone.
|
|
311
|
+
|
|
312
|
+
### Parallelism and sharding basics
|
|
313
|
+
|
|
314
|
+
- Control in-machine parallelism with workers (`workers`). If tests are
|
|
315
|
+
truly independent, the default parallel execution works as-is
|
|
316
|
+
- As the test count grows, split across multiple machines:
|
|
317
|
+
`npx playwright test --shard=1/4`
|
|
318
|
+
- If a test starts failing under parallel execution, fix the test's
|
|
319
|
+
independence (data collisions, shared state) rather than lowering
|
|
320
|
+
parallelism
|
|
321
|
+
|
|
322
|
+
```yaml
|
|
323
|
+
# CI example: matrix run with 4-way sharding
|
|
324
|
+
strategy:
|
|
325
|
+
matrix:
|
|
326
|
+
shard: [1/4, 2/4, 3/4, 4/4]
|
|
327
|
+
steps:
|
|
328
|
+
- run: npx playwright test --shard=${{ matrix.shard }}
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
## Related
|
|
332
|
+
|
|
333
|
+
- Use `/test-coverage` to check coverage
|
|
334
|
+
- Use `/code-review` to review implementation and test code
|