@tamsensedev/runner 0.13.0 → 0.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (82) hide show
  1. package/LICENSE +5 -5
  2. package/README.md +76 -57
  3. package/dist/cli.js +757 -379
  4. package/dist/cli.js.map +1 -1
  5. package/dist/recorder.cjs.map +1 -1
  6. package/library/.claude-plugin/plugin.json +5 -5
  7. package/library/THIRD_PARTY_NOTICES.md +123 -123
  8. package/library/agents/code-architect.md +67 -67
  9. package/library/agents/code-explorer.md +61 -61
  10. package/library/agents/code-reviewer.md +73 -73
  11. package/library/agents/code-simplifier.md +62 -62
  12. package/library/agents/design-critic.md +52 -52
  13. package/library/agents/second-opinion.md +64 -64
  14. package/library/agents/security-auditor.md +86 -86
  15. package/library/agents/silent-failure-hunter.md +70 -70
  16. package/library/agents/test-coverage-analyzer.md +78 -78
  17. package/library/agents/test-runner.md +48 -48
  18. package/library/agents/ui-tester.md +50 -50
  19. package/library/licenses/Apache-2.0.txt +202 -202
  20. package/library/skills/accessibility/SKILL.md +144 -144
  21. package/library/skills/accessibility/references/wcag-checks.md +168 -168
  22. package/library/skills/addressing-review-feedback/SKILL.md +109 -109
  23. package/library/skills/app-screens/SKILL.md +96 -96
  24. package/library/skills/app-screens/references/shell-kpis-charts.md +108 -108
  25. package/library/skills/app-screens/references/states-density-phone.md +54 -54
  26. package/library/skills/app-screens/references/tables-forms-settings.md +104 -104
  27. package/library/skills/avoiding-generic-ai-design/SKILL.md +92 -92
  28. package/library/skills/avoiding-generic-ai-design/references/tells.md +112 -112
  29. package/library/skills/clean-commits/SKILL.md +85 -85
  30. package/library/skills/design-review/SKILL.md +107 -107
  31. package/library/skills/design-review/references/critique-passes.md +101 -101
  32. package/library/skills/design-review/references/screenshot-procedure.md +121 -121
  33. package/library/skills/differential-security-review/LICENSE +427 -427
  34. package/library/skills/differential-security-review/SKILL.md +105 -105
  35. package/library/skills/differential-security-review/references/adversarial.md +115 -115
  36. package/library/skills/differential-security-review/references/methodology.md +145 -145
  37. package/library/skills/differential-security-review/references/patterns.md +108 -108
  38. package/library/skills/false-positive-check/LICENSE +427 -427
  39. package/library/skills/false-positive-check/SKILL.md +95 -95
  40. package/library/skills/false-positive-check/references/bug-class-verification.md +128 -128
  41. package/library/skills/false-positive-check/references/deep-verification.md +113 -113
  42. package/library/skills/false-positive-check/references/evidence-templates.md +93 -93
  43. package/library/skills/false-positive-check/references/false-positive-patterns.md +122 -122
  44. package/library/skills/false-positive-check/references/gate-reviews.md +36 -36
  45. package/library/skills/false-positive-check/references/standard-verification.md +80 -80
  46. package/library/skills/focused-implementation/SKILL.md +104 -104
  47. package/library/skills/implementation-plan/SKILL.md +66 -66
  48. package/library/skills/interface-design/SKILL.md +144 -144
  49. package/library/skills/interface-design/references/craft-defaults.md +130 -130
  50. package/library/skills/interface-design/references/new-direction.md +62 -62
  51. package/library/skills/interface-design/references/writing.md +52 -52
  52. package/library/skills/interface-polish/SKILL.md +103 -103
  53. package/library/skills/interface-polish/references/detail-rules.md +154 -154
  54. package/library/skills/interface-polish/references/worst-case-data.md +158 -158
  55. package/library/skills/motion-design/SKILL.md +106 -106
  56. package/library/skills/motion-design/references/recipes.md +243 -243
  57. package/library/skills/postgres-performance/SKILL.md +97 -97
  58. package/library/skills/postgres-performance/references/rules.md +265 -265
  59. package/library/skills/project-orientation/SKILL.md +77 -77
  60. package/library/skills/reviewing-changes/SKILL.md +183 -183
  61. package/library/skills/safe-database-changes/SKILL.md +111 -111
  62. package/library/skills/safe-database-changes/references/backfills.md +75 -75
  63. package/library/skills/safe-database-changes/references/migration-tools.md +69 -69
  64. package/library/skills/safe-database-changes/references/postgres-ddl.md +199 -199
  65. package/library/skills/security-review/SKILL.md +132 -132
  66. package/library/skills/security-review/references/checks-by-area.md +107 -107
  67. package/library/skills/security-review/references/do-not-report.md +86 -86
  68. package/library/skills/spec-writing/SKILL.md +144 -144
  69. package/library/skills/systematic-debugging/SKILL.md +130 -130
  70. package/library/skills/systematic-debugging/references/flaky-tests.md +51 -51
  71. package/library/skills/systematic-debugging/references/root-cause-tracing.md +55 -55
  72. package/library/skills/task-interview/SKILL.md +84 -84
  73. package/library/skills/test-driven-development/SKILL.md +116 -116
  74. package/library/skills/test-driven-development/references/writing-good-tests.md +85 -85
  75. package/library/skills/ui-testing/SKILL.md +102 -102
  76. package/library/skills/ui-testing/references/headless-check.md +295 -295
  77. package/library/skills/ui-testing/references/local-stand.md +51 -51
  78. package/library/skills/ui-testing/references/playwright-patterns.md +134 -134
  79. package/library/skills/ui-testing/references/recording.md +61 -61
  80. package/library/skills/verification-before-completion/SKILL.md +113 -113
  81. package/library/skills/web-interface-guidelines/SKILL.md +202 -202
  82. package/package.json +46 -46
@@ -1,51 +1,51 @@
1
- # Starting a local copy of the app
2
-
3
- Read this before you start the app for a check. In a TAMsense run the task folder is a fresh checkout: no dependencies, no `.env`, nothing running. You set up a local copy from this checkout yourself, every time, and you take it down at the end. The person's own setup, their databases and every shared server stay untouched.
4
-
5
- ## The rules
6
-
7
- 1. **This task folder only.** Everything you start comes from the checkout in the task folder. Never read or copy `.env` files, keys or settings from elsewhere on this computer, including the person's own checkout of the same project.
8
- 2. **Find the local setup yourself.** Look in the README, compose files named local or dev (`docker-compose.local.yml`, `infra/compose/local.compose.yml`, `compose.dev.yaml`), `.env.example` and other env files meant for a local run (`.env.local.example`, `.env.development`), and the package scripts (`dev`, `db:migrate`, `db:seed`). The CI workflow often shows the exact services and commands.
9
- 3. **Check every address before you start.** In the settings you use, databases, caches, queues and APIs must point at `localhost`, `127.0.0.1` or containers you start. A staging, shared dev or production address, or a cloud host you do not start, means stop (rule 7). Keys of outside services (payments, mail, analytics, storage, AI APIs) stay empty; never fill them with real ones. Check variables by name and address only; never print secrets.
10
- 4. **Your own compose project on free ports.** Start containers as a separate compose project named `tamsense-task-<task id>`, so they never mix with the person's containers. Map them to free host ports with an override file in a temp folder, not by editing the project's files. Find a free port by asking the system (`node -e "const s=require('net').createServer().listen(0,()=>{console.log(s.address().port);s.close()})"`).
11
- 5. **Never touch containers you did not start.** Do not stop, restart, remove or change other containers, volumes or networks, even when a port is taken: pick another port.
12
- 6. **Clean up.** At the end run `docker compose -p tamsense-task-<task id> down -v` with the same files, stop the app by its process id, and delete the temp folder.
13
- 7. **When it cannot run, stop.** If Docker is not running, the project has no local setup, or the settings point outside this computer, do not start anything and do not look for a workaround. In a check step, call `submit_verdict` with `passed: false` and `couldNotRun: true`, write the exact reason and what a person should do in the report ("Docker is not running: start Docker Desktop and run this step again"), and do not hand off. The task then waits on this step for a person. In a code step, say so in your report.
14
-
15
- The browser adds a guard of its own: requests other than GET, HEAD and OPTIONS to an address outside this computer are aborted (see [recording.md](recording.md)). It does not replace rule 3: the app's server talks to its database and services directly.
16
-
17
- ## Example
18
-
19
- Task 40, a project with `infra/compose/local.compose.yml` (postgres and redis), `.env.example` pointing at `localhost:5432` and `localhost:6379`, and a `dev` script.
20
-
21
- ```bash
22
- docker info --format '{{.ServerVersion}}' # an error here means Docker is not running: stop (rule 7)
23
- TMP=$(node -p "require('fs').mkdtempSync(require('path').join(require('os').tmpdir(), 'tamsense-task-40-'))")
24
-
25
- # free host ports for the two services, found as in rule 4
26
- cat > "$TMP/ports.override.yml" <<'EOF'
27
- services:
28
- postgres:
29
- ports: !override
30
- - "55432:5432"
31
- redis:
32
- ports: !override
33
- - "56379:6379"
34
- EOF
35
-
36
- docker compose -p tamsense-task-40 -f infra/compose/local.compose.yml -f "$TMP/ports.override.yml" up -d postgres redis
37
-
38
- pnpm install --frozen-lockfile
39
- # settings from .env.example, with the ports changed to the free ones, passed as variables, not written into the repository
40
- export DATABASE_URL=postgresql://postgres:postgres@localhost:55432/app REDIS_URL=redis://localhost:56379
41
- pnpm db:migrate # migrations go to the local database only
42
- BROWSER=none pnpm dev --port 53100 # the project's own dev script on a free port; note its process id
43
-
44
- # ... the headless check against http://localhost:53100 ...
45
-
46
- # stop the dev server by its process id, then
47
- docker compose -p tamsense-task-40 -f infra/compose/local.compose.yml -f "$TMP/ports.override.yml" down -v
48
- rm -rf "$TMP"
49
- ```
50
-
51
- Before `db:migrate`, look at the address once more: `DATABASE_URL` must name `localhost` and the port of your container. `!override` (Docker Compose 2.24 or newer) replaces the project's own port list instead of adding to it, so nothing binds the ports the person's own containers may use.
1
+ # Starting a local copy of the app
2
+
3
+ Read this before you start the app for a check. In a TAMsense run the task folder is a fresh checkout: no dependencies, no `.env`, nothing running. You set up a local copy from this checkout yourself, every time, and you take it down at the end. The person's own setup, their databases and every shared server stay untouched.
4
+
5
+ ## The rules
6
+
7
+ 1. **This task folder only.** Everything you start comes from the checkout in the task folder. Never read or copy `.env` files, keys or settings from elsewhere on this computer, including the person's own checkout of the same project.
8
+ 2. **Find the local setup yourself.** Look in the README, compose files named local or dev (`docker-compose.local.yml`, `infra/compose/local.compose.yml`, `compose.dev.yaml`), `.env.example` and other env files meant for a local run (`.env.local.example`, `.env.development`), and the package scripts (`dev`, `db:migrate`, `db:seed`). The CI workflow often shows the exact services and commands.
9
+ 3. **Check every address before you start.** In the settings you use, databases, caches, queues and APIs must point at `localhost`, `127.0.0.1` or containers you start. A staging, shared dev or production address, or a cloud host you do not start, means stop (rule 7). Keys of outside services (payments, mail, analytics, storage, AI APIs) stay empty; never fill them with real ones. Check variables by name and address only; never print secrets.
10
+ 4. **Your own compose project on free ports.** Start containers as a separate compose project named `tamsense-task-<task id>`, so they never mix with the person's containers. Map them to free host ports with an override file in a temp folder, not by editing the project's files. Find a free port by asking the system (`node -e "const s=require('net').createServer().listen(0,()=>{console.log(s.address().port);s.close()})"`).
11
+ 5. **Never touch containers you did not start.** Do not stop, restart, remove or change other containers, volumes or networks, even when a port is taken: pick another port.
12
+ 6. **Clean up.** At the end run `docker compose -p tamsense-task-<task id> down -v` with the same files, stop the app by its process id, and delete the temp folder.
13
+ 7. **When it cannot run, stop.** If Docker is not running, the project has no local setup, or the settings point outside this computer, do not start anything and do not look for a workaround. In a check step, call `submit_verdict` with `passed: false` and `couldNotRun: true`, write the exact reason and what a person should do in the report ("Docker is not running: start Docker Desktop and run this step again"), and do not hand off. The task then waits on this step for a person. In a code step, say so in your report.
14
+
15
+ The browser adds a guard of its own: requests other than GET, HEAD and OPTIONS to an address outside this computer are aborted (see [recording.md](recording.md)). It does not replace rule 3: the app's server talks to its database and services directly.
16
+
17
+ ## Example
18
+
19
+ Task 40, a project with `infra/compose/local.compose.yml` (postgres and redis), `.env.example` pointing at `localhost:5432` and `localhost:6379`, and a `dev` script.
20
+
21
+ ```bash
22
+ docker info --format '{{.ServerVersion}}' # an error here means Docker is not running: stop (rule 7)
23
+ TMP=$(node -p "require('fs').mkdtempSync(require('path').join(require('os').tmpdir(), 'tamsense-task-40-'))")
24
+
25
+ # free host ports for the two services, found as in rule 4
26
+ cat > "$TMP/ports.override.yml" <<'EOF'
27
+ services:
28
+ postgres:
29
+ ports: !override
30
+ - "55432:5432"
31
+ redis:
32
+ ports: !override
33
+ - "56379:6379"
34
+ EOF
35
+
36
+ docker compose -p tamsense-task-40 -f infra/compose/local.compose.yml -f "$TMP/ports.override.yml" up -d postgres redis
37
+
38
+ pnpm install --frozen-lockfile
39
+ # settings from .env.example, with the ports changed to the free ones, passed as variables, not written into the repository
40
+ export DATABASE_URL=postgresql://postgres:postgres@localhost:55432/app REDIS_URL=redis://localhost:56379
41
+ pnpm db:migrate # migrations go to the local database only
42
+ BROWSER=none pnpm dev --port 53100 # the project's own dev script on a free port; note its process id
43
+
44
+ # ... the headless check against http://localhost:53100 ...
45
+
46
+ # stop the dev server by its process id, then
47
+ docker compose -p tamsense-task-40 -f infra/compose/local.compose.yml -f "$TMP/ports.override.yml" down -v
48
+ rm -rf "$TMP"
49
+ ```
50
+
51
+ Before `db:migrate`, look at the address once more: `DATABASE_URL` must name `localhost` and the port of your container. `!override` (Docker Compose 2.24 or newer) replaces the project's own port list instead of adding to it, so nothing binds the ports the person's own containers may use.
@@ -1,134 +1,134 @@
1
- # Playwright patterns: locators, waiting, flakes, auth, mocking
2
-
3
- Read this when you write, fix or review browser tests or the `steps` of a headless check. In a TAMsense run every browser context, also those of the project's tests, is recorded for the task; scripts of your own name their recordings as [recording.md](recording.md) shows. Examples are Playwright Test in TypeScript; translate to the project's language and runner. If the project uses Cypress, the same ideas apply (see the last section). Check the installed version (`npx playwright --version`) before using an API you are not sure exists in it.
4
-
5
- ## Locators
6
-
7
- Use the first one that works, in this order:
8
-
9
- ```ts
10
- page.getByRole('button', { name: 'Save changes' }) // 1. role + accessible name
11
- page.getByLabel('Email address') // 2. form fields by label
12
- page.getByText('Welcome back') // 3. non-interactive text
13
- page.getByPlaceholder('Search') // 4. only if there is no label
14
- page.getByAltText('Company logo') // 5. images
15
- page.getByTestId('checkout-summary') // 6. when nothing semantic is stable
16
- page.locator('.legacy-widget button') // 7. CSS or XPath, last resort
17
- ```
18
-
19
- - Role locators survive class renames and component swaps, and they fail when the control is not accessible, which is a finding in itself.
20
- - `exact: true` stops "Log" from matching "Log out".
21
- - Scope instead of indexing: `page.getByRole('dialog', { name: 'Delete project' }).getByRole('button', { name: 'Delete' })` beats `.nth(2)`.
22
- - Rows and cards: `page.getByRole('row').filter({ hasText: 'Invoice 42' }).getByRole('button', { name: 'Open' })`.
23
- - Use the project's existing `data-testid` convention if it has one; do not add test ids to product code during a check step.
24
- - Strict mode errors ("resolved to 2 elements") mean the locator is ambiguous: scope it, do not add `.first()` to silence it.
25
-
26
- ## Waiting and assertions
27
-
28
- Actions (`click`, `fill`, `check`) already wait for the element to be attached, visible, stable and enabled. Assertions on locators retry until they pass or time out. Use them; do not sleep.
29
-
30
- | Need | Use |
31
- |---|---|
32
- | Element shown or hidden | `await expect(locator).toBeVisible()` / `.toBeHidden()` |
33
- | Text | `toHaveText` (full) or `toContainText` |
34
- | Count | `toHaveCount(n)` |
35
- | Input value, attribute, focus | `toHaveValue`, `toHaveAttribute`, `toBeFocused` |
36
- | Navigation | `await expect(page).toHaveURL(/\/dashboard/)` |
37
- | A request or response | register `page.waitForResponse(...)` **before** the action that triggers it |
38
- | A non-DOM condition | `await expect.poll(() => readStatus()).toBe('done')` |
39
-
40
- Wrong and right:
41
-
42
- ```ts
43
- await page.waitForTimeout(2000); // wrong: too slow or too short
44
- expect(await page.getByRole('alert').isVisible()).toBe(true); // wrong: checks once, no retry
45
- expect(await row.textContent()).toBe('Paid'); // wrong: no retry
46
-
47
- await expect(page.getByRole('alert')).toBeVisible(); // right
48
- await expect(row).toHaveText('Paid'); // right
49
-
50
- const saved = page.waitForResponse(r => r.url().includes('/api/profile') && r.ok());
51
- await page.getByRole('button', { name: 'Save' }).click();
52
- await saved; // right: registered before the click
53
- ```
54
-
55
- - `networkidle` is unreliable with websockets, polling or analytics. Wait for the element or URL that proves the page is ready.
56
- - Every `expect` and every action is awaited. A missing `await` makes a test pass before it checked anything.
57
- - Do not wrap assertions in try/catch; do not raise timeouts above about 30 s to make something pass.
58
-
59
- ## Flaky tests
60
-
61
- A flaky test has a cause; retries and longer timeouts only hide it. Classify first:
62
-
63
- | Symptom | Likely cause | How to confirm |
64
- |---|---|---|
65
- | Fails sometimes even alone | Timing: missing await, sleep, race between action and assertion | `npx playwright test <file> --repeat-each=10` (raise the count for rare races) |
66
- | Passes alone, fails in the suite | Isolation: shared state, shared test data, order dependence | `--grep "<name>" --workers=1` passes, full run fails |
67
- | Fails only in CI | Environment: viewport, fonts, slower machine, missing service | Compare traces and screenshots from both |
68
- | Random crashes, "Target closed" | Infrastructure: memory, browser crash | Errors mention the browser, not the app |
69
-
70
- Fixes: replace sleeps with web-first assertions; register waits before actions; give each test its own data (unique names, per-test setup); avoid module-level mutable variables; set viewport and `reducedMotion: 'reduce'` explicitly when animation or layout matters. For evidence, run with `--trace retain-on-failure --reporter=line --output <tmp>/test-results` and read the error message and the error-context and screenshot files in that output folder. Do not open `show-trace`, `show-report`, `--ui` or `--headed`: they open windows, and nobody is there to look.
71
-
72
- ## Authentication
73
-
74
- - Use the project's existing auth setup first: a setup project, a global setup that saves `storageState`, a fixture, a test-only login endpoint.
75
- - Without one, log in once per run through the UI or the API with a **test account the project documents for local development** (seed data, README, fixtures), save `storageState` to the temp folder, and reuse it.
76
- - Never use real user credentials, never read secrets the task did not give you, and never write auth state files inside the repository.
77
- - Testing the login page itself: start from an empty state (`storageState: { cookies: [], origins: [] }`).
78
- - OAuth or SSO with an external provider: do not hit the real provider; check the pages around it or report that the flow could not be exercised.
79
-
80
- ```ts
81
- // global setup or the first step of a headless check
82
- await page.goto('/login');
83
- await page.getByLabel('Email').fill(process.env.E2E_USER ?? 'demo@example.test');
84
- await page.getByLabel('Password').fill(process.env.E2E_PASSWORD ?? 'from-the-seed-file');
85
- await page.getByRole('button', { name: 'Sign in' }).click();
86
- await expect(page).toHaveURL(/dashboard/);
87
- await page.context().storageState({ path: `${tmpDir}/auth.json` });
88
- ```
89
-
90
- ## Network mocking
91
-
92
- Mock what you do not own (payment providers, email, maps, analytics) and error states you cannot trigger otherwise. Do not mock the app's own pages, scripts or styles, and prefer the real backend for the happy path.
93
-
94
- ```ts
95
- await page.route('**/api/orders', route =>
96
- route.fulfill({ status: 500, json: { error: 'Internal error' } })); // error state
97
- await page.route('**/api/orders', route => route.fulfill({ json: [] })); // empty state
98
- await page.route('**/api/orders', async route => { // slow: loading state
99
- await new Promise(r => setTimeout(r, 3000));
100
- await route.continue();
101
- });
102
- await page.route(/analytics|segment|hotjar/, route => route.abort()); // third-party noise
103
- ```
104
-
105
- - Register routes before `page.goto`.
106
- - Every handler calls exactly one of `fulfill`, `continue` or `abort`, otherwise the request hangs.
107
- - Keep globs narrow (`**/api/orders`, not `**/*`).
108
- - Loading, empty and error states are where UI changes break most often; mocking is the cheapest way to see them.
109
-
110
- ## Writing new tests (code step only)
111
-
112
- When the task changes a user flow and the project has an E2E suite:
113
-
114
- 1. **Plan:** list the scenarios from the acceptance criteria: the happy path, the main error state, the empty state, the narrow screen if layout changed. Note what each one must assert.
115
- 2. **Generate:** follow the suite's existing structure, fixtures, page objects and naming. One behaviour per test; several related `expect`s are fine.
116
- 3. **Run** the new tests several times (`--repeat-each=3` at least) and the related existing tests once.
117
- 4. **Repair** failures by finding the cause. If a test fails because the app is wrong, fix the app (in a code step) or report the bug (in a check step).
118
-
119
- Never make a run green by marking tests `test.skip`, `test.fixme`, `test.fail`, `.only`-ing others, deleting assertions, loosening them (`toHaveText` to `toContainText` without reason), or adding sleeps. A healed test that hides a regression is worse than a red one.
120
-
121
- ## Cypress equivalents
122
-
123
- | Playwright | Cypress |
124
- |---|---|
125
- | `getByRole`, `getByLabel` | `cy.findByRole`, `cy.findByLabelText` if Testing Library is installed; otherwise `cy.contains` and the project's `data-cy` / `data-testid` |
126
- | web-first `expect(locator)` | `cy.get(...).should(...)` (retries) |
127
- | `waitForResponse` | `cy.intercept(...).as('x')` then `cy.wait('@x')` |
128
- | `page.route(...).fulfill` | `cy.intercept(method, url, { statusCode, body })` |
129
- | `storageState` | `cy.session()` |
130
- | `npx playwright test <file>` | `npx cypress run --spec <file>` (headless by default; never `cypress open`) |
131
-
132
- The same bans apply: no `cy.wait(<milliseconds>)`, no skipping failing tests.
133
-
134
- Adapted from testdino-hq/playwright-skill guides `core/locators.md`, `core/assertions-and-waiting.md`, `core/flaky-tests.md`, `core/authentication.md`, `core/network-mocking.md` and `core/test-agents.md` (MIT, commit 178d1cb537604279d7b54500c4823a6db42b9de6, Copyright (c) 2026 TestDino). Changed: condensed several thousand lines into one page, removed version-specific APIs and vendor mentions, added unattended rules (test accounts, temp folders, no skips), added the Cypress table. The plan, generate, repair loop restates the idea of the Playwright test agents (microsoft/playwright, Apache-2.0) in our own words; no text copied.
1
+ # Playwright patterns: locators, waiting, flakes, auth, mocking
2
+
3
+ Read this when you write, fix or review browser tests or the `steps` of a headless check. In a TAMsense run every browser context, also those of the project's tests, is recorded for the task; scripts of your own name their recordings as [recording.md](recording.md) shows. Examples are Playwright Test in TypeScript; translate to the project's language and runner. If the project uses Cypress, the same ideas apply (see the last section). Check the installed version (`npx playwright --version`) before using an API you are not sure exists in it.
4
+
5
+ ## Locators
6
+
7
+ Use the first one that works, in this order:
8
+
9
+ ```ts
10
+ page.getByRole('button', { name: 'Save changes' }) // 1. role + accessible name
11
+ page.getByLabel('Email address') // 2. form fields by label
12
+ page.getByText('Welcome back') // 3. non-interactive text
13
+ page.getByPlaceholder('Search') // 4. only if there is no label
14
+ page.getByAltText('Company logo') // 5. images
15
+ page.getByTestId('checkout-summary') // 6. when nothing semantic is stable
16
+ page.locator('.legacy-widget button') // 7. CSS or XPath, last resort
17
+ ```
18
+
19
+ - Role locators survive class renames and component swaps, and they fail when the control is not accessible, which is a finding in itself.
20
+ - `exact: true` stops "Log" from matching "Log out".
21
+ - Scope instead of indexing: `page.getByRole('dialog', { name: 'Delete project' }).getByRole('button', { name: 'Delete' })` beats `.nth(2)`.
22
+ - Rows and cards: `page.getByRole('row').filter({ hasText: 'Invoice 42' }).getByRole('button', { name: 'Open' })`.
23
+ - Use the project's existing `data-testid` convention if it has one; do not add test ids to product code during a check step.
24
+ - Strict mode errors ("resolved to 2 elements") mean the locator is ambiguous: scope it, do not add `.first()` to silence it.
25
+
26
+ ## Waiting and assertions
27
+
28
+ Actions (`click`, `fill`, `check`) already wait for the element to be attached, visible, stable and enabled. Assertions on locators retry until they pass or time out. Use them; do not sleep.
29
+
30
+ | Need | Use |
31
+ |---|---|
32
+ | Element shown or hidden | `await expect(locator).toBeVisible()` / `.toBeHidden()` |
33
+ | Text | `toHaveText` (full) or `toContainText` |
34
+ | Count | `toHaveCount(n)` |
35
+ | Input value, attribute, focus | `toHaveValue`, `toHaveAttribute`, `toBeFocused` |
36
+ | Navigation | `await expect(page).toHaveURL(/\/dashboard/)` |
37
+ | A request or response | register `page.waitForResponse(...)` **before** the action that triggers it |
38
+ | A non-DOM condition | `await expect.poll(() => readStatus()).toBe('done')` |
39
+
40
+ Wrong and right:
41
+
42
+ ```ts
43
+ await page.waitForTimeout(2000); // wrong: too slow or too short
44
+ expect(await page.getByRole('alert').isVisible()).toBe(true); // wrong: checks once, no retry
45
+ expect(await row.textContent()).toBe('Paid'); // wrong: no retry
46
+
47
+ await expect(page.getByRole('alert')).toBeVisible(); // right
48
+ await expect(row).toHaveText('Paid'); // right
49
+
50
+ const saved = page.waitForResponse(r => r.url().includes('/api/profile') && r.ok());
51
+ await page.getByRole('button', { name: 'Save' }).click();
52
+ await saved; // right: registered before the click
53
+ ```
54
+
55
+ - `networkidle` is unreliable with websockets, polling or analytics. Wait for the element or URL that proves the page is ready.
56
+ - Every `expect` and every action is awaited. A missing `await` makes a test pass before it checked anything.
57
+ - Do not wrap assertions in try/catch; do not raise timeouts above about 30 s to make something pass.
58
+
59
+ ## Flaky tests
60
+
61
+ A flaky test has a cause; retries and longer timeouts only hide it. Classify first:
62
+
63
+ | Symptom | Likely cause | How to confirm |
64
+ |---|---|---|
65
+ | Fails sometimes even alone | Timing: missing await, sleep, race between action and assertion | `npx playwright test <file> --repeat-each=10` (raise the count for rare races) |
66
+ | Passes alone, fails in the suite | Isolation: shared state, shared test data, order dependence | `--grep "<name>" --workers=1` passes, full run fails |
67
+ | Fails only in CI | Environment: viewport, fonts, slower machine, missing service | Compare traces and screenshots from both |
68
+ | Random crashes, "Target closed" | Infrastructure: memory, browser crash | Errors mention the browser, not the app |
69
+
70
+ Fixes: replace sleeps with web-first assertions; register waits before actions; give each test its own data (unique names, per-test setup); avoid module-level mutable variables; set viewport and `reducedMotion: 'reduce'` explicitly when animation or layout matters. For evidence, run with `--trace retain-on-failure --reporter=line --output <tmp>/test-results` and read the error message and the error-context and screenshot files in that output folder. Do not open `show-trace`, `show-report`, `--ui` or `--headed`: they open windows, and nobody is there to look.
71
+
72
+ ## Authentication
73
+
74
+ - Use the project's existing auth setup first: a setup project, a global setup that saves `storageState`, a fixture, a test-only login endpoint.
75
+ - Without one, log in once per run through the UI or the API with a **test account the project documents for local development** (seed data, README, fixtures), save `storageState` to the temp folder, and reuse it.
76
+ - Never use real user credentials, never read secrets the task did not give you, and never write auth state files inside the repository.
77
+ - Testing the login page itself: start from an empty state (`storageState: { cookies: [], origins: [] }`).
78
+ - OAuth or SSO with an external provider: do not hit the real provider; check the pages around it or report that the flow could not be exercised.
79
+
80
+ ```ts
81
+ // global setup or the first step of a headless check
82
+ await page.goto('/login');
83
+ await page.getByLabel('Email').fill(process.env.E2E_USER ?? 'demo@example.test');
84
+ await page.getByLabel('Password').fill(process.env.E2E_PASSWORD ?? 'from-the-seed-file');
85
+ await page.getByRole('button', { name: 'Sign in' }).click();
86
+ await expect(page).toHaveURL(/dashboard/);
87
+ await page.context().storageState({ path: `${tmpDir}/auth.json` });
88
+ ```
89
+
90
+ ## Network mocking
91
+
92
+ Mock what you do not own (payment providers, email, maps, analytics) and error states you cannot trigger otherwise. Do not mock the app's own pages, scripts or styles, and prefer the real backend for the happy path.
93
+
94
+ ```ts
95
+ await page.route('**/api/orders', route =>
96
+ route.fulfill({ status: 500, json: { error: 'Internal error' } })); // error state
97
+ await page.route('**/api/orders', route => route.fulfill({ json: [] })); // empty state
98
+ await page.route('**/api/orders', async route => { // slow: loading state
99
+ await new Promise(r => setTimeout(r, 3000));
100
+ await route.continue();
101
+ });
102
+ await page.route(/analytics|segment|hotjar/, route => route.abort()); // third-party noise
103
+ ```
104
+
105
+ - Register routes before `page.goto`.
106
+ - Every handler calls exactly one of `fulfill`, `continue` or `abort`, otherwise the request hangs.
107
+ - Keep globs narrow (`**/api/orders`, not `**/*`).
108
+ - Loading, empty and error states are where UI changes break most often; mocking is the cheapest way to see them.
109
+
110
+ ## Writing new tests (code step only)
111
+
112
+ When the task changes a user flow and the project has an E2E suite:
113
+
114
+ 1. **Plan:** list the scenarios from the acceptance criteria: the happy path, the main error state, the empty state, the narrow screen if layout changed. Note what each one must assert.
115
+ 2. **Generate:** follow the suite's existing structure, fixtures, page objects and naming. One behaviour per test; several related `expect`s are fine.
116
+ 3. **Run** the new tests several times (`--repeat-each=3` at least) and the related existing tests once.
117
+ 4. **Repair** failures by finding the cause. If a test fails because the app is wrong, fix the app (in a code step) or report the bug (in a check step).
118
+
119
+ Never make a run green by marking tests `test.skip`, `test.fixme`, `test.fail`, `.only`-ing others, deleting assertions, loosening them (`toHaveText` to `toContainText` without reason), or adding sleeps. A healed test that hides a regression is worse than a red one.
120
+
121
+ ## Cypress equivalents
122
+
123
+ | Playwright | Cypress |
124
+ |---|---|
125
+ | `getByRole`, `getByLabel` | `cy.findByRole`, `cy.findByLabelText` if Testing Library is installed; otherwise `cy.contains` and the project's `data-cy` / `data-testid` |
126
+ | web-first `expect(locator)` | `cy.get(...).should(...)` (retries) |
127
+ | `waitForResponse` | `cy.intercept(...).as('x')` then `cy.wait('@x')` |
128
+ | `page.route(...).fulfill` | `cy.intercept(method, url, { statusCode, body })` |
129
+ | `storageState` | `cy.session()` |
130
+ | `npx playwright test <file>` | `npx cypress run --spec <file>` (headless by default; never `cypress open`) |
131
+
132
+ The same bans apply: no `cy.wait(<milliseconds>)`, no skipping failing tests.
133
+
134
+ Adapted from testdino-hq/playwright-skill guides `core/locators.md`, `core/assertions-and-waiting.md`, `core/flaky-tests.md`, `core/authentication.md`, `core/network-mocking.md` and `core/test-agents.md` (MIT, commit 178d1cb537604279d7b54500c4823a6db42b9de6, Copyright (c) 2026 TestDino). Changed: condensed several thousand lines into one page, removed version-specific APIs and vendor mentions, added unattended rules (test accounts, temp folders, no skips), added the Cypress table. The plan, generate, repair loop restates the idea of the Playwright test agents (microsoft/playwright, Apache-2.0) in our own words; no text copied.
@@ -1,61 +1,61 @@
1
- # Recording the browser for the task
2
-
3
- Read this before any script that opens a browser during a TAMsense run. People on the task watch what your browser did: every page is recorded as a replay (the DOM, clicks and typing, played back in the TAMsense player) and attached to the task when the run ends.
4
-
5
- ## What happens by itself
6
-
7
- - When the `TAMSENSE_RECORDER` environment variable is set, every Node process you start loads the TAMsense recorder, and every Playwright browser context is recorded, also the ones opened by the project's own E2E tests. You cannot switch this off, and you do not need to.
8
- - Requests from the tested site to the TAMsense tracker (`/api/events` with a `?key=` or sent to the TAMsense server) are answered by the recorder and never leave the browser, so your runs do not reach the product's analytics.
9
- - Requests other than GET, HEAD and OPTIONS to an address outside this computer (anything but `localhost`, `*.localhost` and loopback addresses) are aborted, so a test run never writes to an outside service. The summary from `closeRecorded` counts them in `blockedExternalWrites`. If a flow you check needs such a write, it was set up against an outside service: say so instead of working around it.
10
- - Password fields are masked in the replay.
11
- - Recordings you did not attach yourself are attached when the run ends. Unnamed ones get the step name as their title, so name yours.
12
- - The task shows who made each recording: your role, the step, the round and the computer. You do not write that yourself.
13
-
14
- ## What you add in every browser script
15
-
16
- Load the recorder and open each context through it, with a title and a purpose:
17
-
18
- ```js
19
- import { createRequire } from 'node:module'
20
-
21
- const recorder = process.env.TAMSENSE_RECORDER ? createRequire(import.meta.url)(process.env.TAMSENSE_RECORDER) : null
22
-
23
- const context = recorder
24
- ? await recorder.newRecordedContext(browser, {
25
- title: 'Checkout testing, attempt 2',
26
- purpose: 'Paying with a saved card after the fix to the total',
27
- context: { viewport: { width: 1280, height: 900 } },
28
- })
29
- : await browser.newContext({ viewport: { width: 1280, height: 900 } })
30
- const page = await context.newPage()
31
-
32
- await page.goto(url)
33
- await recorder?.mark(page, 'Opened checkout')
34
- await page.getByRole('button', { name: 'Pay' }).click()
35
- await recorder?.mark(page, 'Paid, the receipt is shown')
36
-
37
- const notes = 'Opened checkout with a saved card and clicked Pay. Expected the receipt with the corrected total of $42: it showed $42. Passed.'
38
- const recorded = recorder ? await recorder.closeRecorded(context, { notes }) : await context.close()
39
- console.log(JSON.stringify(recorded, null, 2))
40
- ```
41
-
42
- - `title`: what it shows, at most 60 characters. Number design variants ("Design option 1", "Design option 2") and name test runs by their goal and attempt ("Feature testing", "Second attempt at feature testing"). Never generic names like "screenshot1" or "test".
43
- - `purpose`: one line, at most 160 characters, on why it was made ("Comparing two layouts for the empty orders state").
44
- - `mark(page, label)`: call it at each meaningful step (opened a screen, submitted a form, saw the error). Marks show on the replay's timeline, and a click on one jumps there.
45
- - `closeRecorded(context, { notes })`: closes the context, finishes the files and returns where they are. Close every context this way, or at least with `context.close()`, before the script ends.
46
- - `notes`: what you tested in this browser and what happened, in your own words, at most 600 characters: what you opened, what you clicked or typed, what you expected, and whether it passed or failed. People see them on the replay card and above the player before they press play, so write them for someone who did not read your report. Write them after the steps ran, from what you actually saw. A recording without notes shows the marks you set instead, and without marks, "No notes from the agent".
47
- - Without `TAMSENSE_RECORDER` (not a TAMsense run) the script works as before.
48
-
49
- ## When to add a video
50
-
51
- A replay rebuilds the page from its DOM, so it cannot show pixels drawn by the page itself. Pass `video: true` to `newRecordedContext` as well when the screen has:
52
-
53
- - a `<canvas>` (charts drawn on canvas, editors, games), a map, or a `<video>` player;
54
- - iframes from another site (payment forms, embedded widgets);
55
- - or when someone in the chat asks for a video.
56
-
57
- `closeRecorded` warns when it saw a canvas, a video or a cross-origin iframe on a page recorded without video. Videos are larger (up to 200 MB each, 5 per run), so keep them short.
58
-
59
- ## Attaching earlier
60
-
61
- Everything is attached at the end of the run. To attach a recording sooner, or with a better title, call `attach_replay` (the `.rrweb.ndjson` file) or `attach_video` (the `.webm` file) with the path from `closeRecorded`, a title, a purpose and notes (the notes you gave `closeRecorded` are used when you leave them out). A run attaches at most 20 replays and 5 videos; the ones you named come first.
1
+ # Recording the browser for the task
2
+
3
+ Read this before any script that opens a browser during a TAMsense run. People on the task watch what your browser did: every page is recorded as a replay (the DOM, clicks and typing, played back in the TAMsense player) and attached to the task when the run ends.
4
+
5
+ ## What happens by itself
6
+
7
+ - When the `TAMSENSE_RECORDER` environment variable is set, every Node process you start loads the TAMsense recorder, and every Playwright browser context is recorded, also the ones opened by the project's own E2E tests. You cannot switch this off, and you do not need to.
8
+ - Requests from the tested site to the TAMsense tracker (`/api/events` with a `?key=` or sent to the TAMsense server) are answered by the recorder and never leave the browser, so your runs do not reach the product's analytics.
9
+ - Requests other than GET, HEAD and OPTIONS to an address outside this computer (anything but `localhost`, `*.localhost` and loopback addresses) are aborted, so a test run never writes to an outside service. The summary from `closeRecorded` counts them in `blockedExternalWrites`. If a flow you check needs such a write, it was set up against an outside service: say so instead of working around it.
10
+ - Password fields are masked in the replay.
11
+ - Recordings you did not attach yourself are attached when the run ends. Unnamed ones get the step name as their title, so name yours.
12
+ - The task shows who made each recording: your role, the step, the round and the computer. You do not write that yourself.
13
+
14
+ ## What you add in every browser script
15
+
16
+ Load the recorder and open each context through it, with a title and a purpose:
17
+
18
+ ```js
19
+ import { createRequire } from 'node:module'
20
+
21
+ const recorder = process.env.TAMSENSE_RECORDER ? createRequire(import.meta.url)(process.env.TAMSENSE_RECORDER) : null
22
+
23
+ const context = recorder
24
+ ? await recorder.newRecordedContext(browser, {
25
+ title: 'Checkout testing, attempt 2',
26
+ purpose: 'Paying with a saved card after the fix to the total',
27
+ context: { viewport: { width: 1280, height: 900 } },
28
+ })
29
+ : await browser.newContext({ viewport: { width: 1280, height: 900 } })
30
+ const page = await context.newPage()
31
+
32
+ await page.goto(url)
33
+ await recorder?.mark(page, 'Opened checkout')
34
+ await page.getByRole('button', { name: 'Pay' }).click()
35
+ await recorder?.mark(page, 'Paid, the receipt is shown')
36
+
37
+ const notes = 'Opened checkout with a saved card and clicked Pay. Expected the receipt with the corrected total of $42: it showed $42. Passed.'
38
+ const recorded = recorder ? await recorder.closeRecorded(context, { notes }) : await context.close()
39
+ console.log(JSON.stringify(recorded, null, 2))
40
+ ```
41
+
42
+ - `title`: what it shows, at most 60 characters. Number design variants ("Design option 1", "Design option 2") and name test runs by their goal and attempt ("Feature testing", "Second attempt at feature testing"). Never generic names like "screenshot1" or "test".
43
+ - `purpose`: one line, at most 160 characters, on why it was made ("Comparing two layouts for the empty orders state").
44
+ - `mark(page, label)`: call it at each meaningful step (opened a screen, submitted a form, saw the error). Marks show on the replay's timeline, and a click on one jumps there.
45
+ - `closeRecorded(context, { notes })`: closes the context, finishes the files and returns where they are. Close every context this way, or at least with `context.close()`, before the script ends.
46
+ - `notes`: what you tested in this browser and what happened, in your own words, at most 600 characters: what you opened, what you clicked or typed, what you expected, and whether it passed or failed. People see them on the replay card and above the player before they press play, so write them for someone who did not read your report. Write them after the steps ran, from what you actually saw. A recording without notes shows the marks you set instead, and without marks, "No notes from the agent".
47
+ - Without `TAMSENSE_RECORDER` (not a TAMsense run) the script works as before.
48
+
49
+ ## When to add a video
50
+
51
+ A replay rebuilds the page from its DOM, so it cannot show pixels drawn by the page itself. Pass `video: true` to `newRecordedContext` as well when the screen has:
52
+
53
+ - a `<canvas>` (charts drawn on canvas, editors, games), a map, or a `<video>` player;
54
+ - iframes from another site (payment forms, embedded widgets);
55
+ - or when someone in the chat asks for a video.
56
+
57
+ `closeRecorded` warns when it saw a canvas, a video or a cross-origin iframe on a page recorded without video. Videos are larger (up to 200 MB each, 5 per run), so keep them short.
58
+
59
+ ## Attaching earlier
60
+
61
+ Everything is attached at the end of the run. To attach a recording sooner, or with a better title, call `attach_replay` (the `.rrweb.ndjson` file) or `attach_video` (the `.webm` file) with the path from `closeRecorded`, a title, a purpose and notes (the notes you gave `closeRecorded` are used when you leave them out). A run attaches at most 20 replays and 5 videos; the ones you named come first.