@tamsensedev/runner 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (79) hide show
  1. package/LICENSE +5 -0
  2. package/README.md +45 -2
  3. package/dist/cli.js +2234 -0
  4. package/dist/cli.js.map +1 -0
  5. package/library/.claude-plugin/plugin.json +5 -0
  6. package/library/THIRD_PARTY_NOTICES.md +123 -0
  7. package/library/agents/code-architect.md +67 -0
  8. package/library/agents/code-explorer.md +61 -0
  9. package/library/agents/code-reviewer.md +73 -0
  10. package/library/agents/code-simplifier.md +62 -0
  11. package/library/agents/design-critic.md +52 -0
  12. package/library/agents/second-opinion.md +64 -0
  13. package/library/agents/security-auditor.md +86 -0
  14. package/library/agents/silent-failure-hunter.md +70 -0
  15. package/library/agents/test-coverage-analyzer.md +78 -0
  16. package/library/agents/test-runner.md +48 -0
  17. package/library/agents/ui-tester.md +47 -0
  18. package/library/licenses/Apache-2.0.txt +202 -0
  19. package/library/skills/accessibility/SKILL.md +144 -0
  20. package/library/skills/accessibility/references/wcag-checks.md +168 -0
  21. package/library/skills/addressing-review-feedback/SKILL.md +109 -0
  22. package/library/skills/app-screens/SKILL.md +96 -0
  23. package/library/skills/app-screens/references/shell-kpis-charts.md +108 -0
  24. package/library/skills/app-screens/references/states-density-phone.md +54 -0
  25. package/library/skills/app-screens/references/tables-forms-settings.md +104 -0
  26. package/library/skills/avoiding-generic-ai-design/SKILL.md +92 -0
  27. package/library/skills/avoiding-generic-ai-design/references/tells.md +112 -0
  28. package/library/skills/clean-commits/SKILL.md +85 -0
  29. package/library/skills/design-review/SKILL.md +107 -0
  30. package/library/skills/design-review/references/critique-passes.md +101 -0
  31. package/library/skills/design-review/references/screenshot-procedure.md +113 -0
  32. package/library/skills/differential-security-review/LICENSE +427 -0
  33. package/library/skills/differential-security-review/SKILL.md +105 -0
  34. package/library/skills/differential-security-review/references/adversarial.md +115 -0
  35. package/library/skills/differential-security-review/references/methodology.md +145 -0
  36. package/library/skills/differential-security-review/references/patterns.md +108 -0
  37. package/library/skills/false-positive-check/LICENSE +427 -0
  38. package/library/skills/false-positive-check/SKILL.md +95 -0
  39. package/library/skills/false-positive-check/references/bug-class-verification.md +128 -0
  40. package/library/skills/false-positive-check/references/deep-verification.md +113 -0
  41. package/library/skills/false-positive-check/references/evidence-templates.md +93 -0
  42. package/library/skills/false-positive-check/references/false-positive-patterns.md +122 -0
  43. package/library/skills/false-positive-check/references/gate-reviews.md +36 -0
  44. package/library/skills/false-positive-check/references/standard-verification.md +80 -0
  45. package/library/skills/focused-implementation/SKILL.md +104 -0
  46. package/library/skills/implementation-plan/SKILL.md +66 -0
  47. package/library/skills/interface-design/SKILL.md +144 -0
  48. package/library/skills/interface-design/references/craft-defaults.md +130 -0
  49. package/library/skills/interface-design/references/new-direction.md +62 -0
  50. package/library/skills/interface-design/references/writing.md +52 -0
  51. package/library/skills/interface-polish/SKILL.md +103 -0
  52. package/library/skills/interface-polish/references/detail-rules.md +154 -0
  53. package/library/skills/interface-polish/references/worst-case-data.md +158 -0
  54. package/library/skills/motion-design/SKILL.md +106 -0
  55. package/library/skills/motion-design/references/recipes.md +243 -0
  56. package/library/skills/postgres-performance/SKILL.md +97 -0
  57. package/library/skills/postgres-performance/references/rules.md +265 -0
  58. package/library/skills/project-orientation/SKILL.md +77 -0
  59. package/library/skills/reviewing-changes/SKILL.md +183 -0
  60. package/library/skills/safe-database-changes/SKILL.md +111 -0
  61. package/library/skills/safe-database-changes/references/backfills.md +75 -0
  62. package/library/skills/safe-database-changes/references/migration-tools.md +69 -0
  63. package/library/skills/safe-database-changes/references/postgres-ddl.md +199 -0
  64. package/library/skills/security-review/SKILL.md +132 -0
  65. package/library/skills/security-review/references/checks-by-area.md +107 -0
  66. package/library/skills/security-review/references/do-not-report.md +86 -0
  67. package/library/skills/spec-writing/SKILL.md +144 -0
  68. package/library/skills/systematic-debugging/SKILL.md +130 -0
  69. package/library/skills/systematic-debugging/references/flaky-tests.md +51 -0
  70. package/library/skills/systematic-debugging/references/root-cause-tracing.md +55 -0
  71. package/library/skills/task-interview/SKILL.md +84 -0
  72. package/library/skills/test-driven-development/SKILL.md +116 -0
  73. package/library/skills/test-driven-development/references/writing-good-tests.md +85 -0
  74. package/library/skills/ui-testing/SKILL.md +99 -0
  75. package/library/skills/ui-testing/references/headless-check.md +262 -0
  76. package/library/skills/ui-testing/references/playwright-patterns.md +134 -0
  77. package/library/skills/verification-before-completion/SKILL.md +113 -0
  78. package/library/skills/web-interface-guidelines/SKILL.md +202 -0
  79. package/package.json +44 -5
@@ -0,0 +1,84 @@
1
+ ---
2
+ name: task-interview
3
+ description: Use when a task or spec leaves a decision open and you are tempted to ask: research first, ask only questions that pass three tests, settle corner cases as assumptions with a reason, and either ask in short rounds with a recommended answer first (when a person can answer, or on the TAMsense spec step with submit_questions) or decide with the recommendation and report it (on any other unattended TAMsense step).
4
+ ---
5
+
6
+ # Task interview
7
+
8
+ Finding facts is your job. Decisions about intent belong to the person who owns the task. Most "questions" are facts you have not looked up yet, or corners with a safe default. Ask only what is left after both.
9
+
10
+ ## 1. Research before any question
11
+
12
+ Look before you ask: the task, the spec and its decisions, the project's rules (`project-orientation`), the code around the change, existing tests, data you can read locally.
13
+
14
+ - Never ask where something is, which component to use or how the code works today. Look it up.
15
+ - If the answer has several plausible candidates in the code, the question becomes "which of these", with the real paths as options.
16
+ - When a person corrects a fact, check it in the code before you rely on it.
17
+
18
+ ## 2. Sweep the corners instead of asking about them
19
+
20
+ Most defects are corners: one part of the change meeting one unusual condition. More questions do not find them; a sweep does. For each part of the change, go through:
21
+
22
+ - empty or missing data, one item, very many items;
23
+ - invalid or too long input, first use and repeated use;
24
+ - missing access, a signed-out user, two people editing at once;
25
+ - a failed or slow external call, old stored data, existing users;
26
+ - phone width; numbers and money (zero, rounding, unknown versus zero); dates and time zones.
27
+
28
+ Settle every condition that applies in one of three ways: an assumption with its reason (a safe default exists), out of scope, or a question (only if it passes the test below).
29
+
30
+ ## 3. The question test
31
+
32
+ Ask a question only if all three hold:
33
+
34
+ 1. **Not discoverable.** The code, the data, the spec and the project's rules do not answer it.
35
+ 2. **It changes the result.** The two most likely answers lead to different changes or different acceptance criteria.
36
+ 3. **No safe default.** Or a wrong default is costly: lost or wrong data, money, permissions and privacy, a public API or stored format, a migration that is hard to undo, or visible text and behaviour the owner cares about.
37
+
38
+ Rank the questions that pass by impact times uncertainty. On a tie: scope first, then data and security, then what the user sees, then technical details. Stop when you can predict the answer to the next questions; record those as assumptions.
39
+
40
+ ## 4. How to write a question
41
+
42
+ ```
43
+ id: empty_state
44
+ Question: What should the task list show when a board has no tasks?
45
+ Why it matters: It decides the empty-state text and whether a "Create a task" button is added.
46
+ Options:
47
+ 1. A short hint (recommended): the page says "No tasks yet", nothing else changes.
48
+ 2. A hint and a button: the page also offers "Create a task", which needs a new route.
49
+ ```
50
+
51
+ - One plain sentence; the owner may not be a developer.
52
+ - A "why it matters" line.
53
+ - 2 to 4 mutually exclusive options, the recommended one first and marked; each option says what happens if it is chosen.
54
+ - Several picks only when the options are additive.
55
+ - Zero questions is a good result.
56
+
57
+ ## 5. Rounds
58
+
59
+ - **Round 1:** at most 4 questions, ideally 2 or 3. Ask only questions that do not depend on another open question of the same round; note dependent decisions for later.
60
+ - **Round 2 (optional):** at most 3 more, only if an answer was vague ("not sure", "depends"), contradicts another answer or the code, or unlocked a dependent decision.
61
+ - No third round. About 6 questions in total. Whatever is left becomes an assumption.
62
+ - An unanswered question takes the recommended option and is recorded as an assumption.
63
+
64
+ ## 6. When nobody can answer (a TAMsense run)
65
+
66
+ On the TAMsense spec step the owner answers after the run: hand the brief and the round's questions in with `submit_questions` and finish. The owner clicks through them on the task page, and the next round or the spec continues from your brief.
67
+
68
+ On any other step of a TAMsense run nobody answers, so never stop to wait:
69
+
70
+ 1. Take the recommended option and keep working.
71
+ 2. If a wrong guess would be costly and hard to undo (deleting data, a destructive migration, money, permissions), build the safe part only and leave the risky part out.
72
+ 3. In the report, list each decision as the question, the options, what you chose and why, so the owner can correct it in one reply. If the board lets you hand the task back to the spec step and the open decision blocks the work, hand it back with these questions in the note.
73
+
74
+ ## Common mistakes
75
+
76
+ | Thought | What to do instead |
77
+ |---|---|
78
+ | "I'll ask which file handles this" | Search the code; ask only to choose between real candidates. |
79
+ | "Better ask about the empty state too" | If a safe default exists, write it as an assumption with the reason. |
80
+ | "Let me ask everything at once" | One round of independent questions; dependent ones wait. |
81
+ | "Whatever you think is best" came back | That is a delegation: take your recommendation and say so. |
82
+ | "I'll wait for an answer" | In a run nobody answers: decide, record, report. |
83
+
84
+ Sources: adapted from openai/codex Plan Mode template and request_user_input tool (Apache-2.0, commit 430a167), github/spec-kit clarify and specify commands (MIT, commit c06e670), mattpocock/skills grilling (MIT, commit 5beb35b), awslabs/aidlc-workflows corner checklist and requirements analysis (MIT-0, commit 95d2b44), addyosmani/agent-skills interview-me (MIT, commit aeac41d) and humanlayer/humanlayer create_plan (Apache-2.0, commit 99abe67). Changed: merged into one question policy, rewritten for batched rounds and unattended runs.
@@ -0,0 +1,116 @@
1
+ ---
2
+ name: test-driven-development
3
+ description: Use when adding behaviour or fixing a bug in a project, before writing the implementation: write a test in the project's own framework, watch it fail for the right reason, make it pass with minimal code, then run the suite. Covers what to do when the project has no test setup.
4
+ ---
5
+
6
+ # Test-driven development
7
+
8
+ If you never saw a test fail, you do not know that it tests anything. Write the test first, watch it fail, then write the code.
9
+
10
+ ## Step 0: Find the project's test setup
11
+
12
+ Before anything else (see `project-orientation`):
13
+
14
+ - Which framework, where tests live, how they are named, which helpers and fixtures exist, and the exact command to run one file and the whole suite.
15
+ - Copy the shape of a neighbouring test: imports, setup, naming, assertion style.
16
+
17
+ **If the project has no test setup at all:** do not add a test framework, new dependencies or a test folder structure the maintainers did not choose. Verify the change another way that you can show evidence for (type check, build, running the code with a fixture input, a throwaway script outside the repository) and write in the report: "No test setup in this repository; verified by: ...".
18
+
19
+ **If tests exist but not for this area:** add the test in the existing framework, next to the code, following the closest existing example.
20
+
21
+ ## When to use it, and when not
22
+
23
+ Use it for new behaviour, bug fixes, and behaviour changes in code that has a reasonable seam to test at.
24
+
25
+ Usually skip it, and say so in the report, for: configuration-only changes, generated code, pure markup or styling with no logic, copy changes, dependency bumps, and code with no seam that can show the behaviour (then explain the gap rather than writing a hollow test). Exploratory spikes are fine, but throw the spike away or start the test-first cycle from it before you commit.
26
+
27
+ ## The cycle
28
+
29
+ ### Red: write one failing test
30
+
31
+ One behaviour per test, a name that reads as a sentence about it, real code rather than mocks where that is cheap.
32
+
33
+ ```ts
34
+ test('retries a failing call up to 3 times, then succeeds', async () => {
35
+ let attempts = 0
36
+ const call = async () => {
37
+ attempts++
38
+ if (attempts < 3) throw new Error('fail')
39
+ return 'ok'
40
+ }
41
+
42
+ await expect(retry(call)).resolves.toBe('ok')
43
+ expect(attempts).toBe(3)
44
+ })
45
+ ```
46
+
47
+ Before writing the body, name the production change that would make this test fail. If you cannot name one, the test is not testing behaviour. More in [references/writing-good-tests.md](references/writing-good-tests.md).
48
+
49
+ ### Verify red: run it and read the failure
50
+
51
+ ```
52
+ pnpm vitest run src/retry.test.ts
53
+ ```
54
+
55
+ - It must **fail**, not error. A typo, a missing import or a syntax error is not a red test.
56
+ - The failure message must be the one you expect ("expected 'ok', received undefined").
57
+ - It must fail because the behaviour is missing, not because of setup.
58
+ - If it passes immediately, it tests existing behaviour or nothing at all: fix the test.
59
+
60
+ ### Green: the minimal code that passes
61
+
62
+ Write the simplest code that makes this test pass. No extra options, no handling for cases no test asks for yet. Do not change the test to fit the code.
63
+
64
+ ### Verify green
65
+
66
+ Run the test again, then the tests around it, then the project's full suite (or the widest suite that runs in reasonable time). A green file is not a green suite. Output should be clean: no new warnings or errors in the log.
67
+
68
+ Any failure in the wider run goes in the report by name, including ones you did not cause (check against the baseline from `project-orientation`).
69
+
70
+ ### Refactor
71
+
72
+ Only on green: remove duplication, improve names, extract helpers. Re-run after each change. Do not add behaviour while refactoring.
73
+
74
+ Repeat for the next behaviour: one test, one minimal implementation, at a time. Do not write the whole test file first.
75
+
76
+ ## If you wrote the code first
77
+
78
+ It happens: an exploration turned into the fix, or the change was made before this skill was loaded. Do not pretend. Prove the test can fail:
79
+
80
+ 1. Save your change to the production file: `git diff -- path/to/file > "$TMPDIR/fix.patch"` (any temp location outside the repository).
81
+ 2. Restore the original: `git checkout -- path/to/file`.
82
+ 3. Run the new test and confirm it fails for the right reason.
83
+ 4. Re-apply: `git apply "$TMPDIR/fix.patch"`, check with `git diff --stat` that the change is back, and run the test again to see it pass.
84
+
85
+ If the test passes without your change, it does not cover the change. Fix the test.
86
+
87
+ ## Bug fixes
88
+
89
+ 1. Write a test that reproduces the reported bug exactly (the input, the wrong output). For a hard bug, follow `systematic-debugging` first.
90
+ 2. Watch it fail with the symptom from the report.
91
+ 3. Fix the cause. Watch it pass.
92
+ 4. Run the suite.
93
+
94
+ ## Keep tests deterministic
95
+
96
+ Fixed dates and time zones, seeded random data, no dependence on test order, no real network, temp folders per test. Wait for conditions, not for time (see `systematic-debugging`, flaky tests reference).
97
+
98
+ ## Common rationalisations
99
+
100
+ | Excuse | Reality |
101
+ |---|---|
102
+ | "Too simple to test" | Simple code breaks too; the test takes a minute. |
103
+ | "I'll add tests after" | Tests written after pass immediately and prove nothing; they check what you built, not what was asked. |
104
+ | "I already tried it by hand" | A manual try leaves no record and is not repeated on the next change. |
105
+ | "The existing code has no tests" | Add tests for the part you change, in the project's framework. |
106
+ | "Hard to test" | Often a sign the design is hard to use. Find a better seam, or report the gap. |
107
+ | "I need to mock everything" | The code is too coupled, or the test is at the wrong level. Test higher up with real collaborators. |
108
+
109
+ ## Done when
110
+
111
+ - [ ] Each new behaviour and each fixed bug has a test that you saw fail for the right reason, then pass.
112
+ - [ ] Tests assert real behaviour through the public surface, with expected values written out by hand.
113
+ - [ ] The full relevant suite passes, or failures are listed with their cause (yours fixed, pre-existing reported).
114
+ - [ ] The report says which tests were added, or why none were (no test setup, no seam, not applicable).
115
+
116
+ Sources: obra/superpowers (MIT, commit bb92a77) test-driven-development. Changed: rewritten for unattended work in existing repositories.
@@ -0,0 +1,85 @@
1
+ # Writing good tests
2
+
3
+ Read this when writing or changing a test, adding a mock, or adding a test helper.
4
+
5
+ A test exists to catch a specific break. Two rules carry most of the weight:
6
+
7
+ 1. Every test names the break it catches.
8
+ 2. Every test exercises the real thing.
9
+
10
+ ## Rule 1: name the break
11
+
12
+ Before writing the body, answer: which production change would make this test fail, and is that change a bug or a deliberate decision? A good test catches a wrong branch, a missing side effect, a wrong argument, a boundary case or a broken contract.
13
+
14
+ **Derive expected values independently.** Use literals and hand-checked fixtures. An expected value computed by the code under test (or its helpers) passes whatever the code does.
15
+
16
+ ```ts
17
+ // Bad: the same builder produces both sides, so it is always equal
18
+ const expected = buildSearchQuery({ tag: 'urgent' })
19
+ expect(buildSearchQuery({ tag: 'urgent' })).toBe(expected)
20
+
21
+ // Bad: recomputes the answer the way the code does
22
+ expect(add(a, b)).toBe(a + b)
23
+
24
+ // Good: written out by hand
25
+ expect(buildSearchQuery({ tag: 'urgent' })).toBe('tag:"urgent"')
26
+ ```
27
+
28
+ Table-driven tests with literal expected values are a good shape for many cases.
29
+
30
+ **No change detectors.** A test that only fails on intentional decisions (a constant's value, exact message wording, private structure) fires on every redesign and sleeps through real bugs. Test the behaviour that depends on the decision: not `expect(MAX_RETRIES).toBe(5)` but "a failing call is retried 5 times and a sixth attempt never happens".
31
+
32
+ **Behaviour, not source text.** Asserting that a file contains a certain line proves only that the file is the file. Run the script, config or template against a controlled input and check the output, side effect or exit code.
33
+
34
+ **Your code, not the framework.** Test the contract your code makes at its boundary: the route you register, the query you build, the payload you send. Do not test that the router calls a registered handler; that is the framework's job. Constructors, getters and simple pass-throughs earn a test only when they validate, normalise, default, derive or cause a side effect.
35
+
36
+ **Through the public surface.** Test via the function, endpoint, component or command that callers use. A test that breaks when you refactor internals without changing behaviour is coupled to implementation. Avoid checking results through a side channel (querying the database directly) when the public interface can show them.
37
+
38
+ ## Rule 2: exercise the real thing
39
+
40
+ **Never assert on a mock.** An assertion that a mock was rendered or exists passes when the mock is there and says nothing about the component.
41
+
42
+ ```ts
43
+ // Bad: tests the mock
44
+ expect(screen.getByTestId('sidebar-mock')).toBeInTheDocument()
45
+
46
+ // Good: tests the real behaviour
47
+ expect(screen.getByRole('navigation')).toBeInTheDocument()
48
+ ```
49
+
50
+ **Mock at the right level.** Before replacing a method, learn all its side effects. Mock the slow or external part below it (network, paid API, clock, email) and keep what the test depends on real. A mock that swallows a write that the code later reads makes the test pass while the real thing breaks.
51
+
52
+ **Make doubles specific.** When arguments, call counts or order are part of the contract, assert them. Give success, error and malformed responses their own fixtures, so the wrong branch cannot satisfy the test.
53
+
54
+ **Mirror real data completely.** Fake responses should have the full structure of the real one, not only the fields this test reads. Partial fakes hide failures in code that reads the other fields.
55
+
56
+ **Production code carries production methods only.** Cleanup or reset methods that only tests call belong in test utilities, not on the production class.
57
+
58
+ **Prefer real components over elaborate mocks.** When mock setup is longer than the test logic, or the test breaks whenever the mock changes, switch to a test with real collaborators (an in-memory database, the real module, the project's existing test server).
59
+
60
+ ## The mutation check
61
+
62
+ Before finishing, imagine small breaks in the production code. At least one test should fail for each realistic one:
63
+
64
+ - A wrong constant or argument.
65
+ - A wrong branch taken.
66
+ - A missing state change or side effect.
67
+ - An empty or default return value.
68
+ - Missing validation for zero, empty, null, unauthorised or malformed input.
69
+
70
+ If nothing fails for a realistic mutation, that behaviour is unprotected. For an important change you can make the mutation for real: invert one condition, run the tests, confirm a failure, restore the line, and confirm with `git diff` that it is restored.
71
+
72
+ ## Warning signs
73
+
74
+ - Setup and assertion share the same object, so they are equal by construction.
75
+ - The test can only fail by crashing or by a missing selector.
76
+ - The test fails on every intentional change but never on accidental breakage.
77
+ - Expected values are hidden behind loops, builders or helpers.
78
+ - The test greps source text.
79
+ - The test exists for coverage and checks no outcome.
80
+ - An assertion checks a `*-mock` test id, or fails if you remove the mock.
81
+ - A method is called only from test files.
82
+ - Mock setup is more than half the test, or you cannot say why the mock is needed.
83
+ - Writing all the tests for a feature before any implementation: tests for imagined behaviour test the shape of things, not what users see. Go one test at a time.
84
+
85
+ Sources: obra/superpowers (MIT, commit bb92a77) writing-good-tests and mattpocock/skills (MIT, commit 5beb35b) tdd. Changed: shortened and merged.
@@ -0,0 +1,99 @@
1
+ ---
2
+ name: ui-testing
3
+ description: Use when a change affects what users see or click in a web app and should be checked in a real browser, when writing or fixing Playwright or Cypress tests, or when a test step must verify UI behaviour. Runs the project's E2E suite or a headless check from a temp folder and reports screenshots and errors as evidence.
4
+ ---
5
+
6
+ # UI testing in a headless browser
7
+
8
+ Reading the code tells you what should happen; a browser shows what does. This skill gets the app running, drives the changed flows headless, and turns what it saw into evidence for the report. Nobody watches the run, so every step must finish on its own and leave the machine as it was.
9
+
10
+ The project's own setup wins: its E2E framework, config, fixtures, page objects, test data, auth helpers and scripts. Use them before anything from here.
11
+
12
+ ## Hard rules
13
+
14
+ 1. **Headless only.** Never open a window: no `--headed`, `--ui`, `--debug`, `show-report`, `show-trace`, `cypress open`, `PWDEBUG`. Playwright's HTML reporter opens a browser on failure when not in CI, so run with `--reporter=line` and `PW_TEST_HTML_REPORT_OPEN=never`. Set `BROWSER=none` when starting dev servers so they do not open one either.
15
+ 2. **No new dependencies.** Do not add packages to the repository and do not download browsers (`playwright install`, `npx some-tool@latest`) just to check. Use what is installed. If the project has no Playwright but the `TAMSENSE_PLAYWRIGHT` environment variable is set, the TAMsense runner has already installed `playwright-core` and a headless browser outside the project, and the headless check script uses them. If nothing is, check from the code and say the browser check could not run.
16
+ 3. **The project's own start command, on a free port.** Start the app with its dev or preview script, never a hand-made server that skips its config. Prefer the headless check script, which starts the server and stops its whole process tree for you. Stop it at the end, including child processes, and confirm the port is free again.
17
+ 8. **Stop only what you started, by process id.** This is someone's own computer and their other programs may be running. Never kill processes by name or pattern: no `taskkill /IM`, `pkill`, `killall`, `Stop-Process -Name`. If you start a server yourself, record its PID and stop that process tree (`taskkill /PID <pid> /T /F` on Windows, `kill -- -<pgid>` elsewhere). If a process you started cannot be stopped that way, say so in the report and leave the rest alone.
18
+ 4. **Outputs outside the repository.** Screenshots, traces, reports, scratch scripts and auth state go to a new folder in the OS temp directory. `git status` must show nothing new from you when you finish.
19
+ 5. **Never fake it.** If the app cannot start (missing environment variables, secrets, a database or service you may not use), report exactly what is missing. Do not mock the whole backend and call the result a pass, and do not invent screenshots or results.
20
+ 6. **Do not write to shared systems.** If the local app is configured against a shared or production backend or database, look but do not submit forms or trigger writes.
21
+ 7. **Never hide failures.** Do not skip, `fixme`, delete or loosen a failing test to get a green run. In a check step, a failing test is a finding.
22
+
23
+ ## Process
24
+
25
+ ### 1. Decide what to check
26
+
27
+ From the task, the spec's acceptance criteria and the diff (`git diff origin/<base>...HEAD --stat`, then the changed components, routes and styles), write a short list of scenarios before touching the browser:
28
+
29
+ - The main flow the task changed, end to end.
30
+ - Its states: loading, empty, error, long or unusual content, many items, disabled controls.
31
+ - A narrow screen (390 px wide) if layout or styles changed, plus a desktop width.
32
+ - Keyboard use for any new or changed control (Tab to it, activate with Enter or Space, Escape closes overlays). The `accessibility` skill has the full checks.
33
+
34
+ ### 2. Find how this project runs and tests its UI
35
+
36
+ - E2E setup: `playwright.config.*`, `cypress.config.*`, `e2e/` or `tests/` folders, scripts like `test:e2e` in `package.json`, and the CI workflow steps that run them (they show the exact command, env vars and services).
37
+ - The app: dev and preview scripts, required env vars (`.env.example`, config validation, README), services it needs (database, cache, other apps in a monorepo). Check whether required variables are set by name; never print their values.
38
+ - Auth: seeded test users, a test login endpoint, existing storage-state setup.
39
+ - The installed Playwright or Cypress version.
40
+
41
+ ### 3a. The project has an E2E suite
42
+
43
+ Run the specs related to the change, then the whole suite if it is fast enough.
44
+
45
+ - Playwright: `PW_TEST_HTML_REPORT_OPEN=never npx playwright test <specs> --reporter=line --output <tmp>/test-results` (use the package manager the project uses). Its `webServer` config usually starts the app: check the port and `reuseExistingServer`, and make sure no stale server of yours is still running.
46
+ - Cypress: `npx cypress run --spec <specs> --config screenshotsFolder=<tmp>/screens,videosFolder=<tmp>/videos,video=false` (headless by default).
47
+ - If a run fails, rerun the failing spec alone to separate a real failure from a flaky one, and read the error and screenshot. Decide whether the failure comes from the change (the diff touches that flow) or existed before; say which and why.
48
+
49
+ In a code step, add or update tests for the changed flow following the suite's structure; see [references/playwright-patterns.md](references/playwright-patterns.md) for locators, waiting, auth, mocking and the plan, write, repair loop. In a review or test step, do not edit the repository's tests; check extra scenarios with 3b instead.
50
+
51
+ ### 3b. No suite, or scenarios the suite does not cover
52
+
53
+ Use the headless check in [references/headless-check.md](references/headless-check.md): a Node.js script you write to a temp folder that resolves Playwright from the project, starts the app with the project's command on a free port, visits the pages at 1280 and 390 px, runs your steps, saves screenshots and a JSON summary (console errors, failed requests, horizontal overflow, optional axe scan), and stops the app.
54
+
55
+ If Playwright is not installed in the project and no other headless browser tool is available, do not install one. Check the change from the code and say so.
56
+
57
+ ### 4. Look at the evidence
58
+
59
+ - Open every screenshot with the Read tool and compare it with what the scenario expects. Describe what you see in plain words: what is on the screen, what is wrong, where.
60
+ - Console errors, page errors and failed requests from the changed code are findings; third-party noise is not, but name it so the reader knows you saw it.
61
+ - Sideways scrolling at 390 px, cut-off text, overlapping elements, missing states and dead buttons are findings.
62
+ - For visual quality and consistency with the design system, apply the `design-review` skill; for the rules list, `web-interface-guidelines`; for accessibility, `accessibility`.
63
+
64
+ ### 5. Clean up
65
+
66
+ Stop the servers you started, by their process id only (hard rule 8), check the port no longer answers, remove containers and temp folders you created unless the task asks to keep artifacts, and run `git status` in every repository to confirm you left nothing behind.
67
+
68
+ ## Report
69
+
70
+ ```
71
+ UI check: <passed | failed | could not run>
72
+ How: <suite command or headless check>, browser <chromium | chrome | msedge>, widths <1280, 390>
73
+ Scenarios:
74
+ - <scenario>: pass | fail | not checked (why). Evidence: <what the screenshot or log shows>
75
+ Findings:
76
+ - <file:line if traced> <what is wrong, how to reproduce, what was expected>
77
+ Not checked: <and why: missing env vars X, Y; no browser available; flow needs an external provider>
78
+ Assumptions: <test account used, data you created locally, version>
79
+ ```
80
+
81
+ In a test step, base the verdict on the scenarios: a broken acceptance criterion, an error from the changed code or a layout broken on narrow screens fails it. If the browser check could not run, do not report it as passed; judge from the other evidence (tests, code) and state clearly that the UI was not seen. If the change itself stops the app from starting or building, that is a failure.
82
+
83
+ ## Common mistakes
84
+
85
+ - Assuming the page works because the build passed.
86
+ - Fixed sleeps instead of waiting for the element that proves readiness.
87
+ - Letting the HTML reporter or a dev server open a browser window.
88
+ - Leaving the dev server running, or `test-results/` and screenshots inside the repository.
89
+ - Testing only the happy path at desktop width.
90
+ - Pointing the check at staging or production because the local app did not start.
91
+ - Listing screenshot paths without saying what they show.
92
+
93
+ ## Done when
94
+
95
+ - Every scenario from step 1 is checked or listed as not checked with a reason.
96
+ - Evidence (described screenshots, error lines, failing assertions) backs each finding.
97
+ - No window was opened, no dependency added, nothing left running or written into the repository.
98
+
99
+ Sources: the start, wait, act, stop approach of anthropics/skills webapp-testing (Apache-2.0, commit dbd4588), testdino-hq/playwright-skill (MIT, commit 178d1cb), ideas from the Playwright test agents (Apache-2.0). Changed: rewritten for Node and the project's own setup, unattended rules added.
@@ -0,0 +1,262 @@
1
+ # Headless check without an E2E suite
2
+
3
+ Read this when the project has no Playwright or Cypress tests you can run for the change, but you still need to see the pages: screenshots, console errors, failed requests, horizontal overflow on narrow screens, and optionally an axe scan.
4
+
5
+ The script below starts the app with the project's own command on a free port, waits for it, opens each page headless at a desktop and a phone width, runs optional steps you write, saves screenshots and a JSON summary to a temp folder, and stops the app and all its child processes at the end.
6
+
7
+ ## What it needs
8
+
9
+ - Node.js 20 or newer.
10
+ - The `playwright`, `@playwright/test` or `playwright-core` package already installed in the project (it is resolved from the project folder you give it; in a monorepo, use the package folder of the web app). If the project has none and the `TAMSENSE_PLAYWRIGHT` environment variable is set, the script loads the `playwright-core` that the TAMsense runner installed in that folder, with the headless browser from the `TAMSENSE_PLAYWRIGHT_BROWSERS` folder. Nothing is installed.
11
+ - A browser that is already on the machine: Playwright's downloaded Chromium (the project's or the runner's), or the system Chrome or Edge as a fallback. If none works, the script stops and says so. Do not run `playwright install` or add packages to get one; report that the check could not run.
12
+ - `axe-core` is optional: if the project already has it, each page also gets an accessibility scan.
13
+
14
+ ## How to use it
15
+
16
+ 1. Make a new folder in the OS temp directory (`node -p "require('os').tmpdir()"`), for example `<tmp>/ui-check-<random>`. Never inside the repository.
17
+ 2. Write the script there as `ui-check.mjs` and edit only the `CONFIG` block and the `steps` function.
18
+ 3. Run `node <tmp>/ui-check-<random>/ui-check.mjs` with a timeout of a few minutes. It prints the summary; screenshots are in `CONFIG.outDir`.
19
+ 4. Open the screenshots with the Read tool and look at them. Describe in the report what each one shows; the paths alone mean nothing to the reader.
20
+ 5. Delete the folder when the report is written, unless the task asks you to keep artifacts.
21
+
22
+ `startCommand` is the project's own dev or preview command with `{port}` where the port goes, for example `pnpm dev --port {port} --strictPort` (Vite), `npm run dev -- -p {port}` (Next.js), `pnpm nuxt dev --port {port}`, `bin/rails server -p {port}`, `python manage.py runserver 127.0.0.1:{port}`. The script also sets `PORT`. Leave it empty and set `baseUrl` if the app is already running or is static files you serve another way.
23
+
24
+ On Windows the command runs in `cmd.exe`, which may not see the same PATH as your shell. If the summary says the command is not recognized, use absolute paths (for example the full path to `node.exe` and the package manager's script) rather than changing anything in the project.
25
+
26
+ ## The script
27
+
28
+ ```js
29
+ import { createRequire } from 'node:module'
30
+ import { spawn, execFileSync } from 'node:child_process'
31
+ import { mkdirSync, writeFileSync } from 'node:fs'
32
+ import { join } from 'node:path'
33
+ import net from 'node:net'
34
+
35
+ const CONFIG = {
36
+ projectDir: '/absolute/path/to/web/app',
37
+ outDir: '/absolute/path/to/tmp/ui-check-1234/out',
38
+ startCommand: 'pnpm dev --port {port} --strictPort',
39
+ baseUrl: '',
40
+ paths: ['/'],
41
+ widths: [1280, 390],
42
+ startTimeoutMs: 120_000,
43
+ navigationTimeoutMs: 30_000,
44
+ }
45
+
46
+ async function steps(page, { path, width }) {
47
+ }
48
+
49
+ const MAX_ITEMS = 20
50
+ const serverOutput = []
51
+
52
+ function keepTail(chunk) {
53
+ serverOutput.push(...String(chunk).split('\n').filter(Boolean))
54
+ serverOutput.splice(0, Math.max(0, serverOutput.length - 40))
55
+ }
56
+
57
+ function requireFrom(dir, names) {
58
+ const require = createRequire(join(dir, 'package.json'))
59
+ for (const name of names) {
60
+ try {
61
+ return require(name)
62
+ } catch {}
63
+ }
64
+ return null
65
+ }
66
+
67
+ function loadFromProject(names) {
68
+ const found = requireFrom(CONFIG.projectDir, names)
69
+ const runnerCopy = process.env.TAMSENSE_PLAYWRIGHT
70
+ if (found || !names.includes('playwright-core') || !runnerCopy) return found
71
+ if (process.env.TAMSENSE_PLAYWRIGHT_BROWSERS) process.env.PLAYWRIGHT_BROWSERS_PATH = process.env.TAMSENSE_PLAYWRIGHT_BROWSERS
72
+ return requireFrom(runnerCopy, ['playwright-core'])
73
+ }
74
+
75
+ function freePort() {
76
+ return new Promise((resolve, reject) => {
77
+ const server = net.createServer()
78
+ server.on('error', reject)
79
+ server.listen(0, '127.0.0.1', () => {
80
+ const { port } = server.address()
81
+ server.close(() => resolve(port))
82
+ })
83
+ })
84
+ }
85
+
86
+ function startServer(command, port) {
87
+ const child = spawn(command.replaceAll('{port}', String(port)), {
88
+ cwd: CONFIG.projectDir,
89
+ shell: true,
90
+ env: { ...process.env, PORT: String(port), BROWSER: 'none' },
91
+ detached: process.platform !== 'win32',
92
+ windowsHide: true,
93
+ stdio: ['ignore', 'pipe', 'pipe'],
94
+ })
95
+ child.stdout.on('data', keepTail)
96
+ child.stderr.on('data', keepTail)
97
+ return child
98
+ }
99
+
100
+ function stopServer(child) {
101
+ if (!child || child.exitCode !== null) return 'not running'
102
+ try {
103
+ if (process.platform === 'win32') {
104
+ const taskkill = join(process.env.SystemRoot ?? 'C:\\Windows', 'System32', 'taskkill.exe')
105
+ execFileSync(taskkill, ['/pid', String(child.pid), '/T', '/F'], { stdio: 'ignore' })
106
+ } else {
107
+ process.kill(-child.pid, 'SIGKILL')
108
+ }
109
+ return 'stopped'
110
+ } catch (error) {
111
+ child.kill('SIGKILL')
112
+ return `could not stop the process tree of pid ${child.pid}: ${error.message}`
113
+ } finally {
114
+ child.stdout.destroy()
115
+ child.stderr.destroy()
116
+ child.unref()
117
+ }
118
+ }
119
+
120
+ async function waitForServer(url, child) {
121
+ const deadline = Date.now() + CONFIG.startTimeoutMs
122
+ while (Date.now() < deadline) {
123
+ if (child && child.exitCode !== null) throw new Error(`start command exited with code ${child.exitCode}`)
124
+ try {
125
+ await fetch(url, { redirect: 'manual', signal: AbortSignal.timeout(5_000) })
126
+ return
127
+ } catch {}
128
+ await new Promise(resolve => setTimeout(resolve, 1_000))
129
+ }
130
+ throw new Error(`no response from ${url} within ${CONFIG.startTimeoutMs / 1000}s`)
131
+ }
132
+
133
+ async function launchBrowser(chromium) {
134
+ const failures = []
135
+ for (const channel of [undefined, 'chromium', 'chrome', 'msedge']) {
136
+ try {
137
+ return await chromium.launch({ headless: true, channel })
138
+ } catch (error) {
139
+ failures.push(`${channel ?? 'playwright chromium'}: ${String(error.message).split('\n')[0]}`)
140
+ }
141
+ }
142
+ throw new Error(`no browser available without downloading one: ${failures.join(' | ')}`)
143
+ }
144
+
145
+ async function runAxe(page, axeSource) {
146
+ try {
147
+ await page.addScriptTag({ content: axeSource })
148
+ return await page.evaluate(async () => {
149
+ const result = await window.axe.run(document, {
150
+ runOnly: { type: 'tag', values: ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'] },
151
+ })
152
+ return result.violations.map(violation => ({
153
+ rule: violation.id,
154
+ impact: violation.impact,
155
+ help: violation.help,
156
+ count: violation.nodes.length,
157
+ targets: violation.nodes.slice(0, 5).map(node => JSON.stringify(node.target)),
158
+ }))
159
+ })
160
+ } catch (error) {
161
+ return { error: String(error.message).split('\n')[0] }
162
+ }
163
+ }
164
+
165
+ async function checkPage(browser, url, path, width, axeSource) {
166
+ const context = await browser.newContext({ viewport: { width, height: 900 } })
167
+ const page = await context.newPage()
168
+ const result = { path, width, consoleErrors: [], pageErrors: [], failedRequests: [], badResponses: [] }
169
+ const add = (list, item) => { if (list.length < MAX_ITEMS) list.push(item) }
170
+ page.on('console', message => { if (message.type() === 'error') add(result.consoleErrors, message.text()) })
171
+ page.on('pageerror', error => add(result.pageErrors, String(error.message)))
172
+ page.on('requestfailed', request => add(result.failedRequests, `${request.method()} ${request.url()} ${request.failure()?.errorText ?? ''}`))
173
+ page.on('response', response => { if (response.status() >= 400) add(result.badResponses, `${response.status()} ${response.url()}`) })
174
+ try {
175
+ const response = await page.goto(url, { waitUntil: 'load', timeout: CONFIG.navigationTimeoutMs })
176
+ result.status = response?.status() ?? null
177
+ await page.waitForLoadState('networkidle', { timeout: 5_000 }).catch(() => {})
178
+ result.steps = await steps(page, { path, width })
179
+ result.title = await page.title()
180
+ result.finalUrl = page.url()
181
+ result.horizontalOverflowPx = await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth)
182
+ const name = `${path.replace(/[^a-z0-9]+/gi, '_').replace(/^_|_$/g, '') || 'root'}-${width}.png`
183
+ result.screenshot = join(CONFIG.outDir, name)
184
+ await page.screenshot({ path: result.screenshot, fullPage: true })
185
+ if (axeSource) result.axe = await runAxe(page, axeSource)
186
+ } catch (error) {
187
+ result.error = String(error.message).split('\n')[0]
188
+ } finally {
189
+ await context.close()
190
+ }
191
+ return result
192
+ }
193
+
194
+ async function main() {
195
+ mkdirSync(CONFIG.outDir, { recursive: true })
196
+ const summary = { pages: [], serverOutputTail: serverOutput }
197
+ const playwright = loadFromProject(['playwright', '@playwright/test', 'playwright-core'])
198
+ if (!playwright?.chromium) throw new Error(`Playwright is not installed in ${CONFIG.projectDir} and TAMSENSE_PLAYWRIGHT is not set`)
199
+ const axeSource = loadFromProject(['axe-core'])?.source ?? null
200
+ summary.axe = axeSource ? 'axe-core found in the project' : 'axe-core not installed, scan skipped'
201
+
202
+ let server = null
203
+ let browser = null
204
+ try {
205
+ let baseUrl = CONFIG.baseUrl
206
+ if (CONFIG.startCommand) {
207
+ const port = await freePort()
208
+ server = startServer(CONFIG.startCommand, port)
209
+ baseUrl = `http://localhost:${port}`
210
+ }
211
+ if (!baseUrl) throw new Error('set CONFIG.startCommand or CONFIG.baseUrl')
212
+ summary.baseUrl = baseUrl
213
+ await waitForServer(baseUrl, server)
214
+ browser = await launchBrowser(playwright.chromium)
215
+ for (const path of CONFIG.paths) {
216
+ for (const width of CONFIG.widths) {
217
+ summary.pages.push(await checkPage(browser, new URL(path, baseUrl).href, path, width, axeSource))
218
+ }
219
+ }
220
+ } catch (error) {
221
+ summary.setupError = String(error.message)
222
+ process.exitCode = 2
223
+ } finally {
224
+ await browser?.close().catch(() => {})
225
+ if (server) summary.server = stopServer(server)
226
+ writeFileSync(join(CONFIG.outDir, 'summary.json'), JSON.stringify(summary, null, 2))
227
+ console.log(JSON.stringify(summary, null, 2))
228
+ }
229
+ }
230
+
231
+ main().catch(error => {
232
+ console.error(error.message)
233
+ process.exitCode = 2
234
+ })
235
+ ```
236
+
237
+ ## Writing steps
238
+
239
+ `steps` runs on every page and width before the screenshot. Use it to reach the state the task changed: open the dialog, fill and submit the form, switch the tab. Branch on `path` when pages need different steps. Whatever it returns is saved as `steps` in that page's summary (for example the focus order from a keyboard pass, see the `accessibility` skill). Use the same rules as in tests: role and label locators, web-first waits, no fixed sleeps (see [playwright-patterns.md](playwright-patterns.md)).
240
+
241
+ ```js
242
+ async function steps(page, { path }) {
243
+ if (path !== '/settings') return
244
+ await page.getByRole('button', { name: 'Edit profile' }).click()
245
+ await page.getByLabel('Display name').fill('A very long display name that should wrap or truncate')
246
+ await page.getByRole('button', { name: 'Save' }).click()
247
+ await page.getByRole('status').waitFor()
248
+ }
249
+ ```
250
+
251
+ Steps that write data only touch the local app and whatever local database it uses. If the dev server is configured to talk to a shared or production backend or database, do not submit forms or trigger writes; check that first (see the `safe-database-changes` skill for how to read where the app connects).
252
+
253
+ ## Reading the summary
254
+
255
+ - `server`: should be `stopped`. Anything else names a process you must stop yourself before finishing.
256
+ - `setupError`: the app or the browser did not start. Report it with the last lines of `serverOutputTail` (strip anything that looks like a secret). Do not retry with a different command unless the first one was clearly wrong, and never invent results.
257
+ - `status` 4xx or 5xx, `pageErrors`, `consoleErrors`: real findings when they come from the changed code; noise from browser extensions does not exist here, but third-party scripts and dev-server warnings do. Say which is which.
258
+ - `badResponses` and `failedRequests`: API calls the page needed that failed. A 401 or a redirect to a login page means the page needs a signed-in user (see the auth section of [playwright-patterns.md](playwright-patterns.md)).
259
+ - `horizontalOverflowPx` above 0 at 390 px: the page scrolls sideways on phones.
260
+ - `axe`: list each rule with impact and a target; the `accessibility` skill explains how to judge them.
261
+
262
+ Written for this library. The start, wait, act, stop approach follows anthropics/skills `webapp-testing` (Apache-2.0, commit dbd4588f9e1033efb41dad4bef2f7947c8993d44, Copyright 2026 Anthropic, PBC); the script is new Node.js code, not a translation of its Python helper.