@llipe.com/dev-tasks 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/developer.md +11 -0
- package/.claude/agents/qa-engineer.md +31 -2
- package/.claude/agents/researcher.md +96 -0
- package/.claude/commands/planner.md +22 -7
- package/.claude/commands/product-engineer.md +37 -10
- package/.claude/commands/researcher.md +28 -0
- package/.claude/skills/activity-codebase-research/SKILL.md +158 -0
- package/.claude/skills/activity-contract-validation/SKILL.md +167 -0
- package/.claude/skills/activity-e2e-test-implementation/SKILL.md +156 -0
- package/.claude/skills/activity-integration-test-implementation/SKILL.md +156 -0
- package/.claude/skills/activity-test-standards/SKILL.md +41 -0
- package/.github/agents/developer.agent.md +11 -0
- package/.github/agents/planner.agent.md +22 -7
- package/.github/agents/product-engineer.agent.md +37 -10
- package/.github/agents/qa-engineer.agent.md +30 -3
- package/.github/agents/researcher.agent.md +96 -0
- package/.github/prompts/researcher.prompt.md +29 -0
- package/.github/skills/activity-codebase-research/SKILL.md +158 -0
- package/.github/skills/activity-contract-validation/SKILL.md +167 -0
- package/.github/skills/activity-e2e-test-implementation/SKILL.md +156 -0
- package/.github/skills/activity-integration-test-implementation/SKILL.md +156 -0
- package/.github/skills/activity-test-standards/SKILL.md +41 -0
- package/.kiro/agents/developer.md +11 -0
- package/.kiro/agents/planner.md +22 -7
- package/.kiro/agents/product-engineer.md +37 -10
- package/.kiro/agents/qa-engineer.md +30 -3
- package/.kiro/agents/researcher.md +99 -0
- package/.kiro/agents/verifier.md +1 -1
- package/.kiro/skills/activity-codebase-research/SKILL.md +158 -0
- package/.kiro/skills/activity-contract-validation/SKILL.md +167 -0
- package/.kiro/skills/activity-e2e-test-implementation/SKILL.md +156 -0
- package/.kiro/skills/activity-integration-test-implementation/SKILL.md +156 -0
- package/.kiro/skills/activity-test-standards/SKILL.md +41 -0
- package/AGENTS.md +91 -215
- package/CHANGELOG.md +33 -0
- package/CLAUDE.md +3 -2
- package/README.md +1 -0
- package/TESTING.md +19 -11
- package/package.json +1 -1
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: activity-e2e-test-implementation
|
|
3
|
+
description: "Author Playwright end-to-end tests from verifier scenario tables. Covers auth, state reset, CI config, and scenario-to-spec traceability. Use when writing or backfilling E2E tests."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Activity: E2E Test Implementation (Playwright)
|
|
7
|
+
|
|
8
|
+
Convert `verifier` Design Mode scenario tables into executable Playwright specs. This skill is the execution counterpart to `activity-e2e-test-design`, which produces the scenario tables.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
> **RFC 2119 Notice:** The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, **SHOULD**, **SHOULD NOT**, **RECOMMENDED**, **MAY**, and **OPTIONAL** in this document are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119).
|
|
13
|
+
|
|
14
|
+
## Goal
|
|
15
|
+
|
|
16
|
+
Produce Playwright E2E specs that fail when user-facing behavior is wrong. E2E tests assert on observable outcomes — never on internal state, database rows, or implementation details.
|
|
17
|
+
|
|
18
|
+
Consult `/TESTING.md` first. Its E2E layer boundary governs; this skill supplies the method.
|
|
19
|
+
|
|
20
|
+
## E2E Layer Boundary
|
|
21
|
+
|
|
22
|
+
- **MUST** assert on user-visible behavior (page content, navigation, HTTP responses).
|
|
23
|
+
- **MUST NOT** assert on internal state, database contents, or implementation details.
|
|
24
|
+
- **MUST NOT** mock the application under test. The full stack runs.
|
|
25
|
+
- **MAY** seed test data via API or database before the test begins.
|
|
26
|
+
|
|
27
|
+
## Part 1 — Playwright Prerequisite Contract
|
|
28
|
+
|
|
29
|
+
Before authoring specs, verify or establish these prerequisites:
|
|
30
|
+
|
|
31
|
+
### Authentication strategy
|
|
32
|
+
|
|
33
|
+
| Strategy | When to Use | How |
|
|
34
|
+
| ---------------------------------------------- | --------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
|
35
|
+
| **`storageState` via setup project** (default) | Multi-test suites sharing a logged-in session | A `global-setup.ts` or setup project authenticates once, saves `storageState.json`, and all specs reuse it. |
|
|
36
|
+
| **Per-test programmatic login** | Tests that need different users or roles | Each spec calls login API directly in `beforeEach`. Slower but flexible. |
|
|
37
|
+
| **API-token seeding** | API-only E2E, no browser UI | Set auth headers directly via `request.newContext()`. No browser needed. |
|
|
38
|
+
|
|
39
|
+
Default to `storageState` unless the test requires a different user or role per test.
|
|
40
|
+
|
|
41
|
+
### Seeded test users
|
|
42
|
+
|
|
43
|
+
- Create dedicated test users (not production accounts).
|
|
44
|
+
- Credentials reach CI via environment variables or secrets — never hardcoded in the repository.
|
|
45
|
+
- Use a deterministic naming convention: `e2e-user-admin@test.local`, `e2e-user-member@test.local`.
|
|
46
|
+
- Document how to create/reset test users in the project's test setup README.
|
|
47
|
+
|
|
48
|
+
### Base URL and environment resolution
|
|
49
|
+
|
|
50
|
+
- Use `baseURL` in `playwright.config.ts`, resolved from `process.env.BASE_URL` or `process.env.PLAYWRIGHT_BASE_URL`.
|
|
51
|
+
- Never hardcode `localhost:3000` — CI and staging use different URLs.
|
|
52
|
+
- Provide a `.env.test.example` showing required variables.
|
|
53
|
+
|
|
54
|
+
### Database state reset between runs
|
|
55
|
+
|
|
56
|
+
Choose one:
|
|
57
|
+
|
|
58
|
+
- **Migration + seed** (preferred): drop, migrate, seed before the test suite.
|
|
59
|
+
- **Transaction rollback**: wrap each test in a transaction (complex for E2E).
|
|
60
|
+
- **Snapshot restore**: restore a known-good database snapshot before runs.
|
|
61
|
+
|
|
62
|
+
State reset **MUST** run in `globalSetup` or a setup project, not inside individual specs.
|
|
63
|
+
|
|
64
|
+
### Trace, screenshot, and video retention
|
|
65
|
+
|
|
66
|
+
- **Default:** `on-first-retry` for traces, `only-on-failure` for screenshots, `off` for video.
|
|
67
|
+
- **CI:** retain artifacts as test attachments. Configure `outputDir` and artifact upload in CI workflow.
|
|
68
|
+
- Keep retention bounded — do not retain video for every passing test.
|
|
69
|
+
|
|
70
|
+
### Browser install in CI
|
|
71
|
+
|
|
72
|
+
- CI **MUST** install browsers explicitly: `npx playwright install --with-deps chromium`.
|
|
73
|
+
- Locally: `npx playwright install` on first run.
|
|
74
|
+
- Pin Playwright version in `package.json` — do not use `latest`.
|
|
75
|
+
|
|
76
|
+
### Sharding (large suites)
|
|
77
|
+
|
|
78
|
+
- For suites >50 specs: shard across CI workers using `--shard=1/4`, `--shard=2/4`, etc.
|
|
79
|
+
- Configure in CI matrix, not in `playwright.config.ts`.
|
|
80
|
+
|
|
81
|
+
## Part 2 — Scenario-to-Spec Mapping
|
|
82
|
+
|
|
83
|
+
### Traceability convention
|
|
84
|
+
|
|
85
|
+
Every Playwright spec **MUST** be traceable to a `verifier` Design Mode scenario.
|
|
86
|
+
|
|
87
|
+
**Annotation format:** Include `@scenario SC-{n}` in the test title or as a comment:
|
|
88
|
+
|
|
89
|
+
```typescript
|
|
90
|
+
// @scenario SC-1
|
|
91
|
+
test("SC-1: Successful login with valid credentials", async ({ page }) => {
|
|
92
|
+
// ...
|
|
93
|
+
});
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Or in the test title directly:
|
|
97
|
+
|
|
98
|
+
```typescript
|
|
99
|
+
test("[SC-1] User logs in with valid credentials", async ({ page }) => {
|
|
100
|
+
// ...
|
|
101
|
+
});
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
### Traceability chain
|
|
105
|
+
|
|
106
|
+
```
|
|
107
|
+
AC-{n} → SC-{n} (test plan) → .spec.ts file / test block (@scenario SC-{n}) → Pass/Fail
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
The `verifier` audit resolves scenario IDs to spec files by searching for `@scenario SC-{n}` or `SC-{n}:` patterns. Orphaned scenarios (in the plan but not in specs) are reported as uncovered.
|
|
111
|
+
|
|
112
|
+
### File naming
|
|
113
|
+
|
|
114
|
+
- One spec file per feature or user journey: `login.spec.ts`, `checkout.spec.ts`.
|
|
115
|
+
- Group related scenarios in one file. Do not create one file per scenario unless they are independent flows.
|
|
116
|
+
|
|
117
|
+
## Part 3 — Converting Scenario Tables to Specs
|
|
118
|
+
|
|
119
|
+
Given a scenario table from `activity-e2e-test-design`:
|
|
120
|
+
|
|
121
|
+
1. **Read the scenario.** Extract: preconditions, steps, expected result, pass criteria.
|
|
122
|
+
2. **Write the spec.** Map steps to Playwright actions. Map pass criteria to assertions.
|
|
123
|
+
3. **Annotate.** Add `@scenario SC-{n}` to the test.
|
|
124
|
+
4. **Handle preconditions.** Seed required state in `beforeAll`/`beforeEach` or a fixture.
|
|
125
|
+
5. **Assert observable outcomes.** Use `expect(page)` or `expect(response)` — never internal state.
|
|
126
|
+
|
|
127
|
+
### Retry and flakiness
|
|
128
|
+
|
|
129
|
+
- Use Playwright's built-in retry (`retries: 2` in config) for CI.
|
|
130
|
+
- Prefer explicit `await expect(locator).toBeVisible()` over arbitrary `waitForTimeout`.
|
|
131
|
+
- If a test is flaky, fix the root cause (race condition, state leak) rather than increasing retries.
|
|
132
|
+
- A flaky test that occasionally passes is worse than no test.
|
|
133
|
+
|
|
134
|
+
### Test isolation
|
|
135
|
+
|
|
136
|
+
- Each spec **MUST** be independent — runnable in any order, in parallel.
|
|
137
|
+
- Never rely on state from a previous test. Use fixtures or setup projects.
|
|
138
|
+
- Use `test.describe.configure({ mode: 'serial' })` only when tests represent a true sequential flow.
|
|
139
|
+
|
|
140
|
+
## Output
|
|
141
|
+
|
|
142
|
+
- Specs written or updated, by path
|
|
143
|
+
- Scenario-to-spec mapping table: `SC-{n}` → file path and test name
|
|
144
|
+
- Uncovered scenarios (in the plan but not yet implemented)
|
|
145
|
+
- Prerequisites missing (base URL, auth setup, browser install, etc.)
|
|
146
|
+
- Stated limitations (e.g., "no staging environment — E2E limited to localhost")
|
|
147
|
+
|
|
148
|
+
## Final Instructions
|
|
149
|
+
|
|
150
|
+
1. You **MUST** consult `/TESTING.md` and respect its E2E layer boundary.
|
|
151
|
+
2. You **MUST** annotate every spec with its scenario ID (`@scenario SC-{n}`).
|
|
152
|
+
3. You **MUST NOT** assert on internal state or implementation details.
|
|
153
|
+
4. You **MUST** verify Playwright prerequisites before authoring specs.
|
|
154
|
+
5. You **MUST** report uncovered scenarios from the test plan.
|
|
155
|
+
6. You **MUST** address flakiness causes, not mask them with retries.
|
|
156
|
+
7. You **MUST NOT** install dependencies — detect and recommend only.
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: activity-integration-test-implementation
|
|
3
|
+
description: "Author integration tests (Layer 2.5) against real databases — local via testcontainers/docker-compose/Supabase CLI, remote via testing environments. Covers migrations, RLS, pgTAP, fixtures, and fallback paths. Use when writing or backfilling integration tests."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Activity: Integration Test Implementation (Layer 2.5)
|
|
7
|
+
|
|
8
|
+
Author integration tests that exercise code against a real database. Layer 2.5 exists because mocking the data layer makes SQL errors, type mismatches, NULL handling, malformed joins, and RLS policies structurally invisible.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
> **RFC 2119 Notice:** The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, **SHOULD**, **SHOULD NOT**, **RECOMMENDED**, **MAY**, and **OPTIONAL** in this document are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119).
|
|
13
|
+
|
|
14
|
+
## Goal
|
|
15
|
+
|
|
16
|
+
Produce integration tests that fail when the data layer is wrong. If the database is mocked, the test belongs at Layer 2, not here.
|
|
17
|
+
|
|
18
|
+
Consult `/TESTING.md` first. Its Layer 2.5 boundary governs; this skill supplies the method.
|
|
19
|
+
|
|
20
|
+
## Layer 2.5 Boundary
|
|
21
|
+
|
|
22
|
+
- **MUST** use a real database (Postgres, SQLite, etc.) — not a mock or in-memory fake.
|
|
23
|
+
- **MUST NOT** mock the data layer. If you mock `sql`, `prisma`, `drizzle`, or the ORM client, the test is Layer 2.
|
|
24
|
+
- **MUST NOT** assert on SQL text or internal query shape. Assert on persisted state or returned data.
|
|
25
|
+
- **MAY** mock third-party external services (payment gateways, email providers) at the HTTP boundary.
|
|
26
|
+
|
|
27
|
+
## Part 1 — Local Integration Environment Detection
|
|
28
|
+
|
|
29
|
+
Detect the easiest applicable local option. Prefer in this order:
|
|
30
|
+
|
|
31
|
+
### 1. Testcontainers (preferred for isolation)
|
|
32
|
+
|
|
33
|
+
- Look for `@testcontainers/postgresql` or equivalent in `package.json`/`pom.xml`/`Cargo.toml`.
|
|
34
|
+
- Requires Docker. Check: `docker info` succeeds.
|
|
35
|
+
- Each test suite gets an ephemeral container — full isolation, no cleanup conflicts.
|
|
36
|
+
|
|
37
|
+
### 2. Docker Compose
|
|
38
|
+
|
|
39
|
+
- Look for `docker-compose.yml` or `compose.yml` with a database service.
|
|
40
|
+
- Requires Docker. Shared instance across tests — needs explicit state reset.
|
|
41
|
+
|
|
42
|
+
### 3. Supabase Local CLI
|
|
43
|
+
|
|
44
|
+
- Look for `supabase/config.toml` or `.supabase/` directory.
|
|
45
|
+
- Check: `supabase status` succeeds.
|
|
46
|
+
- Provides real Postgres with auth, storage, and edge functions locally.
|
|
47
|
+
|
|
48
|
+
### 4. Fallback — No Local Environment
|
|
49
|
+
|
|
50
|
+
- If Docker is unavailable and Supabase CLI is not installed, report the limitation.
|
|
51
|
+
- Emit: `SKIPPED(no local database environment — Docker unavailable, Supabase CLI not installed)`.
|
|
52
|
+
- Recommend environment setup as a follow-up. Do not silently omit.
|
|
53
|
+
|
|
54
|
+
## Part 2 — Fixtures, Seeding, and Lifecycle
|
|
55
|
+
|
|
56
|
+
### Test isolation
|
|
57
|
+
|
|
58
|
+
- Each test **SHOULD** run in a transaction that rolls back on completion (preferred).
|
|
59
|
+
- Alternative: truncate affected tables in `beforeEach`/`afterEach`.
|
|
60
|
+
- Alternative for testcontainers: fresh container per suite (slower but fully isolated).
|
|
61
|
+
|
|
62
|
+
### Seeding
|
|
63
|
+
|
|
64
|
+
- Shared seed data lives in one location per package (e.g., `test/fixtures/seed.sql` or `test/helpers/seed.ts`).
|
|
65
|
+
- Seeds **MUST** be idempotent — running twice produces the same state.
|
|
66
|
+
- Seeds **MUST NOT** depend on auto-increment IDs. Use deterministic UUIDs or named references.
|
|
67
|
+
|
|
68
|
+
### Deterministic teardown
|
|
69
|
+
|
|
70
|
+
- Every integration test file **MUST** clean up after itself or use rollback isolation.
|
|
71
|
+
- Tests **MUST NOT** depend on execution order.
|
|
72
|
+
- Tests **MUST NOT** leave residual data that affects other test files.
|
|
73
|
+
|
|
74
|
+
## Part 3 — Migration Clean-Apply Tests
|
|
75
|
+
|
|
76
|
+
- **MUST** test that all migrations apply cleanly from an empty database.
|
|
77
|
+
- **MUST** verify apply order (no duplicate prefixes, no gaps).
|
|
78
|
+
- **MUST** verify idempotency: running migrations twice does not error.
|
|
79
|
+
- For large stacks (50+ migrations): run full clean-apply on CI/scheduled; run latest N on PR.
|
|
80
|
+
- Report duplicate migration prefixes as a finding.
|
|
81
|
+
|
|
82
|
+
## Part 4 — Row-Level Security (RLS) Policy Tests
|
|
83
|
+
|
|
84
|
+
RLS tests **MUST** exercise the real database policy, not application-level tenant checks in JavaScript.
|
|
85
|
+
|
|
86
|
+
### Multi-session setup
|
|
87
|
+
|
|
88
|
+
- Create at least two database roles or authenticated sessions (Tenant A, Tenant B).
|
|
89
|
+
- Use `SET ROLE` or separate connections per tenant.
|
|
90
|
+
- For Supabase: use `supabase.auth.admin.createUser()` for test users, then authenticate with their JWTs.
|
|
91
|
+
|
|
92
|
+
### Required assertions
|
|
93
|
+
|
|
94
|
+
| Case | Assertion |
|
|
95
|
+
| ---------------------------- | ----------------------------------- |
|
|
96
|
+
| Tenant A reads own data | Returns expected rows |
|
|
97
|
+
| Tenant A reads Tenant B data | Returns empty set or error |
|
|
98
|
+
| Tenant A writes to Tenant B | Insert/update fails or is invisible |
|
|
99
|
+
| Unauthenticated access | Denied by policy |
|
|
100
|
+
| Service role bypasses RLS | Returns all rows (when expected) |
|
|
101
|
+
|
|
102
|
+
### Record limitations
|
|
103
|
+
|
|
104
|
+
If isolation is asserted against application-level checks rather than the real RLS policy, you **MUST** record: "Production RLS policy unverified — tested against application-layer tenant filtering only."
|
|
105
|
+
|
|
106
|
+
## Part 5 — pgTAP / Schema Contract Tests
|
|
107
|
+
|
|
108
|
+
Use pgTAP (or equivalent: `pg_prove`, raw SQL assertions) for:
|
|
109
|
+
|
|
110
|
+
- Table and column existence.
|
|
111
|
+
- Constraint verification (NOT NULL, UNIQUE, CHECK, FK).
|
|
112
|
+
- Function and trigger existence and signatures.
|
|
113
|
+
- Permission grants per role.
|
|
114
|
+
- Index existence for query-critical paths.
|
|
115
|
+
|
|
116
|
+
When pgTAP is unavailable, use raw `SELECT` assertions against `information_schema` or `pg_catalog`.
|
|
117
|
+
|
|
118
|
+
## Part 6 — Remote Integration Testing
|
|
119
|
+
|
|
120
|
+
### Default: read-only
|
|
121
|
+
|
|
122
|
+
- Inspect schema, permissions, RLS policies, and data shape without mutation.
|
|
123
|
+
- Use `EXPLAIN` for query plan verification without executing writes.
|
|
124
|
+
- No approval required for read-only operations.
|
|
125
|
+
|
|
126
|
+
### Testing environment writes
|
|
127
|
+
|
|
128
|
+
- **REQUIRE** explicit user approval before any write or migration apply.
|
|
129
|
+
- Connect only to a dedicated testing environment — never to production for writes.
|
|
130
|
+
- After writes: verify applied state, then clean up test data.
|
|
131
|
+
- Record: environment name, operation performed, approval evidence.
|
|
132
|
+
|
|
133
|
+
### No testing environment available
|
|
134
|
+
|
|
135
|
+
- Report: "Remote integration limited to read-only inspection. No testing environment configured."
|
|
136
|
+
- Recommend environment setup. Do not fabricate a pass.
|
|
137
|
+
|
|
138
|
+
## Output
|
|
139
|
+
|
|
140
|
+
- Tests written or updated, by path and layer
|
|
141
|
+
- Local environment detection result (which option selected, or fallback triggered)
|
|
142
|
+
- Migration clean-apply result
|
|
143
|
+
- RLS policy test coverage per table with RLS enabled
|
|
144
|
+
- Schema contract coverage (tables, constraints, functions)
|
|
145
|
+
- Remote integration status (read-only / testing-env / unavailable)
|
|
146
|
+
- Stated limitations where real-policy testing was not possible
|
|
147
|
+
|
|
148
|
+
## Final Instructions
|
|
149
|
+
|
|
150
|
+
1. You **MUST** consult `/TESTING.md` and respect its Layer 2.5 boundary.
|
|
151
|
+
2. You **MUST NOT** mock the data layer. If the database is mocked, the test is Layer 2.
|
|
152
|
+
3. You **MUST** detect the local environment and recommend the easiest applicable option.
|
|
153
|
+
4. You **MUST** report a limitation when no local or remote environment is available.
|
|
154
|
+
5. You **MUST** test RLS against the real policy layer, not application-level checks.
|
|
155
|
+
6. You **MUST** require explicit approval before any remote write operation.
|
|
156
|
+
7. You **MUST NOT** install dependencies — detect and recommend only.
|
|
@@ -92,6 +92,47 @@ A monorepo defines `test:node` as `app && scraper && db` while the `api` package
|
|
|
92
92
|
|
|
93
93
|
Correct output: **defect** — `api` is unreachable from the aggregate; the CI gate and the deploy gate both let untested backend code through. Reporting "all script names conform" here is a false pass.
|
|
94
94
|
|
|
95
|
+
## Part 4 — Integration, E2E, and Contract Infrastructure Detection
|
|
96
|
+
|
|
97
|
+
Detect the presence and configuration state of higher-layer test infrastructure. Report findings as **informational** — missing infrastructure is not a defect, but it informs the gap analysis.
|
|
98
|
+
|
|
99
|
+
### Integration testing infrastructure
|
|
100
|
+
|
|
101
|
+
| Check | What to look for |
|
|
102
|
+
| ----------------------- | ------------------------------------------------------------- |
|
|
103
|
+
| **Testcontainers** | `@testcontainers/postgresql` or similar in dependencies |
|
|
104
|
+
| **Docker Compose** | `docker-compose.yml` or `compose.yml` with a database service |
|
|
105
|
+
| **Supabase local CLI** | `supabase/config.toml` or `.supabase/` directory |
|
|
106
|
+
| **Docker availability** | `docker info` succeeds (report if unavailable) |
|
|
107
|
+
|
|
108
|
+
### E2E testing infrastructure
|
|
109
|
+
|
|
110
|
+
| Check | What to look for |
|
|
111
|
+
| ------------------------- | -------------------------------------------------------------------------------- |
|
|
112
|
+
| **Playwright config** | `playwright.config.ts` or `playwright.config.js` present |
|
|
113
|
+
| **Base URL configured** | Config references `process.env.BASE_URL` or equivalent (not hardcoded localhost) |
|
|
114
|
+
| **Auth setup project** | A `global-setup` or setup project defined in Playwright config |
|
|
115
|
+
| **Browser install in CI** | CI workflow includes `playwright install` step |
|
|
116
|
+
| **`test:e2e` script** | Present in `package.json` and reachable from aggregate |
|
|
117
|
+
|
|
118
|
+
### Contract validation infrastructure
|
|
119
|
+
|
|
120
|
+
| Check | What to look for |
|
|
121
|
+
| -------------------------- | ------------------------------------------------------ |
|
|
122
|
+
| **OpenAPI spec** | `openapi.yaml`, `openapi.json`, `swagger.*` in repo |
|
|
123
|
+
| **AsyncAPI spec** | `asyncapi.yaml`, `asyncapi.json` in repo |
|
|
124
|
+
| **`dt` CLI** | `which dt` succeeds |
|
|
125
|
+
| **`test:contract` script** | Present in `package.json` and reachable from aggregate |
|
|
126
|
+
|
|
127
|
+
### Reporting
|
|
128
|
+
|
|
129
|
+
Report each finding as:
|
|
130
|
+
|
|
131
|
+
- `[INFO] <infrastructure> detected: <path or status>`
|
|
132
|
+
- `[INFO] <infrastructure> not found — Layer {X} tests require this`
|
|
133
|
+
|
|
134
|
+
These are informational findings for the gap analysis. They do **NOT** block or fail the standards check.
|
|
135
|
+
|
|
95
136
|
## Failure Modes
|
|
96
137
|
|
|
97
138
|
| Condition | Required behavior |
|
|
@@ -138,12 +138,23 @@ Follow `.kiro/steering/implement.md`:
|
|
|
138
138
|
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
139
139
|
| `product-engineer` | Produces the task lists and refined issues that `developer` executes |
|
|
140
140
|
| `planner` | Orchestrates multi-story runs — delegates each story to `developer` in Execute Mode with an integration branch override |
|
|
141
|
+
| `researcher` | May be invoked by `developer` for troubleshooting/diagnosis when a bug or regression is unclear; conditional, never mandatory |
|
|
141
142
|
| `verifier` | Invoked by `developer` in `audit` mode, post-implementation and pre-PR-ready, mandatory and non-skippable — reports fidelity/drift; findings route to `product-engineer`'s `activity-drift-reconciliation` for remediation |
|
|
142
143
|
| `qa-engineer` | Invoked by `developer` at the completion gate, before the `verifier` audit — owns the testing standard, missing test harnesses, and coverage/gap reporting; `developer` still authors feature tests under rule 19 |
|
|
143
144
|
| `technical-writer` | Invoked by `developer` before PR is marked ready — updates `/docs` |
|
|
144
145
|
| `housekeeping` | Can be invoked during implementation for lint/type/test-wiring fixes |
|
|
145
146
|
| `github-ops` | Defines conventions for all GitHub artifacts — `developer` follows these rules |
|
|
146
147
|
|
|
148
|
+
### Codebase Research for Troubleshooting (Conditional)
|
|
149
|
+
|
|
150
|
+
When diagnosing a bug, regression, or unclear failure during implementation, you **SHOULD** invoke `researcher` if:
|
|
151
|
+
|
|
152
|
+
- The failure spans multiple modules and the root cause is not immediately apparent.
|
|
153
|
+
- The area is unfamiliar or lacks documentation.
|
|
154
|
+
- Multiple hypotheses need codebase evidence to disambiguate.
|
|
155
|
+
|
|
156
|
+
**Skip when:** the failure is localized to a single file or function, or the root cause is already identified. This is a recommended diagnostic tool, not a mandatory gate.
|
|
157
|
+
|
|
147
158
|
---
|
|
148
159
|
|
|
149
160
|
## Autonomous Behavior Contract
|
package/.kiro/agents/planner.md
CHANGED
|
@@ -84,6 +84,20 @@ If any required input is missing, ask one focused question with a default option
|
|
|
84
84
|
|
|
85
85
|
---
|
|
86
86
|
|
|
87
|
+
## Codebase Research for Pre-Orchestration (Conditional)
|
|
88
|
+
|
|
89
|
+
Before parsing stories and building the dependency graph, you **SHOULD** invoke `researcher` if:
|
|
90
|
+
|
|
91
|
+
- The stories span multiple modules or packages whose dependencies are not yet established.
|
|
92
|
+
- The orchestration involves unfamiliar areas of the codebase.
|
|
93
|
+
- Story ordering requires understanding cross-module data flow or contract boundaries.
|
|
94
|
+
|
|
95
|
+
**Skip when:** story dependencies are already explicit in the task file, the codebase is small and well-understood, or a non-stale research artifact already covers the scope.
|
|
96
|
+
|
|
97
|
+
When triggered, delegate to `researcher` with the orchestration question (e.g., "What are the dependency relationships between modules X, Y, and Z?"). Consume the resulting `/workstream/research-*.md` artifact to inform dependency ordering in Phase 1.
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
87
101
|
## Phase 0 - Discover Task Source
|
|
88
102
|
|
|
89
103
|
### Option A - File in /workstream
|
|
@@ -456,15 +470,16 @@ Planner **MUST NOT** allow multiple open story PRs to merge concurrently.
|
|
|
456
470
|
After all stories are merged into integration:
|
|
457
471
|
|
|
458
472
|
1. Run full integration test suite on integration branch.
|
|
459
|
-
2. Invoke `
|
|
460
|
-
3. Invoke `
|
|
461
|
-
4.
|
|
462
|
-
5.
|
|
463
|
-
6.
|
|
464
|
-
7.
|
|
473
|
+
2. Invoke `qa-engineer` at **PRD scope** (all packages affected by the merged stories, all layers) for a coverage and gap rollup. Report a PRD-level `coverage_gate` that aggregates per-story gates. Scope to packages affected by the merged stories, not the entire workspace.
|
|
474
|
+
3. Invoke `verifier` in `audit` mode for a **PRD-level rollup audit** against the full integrated scope (all merged stories on the integration branch, cross-checked against the source PRD/spec/milestone). This is mandatory and non-skippable — it runs whether or not any story-level drift was previously reported. Post the resulting human-readable summary to the plan/milestone issue via `github-ops` comment conventions. Rollup drift findings **MUST NOT** block the consolidated PR handoff; Unintended/Intended drift routes to `product-engineer`'s `activity-drift-reconciliation` flow.
|
|
475
|
+
4. Invoke `technical-writer` for a planner-level drift/stale-doc validation pass against integrated changes; unresolved drift **MUST** block handoff.
|
|
476
|
+
5. Open one consolidated PR from integration branch to `main`.
|
|
477
|
+
6. **Do NOT merge.** Notify the user that the consolidated PR is ready for their review.
|
|
478
|
+
7. Wait for the user to approve and merge the PR into `main`.
|
|
479
|
+
8. Before final handoff, **MUST** ensure the local working branch is the integration branch used for this run:
|
|
465
480
|
- Preferred: run `git checkout integration/<plan-id>-<short-description>`.
|
|
466
481
|
- Alternative (if checkout is not possible in the current runtime): explicitly verify and report current branch, and provide the exact checkout command the user can run.
|
|
467
|
-
|
|
482
|
+
9. Final user response **MUST** include a `PR Directives (User Action Required)` section with:
|
|
468
483
|
- consolidated PR URL
|
|
469
484
|
- current CI/check status
|
|
470
485
|
- exact required user actions: review, approve, and merge
|
|
@@ -69,11 +69,13 @@ Chains: `refine` → `plan`. Produces a lightweight refinement doc under `/works
|
|
|
69
69
|
|
|
70
70
|
Detect mode from user input:
|
|
71
71
|
|
|
72
|
-
| Input | Mode | Activity Chain
|
|
73
|
-
| --------------------------------- | ---------------- |
|
|
74
|
-
| "init" or foundation request | **Init Mode** | `activity-init`
|
|
75
|
-
| Feature description / PRD request | **Feature Mode** | `activity-refine` → `activity-generate-spec` → `activity-generate-stories` → `activity-publish-github` → `plan` → `verifier` (Design Mode) recommendation |
|
|
76
|
-
| GitHub Issue number + repo | **Issue Mode** | `activity-refine` → `plan` → `verifier` (Design Mode) recommendation |
|
|
72
|
+
| Input | Mode | Activity Chain |
|
|
73
|
+
| --------------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
74
|
+
| "init" or foundation request | **Init Mode** | `activity-init` |
|
|
75
|
+
| Feature description / PRD request | **Feature Mode** | `activity-refine` → [`researcher`] → `activity-generate-spec` → `activity-generate-stories` → `activity-publish-github` → `plan` → `verifier` (Design Mode) recommendation |
|
|
76
|
+
| GitHub Issue number + repo | **Issue Mode** | [`researcher`] → `activity-refine` → `plan` → `verifier` (Design Mode) recommendation |
|
|
77
|
+
|
|
78
|
+
> **Note:** `[researcher]` steps are conditional — recommended when trigger heuristics are met, skipped for trivial or single-file changes. See "Codebase Research (Conditional)" below.
|
|
77
79
|
|
|
78
80
|
If the user explicitly asks to start from a later activity (e.g., "generate stories from this spec"), you **MAY** skip earlier steps when the required input artifacts already exist and are approved.
|
|
79
81
|
|
|
@@ -165,6 +167,7 @@ This agent invokes the following **skills** for each activity. You **MUST** load
|
|
|
165
167
|
| -------------------------------- | ------------------------------------------ |
|
|
166
168
|
| Initialize foundation | `activity-init` |
|
|
167
169
|
| Refine scope / create PRD | `activity-refine` |
|
|
170
|
+
| Codebase research (conditional) | `activity-codebase-research` |
|
|
168
171
|
| Generate technical specification | `activity-generate-spec` |
|
|
169
172
|
| Generate user stories | `activity-generate-stories` |
|
|
170
173
|
| Publish stories to GitHub | `activity-publish-github` |
|
|
@@ -219,7 +222,19 @@ Follow the `activity-refine` skill (PRD mode):
|
|
|
219
222
|
4. Produce PRD: `/docs/requirements/prd-[feature-name].md`
|
|
220
223
|
5. Present for user review and iterate.
|
|
221
224
|
|
|
222
|
-
#### Phase 2 —
|
|
225
|
+
#### Phase 2 — Codebase Research (Conditional)
|
|
226
|
+
|
|
227
|
+
After refining and before generating the specification, evaluate whether `researcher` SHOULD be invoked. Trigger heuristics:
|
|
228
|
+
|
|
229
|
+
- A spec is about to be written for an area with existing implementation.
|
|
230
|
+
- The change plausibly spans more than one module, package, or repository.
|
|
231
|
+
- The target area is unfamiliar or undocumented in `/workstream`.
|
|
232
|
+
|
|
233
|
+
**Skip when:** the feature is entirely greenfield with no existing code to map, or a non-stale research artifact already covers the scope.
|
|
234
|
+
|
|
235
|
+
When triggered, delegate to `researcher` with the PRD scope as the research question. Consume the resulting `/workstream/research-*.md` artifact as input to Phase 3 (Generate Specification).
|
|
236
|
+
|
|
237
|
+
#### Phase 3 — Generate Specification
|
|
223
238
|
|
|
224
239
|
Follow the `activity-generate-spec` skill:
|
|
225
240
|
|
|
@@ -228,7 +243,7 @@ Follow the `activity-generate-spec` skill:
|
|
|
228
243
|
3. Produce specification: `/workstream/specification-[prd-name].md`
|
|
229
244
|
4. Present for user review and iterate.
|
|
230
245
|
|
|
231
|
-
#### Phase
|
|
246
|
+
#### Phase 4 — Generate Stories
|
|
232
247
|
|
|
233
248
|
Follow the `activity-generate-stories` skill:
|
|
234
249
|
|
|
@@ -237,7 +252,7 @@ Follow the `activity-generate-stories` skill:
|
|
|
237
252
|
3. Produce stories: `/workstream/user-stories-[prd-name].md`
|
|
238
253
|
4. Present for user review and iterate.
|
|
239
254
|
|
|
240
|
-
#### Phase
|
|
255
|
+
#### Phase 5 — Publish to GitHub
|
|
241
256
|
|
|
242
257
|
Follow the `activity-publish-github` skill:
|
|
243
258
|
|
|
@@ -245,7 +260,7 @@ Follow the `activity-publish-github` skill:
|
|
|
245
260
|
2. Publish each story as a GitHub Issue (delegate to `github-ops`).
|
|
246
261
|
3. Produce publication report: `/workstream/github-publication-[prd-name].md`
|
|
247
262
|
|
|
248
|
-
#### Phase
|
|
263
|
+
#### Phase 6 — Plan
|
|
249
264
|
|
|
250
265
|
Follow the `plan` steering document:
|
|
251
266
|
|
|
@@ -253,7 +268,7 @@ Follow the `plan` steering document:
|
|
|
253
268
|
2. Generate task list: `/workstream/tasks-[prd-name]-plan.md`
|
|
254
269
|
3. Update GitHub Issues with checklists.
|
|
255
270
|
|
|
256
|
-
#### Phase
|
|
271
|
+
#### Phase 7 — Verifier Design Recommendation (Test-First)
|
|
257
272
|
|
|
258
273
|
After the task list is finalized:
|
|
259
274
|
|
|
@@ -265,6 +280,18 @@ This enforces the repository's **test-first design** default: tests and acceptan
|
|
|
265
280
|
|
|
266
281
|
### Issue Mode
|
|
267
282
|
|
|
283
|
+
#### Codebase Research (Conditional)
|
|
284
|
+
|
|
285
|
+
Before refining, evaluate whether `researcher` SHOULD be invoked. Trigger heuristics:
|
|
286
|
+
|
|
287
|
+
- The target area is unfamiliar or undocumented in `/workstream`.
|
|
288
|
+
- The change plausibly spans more than one module, package, or repository.
|
|
289
|
+
- The task is diagnostic (bug, regression, "why does X happen").
|
|
290
|
+
|
|
291
|
+
**Skip when:** the issue is a trivial single-file change, a typo/copy fix, or a non-stale research artifact already covers the same scope.
|
|
292
|
+
|
|
293
|
+
When triggered, delegate to `researcher` with the issue title/description as the research question. Consume the resulting `/workstream/research-*.md` artifact as input to Phase A.
|
|
294
|
+
|
|
268
295
|
#### Phase A — Refine Issue
|
|
269
296
|
|
|
270
297
|
Follow the `activity-refine` skill (Issue Refinement mode):
|
|
@@ -40,7 +40,7 @@ Invoke `activity-test-standards`. It establishes or refreshes `/TESTING.md` for
|
|
|
40
40
|
- Verify script reachability: every package containing tests **MUST** be reachable from the aggregate test command, and the CI and deploy gates **MUST** invoke that aggregate. A correct script name that omits a package is a defect.
|
|
41
41
|
- Preserve consumer-authored content. Additions are additive.
|
|
42
42
|
|
|
43
|
-
### 2. Author or fill missing tests
|
|
43
|
+
### 2. Author or fill missing tests (Layers 1-2)
|
|
44
44
|
|
|
45
45
|
Invoke `activity-test-implementation` for the requested scope.
|
|
46
46
|
|
|
@@ -49,7 +49,34 @@ Invoke `activity-test-implementation` for the requested scope.
|
|
|
49
49
|
- Security-negative tests are **mandatory** for every authentication and authorization path: invalid signature, expired credential, wrong issuer or audience, tampered claims.
|
|
50
50
|
- A passing suite over a permissive implementation is a finding, not a pass.
|
|
51
51
|
|
|
52
|
-
###
|
|
52
|
+
### 2.5. Integration tests (Layer 2.5) — conditional
|
|
53
|
+
|
|
54
|
+
Invoke `activity-integration-test-implementation` for the requested scope.
|
|
55
|
+
|
|
56
|
+
- **Condition:** Run only when Layer 2.5 is configured in `/TESTING.md` (not `<!-- unfilled -->`). If Layer 2.5 is not configured, emit `SKIPPED(Layer 2.5 not configured in TESTING.md)` and proceed to the next step.
|
|
57
|
+
- Detect the local environment (testcontainers, docker-compose, Supabase local CLI) and recommend the easiest option.
|
|
58
|
+
- Write integration tests: migration clean-apply, RLS policy assertions, schema contracts.
|
|
59
|
+
- Report limitations when no local or remote environment is available.
|
|
60
|
+
|
|
61
|
+
### 3. E2E tests — conditional
|
|
62
|
+
|
|
63
|
+
Invoke `activity-e2e-test-implementation` for the requested scope.
|
|
64
|
+
|
|
65
|
+
- **Condition:** Run only when the E2E layer is configured in `/TESTING.md` (not `<!-- unfilled -->`). If E2E is not configured, emit `SKIPPED(E2E layer not configured in TESTING.md)` and proceed to the next step.
|
|
66
|
+
- Convert verifier scenario tables into Playwright specs with `@scenario SC-{n}` traceability.
|
|
67
|
+
- Verify Playwright prerequisites (auth, base URL, state reset, browser install).
|
|
68
|
+
- Report uncovered scenarios from the test plan.
|
|
69
|
+
|
|
70
|
+
### 4. Contract validation — conditional
|
|
71
|
+
|
|
72
|
+
Invoke `activity-contract-validation` for the requested scope.
|
|
73
|
+
|
|
74
|
+
- **Condition:** Run only when the Contract Validation layer is configured in `/TESTING.md` (not `<!-- unfilled -->`) or when OpenAPI/AsyncAPI specs are detected in the repository. If neither condition is met, emit `SKIPPED(no contract validation layer configured and no API specs found)` and proceed to the next step.
|
|
75
|
+
- Detect `dt` availability. If unavailable, emit `SKIPPED(dt not installed)` with manual instructions.
|
|
76
|
+
- Run `dt verify contract-diff`, `impact`, and `drift`.
|
|
77
|
+
- Report breaking changes as critical findings. Report drift as non-blocking.
|
|
78
|
+
|
|
79
|
+
### 5. Coverage and gap report
|
|
53
80
|
|
|
54
81
|
Invoke `activity-coverage-gap-analysis`.
|
|
55
82
|
|
|
@@ -74,7 +101,7 @@ You **MUST NOT** edit:
|
|
|
74
101
|
|
|
75
102
|
## Non-Negotiable Rules
|
|
76
103
|
|
|
77
|
-
1. You **MUST** run all
|
|
104
|
+
1. You **MUST** run all steps in order, every invocation. Steps 2.5, 3, and 4 are conditional — skip with a reason when the layer is not configured.
|
|
78
105
|
2. You **MUST NOT** report a pass for anything you could not measure.
|
|
79
106
|
3. You **MUST** report an unfilled `/TESTING.md` placeholder as unfilled.
|
|
80
107
|
4. You **MUST** treat a green suite over an insecure or permissive implementation as a finding.
|