@1aboveio/skills 0.0.0-stage → 1.0.0

package/skills/engineering/e2e-test/references/authoring/auth-flows.md

@@ -0,0 +1,159 @@
+# Auth Flows (login, signup, forgot-password)
+
+> Read when: testing authentication surfaces — login, signup, password reset, magic link, OAuth callback. These span three tiers and cross an external boundary (email/SMS), which is why they are the most commonly mis-tested feature in a codebase.
+
+Auth is special on two counts, both easy to get wrong:
+
+1. **It spans all three tiers** — the same feature owes a render-health test, functional negatives, *and* a real journey. Confusing them produces a test that looks like a journey but proves nothing.
+2. **It crosses an external boundary you don't own** — reset/verification email. The token round-trip is the whole point, and must be proven without calling a live provider.
+
+## The anti-pattern this exists to kill
+
+A test tagged `@level journey` that only checks *a form element is visible*:
+
+```typescript
+// ✗ A smoke test wearing a journey label. Proves nothing; can't fail the way the
+//   name promises. When the form fails to render at all (e.g. a seed/cache race),
+//   the disjunction just goes falsy with no specific signal.
+test('Customer login page loads and renders login form', async ({ page }) => {
+  const response = await page.goto(`/${MERCHANT}/account/login`);
+  expect(response?.ok()).toBeTruthy();
+  const hasEmail = await page.getByLabel(/email/i).isVisible().catch(() => false);
+  const hasPassword = await page.getByLabel(/password/i).isVisible().catch(() => false);
+  const hasButton = await page.getByRole('button', { name: /login/i }).isVisible().catch(() => false);
+  expect(hasEmail || hasPassword || hasButton).toBeTruthy();   // weak disjunction
+});
+```
+
+Two syntactic tells — `.catch(() => false)` (swallows a locator throw, defeating web-first retry) and `expect(a || b || c).toBeTruthy()` (passes if *any* branch holds) — are flagged as leads for the reviewer (`swallowed-assertion`, `weak-assertion`). Writing them means a vacuous check, not a journey — replace with a real journey below.
+
+## Login
+
+Separate two jobs that get conflated:
+
+### 1. Proving the login flow itself
+
+A Tier-3 journey drives credentials to an **authenticated outcome** (not form presence), backed by Tier-2 functional negatives where most real bugs live.
+
+```typescript
+// @covers /[merchant]/account/login
+// @level journey
+test('customer logs in and lands authenticated', async ({ page }) => {
+  const user = await factory.customer({ password: 'Correct-horse-9' }); // seed real state
+  await page.goto(`/${MERCHANT}/account/login`);
+  await page.getByLabel(/email/i).fill(user.email);
+  await page.getByLabel(/password/i).fill('Correct-horse-9');
+  await page.getByRole('button', { name: /log in|sign in/i }).click();
+  // Assert a CONCRETE post-login signal — redirect + authed-only content:
+  await expect(page).toHaveURL(new RegExp(`/${MERCHANT}/account$`));
+  await expect(page.getByRole('heading', { name: /my account/i })).toBeVisible();
+});
+```
+
+Back it with **Tier-2 functional negatives** where most real bugs live — e.g. `@level functional` "wrong password shows error and stays unauthenticated": fill a bad password, assert the error text is visible *and* `toHaveURL(/login/)` (did NOT get in).
+
+Rules:
+
+- **Seed the user** with a factory/seed script — never depend on ambient data. Missing state is the test's *arrange* step, never a skip reason.
+- **Name every seeded account with the `e2e-` prefix** (see [the naming convention](#the-e2e--account-naming-convention) below) — the factory should emit it by default, not leave it to each call site.
+- **Assert authenticated state**, not form presence: a redirect to an authed route, an authed-only element, or the session cookie. "A button is visible" is Tier-1 render-health.
+- **Don't mock first-party auth endpoints** in the journey — real DB, real session. Intercepting `/api/auth/*` makes it `Over-mocked` (`ensure-coverage` → `references/depth/mock-policy.md`).
+
+### 2. Login as a precondition for other journeys
+
+Don't re-type credentials in every test. Log in once and reuse the session via a shared auth fixture (`storageState`) or `E2E_AUTH_BYPASS`; other journeys start already-authed and only the login spec proves login itself. The cure for "no session" is to seed it, never skip.
+
+## A session fixture asserts its own preconditions, and throws
+
+A minted session the app then rejects is the worst failure mode in this file: the request fails closed, nothing renders, and the spec dies on an **undiagnostic timeout**. Nothing in the diff looks wrong — there is no assertion against a 404 to spot, because the assertion was never reached. The lane goes on reporting green-or-flaky while proving nothing.
+
+The recurring cause is a value the **app and the harness each default independently**: a tenant/org slug, an auth secret, a base URL, an API version. They agree until one side's default moves.
+
+**Rules, in the order they close the hole:**
+
+- **One owner per shared value.** A single constant mirrors the app's own resolver (`getRequiredOrgSlug()`, not a re-typed `?? "skyee"`), and the Playwright config sets the env var *from that constant* — in `webServer.env` **and** any inline command string, which drift apart otherwise.
+- **Make the wrong value inexpressible.** Identities come from constructor functions (`authorizedIdentity()`, `denyOnlyIdentity()`) that fill the shared value themselves. If no call site can state it, no call site can state it wrongly — a rule prose cannot enforce, structure can.
+- **Assert at mint time and throw with a diagnosis.** The fixture checks the value as it mints and, on mismatch, throws naming the identity id, *both* values, and the owning constant's file path. A loud throw is the entire difference between a five-minute fix and a five-round one; a timeout names nothing.
+- **Close the vacuous-pass hole in deny fixtures.** A fixture that deliberately uses a wrong value opts in explicitly (`{ allowWrongOrg: true }`), and that flag is **rejected on a right-value identity** — otherwise a denial assertion passes for the wrong reason, and the negative test is as vacuous as the positive one.
+
+The first two rules are what make this stick: a value no call site can express is a value no call site can get wrong. (Named incident: skyee-ai-risk #1144, where a tenant slug defaulted independently in four files kept authorized lanes green for five review rounds while exercising nothing.)
+
+## The `e2e-*` account naming convention
+
+Every account, email, and tenant a test seeds carries an **`e2e-` prefix**:
+
+```
+account   e2e-<role>-<runid>@<test-domain>     e2e-customer-x7f2@example.test
+tenant    e2e-<name>                           e2e-synthetic
+username  e2e-<role>-<runid>                   e2e-admin-x7f2
+```
+
+Bake it into the **factory default** — `factory.customer()` emits a conforming email without the call site asking — so the convention can't erode one test at a time. Why a naming convention and not just "use seeded data":
+
+- **Identifiable at a glance.** In a DB dump, log line, admin UI, or bug report, `e2e-` says "test artifact" with no lookup.
+- **Bulk-cleanable.** `DELETE FROM users WHERE email LIKE 'e2e-%'` is a safe janitor job; without the prefix, cleanup needs a bookkeeping table that drifts.
+- **Mechanically guarded.** The app/monitoring can special-case the prefix (exclude from analytics/emails/billing), and `golden-path`'s deployed-target rule builds on it: a mutating golden path off-`local` runs only against a declared synthetic tenant, and `golden-path validate` warns when a tenant is not `e2e-`-prefixed. (Production is governed by the stricter **read-only production profile** (`prod-preview`/`prod-active`), where a tenant grants nothing — this convention is what makes *staging* runs safe, not prod ones.)
+- **Unambiguous in shared environments.** On staging/preview where real-ish data coexists, the prefix keeps a cleanup script or mutating smoke run from touching anything live.
+
+Use a **run-scoped suffix** (a short random id per run) so parallel workers and repeated CI runs never collide on unique columns — never a fixed `e2e-test@example.com` singleton, which serializes workers and flakes on re-runs.
+
+## Forgot password — the email boundary is the whole flow
+
+Email/SMS is an **external boundary you don't own** → *real internals, faked externals, nothing in between* (`ensure-coverage/references/depth/mock-policy.md`).
+
+- ✗ Don't call a live email provider in CI (non-deterministic, flaky, leaks keys).
+- ✗ Don't skip it because "email is external" — the exact rationalization the no-skip policy forbids; **sandbox the boundary** instead.
+- ✗ Don't assert "the form renders" and call it a journey.
+
+✓ Fake the mail adapter so the app writes to a test outbox, then drive the **full durable-state** round-trip:
+
+```typescript
+// @covers /[merchant]/account/forgot-password
+// @level journey
+test('reset password end to end', async ({ page }) => {
+  const user = await factory.customer({ password: 'old-pass-1' });
+
+  // 1. Request reset
+  await page.goto(`/${MERCHANT}/account/forgot-password`);
+  await page.getByLabel(/email/i).fill(user.email);
+  await page.getByRole('button', { name: /reset|send/i }).click();
+  await expect(page.getByText(/check your (email|inbox)/i)).toBeVisible();
+
+  // 2. Read the token from the FAKE outbox (or directly from the DB token table —
+  //    both are real internals; live SMTP is the banned outward reach)
+  const link = await fakeMail.lastResetLink(user.email);
+
+  // 3. Complete the reset
+  await page.goto(link);
+  await page.getByLabel(/new password/i).fill('new-pass-2');
+  await page.getByRole('button', { name: /set|update/i }).click();
+
+  // 4. Prove the durable change: new password works
+  await login(page, { email: user.email, password: 'new-pass-2' });
+  await expect(page.getByRole('heading', { name: /my account/i })).toBeVisible();
+});
+```
+
+- Reading the token from the **fake outbox** also proves the send wiring; reading it from the **DB token table** proves only the token logic. Either is real-internal and valid — pick based on whether email dispatch is part of the contract you're proving.
+- Cover the negatives as Tier-2 functional against the real DB: **expired token**, **already-used token**, **unknown email** (should not reveal account existence).
+
+## CI wiring
+
+| Concern | Practice |
+|---|---|
+| **Where they run** | The **blocking** E2E lane. Once email is faked, both flows are fully hermetic — *no quarantine*. Quarantine is only for genuinely un-sandboxable externals (e.g. a real-provider deliverability check), which is a separate secret-gated test with a dedicated mailbox ([ci-integration.md](../ci-integration.md) → Quarantine lane). |
+| **Seed timing** | Migrate → seed → **start servers** → run. Seed the user/tenant *before* servers boot, or bypass caches for E2E (e.g. an `ALLOW_E2E_ROUTES` flag). A freshly-seeded row a cached read can't yet see is a classic auth-journey flake. |
+| **Email adapter** | Inject a fake/in-memory mail adapter via env in CI; never real keys, never a live provider. |
+| **Presentation** | The presentation sweep ([presentation-sweep.md](../presentation-sweep.md)) catches "the form didn't render at all" — the bug a green functional run sails past. It is a **separate run**; the journey assertion is not a substitute. |
+| **Getting to green** | Never weaken assertions or skip to pass — the reviewer's manufactured-green check and the `weak-assertion`/`swallowed-assertion` leads flag exactly that. |
+
+## Checklist
+
+- [ ] Login journey asserts **authenticated state** (redirect / authed element / session cookie), not form presence.
+- [ ] User/session **seeded** before server boot; caches bypassed for E2E.
+- [ ] Values the app and harness share (tenant/org, secret, base URL) have **one owner**, and the session fixture **throws** on mismatch rather than letting the lane time out.
+- [ ] Seeded accounts/tenants carry the **`e2e-` prefix** with a run-scoped suffix (factory default, not per-call-site).
+- [ ] Forgot-password drives the **full reset → new-password-works** loop with a **faked mail boundary**.
+- [ ] Negatives (wrong password, expired/used token, unknown email) are **Tier-2 functional** against the real DB.
+- [ ] No first-party `page.route`/MSW interception in the journeys; no `.catch(() => false)`; no disjunctive `toBeTruthy()`.
+- [ ] No skips; the presentation sweep runs separately from the journeys.

package/skills/engineering/e2e-test/references/authoring/playwright-config.md

@@ -0,0 +1,73 @@
+# Playwright Config
+
+> Read when: creating or tuning `playwright.config.ts`, or dropping the E2E rules into a repo's AGENTS.md.
+
+```typescript
+import { defineConfig, devices } from '@playwright/test';
+
+export default defineConfig({
+  testDir: './e2e',
+  // Timeout convention: 15s for any single action/assertion — a real state change
+  // arrives in seconds; anything slower is a bug the test should surface, not wait out.
+  // The per-test ceiling stays higher (a journey is many 15s-bounded steps).
+  timeout: 60_000,
+  expect: { timeout: 15_000 },
+  fullyParallel: true,
+  forbidOnly: !!process.env.CI,
+  retries: process.env.CI ? 2 : 0,
+  // Parallel in CI too — wall clock ≈ slowest spec, not the sum. E2E_WORKERS is the
+  // CI-side knob (see ../ci-playwright-container.md → Resource Envelope and Workers);
+  // 4 fits a 4 CPU / 16 GB job. Drop to 2 only on OOM/browser crashes.
+  workers: process.env.CI ? Number(process.env.E2E_WORKERS) || 4 : undefined,
+  reporter: process.env.CI
+    ? [['github'], ['json', { outputFile: 'e2e-results.json' }], ['html', { open: 'never' }]]
+    : [['list'], ['html', { open: 'never' }]],
+  use: {
+    baseURL: process.env.BASE_URL || 'http://localhost:3000',
+    actionTimeout: 15_000,
+    navigationTimeout: 15_000,
+    trace: 'retain-on-failure',
+    screenshot: 'only-on-failure',
+    ignoreHTTPSErrors: true,
+  },
+  projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],
+  webServer: {
+    command: 'npm run dev',
+    url: process.env.BASE_URL || 'http://localhost:3000',
+    timeout: 120_000,
+    reuseExistingServer: !process.env.CI,
+  },
+});
+```
+
+## AGENTS.md section
+
+Drop into each repo's AGENTS.md or CLAUDE.md:
+
+```markdown
+## E2E Testing
+
+### Rules
+1. Every new route/endpoint MUST be added to `e2e/route-manifest.json` (the coverage audit will fail otherwise)
+2. Every new route needs at least a Tier 1 render-health test
+3. Interactive routes need Tier 2 functional tests (success + error + edge cases)
+4. Core business flows need Tier 3 journey tests — the golden paths (multi-step, real API); tag the do-not-ship subset `@golden-path`
+5. Directory: `render/`, `functional/`, `journeys/`
+6. Use `getByRole`/`getByLabel`/`getByText` -- not CSS selectors
+7. No `waitForTimeout()` -- wait for real state
+8. Run route discovery and reachability analysis before E2E coverage audits; do not audit from memory, sampling, or file enumeration alone
+9. Never skip a test: seed the state, route by project, quarantine with a dated exit (`@quarantine(until=…, issue=#…)`), or delete it
+10. Auth in shared fixtures (`e2e/fixtures/auth.ts`); test accounts and synthetic tenants carry the `e2e-` prefix (e.g. `e2e-customer-x1@example.test`, tenant `e2e-synthetic`)
+11. Timeouts: 15s per action/assertion (`actionTimeout`/`expect.timeout`); no per-test timeout raises without justification
+
+### Checks before a PR
+- Run route detection -- a page/endpoint without a manifest entry is a gap
+- Run the presentation sweep -- a route that throws on render is a bug even when a functional spec passes
+- Reviewer will block PRs with uncovered interactive routes without justification
+
+### CI
+- Curated golden-path smoke is the sole CI gate
+- Smoke drives the real app with deterministic seeded state and no first-party interception
+- the job passes only on an ALIVE verdict (DEAD, CANNOT-RUN, and EMPTY fail it)
+- Reviewer will block uncovered interactive routes based on the audit evidence, not CI wiring
+```

package/skills/engineering/e2e-test/references/authoring/playwright-patterns.md

@@ -0,0 +1,144 @@
+# Playwright Patterns
+
+> Read when: writing a spec: locators, waits, tier examples, and how to call the presentation instruments.
+
+## Locators
+
+```typescript
+// Prefer semantic locators
+page.getByRole('button', { name: 'Submit' })
+page.getByLabel('Email')
+page.getByText(/success/i)
+
+// Avoid CSS selectors
+page.locator('.btn-primary.text-red-500')  // breaks on style changes
+```
+
+## Waiting
+
+```typescript
+// Wait for real state
+await page.getByRole('button', { name: 'Save' }).click();
+await expect(page.getByText('Saved')).toBeVisible();
+
+// No fixed sleeps
+await page.waitForTimeout(3000);  // flaky
+```
+
+## Assertions
+
+```typescript
+// Assert on behavior
+await expect(page.getByText(/invalid.*range/i)).toBeVisible();
+
+// Don't assert CSS classes
+await expect(page.locator('.text-red-500')).toBeVisible();  // implementation detail
+```
+
+## No skips
+
+Seed the missing state, route by Playwright project, quarantine with a dated exit, or delete. `if (!order) return` is the worst form: it reports passed and asserts nothing. Policy: `ensure-coverage/references/process/test-strategy.md`.
+
+## Tier Examples
+
+### Tier 1 — Render-health
+
+One test per route: navigate, verify no crash.
+
+```typescript
+test.describe('Render-health: all routes load', () => {
+  for (const route of ['/', '/login', '/dashboard', '/settings']) {
+    test(`${route} loads`, async ({ page }) => {
+      const errors: string[] = [];
+      page.on('pageerror', (err) => errors.push(err.message));
+      await page.goto(route);
+      await expect(page.locator('body')).toBeVisible();
+      expect(errors).toEqual([]);
+    });
+  }
+});
+```
+
+### Tier 2 — Functional
+
+Cover success + failure + edge cases per feature.
+
+```typescript
+test.describe('Date Range Filter', () => {
+  test('applies range and updates data', async ({ page }) => {
+    await page.goto('/dashboard');
+    await page.getByLabel('From').fill('2026-01-01');
+    await page.getByRole('button', { name: 'Apply' }).click();
+    await expect(page.getByTestId('data-table')).toBeVisible();
+  });
+
+  test('shows error for invalid range', async ({ page }) => {
+    await page.goto('/dashboard');
+    await page.getByLabel('From').fill('2026-12-31');
+    await page.getByLabel('To').fill('2026-01-01');
+    await page.getByRole('button', { name: 'Apply' }).click();
+    await expect(page.getByText(/invalid.*range/i)).toBeVisible();
+  });
+});
+```
+
+### Tier 3 — Journey (golden paths)
+
+Multi-step flow, real API calls. Mock only external third-party APIs. These are the golden paths; the do-not-ship subset is tagged `@golden-path` for the golden-path gate.
+
+```typescript
+test('login then view dashboard then filter', async ({ page }) => {
+  await page.goto('/login');
+  await page.getByLabel('Token').fill(process.env.E2E_TOKEN!);
+  await page.getByRole('button', { name: 'Login' }).click();
+  await expect(page.getByRole('heading', { name: /dashboard/i })).toBeVisible();
+  await page.getByRole('button', { name: '7d' }).click();
+  const resp = await page.waitForResponse('**/api/data*');
+  expect(resp.status()).toBe(200);
+});
+```
+
+## Route Mocking (Tier 2 only)
+
+```typescript
+await page.route('**/api/data*', (route) => {
+  void route.fulfill({ status: 500, contentType: 'application/json',
+    body: JSON.stringify({ error: 'SERVER_ERROR', message: 'boom' }) });
+});
+await page.goto('/dashboard');
+await expect(page.getByText(/failed to fetch/i)).toBeVisible();
+```
+
+## Presentation checks
+
+A functional journey proves the page *works*; it does not prove a human can *see and use* it. The instruments in `scripts/` add per-route checks against the real page. Tag the spec with the level so the ledger resolves it; a DOM-only or JSDOM equivalent grades `Over-mocked`.
+
+```typescript
+// @covers /dashboard
+// @level browser-journey
+// @level presentation:spatial
+import { assertNoHorizontalOverflow, assertNoOverlap } from '<skill-dir>/scripts/presentation-checks.mjs';
+import { assertContrast } from '<skill-dir>/scripts/presentation-perceivability.mjs';
+import { assertTargetSizes, assertFocusVisible } from '<skill-dir>/scripts/presentation-reachability.mjs';
+import { assertNoFirstPartyShift } from '<skill-dir>/scripts/presentation-temporal.mjs';
+
+test('dashboard is laid out, perceivable, actionable, and still', async ({ page }) => {
+  await page.goto('/dashboard');
+  await expect(page.getByRole('heading', { name: /dashboard/i })).toBeVisible();
+  await assertNoHorizontalOverflow(page);            // spatial, at 375 + 1280
+  await assertNoOverlap(page, { allowlist: [{ match: 'carousel', category: 'decorative', reason: 'z-stacked slides' }] });
+  await assertContrast(page);                        // perceivability; throws AbstainError on an axe abstain (`Unverified`)
+  await assertTargetSizes(page);                     // reachability: >=44px targets
+  await assertFocusVisible(page, page.getByRole('button', { name: 'Save' }));
+  await assertNoFirstPartyShift(page, {              // temporal: hermetic, gated on a stable condition, never a sleep
+    trigger: () => page.getByRole('button', { name: 'Load more' }).click(),
+    settle: page.getByTestId('row-50'),
+  });
+});
+```
+
+- Each level is a separate obligation; a journey alone satisfies none of them.
+- Suppress intentional overflow or overlap only through the gated allowlist (`category` + `reason`); an entry that would hide an overlap on an interactive control is rejected.
+- Disabled controls are WCAG-exempt and skipped by `assertContrast`, not false-flagged.
+- `assertNoNavOrphans(page, manifestRoutes, { ignore: [...] })` runs once per app: every manifest route must be reachable by crawling nav from `/`.
+- Third-party payfields (`assertNoThirdPartyShift`) touch a provider sandbox iframe, so that check lives in the quarantine lane, never the blocking one.

package/skills/engineering/e2e-test/references/ci-integration.md

@@ -0,0 +1,25 @@
+# CI Integration
+
+Read only when the user has asked to wire gates. **Gates are added by humans only**: do not add, recommend, or audit CI gates on your own. The only supported CI shape is ensure-coverage's smoke-only contract (`ensure-coverage/references/enforcement/ci-contract.md`). The ledger, route inventory, presentation sweep, broad suite, lint, and quarantine checks are local or review-time evidence, not CI gates.
+
+## Wire golden-path smoke
+
+- **GitHub Actions:** the `golden-path` skill generates `.github/workflows/golden-paths.yml` from the repo's `golden-paths.json` (`golden-path init --write`); its `check` step fails CI when the workflow drifts from the contract.
+- **Cloud Build:** copy [`assets/ci-gates.cloudbuild.yaml`](../assets/ci-gates.cloudbuild.yaml) and fill the `<…>` placeholders. It calls the engine copy `golden-path init` puts in the repo (`scripts/golden-paths/golden-path.mjs`), so the runner needs no skills tree.
+
+Then verify that the golden-path verdict is present and blocking:
+
+```bash
+node <ensure-coverage>/scripts/ci-audit.mjs --files <your ci configs>
+```
+
+## Smoke job rules
+
+- Run the curated do-not-ship journeys against the real application and deterministic seeded state.
+- Do not intercept first-party requests or replace owned storage and services.
+- Keep the job blocking; do not use `continue-on-error`, `allowFailure`, `|| true`, `--warn-only`, or `set +e`.
+- Fail the job unless the verdict is ALIVE (`fail` is DEAD; `blocked`, `cannot-run`, or `missing` is CANNOT-RUN).
+- Keep the contract small enough for the PR lane. Parallelize paths rather than deleting them.
+- Use provider test credentials, never live credentials.
+
+The full presentation sweep and broad E2E suite may still be run by developers, reviewers, scheduled diagnostics, or another owner-defined workflow. This skill does not require or audit them as CI gates.