@harness-engineering/cli 1.23.2 → 1.24.0

package/dist/agents/skills/claude-code/test-playwright-setup/SKILL.md

@@ -0,0 +1,178 @@
+# Test Playwright Setup
+
+> Configure Playwright test runner with fixtures, reporters, and browser contexts
+
+## When to Use
+
+- Setting up Playwright for end-to-end testing in a web project
+- Configuring multiple browsers, reporters, and test environments
+- Setting up authentication state sharing across tests
+- Configuring CI-specific Playwright settings
+
+## Instructions
+
+1. **Install Playwright:**
+
+```bash
+npm init playwright@latest
+# or
+npm install -D @playwright/test
+npx playwright install
+```
+
+2. **Configure `playwright.config.ts`:**
+
+```typescript
+import { defineConfig, devices } from '@playwright/test';
+
+export default defineConfig({
+  testDir: './e2e',
+  fullyParallel: true,
+  forbidOnly: !!process.env.CI,
+  retries: process.env.CI ? 2 : 0,
+  workers: process.env.CI ? 1 : undefined,
+  reporter: process.env.CI ? 'github' : 'html',
+
+  use: {
+    baseURL: 'http://localhost:3000',
+    trace: 'on-first-retry',
+    screenshot: 'only-on-failure',
+  },
+
+  projects: [
+    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
+    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
+    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
+    { name: 'mobile-chrome', use: { ...devices['Pixel 5'] } },
+  ],
+
+  webServer: {
+    command: 'npm run dev',
+    url: 'http://localhost:3000',
+    reuseExistingServer: !process.env.CI,
+    timeout: 120_000,
+  },
+});
+```
+
+3. **Auto-start the dev server** with `webServer` config — Playwright starts and waits for the server automatically.
+
+4. **Set up authenticated state** with global setup:
+
+```typescript
+// e2e/auth.setup.ts
+import { test as setup, expect } from '@playwright/test';
+
+setup('authenticate', async ({ page }) => {
+  await page.goto('/login');
+  await page.getByLabel('Email').fill('admin@test.com');
+  await page.getByLabel('Password').fill('password');
+  await page.getByRole('button', { name: 'Log in' }).click();
+  await expect(page.getByText('Dashboard')).toBeVisible();
+
+  await page.context().storageState({ path: 'e2e/.auth/user.json' });
+});
+```
+
+```typescript
+// playwright.config.ts — add setup project
+projects: [
+  { name: 'setup', testMatch: /.*\.setup\.ts/ },
+  {
+    name: 'chromium',
+    use: {
+      ...devices['Desktop Chrome'],
+      storageState: 'e2e/.auth/user.json',
+    },
+    dependencies: ['setup'],
+  },
+],
+```
+
+5. **Custom fixtures** for reusable test helpers:
+
+```typescript
+// e2e/fixtures.ts
+import { test as base } from '@playwright/test';
+
+type Fixtures = {
+  todoPage: TodoPage;
+};
+
+export const test = base.extend<Fixtures>({
+  todoPage: async ({ page }, use) => {
+    const todoPage = new TodoPage(page);
+    await todoPage.goto();
+    await use(todoPage);
+  },
+});
+
+export { expect } from '@playwright/test';
+```
+
+6. **Configure reporters:**
+
+```typescript
+reporter: [
+  ['html', { open: 'never' }],
+  ['junit', { outputFile: 'test-results/junit.xml' }],
+  ['list'],
+],
+```
+
+7. **Trace viewer** for debugging failed tests:
+
+```typescript
+use: {
+  trace: 'on-first-retry', // Records trace on first retry of failed tests
+},
+```
+
+View with: `npx playwright show-trace test-results/trace.zip`
+
+8. **Add to `.gitignore`:**
+
+```
+test-results/
+playwright-report/
+e2e/.auth/
+```
+
+## Details
+
+Playwright runs tests across real browsers (Chromium, Firefox, WebKit) with full automation capabilities. Each test gets a fresh browser context by default, providing natural test isolation.
+
+**Browser contexts:** Each test runs in an isolated browser context (like an incognito window). Cookies, localStorage, and session state do not leak between tests. This is more isolated than sharing a single browser instance.
+
+**`webServer` configuration:** Playwright can start your dev/preview server automatically before running tests. Set `reuseExistingServer: true` for development (uses an already-running server) and `false` for CI (starts a fresh server).
+
+**Storage state:** Authentication state (cookies, localStorage) can be saved and reused across tests. This avoids logging in for every test, significantly speeding up the suite.
+
+**Parallelism:** `fullyParallel: true` runs tests in the same file in parallel. Each test gets its own worker process. Set `workers: 1` in CI if your app cannot handle concurrent users.
+
+**Trade-offs:**
+
+- Multi-browser testing catches browser-specific bugs — but triples test execution time
+- `webServer` auto-start is convenient — but can mask server startup issues
+- Storage state sharing speeds up tests — but tests may fail if the auth state expires
+- Trace viewer is invaluable for debugging — but traces consume significant disk space
+
+## Source
+
+https://playwright.dev/docs/intro
+
+## Process
+
+1. Read the instructions and examples in this document.
+2. Apply the patterns to your implementation, adapting to your specific context.
+3. Verify your implementation against the details and edge cases listed above.
+
+## Harness Integration
+
+- **Type:** knowledge — this skill is a reference document, not a procedural workflow.
+- **No tools or state** — consumed as context by other skills and agents.
+
+## Success Criteria
+
+- The patterns described in this document are applied correctly in the implementation.
+- Edge cases and anti-patterns listed in this document are avoided.

package/dist/agents/skills/claude-code/test-playwright-setup/skill.yaml

@@ -0,0 +1,42 @@
+name: test-playwright-setup
+version: "1.0.0"
+description: Configure Playwright test runner with fixtures, reporters, and browser contexts
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths:
+  - "**/*.test.ts"
+  - "**/*.spec.ts"
+  - "**/*.test.tsx"
+  - "vitest.config.*"
+  - "jest.config.*"
+  - "playwright.config.*"
+related_skills:
+  - test-playwright-patterns
+  - test-e2e-strategy
+stack_signals:
+  - vitest
+  - jest
+  - playwright
+  - testing-library
+keywords:
+  - playwright
+  - e2e-testing
+  - test-fixtures
+  - browser-automation
+  - trace-viewer
+metadata:
+  author: community
+  upstream: playwright.dev/docs/intro
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/claude-code/test-property-based/SKILL.md

@@ -0,0 +1,172 @@
+# Test Property-Based
+
+> Generate exhaustive test cases automatically using fast-check property-based testing
+
+## When to Use
+
+- Testing functions where the set of valid inputs is large or complex
+- Verifying mathematical properties (commutativity, associativity, idempotence)
+- Finding edge cases that hand-written examples miss
+- Testing serialization/deserialization roundtrips
+
+## Instructions
+
+1. **Install fast-check:**
+
+```bash
+npm install -D fast-check
+```
+
+2. **Basic property test:**
+
+```typescript
+import fc from 'fast-check';
+import { describe, it, expect } from 'vitest';
+
+describe('sort', () => {
+  it('produces output with same length as input', () => {
+    fc.assert(
+      fc.property(fc.array(fc.integer()), (arr) => {
+        expect(arr.sort()).toHaveLength(arr.length);
+      })
+    );
+  });
+
+  it('produces sorted output', () => {
+    fc.assert(
+      fc.property(fc.array(fc.integer()), (arr) => {
+        const sorted = [...arr].sort((a, b) => a - b);
+        for (let i = 1; i < sorted.length; i++) {
+          expect(sorted[i]).toBeGreaterThanOrEqual(sorted[i - 1]);
+        }
+      })
+    );
+  });
+});
+```
+
+3. **Common arbitraries** (data generators):
+
+```typescript
+fc.string(); // random strings
+fc.integer({ min: 0, max: 100 }); // bounded integers
+fc.float({ noNaN: true }); // floats without NaN
+fc.boolean(); // true/false
+fc.date(); // Date objects
+fc.uuid(); // UUID strings
+fc.emailAddress(); // valid email addresses
+fc.array(fc.integer()); // arrays of integers
+fc.record({ name: fc.string(), age: fc.integer({ min: 0 }) }); // objects
+fc.oneof(fc.string(), fc.integer()); // union of types
+```
+
+4. **Test roundtrip properties** (encode/decode, serialize/deserialize):
+
+```typescript
+it('JSON roundtrip preserves data', () => {
+  fc.assert(
+    fc.property(fc.record({ name: fc.string(), age: fc.integer() }), (obj) => {
+      expect(JSON.parse(JSON.stringify(obj))).toEqual(obj);
+    })
+  );
+});
+```
+
+5. **Test idempotence:**
+
+```typescript
+it('normalizing email is idempotent', () => {
+  fc.assert(
+    fc.property(fc.emailAddress(), (email) => {
+      const once = normalizeEmail(email);
+      const twice = normalizeEmail(once);
+      expect(once).toBe(twice);
+    })
+  );
+});
+```
+
+6. **Custom arbitraries** for domain types:
+
+```typescript
+const userArbitrary = fc.record({
+  id: fc.uuid(),
+  name: fc.string({ minLength: 1, maxLength: 100 }),
+  email: fc.emailAddress(),
+  age: fc.integer({ min: 0, max: 150 }),
+});
+
+it('validates all valid users', () => {
+  fc.assert(
+    fc.property(userArbitrary, (user) => {
+      expect(validateUser(user).ok).toBe(true);
+    })
+  );
+});
+```
+
+7. **Shrinking** — when a test fails, fast-check automatically finds the smallest failing input:
+
+```typescript
+// If sort fails on [3, 1, -5, 7, 2], fast-check shrinks to
+// the minimal failing case, e.g., [1, -1]
+```
+
+8. **Configure test parameters:**
+
+```typescript
+fc.assert(
+  fc.property(fc.string(), (s) => {
+    /* ... */
+  }),
+  {
+    numRuns: 1000, // Number of random inputs (default: 100)
+    seed: 42, // Deterministic seed for reproducibility
+    verbose: true, // Log generated values
+    endOnFailure: true, // Stop at first failure
+  }
+);
+```
+
+## Details
+
+Property-based testing inverts the traditional testing model: instead of specifying inputs and expected outputs, you specify properties that must hold for ALL inputs. The framework generates random inputs and checks the property, then shrinks failing cases to minimal counterexamples.
+
+**What makes a good property:**
+
+- **Roundtrip** — `decode(encode(x)) === x`
+- **Idempotence** — `f(f(x)) === f(x)`
+- **Invariant** — `sorted(x).length === x.length`
+- **Oracle** — compare output against a simple but slow reference implementation
+- **Negation** — `isValid(invalidInput) === false` using generated invalid inputs
+
+**Shrinking:** When fast-check finds a failing input, it tries smaller variants to find the minimal counterexample. This is invaluable for debugging — instead of "fails on a 500-element array," you get "fails on [0, -1]."
+
+**Integration with Vitest:** fast-check works as a library called inside `it()` blocks. No special test runner integration needed. The `fc.assert()` call throws on failure, which Vitest catches.
+
+**Trade-offs:**
+
+- Property tests find edge cases humans miss — but properties can be hard to formulate
+- 100+ random runs per property catch more bugs — but run slower than example-based tests
+- Shrinking produces minimal counterexamples — but shrinking can be slow for complex data structures
+- Deterministic seeds enable reproducibility — but random runs are more valuable for catching new bugs
+
+## Source
+
+https://fast-check.dev/docs/
+
+## Process
+
+1. Read the instructions and examples in this document.
+2. Apply the patterns to your implementation, adapting to your specific context.
+3. Verify your implementation against the details and edge cases listed above.
+
+## Harness Integration
+
+- **Type:** knowledge — this skill is a reference document, not a procedural workflow.
+- **No tools or state** — consumed as context by other skills and agents.
+
+## Success Criteria
+
+- The patterns described in this document are applied correctly in the implementation.
+- Edge cases and anti-patterns listed in this document are avoided.

package/dist/agents/skills/claude-code/test-property-based/skill.yaml

@@ -0,0 +1,42 @@
+name: test-property-based
+version: "1.0.0"
+description: Generate exhaustive test cases automatically using fast-check property-based testing
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths:
+  - "**/*.test.ts"
+  - "**/*.spec.ts"
+  - "**/*.test.tsx"
+  - "vitest.config.*"
+  - "jest.config.*"
+  - "playwright.config.*"
+related_skills:
+  - test-unit-patterns
+  - ts-generics-pattern
+stack_signals:
+  - vitest
+  - jest
+  - playwright
+  - testing-library
+keywords:
+  - fast-check
+  - property-based-testing
+  - arbitraries
+  - shrinking
+  - generative-testing
+metadata:
+  author: community
+  upstream: fast-check.dev/docs/
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/claude-code/test-snapshot-patterns/SKILL.md

@@ -0,0 +1,142 @@
+# Test Snapshot Patterns
+
+> Use snapshot testing selectively for stable outputs, knowing when to avoid it
+
+## When to Use
+
+- Testing serialized output (JSON, HTML, error messages) that should not change unexpectedly
+- Verifying component render output for regression detection
+- Capturing complex object shapes that are tedious to assert manually
+- NOT for rapidly changing UI or non-deterministic output
+
+## Instructions
+
+1. **Basic snapshot:**
+
+```typescript
+it('serializes user data correctly', () => {
+  const user = formatUser({ id: '1', name: 'Alice', email: 'alice@test.com' });
+
+  expect(user).toMatchSnapshot();
+});
+```
+
+First run creates the snapshot file. Subsequent runs compare against it.
+
+2. **Inline snapshots** — store the expected value in the test file:
+
+```typescript
+it('formats the error message', () => {
+  const error = formatError({ code: 404, resource: 'User' });
+
+  expect(error).toMatchInlineSnapshot(`"User not found (404)"`);
+});
+```
+
+Vitest/Jest auto-fills the inline snapshot on first run.
+
+3. **Snapshot specific properties:**
+
+```typescript
+it('creates a user with generated fields', () => {
+  const user = createUser({ name: 'Alice' });
+
+  expect(user).toMatchSnapshot({
+    id: expect.any(String),
+    createdAt: expect.any(Date),
+  });
+});
+```
+
+This snapshots the structure but uses matchers for non-deterministic fields.
+
+4. **Update snapshots** when intentional changes occur:
+
+```bash
+vitest --update  # or vitest -u
+```
+
+Review the diff before committing updated snapshots.
+
+5. **Prefer inline snapshots for small values** — they are easier to review in PRs:
+
+```typescript
+it('generates correct SQL', () => {
+  const query = buildQuery({ table: 'users', where: { role: 'admin' } });
+
+  expect(query).toMatchInlineSnapshot(`
+    "SELECT * FROM users WHERE role = 'admin'"
+  `);
+});
+```
+
+6. **Snapshot file-based output** for large payloads:
+
+```typescript
+it('generates correct API response', () => {
+  const response = buildResponse(testData);
+  expect(response).toMatchSnapshot(); // Stored in __snapshots__/
+});
+```
+
+7. **Avoid snapshots for:**
+   - UI components that change frequently (use specific assertions instead)
+   - Non-deterministic output (timestamps, random IDs)
+   - Very large objects (snapshots become unreadable)
+   - Behavior testing (use explicit assertions)
+
+8. **Good snapshot candidates:**
+   - Serialization formats (JSON output, GraphQL schema)
+   - Error messages and validation output
+   - Configuration generation
+   - CLI output formatting
+
+## Details
+
+Snapshot testing captures the serialized output of a value and compares it against a stored reference. It is a regression detection tool — it tells you when output changes, but not whether the change is correct.
+
+**Snapshot workflow:**
+
+1. First run: snapshot is created and stored (in `__snapshots__/` or inline)
+2. Subsequent runs: output is compared against the stored snapshot
+3. On mismatch: test fails with a diff
+4. To accept the change: run with `--update` flag and review the diff
+
+**Snapshot hygiene:**
+
+- Review snapshot diffs in PRs as carefully as code changes
+- Delete snapshots for removed tests — stale snapshots cause confusion
+- Keep snapshots small — large snapshots are rubber-stamped in review
+- Use `toMatchInlineSnapshot` for values under 5 lines — inline snapshots are easier to review
+
+**`toMatchSnapshot` vs `toMatchInlineSnapshot`:**
+
+- File-based: stored separately, good for large output, harder to review in PRs
+- Inline: stored in the test file, easy to review, best for small values
+
+**Trade-offs:**
+
+- Snapshots catch unintentional changes with minimal effort — but also flag intentional changes, requiring update ceremonies
+- Large snapshot files are easy to create — but become "approve without reading" in code review
+- Inline snapshots are reviewable — but clutter test files with literal output
+- Snapshot tests are fast to write — but can mask poor test design (asserting everything instead of specific behavior)
+
+## Source
+
+https://vitest.dev/guide/snapshot.html
+
+## Process
+
+1. Read the instructions and examples in this document.
+2. Apply the patterns to your implementation, adapting to your specific context.
+3. Verify your implementation against the details and edge cases listed above.
+
+## Harness Integration
+
+- **Type:** knowledge — this skill is a reference document, not a procedural workflow.
+- **No tools or state** — consumed as context by other skills and agents.
+
+## Success Criteria
+
+- The patterns described in this document are applied correctly in the implementation.
+- Edge cases and anti-patterns listed in this document are avoided.

package/dist/agents/skills/claude-code/test-snapshot-patterns/skill.yaml

@@ -0,0 +1,41 @@
+name: test-snapshot-patterns
+version: "1.0.0"
+description: Use snapshot testing selectively for stable outputs, knowing when to avoid it
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths:
+  - "**/*.test.ts"
+  - "**/*.spec.ts"
+  - "**/*.test.tsx"
+  - "vitest.config.*"
+  - "jest.config.*"
+  - "playwright.config.*"
+related_skills:
+  - test-unit-patterns
+  - test-component-react
+stack_signals:
+  - vitest
+  - jest
+  - playwright
+  - testing-library
+keywords:
+  - snapshots
+  - toMatchSnapshot
+  - toMatchInlineSnapshot
+  - snapshot-testing
+metadata:
+  author: community
+  upstream: vitest.dev/guide/snapshot.html
+state:
+  persistent: false
+  files: []
+depends_on: []