@schalkneethling/toolkit 0.2.0 → 0.4.0

package/skills/frontend-testing/references/visual-regression.md

@@ -0,0 +1,466 @@
+# Visual Regression Testing
+
+Visual regression testing (VRT) catches unintended visual changes by comparing screenshots against approved baselines.
+
+## When to Use VRT
+
+**Good candidates:**
+- Design system components
+- Critical landing pages
+- Complex layouts (grids, dashboards)
+- Components with CSS that changes frequently
+- Cross-browser visual consistency
+
+**Poor candidates:**
+- Pages with frequently changing dynamic content
+- Components with animations
+- Time-dependent displays
+- Pages with third-party content (ads, embeds)
+
+## Playwright Visual Testing
+
+### Basic Screenshot Comparison
+
+```javascript
+import { test, expect } from "@playwright/test";
+
+test("homepage looks correct", async ({ page }) => {
+  await page.goto("/");
+  
+  await expect(page).toHaveScreenshot();
+});
+```
+
+First run creates the baseline. Subsequent runs compare against it.
+
+### Named Screenshots
+
+```javascript
+test("hero section renders correctly", async ({ page }) => {
+  await page.goto("/");
+  
+  await expect(page).toHaveScreenshot("homepage-hero.png");
+});
+```
+
+### Element Screenshots
+
+More stable than full-page screenshots:
+
+```javascript
+test("product card renders correctly", async ({ page }) => {
+  await page.goto("/products/123");
+  
+  const productCard = page.getByTestId("product-card");
+  await expect(productCard).toHaveScreenshot("product-card.png");
+});
+```
+
+### Full Page Screenshots
+
+```javascript
+test("full page layout", async ({ page }) => {
+  await page.goto("/about");
+  
+  await expect(page).toHaveScreenshot("about-page-full.png", {
+    fullPage: true,
+  });
+});
+```
+
+## Handling Instability
+
+Visual tests fail due to timing, rendering, or environment differences. Stabilize tests before trusting results.
+
+### Wait for Network and Fonts
+
+```javascript
+test("page renders after loading", async ({ page }) => {
+  await page.goto("/");
+  
+  // Wait for network to be idle
+  await page.waitForLoadState("networkidle");
+  
+  // Wait for web fonts to load
+  await page.evaluate(() => document.fonts.ready);
+  
+  await expect(page).toHaveScreenshot();
+});
+```
+
+### Disable Animations
+
+```javascript
+// playwright.config.js
+export default defineConfig({
+  use: {
+    // Disable CSS animations and transitions
+    contextOptions: {
+      reducedMotion: "reduce",
+    },
+  },
+});
+```
+
+Or in test:
+
+```javascript
+test("static screenshot", async ({ page }) => {
+  await page.goto("/");
+  
+  // Inject CSS to disable animations
+  await page.addStyleTag({
+    content: `
+      *, *::before, *::after {
+        animation-duration: 0s !important;
+        transition-duration: 0s !important;
+      }
+    `,
+  });
+  
+  await expect(page).toHaveScreenshot();
+});
+```
+
+### Mask Dynamic Content
+
+Hide elements that change between runs:
+
+```javascript
+test("page with masked dynamic content", async ({ page }) => {
+  await page.goto("/dashboard");
+  
+  await expect(page).toHaveScreenshot({
+    mask: [
+      page.locator("[data-testid='current-time']"),
+      page.locator("[data-testid='user-avatar']"),
+      page.locator(".ad-container"),
+    ],
+  });
+});
+```
+
+### Fixed Viewport
+
+Lock viewport size for consistent screenshots:
+
+```javascript
+// playwright.config.js
+export default defineConfig({
+  use: {
+    viewport: { width: 1280, height: 720 },
+  },
+});
+```
+
+Or per-test:
+
+```javascript
+test("mobile view", async ({ page }) => {
+  await page.setViewportSize({ width: 375, height: 667 });
+  await page.goto("/");
+  await expect(page).toHaveScreenshot("homepage-mobile.png");
+});
+```
+
+### Hide Scrollbars
+
+Scrollbars vary by OS:
+
+```javascript
+await page.addStyleTag({
+  content: `
+    ::-webkit-scrollbar { display: none; }
+    * { scrollbar-width: none; }
+  `,
+});
+```
+
+## Tolerance Settings
+
+Allow minor pixel differences to reduce flakiness.
+
+### Global Configuration
+
+```javascript
+// playwright.config.js
+export default defineConfig({
+  expect: {
+    toHaveScreenshot: {
+      maxDiffPixels: 100,        // Allow up to 100 different pixels
+      maxDiffPixelRatio: 0.01,   // Or 1% of total pixels
+      threshold: 0.2,           // Color difference threshold (0-1)
+    },
+  },
+});
+```
+
+### Per-Screenshot Settings
+
+```javascript
+await expect(page).toHaveScreenshot({
+  maxDiffPixels: 50,
+  threshold: 0.3,
+});
+```
+
+### Threshold Guidelines
+
+| Setting | Value | Use Case |
+|---------|-------|----------|
+| `threshold` | 0.1 | Strict pixel matching |
+| `threshold` | 0.2 | Standard tolerance (default) |
+| `threshold` | 0.3 | Generous for anti-aliasing |
+| `maxDiffPixels` | 50 | Small components |
+| `maxDiffPixels` | 200 | Full pages |
+| `maxDiffPixelRatio` | 0.01 | 1% tolerance |
+
+## Updating Baselines
+
+### When to Update
+
+- Intentional design changes
+- New features affecting layout
+- Browser rendering engine updates
+- After fixing visual bugs
+
+### Update Command
+
+```bash
+npx playwright test --update-snapshots
+```
+
+**Never update blindly.** Review changes before committing:
+
+```bash
+# See what changed
+git diff --stat
+# Review each screenshot
+open tests/screenshots/
+```
+
+### Workflow
+
+1. Test fails due to visual difference
+2. Open test report, review diff image
+3. If change is intentional:
+   - Update baseline with `--update-snapshots`
+   - Commit new baseline
+4. If change is unintentional:
+   - Fix the code
+   - Run test again
+
+## Cross-Browser Testing
+
+Screenshots differ between browsers due to rendering differences.
+
+### Browser-Specific Baselines
+
+Playwright automatically creates separate baselines per browser:
+
+```
+example.spec.ts-snapshots/
+├── homepage-chromium-linux.png
+├── homepage-firefox-linux.png
+└── homepage-webkit-linux.png
+```
+
+### Testing Specific Browsers
+
+```javascript
+// playwright.config.js
+export default defineConfig({
+  projects: [
+    {
+      name: "chromium",
+      use: { ...devices["Desktop Chrome"] },
+    },
+    {
+      name: "firefox",
+      use: { ...devices["Desktop Firefox"] },
+    },
+    {
+      name: "webkit",
+      use: { ...devices["Desktop Safari"] },
+    },
+  ],
+});
+```
+
+### OS Differences
+
+Font rendering varies by OS. Options:
+- Run visual tests in CI only (consistent environment)
+- Use Docker with consistent fonts
+- Generate baselines in CI, not locally
+
+## CI/CD Configuration
+
+### GitHub Actions Example
+
+```yaml
+- name: Run visual tests
+  run: npx playwright test --project=chromium
+  
+- name: Upload diff artifacts on failure
+  if: failure()
+  uses: actions/upload-artifact@v7
+  with:
+    name: visual-diff
+    path: test-results/
+```
+
+### Generate Baselines in CI
+
+For consistent baselines, generate in CI:
+
+```yaml
+- name: Update baselines
+  run: npx playwright test --update-snapshots
+  
+- name: Commit baselines
+  run: |
+    git config user.name "CI Bot"
+    git add "**/*.png"
+    git commit -m "Update visual baselines"
+    git push
+```
+
+## Organizing Screenshots
+
+### Directory Structure
+
+```
+tests/
+├── visual/
+│   ├── homepage.spec.ts
+│   └── components/
+│       ├── button.spec.ts
+│       └── card.spec.ts
+└── visual.spec.ts-snapshots/
+    ├── homepage-chromium.png
+    └── button-default-chromium.png
+```
+
+### Custom Path Template
+
+```javascript
+// playwright.config.js
+export default defineConfig({
+  snapshotPathTemplate: "{testDir}/__snapshots__/{projectName}/{testFilePath}/{arg}{ext}",
+});
+```
+
+## Reporting and Review
+
+### Test Report
+
+```bash
+npx playwright show-report
+```
+
+Shows:
+- Expected (baseline) image
+- Actual (current) image
+- Diff highlighting changed pixels
+
+### Artifacts on Failure
+
+Configure to save all artifacts:
+
+```javascript
+// playwright.config.js
+export default defineConfig({
+  use: {
+    trace: "on-first-retry",
+    screenshot: "only-on-failure",
+  },
+  outputDir: "test-results/",
+});
+```
+
+## Anti-Patterns
+
+### Removing Platform Suffix
+
+```javascript
+// BAD: Removes platform/browser suffix from snapshot names
+test.beforeEach(async ({}, testInfo) => {
+  testInfo.snapshotSuffix = "";
+});
+
+// Results in single baseline used across all platforms:
+// button.png  ← Used for Chrome/Mac, Firefox/Linux, Safari/Windows...
+```
+
+This seems appealing for "cleaner" snapshot names, but it breaks visual testing. Per [Playwright documentation](https://playwright.dev/docs/test-snapshots):
+
+> "Different snapshots are needed for different browsers and platforms due to variations in rendering and fonts."
+
+**Why it fails:**
+- Font rendering differs between macOS, Windows, and Linux
+- Anti-aliasing algorithms vary by OS and browser
+- Subpixel rendering produces different results
+- You'll get constant false positives or false negatives
+
+**Keep the default naming:**
+```
+button-chromium-darwin.png
+button-chromium-linux.png
+button-firefox-linux.png
+```
+
+Yes, it's more files. But each baseline accurately represents that specific environment.
+
+### Too Many Full-Page Screenshots
+
+```javascript
+// BAD: Full page with dynamic content
+await expect(page).toHaveScreenshot({ fullPage: true });
+
+// GOOD: Target stable regions
+const header = page.getByRole("banner");
+await expect(header).toHaveScreenshot("header.png");
+```
+
+### No Stabilization
+
+```javascript
+// BAD: Screenshot immediately
+await page.goto("/");
+await expect(page).toHaveScreenshot();
+
+// GOOD: Wait for stable state
+await page.goto("/");
+await page.waitForLoadState("networkidle");
+await page.evaluate(() => document.fonts.ready);
+await expect(page).toHaveScreenshot();
+```
+
+### Overly Strict Tolerance
+
+```javascript
+// BAD: Zero tolerance causes flaky tests
+await expect(page).toHaveScreenshot({ maxDiffPixels: 0 });
+
+// GOOD: Allow minor rendering differences
+await expect(page).toHaveScreenshot({ maxDiffPixels: 50 });
+```
+
+### Updating Without Review
+
+```bash
+# BAD: Blindly update all baselines
+npx playwright test --update-snapshots
+
+# GOOD: Review changes first
+npx playwright test
+npx playwright show-report
+# Review diffs, then update if intentional
+npx playwright test --update-snapshots
+```
+
+## References
+
+- [Playwright Visual Comparisons](https://playwright.dev/docs/test-snapshots)
+- [Pixelmatch](https://github.com/mapbox/pixelmatch) — Image comparison library used by Playwright

package/skills/refined-plan-mode/SKILL.md

@@ -0,0 +1,84 @@
+# Refined Plan Mode
+
+Use this skill when the user starts a planning session using plan mode and wants the Refined Plan Mode review loop.
+
+This skill is additive to the agent's current plan-mode guidance. It turns the plan into a versioned Markdown artifact that can be reviewed with line, range, and text-selection comments. The agent remains responsible for reading the feedback, revising the plan, and moving only when the user has approved the plan or explicitly asks to proceed.
+
+## Core Protocol
+
+1. Clarify only what is necessary to produce a useful plan.
+2. Write the complete plan to `.plan-review/plans/plan-vN.md`.
+3. Write the active version, such as `v1`, to `.plan-review/.current-version`.
+4. Present a short summary to the user and point them at Refined Plan Mode for review.
+5. After review, read `.plan-review/feedback/plan-vN-feedback.json`.
+6. Address every feedback item in the next plan version.
+7. Update `.plan-review/.current-version` to the new version.
+8. Repeat until the plan is approved.
+9. When the plan is approved, read `.plan-review/approved-plan.md` and execute it carefully.
+
+## File Convention
+
+```text
+.plan-review/
+  .current-version
+  plans/
+    plan-v1.md
+    plan-v2.md
+  feedback/
+    plan-v1-feedback.json
+    plan-v2-feedback.json
+  approved-plan.md
+```
+
+Create missing directories when needed. Never summarize or truncate the plan file itself. The file should be self-contained enough for another agent to understand the goal, context, constraints, implementation steps, validation steps, and open questions.
+
+## Plan Shape
+
+Prefer this structure unless the task clearly calls for something else:
+
+```markdown
+# Plan vN: Short Title
+
+## Goal
+
+## Current Understanding
+
+## Assumptions
+
+## Open Questions
+
+## Proposed Changes
+
+## Validation
+
+## Risks
+
+## Rollback or Recovery
+```
+
+Keep the plan practical. Include file paths, commands, and decision points when known. Call out assumptions explicitly instead of hiding uncertainty inside confident prose.
+
+## Feedback Handling
+
+When feedback exists:
+
+- Read the relevant JSON feedback file before revising.
+- Treat every unresolved comment as actionable until clearly addressed.
+- Preserve useful parts of the previous plan.
+- Add a short `Feedback Addressed` section to the revised plan that maps comments to the changes made.
+- If feedback conflicts or cannot be satisfied safely, explain that in the revised plan and ask the user for the smallest useful decision.
+
+## Execution Gate
+
+Do not begin implementation from an unapproved plan unless the user explicitly asks you to proceed. Once `.plan-review/approved-plan.md` exists or the user directly approves the plan in conversation, execute the approved plan and keep the normal agent workflow: inspect files, make focused edits, validate, and report the outcome.
+
+## User Updates
+
+In conversation, keep updates brief:
+
+- Say which plan version was written.
+- Say where feedback should be submitted.
+- Say which feedback file was read when revising.
+- Say when the plan is approved and execution is beginning.
+
+The plan file carries the detail; the chat message should help the user orient without duplicating the full artifact.