@arcadialdev/arcality 2.2.0

package/.agents/skills/playwright-best-practices/testing-patterns/visual-regression.md

@@ -0,0 +1,634 @@
+# Visual Regression Testing
+
+## Table of Contents
+
+1. [Quick Reference](#quick-reference)
+2. [Patterns](#patterns)
+3. [Decision Guide](#decision-guide)
+4. [Anti-Patterns](#anti-patterns)
+5. [Troubleshooting](#troubleshooting)
+
+> **When to use**: Detecting unintended visual changes—layout shifts, style regressions, broken responsive designs—that functional assertions miss.
+
+## Quick Reference
+
+```typescript
+// Element screenshot
+await expect(page.getByTestId('product-card')).toHaveScreenshot();
+
+// Full page screenshot
+await expect(page).toHaveScreenshot('landing-hero.png');
+
+// Threshold for minor pixel variance
+await expect(page).toHaveScreenshot({ maxDiffPixelRatio: 0.01 });
+
+// Mask volatile content
+await expect(page).toHaveScreenshot({
+  mask: [page.getByTestId('clock'), page.getByRole('img', { name: 'User photo' })],
+});
+
+// Disable CSS animations
+await expect(page).toHaveScreenshot({ animations: 'disabled' });
+
+// Update baselines
+npx playwright test --update-snapshots
+```
+
+## Patterns
+
+### Masking Volatile Content
+
+**Use when**: Page contains timestamps, avatars, ad slots, relative dates, random images, or A/B variants.
+
+The `mask` option overlays a solid box over specified locators before capturing.
+
+```typescript
+test('analytics panel with masked dynamic elements', async ({ page }) => {
+  await page.goto('/analytics');
+
+  await expect(page).toHaveScreenshot('analytics.png', {
+    mask: [
+      page.getByTestId('last-updated'),
+      page.getByTestId('profile-avatar'),
+      page.getByTestId('active-users'),
+      page.locator('.promo-banner'),
+    ],
+    maskColor: '#FF00FF',
+  });
+});
+
+test('activity stream with relative times', async ({ page }) => {
+  await page.goto('/activity');
+
+  await expect(page).toHaveScreenshot('activity.png', {
+    mask: [page.locator('time[datetime]')],
+  });
+});
+```
+
+**Alternative: freeze content with JavaScript** when masking affects layout:
+
+```typescript
+test('freeze timestamps before capture', async ({ page }) => {
+  await page.goto('/analytics');
+
+  await page.evaluate(() => {
+    document.querySelectorAll('[data-testid="time-display"]').forEach((el) => {
+      el.textContent = 'Jan 1, 2025 12:00 PM';
+    });
+  });
+
+  await expect(page).toHaveScreenshot('analytics-frozen.png');
+});
+```
+
+### Disabling Animations
+
+**Use when**: Always. CSS animations and transitions are the primary cause of flaky visual diffs.
+
+```typescript
+test('renders without animation interference', async ({ page }) => {
+  await page.goto('/');
+
+  await expect(page).toHaveScreenshot('home.png', {
+    animations: 'disabled',
+  });
+});
+```
+
+**Set globally** in config:
+
+```typescript
+// playwright.config.ts
+export default defineConfig({
+  expect: {
+    toHaveScreenshot: {
+      animations: 'disabled',
+    },
+  },
+});
+```
+
+When `animations: 'disabled'` is set, Playwright injects CSS forcing animation/transition duration to 0s, waits for running animations to finish, then captures.
+
+For JavaScript-driven animations (GSAP, Framer Motion), wait for stability:
+
+```typescript
+test('page with JS animations', async ({ page }) => {
+  await page.goto('/animated-hero');
+
+  const heroBanner = page.getByTestId('hero-banner');
+  await heroBanner.waitFor({ state: 'visible' });
+  
+  // Wait for animation to complete by checking for stable state
+  await expect(heroBanner).not.toHaveClass(/animating/);
+
+  await expect(page).toHaveScreenshot('hero.png', {
+    animations: 'disabled',
+  });
+});
+```
+
+### Configuring Thresholds
+
+**Use when**: Minor rendering differences from anti-aliasing, font hinting, or sub-pixel rendering cause false failures.
+
+| Option | Controls | Typical Value |
+|---|---|---|
+| `maxDiffPixels` | Absolute pixel count that can differ | `100` for pages, `10` for components |
+| `maxDiffPixelRatio` | Fraction of total pixels (0-1) | `0.01` (1%) for pages |
+| `threshold` | Per-pixel color tolerance (0-1) | `0.2` for most UIs, `0.1` for design systems |
+
+```typescript
+test('control panel allows minor variance', async ({ page }) => {
+  await page.goto('/control-panel');
+
+  await expect(page).toHaveScreenshot('control-panel.png', {
+    maxDiffPixelRatio: 0.01,
+  });
+});
+
+test('brand logo renders pixel-perfect', async ({ page }) => {
+  await page.goto('/brand');
+
+  await expect(page.getByTestId('brand-logo')).toHaveScreenshot('brand-logo.png', {
+    maxDiffPixels: 0,
+    threshold: 0,
+  });
+});
+
+test('graph allows anti-aliasing differences', async ({ page }) => {
+  await page.goto('/reports');
+
+  await expect(page.getByTestId('sales-graph')).toHaveScreenshot('sales-graph.png', {
+    threshold: 0.3,
+    maxDiffPixels: 200,
+  });
+});
+```
+
+**Global thresholds** in config:
+
+```typescript
+// playwright.config.ts
+export default defineConfig({
+  expect: {
+    toHaveScreenshot: {
+      maxDiffPixelRatio: 0.01,
+      threshold: 0.2,
+      animations: 'disabled',
+    },
+  },
+});
+```
+
+### CI Configuration
+
+**Use when**: Running visual tests in CI. Consistent rendering is critical—the same test must produce identical screenshots every time.
+
+**The problem**: Font rendering and anti-aliasing differ across operating systems. macOS snapshots won't match Linux.
+
+**The solution**: Run visual tests in Docker using the official Playwright container. Generate and update snapshots from the same container.
+
+**GitHub Actions with Docker**
+
+```yaml
+# .github/workflows/visual-tests.yml
+name: Visual Regression Tests
+on: [push, pull_request]
+
+jobs:
+  visual-tests:
+    runs-on: ubuntu-latest
+    container:
+      image: mcr.microsoft.com/playwright:v1.48.0-noble
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: lts/*
+          cache: npm
+
+      - run: npm ci
+
+      - name: Run visual tests
+        run: npx playwright test --project=visual
+        env:
+          HOME: /root
+
+      - uses: actions/upload-artifact@v4
+        if: failure()
+        with:
+          name: visual-test-report
+          path: playwright-report/
+          retention-days: 14
+```
+
+**Updating snapshots locally using Docker**:
+
+```bash
+docker run --rm -v $(pwd):/work -w /work \
+  mcr.microsoft.com/playwright:v1.48.0-noble \
+  npx playwright test --update-snapshots --project=visual
+```
+
+**Add script to `package.json`**:
+
+```json
+{
+  "scripts": {
+    "test:visual": "npx playwright test --project=visual",
+    "test:visual:update": "docker run --rm -v $(pwd):/work -w /work mcr.microsoft.com/playwright:v1.48.0-noble npx playwright test --update-snapshots --project=visual"
+  }
+}
+```
+
+**Platform-agnostic snapshots** (requires Docker for generation):
+
+```typescript
+// playwright.config.ts
+export default defineConfig({
+  snapshotPathTemplate: '{testDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{ext}',
+  projects: [
+    {
+      name: 'visual',
+      testMatch: '**/*.visual.spec.ts',
+      use: { ...devices['Desktop Chrome'] },
+    },
+  ],
+});
+```
+
+### Full Page vs Element Screenshots
+
+**Use when**: Deciding scope. Full page catches layout shifts. Element screenshots isolate components and are more stable.
+
+```typescript
+test('full page captures layout shifts', async ({ page }) => {
+  await page.goto('/');
+
+  // Visible viewport
+  await expect(page).toHaveScreenshot('home-viewport.png');
+
+  // Entire scrollable page
+  await expect(page).toHaveScreenshot('home-full.png', {
+    fullPage: true,
+  });
+});
+
+test('element screenshot isolates component', async ({ page }) => {
+  await page.goto('/catalog');
+
+  await expect(page.getByRole('table')).toHaveScreenshot('catalog-table.png');
+  await expect(page.getByTestId('featured-item')).toHaveScreenshot('featured-item.png');
+});
+```
+
+**Rule of thumb**: Element screenshots for independently changing components. Full page screenshots for key layouts where spacing matters.
+
+### Responsive Visual Testing
+
+**Use when**: Application has responsive breakpoints requiring verification at different viewport sizes.
+
+```typescript
+const breakpoints = [
+  { name: 'phone', width: 375, height: 812 },
+  { name: 'tablet', width: 768, height: 1024 },
+  { name: 'desktop', width: 1440, height: 900 },
+];
+
+for (const bp of breakpoints) {
+  test(`landing at ${bp.name} (${bp.width}x${bp.height})`, async ({ page }) => {
+    await page.setViewportSize({ width: bp.width, height: bp.height });
+    await page.goto('/');
+
+    await expect(page).toHaveScreenshot(`landing-${bp.name}.png`, {
+      animations: 'disabled',
+      fullPage: true,
+    });
+  });
+}
+```
+
+**Alternative: use projects for responsive testing**:
+
+```typescript
+// playwright.config.ts
+export default defineConfig({
+  projects: [
+    {
+      name: 'desktop',
+      testMatch: '**/*.visual.spec.ts',
+      use: {
+        ...devices['Desktop Chrome'],
+        viewport: { width: 1440, height: 900 },
+      },
+    },
+    {
+      name: 'tablet',
+      testMatch: '**/*.visual.spec.ts',
+      use: { ...devices['iPad (gen 7)'] },
+    },
+    {
+      name: 'mobile',
+      testMatch: '**/*.visual.spec.ts',
+      use: { ...devices['iPhone 14'] },
+    },
+  ],
+});
+```
+
+### Component Visual Testing
+
+**Use when**: Testing individual UI components in isolation—buttons, cards, forms, modals. Faster and more stable than full-page screenshots.
+
+```typescript
+test.describe('Button visual states', () => {
+  test('primary button', async ({ page }) => {
+    await page.goto('/storybook/iframe.html?id=button--primary');
+    const btn = page.getByRole('button');
+    await expect(btn).toHaveScreenshot('btn-primary.png', {
+      animations: 'disabled',
+    });
+  });
+
+  test('primary button hover', async ({ page }) => {
+    await page.goto('/storybook/iframe.html?id=button--primary');
+    const btn = page.getByRole('button');
+    await btn.hover();
+    await expect(btn).toHaveScreenshot('btn-primary-hover.png', {
+      animations: 'disabled',
+    });
+  });
+
+  test('button sizes', async ({ page }) => {
+    for (const size of ['small', 'medium', 'large']) {
+      await page.goto(`/storybook/iframe.html?id=button--${size}`);
+      const btn = page.getByRole('button');
+      await expect(btn).toHaveScreenshot(`btn-${size}.png`, {
+        animations: 'disabled',
+      });
+    }
+  });
+});
+```
+
+**Using a dedicated test harness** instead of Storybook:
+
+```typescript
+test.describe('Card component', () => {
+  test.beforeEach(async ({ page }) => {
+    await page.goto('/test-harness/card');
+  });
+
+  test('default state', async ({ page }) => {
+    await expect(page.getByTestId('card')).toHaveScreenshot('card-default.png', {
+      animations: 'disabled',
+    });
+  });
+
+  test('truncates long content', async ({ page }) => {
+    await page.goto('/test-harness/card?content=long');
+    await expect(page.getByTestId('card')).toHaveScreenshot('card-long.png', {
+      animations: 'disabled',
+    });
+  });
+});
+```
+
+### Updating Snapshots
+
+**Use when**: Intentionally changed UI—design refresh, rebrand, new feature. Never update when diff is unexpected.
+
+```bash
+# Update all snapshots
+npx playwright test --update-snapshots
+
+# Update for specific file
+npx playwright test tests/landing.spec.ts --update-snapshots
+
+# Update for specific project
+npx playwright test --project=chromium --update-snapshots
+```
+
+**Workflow for reviewing changes:**
+
+1. Run tests and view failures in HTML report:
+   ```bash
+   npx playwright test
+   npx playwright show-report
+   ```
+   The report shows expected, actual, and diff images side-by-side.
+
+2. If changes are intentional, update:
+   ```bash
+   npx playwright test --update-snapshots
+   ```
+
+3. Review updated snapshots before committing:
+   ```bash
+   git diff --name-only
+   ```
+
+**Tag visual tests for selective updates:**
+
+```typescript
+test('landing visual @visual', async ({ page }) => {
+  await page.goto('/');
+  await expect(page).toHaveScreenshot('landing.png', {
+    animations: 'disabled',
+  });
+});
+```
+
+```bash
+npx playwright test --grep @visual --update-snapshots
+```
+
+### Cross-Browser Visual Testing
+
+**Use when**: Users span Chrome, Firefox, Safari and you need per-browser rendering verification.
+
+Playwright separates snapshots by project name automatically. Each browser gets its own baseline—browsers render fonts and shadows differently.
+
+```typescript
+// playwright.config.ts
+export default defineConfig({
+  expect: {
+    toHaveScreenshot: {
+      animations: 'disabled',
+      maxDiffPixelRatio: 0.01,
+    },
+  },
+  projects: [
+    {
+      name: 'chromium',
+      use: { ...devices['Desktop Chrome'] },
+    },
+    {
+      name: 'firefox',
+      use: { ...devices['Desktop Firefox'] },
+    },
+    {
+      name: 'webkit',
+      use: { ...devices['Desktop Safari'] },
+    },
+  ],
+});
+```
+
+**Strategy**: Run visual tests in a single browser (Chromium on Linux in CI) to minimize snapshot count. Add other browsers only when you have actual cross-browser rendering bugs:
+
+```typescript
+// playwright.config.ts
+export default defineConfig({
+  projects: [
+    {
+      name: 'visual',
+      testMatch: '**/*.visual.spec.ts',
+      use: { ...devices['Desktop Chrome'] },
+    },
+    {
+      name: 'chromium',
+      testIgnore: '**/*.visual.spec.ts',
+      use: { ...devices['Desktop Chrome'] },
+    },
+    {
+      name: 'firefox',
+      testIgnore: '**/*.visual.spec.ts',
+      use: { ...devices['Desktop Firefox'] },
+    },
+  ],
+});
+```
+
+## Decision Guide
+
+| Scenario | Approach | Rationale |
+|---|---|---|
+| Key landing/marketing pages | Full page, `fullPage: true` | Catches layout shifts, spacing, overall harmony |
+| Individual components | Element screenshot | Isolated, fast, immune to unrelated changes |
+| Page with dynamic content | Full page + `mask` | Covers layout while ignoring volatile content |
+| Design system library | Element per variant, zero threshold | Pixel-perfect enforcement |
+| Responsive verification | Screenshot per viewport | Catches breakpoint bugs |
+| Cross-browser consistency | Separate snapshots per browser | Browsers render differently |
+| CI pipeline | Docker container, Linux-only snapshots | Consistent rendering |
+| Threshold: design system | `threshold: 0`, `maxDiffPixels: 0` | Zero tolerance |
+| Threshold: content pages | `maxDiffPixelRatio: 0.01`, `threshold: 0.2` | Minor anti-aliasing variance |
+| Threshold: charts/graphs | `maxDiffPixels: 200`, `threshold: 0.3` | Anti-aliasing on curves varies |
+
+## Anti-Patterns
+
+| Don't | Problem | Do Instead |
+|---|---|---|
+| Visual test every page | Massive maintenance, constant false failures | Pick 5-10 key pages and critical components |
+| Skip masking dynamic content | Screenshots differ every run, permanently flaky | Use `mask` for all volatile elements |
+| Run across macOS, Linux, Windows | Font rendering differs, snapshots never match | Standardize on Linux via Docker |
+| Skip Docker in CI | OS updates shift rendering silently | Pin specific Playwright Docker image |
+| Blindly run `--update-snapshots` | Accepts unintentional regressions | Always review diff in HTML report first |
+| Skip `animations: 'disabled'` | CSS transitions create random diffs | Set globally in config |
+| Replace functional assertions with visual tests | Diffs don't tell you *what* broke | Visual tests complement, never replace |
+| Commit snapshots from different platforms | Tests fail for everyone | All team members use same Docker container |
+| Set threshold too high (`0.1`) | 10% pixel change passes, defeats purpose | Start with `0.01`, adjust per-test |
+| Full page on infinite scroll pages | Page height nondeterministic | Element screenshots on above-the-fold content |
+
+## Troubleshooting
+
+### "Screenshot comparison failed" on first CI run after local development
+
+**Cause**: Snapshots generated on macOS locally. CI runs on Linux. Font rendering differs.
+
+**Fix**: Generate snapshots using Docker:
+
+```bash
+docker run --rm -v $(pwd):/work -w /work \
+  mcr.microsoft.com/playwright:v1.48.0-noble \
+  npx playwright test --update-snapshots --project=visual
+```
+
+Commit Linux-generated snapshots.
+
+### "Expected screenshot to match but X pixels differ"
+
+**Cause**: Anti-aliasing, font hinting, sub-pixel rendering differences.
+
+**Fix**: Add tolerance:
+
+```typescript
+await expect(page).toHaveScreenshot('page.png', {
+  maxDiffPixelRatio: 0.01,
+  threshold: 0.2,
+});
+```
+
+Check HTML report diff image to determine if it's regression or noise.
+
+### Visual tests pass locally but fail in CI (even with Docker)
+
+**Cause**: Different Playwright versions locally vs CI.
+
+**Fix**: Ensure `package.json` version matches Docker image tag:
+
+```json
+{
+  "devDependencies": {
+    "@playwright/test": "latest"
+  }
+}
+```
+
+```yaml
+container:
+  image: mcr.microsoft.com/playwright:v1.48.0-noble
+```
+
+### Animations cause random diff failures
+
+**Cause**: CSS animations captured mid-frame.
+
+**Fix**: Set `animations: 'disabled'` globally:
+
+```typescript
+// playwright.config.ts
+export default defineConfig({
+  expect: {
+    toHaveScreenshot: {
+      animations: 'disabled',
+    },
+  },
+});
+```
+
+For JS animations, wait for stable state before capture.
+
+### Snapshot file names conflict between tests
+
+**Cause**: Two tests use same screenshot name without unique paths.
+
+**Fix**: Use explicit unique names:
+
+```typescript
+await expect(page).toHaveScreenshot('auth-home.png');
+await expect(page).toHaveScreenshot('public-home.png');
+```
+
+Or customize snapshot path template:
+
+```typescript
+export default defineConfig({
+  snapshotPathTemplate: '{testDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{ext}',
+});
+```
+
+### Too many snapshot files to maintain
+
+**Cause**: Visual tests for every page, browser, viewport.
+
+**Fix**: Be selective. Visual test only high-risk pages:
+- Landing and marketing pages
+- Design system components
+- Complex layouts (dashboards, data tables)
+- Pages after major refactor
+
+Skip pages where functional assertions cover key elements.

package/.env.example

@@ -0,0 +1,21 @@
+# ===================================================
+# Arcality — Internal Tool Configuration
+# 
+# RECOMMENDED: Use `arcality init` to configure your
+# project. This creates an arcality.config file with
+# all settings (API key, project, credentials).
+#
+# This .env file contains INTERNAL tool settings only.
+# ===================================================
+
+# ─── INTERNAL (do not modify) ───────────────────────
+# API URL for the Arcality backend
+# This is an internal setting. NOT user-configurable.
+ARCALITY_API_URL=http://localhost:5164
+
+# ─── AI Configuration ──────────────────────────────
+# AI model for test generation and autonomous agent
+CLAUDE_MODEL=claude-4-5-sonnet-20250929
+
+# [Optional] Anthropic API Key (only needed if not using AI proxy)
+# ANTHROPIC_API_KEY=

package/README.md

@@ -0,0 +1,30 @@
+# Arcality
+Herramienta QA para pruebas de UI con IA
+
+Meta 1.0.0: 
+ - Start CLI (interactive test chooser): `arcality` or `npm run arcality` (see [package.json](package.json)).
+ - Run Playwright directly: `npx playwright test` or `npx playwright test <path/to/spec>`.
+ - Auto-generate + run spec (uses Ollama model): `node scripts/gen-and-run.mjs "<prompt>"` or run without args to execute last generated spec (see [scripts/gen-and-run.mjs](scripts/gen-and-run.mjs)).
+ - .env management: the CLI will prompt and write `.env` when missing via [src/envSetup.ts](src/envSetup.ts).
+
+Important: the project reads configuration from the repository `.env` automatically. Do NOT pass `BASE_URL`, `LOGIN_USER`, or `LOGIN_PASSWORD` on the command line — the CLI, scripts, and tests load these values from `.env` internally. Example correct commands (no credentials in the command):
+
+```powershell
+node .\scripts\discover-view.mjs /categories
+node .\scripts\extract-view.mjs /categories
+node .\scripts\gen-and-run.mjs --discover /categories "Crear una categoría de ejemplo"
+```
+
+If `.env` is missing, the interactive setup in `src/envSetup.ts` will prompt to create it; otherwise the tools will use values found in `.env`.
+
+Meta 1.0.1:
+-Hacer personalizable el login, sin depender de Id directamente
+-Tener el test de creación de categoría
+-Tener el test de creación de marca
+-Tener el test de creación de producto
+
+Meta 1.0.2:
+-Pedir la URL inicial, usuario y contraseña para .env
+-Implementar una prueba de ZeroSteps
+-Crear un template para leer YAML
+-Leer un YAML y convertir en el template