@rettangoli/vt 0.0.14 → 1.0.0-rc12

package/README.md

@@ -1,98 +1,198 @@
-
 # Rettangoli Visual Testing
 
-A visual testing framework for UI components using Playwright and screenshot comparison. Perfect for regression testing and ensuring UI consistency across changes.
+Visual regression testing for Rettangoli specs using Playwright screenshots.
 
-**In production**, this package is typically used through the `rtgl` CLI tool. For development and testing of this package itself, you should call the local CLI directly.
+## Commands
 
-## Features
+- `rtgl vt generate`
+- `rtgl vt screenshot`
+- `rtgl vt report`
+- `rtgl vt accept`
 
-- **Screenshot Generation** - Automatically generate screenshots from HTML specifications
-- **Visual Comparison** - Compare screenshots to detect visual changes
-- **Test Reports** - Generate detailed reports with diff highlights
-- **Playwright Integration** - Uses Playwright for reliable cross-browser testing
-- **Template System** - Liquid templates for flexible HTML generation
-- **Configuration** - YAML-based configuration for easy customization
+Behavior split:
 
-## Development
+- `generate` builds candidate HTML only (no Playwright capture)
+- `screenshot` runs generate flow and captures candidate screenshots
+- `report` compares existing artifacts only (does not run generate/screenshot)
 
-### Prerequisites
+## Public Screenshot Options
 
-- Node.js 18+ or Bun
-- Playwright browsers (automatically installed)
+- `--headed`
+- `--concurrency <number>`
+- `--timeout <ms>`
+- `--wait-event <name>`
+- `--folder <path>` (repeatable)
+- `--group <section-key>` (repeatable)
+- `--item <spec-path>` (repeatable)
 
-### Setup
+## Public Report Options
 
-1. **Install dependencies**:
-```bash
-bun install
-```
+- `--compare-method <method>`
+- `--color-threshold <number>`
+- `--diff-threshold <number>`
+- `--folder <path>` (repeatable)
+- `--group <section-key>` (repeatable)
+- `--item <spec-path>` (repeatable)
+
+## Scoped Runs
+
+Use selectors to run only part of VT in both `screenshot` and `report`:
+
+- `folder`: matches specs by folder prefix under `vt/specs` (example: `components/forms`)
+- `group`: matches derived section page key from `vt.sections` titles (`kebab-case(title)`)
+- `item`: matches a single spec path relative to `vt/specs` (with or without extension)
+
+Selector rules:
+
+- selectors are unioned (OR); any matched item is included
+- if no selector is provided, all items are included
+
+Examples:
 
-2. **Install Playwright browsers** (if not already installed):
 ```bash
-npx playwright install
+# Only specs under a folder
+rtgl vt screenshot --folder components/forms
+
+# Only one section/group key from vt.sections
+rtgl vt screenshot --group components-basic
+
+# Only one spec item (extension optional)
+rtgl vt screenshot --item components/forms/login
+rtgl vt screenshot --item components/forms/login.html
+
+# Combine selectors (union)
+rtgl vt screenshot --group components-basic --item pages/home
+
+# Same selectors for report
+rtgl vt report --folder components/forms
+rtgl vt report --group components-basic
+rtgl vt report --item components/forms/login
 ```
 
-### Project Structure
+Everything else in capture is internal and intentionally not user-configurable.
+
+## Config
 
+`rettangoli.config.yaml`:
+
+```yaml
+vt:
+  path: ./vt
+  port: 3001
+  url: http://127.0.0.1:4173
+  service:
+    start: bun run preview
+  concurrency: 4
+  timeout: 30000
+  waitEvent: vt:ready
+  viewport:
+    id: desktop
+    width: 1280
+    height: 720
+  sections:
+    - title: Components Basic
+      files: components
 ```
-src/
-├── cli/
-│   ├── generate.js    # Generate screenshots from specifications
-│   ├── report.js      # Generate visual comparison reports
-│   ├── accept.js      # Accept screenshot changes as new reference
-│   ├── templates/     # HTML templates for reports
-│   └── static/        # Static assets (CSS, etc.)
-├── common.js          # Shared utilities and functions
-└── index.js           # Main export (empty - CLI focused)
+
+Notes:
+
+- `vt.sections` is required.
+- `vt.service` is optional. When set, VT starts the command before capture, waits for `vt.url`, then stops it after capture.
+- when `vt.service` is omitted and `vt.url` is set, VT expects that URL to already be running.
+- Section page keys are derived as `kebab-case(title)` for flat sections and group `items[].title`.
+- Derived section page keys must be unique case-insensitively.
+- `vt.viewport` supports object or array; each viewport requires `id`, `width`, `height`.
+- `vt.capture` is internal and must be omitted.
+- Viewport contract details: `docs/viewport-contract.md`.
+
+## Spec Frontmatter
+
+Supported frontmatter keys per spec file:
+
+- `title`
+- `description`
+- `template`
+- `url`
+- `waitEvent`
+- `waitSelector`
+- `waitStrategy` (`networkidle` | `load` | `event` | `selector`)
+- `viewport` (object or array of viewport objects)
+- `skipScreenshot`
+- `skipInitialScreenshot`
+- `specs`
+- `steps`
+
+Step action reference:
+
+- `docs/step-actions.md`
+- canonical format is structured action objects (`- action: ...`); legacy one-line string steps are not supported.
+- `assert` supports `js` deep-equal checks for object/array values.
+
+Screenshot naming:
+
+- By default, VT takes an immediate first screenshot before running `steps`.
+- Set `skipInitialScreenshot: true` in frontmatter to skip that immediate first screenshot.
+- First captured screenshot is `-01`.
+- Then `-02`, `-03`, up to `-99`.
+- When viewport id is configured, filenames include `--<viewportId>` before ordinal (for example `pages/home--mobile-01.webp`).
+
+## Docker
+
+A pre-built Docker image with `rtgl` and Playwright browsers is available:
+
+```bash
+docker pull han4wluc/rtgl:playwright-v1.57.0-rtgl-v1.0.0-rc12
 ```
 
-### Core Functionality
+Run commands against a local project:
 
-The visual testing framework provides three main commands:
+```bash
+docker run --rm -v "$(pwd):/workspace" han4wluc/rtgl:playwright-v1.57.0-rtgl-v1.0.0-rc12 rtgl vt screenshot
+docker run --rm -v "$(pwd):/workspace" han4wluc/rtgl:playwright-v1.57.0-rtgl-v1.0.0-rc12 rtgl vt report
+docker run --rm -v "$(pwd):/workspace" han4wluc/rtgl:playwright-v1.57.0-rtgl-v1.0.0-rc12 rtgl vt accept
+```
 
-#### 1. Generate (`vt generate`)
-- Reads HTML specifications from `vt/specs/` directory
-- Generates screenshots using Playwright
-- Saves candidate screenshots for comparison
-- Creates a static site for viewing results
+Note:
 
-#### 2. Report (`vt report`)
-- Compares candidate screenshots with reference screenshots
-- Generates visual diff reports highlighting changes
-- Creates an HTML report with before/after comparisons
-- Uses `looks-same` library for pixel-perfect comparison
+- Image default working directory is `/workspace`.
+- Use `-w /workspace/<subdir>` only when running commands from a subfolder within the mounted project.
 
-#### 3. Accept (`vt accept`)
-- Accepts candidate screenshots as new reference images
-- Updates the golden/reference screenshot directory
-- Used when visual changes are intentional
+Supports `linux/amd64` and `linux/arm64`.
 
-### Configuration
+## Development
 
-The framework reads configuration from `rettangoli.config.yaml`:
+Run unit tests:
 
-```yaml
-vt:
-  port: 3001
-  screenshotWaitTime: 500
-  skipScreenshots: false
+```bash
+bun test
 ```
 
-### Testing Your Changes
+Default unit run behavior:
+
+- `bun test` skips the real-browser smoke tests in `spec/e2e-smoke.spec.js` unless `VT_E2E=1`.
+- Skipped smoke tests are:
+  - `runs generate, accept, and report with real screenshots`
+  - `supports waitEvent readiness with real browser screenshots`
+  - `supports managed service lifecycle with vt.service.start and vt.url`
 
-**Note**: This package doesn't include example files in its directory. For testing during development, use examples from other packages (like `rettangoli-ui`) and call the CLI directly:
+Run real-browser smoke:
 
 ```bash
-# Call the local CLI directly for development
-node ../rettangoli-cli/cli.js vt generate
-node ../rettangoli-cli/cli.js vt report
-node ../rettangoli-cli/cli.js vt accept
+VT_E2E=1 bun test spec/e2e-smoke.spec.js
 ```
 
-**Production usage** (when rtgl is installed globally):
+Run Docker E2E tests (requires Docker daemon running):
+
+```bash
+# Full pipeline: build test image → run all E2E scenarios
+bun run test:e2e:full
+
+# Scenarios only (skip image build, assumes image already exists)
+bun run test:e2e
+```
+
+Optional benchmark fixture:
+
 ```bash
-rtgl vt generate
-rtgl vt report
-rtgl vt accept
+bun run bench:capture
 ```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@rettangoli/vt",
-  "version": "0.0.14",
+  "version": "1.0.0-rc12",
   "description": "Rettangoli Visual Testing",
   "type": "module",
   "main": "./src/index.js",
@@ -9,7 +9,10 @@
     "./cli": "./src/cli/index.js"
   },
   "scripts": {
-    "test": "echo \"Error: no test specified\" && exit 1"
+    "test": "vitest run --reporter=verbose",
+    "test:e2e": "node e2e/run.js",
+    "test:e2e:full": "bash scripts/docker-e2e-test.sh",
+    "bench:capture": "node ./scripts/benchmark-capture.js"
   },
   "dependencies": {
     "commander": "^13.1.0",
@@ -19,5 +22,9 @@
     "playwright": "1.57.0",
     "sharp": "^0.34.5",
     "shiki": "^3.3.0"
+  },
+  "devDependencies": {
+    "puty": "^0.1.1",
+    "vitest": "^4.0.15"
   }
 }

package/src/capture/capture-scheduler.js

@@ -0,0 +1,206 @@
+import { chromium } from "playwright";
+import { PlaywrightRunner } from "./playwright-runner.js";
+import { ResultCollector } from "./result-collector.js";
+import { resolveWorkerPlan } from "./worker-plan.js";
+
+function nowMs() {
+  return performance.now();
+}
+
+function createTaskQueues(tasks) {
+  const freshQueue = tasks
+    .map((task, sourceOrder) => ({
+      task,
+      sourceOrder,
+      attempt: 1,
+      queueType: "fresh",
+      enqueuedAtMs: nowMs(),
+    }))
+    .sort((left, right) => {
+      const leftCost = left.task.estimatedCost ?? 0;
+      const rightCost = right.task.estimatedCost ?? 0;
+      if (leftCost !== rightCost) {
+        return leftCost - rightCost;
+      }
+      return right.sourceOrder - left.sourceOrder;
+    });
+
+  const retryQueue = [];
+  let dispatchCount = 0;
+  const fairRetryInterval = 4; // 3 fresh dispatches then 1 retry when available
+
+  const getNextTask = () => {
+    const shouldDispatchRetry = retryQueue.length > 0
+      && (freshQueue.length === 0 || (dispatchCount % fairRetryInterval) === (fairRetryInterval - 1));
+
+    if (shouldDispatchRetry) {
+      dispatchCount += 1;
+      return retryQueue.shift();
+    }
+    if (freshQueue.length > 0) {
+      dispatchCount += 1;
+      return freshQueue.pop();
+    }
+    if (retryQueue.length > 0) {
+      dispatchCount += 1;
+      return retryQueue.shift();
+    }
+    return null;
+  };
+
+  const enqueueRetry = (item) => {
+    retryQueue.push({
+      task: item.task,
+      attempt: item.attempt + 1,
+      queueType: "retry",
+      enqueuedAtMs: nowMs(),
+    });
+  };
+
+  return {
+    getNextTask,
+    enqueueRetry,
+  };
+}
+
+export async function runCaptureScheduler(options) {
+  const {
+    tasks,
+    screenshotsDir,
+    workerCount: requestedWorkers,
+    isolationMode,
+    screenshotWaitTime,
+    waitEvent,
+    waitSelector,
+    waitStrategy,
+    navigationTimeout,
+    readyTimeout,
+    screenshotTimeout,
+    maxRetries,
+    recycleEvery,
+    metricsPath,
+    headless,
+  } = options;
+
+  const { workerCount, adaptivePolicy } = resolveWorkerPlan(requestedWorkers);
+  console.log(
+    `Capture scheduler: mode=${adaptivePolicy.mode}, workers=${workerCount}, isolation=${isolationMode}`,
+  );
+
+  const collector = new ResultCollector({
+    totalTasks: tasks.length,
+    metricsPath,
+    workerCount,
+    isolationMode,
+    maxRetries,
+    adaptivePolicy,
+    schedulingPolicy: {
+      type: "duration-aware-fair-retry",
+      freshBeforeRetry: 3,
+    },
+  });
+
+  if (!tasks.length) {
+    const { summary } = collector.finalize();
+    return {
+      summary,
+      failures: [],
+    };
+  }
+
+  const queue = createTaskQueues(tasks);
+  const getNextTask = queue.getNextTask;
+
+  const browser = await chromium.launch({ headless });
+  const runners = [];
+
+  try {
+    for (let index = 0; index < workerCount; index += 1) {
+      const runner = new PlaywrightRunner({
+        workerId: index + 1,
+        browser,
+        screenshotsDir,
+        isolationMode,
+        screenshotWaitTime,
+        waitEvent,
+        waitSelector,
+        waitStrategy,
+        navigationTimeout,
+        readyTimeout,
+        screenshotTimeout,
+      });
+      await runner.initialize();
+      runners.push(runner);
+    }
+
+    const workerLoops = runners.map(async (runner) => {
+      let processedSinceRecycle = 0;
+
+      while (true) {
+        const item = getNextTask();
+        if (!item) {
+          break;
+        }
+
+        const queueWaitMs = Math.max(0, nowMs() - item.enqueuedAtMs);
+        const attemptStartMs = nowMs();
+
+        try {
+          const result = await runner.runTask(item.task, item.attempt);
+          const attemptMs = nowMs() - attemptStartMs;
+          collector.recordSuccess(item.task, result, {
+            workerId: runner.workerId,
+            queueType: item.queueType,
+            queueWaitMs,
+            attemptMs,
+          });
+          processedSinceRecycle += 1;
+
+          if (
+            isolationMode === "fast"
+            && recycleEvery > 0
+            && processedSinceRecycle >= recycleEvery
+          ) {
+            await runner.recycleSharedContext();
+            collector.recordRecycle(runner.workerId, `recycleEvery=${recycleEvery}`);
+            processedSinceRecycle = 0;
+          }
+        } catch (error) {
+          const attemptMs = nowMs() - attemptStartMs;
+          if (item.attempt <= maxRetries) {
+            collector.recordRetry(item.task, item.attempt, error.message, {
+              workerId: runner.workerId,
+              queueType: item.queueType,
+              queueWaitMs,
+              attemptMs,
+            });
+            queue.enqueueRetry(item);
+            continue;
+          }
+
+          collector.recordFailure(item.task, item.attempt, error.message, {
+            workerId: runner.workerId,
+            queueType: item.queueType,
+            queueWaitMs,
+            attemptMs,
+          });
+        }
+      }
+    });
+
+    await Promise.all(workerLoops);
+  } finally {
+    await Promise.all(
+      runners.map(async (runner) => {
+        await runner.dispose();
+      }),
+    );
+    await browser.close();
+  }
+
+  const { summary, failures } = collector.finalize();
+  return {
+    summary,
+    failures,
+  };
+}