@trylayout/qa 0.1.3 → 0.1.5

package/README.md

@@ -1,5 +1,9 @@
 # Layout QA
 
+[![npm](https://img.shields.io/npm/v/@trylayout/qa?label=npm)](https://www.npmjs.com/package/@trylayout/qa)
+[![CI](https://github.com/Layout-App/layout-qa/actions/workflows/ci.yml/badge.svg)](https://github.com/Layout-App/layout-qa/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-black.svg)](LICENSE)
+
 Layout QA is a local browser QA protocol and runner for AI-built frontends. It runs deterministic flows against a local or preview URL, captures screenshots at meaningful checkpoints, checks browser health, and writes a static HTML report.
 
 The core loop is intentionally local:
@@ -12,6 +16,10 @@ npx @trylayout/qa run --target-url http://localhost:5173 --scenario happy_path -
 
 No account, upload, hosted service, or external docs are required.
 
+## Example Report
+
+![Layout QA HTML report for a synthetic CRM demo](docs/assets/layout-qa-report.png)
+
 Package names:
 
 - Canonical npm package: `@trylayout/qa`
@@ -29,8 +37,8 @@ npx layout-qa run --target-url http://localhost:5173 --scenario happy_path --ope
 
 Frontend agents can move faster when they have a visual feedback loop they can run themselves. Layout gives the agent a small protocol:
 
-- Wire deterministic API/auth mocks behind a local env flag such as `VITE_LAYOUT_QA_MOCKS=1`.
-- Switch mock states with `localStorage["layout.qa.scenario"]`.
+- Wire deterministic API/auth responses behind a local env flag such as `VITE_LAYOUT_QA_MOCKS=1`.
+- Switch response states with `localStorage["layout.qa.scenario"]`.
 - Declare high-value browser flows in `.layout/qa-flows.json`.
 - Run the CLI locally and inspect the generated screenshots/report.
 
@@ -71,7 +79,7 @@ Create a starter flow manifest:
 npx @trylayout/qa init
 ```
 
-Start your app with whatever mock flag your project uses:
+Start your app with whatever local QA flag your project uses:
 
 ```bash
 VITE_LAYOUT_QA_MOCKS=1 npm run dev
@@ -111,7 +119,7 @@ Options:
 
 ```text
 --target-url <url>     URL of the running frontend to test.
---scenario <name>      Mock scenario to activate. Defaults to happy_path.
+--scenario <name>      Scenario to activate. Defaults to happy_path.
 --flows <path>         Flow manifest path. Defaults to .layout/qa-flows.json.
 --out <path>           Artifact directory. Defaults to .layout/runs.
 --viewport <value>     Viewport preset or size. Use desktop, tablet, mobile, or WIDTHxHEIGHT. Defaults to desktop.
@@ -241,7 +249,7 @@ Presets:
 
 The selected viewport is written to `result.json`, shown in the HTML report, and included in the run directory name.
 
-## Mock Scenarios
+## Scenarios
 
 Before the app loads, the runner sets:
 
@@ -250,7 +258,7 @@ localStorage.setItem("layout.qa.scenario", "<scenario>");
 sessionStorage.setItem("layout.qa.runner", "1");
 ```
 
-Your app can use `layout.qa.scenario` to switch deterministic mock states:
+Your app can use `layout.qa.scenario` to switch deterministic API/auth response states:
 
 - `happy_path`: normal populated data.
 - `empty`: successful responses with empty states.
@@ -271,14 +279,14 @@ Create a local-only browser QA loop that an agent can run while changing fronten
 Rules:
 - Do not add a standalone mock server.
 - Do not require a hosted Layout service.
-- Keep all mock data local to this app.
-- Gate mocks behind a local-only env flag such as VITE_LAYOUT_QA_MOCKS=1, NEXT_PUBLIC_LAYOUT_QA_MOCKS=1, or the framework-appropriate equivalent.
-- Use localStorage["layout.qa.scenario"] to select at least happy_path, empty, and error mock states.
+- Keep all deterministic response fixtures local to this app.
+- Gate deterministic API/auth responses behind a local-only env flag such as VITE_LAYOUT_QA_MOCKS=1, NEXT_PUBLIC_LAYOUT_QA_MOCKS=1, or the framework-appropriate equivalent.
+- Use localStorage["layout.qa.scenario"] to select at least happy_path, empty, and error response states.
 - Hide any local QA switcher or debug controls when sessionStorage["layout.qa.runner"] === "1".
 
 Implementation:
 - Add deterministic API fixtures for the highest-value frontend route.
-- If the app has a central auth/session abstraction, add a deterministic mock user only when the Layout QA env flag is enabled.
+- If the app has a central auth/session abstraction, add a deterministic QA user only when the Layout QA env flag is enabled.
 - If auth is scattered or provider-SDK-only, leave a clear note in the PR/code comments and start with public or logged-out flows.
 - Add .layout/qa-flows.json with one smoke flow for the most important page.
 - Prefer visible text and stable selectors.
@@ -332,3 +340,7 @@ This package is intentionally small:
 - It does not perform AI review by itself.
 
 Those hosted/reporting layers can be added later without changing the local protocol.
+
+## Feedback
+
+Issues and examples are welcome in [GitHub Issues](https://github.com/Layout-App/layout-qa/issues). You can also reach me on X at [@tscepo](https://x.com/tscepo).

package/build/cli/layoutQa.js

@@ -26,7 +26,7 @@ Commands:
 
 Options:
   --target-url <url>     URL of the running frontend to test.
-  --scenario <name>      Mock scenario to activate. Defaults to happy_path.
+  --scenario <name>      Scenario to activate. Defaults to happy_path.
   --flows <path>         Flow manifest path. Defaults to .layout/qa-flows.json.
   --out <path>           Artifact directory. Defaults to .layout/runs.
   --viewport <value>     Viewport preset or size. Use desktop, tablet, mobile, or WIDTHxHEIGHT. Defaults to desktop.

package/build/report.js

@@ -57,7 +57,7 @@ function renderStatusBadge(status) {
 function renderCheckList(result) {
     return result.checks
         .map(check => `<li class="row">
-        <span class="status ${check.passed ? 'passed' : 'failed'}">${check.passed ? 'PASS' : 'FAIL'}</span>
+        <span class="status ${check.passed ? 'passed' : 'failed'}">${check.passed ? 'Pass' : 'Fail'}</span>
         <div>
           <p class="row-title">${escapeHtml(check.label)}</p>
           ${check.detail
@@ -123,81 +123,149 @@ function renderReport(input) {
   <style>
     :root {
       color-scheme: light;
-      --bg: #f7f7f2;
-      --panel: #fffdf8;
-      --line: #e4e1d8;
-      --text: #1b1a17;
-      --muted: #65635d;
-      --green: #2f7a45;
-      --green-bg: #e6f4ea;
-      --red: #9b2c2c;
-      --red-bg: #fff0f0;
-      --amber: #7a5c14;
-      --amber-bg: #fff5d6;
+      --bg: #ffffff;
+      --surface: #f5f5f7;
+      --surface-strong: #fbfbfd;
+      --line: #d2d2d7;
+      --line-soft: #e8e8ed;
+      --text: #1d1d1f;
+      --muted: #6e6e73;
+      --blue: #06c;
+      --green: #248a3d;
+      --red: #d70015;
+      --amber: #b25000;
     }
     * { box-sizing: border-box; }
     body {
       margin: 0;
       background: var(--bg);
       color: var(--text);
-      font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
-      line-height: 1.5;
+      font-family: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "SF Pro Text", "Segoe UI", sans-serif;
+      line-height: 1.47059;
+      -webkit-font-smoothing: antialiased;
+      text-rendering: optimizeLegibility;
+    }
+    main { max-width: 1180px; margin: 0 auto; padding: 56px 28px 72px; }
+    header.page {
+      display: flex;
+      justify-content: space-between;
+      gap: 32px;
+      align-items: flex-start;
+      border-bottom: 1px solid var(--line-soft);
+      padding-bottom: 36px;
     }
-    main { max-width: 1120px; margin: 0 auto; padding: 32px 24px 56px; }
-    header.page { display: flex; justify-content: space-between; gap: 24px; align-items: flex-start; border-bottom: 1px solid var(--line); padding-bottom: 24px; }
     h1, h2, h3, p { margin: 0; }
-    h1 { font-size: clamp(2rem, 5vw, 4rem); line-height: 1; font-weight: 600; letter-spacing: 0; }
-    h2 { font-size: 1rem; font-weight: 650; margin-bottom: 12px; }
-    h3 { font-size: 1rem; font-weight: 650; }
-    section { margin-top: 28px; }
-    a { color: inherit; }
-    .eyebrow { color: var(--muted); font-size: .78rem; font-weight: 650; text-transform: uppercase; letter-spacing: .04em; }
-    .summary { display: grid; grid-template-columns: repeat(5, minmax(0, 1fr)); gap: 12px; margin-top: 24px; }
+    h1 {
+      font-size: clamp(2.75rem, 7vw, 5.5rem);
+      line-height: .95;
+      font-weight: 500;
+      letter-spacing: 0;
+    }
+    h2 { font-size: 1.375rem; line-height: 1.2; font-weight: 500; margin-bottom: 18px; }
+    h3 { font-size: 1.0625rem; line-height: 1.25; font-weight: 500; }
+    section { margin-top: 42px; }
+    a { color: var(--blue); text-decoration: none; }
+    a:hover { text-decoration: underline; }
+    .eyebrow {
+      color: var(--muted);
+      font-size: .8125rem;
+      font-weight: 500;
+      text-transform: uppercase;
+      letter-spacing: .04em;
+      margin-bottom: 10px;
+    }
+    .summary {
+      display: grid;
+      grid-template-columns: repeat(5, minmax(0, 1fr));
+      gap: 1px;
+      margin-top: 32px;
+      border: 1px solid var(--line-soft);
+      border-radius: 14px;
+      overflow: hidden;
+      background: var(--line-soft);
+    }
     .metric, .panel, .step, .issue {
-      border: 1px solid var(--line);
-      background: var(--panel);
-      border-radius: 8px;
+      background: var(--surface-strong);
+    }
+    .metric { padding: 18px 20px; min-width: 0; }
+    .metric dt { color: var(--muted); font-size: .8125rem; }
+    .metric dd { margin: 5px 0 0; font-weight: 500; overflow-wrap: anywhere; }
+    .panel {
+      padding: 22px;
+      border: 1px solid var(--line-soft);
+      border-radius: 14px;
+    }
+    .stack { display: grid; gap: 0; list-style: none; margin: 0; padding: 0; }
+    .row {
+      display: grid;
+      grid-template-columns: 64px minmax(0, 1fr);
+      gap: 16px;
+      padding: 14px 0;
+      border-top: 1px solid var(--line-soft);
     }
-    .metric { padding: 14px; min-width: 0; }
-    .metric dt { color: var(--muted); font-size: .82rem; }
-    .metric dd { margin: 4px 0 0; font-weight: 650; overflow-wrap: anywhere; }
-    .panel { padding: 16px; }
-    .stack { display: grid; gap: 10px; list-style: none; margin: 0; padding: 0; }
-    .row { display: grid; grid-template-columns: 56px minmax(0, 1fr); gap: 12px; padding: 10px 0; border-top: 1px solid var(--line); }
     .row:first-child { border-top: 0; padding-top: 0; }
     .row:last-child { padding-bottom: 0; }
-    .row-title { font-weight: 650; overflow-wrap: anywhere; }
-    .muted { color: var(--muted); font-size: .92rem; overflow-wrap: anywhere; }
-    .detail { margin-top: 8px; color: #3f3d38; overflow-wrap: anywhere; }
+    .row-title { font-weight: 500; overflow-wrap: anywhere; }
+    .muted { color: var(--muted); font-size: .9375rem; overflow-wrap: anywhere; }
+    .detail { margin-top: 10px; color: #424245; overflow-wrap: anywhere; }
     .break { word-break: break-all; }
-    .status { font-size: .78rem; font-weight: 750; padding-top: 2px; }
+    .status { font-size: .8125rem; font-weight: 500; padding-top: 2px; }
     .status.passed { color: var(--green); }
     .status.failed { color: var(--red); }
     .badge {
       display: inline-flex;
       align-items: center;
-      border-radius: 6px;
-      border: 1px solid var(--line);
-      padding: 4px 8px;
-      font-size: .82rem;
-      font-weight: 650;
+      border-radius: 999px;
+      border: 1px solid var(--line-soft);
+      padding: 5px 10px;
+      font-size: .875rem;
+      font-weight: 500;
       text-transform: capitalize;
       white-space: nowrap;
+      background: var(--surface);
+    }
+    .badge.passed { color: var(--green); }
+    .badge.failed { color: var(--red); }
+    .badge.skipped { color: var(--amber); }
+    .badge.hero { font-size: 1rem; padding: 8px 14px; }
+    .step {
+      padding: 22px;
+      margin-top: 16px;
+      border: 1px solid var(--line-soft);
+      border-radius: 14px;
     }
-    .badge.passed { color: var(--green); background: var(--green-bg); border-color: #c8e6d0; }
-    .badge.failed { color: var(--red); background: var(--red-bg); border-color: #f0c9c9; }
-    .badge.skipped { color: var(--amber); background: var(--amber-bg); border-color: #f0dc9f; }
-    .badge.hero { font-size: .95rem; padding: 8px 12px; }
-    .step { padding: 16px; margin-top: 12px; }
     .step-header { display: flex; justify-content: space-between; gap: 16px; align-items: flex-start; }
-    .screenshot-link { display: block; margin-top: 14px; border: 1px solid var(--line); border-radius: 8px; overflow: hidden; background: white; }
+    .screenshot-link {
+      display: block;
+      margin-top: 18px;
+      border: 1px solid var(--line);
+      border-radius: 12px;
+      overflow: hidden;
+      background: white;
+      box-shadow: 0 12px 36px rgba(0, 0, 0, .06);
+    }
+    .screenshot-link:hover { text-decoration: none; }
     img { display: block; width: 100%; height: auto; }
-    .issue { padding: 12px; border-color: #f0c9c9; background: #fff8f8; }
-    .next { border-color: #d9d6cb; }
+    .issue {
+      padding: 16px;
+      border: 1px solid #ffd7d9;
+      border-radius: 12px;
+      background: #fff7f7;
+    }
+    .next { background: var(--surface); }
     .empty { color: var(--muted); }
-    .footer { margin-top: 28px; color: var(--muted); font-size: .9rem; }
+    .footer {
+      margin-top: 42px;
+      padding-top: 20px;
+      border-top: 1px solid var(--line-soft);
+      color: var(--muted);
+      font-size: .9375rem;
+      overflow-wrap: anywhere;
+    }
+    ul:not(.stack) { margin: 14px 0 0; padding-left: 1.2rem; color: #424245; }
+    li + li { margin-top: 6px; }
     @media (max-width: 760px) {
-      main { padding: 24px 16px 40px; }
+      main { padding: 34px 18px 48px; }
       header.page { display: grid; }
       .summary { grid-template-columns: 1fr; }
     }

package/build/runner.js

@@ -81,7 +81,7 @@ function buildChecks(input) {
         },
         {
             id: 'scenario_ready',
-            label: 'Mock scenario is available',
+            label: 'Scenario flag is available',
             passed: scenarioReady,
             detail: input.controlsPresent
                 ? `Layout QA controls detected; requested ${input.scenario}.`
@@ -126,11 +126,11 @@ function buildNextAction(input) {
         return {
             category: 'target_unreachable',
             title: 'Target URL did not load cleanly',
-            detail: 'Layout could not reach the target page well enough to evaluate mock states.',
+            detail: 'Layout could not reach the target page well enough to evaluate deterministic response states.',
             docsPath: flows_1.FLOW_MANIFEST_PATH,
             nextSteps: [
                 'Start the app or deploy preview URL before running Layout QA.',
-                'Use the URL where the frontend is served with the Layout mock env flag enabled.',
+                'Use the URL where the frontend is served with the Layout QA env flag enabled.',
                 'Retry the same scenario after the target URL is reachable from the runner.',
             ],
         };
@@ -138,13 +138,13 @@ function buildNextAction(input) {
     if (failedCheckIds.has('scenario_ready')) {
         return {
             category: 'fixtures',
-            title: 'Mock scenario was not active',
-            detail: 'The page loaded, but Layout could not confirm that the requested mock scenario was available.',
+            title: 'Scenario flag was not active',
+            detail: 'The page loaded, but Layout could not confirm that the requested scenario was available.',
             docsPath: flows_1.FLOW_MANIFEST_PATH,
             nextSteps: [
-                'Confirm the target is running with the Layout mock env flag set to 1.',
+                'Confirm the target is running with the Layout QA env flag set to 1.',
                 'Check that the app reads localStorage["layout.qa.scenario"] before API calls run.',
-                `Review ${flows_1.FLOW_MANIFEST_PATH}, the Layout QA docs, and the fixture file for missing handlers.`,
+                `Review ${flows_1.FLOW_MANIFEST_PATH}, the Layout QA docs, and the API/auth response fixtures for missing handlers.`,
             ],
         };
     }
@@ -154,11 +154,11 @@ function buildNextAction(input) {
             category: 'flow',
             title: 'Flow step needs review',
             detail: failedStep?.detail ||
-                'The target loaded with mocks, but a declared QA flow step failed.',
+                'The target loaded with deterministic responses, but a declared QA flow step failed.',
             docsPath: flows_1.FLOW_MANIFEST_PATH,
             nextSteps: [
                 `Inspect ${flows_1.FLOW_MANIFEST_PATH} and confirm the failing step still matches the app UI.`,
-                'Update selectors, visible text assertions, or scenario fixtures so the flow follows real user behavior.',
+                'Update selectors, visible text assertions, or scenario responses so the flow follows real user behavior.',
                 `Use ${flows_1.QA_DOCS_URL} for the supported flow step schema.`,
             ],
         };
@@ -169,24 +169,24 @@ function buildNextAction(input) {
         return {
             category: 'browser_errors',
             title: 'Browser errors need review',
-            detail: 'The target loaded with mocks, but browser errors or failed requests were observed.',
+            detail: 'The target loaded with deterministic responses, but browser errors or failed requests were observed.',
             docsPath: flows_1.FLOW_MANIFEST_PATH,
             nextSteps: [
                 'Inspect the issues captured on this QA run.',
-                'Add or correct fixtures for unhandled frontend API requests.',
-                'Fix app code that throws under the selected mock scenario, then rerun.',
+                'Add or correct fixtures for unhandled frontend API/auth requests.',
+                'Fix app code that throws under the selected scenario, then rerun.',
             ],
         };
     }
     if (appearsToBePublicOrAuthSurface(input.bodyTextSample)) {
         return {
             category: 'auth_boundary',
-            title: 'Public surface reached; wire mock auth next',
-            detail: 'The run passed the basic mock-browser checks, but the page appears to be a logged-out or public surface. Authenticated flows need a mockable auth boundary before Layout can test them end to end.',
+            title: 'Public surface reached; wire deterministic auth next',
+            detail: 'The run passed the basic browser checks, but the page appears to be a logged-out or public surface. Authenticated flows need a deterministic auth boundary before Layout can test them end to end.',
             docsPath: flows_1.FLOW_MANIFEST_PATH,
             nextSteps: [
-                `Use ${flows_1.FLOW_MANIFEST_PATH} and the Layout QA docs to add or confirm a central mock auth boundary.`,
-                'Expose a deterministic mock user/session only when the Layout mock env flag is enabled.',
+                `Use ${flows_1.FLOW_MANIFEST_PATH} and the Layout QA docs to add or confirm a central auth boundary for QA runs.`,
+                'Expose a deterministic user/session only when the Layout QA env flag is enabled.',
                 'Point the next QA run at an authenticated route and rerun happy_path, empty, and error scenarios.',
             ],
         };
@@ -194,11 +194,11 @@ function buildNextAction(input) {
     return {
         category: 'ready',
         title: 'Ready for deeper flow coverage',
-        detail: 'The target loaded with the requested mock scenario and no basic browser issues were detected.',
+        detail: 'The target loaded with the requested scenario and no basic browser issues were detected.',
         docsPath: flows_1.FLOW_MANIFEST_PATH,
         nextSteps: [
             `Add route-specific Playwright-style flow steps to ${flows_1.FLOW_MANIFEST_PATH} for the highest-value user path.`,
-            'Expand fixtures for any API calls encountered by that flow.',
+            'Expand deterministic API/auth responses for any requests encountered by that flow.',
             'Run the same flow across happy_path, empty, and error scenarios.',
         ],
     };
@@ -635,7 +635,7 @@ function buildRunnerErrorResult(message, viewport = viewports_1.DEFAULT_VIEWPORT
             docsPath: flows_1.FLOW_MANIFEST_PATH,
             nextSteps: [
                 'Confirm the target URL is reachable by the runner.',
-                'Confirm the app is served with the Layout mock env flag enabled.',
+                'Confirm the app is served with the Layout QA env flag enabled.',
                 'Retry after the target loads consistently in a browser.',
                 `Viewport used for this run: ${(0, viewports_1.formatViewport)(viewport)}.`,
             ],

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@trylayout/qa",
-  "version": "0.1.3",
+  "version": "0.1.5",
   "description": "Local browser QA runner and HTML reports for AI-built frontends.",
   "license": "MIT",
   "author": "Layout",
@@ -26,6 +26,7 @@
   },
   "files": [
     "build",
+    "docs/assets/layout-qa-report.png",
     "README.md",
     "LICENSE"
   ],
@@ -38,6 +39,7 @@
   "scripts": {
     "build": "tsc",
     "check": "tsc --noEmit",
+    "publish:alias": "cd packages/layout-qa && npm publish --access public",
     "prepack": "npm run build"
   },
   "dependencies": {