laxy-verify 1.2.2 → 1.3.0

package/README.md

@@ -2,7 +2,7 @@
 
 `laxy-verify` is a deployment blocker gate for frontend apps.
 
-Every verification run includes the same build, Lighthouse, E2E, multi-viewport, security, visual diff, and stability coverage regardless of plan.
+Every verification run includes build, Lighthouse, E2E, multi-viewport, security audit, visual diff, broken links, console error monitoring, and stability coverage regardless of plan. Optional opt-in checks add TypeScript type checking, secret scanning, bundle size analysis, outdated dependency detection, deep accessibility audit, deep SEO audit, and Core Web Vitals budget enforcement.
 
 ## Quick start
 
@@ -71,7 +71,17 @@ This is most useful if you ship frontend apps and want a practical gate before:
 | User-flow E2E verification | Yes | No | Manual setup | No |
 | Lighthouse scoring | Yes | Yes | No | No |
 | Visual regression check | Yes | No | No | Yes |
+| Broken link detection | Yes | No | No | No |
+| Console error monitoring | Yes | No | No | No |
+| Cross-browser (Firefox, WebKit) | Yes | No | Yes | No |
 | Release decision (`hold` / `client-ready`) | Yes | No, score only | No | No |
+| TypeScript type check | Yes | No | No | No |
+| Hardcoded secret scanning | Yes | No | No | No |
+| Bundle size analysis | Yes | No | No | No |
+| Outdated dependency check | Yes | No | No | No |
+| Deep WCAG audit (axe-core) | Yes | No | No | No |
+| Deep SEO audit | Yes | No | No | No |
+| Core Web Vitals budget | Yes | No | No | No |
 | Zero-config local start | Yes | No | No | No |
 
 LHCI measures Lighthouse. `laxy-verify` is for deciding whether this frontend is actually safe to ship.
@@ -85,22 +95,42 @@ Use `laxy-verify` when you want to catch things like:
 - desktop looks fine, but a mobile CTA is pushed out of view
 - Lighthouse looks acceptable, but the user-visible path is still not safe to ship
 - a PR needs a clear hold reason instead of a vague "something failed"
+- hardcoded API keys or secrets are about to be pushed to a public repo
+- TypeScript type errors could cause runtime crashes
+- bundle size has silently grown past acceptable limits
+- critical WCAG violations block real users
+- missing SEO meta tags hurt discoverability
 
 ## What it actually checks
 
-A standard run includes:
-
-- production build success
-- Lighthouse thresholds (3 runs for stable evidence)
-- E2E scenarios for user-visible flows
-- multi-viewport checks (desktop, tablet, mobile)
-- blocker-aware reporting and release decisions
-
 Every run includes:
 
-- security audit
-- visual diff evidence
-- stability pass (second E2E run)
+- **production build** — runs your actual build command, exit code determines pass/fail
+- **Lighthouse** — 3 runs averaged for stable performance, accessibility, SEO, and best practices scores
+- **E2E scenarios** — Puppeteer-based user flow testing (auto-detected or configured via `.laxy.yml`)
+- **stability pass** — E2E runs a second time to catch flaky behavior
+- **multi-viewport** — Lighthouse at desktop (1350px), tablet (1024px), and mobile (390px)
+- **security audit** — `npm audit` dependency vulnerability scan
+- **visual diff** — pixel-level screenshot comparison against baseline
+- **broken links** — crawls all internal links and validates HTTP responses
+- **console error monitoring** — captures browser JS errors during E2E execution
+- **cross-browser** — Playwright on Firefox and WebKit if `browsers` is configured in `.laxy.yml`
+
+### Opt-in checks
+
+These checks are off by default. Enable them via CLI flags or `.laxy.yml`.
+
+| Flag | What it checks | Blocker or Advisory |
+|------|----------------|---------------------|
+| `--typecheck` | TypeScript type errors via `tsc --noEmit` | **Blocker** (5+ errors → high severity) |
+| `--secret-scan` | Hardcoded secrets: AWS keys, GitHub tokens, Stripe keys, private keys, Bearer tokens, JWTs, generic password/token assignments | **Blocker** (always high severity) |
+| `--bundle-size` | Next.js or Vite bundle size analysis (first-load JS, largest chunk) | Advisory |
+| `--outdated-check` | Outdated dependencies via `npm outdated --json` | Advisory (major behind → warning) |
+| `--a11y-deep` | Deep WCAG audit via axe-core (critical/serious violations) | **Blocker** (critical → high severity) |
+| `--seo-deep` | SEO meta tags, canonical, robots, Open Graph, Twitter Card, JSON-LD | Advisory |
+| `--vitals-budget` | Core Web Vitals budget: LCP ≤ 2500ms, CLS ≤ 0.1, INP ≤ 200ms | Advisory |
+
+Secret scan never exposes actual credential values — findings are masked with `***` in all output formats.
 
 ## What you get from one run
 
@@ -111,48 +141,26 @@ Every run includes:
 
 Grades still exist, but they are not the main point. The main point is whether the run found blockers you should stop on.
 
-## Quick start
-
-Run it on a frontend app:
+## Example workflow
 
-```bash
-cd your-project
-npx laxy-verify .
-```
+1. Run `npx laxy-verify .` locally before opening or merging a PR.
+2. Fix broken builds, broken flows, and visible regressions.
+3. Commit `.laxy.yml`.
+4. Run `npx laxy-verify --init`.
+5. Let the GitHub Action apply the same gate on every PR.
 
-Generate config plus CI workflow:
+Full verification with all opt-in checks:
 
 ```bash
-npx laxy-verify --init
+npx laxy-verify . --typecheck --secret-scan --bundle-size --outdated-check --a11y-deep --seo-deep --vitals-budget
 ```
 
-That creates:
-
-- `.laxy.yml`
-- `.github/workflows/laxy-verify.yml`
-
-Optional: log in to connect the CLI to your Laxy account:
+Or enable them in `.laxy.yml` and just run:
 
 ```bash
-npx laxy-verify login
-npx laxy-verify whoami
-```
-
-For CI, set `LAXY_TOKEN` instead of interactive login:
-
-```yaml
-env:
-  LAXY_TOKEN: ${{ secrets.LAXY_TOKEN }}
+npx laxy-verify .
 ```
 
-## Example workflow
-
-1. Run `npx laxy-verify .` locally before opening or merging a PR.
-2. Fix broken builds, broken flows, and visible regressions.
-3. Commit `.laxy.yml`.
-4. Run `npx laxy-verify --init`.
-5. Let the GitHub Action apply the same gate on every PR.
-
 ## Example output
 
 ```text
@@ -161,13 +169,35 @@ Grade: Gold
 
 Passed:
 - production build
-- Lighthouse thresholds
-- core user flows
+- Lighthouse thresholds (Performance 92, Accessibility 97, SEO 90, Best Practices 95)
+- core user flows (5/5 scenarios passed)
+- stability pass (second E2E run passed)
 - desktop, tablet, and mobile viewport checks
+- security audit (no known vulnerabilities)
+- visual diff (no regressions)
+- broken links (0 broken / 12 checked)
+- console errors (0 detected)
+- TypeScript (0 errors)
+- Secret scan (0 findings, 42 files)
+- Bundle size (vite, within thresholds)
+- Outdated deps (0 outdated)
+- WCAG deep (0 critical)
+- SEO deep (0 errors)
+- Core Web Vitals budget (LCP 1200ms, CLS 0.05, INP 80ms)
 
 Artifacts:
 - .laxy-result.json
 - laxy-verify-report.md
+
+  Badge (auto-updates with each run):
+    [![Laxy Verify](https://laxy.app/api/badge/your-repo-id)](https://laxy.app)
+```
+
+For Free accounts, the CLI prints a tip instead:
+
+```text
+  Tip: Pro tracks your last 30 runs so you can see if Performance or Grade is improving.
+       https://laxy.app/pricing
 ```
 
 ## The decision it helps you make
@@ -208,7 +238,7 @@ package_manager: auto
 port: 3000
 build_timeout: 300
 dev_timeout: 60
-lighthouse_runs: 1
+lighthouse_runs: 3
 fail_on: bronze
 
 thresholds:
@@ -222,6 +252,31 @@ max_crawl_depth: 3
 max_crawl_pages: 10
 browsers:
   - chromium
+
+# Explicit route list for Lighthouse (optional)
+lighthouse_routes: []
+# Extra routes not discoverable by the crawler (optional)
+extra_routes: []
+# Max crawl-discovered routes to run Lighthouse on (default: 5)
+max_lighthouse_routes: 5
+
+# Visual diff fine-tuning (optional)
+visual_diff:
+  pixelmatch_threshold: 0.1
+  warn_threshold: 30
+  rollback_threshold: 60
+  ignore_selectors: []
+  disable_animations: true
+
+# Opt-in checks (off by default)
+typecheck: false
+secret_scan: false
+secret_scan_ignore_paths: []
+bundle_size: false
+outdated_check: false
+a11y_deep: false
+seo_deep: false
+vitals_budget: false
 ```
 
 Useful adjustments:
@@ -246,6 +301,16 @@ Options:
   --badge
   --init
   --multi-viewport
+  --crawl                Crawl the app to discover routes before E2E
+  --typecheck            Run TypeScript type check (tsc --noEmit)
+  --secret-scan          Scan for hardcoded secrets and credentials
+  --bundle-size          Analyze bundle size (Next.js/Vite)
+  --outdated-check       Check for outdated dependencies
+  --a11y-deep            Deep accessibility audit (axe-core)
+  --seo-deep             Deep SEO audit (meta, OG, JSON-LD)
+  --vitals-budget        Core Web Vitals budget check (LCP, CLS, INP)
+  --share                  Save result and get a shareable URL (Pro)
+  --compare <url>          Compare Lighthouse scores against a reference URL (Pro)
   --help
 
 Subcommands:
@@ -256,6 +321,60 @@ Subcommands:
 
 `--plan-override` is only for plan-label and automation testing. Verification coverage stays the same on every plan.
 
+## Pro features
+
+Pro and Team accounts unlock additional capabilities on top of the same verification run:
+
+- **Result saving and sharing** — `--share` saves the run to your dashboard and returns a shareable URL
+- **Environment comparison** — `--compare <url>` runs Lighthouse against a reference URL (e.g. staging or production) and shows score deltas between your local build and the reference
+- **AI failure analysis** — when a run ends in `hold`, Claude analyzes the failure context and returns a root cause summary with top fix suggestions
+- **History trend tracking** — the last 30 runs are stored so you can see whether your grade and performance scores are improving or regressing over time
+- **Dynamic README badge** — after each run, the CLI prints a Markdown badge snippet that links to your live verification status. The badge auto-updates with every run so your README always reflects current grade
+- **Team Slack / Discord alerts** — grade drops and `hold` verdicts fire webhook notifications with score deltas and blocker details
+- **Weekly team report** — automated weekly summary of verification activity across your team's repos
+
+Free accounts see a hint after each run pointing to the history trend feature.
+
+## Secret scan patterns
+
+When `--secret-scan` is enabled, the following patterns are detected:
+
+| Pattern | Example |
+|---------|--------|
+| AWS Access Key | `AKIA...` (20 chars) |
+| AWS Secret Key | `aws_secret_access_key = '...'` |
+| GitHub Token | `ghp_`, `gho_`, `ghs_`, `ghu_` |
+| Slack Token / Webhook | `xoxb-...`, `hooks.slack.com/...` |
+| Private Key Block | `-----BEGIN RSA PRIVATE KEY-----` |
+| Google API Key | `AIza...` |
+| Stripe Key | `sk_live_...`, `pk_live_...` |
+| Twilio / SendGrid / Mailgun | `SK...`, `SG...`, `key-...` |
+| Hardcoded Bearer Token | `Bearer abc123...` |
+| JWT-like Secret | `eyJ...` |
+| Generic Secret Assignment | `password = '...'`, `api_key = '...'` |
+
+False positive filtering:
+
+- `process.env.*`, `import.meta.env.*`, `NEXT_PUBLIC_*`, `VITE_*` references are ignored
+- GitHub Actions template variables (`${{ secrets.* }}`) are ignored
+- `test/`, `tests/`, `__tests__/`, `spec/` directories are excluded
+- Comment lines (`//`, `*`, `<!--`) are excluded — commented-out secrets are not flagged
+- Placeholder/example values are ignored
+- All secret values in findings are **masked with `***`** — never exposed in output
+
+## Test fixture
+
+A sample Vite + React + TypeScript app is included at `fixtures/sample-app/` for testing all verification checks:
+
+```bash
+cd fixtures/sample-app
+npm install
+npm run build
+npx laxy-verify . --typecheck --secret-scan --bundle-size --outdated-check --skip-lighthouse
+```
+
+See `fixtures/sample-app/README.md` for details on what each check finds in the fixture.
+
 ## Result files
 
 Every run writes `.laxy-result.json`.
@@ -311,6 +430,44 @@ It is a pre-merge and pre-release verification layer, not your entire quality sy
 - a frontend app with a runnable build flow
 - optional: `playwright` if your project already uses it
 
+## Changelog
+
+### v1.3.0 — Pro history trend, dynamic badge, CLI nudge
+
+- **History trend tracking** — Pro accounts now store the last 30 verification runs. Grade and performance scores accumulate so you can see whether your app is improving or regressing over time
+- **Dynamic README badge** — after each run, Pro accounts see a Markdown badge snippet in the CLI output. The badge auto-updates with every run via the `/api/badge/:id` endpoint
+- **CLI nudge** — Free accounts see a one-line tip after each run pointing to the history trend feature
+- Added `generateDynamicBadgeMarkdown(repoId, apiUrl)` export to the `badge` module
+- Added `share_result` and `history_trend` flags to `EntitlementFeatures` interface
+- 17 test files, 88 unit tests
+
+### v1.2.3 — Bug fix
+
+- Fix E2E and visual diff connection failure on Windows with Node.js 17+: `verifyUrl` now uses `localhost` instead of `127.0.0.1`. On Windows, Vite binds to `::1` (IPv6) which does not accept IPv4 connections.
+
+### v1.2.2 — Opt-in verification checks
+
+Added 7 opt-in checks (off by default, enable via CLI flags or `.laxy.yml`):
+
+- `--typecheck` — TypeScript type error detection via `tsc --noEmit`
+- `--secret-scan` — Hardcoded secret scanning with 11 regex patterns and `***` masking
+- `--bundle-size` — Next.js/Vite bundle size analysis
+- `--outdated-check` — Outdated dependency detection via `npm outdated`
+- `--a11y-deep` — Deep WCAG audit via axe-core + Puppeteer
+- `--seo-deep` — SEO meta/OG/JSON-LD audit
+- `--vitals-budget` — Core Web Vitals budget enforcement (LCP, CLS, INP)
+
+Other changes:
+
+- Secret scan findings are **masked** — real credential values never appear in JSON, markdown, or console output
+- Comment lines excluded from secret scanning to reduce false positives
+- `test/`, `tests/`, `__tests__/`, `spec/` directories excluded from scanning
+- `.laxy.yml` supports `secret_scan_ignore_paths` for custom exclusions
+- Verification report includes all new checks in passes checklist and improvement recommendations
+- Markdown report includes all new checks in the delivery evidence table
+- Added `fixtures/sample-app/` test fixture (Vite + React + TypeScript)
+- 17 test files, 88 unit tests
+
 ## Links
 
 - GitHub: https://github.com/SUNgm24/Laxy/tree/main/laxy-verify

package/dist/a11y-deep.d.ts

@@ -0,0 +1,20 @@
+export interface A11yViolation {
+    id: string;
+    impact: "critical" | "serious" | "moderate" | "minor";
+    description: string;
+    helpUrl: string;
+    nodeCount: number;
+    selector: string;
+}
+export interface A11yDeepResult {
+    passed: boolean;
+    criticalCount: number;
+    seriousCount: number;
+    moderateCount: number;
+    minorCount: number;
+    violations: A11yViolation[];
+    url: string;
+    skipped: boolean;
+    summary: string;
+}
+export declare function runA11yDeep(url: string): Promise<A11yDeepResult>;

package/dist/a11y-deep.js

@@ -0,0 +1,161 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.runA11yDeep = runA11yDeep;
+/**
+ * Deep accessibility audit using axe-core + Puppeteer.
+ *
+ * Runs axe-core against the running dev server to produce a detailed
+ * WCAG violation report beyond what Lighthouse provides.
+ * Blocker-capable: critical/serious violations can elevate to warnings.
+ */
+let puppeteerModule = null;
+try {
+    puppeteerModule = require("puppeteer");
+}
+catch {
+    // Puppeteer not available — a11y deep will be skipped
+}
+const IMPACT_ORDER = {
+    critical: 0,
+    serious: 1,
+    moderate: 2,
+    minor: 3,
+};
+async function runA11yDeep(url) {
+    if (!puppeteerModule) {
+        return {
+            passed: true,
+            criticalCount: 0,
+            seriousCount: 0,
+            moderateCount: 0,
+            minorCount: 0,
+            violations: [],
+            url,
+            skipped: true,
+            summary: "Skipped (puppeteer not available)",
+        };
+    }
+    console.log("  Running deep accessibility audit (axe-core)...");
+    const puppeteer = puppeteerModule.default || puppeteerModule;
+    let browser;
+    try {
+        browser = await puppeteer.launch({
+            headless: true,
+            args: ["--no-sandbox", "--disable-setuid-sandbox"],
+        });
+        const page = await browser.newPage();
+        await page.goto(url, { waitUntil: "networkidle2", timeout: 20000 });
+        // Inject and run axe-core
+        const results = await page.evaluate(async () => {
+            // @ts-expect-error — axe is injected at runtime
+            if (typeof window.axe === "undefined") {
+                try {
+                    // Dynamically import axe-core from CDN as fallback
+                    const script = document.createElement("script");
+                    script.src = "https://cdnjs.cloudflare.com/ajax/libs/axe-core/4.9.1/axe.min.js";
+                    script.crossOrigin = "anonymous";
+                    document.head.appendChild(script);
+                    await new Promise((resolve, reject) => {
+                        script.onload = () => resolve();
+                        script.onerror = () => reject(new Error("axe-core CDN load failed"));
+                        setTimeout(() => reject(new Error("axe-core load timed out")), 5000);
+                    });
+                }
+                catch {
+                    return null;
+                }
+            }
+            // @ts-expect-error — axe is now available
+            const result = await window.axe.run(document, {
+                runOnly: {
+                    type: "tag",
+                    values: ["wcag2a", "wcag2aa", "wcag21a", "wcag21aa"],
+                },
+            });
+            return {
+                violations: result.violations.map((v) => ({
+                    id: v.id,
+                    impact: v.impact,
+                    description: v.description,
+                    helpUrl: v.helpUrl,
+                    nodeCount: v.nodes?.length ?? 0,
+                    selector: v.nodes?.[0]?.target?.[0] ?? "",
+                })),
+            };
+        });
+        if (!results) {
+            console.log("  A11y deep: skipped (axe-core could not be loaded)");
+            return {
+                passed: true,
+                criticalCount: 0,
+                seriousCount: 0,
+                moderateCount: 0,
+                minorCount: 0,
+                violations: [],
+                url,
+                skipped: true,
+                summary: "Skipped (axe-core could not be loaded)",
+            };
+        }
+        const violations = results.violations
+            .map((v) => ({
+            id: v.id,
+            impact: v.impact ?? "moderate",
+            description: v.description,
+            helpUrl: v.helpUrl,
+            nodeCount: v.nodeCount,
+            selector: v.selector,
+        }))
+            .filter((v) => ["critical", "serious", "moderate", "minor"].includes(v.impact))
+            .sort((a, b) => (IMPACT_ORDER[a.impact] ?? 3) - (IMPACT_ORDER[b.impact] ?? 3));
+        const criticalCount = violations.filter((v) => v.impact === "critical").length;
+        const seriousCount = violations.filter((v) => v.impact === "serious").length;
+        const moderateCount = violations.filter((v) => v.impact === "moderate").length;
+        const minorCount = violations.filter((v) => v.impact === "minor").length;
+        const passed = criticalCount === 0 && seriousCount === 0;
+        const parts = [];
+        if (criticalCount > 0)
+            parts.push(`${criticalCount} critical`);
+        if (seriousCount > 0)
+            parts.push(`${seriousCount} serious`);
+        if (moderateCount > 0)
+            parts.push(`${moderateCount} moderate`);
+        if (minorCount > 0)
+            parts.push(`${minorCount} minor`);
+        const summary = parts.length > 0
+            ? `${parts.join(", ")} WCAG violation(s)`
+            : "No WCAG violations found";
+        console.log(`  A11y deep: ${summary}`);
+        for (const v of violations.slice(0, 5)) {
+            console.error(`    [${v.impact}] ${v.id}: ${v.description} (${v.nodeCount} nodes)`);
+        }
+        return {
+            passed,
+            criticalCount,
+            seriousCount,
+            moderateCount,
+            minorCount,
+            violations,
+            url,
+            skipped: false,
+            summary,
+        };
+    }
+    catch (err) {
+        console.error(`  A11y deep error: ${err instanceof Error ? err.message : String(err)}`);
+        return {
+            passed: true,
+            criticalCount: 0,
+            seriousCount: 0,
+            moderateCount: 0,
+            minorCount: 0,
+            violations: [],
+            url,
+            skipped: true,
+            summary: `Error: ${err instanceof Error ? err.message : String(err)}`,
+        };
+    }
+    finally {
+        await browser?.close();
+    }
+}

package/dist/audit/broken-links.d.ts

@@ -1,21 +1,25 @@
-/**
- * Broken-links audit.
- * Uses the crawl result to find all internal links, then validates each
- * with an HTTP HEAD/GET request. Links that return 4xx/5xx or timeout are
- * reported as blockers.
- */
-import { type CrawlResult } from "../crawler.js";
-export interface BrokenLink {
-    url: string;
-    path: string;
-    status: number;
-    statusText: string;
-    severity: "critical" | "high";
-}
-export interface BrokenLinksResult {
-    brokenLinks: BrokenLink[];
-    checkedCount: number;
-    hasBrokenLinks: boolean;
-    summary: string;
-}
-export declare function auditBrokenLinks(crawlResult: CrawlResult, baseUrl: string, abortSignal?: AbortSignal): Promise<BrokenLinksResult>;
+/**
+ * Broken-links audit.
+ * Uses the crawl result to find all internal links, then validates each
+ * with an HTTP HEAD/GET request. Links that return 4xx/5xx or timeout are
+ * reported as blockers.
+ */
+import { type CrawlResult } from "../crawler.js";
+export interface BrokenLink {
+    url: string;
+    path: string;
+    status: number;
+    statusText: string;
+    severity: "critical" | "high";
+}
+export interface BrokenLinksResult {
+    brokenLinks: BrokenLink[];
+    checkedCount: number;
+    hasBrokenLinks: boolean;
+    summary: string;
+}
+export interface BrokenLinksAuditOptions {
+    extraRoutes?: string[];
+    abortSignal?: AbortSignal;
+}
+export declare function auditBrokenLinks(crawlResult: CrawlResult | undefined, baseUrl: string, options?: BrokenLinksAuditOptions): Promise<BrokenLinksResult>;