devlyn-cli 1.3.0 → 1.3.1

package/config/skills/devlyn:browser-validate/SKILL.md

@@ -1,9 +1,11 @@
 ---
 name: devlyn:browser-validate
-description: Browser-based validation for web applications — smoke tests, user flow testing, visual checks, and runtime error detection. Starts the dev server, navigates the app, and reports what's broken with screenshot evidence. Use this skill whenever the user says "test in browser", "check if it works", "smoke test", "browser test", "validate the UI", "does the app run", or when auto-resolve needs to verify web changes actually render and function correctly. Also use proactively after implementing UI changes to catch runtime errors that static analysis misses.
+description: Browser-based validation for web applications — verifies that implemented features actually work by testing them in a real browser. Starts the dev server, tests the feature end-to-end (click buttons, fill forms, verify results), and reports what's broken with screenshot evidence. Use this skill whenever the user says "test in browser", "check if it works", "does the feature work", "browser test", "validate the UI", or when auto-resolve needs to verify web changes actually function correctly. Also use proactively after implementing UI changes. The primary goal is feature verification, not just checking if pages render.
 ---
 
-Browser validation for web applications. Starts a dev server, tests in a real browser (or falls back to Playwright/curl), and reports findings with evidence. Designed to catch the bugs that pass every static check but break when a user actually clicks something — runtime errors, failed API calls, blank pages, broken interactions.
+Verify that implemented features actually work in the browser. The primary job is to test the feature that was just built — click the button, fill the form, check the result. Smoke tests and visual checks are supporting checks, not the main event.
+
+The whole point of browser validation is to catch the gap between "code looks correct" and "user can actually do the thing." Static analysis and unit tests can confirm the code is well-structured. Browser validation confirms it *works*.
 
 <config>
 $ARGUMENTS
@@ -13,80 +15,74 @@ $ARGUMENTS
 
 ## PHASE 1: DETECT
 
-1. **Framework detection**: Read `package.json` → identify framework from dependencies (`next`, `vite`, `react-scripts`, `nuxt`, `astro`, `svelte`, `remix`, `angular`). Find the start command from `scripts.dev`, `scripts.start`, or `scripts.preview`.
+1. **What was built**: This is the most important input. Read `.claude/done-criteria.md` if it exists — it tells you what the feature is supposed to do. If it doesn't exist, read `git diff --stat` and `git log -1` to understand what changed. You need to know what to test before anything else.
+
+2. **Framework detection**: Read `package.json` → identify framework and start command from `scripts.dev`, `scripts.start`, or `scripts.preview`.
 
-2. **Port inference**: Check framework config files for custom ports. Defaults — Next.js: 3000, Vite: 5173, CRA: 3000, Nuxt: 3000, Astro: 4321, Angular: 4200. Override with `--port` flag if provided.
+3. **Port inference**: Defaults — Next.js: 3000, Vite: 5173, CRA: 3000, Nuxt: 3000, Astro: 4321, Angular: 4200. Override with `--port` flag.
 
-3. **Affected routes**: Run `git diff --name-only` and map changed files to routes (e.g., `app/dashboard/page.tsx` → `/dashboard`, `src/pages/about.vue` → `/about`). These are the pages that need testing.
+4. **Affected routes**: Map changed files to routes (e.g., `app/dashboard/page.tsx` → `/dashboard`).
 
-4. **Tier selection** — pick the best available browser tool:
-   - Check if `mcp__claude-in-chrome__*` tools exist in available tools → **Tier 1** (Chrome DevTools). Read `references/tier1-chrome.md`.
-   - Else check if `mcp__playwright__*` tools exist (Playwright MCP installed via `npx devlyn-cli`) OR run `npx playwright --version 2>/dev/null` → **Tier 2** (Playwright). Read `references/tier2-playwright.md`.
+5. **Tier selection** — pick the best available browser tool:
+   - Check if `mcp__claude-in-chrome__*` tools exist → **Tier 1** (Chrome DevTools). Read `references/tier1-chrome.md`.
+   - Else check if `mcp__playwright__*` tools exist or `npx playwright --version` succeeds → **Tier 2** (Playwright). Read `references/tier2-playwright.md`.
    - Else → **Tier 3** (HTTP smoke). Read `references/tier3-curl.md`.
 
-5. **Skip gate**: If no web-relevant files changed (no `*.tsx`, `*.jsx`, `*.vue`, `*.svelte`, `*.astro`, `*.css`, `*.scss`, `*.html`, `page.*`, `layout.*`, `route.*`, `+page.*`, `+layout.*`), skip the entire phase. Report: "Browser validation skipped — no web changes detected."
+6. **Skip gate**: If no web-relevant files changed (no `*.tsx`, `*.jsx`, `*.vue`, `*.svelte`, `*.astro`, `*.css`, `*.scss`, `*.html`, `page.*`, `layout.*`, `route.*`, `+page.*`, `+layout.*`), skip. Report: "Browser validation skipped — no web changes detected."
 
-6. **Parse flags** from `<config>`:
-   - `--skip-flow` — skip flow testing, only run smoke + visual
+7. **Parse flags** from `<config>`:
+   - `--skip-feature` — skip feature testing, only run smoke + visual
    - `--port PORT` — override detected port
    - `--tier N` — force a specific tier (1, 2, or 3)
-   - `--mobile-only` — only test mobile viewport
-   - `--desktop-only` — only test desktop viewport
+   - `--mobile-only` / `--desktop-only` — limit viewport testing
 
 Announce:
 ```
 Browser validation starting
+Feature: [what was built, from done-criteria or git diff]
 Framework: [detected] | Port: [PORT] | Tier: [N — name]
-Affected routes: [list]
-Phases: Smoke → [Flow] → Visual → Report
+Phases: Server → Smoke → Feature Test → Visual → Report
 ```
 
 ## PHASE 2: SERVER
 
-1. Start the dev server in background: run the detected start command via Bash with `run_in_background: true`.
-2. Health-check loop: poll `http://localhost:PORT` every 2 seconds using `curl -s -o /dev/null -w "%{http_code}"`. Timeout after 20 seconds.
-3. If the server fails to start, capture stderr output and report as BLOCKED:
-   ```
-   Verdict: BLOCKED
-   Reason: Dev server failed to start within 20s
-   Error: [stderr output]
-   ```
-   Write this to `.claude/BROWSER-RESULTS.md` and stop.
-4. Record the server PID for cleanup.
+Get the dev server running. If it doesn't start, diagnose and fix — don't just report failure.
+
+1. Start the dev server in background via Bash with `run_in_background: true`.
+2. Health-check: poll `http://localhost:PORT` every 2s, timeout 30s. Ready when you get an HTTP response.
+3. **If it doesn't come up — troubleshoot** (up to 2 attempts): read stderr for the error, fix it (npm install, port conflict, build error, etc.), restart, re-check.
+4. If still down after 2 attempts: write BLOCKED verdict and stop.
 
-## PHASE 3: SMOKE
+## PHASE 3: SMOKE (quick prerequisite)
 
-Test that the app renders and the runtime is clean. Follow the tier-specific reference file for exact tool calls.
+Quick check that the app is alive. This is not the main test — it's a gate to make sure feature testing is even possible.
 
-For each affected route (and always `/` as the first):
-1. Navigate to the page
-2. Verify the page has meaningful content (not a blank page, not a raw error)
-3. Capture console messages — filter for errors (ignore React dev-mode warnings, HMR noise, favicon 404s)
-4. Capture network requests — flag any 4xx/5xx responses or CORS failures (ignore HMR websocket, source maps)
-5. Take a screenshot as evidence
+Navigate to `/` and each affected route. For each page, judge: is this the actual application, or an error page? A connection error, framework error overlay, or blank shell is not the app. If broken, try to fix (read console errors, fix source, let hot-reload pick it up). Up to 2 fix attempts per route.
 
-A route **fails smoke** if: the page is blank, shows an unhandled error, has console errors (excluding known dev noise), or has failed network requests to the app's own API.
+If the app isn't rendering, the verdict is BLOCKED — feature testing can't happen.
 
-## PHASE 4: FLOW (conditional)
+## PHASE 4: FEATURE TEST (the main event)
 
-Skip if `--skip-flow` is set or if `.claude/done-criteria.md` doesn't exist.
+This is the primary purpose of browser validation. Everything else is in service of getting here.
 
-Read `references/flow-testing.md` for how to convert done-criteria into browser test steps. Then execute each test step using the tier-specific tools.
+Read `.claude/done-criteria.md` (or infer from git diff what was built). For each criterion that describes something a user can do or see in the UI, test it end-to-end in the browser:
 
-For each flow test:
-1. Execute the action sequence (navigate → find → interact → verify)
-2. After each interaction, check console + network for new errors
-3. Screenshot at each verification point
-4. Record pass/fail with evidence
+1. **Plan the test**: What would a user do to verify this feature works? Navigate where, click what, type what, expect what result?
+2. **Execute it**: Navigate to the page, find the interactive elements, perform the actions, verify the outcome. Read `references/flow-testing.md` for patterns on converting criteria to browser steps.
+3. **Capture evidence**: Screenshot at each key step. Record console errors and network failures that happen during the interaction.
+4. **If it fails — try to fix**: Read the error (console, network, or the UI state) to understand why the feature broke. Fix the source code, let hot-reload update, and re-test. Up to 2 fix attempts per criterion.
+5. **Record the result**: For each criterion — PASS (feature works as specified), FAIL (feature doesn't work, include what went wrong), or SKIPPED (criterion isn't browser-testable, e.g., "API returns 401").
 
-## PHASE 5: VISUAL
+The verdict depends primarily on this phase. If the implemented features don't work in the browser, the validation fails — even if every page renders perfectly and the layout looks great.
 
-Test layout integrity at two viewports (skip one if `--mobile-only` or `--desktop-only` is set):
+## PHASE 5: VISUAL (supporting check)
 
-1. **Mobile** (375x812): resize → navigate to each affected route → screenshot → check for overflow, overlapping elements, unreadable text
-2. **Desktop** (1280x800): resize → navigate to each affected route → screenshot → check for broken layouts, missing sections
+Quick layout check at two viewports (skip if `--mobile-only` or `--desktop-only`):
 
-This is judgment-based — the agent looks at screenshots and reports visible issues. Not pixel-diff.
+1. **Mobile** (375x812): screenshot each affected route, check for overflow/overlap/unreadable text
+2. **Desktop** (1280x800): screenshot each affected route, check for broken layouts
+
+Judgment-based — look at the screenshots and report visible issues.
 
 ## PHASE 6: REPORT
 
@@ -96,23 +92,24 @@ Write `.claude/BROWSER-RESULTS.md`:
 # Browser Validation Results
 
 ## Verdict: [PASS / PASS WITH ISSUES / NEEDS WORK / BLOCKED]
-Verdict rules: BLOCKED = app won't start or root page crashes. NEEDS WORK = flow tests fail or console errors on affected routes. PASS WITH ISSUES = visual issues or minor warnings. PASS = clean across all checks.
+Verdict rules:
+- BLOCKED = server won't start or app doesn't render
+- NEEDS WORK = implemented features don't work in the browser (this is the primary failure mode)
+- PASS WITH ISSUES = features work but visual issues or minor warnings exist
+- PASS = features verified working, pages render, layout clean
 
-## Environment
-- Framework: [detected]
-- Dev server: [command] on port [PORT]
-- Browser tier: [1/2/3 — name]
-- Startup time: [N]s
+## What Was Tested
+[Brief description of the feature/task from done-criteria or git diff]
 
-## Smoke Test
-| Route | Renders | Console Errors | Network Failures | Screenshot |
-|-------|---------|---------------|-----------------|------------|
-| / | YES/NO | [count]: [details] | [count]: [details] | [path] |
+## Feature Verification (primary)
+| Criterion | Test Steps | Result | Evidence |
+|-----------|-----------|--------|----------|
+| [what should work] | [what you did] | PASS/FAIL/SKIPPED | [screenshot, errors, what went wrong] |
 
-## Flow Tests
-| Criterion | Steps | Result | Evidence |
-|-----------|-------|--------|----------|
-| [text] | [N] | PASS/FAIL | [screenshot, errors] |
+## Smoke Test (prerequisite)
+| Route | Renders | Console Errors | Network Failures |
+|-------|---------|---------------|-----------------|
+| / | YES/NO | [count] | [count] |
 
 ## Visual Check
 | Viewport | Route | Issues |
@@ -120,15 +117,18 @@ Verdict rules: BLOCKED = app won't start or root page crashes. NEEDS WORK = flow
 | Mobile (375px) | / | [issues or "Clean"] |
 | Desktop (1280px) | / | [issues or "Clean"] |
 
-## Runtime Errors (full log)
-[all unique console errors, deduplicated]
+## Fixes Applied During Validation
+[List any bugs found and fixed during testing — server startup issues, broken routes, feature bugs]
+
+## Runtime Errors
+[Console errors captured during testing]
 
 ## Failed Network Requests
-[all failed requests with URL, status, and context]
+[Failed API calls captured during testing]
 ```
 
 ## PHASE 7: CLEANUP
 
-Kill the dev server process using the stored PID. If running inside auto-resolve pipeline and the `--keep-server` flag was passed (set by the pipeline orchestrator), skip cleanup — the pipeline will handle it.
+Kill the dev server PID. If `--keep-server` was passed (auto-resolve pipeline), skip — the pipeline handles cleanup.
 
 </workflow>

package/config/skills/devlyn:browser-validate/references/tier1-chrome.md

@@ -41,7 +41,7 @@ After navigating, wait 2-3 seconds for client-side rendering, then call `get_pag
 ```
 get_page_text → extract visible text content
 ```
-Page fails if: text is empty, contains only boilerplate (e.g., just "Loading..."), or contains raw error stack traces.
+Read the text and judge: is this the actual application, or an error/fallback page? Browser error pages, framework error overlays, "Unable to connect" screens, and empty shells all have text — but they're not the app. If the page content doesn't look like what the application is supposed to show, it's a failure.
 
 ### Read page structure
 ```

package/config/skills/devlyn:browser-validate/references/tier2-playwright.md

@@ -70,10 +70,24 @@ test.describe('Smoke Tests', () => {
         }
       });
 
+      // If goto throws (connection refused), the test fails — that's correct behavior
       await page.goto(`http://localhost:${PORT}${route}`, { waitUntil: 'networkidle', timeout: 15000 });
 
-      const bodyText = await page.textContent('body');
-      expect(bodyText?.trim().length).toBeGreaterThan(0);
+      // Verify this is the actual application, not an error page.
+      // When a server is down or a route is broken, the browser shows an error page
+      // that still has text content — "Unable to connect", "This site can't be reached", etc.
+      // A naive length check would pass on these. The title is the best signal:
+      // browser error pages have titles like "Problem loading page" or the URL itself,
+      // while real apps have meaningful titles set by the application.
+      const title = await page.title();
+      const bodyText = await page.textContent('body') || '';
+
+      // Page must have substantive content
+      expect(bodyText.trim().length, 'Page body is empty').toBeGreaterThan(0);
+
+      // Fail if the page navigation itself failed (Playwright sets title to the URL on error)
+      const pageUrl = page.url();
+      expect(title, 'Page shows a browser error — server may be down').not.toBe(pageUrl);
 
       await page.screenshot({ path: `.claude/screenshots/smoke${route.replace(/\//g, '-') || '-root'}.png`, fullPage: true });
 

package/config/skills/devlyn:browser-validate/references/tier3-curl.md

@@ -37,8 +37,9 @@ HTML=$(curl -s http://localhost:{PORT}{route} --max-time 10)
 ### Pass Criteria
 
 A route passes if:
-1. `STATUS` is `200` (or `304`)
-2. HTML contains `<body` tag
+1. curl succeeds (doesn't error out with connection refused or timeout)
+2. `STATUS` is `200` (or `301`, `302`, `304`) — not `000`, not `5xx`
+3. HTML contains `<body` tag
 3. HTML body has more than 100 characters of text content (not just empty divs)
 4. HTML does not contain server error indicators: `Internal Server Error`, `500`, `ECONNREFUSED`, `Cannot GET`, `404`
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "devlyn-cli",
-  "version": "1.3.0",
+  "version": "1.3.1",
   "description": "Claude Code configuration toolkit for teams",
   "bin": {
     "devlyn": "bin/devlyn.js"