ccqa 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kazuki Shibutani
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,278 @@
+# ccqa
+
+**Your Claude subscription already includes a QA engineer.**
+
+ccqa turns Claude Code into a browser test recorder.
+
+Write a spec in Markdown, run `ccqa trace`, and Claude drives your app via [agent-browser](https://github.com/vercel-labs/agent-browser) — a lightweight headless browser CLI that runs anywhere without a browser driver or Playwright setup. Because the agent controls the browser through a simple CLI interface, it can handle login flows, intermediate screens, and dynamic UI the same way a human would.
+
+Every action is recorded as structured data and compiled into a deterministic test script you can run in CI. No extra API key. Just `claude`.
+
+## How it works
+
+```mermaid
+flowchart LR
+    A["Write spec\n(test-spec.md)"] --> B["ccqa trace\n(Claude drives browser)"]
+    B --> C["ccqa generate\n(LLM → test script)"]
+    C --> D["ccqa run\n(deterministic replay)"]
+```
+
+`trace` invokes Claude Code with your spec. Claude drives the browser step by step via [agent-browser](https://github.com/vercel-labs/agent-browser), recording every action as structured data. `generate` compiles that data into a vitest-compatible script. `run` replays it deterministically — no LLM involved.
+
+## Install
+
+```bash
+bunx ccqa trace tasks/create-and-complete
+```
+
+Or install globally:
+
+```bash
+bun add -g ccqa
+```
+
+Requires [Claude Code](https://docs.anthropic.com/en/docs/claude-code) and [agent-browser](https://github.com/vercel-labs/agent-browser) installed globally:
+
+```bash
+npm install -g @anthropic-ai/claude-code
+bun add -g agent-browser
+```
+
+## Usage
+
+**1. Write a spec**
+
+```markdown
+<!-- .ccqa/features/tasks/test-cases/create-and-complete/test-spec.md -->
+---
+title: Create a task and mark it complete
+baseUrl: http://localhost:3000
+---
+
+## Steps
+
+### Step 1: Log in
+- **Instruction**: Fill in email and password, submit the form
+- **Expected**: Redirected to /dashboard, user avatar visible in the header
+
+### Step 2: Create a new task
+- **Instruction**: Click "New Task", fill in the title "Fix login bug", set priority to High, save
+- **Expected**: Task appears in the task list with status "Open"
+
+### Step 3: Mark the task as complete
+- **Instruction**: Open the task "Fix login bug", click "Mark as complete"
+- **Expected**: Task status changes to "Done", task moves to the completed section
+```
+
+**2. Trace — Claude drives the browser and records every action**
+
+```bash
+ccqa trace tasks/create-and-complete
+```
+
+```
+▶ trace  tasks/create-and-complete
+  spec    Create a task and mark it complete
+  url     http://localhost:3000
+  steps   3
+
+Running agent-browser session...
+  ● step-01  Log in
+  ● step-02  Create a new task
+  ● step-03  Mark the task as complete
+
+  trace   .ccqa/features/tasks/test-cases/create-and-complete/actions.json
+  actions 24
+  status  PASSED
+```
+
+**3. Generate — convert recorded actions into a replayable test**
+
+```bash
+ccqa generate tasks/create-and-complete
+```
+
+**4. Run — replay deterministically, no LLM involved**
+
+```bash
+ccqa run tasks/create-and-complete
+```
+
+## Setup Specs — Reusable shared procedures
+
+Setup specs let you define reusable procedures (login, data preparation, etc.) that run before your test steps. Define once, use across multiple test specs.
+
+### 1. Write a setup spec
+
+```markdown
+<!-- .ccqa/setups/login/setup-spec.md -->
+---
+title: "Login"
+placeholders:
+  loginUrl:
+    dummy: "http://localhost:3000/login"
+    description: "Login page URL"
+  email:
+    dummy: "user@example.com"
+    description: "Email address"
+  password:
+    dummy: "secret"
+    description: "Password"
+---
+
+## Steps
+
+### Step 1: Open login page
+- **Instruction**: Navigate to {{loginUrl}}
+- **Expected**: Login form is displayed
+
+### Step 2: Enter credentials and log in
+- **Instruction**: Enter email {{email}} and password {{password}}, then submit
+- **Expected**: Login succeeds
+```
+
+The `placeholders` section defines variables with `dummy` values. During `trace-setup`, the dummy values are used for actual browser operation. During `generate-setup`, they are reverse-replaced with `{{key}}` placeholders.
+
+### 2. Trace the setup
+
+```bash
+ccqa trace-setup login
+```
+
+### 3. Generate and validate the setup
+
+```bash
+ccqa generate-setup login
+```
+
+This generates `test.dummy.spec.ts` with dummy values, runs vitest to validate, and applies auto-fix. On success, it reverse-replaces dummy values with placeholders and saves `test.spec.ts`.
+
+If auto-fix fails, edit `test.dummy.spec.ts` manually and re-run:
+
+```bash
+ccqa generate-setup login --from-dummy
+```
+
+### 4. Reference from test specs
+
+```markdown
+---
+title: Create a task
+baseUrl: http://localhost:3000
+setups:
+  - name: login
+    params:
+      loginUrl: "http://localhost:3000/login"
+      email: "admin@example.com"
+      password: "AdminPass123"
+---
+
+## Steps
+### Step 1: Create a new task
+...
+```
+
+When you run `ccqa trace` or `ccqa generate`, the setup's test body is loaded, placeholders are replaced with `params` values, and it runs before your test steps — sharing the same browser session.
+
+## What gets generated
+
+`ab()` is a thin wrapper around [agent-browser](https://github.com/vercel-labs/agent-browser) — a headless browser CLI. Each call spawns `agent-browser <command>` as a subprocess and throws if it exits non-zero. No browser driver setup, no async/await, no `.waitFor()`.
+
+```typescript
+// .ccqa/features/tasks/test-cases/create-and-complete/test.spec.ts
+import { test } from "vitest";
+import { ab, abWait, abAssertUrl, abAssertTextVisible, abAssertEnabled } from "/path/to/test-helpers.ts";
+
+process.env.AGENT_BROWSER_SESSION = `ccqa-run-${Date.now()}`;
+
+test("setup: login", () => {
+  ab("cookies", "clear");
+  ab("open", "http://localhost:3000/login");
+  ab("fill", "[placeholder='Email']", "admin@example.com");
+  ab("fill", "[type='password']", "AdminPass123");
+  ab("press", "Enter");
+}, 3 * 60 * 1000);
+
+test("Create a task", () => {
+  ab("open", "http://localhost:3000");
+
+  // Create a new task
+  ab("click", "[aria-label='New Task']");
+  ab("fill", "[placeholder='Task title']", "Fix login bug");
+  ab("select", "[aria-label='Priority']", "High");
+  ab("click", "[aria-label='Save']");
+  abAssertTextVisible("Fix login bug");
+  abAssertTextVisible("Open");
+}, 5 * 60 * 1000);
+```
+
+Setup and test share the same `AGENT_BROWSER_SESSION` — login state carries over. Each run starts with `cookies clear` to ensure a clean session.
+
+## Assertions
+
+During `trace`, Claude verifies each step with at least two independent signals and emits structured assertions. These become typed helper calls in the generated script:
+
+| Assert | What it checks |
+|--------|---------------|
+| `abAssertTextVisible(text)` | Text appears on page (waits up to 30s) |
+| `abAssertUrl(pattern)` | Current URL contains pattern |
+| `abAssertEnabled(selector)` | Button/input is enabled |
+| `abAssertDisabled(selector)` | Button/input is disabled |
+| `abAssertVisible(selector)` | Element is visible |
+| `abAssertNotVisible(selector)` | Element is hidden |
+| `abAssertChecked(selector)` | Checkbox is checked |
+| `abAssertUnchecked(selector)` | Checkbox is unchecked |
+
+Assertions are stability-aware: Claude skips timestamps, session IDs, and exact counts that vary between runs.
+
+## Auto-fix
+
+If the generated script fails (timing issues, page not ready), `generate` uses an LLM to analyze the failure log and insert `sleep` at the right positions. Control how many attempts with `--max-retries`:
+
+```bash
+ccqa generate tasks/create-and-complete --max-retries 5
+```
+
+## Commands
+
+```
+ccqa trace <feature/spec>          Record browser actions for a test spec
+ccqa generate <feature/spec>       Generate test script from recorded actions
+ccqa run [feature/spec]            Execute generated test scripts
+
+ccqa trace-setup <name>            Record browser actions for a setup spec
+ccqa generate-setup <name>         Generate and validate setup test script
+  --from-dummy                      Resume from manually edited test.dummy.spec.ts
+```
+
+## File structure
+
+```
+.ccqa/
+  setups/
+    login/
+      setup-spec.md              # Setup definition with placeholders
+      test.spec.ts               # Generated setup script (with {{placeholders}})
+  features/
+    tasks/
+      test-cases/
+        create-and-complete/
+          test-spec.md           # Test definition (references setups)
+          actions.json           # Recorded actions from trace
+          test.spec.ts           # Generated test script
+```
+
+## Why not write Playwright tests by hand?
+
+| | ccqa | Hand-written Playwright |
+|---|---|---|
+| Write selectors | Claude picks them from ARIA snapshots | You inspect the DOM |
+| Handle timing | Recorded wait commands, auto-fix sleep | `waitFor`, `expect().toBeVisible()` |
+| Assertions | Auto-generated from verified signals | Written manually |
+| Login / setup | Shared setup specs with placeholders | Custom fixtures per project |
+| Update after UI change | Re-run `trace` | Find and update every affected locator |
+| Runs in CI | Yes (deterministic replay, no LLM) | Yes |
+
+## License
+
+MIT

package/bin/ccqa.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env bun
+import "../src/cli/index.ts";

package/package.json

@@ -0,0 +1,38 @@
+{
+  "name": "ccqa",
+  "version": "0.1.0",
+  "type": "module",
+  "description": "Browser test recorder powered by Claude Code and agent-browser",
+  "bin": {
+    "ccqa": "./bin/ccqa.ts"
+  },
+  "files": [
+    "bin/",
+    "src/"
+  ],
+  "dependencies": {
+    "@anthropic-ai/claude-agent-sdk": "^0.2.87",
+    "@anthropic-ai/claude-code": "^2.1.87",
+    "commander": "^14.0.3",
+    "gray-matter": "^4.0.3",
+    "zod": "^4.3.6"
+  },
+  "peerDependencies": {
+    "vitest": ">=2.0.0",
+    "agent-browser": "*"
+  },
+  "peerDependenciesMeta": {
+    "agent-browser": {
+      "optional": false
+    },
+    "vitest": {
+      "optional": false
+    }
+  },
+  "devDependencies": {
+    "@types/bun": "latest"
+  },
+  "trustedDependencies": [
+    "agent-browser"
+  ]
+}

package/src/claude/invoke.test.ts

@@ -0,0 +1,167 @@
+import { describe, test, expect } from "bun:test";
+import { extractAbActionFromBashCommand, isBlockedAbSubcommand, hasRefSelector, shellTokenize } from "./invoke.ts";
+
+describe("extractAbActionFromBashCommand", () => {
+  test("returns null for non-agent-browser commands", () => {
+    expect(extractAbActionFromBashCommand("ls -la")).toBeNull();
+    expect(extractAbActionFromBashCommand("echo hello")).toBeNull();
+  });
+
+  test("parses cookies clear", () => {
+    expect(
+      extractAbActionFromBashCommand("agent-browser --session s1 cookies clear"),
+    ).toBe("AB_ACTION|cookies_clear");
+  });
+
+  test("parses open", () => {
+    expect(
+      extractAbActionFromBashCommand("agent-browser --session s1 open https://example.com"),
+    ).toBe("AB_ACTION|open|https://example.com");
+  });
+
+  test("parses click with quoted selector", () => {
+    expect(
+      extractAbActionFromBashCommand(`agent-browser --session s1 click "[aria-label='Submit']" "Submit"`),
+    ).toBe("AB_ACTION|click|[aria-label='Submit']|Submit");
+  });
+
+  test("parses fill", () => {
+    expect(
+      extractAbActionFromBashCommand(`agent-browser --session s1 fill "[placeholder='Email']" "test@example.com" "Email"`),
+    ).toBe("AB_ACTION|fill|[placeholder='Email']|test@example.com|Email");
+  });
+
+  test("parses snapshot as null", () => {
+    expect(
+      extractAbActionFromBashCommand("agent-browser --session s1 snapshot"),
+    ).toBeNull();
+  });
+
+  test("parses press", () => {
+    expect(
+      extractAbActionFromBashCommand("agent-browser --session s1 press Enter"),
+    ).toBe("AB_ACTION|press|Enter");
+  });
+
+  test("parses wait", () => {
+    expect(
+      extractAbActionFromBashCommand(`agent-browser --session s1 wait --text "Done"`),
+    ).toBe("AB_ACTION|wait|--text|Done");
+  });
+});
+
+describe("isBlockedAbSubcommand", () => {
+  test("blocks eval with session", () => {
+    expect(isBlockedAbSubcommand(`agent-browser --session s1 eval "document.querySelector('.btn').click()"`)).toBe(true);
+  });
+
+  test("blocks js with session", () => {
+    expect(isBlockedAbSubcommand(`agent-browser --session s1 js "window.scrollTo(0, 100)"`)).toBe(true);
+  });
+
+  test("blocks eval without session flag", () => {
+    expect(isBlockedAbSubcommand(`agent-browser eval "document.click()"`)).toBe(true);
+  });
+
+  test("does not block click", () => {
+    expect(isBlockedAbSubcommand(`agent-browser --session s1 click "[aria-label='Submit']"`)).toBe(false);
+  });
+
+  test("does not block snapshot", () => {
+    expect(isBlockedAbSubcommand("agent-browser --session s1 snapshot")).toBe(false);
+  });
+
+  test("does not block fill", () => {
+    expect(isBlockedAbSubcommand(`agent-browser --session s1 fill "[placeholder='Email']" "test@example.com"`)).toBe(false);
+  });
+
+  test("blocks find command", () => {
+    expect(isBlockedAbSubcommand(`agent-browser --session s1 find text "Select category" click`)).toBe(true);
+  });
+
+  test("blocks label command", () => {
+    expect(isBlockedAbSubcommand(`agent-browser --session s1 label "Settings" click`)).toBe(true);
+  });
+
+  test("blocks textbox command", () => {
+    expect(isBlockedAbSubcommand(`agent-browser --session s1 textbox "Search"`)).toBe(true);
+  });
+
+  test("does not block non-agent-browser commands", () => {
+    expect(isBlockedAbSubcommand("ls -la")).toBe(false);
+    expect(isBlockedAbSubcommand("echo hello")).toBe(false);
+  });
+});
+
+describe("hasRefSelector", () => {
+  test("detects @ref in click", () => {
+    expect(hasRefSelector(`agent-browser --session s1 click "@e14"`)).toBe(true);
+  });
+
+  test("detects @ref in check", () => {
+    expect(hasRefSelector(`agent-browser --session s1 check @e7`)).toBe(true);
+  });
+
+  test("detects @ref in fill", () => {
+    expect(hasRefSelector(`agent-browser --session s1 fill "@e6" "value"`)).toBe(true);
+  });
+
+  test("does not flag aria-label selector", () => {
+    expect(hasRefSelector(`agent-browser --session s1 click "[aria-label='Submit']"`)).toBe(false);
+  });
+
+  test("does not flag text= selector", () => {
+    expect(hasRefSelector(`agent-browser --session s1 click "text=Select category"`)).toBe(false);
+  });
+
+  test("does not flag non-agent-browser commands", () => {
+    expect(hasRefSelector("ls -la")).toBe(false);
+  });
+
+  test("does not flag snapshot (no args)", () => {
+    expect(hasRefSelector("agent-browser --session s1 snapshot")).toBe(false);
+  });
+});
+
+describe("shellTokenize", () => {
+  test("splits simple tokens", () => {
+    expect(shellTokenize("click foo bar")).toEqual(["click", "foo", "bar"]);
+  });
+
+  test("preserves spaces inside double quotes", () => {
+    expect(shellTokenize(`click "[role='dialog'] button:last-child"`)).toEqual([
+      "click",
+      "[role='dialog'] button:last-child",
+    ]);
+  });
+
+  test("preserves spaces inside single quotes", () => {
+    expect(shellTokenize("fill 'text=hello world' value")).toEqual([
+      "fill",
+      "text=hello world",
+      "value",
+    ]);
+  });
+
+  test("handles empty string", () => {
+    expect(shellTokenize("")).toEqual([]);
+  });
+
+  test("handles multiple spaces", () => {
+    expect(shellTokenize("click  foo")).toEqual(["click", "foo"]);
+  });
+});
+
+describe("extractAbActionFromBashCommand with compound selectors", () => {
+  test("parses click with compound selector containing spaces", () => {
+    expect(
+      extractAbActionFromBashCommand(`agent-browser --session s1 click "[role='dialog'] button:last-child"`),
+    ).toBe("AB_ACTION|click|[role='dialog'] button:last-child|");
+  });
+
+  test("parses fill with placeholder containing spaces", () => {
+    expect(
+      extractAbActionFromBashCommand(`agent-browser --session s1 fill ".modal-footer button" "OK"`),
+    ).toBe("AB_ACTION|fill|.modal-footer button|OK|");
+  });
+});