ccqa 0.3.7 → 0.3.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -45,7 +45,7 @@ pnpm add -D agent-browser
45
45
 
46
46
  ## Usage
47
47
 
48
- **1. Write a spec**
48
+ **1. Write a spec** — by hand, or interactively with [`ccqa draft`](#draft--co-author-test-specmd-with-claude)
49
49
 
50
50
  ```markdown
51
51
  <!-- .ccqa/features/tasks/test-cases/create-and-complete/test-spec.md -->
@@ -103,6 +103,74 @@ ccqa generate tasks/create-and-complete
103
103
  ccqa run tasks/create-and-complete
104
104
  ```
105
105
 
106
+ ## Draft — co-author test-spec.md with Claude
107
+
108
+ Writing a `test-spec.md` from scratch means digging into your codebase to find the right aria-labels, URLs, and button text. `ccqa draft` puts Claude in the loop: you describe what you want to test in plain language, Claude reads the relevant code, and you refine the spec interactively.
109
+
110
+ ```bash
111
+ ccqa draft
112
+ ```
113
+
114
+ The first run asks for your intent, proposes a `feature/spec` name, and writes a draft. Each subsequent invocation lets you give a refinement instruction — empty input means "just re-check the current spec against the code." Press `y` at the final "Are you done with this draft?" prompt to end the session.
115
+
116
+ ```
117
+ ccqa draft
118
+
119
+ What do you want to test? > Select a category on the AI Maintenance page and run a check
120
+ Proposing a feature/spec name based on your intent...
121
+ proposed: ai-maintenance/run-check-with-category
122
+ Use this name? [y/N/edit] > y
123
+
124
+ Reading codebase and drafting spec...
125
+ ✓ 5 Read, 3 Grep, 2 Glob (4.2s)
126
+
127
+ ── Review (1 warning, 3 passed) ───────────────────────────────────
128
+
129
+ WARNINGS (1)
130
+ Assertability step-05
131
+ Result row may still show "running" right after the click
132
+ └ ContentQualityCheck.tsx polls every 5s; the status starts at
133
+ IN_PROGRESS and only flips to SUCCEEDED later.
134
+
135
+ PASSED (3)
136
+ Setup references, Step granularity, Unimplemented checks
137
+
138
+ ────────────────────────────────────────────────────────────────────
139
+
140
+ --- proposed changes ---
141
+ + ---
142
+ + title: "AI Maintenance — content quality check"
143
+ ...
144
+
145
+ Apply this patch? [y/N] y
146
+ saved: .ccqa/features/ai-maintenance/test-cases/run-check-with-category/test-spec.md
147
+
148
+ How would you like to refine? (empty = re-validate) >
149
+ ```
150
+
151
+ You can also edit `test-spec.md` directly in your editor between turns — `ccqa draft` re-reads the file each iteration.
152
+
153
+ ### What gets reviewed
154
+
155
+ Every turn Claude grades the spec on four axes and reports issues:
156
+
157
+ | Check | What it verifies |
158
+ |---|---|
159
+ | **Assertability** | Each step's **Expected** references concrete, observable signals (visible text, URL pattern, element state) that actually exist in the code. Flags timestamps, exact counts, and session-specific values that won't be stable across runs. |
160
+ | **Setup references** | Every `setups[].name` in the frontmatter resolves to an existing `.ccqa/setups/<name>/setup-spec.md`, and every `params` key matches that setup's `placeholders`. See [Setup Specs](#setup-specs--reusable-shared-procedures). |
161
+ | **Step granularity** | Steps aren't too coarse (multiple actions in one) or too fine (snapshot-only filler), and the order is logical. |
162
+ | **Unimplemented checks** | Anything the spec describes that Claude couldn't find in the codebase — a hint that you may be specifying behavior that doesn't exist yet. |
163
+
164
+ Findings with severity `WARN` or `ERROR` are shown in full; `OK` checks collapse to a one-line summary.
165
+
166
+ ### Flags
167
+
168
+ ```
169
+ ccqa draft [feature/spec] # arg is optional; Claude proposes a name if omitted
170
+ --instruction <text> # single-shot, non-interactive
171
+ --apply # auto-apply patches without [y/N] confirmation
172
+ ```
173
+
106
174
  ## Setup Specs — Reusable shared procedures
107
175
 
108
176
  Setup specs let you define reusable procedures (login, data preparation, etc.) that run before your test steps. Define once, use across multiple test specs.
@@ -253,6 +321,10 @@ ccqa generate tasks/create-and-complete --max-retries 5
253
321
  ## Commands
254
322
 
255
323
  ```
324
+ ccqa draft [feature/spec] Co-author a test spec with Claude
325
+ --instruction <text> Single-shot, non-interactive
326
+ --apply Auto-apply patches without [y/N] confirmation
327
+
256
328
  ccqa trace <feature/spec> Record browser actions for a test spec
257
329
  ccqa generate <feature/spec> Generate test script from recorded actions
258
330
  --auto Apply auto-fixes without confirmation (CI)
@@ -267,6 +339,8 @@ ccqa generate-setup <name> Generate and validate setup test script
267
339
  --auto / --no-interactive Same semantics as `generate`
268
340
  ```
269
341
 
342
+ All Claude-driven commands (`trace`, `trace-setup`, `generate`, `generate-setup`) accept `-m, --model <name>` to select the Claude model — pass an alias (`sonnet` | `opus` | `haiku`) or a full model ID (e.g. `claude-opus-4-7`). The flag overrides the `CCQA_MODEL` environment variable; when both are unset, the Claude Code CLI default is used. Authentication is handled by your local Claude Code login — no `ANTHROPIC_API_KEY` is required.
343
+
270
344
  `<feature/spec>` is a 2-segment alias for the on-disk path `.ccqa/features/<feature>/test-cases/<spec>/`. Pass the alias, not the full directory path.
271
345
 
272
346
  ## File structure
package/dist/bin/ccqa.mjs CHANGED
@@ -1,7 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
  import { createRequire } from "node:module";
3
3
  import { Command } from "commander";
4
- import { accessSync, readFileSync } from "node:fs";
4
+ import { accessSync, readFileSync, statSync } from "node:fs";
5
5
  import { fileURLToPath } from "node:url";
6
6
  import { access, mkdir, mkdtemp, readFile, readdir, rm, stat, unlink, writeFile } from "node:fs/promises";
7
7
  import { delimiter, dirname, join, resolve } from "node:path";
@@ -10,6 +10,8 @@ import matter from "gray-matter";
10
10
  import { spawn } from "node:child_process";
11
11
  import { createInterface } from "node:readline";
12
12
  import { tmpdir } from "node:os";
13
+ import { createInterface as createInterface$1 } from "node:readline/promises";
14
+ import { z } from "zod";
13
15
  //#region src/prompts/trace.ts
14
16
  function generateSessionName() {
15
17
  return `ccqa-trace-${(/* @__PURE__ */ new Date()).toISOString().replace(/[:.]/g, "-")}`;
@@ -106,6 +108,7 @@ For each step:
106
108
  - Do NOT retry a selector without taking a fresh snapshot first
107
109
  - Do NOT work around blockers (login walls, missing data, captchas) — stop and report
108
110
  - **Do NOT suppress errors** — never use \`2>/dev/null\`, \`|| true\`, \`; other-command\`, or any other technique that hides agent-browser failures. Each \`agent-browser\` command must run standalone so failures are properly detected and recorded.
111
+ - **If \`agent-browser\` is not found, stop immediately.** Do not run \`which\`, \`find\`, \`npm ls\`, \`npm install\`, \`npx\`, \`brew\`, or any other discovery / installation command. Do not try alternate paths. The ccqa host already validates the binary before launching you, so if you see \`command not found\` it is a host-environment problem you cannot fix from inside the test run. Emit one line and terminate: \`ASSERTION_FAILED|step-XX|agent-browser binary not available in PATH\`.
109
112
 
110
113
  ## Source Code Reference
111
114
 
@@ -339,8 +342,14 @@ async function timedPhase(label, fn, scope = "fix") {
339
342
  }
340
343
  //#endregion
341
344
  //#region src/claude/invoke.ts
345
+ function resolveModel(explicit) {
346
+ if (explicit) return explicit;
347
+ const envModel = process.env["CCQA_MODEL"];
348
+ return envModel && envModel.length > 0 ? envModel : void 0;
349
+ }
342
350
  async function invokeClaudeStreaming(options, onEvent) {
343
- const { prompt, systemPrompt, allowedTools, disableBuiltinTools = false, maxTurns, env, onAbAction, onAbActionFailed } = options;
351
+ const { prompt, systemPrompt, allowedTools, disableBuiltinTools = false, maxTurns, env, model, onAbAction, onAbActionFailed, silenceBashLog = false } = options;
352
+ const resolvedModel = resolveModel(model);
344
353
  let lastAbToolUseId = null;
345
354
  const sdkOptions = {
346
355
  systemPrompt,
@@ -348,6 +357,7 @@ async function invokeClaudeStreaming(options, onEvent) {
348
357
  allowedTools: allowedTools ?? ["Bash(*)"],
349
358
  permissionMode: "bypassPermissions",
350
359
  allowDangerouslySkipPermissions: true,
360
+ ...resolvedModel ? { model: resolvedModel } : {},
351
361
  ...env ? { env: {
352
362
  ...process.env,
353
363
  ...env
@@ -390,7 +400,7 @@ async function invokeClaudeStreaming(options, onEvent) {
390
400
  const q = await buildMessageStream(prompt, sdkOptions);
391
401
  for await (const msg of q) {
392
402
  onEvent(msg);
393
- if (msg.type === "assistant") {
403
+ if (msg.type === "assistant" && !silenceBashLog) {
394
404
  for (const block of msg.message.content ?? []) if (block.type === "tool_use" && block.name === "Bash") {
395
405
  const cmd = block.input?.["command"];
396
406
  if (typeof cmd === "string") bash(cmd);
@@ -553,6 +563,16 @@ async function readSpecFile(featureName, specName, cwd) {
553
563
  throw new Error(`Spec file not found: ${specPath}`);
554
564
  });
555
565
  }
566
+ async function tryReadSpecFile(featureName, specName, cwd) {
567
+ return readFile(join(getSpecDir(featureName, specName, cwd), "test-spec.md"), "utf-8").catch(() => null);
568
+ }
569
+ async function saveSpecFile(featureName, specName, content, cwd) {
570
+ const specDir = getSpecDir(featureName, specName, cwd);
571
+ await mkdir(specDir, { recursive: true });
572
+ const specPath = join(specDir, "test-spec.md");
573
+ await writeFile(specPath, content.endsWith("\n") ? content : content + "\n", "utf-8");
574
+ return specPath;
575
+ }
556
576
  async function saveRoute(featureName, specName, route, cwd) {
557
577
  const specDir = getSpecDir(featureName, specName, cwd);
558
578
  await mkdir(specDir, { recursive: true });
@@ -638,6 +658,34 @@ async function listAllSpecs(cwd) {
638
658
  async function listSpecsForFeature(featureName, cwd) {
639
659
  return readdir(join(getFeatureDir(featureName, cwd), "test-cases")).catch(() => []);
640
660
  }
661
+ /**
662
+ * Lists every feature/spec dir under .ccqa/features/, regardless of whether
663
+ * the spec is fully drafted yet. Used by `ccqa draft` to suggest non-colliding
664
+ * feature/spec names that fit the existing structure.
665
+ */
666
+ async function listFeatureTree(cwd) {
667
+ const featuresDir = join(getCcqaDir(cwd), "features");
668
+ const featureDirs = await readdir(featuresDir).catch(() => []);
669
+ return Promise.all(featureDirs.map(async (featureName) => {
670
+ const testCasesDir = join(featuresDir, featureName, "test-cases");
671
+ const specDirs = await readdir(testCasesDir).catch(() => []);
672
+ return {
673
+ featureName,
674
+ specs: await Promise.all(specDirs.map(async (specName) => {
675
+ const content = await readFile(join(testCasesDir, specName, "test-spec.md"), "utf-8").catch(() => null);
676
+ if (content === null) return {
677
+ specName,
678
+ hasSpecFile: false
679
+ };
680
+ return {
681
+ specName,
682
+ hasSpecFile: true,
683
+ title: content.match(/^title:\s*"?([^"\n]+)"?/m)?.[1]?.trim()
684
+ };
685
+ }))
686
+ };
687
+ }));
688
+ }
641
689
  function routeToMarkdown(route) {
642
690
  const lines = [
643
691
  "---",
@@ -827,22 +875,46 @@ function waitExit(child) {
827
875
  //#endregion
828
876
  //#region src/runtime/agent-browser-bin.ts
829
877
  const require$1 = createRequire(import.meta.url);
878
+ function hasAgentBrowserShim(dir) {
879
+ try {
880
+ statSync(join(dir, "agent-browser"));
881
+ return true;
882
+ } catch {
883
+ return false;
884
+ }
885
+ }
886
+ /**
887
+ * Walks up from `start` looking for a `node_modules/.bin/agent-browser` shim.
888
+ * Returns the .bin directory containing the shim, or null if none is found.
889
+ */
890
+ function findNodeModulesBin(start) {
891
+ let cur = start;
892
+ while (true) {
893
+ const candidate = join(cur, "node_modules", ".bin");
894
+ if (hasAgentBrowserShim(candidate)) return candidate;
895
+ const parent = dirname(cur);
896
+ if (parent === cur) return null;
897
+ cur = parent;
898
+ }
899
+ }
830
900
  /**
831
901
  * Resolves the directory containing the `agent-browser` shim that npm/pnpm
832
902
  * exposes on PATH for the peer-installed package. Used by `ccqa trace` to
833
903
  * prepend this directory to PATH so the Claude subprocess can invoke
834
904
  * `agent-browser ...` without requiring a global install.
835
905
  *
836
- * Returns null if agent-browser cannot be resolved (peer not installed).
906
+ * Returns null if agent-browser cannot be located.
837
907
  */
838
908
  function resolveAgentBrowserBinDir() {
839
- let pkgJsonPath;
909
+ const fromCwd = findNodeModulesBin(process.cwd());
910
+ if (fromCwd) return fromCwd;
911
+ const fromSelf = findNodeModulesBin(dirname(require$1.resolve("agent-browser/package.json")));
912
+ if (fromSelf) return fromSelf;
840
913
  try {
841
- pkgJsonPath = require$1.resolve("agent-browser/package.json");
842
- } catch {
843
- return null;
844
- }
845
- return join(dirname(pkgJsonPath), "node_modules", ".bin");
914
+ const candidate = join(dirname(require$1.resolve("agent-browser/package.json")), "node_modules", ".bin");
915
+ if (hasAgentBrowserShim(candidate)) return candidate;
916
+ } catch {}
917
+ return null;
846
918
  }
847
919
  /**
848
920
  * Returns a PATH string with the agent-browser shim directory prepended,
@@ -856,6 +928,48 @@ function pathWithAgentBrowserShim(currentPath) {
856
928
  if (path.split(delimiter).includes(dir)) return path;
857
929
  return dir + delimiter + path;
858
930
  }
931
+ /**
932
+ * Confirms before launching Claude that an `agent-browser` shim is reachable
933
+ * via PATH. We do this up front so a missing peer dependency fails fast with
934
+ * a clear message, instead of Claude burning tokens probing the system with
935
+ * `which`, `find`, `npm install`, etc.
936
+ *
937
+ * The `resolver` argument is for tests; production calls take no args.
938
+ */
939
+ function assertAgentBrowserAvailable(resolver = resolveAgentBrowserBinDir) {
940
+ const dir = resolver();
941
+ if (!dir) throw new AgentBrowserUnavailableError();
942
+ const shim = join(dir, "agent-browser");
943
+ try {
944
+ const s = statSync(shim);
945
+ if (!s.isFile() && !s.isSymbolicLink()) throw new AgentBrowserUnavailableError();
946
+ } catch {
947
+ throw new AgentBrowserUnavailableError();
948
+ }
949
+ return dir;
950
+ }
951
+ var AgentBrowserUnavailableError = class extends Error {
952
+ constructor() {
953
+ super("agent-browser binary not found on PATH");
954
+ this.name = "AgentBrowserUnavailableError";
955
+ }
956
+ };
957
+ /** Human-readable explanation shown to the user when the guard fires. */
958
+ function formatAgentBrowserUnavailableMessage() {
959
+ return [
960
+ "agent-browser is not installed or not on PATH.",
961
+ "",
962
+ "ccqa drives the browser via the peer-installed `agent-browser` package.",
963
+ "Install it in this project:",
964
+ "",
965
+ " pnpm add -D agent-browser",
966
+ " # or",
967
+ " npm install -D agent-browser",
968
+ "",
969
+ "If it is already installed, make sure you are running ccqa from the",
970
+ "project root (or via your package runner, e.g. `pnpm exec ccqa ...`)."
971
+ ].join("\n");
972
+ }
859
973
  //#endregion
860
974
  //#region src/runtime/env-vars.ts
861
975
  const ENV_VAR_RE = /\$\{([A-Z_][A-Z0-9_]*)\}|\$([A-Z_][A-Z0-9_]*)/g;
@@ -908,12 +1022,21 @@ function envRefsToJsExpression(value) {
908
1022
  }
909
1023
  //#endregion
910
1024
  //#region src/cli/trace.ts
911
- const traceCommand = new Command("trace").argument("<feature/spec>", "Spec id in '<feature>/<spec>' form (resolves to .ccqa/features/<feature>/test-cases/<spec>/)").description("Run agent-browser, verify assertions, and record structured actions").action(async (specPath) => {
1025
+ const traceCommand = new Command("trace").argument("<feature/spec>", "Spec id in '<feature>/<spec>' form (resolves to .ccqa/features/<feature>/test-cases/<spec>/)").description("Run agent-browser, verify assertions, and record structured actions").option("-m, --model <name>", "Claude model alias ('sonnet'|'opus'|'haiku') or full ID. Overrides CCQA_MODEL.").action(async (specPath, opts) => {
912
1026
  const { featureName, specName } = parseSpecPath(specPath);
913
- await runTrace(featureName, specName);
1027
+ await runTrace(featureName, specName, opts.model);
914
1028
  });
915
- async function runTrace(featureName, specName) {
1029
+ async function runTrace(featureName, specName, model) {
916
1030
  header("trace", `${featureName}/${specName}`);
1031
+ try {
1032
+ meta("agent-browser", assertAgentBrowserAvailable());
1033
+ } catch (e) {
1034
+ if (e instanceof AgentBrowserUnavailableError) {
1035
+ error(formatAgentBrowserUnavailableMessage());
1036
+ process.exit(1);
1037
+ }
1038
+ throw e;
1039
+ }
917
1040
  await ensureCcqaDir();
918
1041
  const spec = parseTestSpec(await readSpecFile(featureName, specName));
919
1042
  const hasSetups = (spec.setups?.length ?? 0) > 0;
@@ -951,6 +1074,7 @@ async function runTrace(featureName, specName) {
951
1074
  AGENT_BROWSER_SESSION: sessionName,
952
1075
  PATH: pathWithAgentBrowserShim(process.env["PATH"])
953
1076
  },
1077
+ model,
954
1078
  onAbAction: (abAction) => {
955
1079
  const action = parseAbAction(abAction);
956
1080
  if (action) traceActions.push(action);
@@ -1516,7 +1640,7 @@ ${snapshot}
1516
1640
  }
1517
1641
  //#endregion
1518
1642
  //#region src/diagnose/diagnose.ts
1519
- async function diagnose(input) {
1643
+ async function diagnose(input, options = {}) {
1520
1644
  const { result: raw, isError } = await invokeClaudeStreaming({
1521
1645
  prompt: buildDiagnosePrompt(input),
1522
1646
  allowedTools: [
@@ -1524,7 +1648,8 @@ async function diagnose(input) {
1524
1648
  "Grep",
1525
1649
  "Glob"
1526
1650
  ],
1527
- maxTurns: 10
1651
+ maxTurns: 10,
1652
+ model: options.model
1528
1653
  }, () => {});
1529
1654
  if (isError) return {
1530
1655
  result: null,
@@ -1558,7 +1683,7 @@ async function diagnose(input) {
1558
1683
  reason: "diagnose returned no parseable diagnosis JSON"
1559
1684
  },
1560
1685
  confidence: 0,
1561
- reasoning: truncate$1(raw, 1e3)
1686
+ reasoning: truncate$2(raw, 1e3)
1562
1687
  },
1563
1688
  raw,
1564
1689
  sdkError: false
@@ -1615,7 +1740,7 @@ function extractJsonCandidates(raw) {
1615
1740
  }
1616
1741
  return out;
1617
1742
  }
1618
- function truncate$1(s, max) {
1743
+ function truncate$2(s, max) {
1619
1744
  return s.length <= max ? s : `${s.slice(0, max)}... [truncated, ${s.length - max} more chars]`;
1620
1745
  }
1621
1746
  function stripFence(raw) {
@@ -1843,11 +1968,11 @@ async function captureSnapshot(sessionName) {
1843
1968
  resolve(null);
1844
1969
  return;
1845
1970
  }
1846
- resolve(truncate(trimmed, MAX_OUTPUT_BYTES));
1971
+ resolve(truncate$1(trimmed, MAX_OUTPUT_BYTES));
1847
1972
  });
1848
1973
  });
1849
1974
  }
1850
- function truncate(s, maxBytes) {
1975
+ function truncate$1(s, maxBytes) {
1851
1976
  if (s.length <= maxBytes) return s;
1852
1977
  return `${s.slice(0, maxBytes)}\n... [truncated, ${s.length - maxBytes} more chars]`;
1853
1978
  }
@@ -1889,7 +2014,7 @@ const DEFAULT_CONFIDENCE_THRESHOLD = .8;
1889
2014
  * or the diagnose loop chose to bail out early.
1890
2015
  */
1891
2016
  async function runAutoFixLoop(input) {
1892
- const { scriptPath, initialRun, specMarkdown, actions, maxRetries, mode, runVitest, agentBrowserSession, outputLanguage } = input;
2017
+ const { scriptPath, initialRun, specMarkdown, actions, maxRetries, mode, runVitest, agentBrowserSession, outputLanguage, model } = input;
1893
2018
  let { exitCode, output, currentScript } = initialRun;
1894
2019
  if (exitCode === 0) return true;
1895
2020
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
@@ -1905,7 +2030,8 @@ async function runAutoFixLoop(input) {
1905
2030
  failureLog: output,
1906
2031
  pageSnapshot: pageSnapshot ?? void 0,
1907
2032
  mode,
1908
- outputLanguage
2033
+ outputLanguage,
2034
+ model
1909
2035
  });
1910
2036
  if (!fixed) {
1911
2037
  fix("bailed out; see diagnosis above");
@@ -1920,7 +2046,7 @@ async function runAutoFixLoop(input) {
1920
2046
  return false;
1921
2047
  }
1922
2048
  async function diagnoseAndFix(input) {
1923
- const { script, specMarkdown, actions, failureLog, pageSnapshot, mode, outputLanguage } = input;
2049
+ const { script, specMarkdown, actions, failureLog, pageSnapshot, mode, outputLanguage, model } = input;
1924
2050
  const outcome = await timedPhase("diagnose", () => diagnose({
1925
2051
  script,
1926
2052
  specMarkdown,
@@ -1928,11 +2054,11 @@ async function diagnoseAndFix(input) {
1928
2054
  failureLog,
1929
2055
  pageSnapshot,
1930
2056
  outputLanguage
1931
- }), "fix");
2057
+ }, { model }), "fix");
1932
2058
  if (outcome.sdkError) {
1933
2059
  fix("diagnose: SDK error talking to Claude");
1934
2060
  if (outcome.raw) fix(`diagnose raw: ${truncateForLog(outcome.raw)}`);
1935
- hint("re-run later, or check ANTHROPIC_API_KEY / network connectivity");
2061
+ hint("re-run later, or check your Claude Code login / network connectivity");
1936
2062
  return null;
1937
2063
  }
1938
2064
  if (!outcome.result) {
@@ -2057,13 +2183,13 @@ function resolveMode(opts) {
2057
2183
  }
2058
2184
  //#endregion
2059
2185
  //#region src/cli/generate.ts
2060
- const generateCommand = new Command("generate").argument("<feature/spec>", "Spec id in '<feature>/<spec>' form (resolves to .ccqa/features/<feature>/test-cases/<spec>/)").description("Generate agent-browser test script from recorded trace actions. test.spec.ts is regenerated from actions.json on every run; pass --force to overwrite manual edits.").option("--max-retries <n>", "Maximum number of auto-fix retries", "3").option("--auto", "Apply auto-fixes without confirmation regardless of confidence (CI use)").option("--no-interactive", "Never prompt; only auto-apply when confidence is high, otherwise give up").option("--force", "Overwrite an existing test.spec.ts without warning").option("--no-snapshot", "Don't pin AGENT_BROWSER_SESSION / capture page snapshots after a failure (debug toggle)").option("--language <bcp47>", "Language for diagnose reasoning / hint text (e.g. 'en', 'ja')", "en").action(async (specPath, opts) => {
2186
+ const generateCommand = new Command("generate").argument("<feature/spec>", "Spec id in '<feature>/<spec>' form (resolves to .ccqa/features/<feature>/test-cases/<spec>/)").description("Generate agent-browser test script from recorded trace actions. test.spec.ts is regenerated from actions.json on every run; pass --force to overwrite manual edits.").option("--max-retries <n>", "Maximum number of auto-fix retries", "3").option("--auto", "Apply auto-fixes without confirmation regardless of confidence (CI use)").option("--no-interactive", "Never prompt; only auto-apply when confidence is high, otherwise give up").option("--force", "Overwrite an existing test.spec.ts without warning").option("--no-snapshot", "Don't pin AGENT_BROWSER_SESSION / capture page snapshots after a failure (debug toggle)").option("--language <bcp47>", "Language for diagnose reasoning / hint text (e.g. 'en', 'ja')", "en").option("-m, --model <name>", "Claude model alias ('sonnet'|'opus'|'haiku') or full ID. Overrides CCQA_MODEL.").action(async (specPath, opts) => {
2061
2187
  const { featureName, specName } = parseSpecPath(specPath);
2062
2188
  const mode = resolveMode(opts);
2063
2189
  const useSnapshot = opts.snapshot !== false;
2064
- await runGenerate(featureName, specName, parseInt(opts.maxRetries, 10), mode, opts.force ?? false, useSnapshot, opts.language ?? "en");
2190
+ await runGenerate(featureName, specName, parseInt(opts.maxRetries, 10), mode, opts.force ?? false, useSnapshot, opts.language ?? "en", opts.model);
2065
2191
  });
2066
- async function runGenerate(featureName, specName, maxRetries, mode, force, useSnapshot, outputLanguage) {
2192
+ async function runGenerate(featureName, specName, maxRetries, mode, force, useSnapshot, outputLanguage, model) {
2067
2193
  header("generate", `${featureName}/${specName}`);
2068
2194
  await ensureCcqaDir();
2069
2195
  const existingScriptPath = await getTestScript(featureName, specName);
@@ -2083,7 +2209,7 @@ async function runGenerate(featureName, specName, maxRetries, mode, force, useSn
2083
2209
  meta("fix-mode", mode);
2084
2210
  meta("language", outputLanguage);
2085
2211
  blank();
2086
- const cleanedActions = await cleanupActions$1(actions);
2212
+ const cleanedActions = await cleanupActions$1(actions, model);
2087
2213
  if (cleanedActions.length !== actions.length) meta("cleaned", cleanedActions.length);
2088
2214
  const scriptPath = await saveTestScript(featureName, specName, actionsToScript(cleanedActions, spec.title, setupScripts.length > 0 ? setupScripts : void 0));
2089
2215
  meta("saved", scriptPath);
@@ -2114,7 +2240,8 @@ async function runGenerate(featureName, specName, maxRetries, mode, force, useSn
2114
2240
  mode,
2115
2241
  runVitest: runVitestForSession,
2116
2242
  agentBrowserSession,
2117
- outputLanguage
2243
+ outputLanguage,
2244
+ model
2118
2245
  })) {
2119
2246
  hint(`run 'ccqa run ${featureName}/${specName}' to execute the test`);
2120
2247
  return;
@@ -2225,12 +2352,13 @@ async function runVitest$1(scriptPath, agentBrowserSession) {
2225
2352
  currentScript
2226
2353
  };
2227
2354
  }
2228
- async function cleanupActions$1(actions) {
2355
+ async function cleanupActions$1(actions, model) {
2229
2356
  try {
2230
2357
  const { result, isError } = await invokeClaudeStreaming({
2231
2358
  prompt: buildCleanupPrompt(actions),
2232
2359
  disableBuiltinTools: true,
2233
- maxTurns: 1
2360
+ maxTurns: 1,
2361
+ model
2234
2362
  }, () => {});
2235
2363
  if (isError || !result) return actions;
2236
2364
  const json = result.trim().replace(/^```(?:json)?\n?([\s\S]*?)\n?```$/, "$1").trim();
@@ -2409,11 +2537,20 @@ async function resolveSpecs(target) {
2409
2537
  }
2410
2538
  //#endregion
2411
2539
  //#region src/cli/trace-setup.ts
2412
- const traceSetupCommand = new Command("trace-setup").argument("<name>", "Setup name to trace (e.g. login)").description("Trace a setup procedure using dummy placeholder values").action(async (name) => {
2413
- await runTraceSetup(name);
2540
+ const traceSetupCommand = new Command("trace-setup").argument("<name>", "Setup name to trace (e.g. login)").description("Trace a setup procedure using dummy placeholder values").option("-m, --model <name>", "Claude model alias ('sonnet'|'opus'|'haiku') or full ID. Overrides CCQA_MODEL.").action(async (name, opts) => {
2541
+ await runTraceSetup(name, opts.model);
2414
2542
  });
2415
- async function runTraceSetup(name) {
2543
+ async function runTraceSetup(name, model) {
2416
2544
  header("trace-setup", name);
2545
+ try {
2546
+ meta("agent-browser", assertAgentBrowserAvailable());
2547
+ } catch (e) {
2548
+ if (e instanceof AgentBrowserUnavailableError) {
2549
+ error(formatAgentBrowserUnavailableMessage());
2550
+ process.exit(1);
2551
+ }
2552
+ throw e;
2553
+ }
2417
2554
  await ensureCcqaDir();
2418
2555
  const spec = parseSetupSpec(await readSetupSpecFile(name));
2419
2556
  const resolvedSpec = replacePlaceholdersWithDummies(spec);
@@ -2442,6 +2579,7 @@ async function runTraceSetup(name) {
2442
2579
  PATH: pathWithAgentBrowserShim(process.env["PATH"]),
2443
2580
  ANTHROPIC_API_KEY: ""
2444
2581
  },
2582
+ model,
2445
2583
  onAbAction: (abAction) => {
2446
2584
  const action = parseAbAction(scrubSecrets(abAction, secretsToScrub));
2447
2585
  if (action) traceActions.push(action);
@@ -2537,11 +2675,11 @@ function scrubSecrets(line, secrets) {
2537
2675
  }
2538
2676
  //#endregion
2539
2677
  //#region src/cli/generate-setup.ts
2540
- const generateSetupCommand = new Command("generate-setup").argument("<name>", "Setup name to generate (e.g. login)").description("Clean up, validate, and templatize setup actions").option("--max-retries <n>", "Maximum number of auto-fix retries", "3").option("--from-dummy", "Resume from existing test.dummy.spec.ts (after manual fix)").option("--auto", "Apply auto-fixes without confirmation regardless of confidence (CI use)").option("--no-interactive", "Never prompt; only auto-apply when confidence is high, otherwise give up").option("--language <bcp47>", "Language for diagnose reasoning / hint text (e.g. 'en', 'ja')", "en").action(async (name, opts) => {
2678
+ const generateSetupCommand = new Command("generate-setup").argument("<name>", "Setup name to generate (e.g. login)").description("Clean up, validate, and templatize setup actions").option("--max-retries <n>", "Maximum number of auto-fix retries", "3").option("--from-dummy", "Resume from existing test.dummy.spec.ts (after manual fix)").option("--auto", "Apply auto-fixes without confirmation regardless of confidence (CI use)").option("--no-interactive", "Never prompt; only auto-apply when confidence is high, otherwise give up").option("--language <bcp47>", "Language for diagnose reasoning / hint text (e.g. 'en', 'ja')", "en").option("-m, --model <name>", "Claude model alias ('sonnet'|'opus'|'haiku') or full ID. Overrides CCQA_MODEL.").action(async (name, opts) => {
2541
2679
  const mode = resolveMode(opts);
2542
- await runGenerateSetup(name, parseInt(opts.maxRetries, 10), opts.fromDummy ?? false, mode, opts.language ?? "en");
2680
+ await runGenerateSetup(name, parseInt(opts.maxRetries, 10), opts.fromDummy ?? false, mode, opts.language ?? "en", opts.model);
2543
2681
  });
2544
- async function runGenerateSetup(name, maxRetries, fromDummy, mode, outputLanguage) {
2682
+ async function runGenerateSetup(name, maxRetries, fromDummy, mode, outputLanguage, model) {
2545
2683
  header("generate-setup", name);
2546
2684
  await ensureCcqaDir();
2547
2685
  const specContent = await readSetupSpecFile(name);
@@ -2562,7 +2700,7 @@ async function runGenerateSetup(name, maxRetries, fromDummy, mode, outputLanguag
2562
2700
  meta("fix-mode", mode);
2563
2701
  meta("language", outputLanguage);
2564
2702
  blank();
2565
- cleanedActions = await cleanupActions(actions);
2703
+ cleanedActions = await cleanupActions(actions, model);
2566
2704
  if (cleanedActions.length !== actions.length) meta("cleaned", cleanedActions.length);
2567
2705
  await writeFile(dummyPath, actionsToScript(cleanedActions, spec.title), "utf-8");
2568
2706
  meta("saved", dummyPath);
@@ -2588,7 +2726,8 @@ async function runGenerateSetup(name, maxRetries, fromDummy, mode, outputLanguag
2588
2726
  mode,
2589
2727
  runVitest: runVitestForSession,
2590
2728
  agentBrowserSession,
2591
- outputLanguage
2729
+ outputLanguage,
2730
+ model
2592
2731
  });
2593
2732
  if (!passed) {
2594
2733
  warn("auto-fix exhausted; setup test still failing");
@@ -2665,12 +2804,13 @@ async function runVitestResolved(scriptPath, agentBrowserSession) {
2665
2804
  await unlink(tmpPath).catch(() => {});
2666
2805
  }
2667
2806
  }
2668
- async function cleanupActions(actions) {
2807
+ async function cleanupActions(actions, model) {
2669
2808
  try {
2670
2809
  const { result, isError } = await invokeClaudeStreaming({
2671
2810
  prompt: buildCleanupPrompt(actions),
2672
2811
  disableBuiltinTools: true,
2673
- maxTurns: 1
2812
+ maxTurns: 1,
2813
+ model
2674
2814
  }, () => {});
2675
2815
  if (isError || !result) return actions;
2676
2816
  const json = result.trim().replace(/^```(?:json)?\n?([\s\S]*?)\n?```$/, "$1").trim();
@@ -2680,6 +2820,607 @@ async function cleanupActions(actions) {
2680
2820
  return actions;
2681
2821
  }
2682
2822
  //#endregion
2823
+ //#region src/prompts/draft.ts
2824
+ function buildNamingSystemPrompt() {
2825
+ return `You name a new ccqa test case based on the user's intent and the existing feature tree.
2826
+
2827
+ ccqa test cases live under \`.ccqa/features/<featureName>/test-cases/<specName>/test-spec.md\`.
2828
+
2829
+ ## Naming rules
2830
+
2831
+ - featureName and specName are kebab-case ASCII (lowercase, words separated by '-').
2832
+ - featureName: a broad area (e.g. "tasks", "auth", "billing", "search").
2833
+ - specName: a short scenario name (e.g. "create-and-complete", "login-with-email", "search-by-tag").
2834
+ - Reuse existing featureName when the user's intent fits an existing area. Only invent a new featureName when the existing tree clearly does not cover the area.
2835
+ - specName must NOT collide with an existing spec under the chosen feature. If the natural name collides, pick a different one that distinguishes the new scenario from the existing ones.
2836
+ - Use the codebase (Read/Grep/Glob) sparingly to confirm domain vocabulary if helpful. Do not over-explore.
2837
+
2838
+ ## Output (STRICT)
2839
+
2840
+ Output ONE fenced \`\`\`json block, nothing else outside it:
2841
+
2842
+ {
2843
+ "featureName": "<kebab-case>",
2844
+ "specName": "<kebab-case>",
2845
+ "reason": "<one short sentence: why this name and how it relates to existing specs>"
2846
+ }
2847
+ `;
2848
+ }
2849
+ function buildNamingPrompt(intent, tree) {
2850
+ return `## User intent
2851
+
2852
+ ${intent}
2853
+
2854
+ ## Existing feature tree
2855
+
2856
+ ${tree.length === 0 ? "(no existing features yet)" : tree.map((f) => {
2857
+ const specLines = f.specs.length === 0 ? " (no specs yet)" : f.specs.map((s) => ` - ${s.specName}${s.title ? ` — ${s.title}` : ""}`).join("\n");
2858
+ return `- ${f.featureName}/\n${specLines}`;
2859
+ }).join("\n")}
2860
+
2861
+ ## Task
2862
+
2863
+ Pick featureName and specName for the new test case. Follow the naming rules. Avoid colliding with any existing specName under the chosen feature.
2864
+ `;
2865
+ }
2866
+ function buildDraftSystemPrompt() {
2867
+ return `You are a QA engineer drafting and refining a ccqa test-spec.md.
2868
+
2869
+ The CLI runs you in a loop: each turn the user gives an intent (first run) or a refinement instruction (later runs). You read the codebase, validate the spec, and return a single JSON report. The CLI displays a diff and asks the user whether to apply.
2870
+
2871
+ ## test-spec.md format (STRICT)
2872
+
2873
+ YAML frontmatter + Markdown body.
2874
+
2875
+ Frontmatter fields:
2876
+ - title: string (required)
2877
+ - baseUrl: string (required, e.g. http://localhost:3000)
2878
+ - prerequisites: string (optional, free text)
2879
+ - setups: array of { name: string, params?: Record<string,string> } (optional)
2880
+
2881
+ Body must contain a \`## Steps\` section followed by step blocks:
2882
+
2883
+ \`\`\`
2884
+ ### Step 1: <short title>
2885
+ - **Instruction**: <imperative, one sentence>
2886
+ - **Expected**: <observable outcome>
2887
+
2888
+ ### Step 2: <short title>
2889
+ ...
2890
+ \`\`\`
2891
+
2892
+ ## Quality rules
2893
+
2894
+ - One user-facing action per step (login, click, fill, navigate, ...).
2895
+ - **Expected** must be assertion-friendly: visible text, URL pattern, element state.
2896
+ - Forbidden in **Expected**: timestamps, exact counts, session IDs, internal state.
2897
+ - 3–8 steps is typical. Fewer means too coarse; more means too fine.
2898
+
2899
+ ## Workflow (use Read / Grep / Glob extensively)
2900
+
2901
+ 1. Read the codebase under cwd to find concrete strings: routes, button labels, aria-labels, page titles, placeholders. Use those exact strings in **Expected**.
2902
+ 2. If the spec references setups, Read \`.ccqa/setups/<name>/setup-spec.md\` and verify each \`params\` key matches the setup's \`placeholders\`.
2903
+ 3. Validate the (current or proposed) spec on four axes — emit one issue per finding:
2904
+ - **assertable**: each Expected can be verified against a string/URL/state that exists in code.
2905
+ - **setups**: referenced setup exists; params keys match placeholders.
2906
+ - **granularity**: not too coarse (multiple actions per step) nor too fine (snapshot-only steps); order is logical.
2907
+ - **unimplemented**: any feature mentioned in the spec that you cannot find in code.
2908
+
2909
+ ## Output contract (STRICT)
2910
+
2911
+ Output exactly ONE fenced \`\`\`json code block, and nothing else outside it. No prose before or after.
2912
+
2913
+ Schema:
2914
+
2915
+ \`\`\`json
2916
+ {
2917
+ "issues": [
2918
+ {
2919
+ "severity": "OK" | "WARN" | "ERROR",
2920
+ "category": "assertable" | "setups" | "granularity" | "unimplemented",
2921
+ "stepId": "step-01" | null,
2922
+ "message": "<one-line summary>",
2923
+ "detail": "<optional, multiline explanation>"
2924
+ }
2925
+ ],
2926
+ "patch": "<COMPLETE rewritten test-spec.md, or empty string if no changes>"
2927
+ }
2928
+ \`\`\`
2929
+
2930
+ ## Patch rules
2931
+
2932
+ - \`patch\` must be the COMPLETE file content if non-empty (never a diff fragment).
2933
+ - The CLI replaces the file atomically with \`patch\`.
2934
+ - For **create** mode: produce a fresh spec from the user intent.
2935
+ - For **refine** mode with a non-empty user instruction: apply the user's request, plus fix any issues it introduces. Preserve the user's wording elsewhere.
2936
+ - For **refine** mode with an empty user instruction: only fix issues you find against the current spec; if everything is fine, return \`patch: ""\`.
2937
+ - If \`patch\` is the same as the current spec, return \`patch: ""\` instead.
2938
+ `;
2939
+ }
2940
+ function buildDraftPrompt(input) {
2941
+ const { mode, existing, userInput } = input;
2942
+ if (mode === "create") return `## Mode
2943
+
2944
+ create — no spec exists yet at the target path. Produce a fresh test-spec.md.
2945
+
2946
+ ## User intent
2947
+
2948
+ ${userInput}
2949
+
2950
+ ## Task
2951
+
2952
+ Read the codebase under cwd. Discover concrete strings (routes, labels, titles). Produce a complete test-spec.md as the \`patch\` field, plus any issues you'd flag about your own draft.
2953
+ `;
2954
+ return `## Mode
2955
+
2956
+ refine — a spec already exists. Apply the user's instruction (if any) and validate against the codebase.
2957
+
2958
+ ## Current spec
2959
+
2960
+ \`\`\`markdown
2961
+ ${existing}\`\`\`
2962
+
2963
+ ${userInput ? `## User refinement instruction\n\n${userInput}\n` : `## User refinement instruction\n\n(empty — re-validate the current spec against the codebase; only emit a non-empty patch if something is actually wrong)\n`}
2964
+ ## Task
2965
+
2966
+ 1. Read the codebase under cwd and any referenced setups.
2967
+ 2. If the user's instruction is non-empty, apply it to the spec.
2968
+ 3. Validate the resulting spec on the four axes. Emit issues.
2969
+ 4. Return the complete updated spec as \`patch\`. If no changes are needed, return \`patch: ""\`.
2970
+ `;
2971
+ }
2972
+ //#endregion
2973
+ //#region src/types.ts
2974
+ const TestStepSchema = z.object({
2975
+ id: z.string(),
2976
+ title: z.string(),
2977
+ instruction: z.string(),
2978
+ expected: z.string()
2979
+ });
2980
+ const SetupRefSchema = z.object({
2981
+ name: z.string(),
2982
+ params: z.record(z.string(), z.string()).optional()
2983
+ });
2984
+ z.object({
2985
+ title: z.string(),
2986
+ baseUrl: z.string(),
2987
+ prerequisites: z.string().optional(),
2988
+ setups: z.array(SetupRefSchema).optional(),
2989
+ steps: z.array(TestStepSchema)
2990
+ });
2991
+ const PlaceholderDefSchema = z.object({
2992
+ dummy: z.string(),
2993
+ description: z.string().optional()
2994
+ });
2995
+ z.object({
2996
+ title: z.string(),
2997
+ placeholders: z.record(z.string(), PlaceholderDefSchema).optional(),
2998
+ steps: z.array(TestStepSchema)
2999
+ });
3000
+ const RouteStepSchema = z.object({
3001
+ title: z.string(),
3002
+ action: z.string(),
3003
+ observation: z.string(),
3004
+ status: z.enum([
3005
+ "PASSED",
3006
+ "FAILED",
3007
+ "SKIPPED"
3008
+ ]),
3009
+ reason: z.string().optional()
3010
+ });
3011
+ z.object({
3012
+ specName: z.string(),
3013
+ timestamp: z.string(),
3014
+ status: z.enum(["passed", "failed"]),
3015
+ steps: z.array(RouteStepSchema)
3016
+ });
3017
+ const DraftIssueSchema = z.object({
3018
+ severity: z.enum([
3019
+ "OK",
3020
+ "WARN",
3021
+ "ERROR"
3022
+ ]),
3023
+ category: z.enum([
3024
+ "assertable",
3025
+ "setups",
3026
+ "granularity",
3027
+ "unimplemented"
3028
+ ]),
3029
+ stepId: z.string().nullable(),
3030
+ message: z.string(),
3031
+ detail: z.string().optional()
3032
+ });
3033
+ const DraftReportSchema = z.object({
3034
+ issues: z.array(DraftIssueSchema),
3035
+ patch: z.string()
3036
+ });
3037
+ const DraftNamingSchema = z.object({
3038
+ featureName: z.string().min(1),
3039
+ specName: z.string().min(1),
3040
+ reason: z.string().optional()
3041
+ });
3042
+ //#endregion
3043
+ //#region src/cli/draft.ts
3044
+ const CATEGORY_LABEL = {
3045
+ assertable: "Assertability",
3046
+ setups: "Setup references",
3047
+ granularity: "Step granularity",
3048
+ unimplemented: "Unimplemented checks"
3049
+ };
3050
+ const draftCommand = new Command("draft").argument("[feature/spec]", "Optional spec path (e.g. tasks/create-and-complete). If omitted, Claude proposes one from your intent.").description("Interactively draft and refine a test-spec.md with Claude Code").option("--instruction <text>", "Non-interactive single-shot instruction (skips the interactive loop)").option("--apply", "Auto-apply each generated patch without [y/N] confirmation", false).action(async (specPath, opts) => {
3051
+ await ensureCcqaDir();
3052
+ let featureName;
3053
+ let specName;
3054
+ let prefilledIntent = null;
3055
+ if (specPath) ({featureName, specName} = parseSpecPath(specPath));
3056
+ else {
3057
+ const { naming, intent } = await proposeNaming(opts);
3058
+ featureName = naming.featureName;
3059
+ specName = naming.specName;
3060
+ prefilledIntent = intent;
3061
+ }
3062
+ await runDraft(featureName, specName, opts, prefilledIntent);
3063
+ });
3064
+ async function runDraft(featureName, specName, opts, prefilledIntent) {
3065
+ header("draft", `${featureName}/${specName}`);
3066
+ const oneShot = opts.instruction !== void 0;
3067
+ let useIntentOnce = prefilledIntent !== null && !oneShot;
3068
+ while (true) {
3069
+ const existing = await tryReadSpecFile(featureName, specName);
3070
+ const isFirstRun = existing === null;
3071
+ let userInput;
3072
+ if (oneShot) userInput = opts.instruction ?? "";
3073
+ else if (useIntentOnce && isFirstRun) {
3074
+ userInput = prefilledIntent ?? "";
3075
+ useIntentOnce = false;
3076
+ } else userInput = await prompt(isFirstRun ? "What do you want to test? > " : "How would you like to refine? (empty = re-validate) > ");
3077
+ if (isFirstRun && !userInput.trim()) {
3078
+ error("intent required for the first draft (no spec exists yet)");
3079
+ process.exit(1);
3080
+ }
3081
+ const turnResult = await runOneTurn({
3082
+ featureName,
3083
+ specName,
3084
+ existing,
3085
+ userInput: userInput.trim(),
3086
+ autoApply: opts.apply === true
3087
+ });
3088
+ if (oneShot) process.exit(turnResult.hasError && !turnResult.applied ? 1 : 0);
3089
+ blank();
3090
+ if (/^y/i.test(await prompt("Are you done with this draft? [y/N] "))) {
3091
+ info("draft session complete.");
3092
+ hint(`run 'ccqa trace ${featureName}/${specName}' to record actions`);
3093
+ process.exit(0);
3094
+ }
3095
+ }
3096
+ }
3097
+ async function runOneTurn(input) {
3098
+ const { featureName, specName, existing, userInput, autoApply } = input;
3099
+ const isFirstRun = existing === null;
3100
+ const systemPrompt = buildDraftSystemPrompt();
3101
+ const userPrompt = buildDraftPrompt({
3102
+ mode: isFirstRun ? "create" : "refine",
3103
+ existing: existing ?? "",
3104
+ userInput
3105
+ });
3106
+ info(isFirstRun ? "Reading codebase and drafting spec..." : "Re-validating spec against codebase...");
3107
+ const toolCounts = {};
3108
+ const startedAt = Date.now();
3109
+ const { result, isError } = await invokeClaudeStreaming({
3110
+ prompt: userPrompt,
3111
+ systemPrompt,
3112
+ allowedTools: [
3113
+ "Read",
3114
+ "Grep",
3115
+ "Glob"
3116
+ ],
3117
+ silenceBashLog: true
3118
+ }, (msg) => {
3119
+ if (msg.type !== "assistant") return;
3120
+ for (const block of msg.message.content ?? []) if (block.type === "tool_use") toolCounts[block.name] = (toolCounts[block.name] ?? 0) + 1;
3121
+ });
3122
+ printToolSummary(toolCounts, Date.now() - startedAt);
3123
+ if (isError) {
3124
+ error("Claude returned an error result");
3125
+ return {
3126
+ hasError: true,
3127
+ applied: false
3128
+ };
3129
+ }
3130
+ const json = extractJsonBlock(result);
3131
+ if (!json) {
3132
+ error("Claude did not return a json block");
3133
+ warn(`raw tail: ${truncate(result, 200)}`);
3134
+ return {
3135
+ hasError: true,
3136
+ applied: false
3137
+ };
3138
+ }
3139
+ let report;
3140
+ try {
3141
+ report = DraftReportSchema.parse(JSON.parse(json));
3142
+ } catch (e) {
3143
+ error(`failed to parse draft report: ${e.message}`);
3144
+ return {
3145
+ hasError: true,
3146
+ applied: false
3147
+ };
3148
+ }
3149
+ const hasError = printReviewBlock(report.issues);
3150
+ const original = existing ?? "";
3151
+ if (!report.patch || report.patch === original) {
3152
+ blank();
3153
+ info("no changes proposed.");
3154
+ return {
3155
+ hasError,
3156
+ applied: false
3157
+ };
3158
+ }
3159
+ blank();
3160
+ info("--- proposed changes ---");
3161
+ printUnifiedDiff(original, report.patch);
3162
+ blank();
3163
+ if (!(autoApply ? true : /^y/i.test(await prompt("Apply this patch? [y/N] ")))) {
3164
+ info("aborted — no changes applied.");
3165
+ return {
3166
+ hasError,
3167
+ applied: false
3168
+ };
3169
+ }
3170
+ try {
3171
+ parseTestSpec(report.patch);
3172
+ } catch (e) {
3173
+ error(`refused to apply: patch failed validation (${e.message})`);
3174
+ return {
3175
+ hasError: true,
3176
+ applied: false
3177
+ };
3178
+ }
3179
+ meta("saved", await saveSpecFile(featureName, specName, report.patch));
3180
+ return {
3181
+ hasError,
3182
+ applied: true
3183
+ };
3184
+ }
3185
+ async function prompt(question) {
3186
+ const rl = createInterface$1({
3187
+ input: process.stdin,
3188
+ output: process.stdout
3189
+ });
3190
+ rl.on("SIGINT", () => {
3191
+ rl.close();
3192
+ process.exit(130);
3193
+ });
3194
+ try {
3195
+ return (await rl.question(question)).trim();
3196
+ } finally {
3197
+ rl.close();
3198
+ }
3199
+ }
3200
+ /** Aggregated tool-call counts shown after each Claude turn. */
3201
+ function formatToolSummary(counts, elapsedMs) {
3202
+ const entries = Object.entries(counts).filter(([, n]) => n > 0).sort((a, b) => b[1] - a[1]).map(([name, n]) => `${n} ${name}`);
3203
+ return ` ✓ ${entries.length === 0 ? "no tool calls" : entries.join(", ")} (${(elapsedMs / 1e3).toFixed(1)}s)`;
3204
+ }
3205
+ function printToolSummary(counts, elapsedMs) {
3206
+ process.stdout.write(`${formatToolSummary(counts, elapsedMs)}\n`);
3207
+ }
3208
+ /**
3209
+ * Renders the review report as a visually separated block, grouped by
3210
+ * severity. ERROR and WARN findings get full detail; OK findings collapse
3211
+ * to a one-line summary of category names. Returns whether any ERROR
3212
+ * severity was emitted.
3213
+ */
3214
+ function printReviewBlock(issues) {
3215
+ const RULE = "─".repeat(67);
3216
+ const errors = issues.filter((i) => i.severity === "ERROR");
3217
+ const warnings = issues.filter((i) => i.severity === "WARN");
3218
+ const passed = issues.filter((i) => i.severity === "OK");
3219
+ const headerParts = [];
3220
+ if (errors.length) headerParts.push(`${errors.length} error${errors.length > 1 ? "s" : ""}`);
3221
+ if (warnings.length) headerParts.push(`${warnings.length} warning${warnings.length > 1 ? "s" : ""}`);
3222
+ if (passed.length) headerParts.push(`${passed.length} passed`);
3223
+ const headerSuffix = headerParts.length ? ` (${headerParts.join(", ")})` : "";
3224
+ const ruleLen = Math.max(0, 60 - headerSuffix.length);
3225
+ process.stdout.write(`\n── Review${headerSuffix} ${"─".repeat(ruleLen)}\n\n`);
3226
+ if (issues.length === 0) {
3227
+ process.stdout.write(" (no findings)\n");
3228
+ process.stdout.write(`\n${RULE}\n\n`);
3229
+ return false;
3230
+ }
3231
+ if (errors.length) {
3232
+ process.stdout.write(` ERRORS (${errors.length})\n`);
3233
+ for (const issue of errors) writeFinding(issue);
3234
+ process.stdout.write("\n");
3235
+ }
3236
+ if (warnings.length) {
3237
+ process.stdout.write(` WARNINGS (${warnings.length})\n`);
3238
+ for (const issue of warnings) writeFinding(issue);
3239
+ process.stdout.write("\n");
3240
+ }
3241
+ if (passed.length) {
3242
+ const names = passed.map((i) => CATEGORY_LABEL[i.category]).join(", ");
3243
+ process.stdout.write(` PASSED (${passed.length})\n ${names}\n`);
3244
+ }
3245
+ process.stdout.write(`\n${RULE}\n\n`);
3246
+ return errors.length > 0;
3247
+ }
3248
+ function writeFinding(issue) {
3249
+ const stepPart = issue.stepId ? ` ${issue.stepId}` : "";
3250
+ process.stdout.write(` ${CATEGORY_LABEL[issue.category]}${stepPart}\n`);
3251
+ process.stdout.write(` ${issue.message}\n`);
3252
+ if (issue.detail) process.stdout.write(` └ ${issue.detail.replace(/\n/g, "\n ")}\n`);
3253
+ }
3254
+ async function proposeNaming(opts) {
3255
+ const oneShot = opts.instruction !== void 0;
3256
+ const intent = oneShot ? opts.instruction ?? "" : await prompt("What do you want to test? > ");
3257
+ if (!intent.trim()) {
3258
+ error("intent required to propose a feature/spec name");
3259
+ process.exit(1);
3260
+ }
3261
+ const tree = await listFeatureTree();
3262
+ const treeForPrompt = tree.map((f) => ({
3263
+ featureName: f.featureName,
3264
+ specs: f.specs.map((s) => ({
3265
+ specName: s.specName,
3266
+ ...s.title ? { title: s.title } : {}
3267
+ }))
3268
+ }));
3269
+ info("Proposing a feature/spec name based on your intent...");
3270
+ const { result, isError } = await invokeClaudeStreaming({
3271
+ silenceBashLog: true,
3272
+ prompt: buildNamingPrompt(intent.trim(), treeForPrompt),
3273
+ systemPrompt: buildNamingSystemPrompt(),
3274
+ allowedTools: [
3275
+ "Read",
3276
+ "Grep",
3277
+ "Glob"
3278
+ ]
3279
+ }, () => {});
3280
+ if (isError) {
3281
+ error("Claude failed during naming");
3282
+ process.exit(1);
3283
+ }
3284
+ const json = extractJsonBlock(result);
3285
+ if (!json) {
3286
+ error("Claude did not return a json block for naming");
3287
+ process.exit(1);
3288
+ }
3289
+ let proposed;
3290
+ try {
3291
+ proposed = DraftNamingSchema.parse(JSON.parse(json));
3292
+ } catch (e) {
3293
+ error(`failed to parse naming response: ${e.message}`);
3294
+ process.exit(1);
3295
+ }
3296
+ const sanitized = {
3297
+ featureName: sanitizeNamePart(proposed.featureName),
3298
+ specName: sanitizeNamePart(proposed.specName)
3299
+ };
3300
+ if (!sanitized.featureName || !sanitized.specName) {
3301
+ error(`Claude returned an invalid name: ${proposed.featureName}/${proposed.specName}`);
3302
+ process.exit(1);
3303
+ }
3304
+ const final = ensureUnique(tree, sanitized.featureName, sanitized.specName);
3305
+ meta("proposed", `${final.featureName}/${final.specName}`);
3306
+ if (proposed.reason) meta("reason", proposed.reason);
3307
+ if (oneShot || opts.apply === true) return {
3308
+ naming: final,
3309
+ intent: intent.trim()
3310
+ };
3311
+ const answer = await prompt(`Use this name? [y/N/edit] > `);
3312
+ if (/^y/i.test(answer)) return {
3313
+ naming: final,
3314
+ intent: intent.trim()
3315
+ };
3316
+ if (/^e/i.test(answer)) {
3317
+ const manual = await prompt("Enter feature/spec (e.g. tasks/create-and-complete) > ");
3318
+ const parts = manual.split("/");
3319
+ if (parts.length !== 2 || !parts[0] || !parts[1]) {
3320
+ error(`invalid spec path: "${manual}". Expected "<feature>/<spec>"`);
3321
+ process.exit(1);
3322
+ }
3323
+ const featureName = sanitizeNamePart(parts[0]);
3324
+ const specName = sanitizeNamePart(parts[1]);
3325
+ if (!featureName || !specName) {
3326
+ error(`invalid characters in name: ${parts[0]}/${parts[1]}`);
3327
+ process.exit(1);
3328
+ }
3329
+ return {
3330
+ naming: {
3331
+ featureName,
3332
+ specName
3333
+ },
3334
+ intent: intent.trim()
3335
+ };
3336
+ }
3337
+ info("aborted — no draft created.");
3338
+ process.exit(0);
3339
+ }
3340
+ /**
3341
+ * Restrict to kebab-case-friendly characters: lowercase letters, digits, hyphen.
3342
+ * Anything else is dropped or replaced with '-'. Collapses repeated/edge hyphens.
3343
+ */
3344
+ function sanitizeNamePart(raw) {
3345
+ return raw.trim().toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 60);
3346
+ }
3347
+ function ensureUnique(tree, featureName, specName) {
3348
+ const feature = tree.find((f) => f.featureName === featureName);
3349
+ if (!feature) return {
3350
+ featureName,
3351
+ specName
3352
+ };
3353
+ const taken = new Set(feature.specs.map((s) => s.specName));
3354
+ if (!taken.has(specName)) return {
3355
+ featureName,
3356
+ specName
3357
+ };
3358
+ for (let i = 2; i < 100; i++) {
3359
+ const candidate = `${specName}-${i}`;
3360
+ if (!taken.has(candidate)) return {
3361
+ featureName,
3362
+ specName: candidate
3363
+ };
3364
+ }
3365
+ return {
3366
+ featureName,
3367
+ specName: `${specName}-${Date.now()}`
3368
+ };
3369
+ }
3370
+ function extractJsonBlock(text) {
3371
+ const fenced = text.match(/```(?:json)?\s*\n([\s\S]*?)\n```/);
3372
+ if (fenced && fenced[1]) return fenced[1].trim();
3373
+ const trimmed = text.trim();
3374
+ if (trimmed.startsWith("{") && trimmed.endsWith("}")) return trimmed;
3375
+ return null;
3376
+ }
3377
+ function printUnifiedDiff(before, after) {
3378
+ const lines = computeLineDiff(before.split("\n"), after.split("\n"));
3379
+ for (const line of lines) process.stdout.write(line + "\n");
3380
+ }
3381
+ function computeLineDiff(a, b) {
3382
+ const n = a.length;
3383
+ const m = b.length;
3384
+ const dp = Array.from({ length: n + 1 }, () => new Array(m + 1).fill(0));
3385
+ for (let i = n - 1; i >= 0; i--) for (let j = m - 1; j >= 0; j--) dp[i][j] = a[i] === b[j] ? dp[i + 1][j + 1] + 1 : Math.max(dp[i + 1][j], dp[i][j + 1]);
3386
+ const out = [];
3387
+ let i = 0;
3388
+ let j = 0;
3389
+ while (i < n && j < m) if (a[i] === b[j]) {
3390
+ out.push({
3391
+ kind: "ctx",
3392
+ text: a[i]
3393
+ });
3394
+ i++;
3395
+ j++;
3396
+ } else if (dp[i + 1][j] >= dp[i][j + 1]) {
3397
+ out.push({
3398
+ kind: "del",
3399
+ text: a[i]
3400
+ });
3401
+ i++;
3402
+ } else {
3403
+ out.push({
3404
+ kind: "add",
3405
+ text: b[j]
3406
+ });
3407
+ j++;
3408
+ }
3409
+ while (i < n) out.push({
3410
+ kind: "del",
3411
+ text: a[i++]
3412
+ });
3413
+ while (j < m) out.push({
3414
+ kind: "add",
3415
+ text: b[j++]
3416
+ });
3417
+ return out.map((l) => l.kind === "add" ? `+ ${l.text}` : l.kind === "del" ? `- ${l.text}` : ` ${l.text}`);
3418
+ }
3419
+ function truncate(s, n) {
3420
+ if (s.length <= n) return s;
3421
+ return s.slice(s.length - n);
3422
+ }
3423
+ //#endregion
2683
3424
  //#region src/cli/index.ts
2684
3425
  const packageJsonPath = resolvePackageJson();
2685
3426
  const { version } = JSON.parse(readFileSync(packageJsonPath, "utf8"));
@@ -2695,6 +3436,7 @@ function resolvePackageJson() {
2695
3436
  }
2696
3437
  const program = new Command();
2697
3438
  program.name("ccqa").description("E2E test CLI using Claude Code + agent-browser").version(version);
3439
+ program.addCommand(draftCommand);
2698
3440
  program.addCommand(traceCommand);
2699
3441
  program.addCommand(generateCommand);
2700
3442
  program.addCommand(runCommand);
package/dist/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ccqa",
3
- "version": "0.3.7",
3
+ "version": "0.3.9",
4
4
  "type": "module",
5
5
  "description": "Browser test recorder powered by Claude Code and agent-browser",
6
6
  "repository": {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ccqa",
3
- "version": "0.3.7",
3
+ "version": "0.3.9",
4
4
  "type": "module",
5
5
  "description": "Browser test recorder powered by Claude Code and agent-browser",
6
6
  "repository": {