ccqa 0.10.1 → 0.10.2

package/README.md

@@ -69,7 +69,7 @@ ccqa run tasks/create-and-complete      # vitest replays test.spec.ts; no LLM
 ccqa run tasks/create-and-complete      # Claude drives the browser every time
 ```
 
-Live specs can start already-signed-in by pointing `statePath:` at a saved agent-browser state file (cookies + localStorage). Run an interactive login locally once, save the state with `agent-browser state save .ccqa/sessions/<name>.json`, then commit the path (not the file) — see [Pre-authenticated state](#pre-authenticated-state-statepath) below for the local bootstrap and the CI restore pattern.
+Live specs can start already-signed-in by naming a saved session with `session:`. Create it once with `ccqa session bootstrap <name>` (log in by hand, ccqa saves the cookies + localStorage), then specs restore it — see [Saved sessions](#saved-sessions-session) below for the bootstrap and the CI restore pattern.
 
 By default deterministic runs write step-boundary screenshots and metadata to `ccqa-report/evidence/<feature>/<spec>/` so a reviewer can confirm a passing spec actually reached the states its `expected` clauses describe. Disable with `--no-evidence`.
 
@@ -207,71 +207,67 @@ ccqa run --retry 2 tasks/create-and-complete
 
 Constraints on selectors / `agent-browser` subcommands that apply during `ccqa record` (no `eval`, no `@ref`, no bare-tag positional `find`, no chained agent-browser calls) are **relaxed** for live specs — Claude can use any subcommand and any selector style because there is no replay contract to honour.
 
-### Pre-authenticated state (`statePath:`)
+### Saved sessions (`session:`)
 
-By default each `ccqa run` of a live spec spins up a fresh `agent-browser` session and starts signed-out. That keeps runs hermetic but forces every device-trust gate (Slack "we don't recognize this browser", Google's unfamiliar-device prompt, MFA challenges, …) to fire on every run.
+By default each `ccqa run` of a live spec starts signed-out and logs in through its own steps. That's fine for plain form logins, but some providers gate every fresh browser with a device-trust check (an "unrecognized device" e-mail code, an MFA prompt) that a human has to clear by hand — impractical to repeat on every run, impossible in CI.
 
-To skip them, save an authenticated browser state to a JSON file once locally and point the spec at it:
+For those, save the signed-in browser state once and let the spec **restore** it. ccqa does not manage authentication — `session` is purely an optional restore of cookies + localStorage. Specs that can just log in normally don't use it.
 
 ```yaml
-title: Slack App Home — non-admin access denied
+title: Admin can open the settings page
 mode: live
-statePath: .ccqa/sessions/slack-stg.json   # cookies + localStorage to restore
+session: admin            # restore the saved "admin" session before step 1
 steps:
-  - ...
+  - ...                   # no login steps — the spec starts signed-in
 ```
 
-ccqa resolves the path against the project root and passes `--state <path>` to every `agent-browser` invocation in the run (including ccqa's own screenshot calls). The file is **read-only** — `--state` loads it but never writes back to it. Re-running locally or in CI does not mutate it.
+A spec can also restore several sessions at once (e.g. one provider in each), and ccqa merges them:
 
-Bootstrap once locally:
+```yaml
+session:
+  - admin                 # one provider, signed in as admin
+  - admin-chat            # another provider, same person
+```
 
-```bash
-# 1. Log in interactively in a headed browser.
-agent-browser --headed open https://app.slack.com
-# …complete login + device-trust prompts by hand…
+#### Create a session — `ccqa session bootstrap`
 
-# 2. Snapshot cookies + localStorage to the path the spec references.
-mkdir -p .ccqa/sessions
-agent-browser state save .ccqa/sessions/slack-stg.json
-agent-browser close
+```bash
+# Opens a headed browser. Log in by hand (clear any device-trust gate),
+# then press Enter and ccqa saves the session.
+ccqa session bootstrap admin --url https://app.example.com/login
 
-# 3. ccqa run reuses the saved state — no login prompt.
-ccqa run slack/app-home-non-admin-access-denied
+# List saved sessions (names + save times; no secret values shown).
+ccqa session ls
 ```
 
-Add `.ccqa/sessions/` to `.gitignore` — these files contain live auth cookies and must never be committed.
+Sessions are saved to `.ccqa/sessions/<profile>/<name>.json`. The `<profile>` is the same `--profile` that selects the `.ccqa/profiles/<profile>.env` file, so one flag picks both the environment and its sessions bucket; with no `--profile` the bucket is `default`. `ccqa init` drops a self-ignoring `.ccqa/sessions/.gitignore` so saved sessions stay out of git — **they contain live auth cookies and must never be committed.**
 
-#### CI: bring the state file with you
+When a spec names a session that hasn't been created yet, the run stops and tells you which `ccqa session bootstrap` to run, rather than starting unauthenticated.
 
-`statePath:` lives entirely inside `.ccqa/` and never touches `~/`. CI re-uses the state by writing the file into the same path the spec already references:
+#### CI: restore the session
+
+Saved sessions live entirely inside `.ccqa/` and never touch `~/`. In CI, write each session file back to the path the run expects (read it from your secret store):
 
 ```bash
-# Locally, after the interactive bootstrap above:
-base64 -i .ccqa/sessions/slack-stg.json | pbcopy
-# paste into your CI secret store as CCQA_SLACK_STG_STATE_B64
+# Locally, after bootstrapping, copy the file into your CI secret store:
+base64 -i .ccqa/sessions/default/admin.json | pbcopy   # paste as CCQA_SESSION_ADMIN_B64
 ```
 
 ```yaml
-# .github/workflows/ccqa.yml (sketch)
-- name: Restore agent-browser state
+# CI step (sketch) — restore before `ccqa run`
+- name: Restore session
   env:
-    CCQA_SLACK_STG_STATE_B64: ${{ secrets.CCQA_SLACK_STG_STATE_B64 }}
+    CCQA_SESSION_ADMIN_B64: ${{ secrets.CCQA_SESSION_ADMIN_B64 }}
   run: |
-    mkdir -p .ccqa/sessions
-    printf '%s' "$CCQA_SLACK_STG_STATE_B64" | base64 -d \
-      > .ccqa/sessions/slack-stg.json
-
-- name: Run live specs
-  env:
-    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
-  run: pnpm ccqa run --report
+    mkdir -p .ccqa/sessions/default
+    printf '%s' "$CCQA_SESSION_ADMIN_B64" | base64 -d > .ccqa/sessions/default/admin.json
 ```
 
-Caveats:
+Notes:
 
-- **Expiry.** Whatever the upstream service's "remember this device" window is (Slack ≈ 30 days, others vary), the cookies in the state file eventually expire and CI starts failing on the device-trust gate again. Re-bootstrap locally and rotate the secret.
-- **Treat the file as a credential.** It contains live auth cookies. Store it in your CI secret manager (GitHub Actions encrypted secrets, Vault, …) and never commit it.
-- **Deterministic specs ignore `statePath:`.** Today it only affects `mode: live`; vitest-replayed specs always run isolated.
+- **Expiry.** The provider's "remember this device" window eventually lapses and the saved cookies stop working. Re-run `ccqa session bootstrap` locally and rotate the secret.
+- **Treat session files as credentials.** They hold live auth cookies. Keep them in a secret manager; never commit them.
+- **Deterministic specs ignore `session:`.** It only affects `mode: live`; vitest-replayed specs always run isolated.
 
 ### Per-project guidance (`.ccqa/prompts/live.user.md` + `live.agent.md`)
 

package/dist/bin/ccqa.mjs

@@ -6,7 +6,7 @@ import { accessSync, existsSync, readFileSync, statSync } from "node:fs";
 import { fileURLToPath } from "node:url";
 import { access, mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from "node:fs/promises";
 import { homedir, tmpdir } from "node:os";
-import { delimiter, dirname, isAbsolute, join, posix, relative, resolve } from "node:path";
+import { delimiter, dirname, join, posix, relative, resolve } from "node:path";
 import { parse, stringify } from "yaml";
 import { ZodError, z } from "zod";
 import { execFile, spawn, spawnSync } from "node:child_process";
@@ -136,12 +136,27 @@ const StepSchema = z.union([ActionStepSchema, IncludeStepSchema]);
 *     codegen is impractical). Cost ~$0.5 per spec.
 */
 const SpecModeSchema = z.enum(["deterministic", "live"]);
+/**
+* A session name: the identifier of a saved browser session (cookies +
+* localStorage) to restore before the spec runs. Resolved to
+* `.ccqa/sessions/<profile>/<name>.json` at run time. Restricted to a safe
+* slug so the name can't escape the sessions directory.
+*/
+const SessionNameSchema = z.string().min(1).regex(/^[a-z0-9][a-z0-9._-]*$/i, "session name must be a slug (letters, digits, '.', '_', '-'; no path separators)");
+/**
+* Sessions to restore before a `mode: live` spec runs: a single name or a
+* list. Always normalized to an array. Each name maps to a saved
+* agent-browser state file; multiple names are merged (their cookies +
+* localStorage are unioned) and restored together, so a spec can start
+* signed-in to several providers at once.
+*/
+const SessionFieldSchema = z.union([SessionNameSchema, z.array(SessionNameSchema).min(1)]).transform((v) => Array.isArray(v) ? v : [v]);
 /** Top-level spec schema. `.strict()` rejects any unknown key. */
 const TestSpecSchema = z.object({
 	title: z.string().min(1),
 	relatedPaths: z.array(z.string().min(1)).optional(),
 	mode: SpecModeSchema.optional(),
-	statePath: z.string().min(1).optional(),
+	session: SessionFieldSchema.optional(),
 	steps: z.array(StepSchema).min(1)
 }).strict();
 /** Default mode when `mode:` is absent. */
@@ -4584,6 +4599,55 @@ function oneLine$1(s) {
 	return s.replace(/\s+/g, " ").trim();
 }
 //#endregion
+//#region src/runtime/session-state.ts
+/** Default per-profile sessions root, relative to the project (`--cwd`). */
+const SESSIONS_SUBDIR = ".ccqa/sessions";
+/** The per-profile sessions directory: `<cwd>/.ccqa/sessions/<profile>/`. */
+function sessionsDir(profile, cwd) {
+	return join(cwd, SESSIONS_SUBDIR, profile ?? "default");
+}
+/**
+* Resolve a session name to its state file path:
+* `<cwd>/.ccqa/sessions/<profile>/<name>.json`. The name is a slug (validated
+* by SessionNameSchema), so it can't escape the sessions directory.
+*/
+function sessionFilePath(name, profile, cwd) {
+	return join(sessionsDir(profile, cwd), `${name}.json`);
+}
+/** Read and parse a saved session file. Throws if missing or malformed. */
+async function loadStorageState(path) {
+	const raw = await readFile(path, "utf8");
+	const parsed = JSON.parse(raw);
+	if (!Array.isArray(parsed?.cookies) || !Array.isArray(parsed?.origins)) throw new Error(`not a valid agent-browser state file (expected { cookies, origins }): ${path}`);
+	return parsed;
+}
+/**
+* Merge several saved sessions into one. Cookies are unioned by
+* (name, domain, path); origins by `origin`. Later states win on collision.
+* Distinct providers don't collide (different domains / origins), so merging
+* two single-provider sessions yields a combined signed-in state.
+*/
+function mergeStorageStates(states) {
+	const cookies = /* @__PURE__ */ new Map();
+	for (const s of states) for (const c of s.cookies) cookies.set(`${c.name}\t${c.domain}\t${c.path}`, c);
+	const origins = /* @__PURE__ */ new Map();
+	for (const s of states) for (const o of s.origins) origins.set(o.origin, o);
+	return {
+		cookies: [...cookies.values()],
+		origins: [...origins.values()]
+	};
+}
+/**
+* Write a merged state to a fresh temp file and return its path. Source
+* session files are never modified; the temp file is what gets restored via
+* `--state`, so re-runs (local or CI) leave the source-of-truth untouched.
+*/
+async function writeMergedTempState(state) {
+	const file = join(await mkdtemp(join(tmpdir(), "ccqa-session-")), "merged-state.json");
+	await writeFile(file, JSON.stringify(state), "utf8");
+	return file;
+}
+//#endregion
 //#region src/claude/agent-browser-invoke.ts
 function agentBrowserInvokeBase(input) {
 	const env = {
@@ -4641,7 +4705,7 @@ function buildLiveSystemPromptPrefix(input) {
 	const stepsText = input.allSteps.map((s) => `### ${s.id} [${s.source}]
 - **Instruction**: ${s.instruction}
 - **Expected**: ${s.expected}`).join("\n\n");
-	const stateLine = input.statePath ? `\n\nA pre-recorded auth-state file is provided at \`${input.statePath}\` (also in the env var \`CCQA_AB_STATE\`). **Always also pass \`--state "$CCQA_AB_STATE"\`** to every \`agent-browser\` command — this restores cookies and localStorage from a prior interactive login, so the user is already signed in to the application under test from step 1. The file is loaded read-only; do not run \`agent-browser state save\`.` : "";
+	const stateLine = input.statePath ? `\n\nA pre-recorded auth-state file is provided at \`${input.statePath}\` (also in the env var \`CCQA_AB_STATE\`). **Always also pass \`--state "$CCQA_AB_STATE"\`** to every \`agent-browser\` command — this restores cookies and localStorage saved from a prior interactive login (one or more providers), so the user is already signed in to the application under test from step 1. The file is loaded read-only; do not run \`agent-browser state save\`.` : "";
 	return `You are a QA execution agent. You are executing ONE step of a browser-based end-to-end test and judging whether the step's expected outcome was achieved. You are NOT recording a replayable test script — be flexible, explore the DOM as needed, and make a clear pass / fail call at the end.
 
 ## Session
@@ -5174,6 +5238,44 @@ async function runDriftAudit(runs, opts, cwd) {
 	}
 	return out;
 }
+/**
+* Resolve `spec.session` names to a single state file to restore. Each name
+* maps to `.ccqa/sessions/<profile>/<name>.json` and must load as a valid
+* agent-browser state (the spec assumes it starts signed-in). One name is
+* restored from its file directly; several are merged into a temp file. A
+* missing or malformed session fails with a `ccqa session bootstrap` hint
+* instead of running unauthenticated.
+*/
+async function resolveSessionState(names, profile, cwd) {
+	const paths = names.map((name) => ({
+		name,
+		path: sessionFilePath(name, profile, cwd)
+	}));
+	const loaded = [];
+	const broken = [];
+	for (const p of paths) try {
+		loaded.push(await loadStorageState(p.path));
+	} catch {
+		broken.push(p);
+	}
+	if (broken.length > 0) {
+		const list = broken.map((b) => b.name).join(", ");
+		const profileFlag = profile ? ` --profile ${profile}` : "";
+		return {
+			ok: false,
+			error: `session not usable: ${list} (looked under ${dirname(broken[0].path)})`,
+			hint: `create it with: ${broken.map((b) => `ccqa session bootstrap ${b.name}${profileFlag}`).join("  ·  ")}`
+		};
+	}
+	if (paths.length === 1) return {
+		ok: true,
+		statePath: paths[0].path
+	};
+	return {
+		ok: true,
+		statePath: await writeMergedTempState(mergeStorageStates(loaded))
+	};
+}
 async function runOneSpec(args) {
 	const { featureName, specName, opts, userPromptSuffix, cwd } = args;
 	const specDir = getSpecDir(featureName, specName, cwd);
@@ -5198,21 +5300,20 @@ async function runOneSpec(args) {
 	const sessionName = generateLiveSessionName();
 	meta("session", sessionName);
 	let statePath = null;
-	if (spec.statePath) {
-		statePath = isAbsolute(spec.statePath) ? spec.statePath : resolve(cwd, spec.statePath);
-		try {
-			await access(statePath);
-		} catch {
-			const msg = `spec.statePath points to a missing file: ${statePath}`;
-			error(msg);
+	if (spec.session && spec.session.length > 0) {
+		const resolution = await resolveSessionState(spec.session, opts.profile, cwd);
+		if (!resolution.ok) {
+			error(resolution.error);
+			hint(resolution.hint);
 			return {
 				kind: "error",
 				featureName,
 				specName,
-				error: msg
+				error: resolution.error
 			};
 		}
-		meta("state", statePath);
+		statePath = resolution.statePath;
+		meta("state", spec.session.join(", "));
 	}
 	const runId = buildRunId();
 	const runDir = opts.out ?? join(specDir, "runs", runId);
@@ -5576,6 +5677,7 @@ async function runDispatcher(targets, opts) {
 		...reportDir ? { reportDir } : {},
 		...typeof opts.retry === "number" ? { retry: opts.retry } : {},
 		concurrency: opts.concurrency ?? 1,
+		...opts.profile ? { profile: opts.profile } : {},
 		...reportDir && opts.driftAudit !== false ? { driftAudit: true } : {},
 		...reportDir && opts.failureAnalysis === false ? { failureAnalysis: false } : {}
 	});
@@ -9669,6 +9771,13 @@ Write stable, hand-maintained context here for the trace phase of 'ccqa record'.
 		content: `# Agent learnings for ccqa record
 
 This file is updated by 'ccqa record --update-agent-prompt'. Same convention as live.agent.md — stable rules go in record.user.md.
+`
+	},
+	{
+		relPath: ".ccqa/sessions/.gitignore",
+		content: `# Saved browser sessions contain live auth cookies. Never commit them.
+*
+!.gitignore
 `
 	}
 ];
@@ -9676,6 +9785,7 @@ const initCommand = new Command("init").description("Create .ccqa/prompts/{live,
 	const cwd = resolveCwd(opts.cwd);
 	header("init", cwd);
 	await mkdir(join(cwd, ".ccqa", "prompts"), { recursive: true });
+	await mkdir(join(cwd, ".ccqa", "sessions"), { recursive: true });
 	const created = [];
 	const skipped = [];
 	for (const t of TEMPLATES) if (await writeTemplate(join(cwd, t.relPath), t.content, opts.force ?? false)) created.push(t.relPath);
@@ -10183,6 +10293,89 @@ function mdCell(value) {
 	return value.replace(/\|/g, "\\|").replace(/\n/g, " ");
 }
 //#endregion
+//#region src/cli/session.ts
+const AB = createRequire(import.meta.url).resolve("agent-browser/bin/agent-browser.js");
+/**
+* Run agent-browser attached to the user's terminal (no timeout, inherited
+* stdio) so a human can complete an interactive login during `bootstrap`.
+* Distinct from runtime/spawn-ab.ts, which pipes stdio and hard-times-out for
+* non-interactive automation.
+*/
+function runAbInteractive(args) {
+	return spawnSync(AB, args, { stdio: "inherit" }).status ?? 1;
+}
+function validateName(name) {
+	const parsed = SessionNameSchema.safeParse(name);
+	if (!parsed.success) {
+		error(`invalid session name "${name}": ${parsed.error.issues[0]?.message ?? "bad name"}`);
+		process.exit(2);
+	}
+	return parsed.data;
+}
+const profileOption = ["--profile <name>", "Sessions bucket to read/write (.ccqa/sessions/<profile>/). Defaults to 'default'."];
+const bootstrapCommand = new Command("bootstrap").description("Open a headed browser so you can log in by hand, then save the resulting session (cookies + localStorage) for `session:` specs to restore. The saved file holds live auth cookies — keep .ccqa/sessions/ gitignored.").argument("<name>", "Session name to save (a slug; resolves to .ccqa/sessions/<profile>/<name>.json)").option("--url <url>", "URL to open first (e.g. the login page). Omit to start with a blank tab.").option(...profileOption).option("--cwd <path>", "Project root containing .ccqa/ (defaults to the current directory).").action(async (rawName, opts) => {
+	const name = validateName(rawName);
+	const cwd = resolveCwd(opts.cwd);
+	const dest = sessionFilePath(name, opts.profile, cwd);
+	header("session bootstrap", name);
+	meta("profile", opts.profile ?? "default");
+	meta("save to", dest);
+	blank();
+	const openArgs = [
+		"--headed",
+		"open",
+		...opts.url ? [opts.url] : ["about:blank"]
+	];
+	info("opening a browser — log in by hand, then return here.");
+	const openStatus = runAbInteractive(openArgs);
+	if (openStatus !== 0) {
+		error(`agent-browser open exited ${openStatus}`);
+		process.exit(1);
+	}
+	const rl = createInterface$1({
+		input: process.stdin,
+		output: process.stdout
+	});
+	await rl.question("\nPress Enter once you are fully logged in to save the session… ");
+	rl.close();
+	await mkdir(dirname(dest), { recursive: true });
+	const saveStatus = runAbInteractive([
+		"state",
+		"save",
+		dest
+	]);
+	runAbInteractive(["close"]);
+	if (saveStatus !== 0) {
+		error(`agent-browser state save exited ${saveStatus}`);
+		process.exit(1);
+	}
+	blank();
+	info(`saved session "${name}" → ${dest}`);
+	hint("reference it from a spec with:  session: " + name);
+});
+const lsCommand = new Command("ls").description("List saved sessions for a profile (names + last-saved times). No secret values are shown.").option(...profileOption).option("--cwd <path>", "Project root containing .ccqa/ (defaults to the current directory).").action(async (opts) => {
+	const cwd = resolveCwd(opts.cwd);
+	const profile = opts.profile ?? "default";
+	const dir = sessionsDir(profile, cwd);
+	header("sessions", profile);
+	let entries;
+	try {
+		entries = (await readdir(dir)).filter((f) => f.endsWith(".json"));
+	} catch {
+		entries = [];
+	}
+	if (entries.length === 0) {
+		info(`no saved sessions in ${dir}`);
+		hint("create one with:  ccqa session bootstrap <name>");
+		return;
+	}
+	for (const file of entries.sort()) {
+		const info = await stat(join(dir, file));
+		meta(file.replace(/\.json$/, ""), `saved ${info.mtime.toISOString()}`);
+	}
+});
+const sessionCommand = new Command("session").description("Manage saved browser sessions (cookies + localStorage) for `session:` specs.").addCommand(bootstrapCommand).addCommand(lsCommand);
+//#endregion
 //#region src/cli/index.ts
 function resolvePackageJson() {
 	const distCandidate = fileURLToPath(new URL("../package.json", import.meta.url));
@@ -10203,6 +10396,7 @@ program.addCommand(perspectivesCommand);
 program.addCommand(recordCommand);
 program.addCommand(runCommand);
 program.addCommand(driftCommand);
+program.addCommand(sessionCommand);
 program.parse();
 //#endregion
 export {};

package/dist/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ccqa",
-  "version": "0.10.1",
+  "version": "0.10.2",
   "type": "module",
   "description": "Browser test recorder powered by Claude Code and agent-browser",
   "repository": {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ccqa",
-  "version": "0.10.1",
+  "version": "0.10.2",
   "type": "module",
   "description": "Browser test recorder powered by Claude Code and agent-browser",
   "repository": {