pi-ui-extend 1.0.20 → 1.0.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -247,13 +247,15 @@ For an oh-my-openagent-style workflow, run `/ultrawork` or `/ulw` to ask the par
247
247
 
248
248
  ### Private browser QA and project auth
249
249
 
250
- The built-in `browser-qa` role runs on `antigravity/gemini-3-flash-preview`, with
251
- `openai-codex/gpt-5.4-mini` as its fallback. Its browser workflow is an explicit
252
- private skill under `src/async-subagents/private-skills/`, outside normal Pi skill
253
- discovery. The role's first-class `isolatedSkills` setting launches the child with
254
- `--no-skills` plus the private workflow and the packaged `skills/playwright-cli`
255
- skill. Both are mandatory even when configuration adds other isolated skills;
256
- parent and ordinary sub-agent sessions do not discover the private workflow.
250
+ The built-in `browser-qa` role runs on `openai-codex/gpt-5.4-mini`, with
251
+ `antigravity/gemini-3-flash-preview` and then `zai/glm-5.3` as fallbacks. Its browser
252
+ workflow is an explicit private skill under `src/async-subagents/private-skills/`,
253
+ outside normal Pi skill discovery. The role's first-class `isolatedSkills` setting launches the child with
254
+ `--no-skills` plus one self-contained private workflow. It bundles the relevant
255
+ scenario-design, locator, waiting, assertion, evidence, and cleanup guidance next
256
+ to its trusted runner, so browser QA does not depend on a separately installed
257
+ skill or CLI. The private workflow remains mandatory when configuration appends
258
+ other isolated skills; parent and ordinary sub-agent sessions do not discover it.
257
259
 
258
260
  Keep named dev/staging auth profiles in project `.pi/qa_auth.jsonc` (there is no
259
261
  `/qa-auth` command). The private runner supports `form`, `cookie`, `localStorage`,
@@ -286,15 +288,23 @@ blocked. Example:
286
288
  }
287
289
  ```
288
290
 
289
- Do not place credential values in prompts, QA flows, shell arguments, reports,
290
- or evidence. The helper reads JSONC internally, emits only redacted statuses, and
291
- caches generated storage state under `.pi/qa-auth-state`. Evidence is isolated by
292
- run/profile under `.pi/qa-runs`; trace archives have network records and non-image
293
- resources removed, then known configured/runtime credential values are redacted
294
- and verified before the trace is retained. Missing, ambiguous, rejected, or
295
- expired auth returns `QA_PROFILE_REQUIRED` or `QA_AUTH_UPDATE_REQUIRED`, naming
296
- only the profile/file/reason needed for the parent to ask the user for an update
297
- and rerun. See
291
+ Do not place credential values in prompts, QA flows, shell arguments, or reports.
292
+ The helper reads JSONC internally and emits only redacted statuses. For form auth,
293
+ video recording begins on the login page and captures the field-filling and submit
294
+ sequence; password inputs remain browser-masked, but the private video may show
295
+ other visible login identifiers and must be treated as sensitive evidence. Tracing
296
+ starts only after login succeeds and is sanitized before retention. The launcher
297
+ provides each browser QA process with its own
298
+ `.pi/subagents/<run>/<agent-id>/browser-qa/` workspace. Declarative flows,
299
+ screenshots, video, sanitized traces, and result manifests stay there, so normal
300
+ session shutdown or `subagents cleanup` removes them with the run directory.
301
+ The runner validates the owning agent metadata and refuses flows outside that
302
+ workspace; reusing an agent id clears stale browser QA files first. Trace archives
303
+ have network records and non-image resources removed, then known
304
+ configured/runtime credential values are redacted and verified before retention.
305
+ Missing, ambiguous, rejected, or expired auth returns `QA_PROFILE_REQUIRED` or
306
+ `QA_AUTH_UPDATE_REQUIRED`, naming only the profile/file/reason needed for the
307
+ parent to ask the user for an update and rerun. See
298
308
  `src/async-subagents/private-skills/browser-qa/references/qa-auth.example.jsonc`
299
309
  for complete profile shapes and `references/qa-flow.example.jsonc` beside it for
300
310
  the declarative, non-executable QA action/assertion format.
@@ -17,9 +17,10 @@ evidence. The role uses `antigravity/gemini-3-flash-preview`, falling back to
17
17
  Antigravity-backed role unavailable.
18
18
  - A type profile may declare `isolatedSkills`. Spawning that profile adds
19
19
  `--no-skills` followed by one explicit `--skill` per configured path.
20
- - The `browser-qa` profile always loads both its private workflow and the
21
- packaged `skills/playwright-cli` skill. Configuration may append isolated
22
- skills but cannot remove either mandatory browser skill.
20
+ - The `browser-qa` profile always loads one self-contained private workflow.
21
+ Relevant browser-test design guidance is bundled beside its trusted runner;
22
+ no separately discovered skill or browser CLI is required. Configuration may
23
+ append isolated skills but cannot remove the mandatory private workflow.
23
24
  - Other sub-agent profiles and the parent session must not discover the private
24
25
  skill automatically.
25
26
 
@@ -40,9 +41,11 @@ evidence. The role uses `antigravity/gemini-3-flash-preview`, falling back to
40
41
  - The bundled runner reads secrets internally. Credentials must never be copied
41
42
  into prompts, generated QA flows, shell arguments, transcripts, reports,
42
43
  or QA evidence.
43
- - Generated browser state is private cache under `.pi/qa-auth-state`; evidence
44
- is written under `.pi/qa-runs`. Multiple profiles always use separate browser
45
- contexts and evidence directories.
44
+ - Generated browser state is private cache under `.pi/qa-auth-state`. Ephemeral
45
+ flows, evidence, and result manifests are written under the owning agent's
46
+ `.pi/subagents/<run>/<agent-id>/browser-qa/` workspace. Multiple profiles use
47
+ separate browser contexts/evidence directories, and normal sub-agent shutdown
48
+ or cleanup removes the whole workspace with its run.
46
49
  - Missing, ambiguous, rejected, or expired auth returns a machine-readable
47
50
  update-required/profile-required status naming only the profile id, config
48
51
  file, and redacted reason. The parent asks the user to update the file and
@@ -54,6 +57,10 @@ evidence. The role uses `antigravity/gemini-3-flash-preview`, falling back to
54
57
  trusted runner implements a bounded set of navigation, interaction,
55
58
  assertion, screenshot, and auth-rejection actions and never gives the flow a
56
59
  Playwright context or credential values.
60
+ - The launcher injects `PI_SUBAGENT_AGENT_DIR`, pre-creates a private
61
+ `browser-qa/flows/` workspace, and clears stale browser QA files when an agent
62
+ id is reused. The runner validates the directory's project/type metadata and
63
+ refuses flows outside it; the model cannot select a shared evidence root.
57
64
  - The runner owns browser lifecycle, origin checks, auth application, tracing,
58
65
  screenshots, video finalization, and redacted result output. Before retaining
59
66
  a trace it removes network/non-image resource entries, redacts configured and
@@ -69,10 +76,10 @@ evidence. The role uses `antigravity/gemini-3-flash-preview`, falling back to
69
76
 
70
77
  ## Acceptance criteria
71
78
 
72
- 1. `browser-qa` resolves to the intended model/fallback, private workflow, and
73
- packaged `playwright-cli` skill, and its isolated child process can register
74
- the configured Antigravity model.
75
- 2. Spawn args contain `--no-skills` and both mandatory explicit skills for this
79
+ 1. `browser-qa` resolves to the intended model/fallback and its self-contained
80
+ private workflow, and its isolated child process can register the configured
81
+ Antigravity model.
82
+ 2. Spawn args contain `--no-skills` and the mandatory private skill for this
76
83
  profile; ordinary profiles retain existing skill discovery behavior.
77
84
  3. Auth profile listing and all error output are redacted; model-authored input
78
85
  cannot execute code in the credential-bearing process.
@@ -80,9 +87,11 @@ evidence. The role uses `antigravity/gemini-3-flash-preview`, falling back to
80
87
  origins, path/mode hardening, private empty-template creation, explicit
81
88
  credential requests, non-executable flows, and successful redacted evidence
82
89
  creation.
83
- 5. Completed test runs report clickable screenshot, video, and trace links
90
+ 5. Browser QA flows/evidence live only inside the owning sub-agent directory;
91
+ deleting the run removes them while persistent auth config/state remains.
92
+ 6. Completed test runs report clickable screenshot, video, and trace links
84
93
  whenever those artifacts exist.
85
- 6. Suite tests/typecheck, host checks, and suite sync pass.
94
+ 7. Suite tests/typecheck, host checks, and suite sync pass.
86
95
 
87
96
  ## Real-browser regression test
88
97
 
@@ -96,6 +105,8 @@ npm run test:browser-qa-e2e
96
105
  ```
97
106
 
98
107
  Normal suite tests keep this case skipped; the Publish workflow runs it on
99
- Linux after installing Chromium. Explicit runs retain the latest artifacts in
100
- `.pi/qa-runs/browser-qa-e2e/latest/` and print a clickable link for every file.
101
- Set `BROWSER_QA_KEEP_EVIDENCE=0` to clean up without publishing evidence.
108
+ Linux after installing Chromium. The runner writes into a temporary simulated
109
+ sub-agent directory. For manual inspection only, explicit E2E runs copy the
110
+ latest artifacts to `.pi/qa-runs/browser-qa-e2e/latest/` and print clickable
111
+ links; this test-only published copy is not the runtime storage contract. Set
112
+ `BROWSER_QA_KEEP_EVIDENCE=0` to skip that copy.
@@ -116,7 +116,7 @@
116
116
  "research": { "model": "zai/glm-5-turbo", "thinking": "low" },
117
117
  "docs": { "model": "zai/glm-4.5-air", "thinking": "low" },
118
118
  "frontend": { "model": "antigravity/gemini-3-flash-preview", "fallbackModels": ["zai/glm-5.3"], "thinking": "medium" },
119
- "browser-qa": { "model": "antigravity/gemini-3-flash-preview", "fallbackModels": ["openai-codex/gpt-5.4-mini"], "thinking": "medium" },
119
+ "browser-qa": { "model": "openai-codex/gpt-5.4-mini", "fallbackModels": ["antigravity/gemini-3-flash-preview", "zai/glm-5.3"], "thinking": "medium" },
120
120
  "tests": { "model": "zai/glm-5-turbo", "thinking": "medium" },
121
121
  "review": { "model": "zai/glm-5.3", "thinking": "high" },
122
122
  "implement": { "model": "zai/glm-5.3", "thinking": "high" },
@@ -132,7 +132,7 @@
132
132
  "research": { "model": "openai-codex/gpt-5.6-terra", "fallbackModels": ["zai/glm-5-turbo"], "thinking": "low" },
133
133
  "docs": { "model": "openai-codex/gpt-5.6-luna", "fallbackModels": ["zai/glm-4.5-air"], "thinking": "low" },
134
134
  "frontend": { "model": "openai-codex/gpt-5.6-terra", "fallbackModels": ["antigravity/gemini-3-flash-preview", "zai/glm-5.3"], "thinking": "medium" },
135
- "browser-qa": { "model": "antigravity/gemini-3-flash-preview", "fallbackModels": ["openai-codex/gpt-5.4-mini"], "thinking": "medium" },
135
+ "browser-qa": { "model": "openai-codex/gpt-5.4-mini", "fallbackModels": ["antigravity/gemini-3-flash-preview", "zai/glm-5.3"], "thinking": "medium" },
136
136
  "tests": { "model": "openai-codex/gpt-5.6-terra", "fallbackModels": ["zai/glm-5-turbo"], "thinking": "medium" },
137
137
  "review": { "model": "openai-codex/gpt-5.6-sol", "fallbackModels": ["zai/glm-5.3"], "thinking": "high" },
138
138
  "implement": { "model": "openai-codex/gpt-5.6-sol", "fallbackModels": ["zai/glm-5.3"], "thinking": "high" },
@@ -148,7 +148,7 @@
148
148
  "research": { "model": "antigravity/gemini-3.1-pro-preview", "fallbackModels": ["openai-codex/gpt-5.4-mini", "zai/glm-5-turbo"], "thinking": "medium" },
149
149
  "docs": { "model": "antigravity/gemini-2.5-flash", "fallbackModels": ["openai-codex/gpt-5.3-codex-spark", "zai/glm-4.5-air"], "thinking": "medium" },
150
150
  "frontend": { "model": "antigravity/gemini-3.1-pro-preview-customtools", "fallbackModels": ["openai-codex/gpt-5.4-mini", "zai/glm-5.3"], "thinking": "low" },
151
- "browser-qa": { "model": "antigravity/gemini-3-flash-preview", "fallbackModels": ["openai-codex/gpt-5.4-mini"], "thinking": "medium" },
151
+ "browser-qa": { "model": "openai-codex/gpt-5.4-mini", "fallbackModels": ["antigravity/gemini-3-flash-preview", "zai/glm-5.3"], "thinking": "medium" },
152
152
  "tests": { "model": "antigravity/antigravity-claude-sonnet-4-6", "fallbackModels": ["openai-codex/gpt-5.4-mini", "zai/glm-5-turbo"], "thinking": "high" },
153
153
  "review": { "model": "antigravity/antigravity-claude-sonnet-4-6", "fallbackModels": ["openai-codex/gpt-5.6-sol", "zai/glm-5.3"], "thinking": "high" },
154
154
  "implement": { "model": "openai-codex/gpt-5.6-sol", "fallbackModels": ["zai/glm-5.3"], "thinking": "high" },
@@ -199,8 +199,8 @@
199
199
 
200
200
  "browser-qa": {
201
201
  "description": "Use for browser-based visual QA: reproduce UI bugs and verify fixes with deterministic assertions, screenshots, video, and traces.",
202
- "model": "antigravity/gemini-3-flash-preview",
203
- "fallbackModels": ["openai-codex/gpt-5.4-mini"],
202
+ "model": "openai-codex/gpt-5.4-mini",
203
+ "fallbackModels": ["antigravity/gemini-3-flash-preview", "zai/glm-5.3"],
204
204
  "thinking": "medium",
205
205
  "tools": ["read", "grep", "bash"]
206
206
  },
@@ -207,8 +207,8 @@ const BUILTIN_CONFIG: SubagentConfig = {
207
207
  },
208
208
  "browser-qa": {
209
209
  description: "Use for browser-based visual QA: reproduce UI bugs and verify fixes with deterministic assertions, screenshots, video, and traces.",
210
- model: "antigravity/gemini-3-flash-preview",
211
- fallbackModels: ["openai-codex/gpt-5.4-mini"],
210
+ model: "openai-codex/gpt-5.4-mini",
211
+ fallbackModels: ["antigravity/gemini-3-flash-preview", "zai/glm-5.3"],
212
212
  thinking: "medium",
213
213
  tools: ["read", "grep", "bash"],
214
214
  isolatedSkills: [getBrowserQaSkillPath()],
@@ -288,21 +288,6 @@ export function getBrowserQaSkillPath(): string {
288
288
  return path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "private-skills", "browser-qa", "SKILL.md");
289
289
  }
290
290
 
291
- export function getPlaywrightCliSkillPath(): string {
292
- const bundledPath = path.resolve(
293
- path.dirname(fileURLToPath(import.meta.url)),
294
- "../../../../..",
295
- "skills",
296
- "playwright-cli",
297
- "SKILL.md",
298
- );
299
- const installedPaths = [
300
- path.join(os.homedir(), ".agents", "skills", "playwright-cli", "SKILL.md"),
301
- path.join(os.homedir(), ".pi", "agent", "skills", "playwright-cli", "SKILL.md"),
302
- ];
303
- return [bundledPath, ...installedPaths].find((candidate) => fs.existsSync(candidate)) ?? bundledPath;
304
- }
305
-
306
291
  export function getSubagentConfigInitTargetPath(cwd: string, env: NodeJS.ProcessEnv = process.env): string {
307
292
  return explicitSubagentConfigPath(cwd, env) ?? getDefaultSubagentConfigPath();
308
293
  }
@@ -417,7 +402,7 @@ export function resolveAgentTaskConfig(
417
402
  function resolveIsolatedSkills(selectedType: string | undefined, profile: SubagentTypeConfig | undefined): string[] {
418
403
  const configured = arrayOfStrings(profile?.isolatedSkills) ?? [];
419
404
  if (selectedType !== "browser-qa") return configured;
420
- return [...new Set([getBrowserQaSkillPath(), getPlaywrightCliSkillPath(), ...configured])];
405
+ return [...new Set([getBrowserQaSkillPath(), ...configured])];
421
406
  }
422
407
 
423
408
  export function resolveSubagentRoutingConfig(config: SubagentConfig): ResolvedSubagentRoutingConfig {
@@ -24,6 +24,8 @@ export interface SpawnAgentOptions {
24
24
  }
25
25
 
26
26
  export const DEFAULT_AGENT_TIMEOUT_MS = 30 * 60 * 1000;
27
+ const BROWSER_QA_WORKSPACE_DIR = "browser-qa";
28
+ const SUBAGENT_AGENT_DIR_ENV = "PI_SUBAGENT_AGENT_DIR";
27
29
  const AGENT_TIMEOUT_EXIT_CODE = 124;
28
30
  const AGENT_TIMEOUT_KILL_GRACE_MS = 5_000;
29
31
  const AGENT_SETTLED_TERMINATE_GRACE_MS = 50;
@@ -46,6 +48,7 @@ export function spawnAgent(
46
48
  validateBasename(task.id, "task.id");
47
49
  const agentDir = path.join(runDir, task.id);
48
50
  fs.mkdirSync(agentDir, { recursive: true });
51
+ prepareBrowserQaWorkspace(agentDir, task.subagentType);
49
52
 
50
53
  // Clean previous state when reusing a run directory/agent id.
51
54
  for (const f of [
@@ -136,7 +139,7 @@ export function spawnAgent(
136
139
 
137
140
  const proc = spawn(invocation.command, invocation.args, {
138
141
  cwd,
139
- env: subagentEnvironment(process.env),
142
+ env: subagentEnvironment(process.env, task.subagentType === "browser-qa" ? agentDir : undefined),
140
143
  stdio: ["pipe", "pipe", "pipe"],
141
144
  });
142
145
  proc.stdin.on("error", (error: NodeJS.ErrnoException) => {
@@ -698,8 +701,8 @@ function getEnvModel(): string | undefined {
698
701
  return trimmed ? trimmed : undefined;
699
702
  }
700
703
 
701
- function subagentEnvironment(env: NodeJS.ProcessEnv): NodeJS.ProcessEnv {
702
- return {
704
+ function subagentEnvironment(env: NodeJS.ProcessEnv, agentDir?: string): NodeJS.ProcessEnv {
705
+ const result: NodeJS.ProcessEnv = {
703
706
  ...env,
704
707
  PI_MODEL_SUITABLE_TOOLS_PRESERVE_SELECTION: "1",
705
708
  PI_TERMINAL_BELL_DISABLED: "1",
@@ -709,6 +712,21 @@ function subagentEnvironment(env: NodeJS.ProcessEnv): NodeJS.ProcessEnv {
709
712
  "question",
710
713
  ]),
711
714
  };
715
+ delete result[SUBAGENT_AGENT_DIR_ENV];
716
+ if (agentDir) result[SUBAGENT_AGENT_DIR_ENV] = fs.realpathSync(agentDir);
717
+ return result;
718
+ }
719
+
720
+ function prepareBrowserQaWorkspace(agentDir: string, subagentType: string | undefined): void {
721
+ const workspace = path.join(agentDir, BROWSER_QA_WORKSPACE_DIR);
722
+ fs.rmSync(workspace, { recursive: true, force: true });
723
+ if (subagentType !== "browser-qa") return;
724
+ const flows = path.join(workspace, "flows");
725
+ fs.mkdirSync(flows, { recursive: true, mode: 0o700 });
726
+ if (process.platform !== "win32") {
727
+ fs.chmodSync(workspace, 0o700);
728
+ fs.chmodSync(flows, 0o700);
729
+ }
712
730
  }
713
731
 
714
732
  function appendEnvList(value: string | undefined, items: readonly string[]): string {
@@ -24,7 +24,6 @@ export {
24
24
  existingSubagentConfigFiles,
25
25
  getDefaultSubagentConfigPath,
26
26
  getBrowserQaSkillPath,
27
- getPlaywrightCliSkillPath,
28
27
  getSubagentConfigInitTargetPath,
29
28
  getSubagentConfigSamplePath,
30
29
  isBlindModelRef,
@@ -1,31 +1,71 @@
1
1
  ---
2
2
  name: browser-qa-private
3
- description: Private workflow for deterministic browser bug reproduction and fix verification with redacted project auth and Playwright evidence.
3
+ description: Private self-contained workflow for deterministic browser bug reproduction and fix verification with redacted project auth and Playwright evidence.
4
4
  ---
5
5
 
6
6
  # Browser QA
7
7
 
8
- Use the bundled runner; do not read, print, grep, copy, or edit credential values
9
- from `.pi/qa_auth.jsonc` yourself.
8
+ Use this skill's bundled runner as the only browser interface. It already owns
9
+ Playwright, browser/context lifecycle, tracing, video, screenshots, origin
10
+ isolation, authentication, redaction, and cleanup. Do not invoke another browser
11
+ CLI, create shared/default browser sessions, or generate executable browser code.
12
+
13
+ Never read, print, grep, copy, or edit credential values from
14
+ `.pi/qa_auth.jsonc` yourself.
10
15
 
11
16
  ## Workflow
12
17
 
13
18
  1. Resolve `scripts/browser-qa-runner.mjs` relative to this skill.
14
- 2. Run `node <runner> profiles`. Choose a profile only when the task names its
19
+ 2. Use the launcher-provided `$PI_SUBAGENT_AGENT_DIR/browser-qa/` workspace.
20
+ The launcher creates its private `flows/` directory and the runner rejects
21
+ flows or evidence destinations outside this owning sub-agent directory. Do
22
+ not override `PI_SUBAGENT_AGENT_DIR` or copy evidence to shared project paths.
23
+ 3. Discover the requested target, expected behavior, and the smallest scenario
24
+ that can prove it. If the target cannot be reached or started, report the
25
+ concrete blocker instead of substituting static checks for browser QA.
26
+ 4. Run `node <runner> profiles`. Choose a profile only when the task names its
15
27
  id or safe profile traits make the choice unambiguous. Otherwise stop with
16
28
  `QA_PROFILE_REQUIRED` and list only ids, descriptions, and traits.
17
- 3. Inspect the target code and write a declarative JSONC flow under
18
- `.pi/qa-flows/`. Never put credentials or executable JavaScript in it.
19
- 4. Run:
29
+ 5. Inspect the target code and write a declarative JSONC flow under
30
+ `$PI_SUBAGENT_AGENT_DIR/browser-qa/flows/`. Never put credentials or
31
+ executable JavaScript in it.
32
+ 6. Run:
20
33
  `node <runner> run --profile <id> --base-url <url> --flow <flow.jsonc>`.
21
34
  Profile id, URL, and flow path are non-secret; never pass credentials as
22
35
  arguments or environment variables.
23
- 5. Report deterministic assertions and every artifact returned by the runner.
36
+ 7. Report deterministic assertions and every artifact returned by the runner.
24
37
  For each screenshot, video, or trace, emit a separate clickable Markdown
25
38
  link using its `uri` and also show its absolute `path`. Do this for failed
26
39
  runs too whenever `artifacts` is present; never report only `evidenceDir`.
27
40
  Visual inspection supplements assertions; it does not replace them.
28
41
 
42
+ ## Scenario design
43
+
44
+ - Define the expected postcondition before writing interactions. A successful
45
+ click or navigation is not proof; assert the resulting URL, text, value,
46
+ count, visibility, enabled state, or checked state.
47
+ - Keep the flow minimal and reproducible. Capture setup, the action under test,
48
+ and at least one observable outcome; add a screenshot at the state that best
49
+ explains the result.
50
+ - Prefer stable user-facing locators in this order: `testId`; semantic `role`
51
+ plus accessible `name`; `label`; `placeholder`; visible `text`; CSS only as a
52
+ last resort. Use `exact: true` when similar elements could make a match
53
+ ambiguous.
54
+ - Let locator actions auto-wait. Use `waitFor` for an explicit UI state and use
55
+ `waitForTimeout` only for a short, unavoidable animation/debounce—not as a
56
+ substitute for an assertion. Set flow `timeoutMs` only as high as the target
57
+ legitimately needs.
58
+ - Place `authRejectedIf` immediately after navigation or any transition that
59
+ may reveal expired authentication.
60
+ - Never weaken an assertion merely to make a failing run pass. If the observed
61
+ product behavior differs from the expectation, preserve the failure evidence
62
+ and report the mismatch.
63
+
64
+ Read `references/qa-design.md` when designing a non-trivial flow, diagnosing an
65
+ ambiguous failure, or deciding what evidence proves the result.
66
+
67
+ ## Flow contract
68
+
29
69
  The flow is `{ "steps": [...] }` with at most 100 steps. Supported actions:
30
70
 
31
71
  - navigation: `goto`, `reload`, `waitFor`, `waitForTimeout`
@@ -50,16 +90,23 @@ assertions require exactly one of `equals` or `includes`.
50
90
  "locator": { "role": "heading", "name": "Settings", "exact": true }
51
91
  },
52
92
  { "action": "click", "locator": { "testId": "save-settings" } },
53
- { "action": "assertText", "locator": { "testId": "toast" }, "includes": "Saved" }
93
+ { "action": "assertText", "locator": { "testId": "toast" }, "includes": "Saved" },
94
+ { "action": "screenshot", "name": "settings-saved" }
54
95
  ]
55
96
  }
56
97
  ```
57
98
 
58
- The runner owns Playwright/browser lifecycle, strict network-origin boundaries,
59
- auth application, assertions, screenshots, video, trace sanitization, and
60
- cleanup. Do not start an additional shared/default browser session. For multiple
61
- profiles, invoke the runner separately; each invocation gets an isolated context
62
- and exclusive evidence directory.
99
+ For multiple profiles, invoke the runner separately. Every invocation gets an
100
+ isolated browser context and exclusive evidence directory; the runner closes
101
+ all owned browser resources on success and failure. Flows, screenshots, video,
102
+ sanitized traces, and runner result manifests remain under
103
+ `$PI_SUBAGENT_AGENT_DIR/browser-qa/` so normal sub-agent shutdown or cleanup
104
+ deletes them with the run directory. For form auth, recording starts on the login
105
+ page and includes field filling and submission; password inputs remain masked,
106
+ but the private video may show visible login identifiers. Tracing starts only
107
+ after login succeeds so credentials are not captured in the trace.
108
+
109
+ ## Credentials and blocked runs
63
110
 
64
111
  If the runner returns `QA_AUTH_UPDATE_REQUIRED`, stop and explicitly report that
65
112
  browser QA requires credentials or an auth-config update. Ask the user to fill
@@ -68,9 +115,13 @@ empty template was created at that path. Relay only the runner's profile, file,
68
115
  reason, action, and template-created state; never read the generated file or
69
116
  attempt to recover by exposing or replaying credentials.
70
117
 
118
+ For any other blocked run, report the runner status and redacted reason. Do not
119
+ claim that browser QA passed based on source inspection, unit tests, or a build.
120
+
71
121
  After any runner invocation that actually performed browser testing, include
72
122
  all non-empty `artifacts.screenshots`, `artifacts.videos`, and
73
123
  `artifacts.traces` groups in the final response. These links are mandatory so
74
124
  the user can open the evidence directly.
75
125
 
76
- See `references/qa-auth.example.jsonc` and `references/qa-flow.example.jsonc`.
126
+ See `references/qa-auth.example.jsonc`, `references/qa-flow.example.jsonc`, and
127
+ `references/qa-design.md`.
@@ -0,0 +1,109 @@
1
+ # Designing deterministic browser QA flows
2
+
3
+ Use this reference with the bundled declarative runner. It intentionally does
4
+ not describe a separate browser CLI or executable Playwright scripts.
5
+
6
+ ## Build the proof before the steps
7
+
8
+ Write down three things first:
9
+
10
+ 1. **Setup:** the page and state needed to expose the behavior.
11
+ 2. **Action:** the smallest user interaction that exercises it.
12
+ 3. **Oracle:** the observable state that proves success or reproduces failure.
13
+
14
+ Good oracles are product-visible and specific: an exact URL, a stable status
15
+ message, a field value, item count, enabled/disabled state, or checked state.
16
+ Avoid treating “the click did not throw” or “the screenshot looks plausible” as
17
+ proof.
18
+
19
+ When verifying a fix, prefer a focused regression flow over a broad tour of the
20
+ application. If multiple independent states matter, assert each one explicitly.
21
+
22
+ ## Choose resilient locators
23
+
24
+ Prefer locators that match how users and accessibility APIs identify controls:
25
+
26
+ 1. `testId` when the product exposes a stable test contract.
27
+ 2. `role` with accessible `name` for buttons, links, headings, dialogs, and
28
+ similar semantic elements.
29
+ 3. `label` for form controls.
30
+ 4. `placeholder` or visible `text` when they are stable product copy.
31
+ 5. `css` only when no semantic contract exists.
32
+
33
+ Use `exact: true` when duplicate or substring matches are possible. Avoid CSS
34
+ that encodes DOM depth, generated classes, styling details, or element order.
35
+ If a locator is ambiguous, inspect nearby source or rendered copy and choose a
36
+ more specific product contract rather than adding arbitrary delays.
37
+
38
+ ## Wait for state, not time
39
+
40
+ Runner interactions inherit Playwright auto-waiting. Usually an action followed
41
+ by an assertion is enough. Use `waitFor` only when the next operation depends on
42
+ a distinct attached/detached/visible/hidden transition.
43
+
44
+ `waitForTimeout` is bounded to five seconds and should be exceptional—for a
45
+ known animation, debounce, or externally scheduled transition with no
46
+ observable intermediate state. Sleeping longer hides races instead of proving
47
+ behavior. If a normal operation legitimately needs more time, adjust the flow's
48
+ `timeoutMs` rather than inserting repeated sleeps.
49
+
50
+ ## Authentication transitions
51
+
52
+ Add `authRejectedIf` directly after initial navigation and after transitions
53
+ that can redirect to login or display an expired-session marker. This converts
54
+ stale credentials into an explicit update request instead of misreporting a
55
+ product regression.
56
+
57
+ Do not encode credentials, tokens, storage values, or login form secrets in the
58
+ flow. The trusted runner applies the selected profile internally. For form auth,
59
+ video starts on the login page and includes field filling and submission; password
60
+ inputs remain masked, but visible identifiers can appear, so treat the video as
61
+ sensitive private evidence. Tracing starts only after login succeeds and is
62
+ sanitized before retention.
63
+
64
+ ## Evidence strategy
65
+
66
+ The runner always attempts a final or failure screenshot, records video from the
67
+ first page, and creates a sanitized post-auth trace. Add named `screenshot`
68
+ steps only at states that materially help explain the result—for example before
69
+ and after a destructive interaction, or when a transient success message is
70
+ the oracle.
71
+
72
+ Use evidence by purpose:
73
+
74
+ - **Screenshot:** quick review of one meaningful visual state.
75
+ - **Video:** chronological confirmation of the complete user flow.
76
+ - **Trace:** action/DOM timing diagnosis for a failed or flaky interaction.
77
+
78
+ Assertions determine pass/fail; evidence explains it. Preserve and link every
79
+ artifact group returned on both passed and failed runs.
80
+
81
+ ## Diagnose failures without weakening the test
82
+
83
+ Classify the first failing step:
84
+
85
+ - wrong target/setup or service unavailable;
86
+ - authentication rejected or expired;
87
+ - locator no longer matches the product contract;
88
+ - expected state never appeared;
89
+ - actual product behavior contradicts the expectation.
90
+
91
+ Fix the flow only when its setup or locator is wrong. Do not replace a precise
92
+ assertion with a vague one, increase timeouts reflexively, or remove the failing
93
+ step to manufacture a pass. Keep the failure artifacts and state the expected
94
+ versus observed behavior.
95
+
96
+ ## Cleanup and isolation
97
+
98
+ Each runner invocation owns one isolated context and evidence directory and
99
+ closes its browser resources in a `finally` path. Do not create parallel shared
100
+ or default sessions outside the runner. Test multiple auth profiles with
101
+ separate invocations so cookies, storage, traces, and evidence cannot mix.
102
+
103
+ Keep the declarative flow and every generated screenshot, video, trace, and
104
+ result manifest inside `$PI_SUBAGENT_AGENT_DIR/browser-qa/`. The launcher owns
105
+ that path and the runner validates it before opening a browser. Do not override
106
+ the environment path or copy evidence into shared `.pi/qa-runs`/`.pi/qa-flows`
107
+ directories: the agent-local workspace is intentionally removed by the normal
108
+ sub-agent shutdown and cleanup lifecycle. Authentication config remains a
109
+ separate persistent input under project `.pi/`.
@@ -1,6 +1,5 @@
1
1
  #!/usr/bin/env node
2
2
 
3
- import { createHash } from "node:crypto";
4
3
  import fs from "node:fs";
5
4
  import path from "node:path";
6
5
  import { createRequire } from "node:module";
@@ -9,8 +8,10 @@ import { parse as parseJsonc, printParseErrorCode } from "jsonc-parser";
9
8
  import { strFromU8, strToU8, unzipSync, zipSync } from "../vendor/fflate.mjs";
10
9
 
11
10
  const CONFIG_RELATIVE = ".pi/qa_auth.jsonc";
12
- const STATE_RELATIVE = path.join(".pi", "qa-auth-state");
13
- const RUNS_RELATIVE = path.join(".pi", "qa-runs");
11
+ const SUBAGENT_AGENT_DIR_ENV = "PI_SUBAGENT_AGENT_DIR";
12
+ const QA_WORKSPACE_RELATIVE = "browser-qa";
13
+ const EVIDENCE_RELATIVE = "evidence";
14
+ const FORM_VIDEO_STEP_DELAY_MS = 250;
14
15
  const EXIT_AUTH_UPDATE_REQUIRED = 42;
15
16
  const EXIT_PROFILE_REQUIRED = 43;
16
17
  const PROFILE_ID = /^[A-Za-z0-9._-]+$/;
@@ -80,23 +81,25 @@ async function main() {
80
81
  const profile = selected.profile;
81
82
  const secrets = collectSecrets(profile.auth);
82
83
  try {
83
- await runQa({ cwd, args, profileId, profile });
84
+ const agentDir = resolveBrowserQaAgentDirectory(cwd, process.env[SUBAGENT_AGENT_DIR_ENV]);
85
+ await runQa({ cwd, agentDir, args, profileId, profile });
84
86
  } catch (error) {
85
87
  if (error instanceof QaStatusError) throw error;
86
88
  throw new QaStatusError("QA_RUN_FAILED", redact(safeReason(error), secrets), 1, profileId);
87
89
  }
88
90
  }
89
91
 
90
- async function runQa({ cwd, args, profileId, profile }) {
92
+ async function runQa({ cwd, agentDir, args, profileId, profile }) {
91
93
  if (!args.flow) throw new QaStatusError("QA_RUN_FAILED", "--flow is required", 1, profileId);
92
- const flowPath = resolveExistingPrivateFile(cwd, args.flow, "QA flow", false);
94
+ const workspaceDir = path.join(agentDir, QA_WORKSPACE_RELATIVE);
95
+ const flowPath = resolveExistingPrivateFile(workspaceDir, args.flow, "QA flow", false);
93
96
  const flow = readFlow(flowPath, profileId);
94
97
  const allowedOrigins = normalizeAllowedOrigins(profile.allowedOrigins, profileId);
95
98
  const baseURL = normalizeBaseUrl(args.baseUrl ?? profile.baseUrl ?? allowedOrigins[0], allowedOrigins, profileId);
96
99
  validateAuthConfiguration(cwd, profile.auth, allowedOrigins, profileId);
97
100
  const runId = safeRunId(args.runId ?? `${timestamp()}-${profileId}`);
98
- const evidenceDir = path.join(cwd, RUNS_RELATIVE, runId, profileId);
99
- createExclusivePrivateDirectory(cwd, evidenceDir);
101
+ const evidenceDir = path.join(workspaceDir, EVIDENCE_RELATIVE, runId, profileId);
102
+ createExclusivePrivateDirectory(agentDir, evidenceDir);
100
103
 
101
104
  const playwright = loadPlaywright(cwd);
102
105
  const browser = await playwright.chromium.launch({ headless: true });
@@ -108,7 +111,7 @@ async function runQa({ cwd, args, profileId, profile }) {
108
111
  const runtimeSecrets = collectSecrets(profile.auth);
109
112
  const tracePath = path.join(evidenceDir, "trace.zip");
110
113
  try {
111
- const contextOptions = await contextOptionsForAuth({ cwd, profileId, profile, allowedOrigins, browser });
114
+ const contextOptions = await contextOptionsForAuth({ cwd, profileId, profile, allowedOrigins });
112
115
  context = await browser.newContext({
113
116
  ...contextOptions,
114
117
  baseURL,
@@ -118,10 +121,11 @@ async function runQa({ cwd, args, profileId, profile }) {
118
121
  });
119
122
  await installOriginGuard(context, allowedOrigins, profile.auth);
120
123
  await applyContextAuth(context, profile.auth, allowedOrigins, baseURL, profileId);
121
- runtimeSecrets.push(...collectStorageStateSecrets(await context.storageState()));
122
- await context.tracing.start({ screenshots: true, snapshots: true, sources: false });
123
124
  page = await context.newPage();
124
125
  video = page.video();
126
+ await applyFormAuth(page, profile.auth, allowedOrigins, profileId);
127
+ runtimeSecrets.push(...collectStorageStateSecrets(await context.storageState()));
128
+ await context.tracing.start({ screenshots: true, snapshots: true, sources: false });
125
129
  await executeFlow({ page, context, baseURL, evidenceDir, flow, allowedOrigins, profileId, secrets: runtimeSecrets });
126
130
  await assertPageDoesNotExposeSecrets(page, runtimeSecrets, profileId);
127
131
  await page.screenshot({ path: path.join(evidenceDir, "final.png"), fullPage: true });
@@ -461,7 +465,7 @@ function isStringArray(value) {
461
465
  return Array.isArray(value) && value.length > 0 && value.every((item) => typeof item === "string");
462
466
  }
463
467
 
464
- async function contextOptionsForAuth({ cwd, profileId, profile, allowedOrigins, browser }) {
468
+ async function contextOptionsForAuth({ cwd, profileId, profile, allowedOrigins }) {
465
469
  const auth = profile.auth;
466
470
  if (auth.type === "storageState") {
467
471
  if (typeof auth.path !== "string") throw authError(profileId, "storageState path is missing");
@@ -469,16 +473,7 @@ async function contextOptionsForAuth({ cwd, profileId, profile, allowedOrigins,
469
473
  return { storageState: filteredStorageState(statePath, allowedOrigins, profileId) };
470
474
  }
471
475
  if (auth.type === "form") {
472
- const stateDirectory = path.join(cwd, STATE_RELATIVE);
473
- const statePath = path.join(stateDirectory, `${profileId}-${authCacheKey(auth, allowedOrigins)}.json`);
474
- if (!fs.existsSync(statePath)) {
475
- ensurePrivateDirectory(cwd, stateDirectory);
476
- for (const name of fs.readdirSync(stateDirectory)) {
477
- if (name.startsWith(`${profileId}-`) && name.endsWith(".json")) fs.rmSync(path.join(stateDirectory, name), { force: true });
478
- }
479
- await createFormState({ statePath, auth, allowedOrigins, browser, profileId });
480
- }
481
- return { storageState: filteredStorageState(statePath, allowedOrigins, profileId) };
476
+ return {};
482
477
  }
483
478
  if (!["cookie", "localStorage", "sessionStorage", "bearer"].includes(auth.type)) {
484
479
  throw authError(profileId, `unsupported auth type: ${auth.type}`);
@@ -486,39 +481,36 @@ async function contextOptionsForAuth({ cwd, profileId, profile, allowedOrigins,
486
481
  return {};
487
482
  }
488
483
 
489
- async function createFormState({ statePath, auth, allowedOrigins, browser, profileId }) {
484
+ async function applyFormAuth(page, auth, allowedOrigins, profileId) {
485
+ if (auth.type !== "form") return;
490
486
  if (typeof auth.loginUrl !== "string" || !isAllowedUrl(auth.loginUrl, allowedOrigins)) {
491
487
  throw authError(profileId, "form loginUrl is missing or outside allowedOrigins");
492
488
  }
493
489
  if (!Array.isArray(auth.fields) || auth.fields.length === 0 || typeof auth.submitSelector !== "string") {
494
490
  throw authError(profileId, "form fields or submitSelector are missing");
495
491
  }
496
- const context = await browser.newContext();
497
492
  try {
498
- await installOriginGuard(context, allowedOrigins, auth);
499
- const page = await context.newPage();
500
- await page.goto(auth.loginUrl);
493
+ const timeout = Math.min(finitePositive(auth.timeoutMs) ?? 15_000, 60_000);
494
+ page.setDefaultTimeout(timeout);
495
+ page.setDefaultNavigationTimeout(timeout);
496
+ await page.goto(auth.loginUrl, { timeout });
497
+ await page.waitForTimeout(FORM_VIDEO_STEP_DELAY_MS);
501
498
  for (const field of auth.fields) {
502
499
  if (!isObject(field) || typeof field.selector !== "string" || typeof field.value !== "string") {
503
500
  throw authError(profileId, "form fields must contain selector/value strings");
504
501
  }
505
502
  await page.locator(field.selector).fill(field.value);
503
+ await page.waitForTimeout(FORM_VIDEO_STEP_DELAY_MS);
506
504
  }
507
505
  await page.locator(auth.submitSelector).click();
508
- const timeout = finitePositive(auth.timeoutMs) ?? 15_000;
509
506
  if (isObject(auth.success) && typeof auth.success.url === "string") await page.waitForURL(auth.success.url, { timeout });
510
507
  if (isObject(auth.success) && typeof auth.success.selector === "string") await page.locator(auth.success.selector).waitFor({ timeout });
511
508
  if (!isObject(auth.success) || (typeof auth.success.url !== "string" && typeof auth.success.selector !== "string")) {
512
509
  throw authError(profileId, "form success.url or success.selector is required");
513
510
  }
514
- fs.mkdirSync(path.dirname(statePath), { recursive: true, mode: 0o700 });
515
- await context.storageState({ path: statePath });
516
- fs.chmodSync(statePath, 0o600);
517
511
  } catch (error) {
518
512
  if (error instanceof QaStatusError) throw error;
519
513
  throw authError(profileId, "form login was rejected or did not reach the configured success condition");
520
- } finally {
521
- await context.close().catch(() => {});
522
514
  }
523
515
  }
524
516
 
@@ -546,10 +538,6 @@ async function applyContextAuth(context, auth, allowedOrigins, baseURL, profileI
546
538
  }
547
539
  }
548
540
 
549
- function authCacheKey(auth, allowedOrigins) {
550
- return createHash("sha256").update(JSON.stringify({ auth, allowedOrigins })).digest("hex").slice(0, 16);
551
- }
552
-
553
541
  async function installOriginGuard(context, allowedOrigins, auth) {
554
542
  await context.route("**/*", async (route) => {
555
543
  const request = route.request();
@@ -779,6 +767,45 @@ function isAllowedUrl(raw, allowedOrigins) {
779
767
  }
780
768
  }
781
769
 
770
+ function resolveBrowserQaAgentDirectory(cwd, value) {
771
+ if (typeof value !== "string" || value.length === 0) {
772
+ throw new Error(`${SUBAGENT_AGENT_DIR_ENV} is required for browser QA runs`);
773
+ }
774
+ const projectRoot = fs.realpathSync(cwd);
775
+ const subagentRoot = path.join(projectRoot, ".pi", "subagents");
776
+ const resolved = path.resolve(value);
777
+ if (!fs.existsSync(resolved)) throw new Error("browser QA agent directory is missing");
778
+ const real = fs.realpathSync(resolved);
779
+ if (!isInside(subagentRoot, real)) throw new Error("browser QA agent directory must be inside .pi/subagents");
780
+ assertNoSymlinkComponents(projectRoot, real, "browser QA agent directory");
781
+ if (!fs.statSync(real).isDirectory()) throw new Error("browser QA agent directory must be a real directory");
782
+
783
+ const promptFile = path.join(real, "prompt.md");
784
+ const projectFile = path.join(real, "project_cwd");
785
+ const typeFile = path.join(real, "subagent_type");
786
+ for (const [file, label] of [[promptFile, "prompt"], [projectFile, "project metadata"], [typeFile, "type metadata"]]) {
787
+ if (!fs.existsSync(file) || !fs.statSync(file).isFile()) throw new Error(`browser QA ${label} is missing`);
788
+ assertNoSymlinkComponents(real, file, `browser QA ${label}`);
789
+ }
790
+ const recordedProject = fs.readFileSync(projectFile, "utf8").trim();
791
+ if (!recordedProject || fs.realpathSync(recordedProject) !== projectRoot) {
792
+ throw new Error("browser QA agent directory belongs to another project");
793
+ }
794
+ if (fs.readFileSync(typeFile, "utf8").trim() !== "browser-qa") {
795
+ throw new Error("browser QA runner requires a browser-qa sub-agent directory");
796
+ }
797
+
798
+ const workspace = path.join(real, QA_WORKSPACE_RELATIVE);
799
+ if (!fs.existsSync(workspace)) throw new Error("browser QA workspace is missing");
800
+ assertNoSymlinkComponents(real, workspace, "browser QA workspace");
801
+ const workspaceStat = fs.statSync(workspace);
802
+ if (!workspaceStat.isDirectory()) throw new Error("browser QA workspace must be a real directory");
803
+ if (process.platform !== "win32" && (workspaceStat.mode & 0o077) !== 0) {
804
+ throw new Error("browser QA workspace must use private directory permissions (0700)");
805
+ }
806
+ return real;
807
+ }
808
+
782
809
  function resolveExistingPrivateFile(cwd, value, label, requirePrivate = true) {
783
810
  if (typeof value !== "string" || value.length === 0) throw new Error(`${label} path is missing`);
784
811
  const root = fs.realpathSync(cwd);
@@ -794,9 +821,13 @@ function resolveExistingPrivateFile(cwd, value, label, requirePrivate = true) {
794
821
  return real;
795
822
  }
796
823
 
797
- function assertInside(root, target, label) {
824
+ function isInside(root, target) {
798
825
  const relative = path.relative(root, target);
799
- if (!relative || (!relative.startsWith("..") && !path.isAbsolute(relative))) return;
826
+ return !relative || (!relative.startsWith("..") && !path.isAbsolute(relative));
827
+ }
828
+
829
+ function assertInside(root, target, label) {
830
+ if (isInside(root, target)) return;
800
831
  throw new Error(`${label} must be project-local`);
801
832
  }
802
833
 
@@ -843,10 +874,6 @@ function createExclusivePrivateDirectory(cwd, target) {
843
874
  createPrivateDirectory(cwd, target, true);
844
875
  }
845
876
 
846
- function ensurePrivateDirectory(cwd, target) {
847
- createPrivateDirectory(cwd, target, false);
848
- }
849
-
850
877
  function createPrivateDirectory(cwd, target, exclusive) {
851
878
  const root = fs.realpathSync(cwd);
852
879
  const resolved = path.resolve(target);
@@ -300,6 +300,13 @@ export const DEFAULT_PI_TOOLS_SUITE_CONFIG_JSONC = String.raw`{
300
300
  "When no mockup exists, choose a clear aesthetic direction and explain it briefly. Verify with targeted build/lint/tests or screenshot-relevant checks when possible."
301
301
  ]
302
302
  },
303
+ "browser-qa": {
304
+ "description": "Use for browser-based visual QA: reproduce UI bugs and verify fixes with deterministic assertions, screenshots, video, and traces.",
305
+ "model": "openai-codex/gpt-5.4-mini",
306
+ "fallbackModels": ["antigravity/gemini-3-flash-preview", "zai/glm-5.3"],
307
+ "thinking": "medium",
308
+ "tools": ["read", "grep", "bash"]
309
+ },
303
310
  "tests": {
304
311
  "description": "Use for tests: locate coverage, find gaps, run/check targeted test commands, diagnose failing tests.",
305
312
  "model": "zai/glm-5-turbo",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-ui-extend",
3
- "version": "1.0.20",
3
+ "version": "1.0.22",
4
4
  "description": "Pix: a workspace-first terminal UI for Pi with tabs, readable tool activity, voice input, and bundled agent tools.",
5
5
  "private": false,
6
6
  "repository": {