pi-ui-extend 1.0.41 → 1.0.45
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +50 -13
- package/dist/app/app.d.ts +13 -0
- package/dist/app/app.js +158 -8
- package/dist/app/cli/install.js +2 -2
- package/dist/app/commands/command-controller.d.ts +4 -0
- package/dist/app/commands/command-controller.js +11 -0
- package/dist/app/commands/command-git-actions.d.ts +25 -0
- package/dist/app/commands/command-git-actions.js +381 -0
- package/dist/app/commands/command-host.d.ts +4 -0
- package/dist/app/commands/command-host.js +5 -2
- package/dist/app/commands/command-model-actions.d.ts +1 -0
- package/dist/app/commands/command-model-actions.js +57 -35
- package/dist/app/commands/command-navigation-actions.d.ts +1 -0
- package/dist/app/commands/command-navigation-actions.js +16 -3
- package/dist/app/commands/command-registry.d.ts +2 -0
- package/dist/app/commands/command-registry.js +15 -1
- package/dist/app/commands/command-session-actions.js +9 -3
- package/dist/app/commands/reload-context-inventory.d.ts +11 -0
- package/dist/app/commands/reload-context-inventory.js +70 -0
- package/dist/app/extensions/extension-actions-controller.d.ts +1 -0
- package/dist/app/extensions/extension-actions-controller.js +6 -0
- package/dist/app/extensions/subagent-catalog-state.d.ts +9 -0
- package/dist/app/extensions/subagent-catalog-state.js +23 -0
- package/dist/app/input/autocomplete-controller.js +61 -34
- package/dist/app/input/input-action-controller.d.ts +2 -0
- package/dist/app/input/input-action-controller.js +8 -1
- package/dist/app/input/input-controller.d.ts +2 -1
- package/dist/app/input/input-controller.js +7 -2
- package/dist/app/input/prompt-enhancer-controller.js +53 -35
- package/dist/app/input/voice-controller.d.ts +41 -45
- package/dist/app/input/voice-controller.js +351 -419
- package/dist/app/model/model-usage-controller.js +17 -7
- package/dist/app/model/model-usage-status.d.ts +4 -1
- package/dist/app/model/model-usage-status.js +270 -46
- package/dist/app/popup/menu-items-controller.d.ts +13 -3
- package/dist/app/popup/menu-items-controller.js +37 -21
- package/dist/app/popup/popup-action-controller.d.ts +12 -2
- package/dist/app/popup/popup-action-controller.js +77 -24
- package/dist/app/popup/popup-menu-controller.d.ts +36 -14
- package/dist/app/popup/popup-menu-controller.js +239 -69
- package/dist/app/rendering/dcp-stats.d.ts +9 -4
- package/dist/app/rendering/dcp-stats.js +40 -425
- package/dist/app/rendering/editor-panels.js +10 -4
- package/dist/app/rendering/popup-menu-renderer.d.ts +3 -5
- package/dist/app/rendering/popup-menu-renderer.js +42 -37
- package/dist/app/rendering/render-controller.js +23 -2
- package/dist/app/rendering/status-line-renderer.d.ts +5 -0
- package/dist/app/rendering/status-line-renderer.js +41 -5
- package/dist/app/rendering/tab-line-renderer.js +26 -20
- package/dist/app/runtime.d.ts +12 -1
- package/dist/app/runtime.js +120 -12
- package/dist/app/screen/mouse-controller.d.ts +2 -0
- package/dist/app/screen/mouse-controller.js +17 -7
- package/dist/app/screen/status-controller.d.ts +4 -0
- package/dist/app/screen/status-controller.js +5 -0
- package/dist/app/session/lazy-session-manager.js +34 -0
- package/dist/app/session/session-event-controller.d.ts +1 -0
- package/dist/app/session/session-event-controller.js +10 -1
- package/dist/app/session/session-history.d.ts +1 -0
- package/dist/app/session/session-history.js +12 -1
- package/dist/app/session/session-lifecycle-controller.d.ts +7 -1
- package/dist/app/session/session-lifecycle-controller.js +16 -1
- package/dist/app/session/tabs-controller.d.ts +26 -1
- package/dist/app/session/tabs-controller.js +454 -147
- package/dist/app/subagents/subagents-files.js +60 -1
- package/dist/app/subagents/subagents-model.d.ts +1 -0
- package/dist/app/subagents/subagents-model.js +18 -2
- package/dist/app/subagents/subagents-widget-controller.d.ts +1 -0
- package/dist/app/subagents/subagents-widget-controller.js +6 -0
- package/dist/app/types.d.ts +16 -1
- package/dist/app/workspace/workspace-actions-controller.js +10 -2
- package/dist/app/workspace/workspace-undo.d.ts +1 -0
- package/dist/app/workspace/workspace-undo.js +1 -0
- package/dist/bundled-extensions/question/index.js +9 -1
- package/dist/bundled-extensions/question/remote.d.ts +4 -0
- package/dist/bundled-extensions/question/remote.js +33 -0
- package/dist/bundled-extensions/telegram-connector/bot.d.ts +43 -0
- package/dist/bundled-extensions/telegram-connector/bot.js +166 -0
- package/dist/bundled-extensions/telegram-connector/config.d.ts +8 -0
- package/dist/bundled-extensions/telegram-connector/config.js +87 -0
- package/dist/bundled-extensions/telegram-connector/coordinator.d.ts +66 -0
- package/dist/bundled-extensions/telegram-connector/coordinator.js +413 -0
- package/dist/bundled-extensions/telegram-connector/index.d.ts +3 -0
- package/dist/bundled-extensions/telegram-connector/index.js +195 -0
- package/dist/bundled-extensions/terminal-bell/index.d.ts +0 -8
- package/dist/bundled-extensions/terminal-bell/index.js +0 -76
- package/dist/bundled-extensions/workspace-undo/index.d.ts +26 -0
- package/dist/bundled-extensions/workspace-undo/index.js +191 -0
- package/dist/config.d.ts +21 -2
- package/dist/config.js +159 -32
- package/dist/default-pix-config.js +25 -5
- package/dist/schemas/index.d.ts +1 -0
- package/dist/schemas/index.js +1 -0
- package/dist/schemas/pi-tools-suite-schema.d.ts +88 -62
- package/dist/schemas/pi-tools-suite-schema.js +54 -83
- package/dist/schemas/pix-schema.d.ts +19 -2
- package/dist/schemas/pix-schema.js +47 -5
- package/dist/schemas/tasks-schema.d.ts +18 -0
- package/dist/schemas/tasks-schema.js +41 -0
- package/docs/concurrency.md +9 -1
- package/docs/desktop-mvp.md +22 -6
- package/docs/desktop-task-manager.md +148 -84
- package/docs/release.md +30 -6
- package/external/pi-tools-suite/README.md +197 -84
- package/external/pi-tools-suite/docs/evals.md +1 -1
- package/external/pi-tools-suite/docs/session-recovery.md +47 -14
- package/external/pi-tools-suite/docs/subagent-model-pools.md +48 -30
- package/external/pi-tools-suite/docs/ui-qa-subagent.md +440 -0
- package/external/pi-tools-suite/package.json +6 -1
- package/external/pi-tools-suite/src/antigravity-auth/auth-store.ts +2 -1
- package/external/pi-tools-suite/src/antigravity-auth/constants.ts +9 -3
- package/external/pi-tools-suite/src/antigravity-auth/headers.ts +41 -3
- package/external/pi-tools-suite/src/antigravity-auth/models.ts +94 -21
- package/external/pi-tools-suite/src/antigravity-auth/oauth.ts +5 -17
- package/external/pi-tools-suite/src/antigravity-auth/payload.ts +72 -7
- package/external/pi-tools-suite/src/antigravity-auth/stream.ts +13 -1
- package/external/pi-tools-suite/src/async-subagents/agents/frontier-review.md +23 -0
- package/external/pi-tools-suite/src/async-subagents/agents/implement.md +1 -1
- package/external/pi-tools-suite/src/async-subagents/agents/presets.jsonc +16 -0
- package/external/pi-tools-suite/src/async-subagents/agents/research.md +5 -3
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/backends/browser.mjs +346 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/backends/desktop.mjs +1038 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/backends/tui.mjs +759 -0
- package/external/pi-tools-suite/src/async-subagents/agents/{browser-qa → ui-qa/browser}/scripts/browser-qa-runner.mjs +32 -23
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/drivers/chrome-devtools/chrome-devtools-provider.mjs +895 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/drivers/linux/linux-atspi.py +494 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/drivers/macos/macos-accessibility.swift +1085 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/drivers/native-terminal/bridge-client.mjs +50 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/drivers/native-terminal/native-terminal-host.mjs +501 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/drivers/windows/windows-uia.ps1 +454 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/browser-auth.md +114 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/browser-chrome-devtools.md +91 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/browser-playwright.md +145 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/browser.md +82 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/desktop-linux.md +26 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/desktop-macos.md +29 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/desktop-windows.md +23 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/desktop.md +54 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/tui-native-terminal.md +59 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/tui-pty.md +37 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/tui.md +60 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/scripts/ui-qa-runner.mjs +535 -0
- package/external/pi-tools-suite/src/async-subagents/agents/ui-qa.md +75 -0
- package/external/pi-tools-suite/src/async-subagents/commands.ts +15 -71
- package/external/pi-tools-suite/src/async-subagents/core/activity.ts +33 -0
- package/external/pi-tools-suite/src/async-subagents/core/agent-catalog.ts +5 -4
- package/external/pi-tools-suite/src/async-subagents/core/agent-strategy.ts +1 -1
- package/external/pi-tools-suite/src/async-subagents/core/agents-dir.ts +8 -5
- package/external/pi-tools-suite/src/async-subagents/core/browser-qa.ts +16 -2
- package/external/pi-tools-suite/src/async-subagents/core/config.ts +160 -327
- package/external/pi-tools-suite/src/async-subagents/core/model-selection.ts +2 -1
- package/external/pi-tools-suite/src/async-subagents/core/registry.ts +158 -33
- package/external/pi-tools-suite/src/async-subagents/core/routing.ts +21 -5
- package/external/pi-tools-suite/src/async-subagents/core/spawn.ts +40 -26
- package/external/pi-tools-suite/src/async-subagents/core/state.ts +40 -0
- package/external/pi-tools-suite/src/async-subagents/core/types.ts +7 -0
- package/external/pi-tools-suite/src/async-subagents/core/ultrawork-auto.ts +51 -45
- package/external/pi-tools-suite/src/async-subagents/index.ts +69 -5
- package/external/pi-tools-suite/src/async-subagents/lib.ts +16 -9
- package/external/pi-tools-suite/src/async-subagents/tools/spawn.ts +8 -5
- package/external/pi-tools-suite/src/async-subagents/tools/subagents.ts +5 -4
- package/external/pi-tools-suite/src/async-subagents/types.ts +1 -0
- package/external/pi-tools-suite/src/coding-discipline/index.ts +90 -59
- package/external/pi-tools-suite/src/config.ts +55 -1
- package/external/pi-tools-suite/src/context-gateway/accounting-log.ts +282 -0
- package/external/pi-tools-suite/src/context-gateway/config.ts +77 -3
- package/external/pi-tools-suite/src/context-gateway/efficiency.ts +414 -0
- package/external/pi-tools-suite/src/context-gateway/enforcement.ts +222 -0
- package/external/pi-tools-suite/src/context-gateway/index.ts +206 -19
- package/external/pi-tools-suite/src/context-gateway/storeless-capabilities.ts +7 -7
- package/external/pi-tools-suite/src/context-gateway/telemetry.ts +101 -28
- package/external/pi-tools-suite/src/context-gateway/types.ts +16 -3
- package/external/pi-tools-suite/src/context-inventory.ts +99 -0
- package/external/pi-tools-suite/src/dcp/auto-compress-budget.ts +39 -4
- package/external/pi-tools-suite/src/dcp/auto-compress.ts +128 -49
- package/external/pi-tools-suite/src/dcp/commands.ts +32 -156
- package/external/pi-tools-suite/src/dcp/compress-tool.ts +177 -46
- package/external/pi-tools-suite/src/dcp/compression-blocks.ts +6 -58
- package/external/pi-tools-suite/src/dcp/compression-preview.ts +9 -0
- package/external/pi-tools-suite/src/dcp/compression-progress.ts +22 -0
- package/external/pi-tools-suite/src/dcp/config.ts +114 -17
- package/external/pi-tools-suite/src/dcp/conversation-index.ts +36 -9
- package/external/pi-tools-suite/src/dcp/diagnostics.ts +41 -0
- package/external/pi-tools-suite/src/dcp/fresh-tool-results.ts +38 -0
- package/external/pi-tools-suite/src/dcp/index.ts +338 -66
- package/external/pi-tools-suite/src/dcp/journal.ts +126 -4
- package/external/pi-tools-suite/src/dcp/progress-controller.ts +2 -1
- package/external/pi-tools-suite/src/dcp/prompts.ts +83 -192
- package/external/pi-tools-suite/src/dcp/protected-continuity.ts +175 -0
- package/external/pi-tools-suite/src/dcp/pruner-candidates.ts +93 -3
- package/external/pi-tools-suite/src/dcp/pruner-message-ids.ts +6 -10
- package/external/pi-tools-suite/src/dcp/pruner-nudge.ts +36 -33
- package/external/pi-tools-suite/src/dcp/pruner-tools.ts +6 -3
- package/external/pi-tools-suite/src/dcp/pruner.ts +1 -0
- package/external/pi-tools-suite/src/dcp/routine-pressure.ts +86 -0
- package/external/pi-tools-suite/src/dcp/state.ts +9 -0
- package/external/pi-tools-suite/src/dcp/statistics.d.ts +8 -0
- package/external/pi-tools-suite/src/dcp/statistics.js +156 -0
- package/external/pi-tools-suite/src/default-pi-tools-suite-config.ts +78 -22
- package/external/pi-tools-suite/src/index.ts +7 -1
- package/external/pi-tools-suite/src/lib/project.ts +36 -1
- package/external/pi-tools-suite/src/model-tools/index.ts +10 -7
- package/external/pi-tools-suite/src/repo-discovery/index.ts +304 -4
- package/external/pi-tools-suite/src/resource-registry/index.ts +2551 -0
- package/external/pi-tools-suite/src/session-recovery/index.ts +17 -0
- package/external/pi-tools-suite/src/shell-command-policy.ts +219 -0
- package/external/pi-tools-suite/src/todo/index.ts +21 -0
- package/external/pi-tools-suite/src/todo/todo.ts +19 -1
- package/external/pi-tools-suite/src/tool-descriptions.ts +47 -19
- package/package.json +10 -9
- package/schemas/pi-tools-suite.json +466 -287
- package/schemas/pix.json +129 -11
- package/schemas/tasks.json +131 -0
- package/skills/simplify/SKILL.md +33 -5
- package/docs/desktop-markdown-media.md +0 -77
- package/external/pi-tools-suite/docs/browser-qa-subagent.md +0 -177
- package/external/pi-tools-suite/docs/dcp-emergency-current-turn.md +0 -102
- package/external/pi-tools-suite/src/async-subagents/agents/browser-qa.md +0 -598
- package/external/pi-tools-suite/src/async-subagents/async-subagents.sample.jsonc +0 -54
- package/external/pi-tools-suite/src/skill-installer/index.ts +0 -333
- package/skills/playwright-cli/SKILL.md +0 -420
- package/skills/playwright-cli/references/element-attributes.md +0 -23
- package/skills/playwright-cli/references/playwright-tests.md +0 -50
- package/skills/playwright-cli/references/request-mocking.md +0 -87
- package/skills/playwright-cli/references/running-code.md +0 -241
- package/skills/playwright-cli/references/session-management.md +0 -273
- package/skills/playwright-cli/references/spec-driven-testing.md +0 -311
- package/skills/playwright-cli/references/storage-state.md +0 -290
- package/skills/playwright-cli/references/test-generation.md +0 -142
- package/skills/playwright-cli/references/tracing.md +0 -154
- package/skills/playwright-cli/references/video-recording.md +0 -147
- package/skills/spec-lite/SKILL.md +0 -140
- /package/external/pi-tools-suite/src/async-subagents/agents/{browser-qa → ui-qa/browser}/examples/qa-auth.example.jsonc +0 -0
- /package/external/pi-tools-suite/src/async-subagents/agents/{browser-qa → ui-qa/browser}/examples/qa-flow.example.jsonc +0 -0
- /package/external/pi-tools-suite/src/async-subagents/agents/{browser-qa → ui-qa/browser}/vendor/fflate.LICENSE +0 -0
- /package/external/pi-tools-suite/src/async-subagents/agents/{browser-qa → ui-qa/browser}/vendor/fflate.mjs +0 -0
package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/drivers/windows/windows-uia.ps1
ADDED
|
@@ -0,0 +1,454 @@
|
|
|
1
|
+
$ErrorActionPreference = "Stop"
|
|
2
|
+
$ProgressPreference = "SilentlyContinue"
|
|
3
|
+
|
|
4
|
+
Add-Type -AssemblyName UIAutomationClient
|
|
5
|
+
Add-Type -AssemblyName UIAutomationTypes
|
|
6
|
+
Add-Type -AssemblyName System.Drawing
|
|
7
|
+
|
|
8
|
+
Add-Type -TypeDefinition @'
|
|
9
|
+
using System;
|
|
10
|
+
using System.Drawing;
|
|
11
|
+
using System.Drawing.Imaging;
|
|
12
|
+
using System.Runtime.InteropServices;
|
|
13
|
+
using System.Text;
|
|
14
|
+
|
|
15
|
+
public static class UiQaWin32 {
|
|
16
|
+
[StructLayout(LayoutKind.Sequential)]
|
|
17
|
+
public struct RECT { public int Left, Top, Right, Bottom; }
|
|
18
|
+
|
|
19
|
+
[StructLayout(LayoutKind.Sequential)]
|
|
20
|
+
struct INPUT { public uint type; public InputUnion U; }
|
|
21
|
+
|
|
22
|
+
[StructLayout(LayoutKind.Explicit)]
|
|
23
|
+
struct InputUnion { [FieldOffset(0)] public KEYBDINPUT ki; }
|
|
24
|
+
|
|
25
|
+
[StructLayout(LayoutKind.Sequential)]
|
|
26
|
+
struct KEYBDINPUT {
|
|
27
|
+
public ushort wVk;
|
|
28
|
+
public ushort wScan;
|
|
29
|
+
public uint dwFlags;
|
|
30
|
+
public uint time;
|
|
31
|
+
public UIntPtr dwExtraInfo;
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
[DllImport("user32.dll", SetLastError=true)]
|
|
35
|
+
public static extern bool GetWindowRect(IntPtr hWnd, out RECT rect);
|
|
36
|
+
|
|
37
|
+
[DllImport("user32.dll", SetLastError=true)]
|
|
38
|
+
public static extern bool PrintWindow(IntPtr hwnd, IntPtr hdcBlt, uint flags);
|
|
39
|
+
|
|
40
|
+
[DllImport("user32.dll")]
|
|
41
|
+
public static extern bool SetForegroundWindow(IntPtr hWnd);
|
|
42
|
+
|
|
43
|
+
[DllImport("user32.dll")]
|
|
44
|
+
public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
|
|
45
|
+
|
|
46
|
+
[DllImport("user32.dll")]
|
|
47
|
+
static extern bool SetCursorPos(int x, int y);
|
|
48
|
+
|
|
49
|
+
[DllImport("user32.dll")]
|
|
50
|
+
static extern void mouse_event(uint flags, uint dx, uint dy, uint data, UIntPtr extra);
|
|
51
|
+
|
|
52
|
+
[DllImport("user32.dll", SetLastError=true)]
|
|
53
|
+
static extern uint SendInput(uint count, INPUT[] inputs, int size);
|
|
54
|
+
|
|
55
|
+
const uint INPUT_KEYBOARD = 1;
|
|
56
|
+
const uint KEYEVENTF_KEYUP = 0x0002;
|
|
57
|
+
const uint KEYEVENTF_UNICODE = 0x0004;
|
|
58
|
+
const uint MOUSEEVENTF_LEFTDOWN = 0x0002;
|
|
59
|
+
const uint MOUSEEVENTF_LEFTUP = 0x0004;
|
|
60
|
+
|
|
61
|
+
static INPUT Key(ushort vk, ushort scan, uint flags) {
|
|
62
|
+
var input = new INPUT();
|
|
63
|
+
input.type = INPUT_KEYBOARD;
|
|
64
|
+
input.U.ki.wVk = vk;
|
|
65
|
+
input.U.ki.wScan = scan;
|
|
66
|
+
input.U.ki.dwFlags = flags;
|
|
67
|
+
return input;
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
public static void SendUnicodeText(string text) {
|
|
71
|
+
foreach (char ch in text) {
|
|
72
|
+
var inputs = new [] {
|
|
73
|
+
Key(0, ch, KEYEVENTF_UNICODE),
|
|
74
|
+
Key(0, ch, KEYEVENTF_UNICODE | KEYEVENTF_KEYUP),
|
|
75
|
+
};
|
|
76
|
+
if (SendInput((uint)inputs.Length, inputs, Marshal.SizeOf(typeof(INPUT))) != (uint)inputs.Length)
|
|
77
|
+
throw new InvalidOperationException("SendInput failed while typing text");
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
public static void SendKeyChord(ushort key, ushort[] modifiers) {
|
|
82
|
+
var inputs = new System.Collections.Generic.List<INPUT>();
|
|
83
|
+
foreach (var modifier in modifiers) inputs.Add(Key(modifier, 0, 0));
|
|
84
|
+
inputs.Add(Key(key, 0, 0));
|
|
85
|
+
inputs.Add(Key(key, 0, KEYEVENTF_KEYUP));
|
|
86
|
+
for (int i = modifiers.Length - 1; i >= 0; i--) inputs.Add(Key(modifiers[i], 0, KEYEVENTF_KEYUP));
|
|
87
|
+
var value = inputs.ToArray();
|
|
88
|
+
if (SendInput((uint)value.Length, value, Marshal.SizeOf(typeof(INPUT))) != (uint)value.Length)
|
|
89
|
+
throw new InvalidOperationException("SendInput failed while sending key chord");
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
public static void ClickPoint(int x, int y) {
|
|
93
|
+
if (!SetCursorPos(x, y)) throw new InvalidOperationException("SetCursorPos failed");
|
|
94
|
+
mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, UIntPtr.Zero);
|
|
95
|
+
mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, UIntPtr.Zero);
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
public static void CaptureWindow(IntPtr hwnd, string file) {
|
|
99
|
+
RECT rect;
|
|
100
|
+
if (hwnd == IntPtr.Zero || !GetWindowRect(hwnd, out rect))
|
|
101
|
+
throw new InvalidOperationException("could not resolve target window rectangle");
|
|
102
|
+
int width = Math.Max(1, rect.Right - rect.Left);
|
|
103
|
+
int height = Math.Max(1, rect.Bottom - rect.Top);
|
|
104
|
+
using (var bitmap = new Bitmap(width, height, PixelFormat.Format32bppArgb)) {
|
|
105
|
+
using (var graphics = Graphics.FromImage(bitmap)) {
|
|
106
|
+
IntPtr hdc = graphics.GetHdc();
|
|
107
|
+
bool printed = false;
|
|
108
|
+
try { printed = PrintWindow(hwnd, hdc, 2); }
|
|
109
|
+
finally { graphics.ReleaseHdc(hdc); }
|
|
110
|
+
if (!printed) graphics.CopyFromScreen(rect.Left, rect.Top, 0, 0, new Size(width, height));
|
|
111
|
+
}
|
|
112
|
+
bitmap.Save(file, ImageFormat.Png);
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
}
|
|
116
|
+
'@
|
|
117
|
+
|
|
118
|
+
function Fail([string]$Message) {
|
|
119
|
+
[Console]::Error.WriteLine($Message)
|
|
120
|
+
exit 1
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
function Parse-Options([string[]]$Values) {
|
|
124
|
+
$result = @{}
|
|
125
|
+
$index = 0
|
|
126
|
+
while ($index -lt $Values.Count) {
|
|
127
|
+
$key = $Values[$index]
|
|
128
|
+
if (-not $key.StartsWith("--")) { Fail "unexpected argument: $key" }
|
|
129
|
+
$name = $key.Substring(2)
|
|
130
|
+
if ($name -in @("all", "print-pid")) {
|
|
131
|
+
$result[$name] = $true
|
|
132
|
+
$index += 1
|
|
133
|
+
continue
|
|
134
|
+
}
|
|
135
|
+
if ($index + 1 -ge $Values.Count) { Fail "missing value for $key" }
|
|
136
|
+
if ($result.ContainsKey($name)) { Fail "duplicate option: $key" }
|
|
137
|
+
$result[$name] = $Values[$index + 1]
|
|
138
|
+
$index += 2
|
|
139
|
+
}
|
|
140
|
+
return $result
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
function Require-Option($Options, [string]$Name) {
|
|
144
|
+
if (-not $Options.ContainsKey($Name) -or [string]::IsNullOrWhiteSpace([string]$Options[$Name])) {
|
|
145
|
+
Fail "missing --$Name"
|
|
146
|
+
}
|
|
147
|
+
return [string]$Options[$Name]
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
function Control-Role([System.Windows.Automation.AutomationElement]$Element) {
|
|
151
|
+
$programmatic = $Element.Current.ControlType.ProgrammaticName
|
|
152
|
+
if ($programmatic -and $programmatic.Contains(".")) { return $programmatic.Split(".")[-1] }
|
|
153
|
+
return [string]$programmatic
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
function Scalar-String($Value) {
|
|
157
|
+
if ($null -eq $Value) { return $null }
|
|
158
|
+
if ($Value -is [bool]) { return $Value.ToString().ToLowerInvariant() }
|
|
159
|
+
return [string]$Value
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
function Element-Attributes([System.Windows.Automation.AutomationElement]$Element) {
|
|
163
|
+
$attributes = [ordered]@{}
|
|
164
|
+
if (-not [string]::IsNullOrEmpty($Element.Current.Name)) { $attributes.title = $Element.Current.Name }
|
|
165
|
+
if (-not [string]::IsNullOrEmpty($Element.Current.ItemStatus)) { $attributes.description = $Element.Current.ItemStatus }
|
|
166
|
+
if (-not [string]::IsNullOrEmpty($Element.Current.AutomationId)) { $attributes.identifier = $Element.Current.AutomationId }
|
|
167
|
+
if (-not [string]::IsNullOrEmpty($Element.Current.HelpText)) { $attributes.help = $Element.Current.HelpText }
|
|
168
|
+
if ($Element.Current.ControlType -eq [System.Windows.Automation.ControlType]::Edit -and $Element.Current.IsPassword) {
|
|
169
|
+
$attributes.value = "<redacted>"
|
|
170
|
+
} else {
|
|
171
|
+
$pattern = $null
|
|
172
|
+
if ($Element.TryGetCurrentPattern([System.Windows.Automation.ValuePattern]::Pattern, [ref]$pattern)) {
|
|
173
|
+
$attributes.value = ([System.Windows.Automation.ValuePattern]$pattern).Current.Value
|
|
174
|
+
} else {
|
|
175
|
+
$toggle = $null
|
|
176
|
+
if ($Element.TryGetCurrentPattern([System.Windows.Automation.TogglePattern]::Pattern, [ref]$toggle)) {
|
|
177
|
+
$toggleState = ([System.Windows.Automation.TogglePattern]$toggle).Current.ToggleState
|
|
178
|
+
if ($toggleState -eq [System.Windows.Automation.ToggleState]::On) { $attributes.value = "1" }
|
|
179
|
+
elseif ($toggleState -eq [System.Windows.Automation.ToggleState]::Off) { $attributes.value = "0" }
|
|
180
|
+
else { $attributes.value = "mixed" }
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
}
|
|
184
|
+
$attributes.enabled = Scalar-String $Element.Current.IsEnabled
|
|
185
|
+
$attributes.focused = Scalar-String $Element.Current.HasKeyboardFocus
|
|
186
|
+
$selection = $null
|
|
187
|
+
if ($Element.TryGetCurrentPattern([System.Windows.Automation.SelectionItemPattern]::Pattern, [ref]$selection)) {
|
|
188
|
+
$attributes.selected = Scalar-String ([System.Windows.Automation.SelectionItemPattern]$selection).Current.IsSelected
|
|
189
|
+
}
|
|
190
|
+
return $attributes
|
|
191
|
+
}
|
|
192
|
+
|
|
193
|
+
function Element-Record([System.Windows.Automation.AutomationElement]$Element, [string]$Path) {
|
|
194
|
+
$rect = $Element.Current.BoundingRectangle
|
|
195
|
+
return [ordered]@{
|
|
196
|
+
path = $Path
|
|
197
|
+
role = Control-Role $Element
|
|
198
|
+
attributes = Element-Attributes $Element
|
|
199
|
+
frame = [ordered]@{
|
|
200
|
+
x = [int][math]::Round($rect.X)
|
|
201
|
+
y = [int][math]::Round($rect.Y)
|
|
202
|
+
width = [int][math]::Round($rect.Width)
|
|
203
|
+
height = [int][math]::Round($rect.Height)
|
|
204
|
+
}
|
|
205
|
+
}
|
|
206
|
+
}
|
|
207
|
+
|
|
208
|
+
function Get-Children([System.Windows.Automation.AutomationElement]$Element) {
|
|
209
|
+
$result = New-Object System.Collections.Generic.List[System.Windows.Automation.AutomationElement]
|
|
210
|
+
$walker = [System.Windows.Automation.TreeWalker]::RawViewWalker
|
|
211
|
+
$child = $walker.GetFirstChild($Element)
|
|
212
|
+
while ($null -ne $child) {
|
|
213
|
+
$result.Add($child)
|
|
214
|
+
$child = $walker.GetNextSibling($child)
|
|
215
|
+
}
|
|
216
|
+
return $result
|
|
217
|
+
}
|
|
218
|
+
|
|
219
|
+
function Get-DescendantPids([int]$OwnerPid) {
|
|
220
|
+
$wanted = New-Object 'System.Collections.Generic.HashSet[int]'
|
|
221
|
+
[void]$wanted.Add($OwnerPid)
|
|
222
|
+
try {
|
|
223
|
+
$processes = @(Get-CimInstance Win32_Process -ErrorAction Stop | Select-Object ProcessId, ParentProcessId)
|
|
224
|
+
$changed = $true
|
|
225
|
+
while ($changed) {
|
|
226
|
+
$changed = $false
|
|
227
|
+
foreach ($process in $processes) {
|
|
228
|
+
if ($wanted.Contains([int]$process.ParentProcessId) -and -not $wanted.Contains([int]$process.ProcessId)) {
|
|
229
|
+
[void]$wanted.Add([int]$process.ProcessId)
|
|
230
|
+
$changed = $true
|
|
231
|
+
}
|
|
232
|
+
}
|
|
233
|
+
}
|
|
234
|
+
} catch {
|
|
235
|
+
# Exact owner PID remains a safe fallback if WMI/CIM is unavailable.
|
|
236
|
+
}
|
|
237
|
+
return ,$wanted
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
function Find-AppWindow($Options) {
|
|
241
|
+
$roots = [System.Windows.Automation.AutomationElement]::RootElement.FindAll(
|
|
242
|
+
[System.Windows.Automation.TreeScope]::Children,
|
|
243
|
+
[System.Windows.Automation.Condition]::TrueCondition
|
|
244
|
+
)
|
|
245
|
+
$matches = New-Object System.Collections.Generic.List[System.Windows.Automation.AutomationElement]
|
|
246
|
+
$pids = $null
|
|
247
|
+
if ($Options.ContainsKey("pid")) {
|
|
248
|
+
$pids = New-Object 'System.Collections.Generic.HashSet[int]'
|
|
249
|
+
[void]$pids.Add([int](Require-Option $Options "pid"))
|
|
250
|
+
} elseif ($Options.ContainsKey("owner-pid")) {
|
|
251
|
+
$pids = Get-DescendantPids ([int](Require-Option $Options "owner-pid"))
|
|
252
|
+
}
|
|
253
|
+
$name = if ($Options.ContainsKey("app")) { [string]$Options.app } else { $null }
|
|
254
|
+
$title = if ($Options.ContainsKey("title")) { [string]$Options.title } else { $null }
|
|
255
|
+
foreach ($root in $roots) {
|
|
256
|
+
if ($pids -and -not $pids.Contains([int]$root.Current.ProcessId)) { continue }
|
|
257
|
+
if ($title -and $root.Current.Name.IndexOf($title, [StringComparison]::OrdinalIgnoreCase) -lt 0) { continue }
|
|
258
|
+
if ($name) {
|
|
259
|
+
$processName = ""
|
|
260
|
+
try { $processName = (Get-Process -Id $root.Current.ProcessId -ErrorAction Stop).ProcessName } catch {}
|
|
261
|
+
if ($root.Current.Name.IndexOf($name, [StringComparison]::OrdinalIgnoreCase) -lt 0 -and
|
|
262
|
+
$processName.IndexOf($name, [StringComparison]::OrdinalIgnoreCase) -lt 0) { continue }
|
|
263
|
+
}
|
|
264
|
+
$matches.Add($root)
|
|
265
|
+
}
|
|
266
|
+
if ($matches.Count -eq 0) { return $null }
|
|
267
|
+
if ($matches.Count -gt 1 -and -not $pids -and -not $title) { Fail "ambiguous app selector ($($matches.Count) windows); use --pid" }
|
|
268
|
+
return $matches[0]
|
|
269
|
+
}
|
|
270
|
+
|
|
271
|
+
function Wait-AppWindow($Options, [double]$TimeoutSeconds) {
|
|
272
|
+
$deadline = [DateTime]::UtcNow.AddSeconds($TimeoutSeconds)
|
|
273
|
+
while ([DateTime]::UtcNow -lt $deadline) {
|
|
274
|
+
$window = Find-AppWindow $Options
|
|
275
|
+
if ($null -ne $window) { return $window }
|
|
276
|
+
Start-Sleep -Milliseconds 100
|
|
277
|
+
}
|
|
278
|
+
Fail "timed out waiting for application window"
|
|
279
|
+
}
|
|
280
|
+
|
|
281
|
+
function Focus-Window([System.Windows.Automation.AutomationElement]$Window) {
|
|
282
|
+
$handle = [IntPtr]$Window.Current.NativeWindowHandle
|
|
283
|
+
if ($handle -ne [IntPtr]::Zero) {
|
|
284
|
+
[void][UiQaWin32]::ShowWindow($handle, 9)
|
|
285
|
+
[void][UiQaWin32]::SetForegroundWindow($handle)
|
|
286
|
+
}
|
|
287
|
+
try { $Window.SetFocus() } catch {}
|
|
288
|
+
}
|
|
289
|
+
|
|
290
|
+
function Walk-Tree([System.Windows.Automation.AutomationElement]$Root, [int]$Depth, [int]$Limit) {
|
|
291
|
+
$records = New-Object System.Collections.Generic.List[object]
|
|
292
|
+
function Visit($Element, [string]$Path, [int]$Level) {
|
|
293
|
+
if ($records.Count -ge $Limit) { return }
|
|
294
|
+
$records.Add((Element-Record $Element $Path))
|
|
295
|
+
if ($Level -ge $Depth) { return }
|
|
296
|
+
$children = @(Get-Children $Element)
|
|
297
|
+
for ($index = 0; $index -lt $children.Count; $index++) {
|
|
298
|
+
Visit $children[$index] "$Path/$index" ($Level + 1)
|
|
299
|
+
if ($records.Count -ge $Limit) { return }
|
|
300
|
+
}
|
|
301
|
+
}
|
|
302
|
+
Visit $Root "0" 0
|
|
303
|
+
return $records
|
|
304
|
+
}
|
|
305
|
+
|
|
306
|
+
function Find-Element([System.Windows.Automation.AutomationElement]$Window, $Options) {
|
|
307
|
+
if ($Options.ContainsKey("path")) {
|
|
308
|
+
$parts = ([string]$Options.path).Split('/') | Where-Object { $_ -ne "" }
|
|
309
|
+
$current = $Window
|
|
310
|
+
$start = if ($parts.Count -gt 0 -and $parts[0] -eq "0") { 1 } else { 0 }
|
|
311
|
+
for ($index = $start; $index -lt $parts.Count; $index++) {
|
|
312
|
+
$childIndex = 0
|
|
313
|
+
if (-not [int]::TryParse($parts[$index], [ref]$childIndex)) { Fail "invalid element path" }
|
|
314
|
+
$children = @(Get-Children $current)
|
|
315
|
+
if ($childIndex -lt 0 -or $childIndex -ge $children.Count) { Fail "element path does not exist" }
|
|
316
|
+
$current = $children[$childIndex]
|
|
317
|
+
}
|
|
318
|
+
return $current
|
|
319
|
+
}
|
|
320
|
+
$match = Require-Option $Options "match"
|
|
321
|
+
$role = if ($Options.ContainsKey("role")) { ([string]$Options.role).Replace("AX", "").Replace(" ", "") } else { $null }
|
|
322
|
+
$occurrence = if ($Options.ContainsKey("occurrence")) { [int]$Options.occurrence } else { 1 }
|
|
323
|
+
$seen = 0
|
|
324
|
+
foreach ($record in (Walk-Tree $Window 20 1000)) {
|
|
325
|
+
$path = [string]$record.path
|
|
326
|
+
$element = Find-Element $Window @{ path = $path }
|
|
327
|
+
$attributes = Element-Attributes $element
|
|
328
|
+
$text = @($attributes.title, $attributes.description, $attributes.identifier, $attributes.help, $attributes.value) -join " "
|
|
329
|
+
if ($text.IndexOf($match, [StringComparison]::OrdinalIgnoreCase) -lt 0) { continue }
|
|
330
|
+
if ($role -and (Control-Role $element).Replace(" ", "") -ne $role) { continue }
|
|
331
|
+
$seen += 1
|
|
332
|
+
if ($seen -eq $occurrence) { return $element }
|
|
333
|
+
}
|
|
334
|
+
Fail "no accessibility element matches the supplied selector"
|
|
335
|
+
}
|
|
336
|
+
|
|
337
|
+
function Format-InspectRecord($Record) {
|
|
338
|
+
$parts = New-Object System.Collections.Generic.List[string]
|
|
339
|
+
$parts.Add([string]$Record.path)
|
|
340
|
+
$parts.Add([string]$Record.role)
|
|
341
|
+
foreach ($entry in $Record.attributes.GetEnumerator()) {
|
|
342
|
+
if ($null -eq $entry.Value -or [string]::IsNullOrEmpty([string]$entry.Value)) { continue }
|
|
343
|
+
$safe = ([string]$entry.Value).Replace("`r", " ").Replace("`n", " ").Replace([string][char]34, ('\' + [char]34))
|
|
344
|
+
$parts.Add("$($entry.Key)=`"$safe`"")
|
|
345
|
+
}
|
|
346
|
+
$frame = $Record.frame
|
|
347
|
+
$parts.Add("frame=$($frame.x),$($frame.y),$($frame.width)x$($frame.height)")
|
|
348
|
+
return $parts -join " "
|
|
349
|
+
}
|
|
350
|
+
|
|
351
|
+
function Invoke-Element([System.Windows.Automation.AutomationElement]$Element) {
|
|
352
|
+
$pattern = $null
|
|
353
|
+
if ($Element.TryGetCurrentPattern([System.Windows.Automation.InvokePattern]::Pattern, [ref]$pattern)) {
|
|
354
|
+
([System.Windows.Automation.InvokePattern]$pattern).Invoke(); return
|
|
355
|
+
}
|
|
356
|
+
if ($Element.TryGetCurrentPattern([System.Windows.Automation.SelectionItemPattern]::Pattern, [ref]$pattern)) {
|
|
357
|
+
([System.Windows.Automation.SelectionItemPattern]$pattern).Select(); return
|
|
358
|
+
}
|
|
359
|
+
if ($Element.TryGetCurrentPattern([System.Windows.Automation.TogglePattern]::Pattern, [ref]$pattern)) {
|
|
360
|
+
([System.Windows.Automation.TogglePattern]$pattern).Toggle(); return
|
|
361
|
+
}
|
|
362
|
+
$rect = $Element.Current.BoundingRectangle
|
|
363
|
+
if ($rect.Width -le 0 -or $rect.Height -le 0) { Fail "element has no actionable accessibility pattern or visible frame" }
|
|
364
|
+
[UiQaWin32]::ClickPoint([int]($rect.X + $rect.Width / 2), [int]($rect.Y + $rect.Height / 2))
|
|
365
|
+
}
|
|
366
|
+
|
|
367
|
+
function Key-Code([string]$Name) {
|
|
368
|
+
$map = @{
|
|
369
|
+
"enter"=0x0D; "return"=0x0D; "escape"=0x1B; "esc"=0x1B; "tab"=0x09; "backspace"=0x08;
|
|
370
|
+
"delete"=0x2E; "left"=0x25; "up"=0x26; "right"=0x27; "down"=0x28; "home"=0x24; "end"=0x23;
|
|
371
|
+
"pageup"=0x21; "pagedown"=0x22; "space"=0x20
|
|
372
|
+
}
|
|
373
|
+
$lower = $Name.ToLowerInvariant()
|
|
374
|
+
if ($map.ContainsKey($lower)) { return [ushort]$map[$lower] }
|
|
375
|
+
if ($Name.Length -eq 1) { return [ushort][char]$Name.ToUpperInvariant() }
|
|
376
|
+
Fail "unsupported key: $Name"
|
|
377
|
+
}
|
|
378
|
+
|
|
379
|
+
function Modifier-Codes([string]$Value) {
|
|
380
|
+
$result = New-Object System.Collections.Generic.List[ushort]
|
|
381
|
+
if ([string]::IsNullOrWhiteSpace($Value)) { return $result.ToArray() }
|
|
382
|
+
foreach ($raw in $Value.Split(',')) {
|
|
383
|
+
switch ($raw.Trim().ToLowerInvariant()) {
|
|
384
|
+
"cmd" { $result.Add([ushort]0x5B) }
|
|
385
|
+
"shift" { $result.Add([ushort]0x10) }
|
|
386
|
+
"ctrl" { $result.Add([ushort]0x11) }
|
|
387
|
+
"opt" { $result.Add([ushort]0x12) }
|
|
388
|
+
"alt" { $result.Add([ushort]0x12) }
|
|
389
|
+
"fn" { Fail "fn modifier is not supported by the Windows UI Automation helper" }
|
|
390
|
+
default { Fail "unsupported modifier: $raw" }
|
|
391
|
+
}
|
|
392
|
+
}
|
|
393
|
+
return $result.ToArray()
|
|
394
|
+
}
|
|
395
|
+
|
|
396
|
+
if ($args.Count -lt 1) { Fail "missing command" }
|
|
397
|
+
$command = [string]$args[0]
|
|
398
|
+
$options = Parse-Options @($args | Select-Object -Skip 1)
|
|
399
|
+
|
|
400
|
+
if ($command -eq "doctor") {
|
|
401
|
+
try {
|
|
402
|
+
[void][System.Windows.Automation.AutomationElement]::RootElement
|
|
403
|
+
Write-Output "uia=available"
|
|
404
|
+
Write-Output "screenshot=available"
|
|
405
|
+
exit 0
|
|
406
|
+
} catch {
|
|
407
|
+
Fail "Windows UI Automation is unavailable: $($_.Exception.Message)"
|
|
408
|
+
}
|
|
409
|
+
}
|
|
410
|
+
|
|
411
|
+
$timeout = if ($options.ContainsKey("timeout")) { [double]$options.timeout } else { 15.0 }
|
|
412
|
+
$window = Wait-AppWindow $options $timeout
|
|
413
|
+
|
|
414
|
+
switch ($command) {
|
|
415
|
+
"wait-window" {
|
|
416
|
+
if ($options.ContainsKey("print-pid")) { Write-Output $window.Current.ProcessId }
|
|
417
|
+
}
|
|
418
|
+
"focus" { Focus-Window $window }
|
|
419
|
+
"inspect" {
|
|
420
|
+
$depth = if ($options.ContainsKey("depth")) { [int]$options.depth } else { 10 }
|
|
421
|
+
$limit = if ($options.ContainsKey("limit")) { [int]$options.limit } else { 80 }
|
|
422
|
+
foreach ($record in (Walk-Tree $window $depth $limit)) { Write-Output (Format-InspectRecord $record) }
|
|
423
|
+
}
|
|
424
|
+
"describe" {
|
|
425
|
+
$element = Find-Element $window $options
|
|
426
|
+
$pathValue = if ($options.ContainsKey("path")) { [string]$options.path } else { "match" }
|
|
427
|
+
(Element-Record $element $pathValue) | ConvertTo-Json -Compress -Depth 5
|
|
428
|
+
}
|
|
429
|
+
"screenshot" {
|
|
430
|
+
$out = Require-Option $options "out"
|
|
431
|
+
$directory = [IO.Path]::GetDirectoryName([IO.Path]::GetFullPath($out))
|
|
432
|
+
if (-not [IO.Directory]::Exists($directory)) { Fail "screenshot output directory does not exist" }
|
|
433
|
+
Focus-Window $window
|
|
434
|
+
[UiQaWin32]::CaptureWindow([IntPtr]$window.Current.NativeWindowHandle, $out)
|
|
435
|
+
}
|
|
436
|
+
"click" { Invoke-Element (Find-Element $window $options) }
|
|
437
|
+
"set-value" {
|
|
438
|
+
$element = Find-Element $window $options
|
|
439
|
+
$value = Require-Option $options "value"
|
|
440
|
+
$pattern = $null
|
|
441
|
+
if (-not $element.TryGetCurrentPattern([System.Windows.Automation.ValuePattern]::Pattern, [ref]$pattern)) { Fail "element does not support ValuePattern" }
|
|
442
|
+
([System.Windows.Automation.ValuePattern]$pattern).SetValue($value)
|
|
443
|
+
}
|
|
444
|
+
"type" {
|
|
445
|
+
Focus-Window $window
|
|
446
|
+
[UiQaWin32]::SendUnicodeText((Require-Option $options "text"))
|
|
447
|
+
}
|
|
448
|
+
"key" {
|
|
449
|
+
Focus-Window $window
|
|
450
|
+
$mods = if ($options.ContainsKey("modifiers")) { [ushort[]]@(Modifier-Codes ([string]$options.modifiers)) } else { [ushort[]]@() }
|
|
451
|
+
[UiQaWin32]::SendKeyChord((Key-Code (Require-Option $options "key")), $mods)
|
|
452
|
+
}
|
|
453
|
+
default { Fail "unsupported command: $command" }
|
|
454
|
+
}
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
# Browser auth guide
|
|
2
|
+
|
|
3
|
+
Load this guide only when a browser task actually requires authentication:
|
|
4
|
+
profile discovery, form-auth scaffolding, or an auth update flow. Public QA
|
|
5
|
+
needs no credentials and must not create `.pi/qa_auth.jsonc`.
|
|
6
|
+
|
|
7
|
+
`PI_BROWSER_QA_RUNNER` names the installed trusted browser backend. Invoke it
|
|
8
|
+
directly only for the commands in this guide; all probe/run browser execution
|
|
9
|
+
still goes through `node "$PI_UI_QA_RUNNER"`.
|
|
10
|
+
|
|
11
|
+
Never read, print, grep, copy, or edit credential values from
|
|
12
|
+
`.pi/qa_auth.jsonc` yourself.
|
|
13
|
+
|
|
14
|
+
## Profile discovery
|
|
15
|
+
|
|
16
|
+
When a task requires authenticated behavior, first run:
|
|
17
|
+
|
|
18
|
+
```sh
|
|
19
|
+
node "$PI_BROWSER_QA_RUNNER" profiles
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
A missing auth config returns an empty list without creating
|
|
23
|
+
`.pi/qa_auth.jsonc`. Choose an auth profile only when the task names its id,
|
|
24
|
+
safe profile traits make the choice unambiguous, or a public run proves that
|
|
25
|
+
the requested page requires login. Profiles live in project-local
|
|
26
|
+
`.pi/qa_auth.jsonc` (mode `0600` on POSIX) and every profile declares its own
|
|
27
|
+
exact `allowedOrigins`; listing output exposes only profile ids, descriptions,
|
|
28
|
+
and traits.
|
|
29
|
+
|
|
30
|
+
When an explicit authenticated request finds no usable profile, run:
|
|
31
|
+
|
|
32
|
+
```sh
|
|
33
|
+
node "$PI_BROWSER_QA_RUNNER" profiles --require-auth
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Only `auth scaffold` and `profiles --require-auth` may create the private auth
|
|
37
|
+
file.
|
|
38
|
+
|
|
39
|
+
## Form-auth scaffolding
|
|
40
|
+
|
|
41
|
+
When a public HTTPS login page (or loopback HTTP page for local development) is
|
|
42
|
+
known and there is no usable profile, invoke the trusted runner once:
|
|
43
|
+
|
|
44
|
+
```sh
|
|
45
|
+
node "$PI_BROWSER_QA_RUNNER" auth scaffold \
|
|
46
|
+
--profile <safe-id> \
|
|
47
|
+
--login-url <public-login-url>
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
The runner discovers the login form and privately writes a target-specific
|
|
51
|
+
`.pi/qa_auth.jsonc` containing `__PI_QA_SECRET_n__` values. It may create the
|
|
52
|
+
file or replace only the runner's own empty generated template; it refuses to
|
|
53
|
+
overwrite invalid or non-empty auth configuration. Never inspect the result.
|
|
54
|
+
Relay `QA_AUTH_SCAFFOLD_CREATED`, the reported file, profile, placeholder count,
|
|
55
|
+
and action, then ask the user to replace only those placeholder values. Browser
|
|
56
|
+
QA must resume in a new run after the user confirms that edit.
|
|
57
|
+
|
|
58
|
+
By default, login succeeds when the discovered form becomes hidden. A form-less
|
|
59
|
+
page therefore requires an explicit deterministic signal using either
|
|
60
|
+
`--success-url <glob>` or `--success-selector <selector>` with optional
|
|
61
|
+
`--success-state attached|detached|visible|hidden`. Use `--base-url <url>` only
|
|
62
|
+
to override the profile's same-origin application root. These are public
|
|
63
|
+
metadata; never put credential values in command arguments or environment
|
|
64
|
+
variables.
|
|
65
|
+
|
|
66
|
+
Do not scaffold cookie, bearer, storage, storage-state, MFA, CAPTCHA, or
|
|
67
|
+
federated login. For those modes, use the generic empty-template flow below and
|
|
68
|
+
ask the user to configure the profile themselves.
|
|
69
|
+
|
|
70
|
+
Do not request credentials merely because `.pi/qa_auth.jsonc` is absent. If the
|
|
71
|
+
task explicitly requires authenticated behavior, or a public run reaches the
|
|
72
|
+
flow's `authRejectedIf` check, and `profiles` returned no usable profile, run
|
|
73
|
+
form-auth scaffolding when supported. Otherwise run
|
|
74
|
+
`node "$PI_BROWSER_QA_RUNNER" profiles --require-auth`. Only these paths
|
|
75
|
+
may create the private auth file.
|
|
76
|
+
|
|
77
|
+
## Scaffold safety and edge cases
|
|
78
|
+
|
|
79
|
+
The trusted runner discovers the most likely form, fillable fields, submit
|
|
80
|
+
control, and form container on the unauthenticated page. It writes only
|
|
81
|
+
discovered selectors and generated placeholders to the private `0600` config.
|
|
82
|
+
The agent treats that config as opaque and user-owned. Scaffold status exposes
|
|
83
|
+
only the file path, profile id, counts, and action, never inspected DOM text,
|
|
84
|
+
input values, URLs, or selectors.
|
|
85
|
+
|
|
86
|
+
Scaffolding rejects pages without a discoverable fillable field or submit
|
|
87
|
+
control, form-less pages without an explicit success condition, remote HTTP
|
|
88
|
+
login pages, cross-origin base URLs, malformed success options, symlinked or
|
|
89
|
+
permissive auth paths, and invalid or non-empty existing configuration. Report
|
|
90
|
+
these blockers; do not bypass them. No generated profile is usable until the
|
|
91
|
+
user replaces its secret placeholders.
|
|
92
|
+
|
|
93
|
+
## Blocked auth runs
|
|
94
|
+
|
|
95
|
+
If a scaffold/profile command or an authenticated run returns
|
|
96
|
+
`QA_AUTH_UPDATE_REQUIRED`, stop and explicitly report that authenticated
|
|
97
|
+
browser QA requires credentials or an auth-config update. Ask the user to fill
|
|
98
|
+
the reported file and rerun QA. If `templateCreated` is true, say that a
|
|
99
|
+
private empty template was created at that path. Relay only the runner's
|
|
100
|
+
profile, file, reason, action, and template-created state; never read the
|
|
101
|
+
generated file or attempt to recover by exposing or replaying credentials. If
|
|
102
|
+
the action is `fill_credentials`, also relay the placeholder count and tell the
|
|
103
|
+
user to replace only the generated placeholder values.
|
|
104
|
+
|
|
105
|
+
For any other blocked run, report the runner status and redacted reason. Do not
|
|
106
|
+
claim that browser QA passed based on source inspection, unit tests, or a
|
|
107
|
+
build.
|
|
108
|
+
|
|
109
|
+
After any runner invocation that actually performed browser testing, include
|
|
110
|
+
all non-empty `artifacts.screenshots`, `artifacts.videos`, and
|
|
111
|
+
`artifacts.traces`, and `artifacts.downloads` groups in the final response.
|
|
112
|
+
These links are mandatory so the user can open the evidence directly. Also
|
|
113
|
+
include `visualInspection` with `inspected` or `unavailable`; never infer a
|
|
114
|
+
visual pass from a successful runner status alone.
|
package/external/pi-tools-suite/src/async-subagents/agents/ui-qa/guides/browser-chrome-devtools.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# Browser detail: Chrome DevTools
|
|
2
|
+
|
|
3
|
+
Load this topic only when the browser router or `selection.guide` selected
|
|
4
|
+
`chrome-devtools`. Probe/run still goes through `PI_UI_QA_RUNNER`; never invoke
|
|
5
|
+
`chrome-devtools` directly from the QA child.
|
|
6
|
+
|
|
7
|
+
## Runner-owned security boundary
|
|
8
|
+
|
|
9
|
+
The provider requires a compatible `chrome-devtools-mcp` CLI. The runner owns a
|
|
10
|
+
random per-run daemon session and stops only that session. It starts an isolated
|
|
11
|
+
Chrome/profile by default and applies fail-closed defaults: JavaScript
|
|
12
|
+
evaluation and optional extension/PWA/experimental categories are disabled,
|
|
13
|
+
usage statistics and CrUX lookup are disabled, network headers are redacted,
|
|
14
|
+
writes are restricted to the run evidence directory, page-id routing is
|
|
15
|
+
enabled, and exact target/allowed origins become the network allowlist.
|
|
16
|
+
|
|
17
|
+
If the CLI/runtime is missing or too old, relay the runner's `BLOCKED` handoff;
|
|
18
|
+
do not install or upgrade it from this child.
|
|
19
|
+
|
|
20
|
+
## Target options
|
|
21
|
+
|
|
22
|
+
Use `target.browserDriver: "chrome-devtools"` when explicit routing is needed.
|
|
23
|
+
`target.profile` is not accepted: trusted QA authentication belongs to the
|
|
24
|
+
Playwright route and credentials must never be moved into DevTools arguments,
|
|
25
|
+
profiles, environment, or JavaScript.
|
|
26
|
+
|
|
27
|
+
Optional `target.devtools` fields:
|
|
28
|
+
|
|
29
|
+
- `headless`: boolean for runner-owned isolated Chrome, default `true`;
|
|
30
|
+
- `browserUrl`: exact credential-free loopback HTTP endpoint for an already
|
|
31
|
+
debug-enabled Chrome; remote/non-loopback endpoints are rejected;
|
|
32
|
+
- `reuseExistingBrowserSession`: boolean, valid only with `browserUrl`, and only
|
|
33
|
+
when the parent/user explicitly requires the existing authenticated session.
|
|
34
|
+
|
|
35
|
+
Even when attached, the default is a task-owned isolated page/context. The
|
|
36
|
+
runner closes only its created page and private daemon; it never stops external
|
|
37
|
+
Chrome or closes unrelated tabs.
|
|
38
|
+
|
|
39
|
+
Top-level deterministic `environment` is intentionally unsupported on this
|
|
40
|
+
route. Split diagnostics into a separate flow instead of approximating the
|
|
41
|
+
Playwright locale/timezone/motion contract.
|
|
42
|
+
|
|
43
|
+
## DevTools flow surface
|
|
44
|
+
|
|
45
|
+
Common actions supported here:
|
|
46
|
+
|
|
47
|
+
- navigation/stability: `goto`, `reload`, `waitFor`, `waitForTimeout`;
|
|
48
|
+
- interaction: `click`, `doubleClick`, `hover`, `fill`, `press`;
|
|
49
|
+
- assertions: `assertVisible`, `assertText`, `assertURL`;
|
|
50
|
+
- evidence: `screenshot`.
|
|
51
|
+
|
|
52
|
+
DevTools-specific actions:
|
|
53
|
+
|
|
54
|
+
- `snapshotAccessibility`: retain a structured accessibility-tree JSON;
|
|
55
|
+
- `assertNoConsoleErrors`: require zero console errors;
|
|
56
|
+
- `assertConsole`: match exactly one of `equals`/`includes`, optional `type`;
|
|
57
|
+
- `assertNetworkRequest`: exact origin-relative `path`, uppercase `method`,
|
|
58
|
+
numeric `status`; optional `origin` must already be allowlisted;
|
|
59
|
+
- `lighthouse`: sanitized summary, mode `navigation` or `snapshot`, optional
|
|
60
|
+
`desktop`/`mobile` device;
|
|
61
|
+
- `performanceTrace`: temporary Chrome trace -> bounded numeric/insight summary
|
|
62
|
+
-> raw trace deletion;
|
|
63
|
+
- `heapSummary`: temporary heapsnapshot -> sanitized numeric summary -> raw
|
|
64
|
+
snapshot close/deletion.
|
|
65
|
+
|
|
66
|
+
Locators are deliberately narrow: `{role,name?,exact?}` or `{text,exact?}`.
|
|
67
|
+
Each locator resolves against the latest accessibility snapshot to one unique
|
|
68
|
+
UID; missing or ambiguous matches fail. If the requested scenario needs richer
|
|
69
|
+
locators, frames/popups, trusted profiles, downloads, or actions absent above,
|
|
70
|
+
use the Playwright route rather than weakening the test.
|
|
71
|
+
|
|
72
|
+
## Oracles, privacy, and evidence
|
|
73
|
+
|
|
74
|
+
Assertions remain the pass/fail oracle. Console/network checks publish only the
|
|
75
|
+
bounded matcher result needed for QA; raw network headers and response bodies
|
|
76
|
+
are not evidence. Never place secrets in expected console text, paths, or
|
|
77
|
+
origins.
|
|
78
|
+
|
|
79
|
+
Retained DevTools evidence may include explicit PNGs, accessibility snapshots,
|
|
80
|
+
sanitized Lighthouse summaries, sanitized performance summaries, and sanitized
|
|
81
|
+
heap summaries. Raw performance traces can contain URLs/query strings/network
|
|
82
|
+
and call-frame details, so they are temporary. Raw heapsnapshots are temporary
|
|
83
|
+
as well. Neither raw form is retained by the runner.
|
|
84
|
+
|
|
85
|
+
This route does not currently promise automatic browser video, downloads, or a
|
|
86
|
+
Playwright trace. Missing evidence types are capability differences, not a
|
|
87
|
+
reason to synthesize evidence from another surface.
|
|
88
|
+
|
|
89
|
+
Inspect representative screenshots before visual claims. Let the runner own
|
|
90
|
+
the page/context, daemon, evidence, and cleanup; never issue provider-level
|
|
91
|
+
start/stop or tab cleanup commands yourself.
|