@fredcallagan/arn-spark 5.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +9 -0
- package/.opencode/plugins/arn-spark.js +272 -0
- package/package.json +17 -0
- package/plugins/arn-spark/.claude-plugin/plugin.json +9 -0
- package/plugins/arn-spark/LICENSE +21 -0
- package/plugins/arn-spark/README.md +25 -0
- package/plugins/arn-spark/agents/arn-spark-brand-strategist.md +299 -0
- package/plugins/arn-spark/agents/arn-spark-dev-env-builder.md +228 -0
- package/plugins/arn-spark/agents/arn-spark-doctor.md +92 -0
- package/plugins/arn-spark/agents/arn-spark-forensic-investigator.md +181 -0
- package/plugins/arn-spark/agents/arn-spark-market-researcher.md +232 -0
- package/plugins/arn-spark/agents/arn-spark-marketing-pm.md +225 -0
- package/plugins/arn-spark/agents/arn-spark-persona-architect.md +259 -0
- package/plugins/arn-spark/agents/arn-spark-persona-impersonator.md +183 -0
- package/plugins/arn-spark/agents/arn-spark-product-strategist.md +191 -0
- package/plugins/arn-spark/agents/arn-spark-prototype-builder.md +497 -0
- package/plugins/arn-spark/agents/arn-spark-scaffolder.md +228 -0
- package/plugins/arn-spark/agents/arn-spark-spike-runner.md +209 -0
- package/plugins/arn-spark/agents/arn-spark-style-capture.md +196 -0
- package/plugins/arn-spark/agents/arn-spark-tech-evaluator.md +229 -0
- package/plugins/arn-spark/agents/arn-spark-ui-interactor.md +235 -0
- package/plugins/arn-spark/agents/arn-spark-use-case-writer.md +280 -0
- package/plugins/arn-spark/agents/arn-spark-ux-judge.md +215 -0
- package/plugins/arn-spark/agents/arn-spark-ux-specialist.md +200 -0
- package/plugins/arn-spark/agents/arn-spark-visual-sketcher.md +285 -0
- package/plugins/arn-spark/agents/arn-spark-visual-test-engineer.md +224 -0
- package/plugins/arn-spark/references/copilot-tools.md +62 -0
- package/plugins/arn-spark/skills/arn-brainstorming/SKILL.md +520 -0
- package/plugins/arn-spark/skills/arn-brainstorming/references/add-feature-flow.md +155 -0
- package/plugins/arn-spark/skills/arn-spark-arch-vision/SKILL.md +226 -0
- package/plugins/arn-spark/skills/arn-spark-arch-vision/references/architecture-vision-template.md +153 -0
- package/plugins/arn-spark/skills/arn-spark-arch-vision/references/technology-evaluation-guide.md +86 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype/SKILL.md +471 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype/references/clickable-prototype-criteria.md +65 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype/references/journey-template.md +62 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype/references/review-report-template.md +75 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype/references/showcase-capture-guide.md +213 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype-teams/SKILL.md +642 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype-teams/references/debate-protocol.md +242 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype-teams/references/debate-review-report-template.md +161 -0
- package/plugins/arn-spark/skills/arn-spark-clickable-prototype-teams/references/expert-interaction-review-template.md +152 -0
- package/plugins/arn-spark/skills/arn-spark-concept-review/SKILL.md +350 -0
- package/plugins/arn-spark/skills/arn-spark-concept-review/references/conflict-resolution-protocol.md +145 -0
- package/plugins/arn-spark/skills/arn-spark-concept-review/references/review-report-template.md +185 -0
- package/plugins/arn-spark/skills/arn-spark-dev-setup/SKILL.md +366 -0
- package/plugins/arn-spark/skills/arn-spark-dev-setup/references/dev-setup-checklist.md +84 -0
- package/plugins/arn-spark/skills/arn-spark-dev-setup/references/dev-setup-template.md +205 -0
- package/plugins/arn-spark/skills/arn-spark-discover/SKILL.md +303 -0
- package/plugins/arn-spark/skills/arn-spark-discover/references/competitive-landscape-template.md +87 -0
- package/plugins/arn-spark/skills/arn-spark-discover/references/discovery-questions.md +120 -0
- package/plugins/arn-spark/skills/arn-spark-discover/references/persona-profile-template.md +97 -0
- package/plugins/arn-spark/skills/arn-spark-discover/references/product-concept-template.md +253 -0
- package/plugins/arn-spark/skills/arn-spark-ensure-config/SKILL.md +23 -0
- package/plugins/arn-spark/skills/arn-spark-ensure-config/references/ensure-config.md +388 -0
- package/plugins/arn-spark/skills/arn-spark-ensure-config/references/step-0-fast-path.md +25 -0
- package/plugins/arn-spark/skills/arn-spark-ensure-config/scripts/cache-check.sh +127 -0
- package/plugins/arn-spark/skills/arn-spark-feature-extract/SKILL.md +483 -0
- package/plugins/arn-spark/skills/arn-spark-feature-extract/references/feature-backlog-template.md +176 -0
- package/plugins/arn-spark/skills/arn-spark-feature-extract/references/feature-entry-template.md +209 -0
- package/plugins/arn-spark/skills/arn-spark-help/SKILL.md +149 -0
- package/plugins/arn-spark/skills/arn-spark-help/references/pipeline-map.md +211 -0
- package/plugins/arn-spark/skills/arn-spark-init/SKILL.md +312 -0
- package/plugins/arn-spark/skills/arn-spark-init/references/agent-models-presets/all-opus.md +23 -0
- package/plugins/arn-spark/skills/arn-spark-init/references/agent-models-presets/balanced.md +23 -0
- package/plugins/arn-spark/skills/arn-spark-init/references/bkt-setup.md +55 -0
- package/plugins/arn-spark/skills/arn-spark-init/references/jira-mcp-setup.md +61 -0
- package/plugins/arn-spark/skills/arn-spark-init/references/platform-labels.md +97 -0
- package/plugins/arn-spark/skills/arn-spark-naming/SKILL.md +275 -0
- package/plugins/arn-spark/skills/arn-spark-naming/references/creative-brief-template.md +146 -0
- package/plugins/arn-spark/skills/arn-spark-naming/references/naming-methodology.md +237 -0
- package/plugins/arn-spark/skills/arn-spark-naming/references/naming-report-template.md +122 -0
- package/plugins/arn-spark/skills/arn-spark-naming/references/trademark-databases.md +88 -0
- package/plugins/arn-spark/skills/arn-spark-naming/references/whois-server-map.md +164 -0
- package/plugins/arn-spark/skills/arn-spark-naming/scripts/whois-check.js +502 -0
- package/plugins/arn-spark/skills/arn-spark-naming/scripts/whois-check.py +533 -0
- package/plugins/arn-spark/skills/arn-spark-prototype-lock/SKILL.md +260 -0
- package/plugins/arn-spark/skills/arn-spark-prototype-lock/references/lock-report-template.md +68 -0
- package/plugins/arn-spark/skills/arn-spark-prototype-lock/references/pretooluse-hook-template.json +35 -0
- package/plugins/arn-spark/skills/arn-spark-prototype-lock/references/prototype-guardrail-rules.md +38 -0
- package/plugins/arn-spark/skills/arn-spark-report/SKILL.md +144 -0
- package/plugins/arn-spark/skills/arn-spark-report/references/issue-template.md +81 -0
- package/plugins/arn-spark/skills/arn-spark-report/references/spark-knowledge-base.md +293 -0
- package/plugins/arn-spark/skills/arn-spark-scaffold/SKILL.md +239 -0
- package/plugins/arn-spark/skills/arn-spark-scaffold/references/scaffold-checklist.md +79 -0
- package/plugins/arn-spark/skills/arn-spark-scaffold/references/scaffold-summary-template.md +74 -0
- package/plugins/arn-spark/skills/arn-spark-spike/SKILL.md +209 -0
- package/plugins/arn-spark/skills/arn-spark-spike/references/spike-report-template.md +123 -0
- package/plugins/arn-spark/skills/arn-spark-static-prototype/SKILL.md +362 -0
- package/plugins/arn-spark/skills/arn-spark-static-prototype/references/review-report-template.md +65 -0
- package/plugins/arn-spark/skills/arn-spark-static-prototype/references/showcase-capture-guide.md +153 -0
- package/plugins/arn-spark/skills/arn-spark-static-prototype/references/static-prototype-criteria.md +54 -0
- package/plugins/arn-spark/skills/arn-spark-static-prototype-teams/SKILL.md +518 -0
- package/plugins/arn-spark/skills/arn-spark-static-prototype-teams/references/debate-protocol.md +230 -0
- package/plugins/arn-spark/skills/arn-spark-static-prototype-teams/references/debate-review-report-template.md +148 -0
- package/plugins/arn-spark/skills/arn-spark-static-prototype-teams/references/expert-visual-review-template.md +130 -0
- package/plugins/arn-spark/skills/arn-spark-stress-competitive/SKILL.md +166 -0
- package/plugins/arn-spark/skills/arn-spark-stress-competitive/references/competitive-report-template.md +139 -0
- package/plugins/arn-spark/skills/arn-spark-stress-competitive/references/gap-analysis-framework.md +111 -0
- package/plugins/arn-spark/skills/arn-spark-stress-interview/SKILL.md +257 -0
- package/plugins/arn-spark/skills/arn-spark-stress-interview/references/interview-protocol.md +140 -0
- package/plugins/arn-spark/skills/arn-spark-stress-interview/references/interview-report-template.md +165 -0
- package/plugins/arn-spark/skills/arn-spark-stress-interview/references/persona-casting-spec.md +138 -0
- package/plugins/arn-spark/skills/arn-spark-stress-premortem/SKILL.md +181 -0
- package/plugins/arn-spark/skills/arn-spark-stress-premortem/references/premortem-protocol.md +112 -0
- package/plugins/arn-spark/skills/arn-spark-stress-premortem/references/premortem-report-template.md +158 -0
- package/plugins/arn-spark/skills/arn-spark-stress-prfaq/SKILL.md +206 -0
- package/plugins/arn-spark/skills/arn-spark-stress-prfaq/references/prfaq-report-template.md +139 -0
- package/plugins/arn-spark/skills/arn-spark-stress-prfaq/references/prfaq-workflow.md +118 -0
- package/plugins/arn-spark/skills/arn-spark-style-explore/SKILL.md +281 -0
- package/plugins/arn-spark/skills/arn-spark-style-explore/references/style-brief-template.md +198 -0
- package/plugins/arn-spark/skills/arn-spark-use-cases/SKILL.md +359 -0
- package/plugins/arn-spark/skills/arn-spark-use-cases/references/expert-review-template.md +94 -0
- package/plugins/arn-spark/skills/arn-spark-use-cases/references/review-protocol.md +150 -0
- package/plugins/arn-spark/skills/arn-spark-use-cases/references/use-case-index-template.md +108 -0
- package/plugins/arn-spark/skills/arn-spark-use-cases/references/use-case-template.md +125 -0
- package/plugins/arn-spark/skills/arn-spark-use-cases-teams/SKILL.md +306 -0
- package/plugins/arn-spark/skills/arn-spark-use-cases-teams/references/debate-protocol.md +272 -0
- package/plugins/arn-spark/skills/arn-spark-use-cases-teams/references/review-report-template.md +112 -0
- package/plugins/arn-spark/skills/arn-spark-visual-readiness/SKILL.md +293 -0
- package/plugins/arn-spark/skills/arn-spark-visual-readiness/references/readiness-checklist.md +196 -0
- package/plugins/arn-spark/skills/arn-spark-visual-sketch/SKILL.md +376 -0
- package/plugins/arn-spark/skills/arn-spark-visual-sketch/references/aesthetic-philosophy.md +210 -0
- package/plugins/arn-spark/skills/arn-spark-visual-sketch/references/sketch-gallery-guide.md +282 -0
- package/plugins/arn-spark/skills/arn-spark-visual-sketch/references/visual-direction-template.md +174 -0
- package/plugins/arn-spark/skills/arn-spark-visual-strategy/SKILL.md +447 -0
- package/plugins/arn-spark/skills/arn-spark-visual-strategy/references/baseline-capture-script-template.js +89 -0
- package/plugins/arn-spark/skills/arn-spark-visual-strategy/references/journey-schema.md +375 -0
- package/plugins/arn-spark/skills/arn-spark-visual-strategy/references/spike-checklist.md +122 -0
- package/plugins/arn-spark/skills/arn-spark-visual-strategy/references/strategy-layers-guide.md +132 -0
- package/plugins/arn-spark/skills/arn-spark-visual-strategy/references/visual-strategy-template.md +141 -0
|
@@ -0,0 +1,375 @@
|
|
|
1
|
+
# Journey Schema Reference
|
|
2
|
+
|
|
3
|
+
This document defines the platform-agnostic journey definition format used for Layer 2 journey-based interaction testing. It is the canonical reference for all skills and agents that create, validate, or execute journey definitions.
|
|
4
|
+
|
|
5
|
+
## Journey Definition Schema
|
|
6
|
+
|
|
7
|
+
A journey is a sequence of steps that model a user interaction flow. Each step has a `type` field that determines the required and optional fields.
|
|
8
|
+
|
|
9
|
+
### Step Types
|
|
10
|
+
|
|
11
|
+
| Type | Required Fields | Optional Fields | Description |
|
|
12
|
+
|------|----------------|-----------------|-------------|
|
|
13
|
+
| `capture` | `name` | `delay` | Takes a screenshot at a checkpoint. The `name` field becomes the screenshot filename and is used for cross-layer matching. |
|
|
14
|
+
| `invoke` | `target` | `delay` | Clicks or activates a UI element. The `target` field is a selector object (see Element Selector Format). |
|
|
15
|
+
| `setValue` | `target`, `value` | `delay` | Sets a value on an input element. The `target` field is a selector object and `value` is the string to enter. |
|
|
16
|
+
| `settle` | _(none)_ | `timeout`, `condition` | Waits for the UI to stabilize before proceeding. Default timeout is 5000ms. The optional `condition` field describes what "settled" means (e.g., "loading spinner disappears"). |
|
|
17
|
+
|
|
18
|
+
### Field Definitions
|
|
19
|
+
|
|
20
|
+
- **`name`** (string) -- Unique identifier for the capture step within its journey. Used as the screenshot filename and for cross-layer screen matching.
|
|
21
|
+
- **`target`** (object) -- Selector object identifying the UI element to interact with. See Element Selector Format below.
|
|
22
|
+
- **`value`** (string) -- The text or value to set on the target element.
|
|
23
|
+
- **`delay`** (number) -- Milliseconds to wait before executing the step. Default: 0.
|
|
24
|
+
- **`timeout`** (number) -- Maximum milliseconds to wait for the settle condition. Default: 5000.
|
|
25
|
+
- **`condition`** (string) -- Human-readable description of what "settled" means. Used by the runner to determine the appropriate wait strategy.
|
|
26
|
+
|
|
27
|
+
### Example: Login Flow Journey
|
|
28
|
+
|
|
29
|
+
```json
|
|
30
|
+
{
|
|
31
|
+
"name": "login-flow",
|
|
32
|
+
"description": "Navigates to login screen, enters credentials, submits, and captures the resulting dashboard",
|
|
33
|
+
"preconditions": [
|
|
34
|
+
"Application is running and showing the main window",
|
|
35
|
+
"No user is currently logged in"
|
|
36
|
+
],
|
|
37
|
+
"steps": [
|
|
38
|
+
{
|
|
39
|
+
"type": "capture",
|
|
40
|
+
"name": "login-screen"
|
|
41
|
+
},
|
|
42
|
+
{
|
|
43
|
+
"type": "setValue",
|
|
44
|
+
"target": { "automationId": "usernameInput", "controlType": "textBox" },
|
|
45
|
+
"value": "testuser@example.com"
|
|
46
|
+
},
|
|
47
|
+
{
|
|
48
|
+
"type": "setValue",
|
|
49
|
+
"target": { "automationId": "passwordInput", "controlType": "textBox" },
|
|
50
|
+
"value": "test-password-123"
|
|
51
|
+
},
|
|
52
|
+
{
|
|
53
|
+
"type": "invoke",
|
|
54
|
+
"target": { "automationId": "loginButton", "controlType": "button" },
|
|
55
|
+
"delay": 200
|
|
56
|
+
},
|
|
57
|
+
{
|
|
58
|
+
"type": "settle",
|
|
59
|
+
"timeout": 10000,
|
|
60
|
+
"condition": "Loading spinner disappears and dashboard content is visible"
|
|
61
|
+
},
|
|
62
|
+
{
|
|
63
|
+
"type": "capture",
|
|
64
|
+
"name": "dashboard-after-login"
|
|
65
|
+
}
|
|
66
|
+
]
|
|
67
|
+
}
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Element Selector Format
|
|
71
|
+
|
|
72
|
+
A selector object identifies a UI element in the application's accessibility tree. Selectors are platform-agnostic -- the platform runner translates them into native API calls.
|
|
73
|
+
|
|
74
|
+
### Selector Properties
|
|
75
|
+
|
|
76
|
+
| Property | Type | Description |
|
|
77
|
+
|----------|------|-------------|
|
|
78
|
+
| `automationId` | string | The element's automation ID (highest priority). Maps to `AutomationId` on Windows UIA and `identifier` on macOS NSAccessibility. |
|
|
79
|
+
| `controlType` | string | The logical control type (e.g., `button`, `textBox`, `menuItem`). Maps to platform-specific control type enumerations. See the Element Mapping Table below. |
|
|
80
|
+
| `name` | string | The element's accessible name (fallback). Maps to `Name` on Windows UIA and `AXTitle` or `AXDescription` on macOS NSAccessibility. |
|
|
81
|
+
|
|
82
|
+
### Resolution Order
|
|
83
|
+
|
|
84
|
+
The platform runner resolves selectors in the following priority order:
|
|
85
|
+
|
|
86
|
+
1. **Custom mappings lookup** -- If the `target` value is a string (logical name), the runner looks it up in the `customMappings` section of the journey manifest. This returns a platform-specific selector with full native properties.
|
|
87
|
+
2. **Direct `automationId`** -- If the selector has an `automationId` field, the runner searches the accessibility tree for an element with that automation ID. This is the most reliable cross-platform identifier.
|
|
88
|
+
3. **`controlType` + `name` combination** -- If `automationId` is absent or resolution fails, the runner searches for an element matching both the `controlType` (mapped to the platform's native type) and the `name` (accessible name). This is the fallback strategy.
|
|
89
|
+
|
|
90
|
+
If all resolution strategies fail, the runner reports a `SELECTOR_NOT_FOUND` error for the step.
|
|
91
|
+
|
|
92
|
+
## Custom Mappings
|
|
93
|
+
|
|
94
|
+
The `customMappings` section in the journey manifest maps logical component names to platform-specific selectors. This bridges the gap between the journey's platform-agnostic step definitions and the actual accessibility tree of the implementation.
|
|
95
|
+
|
|
96
|
+
### Structure
|
|
97
|
+
|
|
98
|
+
```json
|
|
99
|
+
{
|
|
100
|
+
"customMappings": {
|
|
101
|
+
"main-navigation": {
|
|
102
|
+
"windows": {
|
|
103
|
+
"automationId": "NavHost",
|
|
104
|
+
"controlType": "Pane",
|
|
105
|
+
"className": "NavigationView"
|
|
106
|
+
},
|
|
107
|
+
"macos": {
|
|
108
|
+
"role": "AXGroup",
|
|
109
|
+
"subrole": "AXNavigationBar",
|
|
110
|
+
"identifier": "MainNavigation"
|
|
111
|
+
}
|
|
112
|
+
},
|
|
113
|
+
"data-table": {
|
|
114
|
+
"windows": {
|
|
115
|
+
"automationId": "MainDataGrid",
|
|
116
|
+
"controlType": "DataGrid",
|
|
117
|
+
"className": "DataGridView"
|
|
118
|
+
},
|
|
119
|
+
"macos": {
|
|
120
|
+
"role": "AXTable",
|
|
121
|
+
"subrole": "",
|
|
122
|
+
"identifier": "MainDataGrid"
|
|
123
|
+
}
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
### When to Use Custom Mappings
|
|
130
|
+
|
|
131
|
+
Custom mappings are needed when standard selectors (automationId, controlType, name) are insufficient:
|
|
132
|
+
|
|
133
|
+
- **Complex or non-standard components** -- Data grids, chart widgets, custom-drawn controls, and composite components that expose non-obvious accessibility trees.
|
|
134
|
+
- **Platform-divergent accessibility trees** -- Components whose accessibility structure differs significantly between Windows UIA and macOS NSAccessibility (e.g., a navigation rail that is a `Pane` on Windows but an `AXGroup` on macOS).
|
|
135
|
+
- **Components with unreliable `automationId`** -- Controls generated by frameworks that assign dynamic or unstable automation IDs.
|
|
136
|
+
- **Deeply nested elements** -- Components where the target element is several levels deep in the accessibility tree and requires class name or subrole disambiguation.
|
|
137
|
+
|
|
138
|
+
**Note:** `arn-spark-visual-test-engineer` populates custom mappings automatically by scanning the actual implementation's accessibility tree during the spike phase. Manual editing is only needed for fine-tuning.
|
|
139
|
+
|
|
140
|
+
### Platform-Specific Properties
|
|
141
|
+
|
|
142
|
+
**Windows (UIA):**
|
|
143
|
+
- `automationId` -- Maps to `AutomationElement.Current.AutomationId`
|
|
144
|
+
- `controlType` -- Maps to `ControlType.ControlTypeId` (e.g., `DataGrid`, `Pane`)
|
|
145
|
+
- `className` -- Maps to `AutomationElement.Current.ClassName` (disambiguation)
|
|
146
|
+
|
|
147
|
+
**macOS (NSAccessibility):**
|
|
148
|
+
- `role` -- Maps to `AXRole` (e.g., `AXTable`, `AXGroup`, `AXButton`)
|
|
149
|
+
- `subrole` -- Maps to `AXSubrole` (e.g., `AXNavigationBar`, `AXSortButton`)
|
|
150
|
+
- `identifier` -- Maps to `AXIdentifier` (accessibility identifier set by the developer)
|
|
151
|
+
|
|
152
|
+
## UIA / NSAccessibility Element Mapping Table
|
|
153
|
+
|
|
154
|
+
This table maps logical control types used in journey step selectors to their platform-native equivalents.
|
|
155
|
+
|
|
156
|
+
| Logical Type | Windows UIA ControlType | Windows UIA Class | macOS NSAccessibility Role | macOS NSAccessibility Subrole |
|
|
157
|
+
|-------------|------------------------|-------------------|---------------------------|------------------------------|
|
|
158
|
+
| `button` | `Button` | `Button` | `AXButton` | -- |
|
|
159
|
+
| `textBox` | `Edit` | `TextBox`, `TextEdit` | `AXTextField` | -- |
|
|
160
|
+
| `checkBox` | `CheckBox` | `CheckBox` | `AXCheckBox` | -- |
|
|
161
|
+
| `radioButton` | `RadioButton` | `RadioButton` | `AXRadioButton` | -- |
|
|
162
|
+
| `comboBox` | `ComboBox` | `ComboBox` | `AXPopUpButton` | -- |
|
|
163
|
+
| `listItem` | `ListItem` | `ListBoxItem` | `AXCell` | `AXOutlineRow` |
|
|
164
|
+
| `menuItem` | `MenuItem` | `MenuItem` | `AXMenuItem` | -- |
|
|
165
|
+
| `tab` | `TabItem` | `TabItem` | `AXRadioButton` | `AXTabButton` |
|
|
166
|
+
| `treeItem` | `TreeItem` | `TreeViewItem` | `AXRow` | `AXOutlineRow` |
|
|
167
|
+
| `slider` | `Slider` | `Slider` | `AXSlider` | -- |
|
|
168
|
+
| `progressBar` | `ProgressBar` | `ProgressBar` | `AXProgressIndicator` | -- |
|
|
169
|
+
| `dataGrid` | `DataGrid` | `DataGridView` | `AXTable` | -- |
|
|
170
|
+
| `link` | `Hyperlink` | `Hyperlink` | `AXLink` | -- |
|
|
171
|
+
| `image` | `Image` | `Image` | `AXImage` | -- |
|
|
172
|
+
| `window` | `Window` | `Window` | `AXWindow` | `AXStandardWindow` |
|
|
173
|
+
| `scrollBar` | `ScrollBar` | `ScrollBar` | `AXScrollBar` | -- |
|
|
174
|
+
| `toolBar` | `ToolBar` | `ToolBar` | `AXToolbar` | -- |
|
|
175
|
+
| `statusBar` | `StatusBar` | `StatusBar` | `AXGroup` | `AXStatusBar` |
|
|
176
|
+
| `dialog` | `Window` | `Dialog` | `AXSheet` | `AXDialog` |
|
|
177
|
+
| `label` | `Text` | `TextBlock` | `AXStaticText` | -- |
|
|
178
|
+
|
|
179
|
+
## Platform Runner Contract
|
|
180
|
+
|
|
181
|
+
Every platform runner must implement the following interface. Runners are platform-specific executables or scripts that consume the platform-agnostic journey manifest and interact with the native accessibility APIs.
|
|
182
|
+
|
|
183
|
+
### Required Interface
|
|
184
|
+
|
|
185
|
+
| Method | Arguments | Returns | Description |
|
|
186
|
+
|--------|-----------|---------|-------------|
|
|
187
|
+
| `Load(manifestPath)` | Path to `journey-manifest.json` | Parsed manifest object | Reads and validates the journey manifest. Fails if the schema version is unsupported or the JSON is malformed. |
|
|
188
|
+
| `Resolve(selector, platform)` | Selector object, platform identifier (`windows` or `macos`) | Native element reference | Finds the UI element using the selector resolution order: custom mappings first, then `automationId`, then `controlType` + `name`. Returns `null` if not found. |
|
|
189
|
+
| `Execute(step, resolvedElement)` | Step object, resolved native element | Step result (pass/fail) | Performs the step action on the resolved element. For `invoke`: activates/clicks. For `setValue`: sets the text/value. For `settle`: waits until timeout or condition met. For `capture`: delegates to `Capture()`. |
|
|
190
|
+
| `Capture(name, outputDir)` | Screenshot name, output directory path | File path of saved screenshot | Takes a screenshot of the application window and saves it as `{outputDir}/{name}.png`. |
|
|
191
|
+
| `Report(results)` | Array of step results | Execution summary | Outputs the full execution results: pass/fail per step, screenshots captured with paths, errors with messages and selectors, total duration. |
|
|
192
|
+
|
|
193
|
+
### Platform-Specific Notes
|
|
194
|
+
|
|
195
|
+
**Windows: PowerShell + System.Windows.Automation**
|
|
196
|
+
|
|
197
|
+
```powershell
|
|
198
|
+
# Load the UIA assembly
|
|
199
|
+
Add-Type -AssemblyName UIAutomationClient
|
|
200
|
+
Add-Type -AssemblyName UIAutomationTypes
|
|
201
|
+
|
|
202
|
+
# Element resolution using PropertyCondition
|
|
203
|
+
$condition = New-Object System.Windows.Automation.PropertyCondition(
|
|
204
|
+
[System.Windows.Automation.AutomationElement]::AutomationIdProperty,
|
|
205
|
+
"loginButton"
|
|
206
|
+
)
|
|
207
|
+
$element = $rootElement.FindFirst(
|
|
208
|
+
[System.Windows.Automation.TreeScope]::Descendants,
|
|
209
|
+
$condition
|
|
210
|
+
)
|
|
211
|
+
|
|
212
|
+
# Invoke action using InvokePattern
|
|
213
|
+
$invokePattern = $element.GetCurrentPattern(
|
|
214
|
+
[System.Windows.Automation.InvokePattern]::Pattern
|
|
215
|
+
)
|
|
216
|
+
$invokePattern.Invoke()
|
|
217
|
+
|
|
218
|
+
# SetValue action using ValuePattern
|
|
219
|
+
$valuePattern = $element.GetCurrentPattern(
|
|
220
|
+
[System.Windows.Automation.ValuePattern]::Pattern
|
|
221
|
+
)
|
|
222
|
+
$valuePattern.SetValue("testuser@example.com")
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
**macOS: AppleScript/Swift + NSAccessibility**
|
|
226
|
+
|
|
227
|
+
```applescript
|
|
228
|
+
-- Element resolution using accessibility identifier
|
|
229
|
+
tell application "System Events"
|
|
230
|
+
tell process "MyApp"
|
|
231
|
+
set targetElement to first text field whose value of attribute "AXIdentifier" is "usernameInput"
|
|
232
|
+
end tell
|
|
233
|
+
end tell
|
|
234
|
+
|
|
235
|
+
-- Invoke action (AXPress)
|
|
236
|
+
tell application "System Events"
|
|
237
|
+
tell process "MyApp"
|
|
238
|
+
click button "Login" of window 1
|
|
239
|
+
end tell
|
|
240
|
+
end tell
|
|
241
|
+
|
|
242
|
+
-- SetValue action (AXValue)
|
|
243
|
+
tell application "System Events"
|
|
244
|
+
tell process "MyApp"
|
|
245
|
+
set value of text field "usernameInput" of window 1 to "testuser@example.com"
|
|
246
|
+
end tell
|
|
247
|
+
end tell
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
For programmatic access, use `osascript -e '...'` to execute AppleScript from shell scripts, or use Swift with the `AXUIElement` API for more complex resolution logic.
|
|
251
|
+
|
|
252
|
+
### Dry-Run Mode
|
|
253
|
+
|
|
254
|
+
Every platform runner must support a `--dry-run` flag that validates the journey without executing actions:
|
|
255
|
+
|
|
256
|
+
1. **Load** -- Parse the manifest (same as normal mode)
|
|
257
|
+
2. **Resolve** -- Attempt to find every element referenced in all journey steps. Report which selectors resolved and which failed.
|
|
258
|
+
3. **Skip Execute** -- Do not perform any invoke, setValue, or settle actions
|
|
259
|
+
4. **Skip Capture** -- Do not take screenshots
|
|
260
|
+
5. **Report** -- Output a validation summary: total selectors, resolved count, failed count, failed selector details
|
|
261
|
+
|
|
262
|
+
Dry-run mode is used during the spike phase to validate that the journey manifest's selectors match the actual application's accessibility tree before committing to a full execution run.
|
|
263
|
+
|
|
264
|
+
## Journey Manifest Schema
|
|
265
|
+
|
|
266
|
+
The `journey-manifest.json` file is the top-level manifest that contains all journey definitions, custom mappings, and capture configuration for a project.
|
|
267
|
+
|
|
268
|
+
### Complete Schema
|
|
269
|
+
|
|
270
|
+
```json
|
|
271
|
+
{
|
|
272
|
+
"$schema": "journey-manifest-v1",
|
|
273
|
+
"generatedBy": "arn-spark-visual-test-engineer",
|
|
274
|
+
"generatedAt": "2026-02-28T14:30:00Z",
|
|
275
|
+
"journeys": [
|
|
276
|
+
{
|
|
277
|
+
"name": "login-flow",
|
|
278
|
+
"description": "Tests the login sequence from initial screen through dashboard load",
|
|
279
|
+
"preconditions": [
|
|
280
|
+
"Application is running and showing the main window",
|
|
281
|
+
"No user is currently logged in"
|
|
282
|
+
],
|
|
283
|
+
"steps": [
|
|
284
|
+
{ "type": "capture", "name": "login-screen" },
|
|
285
|
+
{
|
|
286
|
+
"type": "setValue",
|
|
287
|
+
"target": { "automationId": "usernameInput", "controlType": "textBox" },
|
|
288
|
+
"value": "testuser@example.com"
|
|
289
|
+
},
|
|
290
|
+
{
|
|
291
|
+
"type": "setValue",
|
|
292
|
+
"target": { "automationId": "passwordInput", "controlType": "textBox" },
|
|
293
|
+
"value": "test-password-123"
|
|
294
|
+
},
|
|
295
|
+
{
|
|
296
|
+
"type": "invoke",
|
|
297
|
+
"target": { "automationId": "loginButton", "controlType": "button" },
|
|
298
|
+
"delay": 200
|
|
299
|
+
},
|
|
300
|
+
{
|
|
301
|
+
"type": "settle",
|
|
302
|
+
"timeout": 10000,
|
|
303
|
+
"condition": "Loading spinner disappears and dashboard content is visible"
|
|
304
|
+
},
|
|
305
|
+
{ "type": "capture", "name": "dashboard-after-login" }
|
|
306
|
+
]
|
|
307
|
+
},
|
|
308
|
+
{
|
|
309
|
+
"name": "settings-navigation",
|
|
310
|
+
"description": "Opens settings from the dashboard and captures each settings panel",
|
|
311
|
+
"preconditions": [
|
|
312
|
+
"User is logged in and on the dashboard"
|
|
313
|
+
],
|
|
314
|
+
"steps": [
|
|
315
|
+
{
|
|
316
|
+
"type": "invoke",
|
|
317
|
+
"target": { "automationId": "settingsButton", "controlType": "button" }
|
|
318
|
+
},
|
|
319
|
+
{
|
|
320
|
+
"type": "settle",
|
|
321
|
+
"timeout": 3000,
|
|
322
|
+
"condition": "Settings panel is fully loaded"
|
|
323
|
+
},
|
|
324
|
+
{ "type": "capture", "name": "settings-general" },
|
|
325
|
+
{
|
|
326
|
+
"type": "invoke",
|
|
327
|
+
"target": { "automationId": "appearanceTab", "controlType": "tab" }
|
|
328
|
+
},
|
|
329
|
+
{
|
|
330
|
+
"type": "settle",
|
|
331
|
+
"timeout": 2000
|
|
332
|
+
},
|
|
333
|
+
{ "type": "capture", "name": "settings-appearance" }
|
|
334
|
+
]
|
|
335
|
+
}
|
|
336
|
+
],
|
|
337
|
+
"customMappings": {
|
|
338
|
+
"main-navigation": {
|
|
339
|
+
"windows": {
|
|
340
|
+
"automationId": "NavHost",
|
|
341
|
+
"controlType": "Pane",
|
|
342
|
+
"className": "NavigationView"
|
|
343
|
+
},
|
|
344
|
+
"macos": {
|
|
345
|
+
"role": "AXGroup",
|
|
346
|
+
"subrole": "AXNavigationBar",
|
|
347
|
+
"identifier": "MainNavigation"
|
|
348
|
+
}
|
|
349
|
+
}
|
|
350
|
+
},
|
|
351
|
+
"captureConfig": {
|
|
352
|
+
"outputDir": "visual-tests/baselines/layer-2/journeys/",
|
|
353
|
+
"format": "png",
|
|
354
|
+
"fullWindow": true
|
|
355
|
+
}
|
|
356
|
+
}
|
|
357
|
+
```
|
|
358
|
+
|
|
359
|
+
### Field Reference
|
|
360
|
+
|
|
361
|
+
| Field | Type | Required | Description |
|
|
362
|
+
|-------|------|----------|-------------|
|
|
363
|
+
| `$schema` | string | Yes | Schema version identifier. Must be `"journey-manifest-v1"`. |
|
|
364
|
+
| `generatedBy` | string | Yes | The tool that generated the manifest. Typically `"arn-spark-visual-test-engineer"`. |
|
|
365
|
+
| `generatedAt` | string | Yes | ISO 8601 timestamp of when the manifest was generated. |
|
|
366
|
+
| `journeys` | array | Yes | Array of journey objects. Each journey defines a complete user interaction flow. |
|
|
367
|
+
| `journeys[].name` | string | Yes | Unique identifier for the journey. Used in reporting and file organization. |
|
|
368
|
+
| `journeys[].description` | string | Yes | Human-readable description of what the journey tests. |
|
|
369
|
+
| `journeys[].preconditions` | array | Yes | List of application state requirements that must be true before the journey starts. |
|
|
370
|
+
| `journeys[].steps` | array | Yes | Ordered array of step objects. Each step is one of: `capture`, `invoke`, `setValue`, `settle`. |
|
|
371
|
+
| `customMappings` | object | No | Maps logical component names to platform-specific selectors. See Custom Mappings section. |
|
|
372
|
+
| `captureConfig` | object | Yes | Configuration for screenshot capture. |
|
|
373
|
+
| `captureConfig.outputDir` | string | Yes | Directory where captured screenshots are saved. Relative to project root. |
|
|
374
|
+
| `captureConfig.format` | string | Yes | Image format for captures. Must be `"png"`. |
|
|
375
|
+
| `captureConfig.fullWindow` | boolean | Yes | Whether to capture the full application window (`true`) or just the client area (`false`). |
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# Visual Strategy Spike Checklist
|
|
2
|
+
|
|
3
|
+
Validation criteria used by `arn-spark-visual-strategy` when running mini-spikes to validate each testing layer. The skill reads this checklist and applies the relevant layer's criteria during Step 4 (Validate Strategy with Mini-Spike).
|
|
4
|
+
|
|
5
|
+
## Layer 1: Browser-Based Capture (Playwright)
|
|
6
|
+
|
|
7
|
+
### Prerequisites
|
|
8
|
+
- [ ] Node.js available in the environment
|
|
9
|
+
- [ ] Playwright installable (`npm install -D @playwright/test`)
|
|
10
|
+
- [ ] Dev server starts and responds at the expected URL
|
|
11
|
+
|
|
12
|
+
### Capture Validation
|
|
13
|
+
- [ ] At least one screenshot captured successfully
|
|
14
|
+
- [ ] Screenshot is at the configured viewport size (check image dimensions)
|
|
15
|
+
- [ ] Screenshot content matches the expected screen (not a blank page, error page, or loading spinner)
|
|
16
|
+
- [ ] Multiple screens captured without navigation errors
|
|
17
|
+
- [ ] Screenshots saved to the configured output directory with correct naming
|
|
18
|
+
|
|
19
|
+
### Comparison Validation
|
|
20
|
+
- [ ] Diff tool (pixelmatch/looks-same) installs and runs
|
|
21
|
+
- [ ] Comparing identical images produces a 0% diff (or below threshold)
|
|
22
|
+
- [ ] Comparing intentionally different images produces a detectable diff above threshold
|
|
23
|
+
- [ ] Diff output is human-readable (diff image or percentage report)
|
|
24
|
+
|
|
25
|
+
### Headless Validation
|
|
26
|
+
- [ ] Capture script runs in headless mode (no display server required)
|
|
27
|
+
- [ ] Headless screenshots match headed screenshots (no rendering differences)
|
|
28
|
+
|
|
29
|
+
### Dark Mode (if applicable)
|
|
30
|
+
- [ ] Dark mode toggle mechanism works in Playwright
|
|
31
|
+
- [ ] Dark mode screenshots captured alongside light mode
|
|
32
|
+
- [ ] Dark mode screenshots show correct dark theme (not just light mode duplicated)
|
|
33
|
+
|
|
34
|
+
## Layer 2: Native Application Capture
|
|
35
|
+
|
|
36
|
+
### Prerequisites
|
|
37
|
+
- [ ] Application builds successfully on the target OS
|
|
38
|
+
- [ ] Built application launches and displays the main window
|
|
39
|
+
- [ ] Screenshot capture tool is available (platform-specific)
|
|
40
|
+
|
|
41
|
+
### Build Validation
|
|
42
|
+
- [ ] Build command completes without errors
|
|
43
|
+
- [ ] Build output exists at the expected path
|
|
44
|
+
- [ ] Build output is executable/launchable
|
|
45
|
+
|
|
46
|
+
### Capture Validation
|
|
47
|
+
- [ ] Screenshot captured of the running application window
|
|
48
|
+
- [ ] Screenshot resolution is correct (matches display/window size)
|
|
49
|
+
- [ ] Screenshot includes native window chrome (title bar, borders)
|
|
50
|
+
- [ ] Transparency effects visible in screenshot (if applicable)
|
|
51
|
+
- [ ] Application window identified correctly (not capturing wrong window)
|
|
52
|
+
|
|
53
|
+
### Cross-Environment Validation (WSL2 to Windows, etc.)
|
|
54
|
+
- [ ] File transfer from dev environment to target OS works (e.g., WSL2 -> /mnt/c/...)
|
|
55
|
+
- [ ] Build command runs on target OS from the transferred files
|
|
56
|
+
- [ ] Screenshot capture runs on target OS
|
|
57
|
+
- [ ] Screenshot files transferred back to dev environment
|
|
58
|
+
- [ ] Diff comparison runs on dev environment with the captured screenshots
|
|
59
|
+
|
|
60
|
+
## Layer 2 -- Journey Interaction
|
|
61
|
+
|
|
62
|
+
> Validate only when `Interaction: journey` is selected for Layer 2. These checks extend the standard Layer 2 checklist above.
|
|
63
|
+
|
|
64
|
+
### Prerequisites
|
|
65
|
+
|
|
66
|
+
- [ ] UIA framework available — Windows: `Add-Type -AssemblyName UIAutomationClient` loads successfully; macOS: `osascript` can query accessibility tree
|
|
67
|
+
- [ ] Accessibility tree inspectable — can enumerate UI elements of the target application
|
|
68
|
+
- [ ] Automation IDs present — key interactive elements (buttons, inputs, menus) have automation IDs assigned
|
|
69
|
+
- [ ] Journey manifest exists — `journey-manifest.json` has been generated by `arn-spark-visual-test-engineer`
|
|
70
|
+
- [ ] Runner script exists — platform-specific runner script has been generated
|
|
71
|
+
|
|
72
|
+
### Journey Execution
|
|
73
|
+
|
|
74
|
+
- [ ] Runner loads journey manifest without errors
|
|
75
|
+
- [ ] Runner resolves element selectors (automationId, controlType, name) for all journey steps
|
|
76
|
+
- [ ] Runner executes steps in order (invoke, setValue, settle, capture)
|
|
77
|
+
- [ ] Capture steps produce screenshots at the correct resolution
|
|
78
|
+
- [ ] Complete journey runs end-to-end without crashes or unhandled errors
|
|
79
|
+
|
|
80
|
+
### Interaction Validation
|
|
81
|
+
|
|
82
|
+
- [ ] `invoke` steps click the correct elements (verify via visible state change)
|
|
83
|
+
- [ ] `setValue` steps populate input fields with the specified values (verify via reading the value back)
|
|
84
|
+
- [ ] `settle` steps wait appropriately (page loads complete, animations finish, spinners disappear)
|
|
85
|
+
- [ ] State transitions are visible in captured screenshots (e.g., login form → dashboard)
|
|
86
|
+
|
|
87
|
+
### macOS Permissions
|
|
88
|
+
|
|
89
|
+
- [ ] Terminal/IDE has Accessibility permission granted (System Preferences > Privacy & Security > Accessibility)
|
|
90
|
+
- [ ] `osascript` accessibility queries succeed without authorization errors
|
|
91
|
+
- [ ] If permission is not granted, agent prompts user with step-by-step instructions before retrying
|
|
92
|
+
|
|
93
|
+
### Custom Mapping Validation
|
|
94
|
+
|
|
95
|
+
- [ ] At least one custom mapping resolves correctly on the target platform
|
|
96
|
+
- [ ] Platform-specific selectors (windows/macos) in custom mappings match actual accessibility tree elements
|
|
97
|
+
- [ ] Mapped elements are interactable (can be invoked, can receive values)
|
|
98
|
+
- [ ] Fallback to automationId works when custom mapping is not defined for an element
|
|
99
|
+
|
|
100
|
+
## Layer 3: Extended Validation
|
|
101
|
+
|
|
102
|
+
### Mobile Viewport (if applicable)
|
|
103
|
+
- [ ] Playwright mobile device emulation configured
|
|
104
|
+
- [ ] Screenshots captured at mobile viewport sizes
|
|
105
|
+
- [ ] Touch interactions simulated where relevant
|
|
106
|
+
- [ ] Responsive layout changes visible in captures
|
|
107
|
+
|
|
108
|
+
### CI Integration (if applicable)
|
|
109
|
+
- [ ] CI workflow file is valid YAML
|
|
110
|
+
- [ ] Visual test step runs in the CI pipeline
|
|
111
|
+
- [ ] Screenshots uploaded as CI artifacts
|
|
112
|
+
- [ ] Diff results reported in CI output
|
|
113
|
+
- [ ] Failure (visual regression detected) causes CI step to fail
|
|
114
|
+
|
|
115
|
+
## Spike Result Classification
|
|
116
|
+
|
|
117
|
+
| Result | Criteria |
|
|
118
|
+
|--------|----------|
|
|
119
|
+
| **Validated** | All prerequisite checks pass AND at least capture + comparison validation pass |
|
|
120
|
+
| **Partially validated** | Prerequisites pass, capture works, but comparison has caveats (e.g., anti-aliasing noise) |
|
|
121
|
+
| **Failed** | Prerequisites fail OR capture does not produce usable screenshots |
|
|
122
|
+
| **Deferred** | Cannot be validated in the current environment (e.g., WSL2 cannot run native Windows capture). Scripts created for manual validation on target OS. |
|
package/plugins/arn-spark/skills/arn-spark-visual-strategy/references/strategy-layers-guide.md
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
# Strategy Layers Guide
|
|
2
|
+
|
|
3
|
+
This guide helps `arn-spark-visual-strategy` recommend appropriate visual testing layers based on the project's application type, rendering context, and development environment. Each layer represents a testing approach with different coverage, complexity, and infrastructure requirements.
|
|
4
|
+
|
|
5
|
+
## Layer Decision Matrix
|
|
6
|
+
|
|
7
|
+
| App Type | Rendering Context | Layer 1 (Recommended) | Layer 2 (Optional) | Layer 3 (Advanced) |
|
|
8
|
+
|----------|-------------------|----------------------|--------------------|--------------------|
|
|
9
|
+
| Pure browser SPA | Browser viewport | Playwright against dev server | Mobile viewport emulation (Playwright devices) | Real device testing (BrowserStack / Sauce Labs) |
|
|
10
|
+
| SSR web app (Next.js, Nuxt, SvelteKit) | Browser viewport | Playwright against dev server | Mobile viewport + SSR snapshot testing | Real device testing |
|
|
11
|
+
| Tauri (no transparency) | Webview in native frame | Playwright against dev server (web content) | Tauri CLI build + OS-native screenshot on target OS | CI matrix with per-OS capture |
|
|
12
|
+
| Tauri (with transparency) | Native window with compositor | Playwright against dev server (web content only) | Cross-environment build + native screenshot (e.g., WSL2->Windows) | Per-OS CI runners with display server |
|
|
13
|
+
| Electron | Chromium webview | Playwright against dev server | Electron `BrowserWindow.capturePage()` API | CI matrix with per-OS capture |
|
|
14
|
+
| React Native (web) | Browser viewport | Playwright against dev server | Mobile emulator screenshots (Android Studio / Xcode Simulator) | Real device testing |
|
|
15
|
+
| Progressive Web App | Browser viewport | Playwright against dev server | Mobile viewport + installable PWA testing | Lighthouse visual audits |
|
|
16
|
+
|
|
17
|
+
## Layer Definitions
|
|
18
|
+
|
|
19
|
+
### Layer 1: Browser-Based Capture (Playwright)
|
|
20
|
+
|
|
21
|
+
**What:** Run the project's dev server, connect Playwright (headless Chromium), navigate to each screen/route, capture screenshots at a configured viewport size.
|
|
22
|
+
|
|
23
|
+
**Captures:** All web UI content — layouts, component rendering, typography, colors, spacing, responsive behavior, hover/focus states (via Playwright interactions).
|
|
24
|
+
|
|
25
|
+
**Does NOT capture:** Native window chrome, OS-level transparency/compositing, system tray icons, notification popups, platform-specific font rendering differences.
|
|
26
|
+
|
|
27
|
+
**Requirements:**
|
|
28
|
+
- Node.js runtime
|
|
29
|
+
- Playwright (`npm install -D @playwright/test`)
|
|
30
|
+
- Dev server that serves the web content (e.g., `npm run dev`)
|
|
31
|
+
|
|
32
|
+
**Environment:** Runs anywhere Node.js runs. Works in WSL2, CI runners, Docker containers (with `--no-sandbox`), macOS, Linux, Windows. Headless mode requires no display server.
|
|
33
|
+
|
|
34
|
+
**When to use:** Always. This is the default first layer for any project with web-based UI. It catches the majority of visual regressions (layout shifts, component styling, color changes, text rendering).
|
|
35
|
+
|
|
36
|
+
### Layer 2: Native Application Capture
|
|
37
|
+
|
|
38
|
+
**What:** Build the full application (not just the web content), run it on the target OS, capture a screenshot of the actual application window as rendered by the OS compositor.
|
|
39
|
+
|
|
40
|
+
**Captures:** Everything visible on screen including native window chrome, transparency effects, blur layers, system integration, platform-specific rendering.
|
|
41
|
+
|
|
42
|
+
**Does NOT capture:** Dynamic OS state (notifications from other apps, wallpaper bleeding through transparency), GPU-specific rendering artifacts.
|
|
43
|
+
|
|
44
|
+
**Interaction mode:**
|
|
45
|
+
|
|
46
|
+
Layer 2 supports two interaction modes:
|
|
47
|
+
|
|
48
|
+
| Mode | Description | Requirements |
|
|
49
|
+
|------|-------------|-------------|
|
|
50
|
+
| `static` | Captures predefined screens without user interaction (current default behavior) | Platform access, capture script |
|
|
51
|
+
| `journey` | Walks through the app using ordered step sequences via OS accessibility APIs (UIA / NSAccessibility), capturing screenshots at checkpoints | UIA/NSAccessibility tooling, automation IDs, journey manifest, runner script |
|
|
52
|
+
|
|
53
|
+
Journey mode uses the platform's native UI automation API to simulate real user flows — clicking buttons, filling forms, navigating between screens — and captures screenshots at each checkpoint step. This enables visual regression detection for state transitions, form validation feedback, navigation flows, and dynamic content that static capture cannot test.
|
|
54
|
+
|
|
55
|
+
Journey mode requirements:
|
|
56
|
+
- Platform automation framework available (Windows UIA or macOS NSAccessibility)
|
|
57
|
+
- Target application exposes automation IDs on interactive elements
|
|
58
|
+
- Journey manifest (`journey-manifest.json`) defining the step sequences
|
|
59
|
+
- Platform-specific runner script generated by `arn-spark-visual-test-engineer`
|
|
60
|
+
|
|
61
|
+
> See `journey-schema.md` in this directory for the complete journey definition schema, element selector format, custom mappings, and platform runner contract.
|
|
62
|
+
|
|
63
|
+
When no `**Interaction:**` field is present in the Layer 2 config, the default is `static` (backward compatible).
|
|
64
|
+
|
|
65
|
+
**Requirements vary by app type:**
|
|
66
|
+
|
|
67
|
+
| App Type | Build Command | Capture Method | OS Requirement |
|
|
68
|
+
|----------|--------------|----------------|----------------|
|
|
69
|
+
| Tauri (Windows) | `cargo tauri build` | PowerShell screen capture script or Nircmd screenshot | Windows 10/11 |
|
|
70
|
+
| Tauri (macOS) | `cargo tauri build` | `screencapture -l <window_id>` | macOS with Xcode |
|
|
71
|
+
| Tauri (Linux) | `cargo tauri build` | `xdotool` + `import` (ImageMagick) or `gnome-screenshot` | Linux with X11/Wayland |
|
|
72
|
+
| Electron | `npm run build` | `BrowserWindow.capturePage()` in test harness | Any (Electron bundles Chromium) |
|
|
73
|
+
|
|
74
|
+
**Cross-Environment Patterns:**
|
|
75
|
+
|
|
76
|
+
**WSL2 to Windows (Tauri):**
|
|
77
|
+
1. Copy project source to a Windows-accessible path (`/mnt/c/Users/.../project`)
|
|
78
|
+
2. Run `cargo tauri build` from Windows (PowerShell or CMD)
|
|
79
|
+
3. Launch the built executable on Windows
|
|
80
|
+
4. Capture screenshot using Windows screenshot tool
|
|
81
|
+
5. Copy screenshot back to WSL2 (`/mnt/c/...` -> project directory)
|
|
82
|
+
6. Run diff comparison on WSL2
|
|
83
|
+
|
|
84
|
+
**CI with OS Matrix:**
|
|
85
|
+
```yaml
|
|
86
|
+
strategy:
|
|
87
|
+
matrix:
|
|
88
|
+
os: [ubuntu-latest, windows-latest, macos-latest]
|
|
89
|
+
steps:
|
|
90
|
+
- uses: actions/checkout@v4
|
|
91
|
+
- name: Build application
|
|
92
|
+
run: [build command]
|
|
93
|
+
- name: Capture screenshots
|
|
94
|
+
run: [platform-specific capture]
|
|
95
|
+
- uses: actions/upload-artifact@v4
|
|
96
|
+
with:
|
|
97
|
+
name: visual-captures-${{ matrix.os }}
|
|
98
|
+
path: visual-tests/captures/
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
### Layer 3: Extended Validation
|
|
102
|
+
|
|
103
|
+
Advanced testing approaches for specific needs. Typically added later in the project lifecycle.
|
|
104
|
+
|
|
105
|
+
- **Real device testing** (BrowserStack, Sauce Labs): For mobile apps or cross-browser validation
|
|
106
|
+
- **Accessibility visual testing**: High-contrast mode, reduced motion, screen magnification
|
|
107
|
+
- **Performance visual testing**: Capture at different network speeds, measure visual stability (CLS)
|
|
108
|
+
|
|
109
|
+
## Comparison Tool Options
|
|
110
|
+
|
|
111
|
+
| Tool | Language | Method | Strengths | Weaknesses |
|
|
112
|
+
|------|----------|--------|-----------|------------|
|
|
113
|
+
| pixelmatch | JS | Pixel-by-pixel RGB diff | Fast, lightweight, zero deps | Sensitive to anti-aliasing |
|
|
114
|
+
| looks-same | JS | Perceptual diff with anti-aliasing tolerance | Handles sub-pixel rendering well | Slower than pixelmatch |
|
|
115
|
+
| reg-suit | JS | Full visual regression framework | CI integration, report generation | Heavier setup |
|
|
116
|
+
| BackstopJS | JS | Playwright + resemblejs | Scenario-based, config-driven | Opinionated structure |
|
|
117
|
+
| Percy (commercial) | SaaS | Cloud-based visual review | Handles cross-browser, team review | Requires account, costs money |
|
|
118
|
+
|
|
119
|
+
**Recommendation:** Start with `pixelmatch` for simplicity. If anti-aliasing noise is a problem (common with text rendering), switch to `looks-same` with tolerance configuration.
|
|
120
|
+
|
|
121
|
+
## Threshold Guidelines
|
|
122
|
+
|
|
123
|
+
| Content Type | Recommended Threshold | Rationale |
|
|
124
|
+
|-------------|----------------------|-----------|
|
|
125
|
+
| Layout-heavy screens | 0.1% | Layout shifts are obvious even at low pixel counts |
|
|
126
|
+
| Text-heavy screens | 0.5% | Sub-pixel font rendering varies across environments |
|
|
127
|
+
| Image/gradient screens | 1.0% | Compression and rendering engine differences |
|
|
128
|
+
| Transparency/blur effects | 2.0% | Compositor-dependent rendering |
|
|
129
|
+
| Journey captures (interaction-dependent states) | 2-5% | State transitions introduce timing-dependent rendering variation |
|
|
130
|
+
| Cross-layer comparison (Layer 1 vs Layer 2) | 5-10% | Different rendering engines (browser vs native) produce expected differences |
|
|
131
|
+
|
|
132
|
+
These are starting points. Adjust per-screen based on spike results.
|