agentic-qe 1.9.4 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/qe-api-contract-validator.md +95 -1336
- package/.claude/agents/qe-chaos-engineer.md +152 -1211
- package/.claude/agents/qe-code-complexity.md +144 -707
- package/.claude/agents/qe-coverage-analyzer.md +147 -743
- package/.claude/agents/qe-deployment-readiness.md +143 -1496
- package/.claude/agents/qe-flaky-test-hunter.md +132 -1529
- package/.claude/agents/qe-fleet-commander.md +12 -12
- package/.claude/agents/qe-performance-tester.md +150 -886
- package/.claude/agents/qe-production-intelligence.md +155 -1396
- package/.claude/agents/qe-quality-analyzer.md +6 -6
- package/.claude/agents/qe-quality-gate.md +151 -648
- package/.claude/agents/qe-regression-risk-analyzer.md +132 -1150
- package/.claude/agents/qe-requirements-validator.md +149 -932
- package/.claude/agents/qe-security-scanner.md +157 -797
- package/.claude/agents/qe-test-data-architect.md +96 -1365
- package/.claude/agents/qe-test-executor.md +8 -8
- package/.claude/agents/qe-test-generator.md +145 -1540
- package/.claude/agents/qe-visual-tester.md +153 -1257
- package/.claude/agents/qx-partner.md +235 -0
- package/.claude/agents/subagents/qe-code-reviewer.md +40 -136
- package/.claude/agents/subagents/qe-coverage-gap-analyzer.md +40 -480
- package/.claude/agents/subagents/qe-data-generator.md +41 -125
- package/.claude/agents/subagents/qe-flaky-investigator.md +55 -411
- package/.claude/agents/subagents/qe-integration-tester.md +53 -141
- package/.claude/agents/subagents/qe-performance-validator.md +54 -130
- package/.claude/agents/subagents/qe-security-auditor.md +56 -114
- package/.claude/agents/subagents/qe-test-data-architect-sub.md +57 -548
- package/.claude/agents/subagents/qe-test-implementer.md +58 -551
- package/.claude/agents/subagents/qe-test-refactorer.md +65 -722
- package/.claude/agents/subagents/qe-test-writer.md +63 -726
- package/.claude/skills/skills-manifest.json +632 -0
- package/.claude/skills/testability-scoring/README.md +71 -0
- package/.claude/skills/testability-scoring/SKILL.md +611 -0
- package/.claude/skills/testability-scoring/resources/templates/config.template.js +84 -0
- package/.claude/skills/testability-scoring/resources/templates/testability-scoring.spec.template.js +532 -0
- package/.claude/skills/testability-scoring/scripts/generate-html-report.js +1007 -0
- package/.claude/skills/testability-scoring/scripts/run-assessment.sh +70 -0
- package/CHANGELOG.md +62 -0
- package/README.md +33 -6
- package/dist/agents/QXPartnerAgent.d.ts +139 -0
- package/dist/agents/QXPartnerAgent.d.ts.map +1 -0
- package/dist/agents/QXPartnerAgent.js +769 -0
- package/dist/agents/QXPartnerAgent.js.map +1 -0
- package/dist/agents/index.d.ts +1 -0
- package/dist/agents/index.d.ts.map +1 -1
- package/dist/agents/index.js +82 -2
- package/dist/agents/index.js.map +1 -1
- package/dist/cli/commands/debug/agent.d.ts.map +1 -1
- package/dist/cli/commands/debug/agent.js +19 -6
- package/dist/cli/commands/debug/agent.js.map +1 -1
- package/dist/cli/commands/debug/health-check.js +20 -7
- package/dist/cli/commands/debug/health-check.js.map +1 -1
- package/dist/cli/commands/init-claude-md-template.d.ts +1 -0
- package/dist/cli/commands/init-claude-md-template.d.ts.map +1 -1
- package/dist/cli/commands/init-claude-md-template.js +4 -3
- package/dist/cli/commands/init-claude-md-template.js.map +1 -1
- package/dist/cli/commands/workflow/cancel.d.ts.map +1 -1
- package/dist/cli/commands/workflow/cancel.js +4 -3
- package/dist/cli/commands/workflow/cancel.js.map +1 -1
- package/dist/cli/commands/workflow/list.d.ts.map +1 -1
- package/dist/cli/commands/workflow/list.js +4 -3
- package/dist/cli/commands/workflow/list.js.map +1 -1
- package/dist/cli/commands/workflow/pause.d.ts.map +1 -1
- package/dist/cli/commands/workflow/pause.js +4 -3
- package/dist/cli/commands/workflow/pause.js.map +1 -1
- package/dist/cli/init/claude-config.d.ts.map +1 -1
- package/dist/cli/init/claude-config.js +3 -8
- package/dist/cli/init/claude-config.js.map +1 -1
- package/dist/cli/init/claude-md.d.ts.map +1 -1
- package/dist/cli/init/claude-md.js +44 -2
- package/dist/cli/init/claude-md.js.map +1 -1
- package/dist/cli/init/database-init.js +1 -1
- package/dist/cli/init/index.d.ts.map +1 -1
- package/dist/cli/init/index.js +13 -6
- package/dist/cli/init/index.js.map +1 -1
- package/dist/cli/init/skills.d.ts.map +1 -1
- package/dist/cli/init/skills.js +2 -1
- package/dist/cli/init/skills.js.map +1 -1
- package/dist/core/memory/AgentDBIntegration.d.ts +24 -6
- package/dist/core/memory/AgentDBIntegration.d.ts.map +1 -1
- package/dist/core/memory/AgentDBIntegration.js +66 -10
- package/dist/core/memory/AgentDBIntegration.js.map +1 -1
- package/dist/core/memory/UnifiedMemoryCoordinator.d.ts +341 -0
- package/dist/core/memory/UnifiedMemoryCoordinator.d.ts.map +1 -0
- package/dist/core/memory/UnifiedMemoryCoordinator.js +986 -0
- package/dist/core/memory/UnifiedMemoryCoordinator.js.map +1 -0
- package/dist/core/memory/index.d.ts +5 -0
- package/dist/core/memory/index.d.ts.map +1 -1
- package/dist/core/memory/index.js +23 -1
- package/dist/core/memory/index.js.map +1 -1
- package/dist/core/optimization/SwarmOptimizer.d.ts +185 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts.map +1 -0
- package/dist/core/optimization/SwarmOptimizer.js +631 -0
- package/dist/core/optimization/SwarmOptimizer.js.map +1 -0
- package/dist/core/optimization/index.d.ts +9 -0
- package/dist/core/optimization/index.d.ts.map +1 -0
- package/dist/core/optimization/index.js +25 -0
- package/dist/core/optimization/index.js.map +1 -0
- package/dist/core/optimization/types.d.ts +53 -0
- package/dist/core/optimization/types.d.ts.map +1 -0
- package/dist/core/optimization/types.js +6 -0
- package/dist/core/optimization/types.js.map +1 -0
- package/dist/core/orchestration/PriorityQueue.d.ts +54 -0
- package/dist/core/orchestration/PriorityQueue.d.ts.map +1 -0
- package/dist/core/orchestration/PriorityQueue.js +122 -0
- package/dist/core/orchestration/PriorityQueue.js.map +1 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts +176 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts.map +1 -0
- package/dist/core/orchestration/WorkflowOrchestrator.js +813 -0
- package/dist/core/orchestration/WorkflowOrchestrator.js.map +1 -0
- package/dist/core/orchestration/index.d.ts +7 -0
- package/dist/core/orchestration/index.d.ts.map +1 -0
- package/dist/core/orchestration/index.js +11 -0
- package/dist/core/orchestration/index.js.map +1 -0
- package/dist/core/orchestration/types.d.ts +96 -0
- package/dist/core/orchestration/types.d.ts.map +1 -0
- package/dist/core/orchestration/types.js +6 -0
- package/dist/core/orchestration/types.js.map +1 -0
- package/dist/core/skills/DynamicSkillLoader.d.ts +96 -0
- package/dist/core/skills/DynamicSkillLoader.d.ts.map +1 -0
- package/dist/core/skills/DynamicSkillLoader.js +353 -0
- package/dist/core/skills/DynamicSkillLoader.js.map +1 -0
- package/dist/core/skills/types.d.ts +118 -0
- package/dist/core/skills/types.d.ts.map +1 -0
- package/dist/core/skills/types.js +7 -0
- package/dist/core/skills/types.js.map +1 -0
- package/dist/core/transport/QUICTransport.d.ts +320 -0
- package/dist/core/transport/QUICTransport.d.ts.map +1 -0
- package/dist/core/transport/QUICTransport.js +711 -0
- package/dist/core/transport/QUICTransport.js.map +1 -0
- package/dist/core/transport/index.d.ts +40 -0
- package/dist/core/transport/index.d.ts.map +1 -0
- package/dist/core/transport/index.js +46 -0
- package/dist/core/transport/index.js.map +1 -0
- package/dist/core/transport/quic-loader.d.ts +123 -0
- package/dist/core/transport/quic-loader.d.ts.map +1 -0
- package/dist/core/transport/quic-loader.js +293 -0
- package/dist/core/transport/quic-loader.js.map +1 -0
- package/dist/core/transport/quic.d.ts +154 -0
- package/dist/core/transport/quic.d.ts.map +1 -0
- package/dist/core/transport/quic.js +214 -0
- package/dist/core/transport/quic.js.map +1 -0
- package/dist/mcp/services/AgentRegistry.d.ts.map +1 -1
- package/dist/mcp/services/AgentRegistry.js +4 -1
- package/dist/mcp/services/AgentRegistry.js.map +1 -1
- package/dist/types/index.d.ts +2 -1
- package/dist/types/index.d.ts.map +1 -1
- package/dist/types/index.js +2 -0
- package/dist/types/index.js.map +1 -1
- package/dist/types/qx.d.ts +397 -0
- package/dist/types/qx.d.ts.map +1 -0
- package/dist/types/qx.js +71 -0
- package/dist/types/qx.js.map +1 -0
- package/dist/visualization/api/RestEndpoints.js +1 -1
- package/dist/visualization/api/RestEndpoints.js.map +1 -1
- package/dist/visualization/api/WebSocketServer.d.ts +44 -0
- package/dist/visualization/api/WebSocketServer.d.ts.map +1 -1
- package/dist/visualization/api/WebSocketServer.js +144 -23
- package/dist/visualization/api/WebSocketServer.js.map +1 -1
- package/dist/visualization/core/DataTransformer.d.ts +10 -0
- package/dist/visualization/core/DataTransformer.d.ts.map +1 -1
- package/dist/visualization/core/DataTransformer.js +60 -5
- package/dist/visualization/core/DataTransformer.js.map +1 -1
- package/dist/visualization/emit-event.d.ts +75 -0
- package/dist/visualization/emit-event.d.ts.map +1 -0
- package/dist/visualization/emit-event.js +213 -0
- package/dist/visualization/emit-event.js.map +1 -0
- package/dist/visualization/index.d.ts +1 -0
- package/dist/visualization/index.d.ts.map +1 -1
- package/dist/visualization/index.js +7 -1
- package/dist/visualization/index.js.map +1 -1
- package/docs/reference/skills.md +63 -1
- package/package.json +4 -4
|
@@ -1,345 +1,98 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: qe-visual-tester
|
|
3
|
-
description:
|
|
3
|
+
description: Visual regression testing with AI-powered screenshot comparison and accessibility validation
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
semantic_understanding: true
|
|
82
|
-
},
|
|
83
|
-
thresholds: {
|
|
84
|
-
pixel_diff_threshold: 0.05, // 5% pixels changed
|
|
85
|
-
structural_similarity_threshold: 0.95, // 95% similar
|
|
86
|
-
acceptable_diff_regions: 3 // Max number of different regions
|
|
87
|
-
}
|
|
88
|
-
};
|
|
89
|
-
|
|
90
|
-
// Execute comparison
|
|
91
|
-
const comparisonResults = await compareVisuals({
|
|
92
|
-
baseline: baselines,
|
|
93
|
-
current: visualComparison.current_screenshots,
|
|
94
|
-
strategy: visualComparison.comparison_strategy,
|
|
95
|
-
thresholds: visualComparison.thresholds
|
|
96
|
-
});
|
|
97
|
-
```
|
|
98
|
-
|
|
99
|
-
### Phase 3: Regression Analysis
|
|
100
|
-
```javascript
|
|
101
|
-
// Analyze detected visual differences
|
|
102
|
-
const regressionAnalysis = {
|
|
103
|
-
differences: comparisonResults.differences,
|
|
104
|
-
classification: await classifyDifferences({
|
|
105
|
-
differences: comparisonResults.differences,
|
|
106
|
-
use_ai: true,
|
|
107
|
-
categories: [
|
|
108
|
-
'layout-shift',
|
|
109
|
-
'color-change',
|
|
110
|
-
'font-change',
|
|
111
|
-
'missing-element',
|
|
112
|
-
'new-element',
|
|
113
|
-
'size-change',
|
|
114
|
-
'position-change'
|
|
115
|
-
]
|
|
116
|
-
}),
|
|
117
|
-
severity_assessment: {
|
|
118
|
-
critical: [], // Blocking issues
|
|
119
|
-
high: [], // Major visual regressions
|
|
120
|
-
medium: [], // Minor visual changes
|
|
121
|
-
low: [] // Acceptable variations
|
|
122
|
-
}
|
|
123
|
-
};
|
|
124
|
-
|
|
125
|
-
// Generate regression report
|
|
126
|
-
const regressionReport = generateRegressionReport({
|
|
127
|
-
analysis: regressionAnalysis,
|
|
128
|
-
include_screenshots: true,
|
|
129
|
-
include_diffs: true,
|
|
130
|
-
include_suggestions: true
|
|
131
|
-
});
|
|
6
|
+
<qe_agent_definition>
|
|
7
|
+
<identity>
|
|
8
|
+
You are the Visual Tester Agent for UI/UX validation and accessibility compliance.
|
|
9
|
+
Mission: Detect visual regressions using AI-powered screenshot comparison and validate WCAG 2.1 compliance.
|
|
10
|
+
</identity>
|
|
11
|
+
|
|
12
|
+
<implementation_status>
|
|
13
|
+
✅ Working:
|
|
14
|
+
- AI-powered visual diff with perceptual-hash algorithm
|
|
15
|
+
- Screenshot comparison with semantic understanding
|
|
16
|
+
- WCAG 2.1 AA/AAA accessibility validation
|
|
17
|
+
- Cross-browser testing (Chromium, Firefox, WebKit)
|
|
18
|
+
- Memory coordination via AQE hooks
|
|
19
|
+
|
|
20
|
+
⚠️ Partial:
|
|
21
|
+
- Smart baseline management with auto-update suggestions
|
|
22
|
+
- Responsive design testing across breakpoints
|
|
23
|
+
|
|
24
|
+
❌ Planned:
|
|
25
|
+
- Visual test generation from component libraries
|
|
26
|
+
- Continuous visual monitoring in production
|
|
27
|
+
</implementation_status>
|
|
28
|
+
|
|
29
|
+
<default_to_action>
|
|
30
|
+
Capture and compare screenshots immediately when provided with URLs or baseline versions.
|
|
31
|
+
Make autonomous decisions about visual regression severity based on diff analysis.
|
|
32
|
+
Detect accessibility violations automatically without confirmation.
|
|
33
|
+
Report findings with diff images and remediation guidance.
|
|
34
|
+
</default_to_action>
|
|
35
|
+
|
|
36
|
+
<parallel_execution>
|
|
37
|
+
Capture screenshots across multiple browsers and viewports simultaneously.
|
|
38
|
+
Execute visual comparison and accessibility validation concurrently.
|
|
39
|
+
Process regression classification and compliance checking in parallel.
|
|
40
|
+
Batch memory operations for baselines, regressions, and reports.
|
|
41
|
+
</parallel_execution>
|
|
42
|
+
|
|
43
|
+
<capabilities>
|
|
44
|
+
- **Visual Comparison**: AI-powered screenshot diff with <2% false positive rate
|
|
45
|
+
- **Accessibility Validation**: WCAG 2.1 AA/AAA compliance with color contrast and keyboard navigation checks
|
|
46
|
+
- **Cross-Browser Testing**: Chromium, Firefox, WebKit across desktop/tablet/mobile viewports
|
|
47
|
+
- **Regression Detection**: Semantic understanding of layout shifts, color changes, missing elements
|
|
48
|
+
- **Responsive Testing**: Validate responsive design across 7+ viewport sizes
|
|
49
|
+
- **Learning Integration**: Query past visual patterns and store regression strategies
|
|
50
|
+
</capabilities>
|
|
51
|
+
|
|
52
|
+
<memory_namespace>
|
|
53
|
+
Reads:
|
|
54
|
+
- aqe/visual/baselines - Baseline screenshot repository
|
|
55
|
+
- aqe/visual/test-config - Visual testing configuration
|
|
56
|
+
- aqe/visual/comparison-thresholds - Acceptable diff thresholds
|
|
57
|
+
- aqe/learning/patterns/visual-testing/* - Learned comparison strategies
|
|
58
|
+
|
|
59
|
+
Writes:
|
|
60
|
+
- aqe/visual/test-results - Visual test execution results
|
|
61
|
+
- aqe/visual/regressions - Detected visual regressions with diff images
|
|
62
|
+
- aqe/visual/accessibility-reports - WCAG compliance reports
|
|
63
|
+
- aqe/visual/cross-browser-matrix - Cross-browser test results
|
|
64
|
+
|
|
65
|
+
Coordination:
|
|
66
|
+
- aqe/visual/status - Current visual testing status
|
|
67
|
+
- aqe/visual/alerts - Visual regression alerts
|
|
68
|
+
- aqe/visual/baseline-updates - Pending baseline updates
|
|
69
|
+
</memory_namespace>
|
|
70
|
+
|
|
71
|
+
<learning_protocol>
|
|
72
|
+
Query before testing:
|
|
73
|
+
```javascript
|
|
74
|
+
mcp__agentic_qe__learning_query({
|
|
75
|
+
agentId: "qe-visual-tester",
|
|
76
|
+
taskType: "visual-testing",
|
|
77
|
+
minReward: 0.8,
|
|
78
|
+
queryType: "all",
|
|
79
|
+
limit: 10
|
|
80
|
+
})
|
|
132
81
|
```
|
|
133
82
|
|
|
134
|
-
|
|
83
|
+
Store after completion:
|
|
135
84
|
```javascript
|
|
136
|
-
// Validate WCAG compliance
|
|
137
|
-
const accessibilityTests = {
|
|
138
|
-
standards: ['WCAG-2.1-AA', 'WCAG-2.2-AAA'],
|
|
139
|
-
validations: [
|
|
140
|
-
'color-contrast',
|
|
141
|
-
'keyboard-navigation',
|
|
142
|
-
'screen-reader-compatibility',
|
|
143
|
-
'focus-indicators',
|
|
144
|
-
'alt-text-presence',
|
|
145
|
-
'aria-labels',
|
|
146
|
-
'semantic-html',
|
|
147
|
-
'form-labels',
|
|
148
|
-
'heading-structure'
|
|
149
|
-
],
|
|
150
|
-
tools: ['axe-core', 'pa11y', 'lighthouse-accessibility']
|
|
151
|
-
};
|
|
152
|
-
|
|
153
|
-
// Execute accessibility tests
|
|
154
|
-
const accessibilityResults = await validateAccessibility({
|
|
155
|
-
pages: baselineConfig.pages,
|
|
156
|
-
standards: accessibilityTests.standards,
|
|
157
|
-
validations: accessibilityTests.validations,
|
|
158
|
-
tools: accessibilityTests.tools
|
|
159
|
-
});
|
|
160
|
-
```
|
|
161
|
-
|
|
162
|
-
## Coordination Protocol
|
|
163
|
-
|
|
164
|
-
This agent uses **AQE hooks (Agentic QE native hooks)** for coordination (zero external dependencies, 100-500x faster).
|
|
165
|
-
|
|
166
|
-
**Automatic Lifecycle Hooks:**
|
|
167
|
-
```typescript
|
|
168
|
-
// Called automatically by BaseAgent
|
|
169
|
-
protected async onPreTask(data: { assignment: TaskAssignment }): Promise<void> {
|
|
170
|
-
// Retrieve baselines
|
|
171
|
-
const baselines = await this.memoryStore.retrieve(`aqe/visual/baselines/${this.version}`, {
|
|
172
|
-
partition: 'visual_baselines'
|
|
173
|
-
});
|
|
174
|
-
|
|
175
|
-
// Retrieve test configuration
|
|
176
|
-
const testConfig = await this.memoryStore.retrieve('aqe/visual/test-config', {
|
|
177
|
-
partition: 'configuration'
|
|
178
|
-
});
|
|
179
|
-
|
|
180
|
-
// Verify environment for visual testing
|
|
181
|
-
const verification = await this.hookManager.executePreTaskVerification({
|
|
182
|
-
task: 'visual-testing',
|
|
183
|
-
context: {
|
|
184
|
-
requiredVars: ['BASELINE_VERSION', 'BROWSER'],
|
|
185
|
-
minMemoryMB: 2048,
|
|
186
|
-
requiredKeys: ['aqe/visual/baselines', 'aqe/visual/test-config']
|
|
187
|
-
}
|
|
188
|
-
});
|
|
189
|
-
|
|
190
|
-
// Emit visual testing starting event
|
|
191
|
-
this.eventBus.emit('visual-tester:starting', {
|
|
192
|
-
agentId: this.agentId,
|
|
193
|
-
pagesCount: testConfig.pages.length,
|
|
194
|
-
browser: process.env.BROWSER || 'chromium'
|
|
195
|
-
});
|
|
196
|
-
|
|
197
|
-
this.logger.info('Visual testing starting', {
|
|
198
|
-
pagesCount: testConfig.pages.length,
|
|
199
|
-
verification: verification.passed
|
|
200
|
-
});
|
|
201
|
-
}
|
|
202
|
-
|
|
203
|
-
protected async onPostTask(data: { assignment: TaskAssignment; result: any }): Promise<void> {
|
|
204
|
-
// Store visual test results
|
|
205
|
-
await this.memoryStore.store(`aqe/visual/test-results/${data.result.testRunId}`, data.result, {
|
|
206
|
-
partition: 'visual_results',
|
|
207
|
-
ttl: 86400 // 24 hours
|
|
208
|
-
});
|
|
209
|
-
|
|
210
|
-
// Store detected regressions
|
|
211
|
-
if (data.result.regressions.length > 0) {
|
|
212
|
-
await this.memoryStore.store(`aqe/visual/regressions/${data.result.buildId}`, data.result.regressions, {
|
|
213
|
-
partition: 'regressions',
|
|
214
|
-
ttl: 604800 // 7 days
|
|
215
|
-
});
|
|
216
|
-
}
|
|
217
|
-
|
|
218
|
-
// Store accessibility reports
|
|
219
|
-
await this.memoryStore.store(`aqe/visual/accessibility/${data.result.page}`, data.result.a11yReport, {
|
|
220
|
-
partition: 'accessibility',
|
|
221
|
-
ttl: 604800 // 7 days
|
|
222
|
-
});
|
|
223
|
-
|
|
224
|
-
// Store visual testing metrics
|
|
225
|
-
await this.memoryStore.store('aqe/visual/metrics', {
|
|
226
|
-
timestamp: Date.now(),
|
|
227
|
-
pagesTested: data.result.pagesTested,
|
|
228
|
-
regressionsFound: data.result.regressions.length,
|
|
229
|
-
a11yViolations: data.result.a11yReport?.violations?.length || 0
|
|
230
|
-
}, {
|
|
231
|
-
partition: 'metrics',
|
|
232
|
-
ttl: 604800 // 7 days
|
|
233
|
-
});
|
|
234
|
-
|
|
235
|
-
// Emit completion event with visual testing stats
|
|
236
|
-
this.eventBus.emit('visual-tester:completed', {
|
|
237
|
-
agentId: this.agentId,
|
|
238
|
-
pagesTested: data.result.pagesTested,
|
|
239
|
-
regressionsFound: data.result.regressions.length
|
|
240
|
-
});
|
|
241
|
-
|
|
242
|
-
// Validate visual testing results
|
|
243
|
-
const validation = await this.hookManager.executePostTaskValidation({
|
|
244
|
-
task: 'visual-testing',
|
|
245
|
-
result: {
|
|
246
|
-
output: data.result,
|
|
247
|
-
regressions: data.result.regressions,
|
|
248
|
-
metrics: {
|
|
249
|
-
pagesTested: data.result.pagesTested,
|
|
250
|
-
regressionsFound: data.result.regressions.length
|
|
251
|
-
}
|
|
252
|
-
}
|
|
253
|
-
});
|
|
254
|
-
|
|
255
|
-
this.logger.info('Visual testing completed', {
|
|
256
|
-
pagesTested: data.result.pagesTested,
|
|
257
|
-
regressionsFound: data.result.regressions.length,
|
|
258
|
-
validated: validation.passed
|
|
259
|
-
});
|
|
260
|
-
}
|
|
261
|
-
|
|
262
|
-
protected async onTaskError(data: { assignment: TaskAssignment; error: Error }): Promise<void> {
|
|
263
|
-
// Store error for fleet analysis
|
|
264
|
-
await this.memoryStore.store(`aqe/errors/${data.assignment.task.id}`, {
|
|
265
|
-
error: data.error.message,
|
|
266
|
-
timestamp: Date.now(),
|
|
267
|
-
agent: this.agentId,
|
|
268
|
-
taskType: 'visual-testing',
|
|
269
|
-
page: data.assignment.task.metadata.page
|
|
270
|
-
}, {
|
|
271
|
-
partition: 'errors',
|
|
272
|
-
ttl: 604800 // 7 days
|
|
273
|
-
});
|
|
274
|
-
|
|
275
|
-
// Emit error event for fleet coordination
|
|
276
|
-
this.eventBus.emit('visual-tester:error', {
|
|
277
|
-
agentId: this.agentId,
|
|
278
|
-
error: data.error.message,
|
|
279
|
-
taskId: data.assignment.task.id
|
|
280
|
-
});
|
|
281
|
-
|
|
282
|
-
this.logger.error('Visual testing failed', {
|
|
283
|
-
error: data.error.message,
|
|
284
|
-
stack: data.error.stack
|
|
285
|
-
});
|
|
286
|
-
}
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
**Event Bus Integration:**
|
|
290
|
-
```typescript
|
|
291
|
-
// Subscribe to visual testing events
|
|
292
|
-
this.registerEventHandler({
|
|
293
|
-
eventType: 'visual:regression-detected',
|
|
294
|
-
handler: async (event) => {
|
|
295
|
-
await this.qualityGate.blockDeployment(event.severity);
|
|
296
|
-
}
|
|
297
|
-
});
|
|
298
|
-
|
|
299
|
-
this.registerEventHandler({
|
|
300
|
-
eventType: 'visual:baseline-updated',
|
|
301
|
-
handler: async (event) => {
|
|
302
|
-
await this.notificationAgent.notifyTeam('New visual baseline created');
|
|
303
|
-
}
|
|
304
|
-
});
|
|
305
|
-
```
|
|
306
|
-
|
|
307
|
-
**Advanced Verification (Optional):**
|
|
308
|
-
```typescript
|
|
309
|
-
// Use VerificationHookManager for comprehensive validation
|
|
310
|
-
const hookManager = new VerificationHookManager(this.memoryStore);
|
|
311
|
-
const verification = await hookManager.executePreTaskVerification({
|
|
312
|
-
task: 'visual-regression-test',
|
|
313
|
-
context: {
|
|
314
|
-
requiredVars: ['BASELINE_VERSION', 'BROWSER'],
|
|
315
|
-
minMemoryMB: 2048,
|
|
316
|
-
requiredKeys: ['aqe/visual/baselines', 'aqe/visual/test-config']
|
|
317
|
-
}
|
|
318
|
-
});
|
|
319
|
-
```
|
|
320
|
-
|
|
321
|
-
## Learning Protocol (Phase 6 - Option C Implementation)
|
|
322
|
-
|
|
323
|
-
**⚠️ MANDATORY**: When executed via Claude Code Task tool, you MUST call learning MCP tools to persist learning data.
|
|
324
|
-
|
|
325
|
-
### Required Learning Actions (Call AFTER Task Completion)
|
|
326
|
-
|
|
327
|
-
**1. Store Learning Experience:**
|
|
328
|
-
```typescript
|
|
329
|
-
// Call this MCP tool after completing your task
|
|
330
85
|
mcp__agentic_qe__learning_store_experience({
|
|
331
86
|
agentId: "qe-visual-tester",
|
|
332
87
|
taskType: "visual-testing",
|
|
333
|
-
reward: 0.95,
|
|
88
|
+
reward: 0.95,
|
|
334
89
|
outcome: {
|
|
335
|
-
// Your actual results (agent-specific)
|
|
336
90
|
regressionsDetected: 3,
|
|
337
91
|
accuracy: 0.98,
|
|
338
92
|
falsePositives: 1,
|
|
339
93
|
executionTime: 8500
|
|
340
94
|
},
|
|
341
95
|
metadata: {
|
|
342
|
-
// Additional context (agent-specific)
|
|
343
96
|
algorithm: "ai-visual-diff",
|
|
344
97
|
threshold: 0.95,
|
|
345
98
|
accessibilityChecked: true
|
|
@@ -347,943 +100,86 @@ mcp__agentic_qe__learning_store_experience({
|
|
|
347
100
|
})
|
|
348
101
|
```
|
|
349
102
|
|
|
350
|
-
|
|
351
|
-
```
|
|
352
|
-
// Store Q-value for the strategy you used
|
|
353
|
-
mcp__agentic_qe__learning_store_qvalue({
|
|
354
|
-
agentId: "qe-visual-tester",
|
|
355
|
-
stateKey: "visual-testing-state",
|
|
356
|
-
actionKey: "ai-screenshot-comparison",
|
|
357
|
-
qValue: 0.85, // Expected value of this approach (based on results)
|
|
358
|
-
metadata: {
|
|
359
|
-
// Strategy details (agent-specific)
|
|
360
|
-
comparisonStrategy: "ai-visual-diff",
|
|
361
|
-
accuracy: 0.98,
|
|
362
|
-
sensitivity: 0.95
|
|
363
|
-
}
|
|
364
|
-
})
|
|
365
|
-
```
|
|
366
|
-
|
|
367
|
-
**3. Store Successful Patterns:**
|
|
368
|
-
```typescript
|
|
369
|
-
// If you discovered a useful pattern, store it
|
|
103
|
+
Store patterns when discovered:
|
|
104
|
+
```javascript
|
|
370
105
|
mcp__agentic_qe__learning_store_pattern({
|
|
371
|
-
agentId: "qe-visual-tester",
|
|
372
106
|
pattern: "AI-powered visual diff with 95% threshold detects regressions with <2% false positives",
|
|
373
|
-
confidence: 0.95,
|
|
107
|
+
confidence: 0.95,
|
|
374
108
|
domain: "visual-regression",
|
|
375
109
|
metadata: {
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
detectionAccuracy: 0.98
|
|
110
|
+
detectionAccuracy: 0.98,
|
|
111
|
+
falsePositiveRate: 0.02
|
|
379
112
|
}
|
|
380
113
|
})
|
|
381
114
|
```
|
|
382
115
|
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
-
|
|
424
|
-
-
|
|
425
|
-
-
|
|
426
|
-
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
await learningEngine.initialize();
|
|
455
|
-
|
|
456
|
-
// Record visual testing episode
|
|
457
|
-
await learningEngine.recordEpisode({
|
|
458
|
-
state: {
|
|
459
|
-
page: 'dashboard',
|
|
460
|
-
browser: 'chromium',
|
|
461
|
-
viewport: 'desktop',
|
|
462
|
-
baselineVersion: 'v2.0.0'
|
|
463
|
-
},
|
|
464
|
-
action: {
|
|
465
|
-
comparisonAlgorithm: 'ai-visual-diff',
|
|
466
|
-
threshold: 0.95,
|
|
467
|
-
ignoreRegions: ['timestamp', 'user-avatar']
|
|
468
|
-
},
|
|
469
|
-
reward: visualRegressionConfirmed ? 1.0 : (falsePositive ? -0.5 : 0.5),
|
|
470
|
-
nextState: {
|
|
471
|
-
regressionsDetected: 2,
|
|
472
|
-
falsePositives: 0,
|
|
473
|
-
missedRegressions: 0
|
|
474
|
-
}
|
|
475
|
-
});
|
|
476
|
-
|
|
477
|
-
// Learn from visual testing outcomes
|
|
478
|
-
await learningEngine.learn();
|
|
479
|
-
|
|
480
|
-
// Get learned visual comparison settings
|
|
481
|
-
const prediction = await learningEngine.predict({
|
|
482
|
-
page: 'dashboard',
|
|
483
|
-
browser: 'chromium',
|
|
484
|
-
viewport: 'desktop'
|
|
485
|
-
});
|
|
486
|
-
```
|
|
487
|
-
|
|
488
|
-
### Reward Function
|
|
489
|
-
|
|
490
|
-
```typescript
|
|
491
|
-
function calculateVisualTestingReward(outcome: VisualTestOutcome): number {
|
|
492
|
-
let reward = 0;
|
|
493
|
-
|
|
494
|
-
// Reward for detecting actual regressions
|
|
495
|
-
reward += outcome.truePositives * 1.0;
|
|
496
|
-
|
|
497
|
-
// Penalty for false positives (wasted effort)
|
|
498
|
-
reward -= outcome.falsePositives * 0.5;
|
|
499
|
-
|
|
500
|
-
// Large penalty for missing regressions (false negatives)
|
|
501
|
-
reward -= outcome.falseNegatives * 2.0;
|
|
502
|
-
|
|
503
|
-
// Reward for correct negative (no regression correctly identified)
|
|
504
|
-
reward += outcome.trueNegatives * 0.2;
|
|
505
|
-
|
|
506
|
-
// Bonus for high accuracy
|
|
507
|
-
const accuracy = (outcome.truePositives + outcome.trueNegatives) /
|
|
508
|
-
(outcome.truePositives + outcome.trueNegatives +
|
|
509
|
-
outcome.falsePositives + outcome.falseNegatives);
|
|
510
|
-
if (accuracy > 0.95) {
|
|
511
|
-
reward += 0.5;
|
|
512
|
-
}
|
|
513
|
-
|
|
514
|
-
return reward;
|
|
515
|
-
}
|
|
516
|
-
```
|
|
517
|
-
|
|
518
|
-
### Learning Metrics
|
|
519
|
-
|
|
520
|
-
Track learning progress:
|
|
521
|
-
- **Detection Accuracy**: Percentage of correctly identified regressions
|
|
522
|
-
- **False Positive Rate**: Incorrectly flagged differences
|
|
523
|
-
- **False Negative Rate**: Missed visual regressions
|
|
524
|
-
- **Algorithm Selection**: Optimal comparison algorithm for each page type
|
|
525
|
-
- **Threshold Optimization**: Learned thresholds per page/browser combination
|
|
526
|
-
|
|
527
|
-
```bash
|
|
528
|
-
# View learning metrics
|
|
529
|
-
aqe learn status --agent qe-visual-tester
|
|
530
|
-
|
|
531
|
-
# Export learning history
|
|
532
|
-
aqe learn export --agent qe-visual-tester --format json
|
|
533
|
-
|
|
534
|
-
# Analyze detection accuracy
|
|
535
|
-
aqe learn analyze --agent qe-visual-tester --metric accuracy
|
|
536
|
-
```
|
|
537
|
-
|
|
538
|
-
### Agent Collaboration
|
|
539
|
-
- **QE Test Executor**: Integrates visual tests into test suites
|
|
540
|
-
- **QE Quality Gate**: Blocks deployments on visual regressions
|
|
541
|
-
- **QE Test Generator**: Generates visual test cases automatically
|
|
542
|
-
- **QE Performance Tester**: Correlates visual issues with performance
|
|
543
|
-
- **Fleet Commander**: Reports visual testing resource usage
|
|
544
|
-
|
|
545
|
-
## Memory Keys
|
|
546
|
-
|
|
547
|
-
### Input Keys
|
|
548
|
-
- `aqe/visual/baselines`: Baseline screenshot repository
|
|
549
|
-
- `aqe/visual/test-config`: Visual testing configuration
|
|
550
|
-
- `aqe/visual/comparison-thresholds`: Acceptable difference thresholds
|
|
551
|
-
- `aqe/visual/ignore-regions`: UI regions to ignore in comparisons
|
|
552
|
-
- `aqe/visual/test-targets`: Pages and components to test
|
|
553
|
-
|
|
554
|
-
### Output Keys
|
|
555
|
-
- `aqe/visual/test-results`: Visual test execution results
|
|
556
|
-
- `aqe/visual/regressions`: Detected visual regressions
|
|
557
|
-
- `aqe/visual/diff-images`: Generated diff images
|
|
558
|
-
- `aqe/visual/accessibility-reports`: WCAG compliance reports
|
|
559
|
-
- `aqe/visual/cross-browser-matrix`: Cross-browser test results
|
|
560
|
-
- `aqe/visual/performance-metrics`: Visual rendering metrics
|
|
561
|
-
|
|
562
|
-
### Coordination Keys
|
|
563
|
-
- `aqe/visual/status`: Current visual testing status
|
|
564
|
-
- `aqe/visual/test-queue`: Queued visual test jobs
|
|
565
|
-
- `aqe/visual/baseline-updates`: Pending baseline updates
|
|
566
|
-
- `aqe/visual/alerts`: Visual testing alerts and warnings
|
|
567
|
-
|
|
568
|
-
## Coordination Protocol
|
|
569
|
-
|
|
570
|
-
### Swarm Integration
|
|
571
|
-
|
|
572
|
-
All swarm coordination is handled via **AQE hooks (Agentic QE native hooks)** and the EventBus. Use Claude Code's Task tool to spawn agents and orchestrate workflows - the native hooks handle all coordination automatically without external MCP commands.
|
|
573
|
-
|
|
574
|
-
## Visual Comparison Algorithms
|
|
575
|
-
|
|
576
|
-
### Pixel-by-Pixel Comparison
|
|
577
|
-
```javascript
|
|
578
|
-
// Traditional pixel difference detection
|
|
579
|
-
const pixelDiff = {
|
|
580
|
-
algorithm: 'pixelmatch',
|
|
581
|
-
options: {
|
|
582
|
-
threshold: 0.1, // Pixel difference threshold
|
|
583
|
-
includeAA: false, // Ignore anti-aliasing
|
|
584
|
-
alpha: 0.1, // Opacity of diff overlay
|
|
585
|
-
diffColor: [255, 0, 0], // Red diff color
|
|
586
|
-
diffMask: false
|
|
587
|
-
}
|
|
588
|
-
};
|
|
589
|
-
|
|
590
|
-
// Execute pixel comparison
|
|
591
|
-
const pixelDiffResult = await compareScreenshots({
|
|
592
|
-
baseline: baselineImage,
|
|
593
|
-
current: currentImage,
|
|
594
|
-
algorithm: pixelDiff
|
|
595
|
-
});
|
|
596
|
-
```
|
|
597
|
-
|
|
598
|
-
### Structural Similarity (SSIM)
|
|
599
|
-
```javascript
|
|
600
|
-
// Perceptual similarity comparison
|
|
601
|
-
const structuralComparison = {
|
|
602
|
-
algorithm: 'ssim',
|
|
603
|
-
options: {
|
|
604
|
-
window_size: 11,
|
|
605
|
-
k1: 0.01,
|
|
606
|
-
k2: 0.03,
|
|
607
|
-
luminance_weight: 1.0,
|
|
608
|
-
contrast_weight: 1.0,
|
|
609
|
-
structure_weight: 1.0
|
|
610
|
-
}
|
|
611
|
-
};
|
|
612
|
-
|
|
613
|
-
// Execute structural comparison
|
|
614
|
-
const ssimResult = await compareScreenshots({
|
|
615
|
-
baseline: baselineImage,
|
|
616
|
-
current: currentImage,
|
|
617
|
-
algorithm: structuralComparison
|
|
618
|
-
});
|
|
619
|
-
```
|
|
620
|
-
|
|
621
|
-
### AI-Powered Visual Diff
|
|
622
|
-
```javascript
|
|
623
|
-
// Semantic visual understanding
|
|
624
|
-
const aiVisualDiff = {
|
|
625
|
-
algorithm: 'ai-visual-diff',
|
|
626
|
-
options: {
|
|
627
|
-
use_neural_network: true,
|
|
628
|
-
model: 'visual-regression-v2',
|
|
629
|
-
semantic_understanding: true,
|
|
630
|
-
context_awareness: true,
|
|
631
|
-
ignore_minor_variations: true,
|
|
632
|
-
classification: [
|
|
633
|
-
'intentional-change',
|
|
634
|
-
'unintentional-regression',
|
|
635
|
-
'acceptable-variation',
|
|
636
|
-
'critical-breakage'
|
|
637
|
-
]
|
|
638
|
-
}
|
|
639
|
-
};
|
|
640
|
-
|
|
641
|
-
// Execute AI comparison
|
|
642
|
-
const aiDiffResult = await compareScreenshots({
|
|
643
|
-
baseline: baselineImage,
|
|
644
|
-
current: currentImage,
|
|
645
|
-
algorithm: aiVisualDiff
|
|
646
|
-
});
|
|
647
|
-
```
|
|
648
|
-
|
|
649
|
-
## Cross-Browser Testing
|
|
650
|
-
|
|
651
|
-
### Browser Matrix
|
|
652
|
-
```javascript
|
|
653
|
-
// Define cross-browser test matrix
|
|
654
|
-
const browserMatrix = {
|
|
655
|
-
browsers: [
|
|
656
|
-
{ name: 'chromium', version: 'latest' },
|
|
657
|
-
{ name: 'chromium', version: 'latest-1' },
|
|
658
|
-
{ name: 'firefox', version: 'latest' },
|
|
659
|
-
{ name: 'webkit', version: 'latest' },
|
|
660
|
-
{ name: 'edge', version: 'latest' }
|
|
661
|
-
],
|
|
662
|
-
viewports: [
|
|
663
|
-
{ name: 'desktop', width: 1920, height: 1080 },
|
|
664
|
-
{ name: 'laptop', width: 1366, height: 768 },
|
|
665
|
-
{ name: 'tablet', width: 768, height: 1024 },
|
|
666
|
-
{ name: 'mobile', width: 375, height: 667 }
|
|
667
|
-
],
|
|
668
|
-
pages: baselineConfig.pages,
|
|
669
|
-
parallel: true,
|
|
670
|
-
max_concurrent: 10
|
|
671
|
-
};
|
|
672
|
-
|
|
673
|
-
// Execute cross-browser tests
|
|
674
|
-
const crossBrowserResults = await executeCrossBrowserTests(browserMatrix);
|
|
675
|
-
```
|
|
676
|
-
|
|
677
|
-
### Browser-Specific Handling
|
|
678
|
-
```javascript
|
|
679
|
-
// Handle browser-specific differences
|
|
680
|
-
const browserSpecificConfig = {
|
|
681
|
-
chromium: {
|
|
682
|
-
ignore_font_smoothing: true,
|
|
683
|
-
ignore_scrollbar: true
|
|
684
|
-
},
|
|
685
|
-
firefox: {
|
|
686
|
-
ignore_svg_rendering: true,
|
|
687
|
-
ignore_css_filters: true
|
|
688
|
-
},
|
|
689
|
-
webkit: {
|
|
690
|
-
ignore_shadow_dom_styles: true,
|
|
691
|
-
ignore_webkit_appearance: true
|
|
692
|
-
}
|
|
693
|
-
};
|
|
694
|
-
```
|
|
695
|
-
|
|
696
|
-
## Accessibility Validation
|
|
697
|
-
|
|
698
|
-
### WCAG 2.1 AA Compliance
|
|
699
|
-
```javascript
|
|
700
|
-
// Comprehensive accessibility testing
|
|
701
|
-
const wcagValidation = {
|
|
702
|
-
standard: 'WCAG-2.1-AA',
|
|
703
|
-
rules: {
|
|
704
|
-
perceivable: [
|
|
705
|
-
'color-contrast',
|
|
706
|
-
'text-alternatives',
|
|
707
|
-
'adaptable-content',
|
|
708
|
-
'distinguishable-content'
|
|
709
|
-
],
|
|
710
|
-
operable: [
|
|
711
|
-
'keyboard-accessible',
|
|
712
|
-
'enough-time',
|
|
713
|
-
'seizure-safe',
|
|
714
|
-
'navigable'
|
|
715
|
-
],
|
|
716
|
-
understandable: [
|
|
717
|
-
'readable',
|
|
718
|
-
'predictable',
|
|
719
|
-
'input-assistance'
|
|
720
|
-
],
|
|
721
|
-
robust: [
|
|
722
|
-
'compatible',
|
|
723
|
-
'parsing-valid'
|
|
724
|
-
]
|
|
725
|
-
}
|
|
726
|
-
};
|
|
727
|
-
|
|
728
|
-
// Execute WCAG validation
|
|
729
|
-
const wcagResults = await validateWCAG({
|
|
730
|
-
pages: baselineConfig.pages,
|
|
731
|
-
standard: wcagValidation.standard,
|
|
732
|
-
rules: wcagValidation.rules
|
|
733
|
-
});
|
|
734
|
-
```
|
|
735
|
-
|
|
736
|
-
### Color Contrast Analysis
|
|
737
|
-
```javascript
|
|
738
|
-
// Validate color contrast ratios
|
|
739
|
-
const colorContrastValidation = {
|
|
740
|
-
minimum_ratio_normal: 4.5, // WCAG AA normal text
|
|
741
|
-
minimum_ratio_large: 3.0, // WCAG AA large text
|
|
742
|
-
minimum_ratio_aaa: 7.0, // WCAG AAA
|
|
743
|
-
analyze_all_elements: true,
|
|
744
|
-
generate_suggestions: true
|
|
745
|
-
};
|
|
746
|
-
|
|
747
|
-
// Execute contrast analysis
|
|
748
|
-
const contrastResults = await analyzeColorContrast({
|
|
749
|
-
pages: baselineConfig.pages,
|
|
750
|
-
validation: colorContrastValidation
|
|
751
|
-
});
|
|
752
|
-
```
|
|
753
|
-
|
|
754
|
-
### Keyboard Navigation Testing
|
|
755
|
-
```javascript
|
|
756
|
-
// Test keyboard accessibility
|
|
757
|
-
const keyboardTests = {
|
|
758
|
-
test_tab_order: true,
|
|
759
|
-
test_focus_indicators: true,
|
|
760
|
-
test_skip_links: true,
|
|
761
|
-
test_keyboard_traps: true,
|
|
762
|
-
test_shortcut_conflicts: false,
|
|
763
|
-
record_navigation_path: true
|
|
764
|
-
};
|
|
765
|
-
|
|
766
|
-
// Execute keyboard navigation tests
|
|
767
|
-
const keyboardResults = await testKeyboardNavigation({
|
|
768
|
-
pages: baselineConfig.pages,
|
|
769
|
-
tests: keyboardTests
|
|
770
|
-
});
|
|
771
|
-
```
|
|
772
|
-
|
|
773
|
-
## Responsive Design Testing
|
|
774
|
-
|
|
775
|
-
### Viewport Testing
|
|
776
|
-
```javascript
|
|
777
|
-
// Test responsive breakpoints
|
|
778
|
-
const responsiveTests = {
|
|
779
|
-
breakpoints: [
|
|
780
|
-
{ name: 'mobile-small', width: 320, height: 568 },
|
|
781
|
-
{ name: 'mobile', width: 375, height: 667 },
|
|
782
|
-
{ name: 'mobile-large', width: 414, height: 896 },
|
|
783
|
-
{ name: 'tablet', width: 768, height: 1024 },
|
|
784
|
-
{ name: 'desktop', width: 1366, height: 768 },
|
|
785
|
-
{ name: 'desktop-large', width: 1920, height: 1080 },
|
|
786
|
-
{ name: '4k', width: 3840, height: 2160 }
|
|
787
|
-
],
|
|
788
|
-
validations: [
|
|
789
|
-
'layout-integrity',
|
|
790
|
-
'text-readability',
|
|
791
|
-
'image-scaling',
|
|
792
|
-
'navigation-usability',
|
|
793
|
-
'content-visibility'
|
|
794
|
-
]
|
|
795
|
-
};
|
|
796
|
-
|
|
797
|
-
// Execute responsive tests
|
|
798
|
-
const responsiveResults = await testResponsiveDesign(responsiveTests);
|
|
799
|
-
```
|
|
800
|
-
|
|
801
|
-
### Orientation Testing
|
|
802
|
-
```javascript
|
|
803
|
-
// Test portrait and landscape orientations
|
|
804
|
-
const orientationTests = {
|
|
805
|
-
devices: ['mobile', 'tablet'],
|
|
806
|
-
orientations: ['portrait', 'landscape'],
|
|
807
|
-
validate_layout_shift: true,
|
|
808
|
-
validate_content_reflow: true
|
|
809
|
-
};
|
|
810
|
-
|
|
811
|
-
// Execute orientation tests
|
|
812
|
-
const orientationResults = await testOrientations(orientationTests);
|
|
813
|
-
```
|
|
814
|
-
|
|
815
|
-
## Performance Metrics
|
|
816
|
-
|
|
817
|
-
### Visual Rendering Performance
|
|
818
|
-
```javascript
|
|
819
|
-
// Measure visual performance metrics
|
|
820
|
-
const performanceMetrics = {
|
|
821
|
-
metrics: [
|
|
822
|
-
'first-contentful-paint',
|
|
823
|
-
'largest-contentful-paint',
|
|
824
|
-
'cumulative-layout-shift',
|
|
825
|
-
'speed-index',
|
|
826
|
-
'time-to-interactive',
|
|
827
|
-
'total-blocking-time'
|
|
828
|
-
],
|
|
829
|
-
thresholds: {
|
|
830
|
-
fcp: 1800, // ms
|
|
831
|
-
lcp: 2500,
|
|
832
|
-
cls: 0.1,
|
|
833
|
-
si: 3000,
|
|
834
|
-
tti: 3800,
|
|
835
|
-
tbt: 200
|
|
836
|
-
}
|
|
837
|
-
};
|
|
838
|
-
|
|
839
|
-
// Measure performance
|
|
840
|
-
const perfResults = await measureVisualPerformance({
|
|
841
|
-
pages: baselineConfig.pages,
|
|
842
|
-
metrics: performanceMetrics
|
|
843
|
-
});
|
|
844
|
-
```
|
|
845
|
-
|
|
846
|
-
### Layout Shift Detection
|
|
847
|
-
```javascript
|
|
848
|
-
// Detect cumulative layout shifts
|
|
849
|
-
const layoutShiftDetection = {
|
|
850
|
-
monitor_duration: 5000, // ms
|
|
851
|
-
threshold: 0.1, // CLS threshold
|
|
852
|
-
track_elements: true,
|
|
853
|
-
identify_causes: true
|
|
854
|
-
};
|
|
855
|
-
|
|
856
|
-
// Execute layout shift detection
|
|
857
|
-
const layoutShiftResults = await detectLayoutShifts({
|
|
858
|
-
pages: baselineConfig.pages,
|
|
859
|
-
detection: layoutShiftDetection
|
|
860
|
-
});
|
|
861
|
-
```
|
|
862
|
-
|
|
863
|
-
## Example Outputs
|
|
864
|
-
|
|
865
|
-
### Visual Regression Report
|
|
866
|
-
```json
|
|
867
|
-
{
|
|
868
|
-
"test_run_id": "vt-2025-09-30-001",
|
|
869
|
-
"status": "completed",
|
|
870
|
-
"execution_time": "3m 42s",
|
|
871
|
-
"summary": {
|
|
872
|
-
"total_pages": 15,
|
|
873
|
-
"total_screenshots": 45,
|
|
874
|
-
"browsers_tested": 3,
|
|
875
|
-
"viewports_tested": 3,
|
|
876
|
-
"regressions_found": 2,
|
|
877
|
-
"accessibility_violations": 5
|
|
878
|
-
},
|
|
879
|
-
"regressions": [
|
|
880
|
-
{
|
|
881
|
-
"page": "dashboard",
|
|
882
|
-
"browser": "chromium",
|
|
883
|
-
"viewport": "desktop",
|
|
884
|
-
"severity": "high",
|
|
885
|
-
"type": "layout-shift",
|
|
886
|
-
"description": "Navigation menu shifted 15px right",
|
|
887
|
-
"affected_area": { "x": 0, "y": 0, "width": 250, "height": 1080 },
|
|
888
|
-
"pixel_diff_percentage": 3.2,
|
|
889
|
-
"baseline_image": "baseline-dashboard-chromium-desktop.png",
|
|
890
|
-
"current_image": "current-dashboard-chromium-desktop.png",
|
|
891
|
-
"diff_image": "diff-dashboard-chromium-desktop.png",
|
|
892
|
-
"suggested_fix": "Check CSS grid template columns in navigation.css"
|
|
893
|
-
},
|
|
894
|
-
{
|
|
895
|
-
"page": "user-profile",
|
|
896
|
-
"browser": "firefox",
|
|
897
|
-
"viewport": "mobile",
|
|
898
|
-
"severity": "medium",
|
|
899
|
-
"type": "color-change",
|
|
900
|
-
"description": "Button color changed from #007bff to #0056b3",
|
|
901
|
-
"affected_area": { "x": 150, "y": 400, "width": 100, "height": 40 },
|
|
902
|
-
"pixel_diff_percentage": 0.8,
|
|
903
|
-
"baseline_image": "baseline-profile-firefox-mobile.png",
|
|
904
|
-
"current_image": "current-profile-firefox-mobile.png",
|
|
905
|
-
"diff_image": "diff-profile-firefox-mobile.png",
|
|
906
|
-
"suggested_fix": "Verify CSS variable --primary-color value"
|
|
907
|
-
}
|
|
908
|
-
],
|
|
909
|
-
"accessibility": {
|
|
910
|
-
"standard": "WCAG-2.1-AA",
|
|
911
|
-
"compliance_score": 91,
|
|
912
|
-
"violations": [
|
|
913
|
-
{
|
|
914
|
-
"rule": "color-contrast",
|
|
915
|
-
"severity": "serious",
|
|
916
|
-
"page": "dashboard",
|
|
917
|
-
"element": "button.secondary",
|
|
918
|
-
"description": "Contrast ratio 3.2:1 insufficient (minimum 4.5:1)",
|
|
919
|
-
"location": ".dashboard-actions > button:nth-child(2)",
|
|
920
|
-
"suggested_fix": "Darken button text or lighten background"
|
|
921
|
-
}
|
|
922
|
-
]
|
|
923
|
-
},
|
|
924
|
-
"cross_browser_consistency": {
|
|
925
|
-
"chromium": "98% consistent",
|
|
926
|
-
"firefox": "95% consistent",
|
|
927
|
-
"webkit": "97% consistent"
|
|
928
|
-
}
|
|
929
|
-
}
|
|
930
|
-
```
|
|
931
|
-
|
|
932
|
-
### Accessibility Report
|
|
933
|
-
```json
|
|
934
|
-
{
|
|
935
|
-
"page": "dashboard",
|
|
936
|
-
"standard": "WCAG-2.1-AA",
|
|
937
|
-
"compliance_score": 91,
|
|
938
|
-
"test_date": "2025-09-30T10:00:00Z",
|
|
939
|
-
"violations": [
|
|
940
|
-
{
|
|
941
|
-
"rule": "button-name",
|
|
942
|
-
"severity": "critical",
|
|
943
|
-
"wcag_criterion": "4.1.2",
|
|
944
|
-
"element": "<button class=\"icon-btn\"></button>",
|
|
945
|
-
"location": ".toolbar > button:nth-child(3)",
|
|
946
|
-
"description": "Button has no accessible name",
|
|
947
|
-
"impact": "Buttons without names are unusable by screen readers",
|
|
948
|
-
"suggested_fix": "Add aria-label or visible text content",
|
|
949
|
-
"code_suggestion": "<button class=\"icon-btn\" aria-label=\"Save changes\"></button>"
|
|
950
|
-
}
|
|
951
|
-
],
|
|
952
|
-
"passes": [
|
|
953
|
-
"html-has-lang",
|
|
954
|
-
"document-title",
|
|
955
|
-
"landmark-one-main",
|
|
956
|
-
"page-has-heading-one"
|
|
957
|
-
],
|
|
958
|
-
"warnings": [
|
|
959
|
-
{
|
|
960
|
-
"rule": "color-contrast-enhanced",
|
|
961
|
-
"description": "Some text does not meet WCAG AAA contrast ratio"
|
|
962
|
-
}
|
|
963
|
-
]
|
|
964
|
-
}
|
|
965
|
-
```
|
|
966
|
-
|
|
967
|
-
## Commands
|
|
968
|
-
|
|
969
|
-
### Basic Operations
|
|
970
|
-
```bash
|
|
971
|
-
# Initialize visual tester
|
|
972
|
-
agentic-qe agent spawn --name qe-visual-tester --type visual-tester
|
|
973
|
-
|
|
974
|
-
# Capture baselines
|
|
975
|
-
agentic-qe visual baseline --pages all --browsers chromium,firefox,webkit
|
|
976
|
-
|
|
977
|
-
# Run visual regression tests
|
|
978
|
-
agentic-qe visual test --compare-baseline v2.0.0
|
|
979
|
-
|
|
980
|
-
# Check test status
|
|
981
|
-
agentic-qe visual status --test-run-id vt-123
|
|
982
|
-
```
|
|
983
|
-
|
|
984
|
-
### Advanced Operations
|
|
985
|
-
```bash
|
|
986
|
-
# Cross-browser visual testing
|
|
987
|
-
agentic-qe visual cross-browser \
|
|
988
|
-
--browsers "chromium,firefox,webkit" \
|
|
989
|
-
--viewports "desktop,tablet,mobile"
|
|
990
|
-
|
|
991
|
-
# Accessibility validation
|
|
992
|
-
agentic-qe visual accessibility \
|
|
993
|
-
--standard WCAG-2.1-AA \
|
|
994
|
-
--pages all
|
|
995
|
-
|
|
996
|
-
# Responsive design testing
|
|
997
|
-
agentic-qe visual responsive \
|
|
998
|
-
--breakpoints "320,768,1366,1920"
|
|
999
|
-
|
|
1000
|
-
# Update baselines
|
|
1001
|
-
agentic-qe visual update-baseline \
|
|
1002
|
-
--page dashboard \
|
|
1003
|
-
--version v2.1.0
|
|
1004
|
-
```
|
|
1005
|
-
|
|
1006
|
-
### Analysis Operations
|
|
1007
|
-
```bash
|
|
1008
|
-
# Analyze regressions
|
|
1009
|
-
agentic-qe visual analyze-regressions \
|
|
1010
|
-
--severity high \
|
|
1011
|
-
--ai-classification
|
|
1012
|
-
|
|
1013
|
-
# Generate diff report
|
|
1014
|
-
agentic-qe visual diff-report \
|
|
1015
|
-
--test-run-id vt-123 \
|
|
1016
|
-
--format html
|
|
1017
|
-
|
|
1018
|
-
# Compare versions
|
|
1019
|
-
agentic-qe visual compare-versions \
|
|
1020
|
-
--baseline v2.0.0 \
|
|
1021
|
-
--target v2.1.0
|
|
1022
|
-
```
|
|
1023
|
-
|
|
1024
|
-
## Quality Metrics
|
|
1025
|
-
|
|
1026
|
-
- **Regression Detection**: >99% accuracy for visual regressions
|
|
1027
|
-
- **False Positive Rate**: <2% false positives with AI-powered diff
|
|
1028
|
-
- **Accessibility Coverage**: 100% WCAG 2.1 AA rule validation
|
|
1029
|
-
- **Cross-Browser Coverage**: 5+ browsers, 7+ viewport sizes
|
|
1030
|
-
- **Performance**: <30 seconds per page cross-browser test
|
|
1031
|
-
- **Baseline Storage**: Lossless compression, <5MB per page
|
|
1032
|
-
- **Test Execution**: Parallel execution across 10 browsers
|
|
1033
|
-
|
|
1034
|
-
## Integration with QE Fleet
|
|
1035
|
-
|
|
1036
|
-
This agent integrates with the Agentic QE Fleet through:
|
|
1037
|
-
- **EventBus**: Real-time visual regression alerts
|
|
1038
|
-
- **MemoryManager**: Baseline and regression data storage
|
|
1039
|
-
- **FleetManager**: Coordinated visual testing workflows
|
|
1040
|
-
- **Neural Network**: AI-powered visual diff and regression classification
|
|
1041
|
-
- **Quality Gate**: Automated deployment blocking on visual regressions
|
|
1042
|
-
|
|
1043
|
-
## Advanced Features
|
|
1044
|
-
|
|
1045
|
-
### AI-Powered Visual Understanding
|
|
1046
|
-
Uses neural networks to understand UI context and semantics, not just pixel differences
|
|
1047
|
-
|
|
1048
|
-
### Smart Baseline Management
|
|
1049
|
-
Automatically suggests baseline updates when intentional design changes are detected
|
|
1050
|
-
|
|
1051
|
-
### Visual Test Generation
|
|
1052
|
-
Generates visual test cases automatically from UI component libraries
|
|
1053
|
-
|
|
1054
|
-
### Continuous Visual Monitoring
|
|
1055
|
-
Monitors production UI for visual degradation in real-time
|
|
1056
|
-
|
|
1057
|
-
## Code Execution Workflows
|
|
1058
|
-
|
|
1059
|
-
Write code to orchestrate visual-tester workflows programmatically using Phase 3 domain-specific tools.
|
|
1060
|
-
|
|
1061
|
-
### AI-Powered Screenshot Comparison
|
|
1062
|
-
|
|
1063
|
-
```typescript
|
|
1064
|
-
/**
|
|
1065
|
-
* Phase 3 Visual Testing Tools
|
|
1066
|
-
* Import path: 'agentic-qe/tools/qe/visual'
|
|
1067
|
-
* Type definitions: 'agentic-qe/tools/qe/shared/types'
|
|
1068
|
-
*/
|
|
1069
|
-
|
|
1070
|
-
import type {
|
|
1071
|
-
CompareScreenshotsParams,
|
|
1072
|
-
ScreenshotComparison,
|
|
1073
|
-
VisualDifference
|
|
1074
|
-
} from 'agentic-qe/tools/qe/visual';
|
|
1075
|
-
|
|
1076
|
-
// Import Phase 3 visual tools
|
|
1077
|
-
import {
|
|
1078
|
-
compareScreenshotsAI,
|
|
1079
|
-
validateAccessibilityWCAG,
|
|
1080
|
-
type WCAGLevel
|
|
1081
|
-
} from 'agentic-qe/tools/qe/visual';
|
|
1082
|
-
|
|
1083
|
-
// Example: AI-powered screenshot comparison
|
|
1084
|
-
const compareParams: CompareScreenshotsParams = {
|
|
1085
|
-
baselineImage: './screenshots/baseline/dashboard.png',
|
|
1086
|
-
currentImage: './screenshots/current/dashboard.png',
|
|
1087
|
-
algorithm: 'perceptual-hash', // AI-powered visual diff
|
|
1088
|
-
threshold: 0.95, // 95% similarity threshold
|
|
1089
|
-
ignoreRegions: [
|
|
1090
|
-
{ x: 0, y: 0, width: 100, height: 50 } // Ignore timestamp
|
|
1091
|
-
]
|
|
1092
|
-
};
|
|
1093
|
-
|
|
1094
|
-
const comparison: ScreenshotComparison = await compareScreenshotsAI(compareParams);
|
|
1095
|
-
|
|
1096
|
-
if (comparison.hasDifferences) {
|
|
1097
|
-
console.log('⚠️ Visual regressions detected:');
|
|
1098
|
-
comparison.differences.forEach((diff: VisualDifference) => {
|
|
1099
|
-
console.log(` Region: (${diff.x}, ${diff.y}) ${diff.width}x${diff.height}`);
|
|
1100
|
-
console.log(` Severity: ${diff.severity}`);
|
|
1101
|
-
console.log(` Pixel diff: ${diff.pixelDiffPercentage.toFixed(2)}%`);
|
|
1102
|
-
console.log(` AI analysis: ${diff.aiDescription}`);
|
|
1103
|
-
});
|
|
1104
|
-
} else {
|
|
1105
|
-
console.log('✅ No visual regressions detected');
|
|
1106
|
-
}
|
|
1107
|
-
|
|
1108
|
-
console.log(`Similarity score: ${comparison.similarityScore.toFixed(4)}`);
|
|
1109
|
-
```
|
|
1110
|
-
|
|
1111
|
-
### WCAG Accessibility Validation
|
|
1112
|
-
|
|
1113
|
-
```typescript
|
|
1114
|
-
import type {
|
|
1115
|
-
ValidateAccessibilityParams,
|
|
1116
|
-
AccessibilityReport,
|
|
1117
|
-
AccessibilityViolation
|
|
1118
|
-
} from 'agentic-qe/tools/qe/visual';
|
|
1119
|
-
|
|
1120
|
-
import {
|
|
1121
|
-
validateAccessibilityWCAG
|
|
1122
|
-
} from 'agentic-qe/tools/qe/visual';
|
|
1123
|
-
|
|
1124
|
-
// Example: Validate WCAG 2.1 AA compliance
|
|
1125
|
-
const accessibilityParams: ValidateAccessibilityParams = {
|
|
1126
|
-
url: 'https://example.com/dashboard',
|
|
1127
|
-
standard: 'WCAG-2.1',
|
|
1128
|
-
level: 'AA' as WCAGLevel,
|
|
1129
|
-
rules: ['color-contrast', 'button-name', 'link-name', 'image-alt'],
|
|
1130
|
-
includeBestPractices: true,
|
|
1131
|
-
screenshot: true
|
|
1132
|
-
};
|
|
1133
|
-
|
|
1134
|
-
const accessibilityReport: AccessibilityReport = await validateAccessibilityWCAG(accessibilityParams);
|
|
1135
|
-
|
|
1136
|
-
console.log(`Accessibility Score: ${accessibilityReport.score}/100`);
|
|
1137
|
-
console.log(`Violations: ${accessibilityReport.violationsCount}`);
|
|
1138
|
-
console.log(`Compliance: ${accessibilityReport.compliance ? '✅ PASS' : '❌ FAIL'}`);
|
|
1139
|
-
|
|
1140
|
-
// Show critical violations
|
|
1141
|
-
accessibilityReport.violations
|
|
1142
|
-
.filter((v: AccessibilityViolation) => v.severity === 'critical')
|
|
1143
|
-
.forEach((violation: AccessibilityViolation) => {
|
|
1144
|
-
console.log(`\n🚨 ${violation.rule}:`);
|
|
1145
|
-
console.log(` Element: ${violation.element}`);
|
|
1146
|
-
console.log(` Issue: ${violation.description}`);
|
|
1147
|
-
console.log(` Fix: ${violation.suggestedFix}`);
|
|
1148
|
-
});
|
|
1149
|
-
|
|
1150
|
-
// Color contrast results
|
|
1151
|
-
if (accessibilityReport.colorContrast) {
|
|
1152
|
-
console.log(`\nColor Contrast: ${accessibilityReport.colorContrast.failedElements} failed elements`);
|
|
1153
|
-
}
|
|
1154
|
-
|
|
1155
|
-
// Keyboard navigation results
|
|
1156
|
-
if (accessibilityReport.keyboardNavigation) {
|
|
1157
|
-
console.log(`Keyboard Navigation: ${accessibilityReport.keyboardNavigation.accessible ? '✅' : '❌'}`);
|
|
1158
|
-
}
|
|
1159
|
-
|
|
1160
|
-
// Screen reader results
|
|
1161
|
-
if (accessibilityReport.screenReader) {
|
|
1162
|
-
console.log(`Screen Reader: ${accessibilityReport.screenReader.score}/100`);
|
|
1163
|
-
}
|
|
1164
|
-
```
|
|
1165
|
-
|
|
1166
|
-
### Cross-Browser Visual Regression Testing
|
|
1167
|
-
|
|
1168
|
-
```typescript
|
|
1169
|
-
import type {
|
|
1170
|
-
CompareScreenshotsParams
|
|
1171
|
-
} from 'agentic-qe/tools/qe/visual';
|
|
1172
|
-
|
|
1173
|
-
import {
|
|
1174
|
-
compareScreenshotsAI
|
|
1175
|
-
} from 'agentic-qe/tools/qe/visual';
|
|
1176
|
-
|
|
1177
|
-
// Example: Test across multiple browsers
|
|
1178
|
-
async function testCrossBrowserVisuals(page: string, browsers: string[]): Promise<void> {
|
|
1179
|
-
console.log(`Testing ${page} across ${browsers.length} browsers...\n`);
|
|
1180
|
-
|
|
1181
|
-
const results: Array<{ browser: string; hasRegressions: boolean; score: number }> = [];
|
|
1182
|
-
|
|
1183
|
-
for (const browser of browsers) {
|
|
1184
|
-
const params: CompareScreenshotsParams = {
|
|
1185
|
-
baselineImage: `./screenshots/baseline/${page}-${browser}.png`,
|
|
1186
|
-
currentImage: `./screenshots/current/${page}-${browser}.png`,
|
|
1187
|
-
algorithm: 'perceptual-hash',
|
|
1188
|
-
threshold: 0.98 // 98% similarity for cross-browser
|
|
1189
|
-
};
|
|
1190
|
-
|
|
1191
|
-
const comparison = await compareScreenshotsAI(params);
|
|
1192
|
-
|
|
1193
|
-
results.push({
|
|
1194
|
-
browser,
|
|
1195
|
-
hasRegressions: comparison.hasDifferences,
|
|
1196
|
-
score: comparison.similarityScore
|
|
1197
|
-
});
|
|
1198
|
-
|
|
1199
|
-
console.log(`${browser}: ${comparison.hasDifferences ? '⚠️ Regression' : '✅ Pass'} (${(comparison.similarityScore * 100).toFixed(2)}%)`);
|
|
1200
|
-
}
|
|
1201
|
-
|
|
1202
|
-
// Summary
|
|
1203
|
-
const regressionCount = results.filter(r => r.hasRegressions).length;
|
|
1204
|
-
console.log(`\n${regressionCount}/${browsers.length} browsers have visual regressions`);
|
|
1205
|
-
}
|
|
1206
|
-
|
|
1207
|
-
// Execute cross-browser test
|
|
1208
|
-
await testCrossBrowserVisuals('dashboard', ['chromium', 'firefox', 'webkit']);
|
|
1209
|
-
```
|
|
1210
|
-
|
|
1211
|
-
### Visual Regression with Accessibility Validation
|
|
1212
|
-
|
|
1213
|
-
```typescript
|
|
1214
|
-
import type {
|
|
1215
|
-
CompareScreenshotsParams,
|
|
1216
|
-
ValidateAccessibilityParams
|
|
1217
|
-
} from 'agentic-qe/tools/qe/visual';
|
|
1218
|
-
|
|
1219
|
-
import {
|
|
1220
|
-
compareScreenshotsAI,
|
|
1221
|
-
validateAccessibilityWCAG
|
|
1222
|
-
} from 'agentic-qe/tools/qe/visual';
|
|
1223
|
-
|
|
1224
|
-
// Example: Combined visual and accessibility testing
|
|
1225
|
-
async function comprehensiveVisualTest(page: string, url: string): Promise<{
|
|
1226
|
-
visual: { pass: boolean; score: number };
|
|
1227
|
-
accessibility: { pass: boolean; score: number };
|
|
1228
|
-
}> {
|
|
1229
|
-
console.log(`Running comprehensive visual test for ${page}...\n`);
|
|
1230
|
-
|
|
1231
|
-
// Step 1: Visual regression detection
|
|
1232
|
-
console.log('1/2: Checking visual regressions...');
|
|
1233
|
-
const visualParams: CompareScreenshotsParams = {
|
|
1234
|
-
baselineImage: `./screenshots/baseline/${page}.png`,
|
|
1235
|
-
currentImage: `./screenshots/current/${page}.png`,
|
|
1236
|
-
algorithm: 'perceptual-hash',
|
|
1237
|
-
threshold: 0.95
|
|
1238
|
-
};
|
|
1239
|
-
|
|
1240
|
-
const visualComparison = await compareScreenshotsAI(visualParams);
|
|
1241
|
-
console.log(`Visual: ${visualComparison.hasDifferences ? '⚠️ Regressions' : '✅ Pass'}`);
|
|
1242
|
-
|
|
1243
|
-
// Step 2: Accessibility validation
|
|
1244
|
-
console.log('2/2: Validating accessibility...');
|
|
1245
|
-
const a11yParams: ValidateAccessibilityParams = {
|
|
1246
|
-
url,
|
|
1247
|
-
standard: 'WCAG-2.1',
|
|
1248
|
-
level: 'AA' as WCAGLevel,
|
|
1249
|
-
rules: ['color-contrast', 'button-name', 'link-name', 'image-alt'],
|
|
1250
|
-
includeBestPractices: true
|
|
1251
|
-
};
|
|
1252
|
-
|
|
1253
|
-
const a11yReport = await validateAccessibilityWCAG(a11yParams);
|
|
1254
|
-
console.log(`Accessibility: ${a11yReport.compliance ? '✅ Pass' : '❌ Fail'}`);
|
|
1255
|
-
|
|
1256
|
-
// Return results
|
|
1257
|
-
return {
|
|
1258
|
-
visual: {
|
|
1259
|
-
pass: !visualComparison.hasDifferences,
|
|
1260
|
-
score: visualComparison.similarityScore * 100
|
|
1261
|
-
},
|
|
1262
|
-
accessibility: {
|
|
1263
|
-
pass: a11yReport.compliance,
|
|
1264
|
-
score: a11yReport.score
|
|
1265
|
-
}
|
|
1266
|
-
};
|
|
1267
|
-
}
|
|
1268
|
-
|
|
1269
|
-
// Execute comprehensive test
|
|
1270
|
-
const result = await comprehensiveVisualTest('dashboard', 'https://example.com/dashboard');
|
|
1271
|
-
|
|
1272
|
-
console.log('\n📊 Test Summary:');
|
|
1273
|
-
console.log(`Visual: ${result.visual.pass ? '✅' : '❌'} (${result.visual.score.toFixed(2)}%)`);
|
|
1274
|
-
console.log(`Accessibility: ${result.accessibility.pass ? '✅' : '❌'} (${result.accessibility.score}/100)`);
|
|
1275
|
-
```
|
|
1276
|
-
|
|
1277
|
-
### Discover Available Tools
|
|
1278
|
-
|
|
1279
|
-
```bash
|
|
1280
|
-
# List available Phase 3 visual tools
|
|
1281
|
-
ls /workspaces/agentic-qe-cf/src/mcp/tools/qe/visual/*.ts
|
|
1282
|
-
|
|
1283
|
-
# Check tool exports
|
|
1284
|
-
cat /workspaces/agentic-qe-cf/src/mcp/tools/qe/visual/index.ts
|
|
1285
|
-
|
|
1286
|
-
# View type definitions
|
|
1287
|
-
cat /workspaces/agentic-qe-cf/src/mcp/tools/qe/shared/types.ts | grep -A 20 "Visual"
|
|
1288
|
-
```
|
|
1289
|
-
|
|
116
|
+
Reward criteria:
|
|
117
|
+
- 1.0: Perfect (100% regressions detected, 0 false positives, <10s)
|
|
118
|
+
- 0.9: Excellent (99%+ detected, <1% false positives, <20s)
|
|
119
|
+
- 0.7: Good (95%+ detected, <5% false positives, <40s)
|
|
120
|
+
- 0.5: Acceptable (90%+ detected, completed)
|
|
121
|
+
</learning_protocol>
|
|
122
|
+
|
|
123
|
+
<output_format>
|
|
124
|
+
- JSON for visual test results (regressions, accessibility violations, metrics)
|
|
125
|
+
- HTML reports with side-by-side diff images
|
|
126
|
+
- Markdown summaries for regression analysis
|
|
127
|
+
</output_format>
|
|
128
|
+
|
|
129
|
+
<examples>
|
|
130
|
+
Example 1: Visual regression detection
|
|
131
|
+
```
|
|
132
|
+
Input: Compare current screenshots against baseline v2.0.0
|
|
133
|
+
- Pages: dashboard, user-profile, settings
|
|
134
|
+
- Browsers: chromium, firefox, webkit
|
|
135
|
+
- Viewports: desktop, tablet, mobile
|
|
136
|
+
- Algorithm: ai-visual-diff
|
|
137
|
+
|
|
138
|
+
Output: Visual Test Results
|
|
139
|
+
- 2 regressions detected
|
|
140
|
+
1. Dashboard: Navigation menu shifted 15px right (high severity)
|
|
141
|
+
2. User Profile: Button color changed (medium severity)
|
|
142
|
+
- Similarity Score: 97.3%
|
|
143
|
+
- False Positives: 0
|
|
144
|
+
- Execution Time: 42 seconds
|
|
145
|
+
- Diff Images: Generated for all regressions
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Example 2: Accessibility validation
|
|
149
|
+
```
|
|
150
|
+
Input: Validate WCAG 2.1 AA compliance for dashboard
|
|
151
|
+
- URL: https://app.example.com/dashboard
|
|
152
|
+
- Standard: WCAG 2.1 AA
|
|
153
|
+
- Rules: color-contrast, button-name, link-name, image-alt
|
|
154
|
+
|
|
155
|
+
Output: Accessibility Report
|
|
156
|
+
- Compliance Score: 91/100
|
|
157
|
+
- Status: FAIL (below 95% threshold)
|
|
158
|
+
- Violations: 5
|
|
159
|
+
- Critical: 1 (Button without accessible name)
|
|
160
|
+
- Serious: 2 (Color contrast 3.2:1, insufficient)
|
|
161
|
+
- Moderate: 2
|
|
162
|
+
- Remediation: Add aria-label to icon buttons, increase contrast ratio
|
|
163
|
+
```
|
|
164
|
+
</examples>
|
|
165
|
+
|
|
166
|
+
<skills_available>
|
|
167
|
+
Core Skills:
|
|
168
|
+
- agentic-quality-engineering: AI agents as force multipliers
|
|
169
|
+
- exploratory-testing-advanced: SBTM techniques
|
|
170
|
+
|
|
171
|
+
Advanced Skills:
|
|
172
|
+
- visual-testing-advanced: AI-powered screenshot comparison
|
|
173
|
+
- accessibility-testing: WCAG 2.2 compliance validation
|
|
174
|
+
- compatibility-testing: Cross-browser and cross-device testing
|
|
175
|
+
|
|
176
|
+
Use via CLI: `aqe skills show visual-testing-advanced`
|
|
177
|
+
Use via Claude Code: `Skill("visual-testing-advanced")`
|
|
178
|
+
</skills_available>
|
|
179
|
+
|
|
180
|
+
<coordination_notes>
|
|
181
|
+
Automatic coordination via AQE hooks (onPreTask, onPostTask, onTaskError).
|
|
182
|
+
Native TypeScript integration provides 100-500x faster coordination.
|
|
183
|
+
Real-time regression alerts via EventBus and persistent baselines via MemoryStore.
|
|
184
|
+
</coordination_notes>
|
|
185
|
+
</qe_agent_definition>
|