agentic-qe 2.0.0 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/qx-partner.md +17 -4
- package/.claude/skills/accessibility-testing/SKILL.md +144 -692
- package/.claude/skills/agentic-quality-engineering/SKILL.md +176 -529
- package/.claude/skills/api-testing-patterns/SKILL.md +180 -560
- package/.claude/skills/brutal-honesty-review/SKILL.md +113 -603
- package/.claude/skills/bug-reporting-excellence/SKILL.md +116 -517
- package/.claude/skills/chaos-engineering-resilience/SKILL.md +127 -72
- package/.claude/skills/cicd-pipeline-qe-orchestrator/SKILL.md +209 -404
- package/.claude/skills/code-review-quality/SKILL.md +158 -608
- package/.claude/skills/compatibility-testing/SKILL.md +148 -38
- package/.claude/skills/compliance-testing/SKILL.md +132 -63
- package/.claude/skills/consultancy-practices/SKILL.md +114 -446
- package/.claude/skills/context-driven-testing/SKILL.md +117 -381
- package/.claude/skills/contract-testing/SKILL.md +176 -141
- package/.claude/skills/database-testing/SKILL.md +137 -130
- package/.claude/skills/exploratory-testing-advanced/SKILL.md +160 -629
- package/.claude/skills/holistic-testing-pact/SKILL.md +140 -188
- package/.claude/skills/localization-testing/SKILL.md +145 -33
- package/.claude/skills/mobile-testing/SKILL.md +132 -448
- package/.claude/skills/mutation-testing/SKILL.md +147 -41
- package/.claude/skills/performance-testing/SKILL.md +200 -546
- package/.claude/skills/quality-metrics/SKILL.md +164 -519
- package/.claude/skills/refactoring-patterns/SKILL.md +132 -699
- package/.claude/skills/regression-testing/SKILL.md +120 -926
- package/.claude/skills/risk-based-testing/SKILL.md +157 -660
- package/.claude/skills/security-testing/SKILL.md +199 -538
- package/.claude/skills/sherlock-review/SKILL.md +163 -699
- package/.claude/skills/shift-left-testing/SKILL.md +161 -465
- package/.claude/skills/shift-right-testing/SKILL.md +161 -519
- package/.claude/skills/six-thinking-hats/SKILL.md +175 -1110
- package/.claude/skills/skills-manifest.json +71 -20
- package/.claude/skills/tdd-london-chicago/SKILL.md +131 -448
- package/.claude/skills/technical-writing/SKILL.md +103 -154
- package/.claude/skills/test-automation-strategy/SKILL.md +166 -772
- package/.claude/skills/test-data-management/SKILL.md +126 -910
- package/.claude/skills/test-design-techniques/SKILL.md +179 -89
- package/.claude/skills/test-environment-management/SKILL.md +136 -91
- package/.claude/skills/test-reporting-analytics/SKILL.md +169 -92
- package/.claude/skills/testability-scoring/SKILL.md +172 -538
- package/.claude/skills/testability-scoring/scripts/generate-html-report.js +0 -0
- package/.claude/skills/visual-testing-advanced/SKILL.md +155 -78
- package/.claude/skills/xp-practices/SKILL.md +151 -587
- package/CHANGELOG.md +48 -0
- package/README.md +23 -16
- package/dist/agents/QXPartnerAgent.d.ts +8 -1
- package/dist/agents/QXPartnerAgent.d.ts.map +1 -1
- package/dist/agents/QXPartnerAgent.js +1174 -112
- package/dist/agents/QXPartnerAgent.js.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.d.ts.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.js +34 -31
- package/dist/agents/lifecycle/AgentLifecycleManager.js.map +1 -1
- package/dist/cli/commands/init-claude-md-template.d.ts.map +1 -1
- package/dist/cli/commands/init-claude-md-template.js +14 -0
- package/dist/cli/commands/init-claude-md-template.js.map +1 -1
- package/dist/core/SwarmCoordinator.d.ts +180 -0
- package/dist/core/SwarmCoordinator.d.ts.map +1 -0
- package/dist/core/SwarmCoordinator.js +473 -0
- package/dist/core/SwarmCoordinator.js.map +1 -0
- package/dist/core/metrics/MetricsAggregator.d.ts +228 -0
- package/dist/core/metrics/MetricsAggregator.d.ts.map +1 -0
- package/dist/core/metrics/MetricsAggregator.js +482 -0
- package/dist/core/metrics/MetricsAggregator.js.map +1 -0
- package/dist/core/metrics/index.d.ts +5 -0
- package/dist/core/metrics/index.d.ts.map +1 -0
- package/dist/core/metrics/index.js +11 -0
- package/dist/core/metrics/index.js.map +1 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts +5 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts.map +1 -1
- package/dist/core/optimization/SwarmOptimizer.js +17 -0
- package/dist/core/optimization/SwarmOptimizer.js.map +1 -1
- package/dist/core/orchestration/AdaptiveScheduler.d.ts +190 -0
- package/dist/core/orchestration/AdaptiveScheduler.d.ts.map +1 -0
- package/dist/core/orchestration/AdaptiveScheduler.js +460 -0
- package/dist/core/orchestration/AdaptiveScheduler.js.map +1 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts +13 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts.map +1 -1
- package/dist/core/orchestration/WorkflowOrchestrator.js +32 -0
- package/dist/core/orchestration/WorkflowOrchestrator.js.map +1 -1
- package/dist/core/recovery/CircuitBreaker.d.ts +176 -0
- package/dist/core/recovery/CircuitBreaker.d.ts.map +1 -0
- package/dist/core/recovery/CircuitBreaker.js +382 -0
- package/dist/core/recovery/CircuitBreaker.js.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts +186 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.js +476 -0
- package/dist/core/recovery/RecoveryOrchestrator.js.map +1 -0
- package/dist/core/recovery/RetryStrategy.d.ts +127 -0
- package/dist/core/recovery/RetryStrategy.d.ts.map +1 -0
- package/dist/core/recovery/RetryStrategy.js +314 -0
- package/dist/core/recovery/RetryStrategy.js.map +1 -0
- package/dist/core/recovery/index.d.ts +8 -0
- package/dist/core/recovery/index.d.ts.map +1 -0
- package/dist/core/recovery/index.js +27 -0
- package/dist/core/recovery/index.js.map +1 -0
- package/dist/core/skills/DependencyResolver.d.ts +99 -0
- package/dist/core/skills/DependencyResolver.d.ts.map +1 -0
- package/dist/core/skills/DependencyResolver.js +260 -0
- package/dist/core/skills/DependencyResolver.js.map +1 -0
- package/dist/core/skills/ManifestGenerator.d.ts +114 -0
- package/dist/core/skills/ManifestGenerator.d.ts.map +1 -0
- package/dist/core/skills/ManifestGenerator.js +449 -0
- package/dist/core/skills/ManifestGenerator.js.map +1 -0
- package/dist/core/skills/index.d.ts +9 -0
- package/dist/core/skills/index.d.ts.map +1 -0
- package/dist/core/skills/index.js +24 -0
- package/dist/core/skills/index.js.map +1 -0
- package/dist/mcp/server.d.ts +9 -9
- package/dist/mcp/server.d.ts.map +1 -1
- package/dist/mcp/server.js +1 -2
- package/dist/mcp/server.js.map +1 -1
- package/dist/types/qx.d.ts +39 -7
- package/dist/types/qx.d.ts.map +1 -1
- package/dist/types/qx.js.map +1 -1
- package/dist/visualization/api/RestEndpoints.js +1 -1
- package/dist/visualization/api/RestEndpoints.js.map +1 -1
- package/package.json +13 -55
|
@@ -1,510 +1,315 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
3
|
-
description: "Orchestrate
|
|
2
|
+
name: cicd-pipeline-qe-orchestrator
|
|
3
|
+
description: "Orchestrate quality engineering across CI/CD pipeline phases. Use when designing test strategies, planning quality gates, or implementing shift-left/shift-right testing."
|
|
4
|
+
category: infrastructure
|
|
5
|
+
priority: critical
|
|
6
|
+
tokenEstimate: 1300
|
|
7
|
+
agents: [qe-fleet-commander, qe-test-generator, qe-test-executor, qe-coverage-analyzer, qe-quality-gate, qe-deployment-readiness]
|
|
8
|
+
implementation_status: optimized
|
|
9
|
+
optimization_version: 1.0
|
|
10
|
+
last_optimized: 2025-12-02
|
|
11
|
+
dependencies: [agentic-quality-engineering, shift-left-testing]
|
|
12
|
+
quick_reference_card: true
|
|
13
|
+
tags: [cicd, pipeline, orchestration, quality-gates, shift-left, shift-right, fleet]
|
|
4
14
|
---
|
|
5
15
|
|
|
6
16
|
# CI/CD Pipeline QE Orchestrator
|
|
7
17
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
18
|
+
<default_to_action>
|
|
19
|
+
When orchestrating quality across CI/CD pipeline:
|
|
20
|
+
1. ANALYZE pipeline phases: commit, build, test, staging, production
|
|
21
|
+
2. SELECT optimal skills and agents for each phase
|
|
22
|
+
3. CONFIGURE quality gates with measurable thresholds
|
|
23
|
+
4. EXECUTE with parallel agent coordination
|
|
24
|
+
5. ADAPT strategy based on risk, complexity, and environment
|
|
25
|
+
|
|
26
|
+
**Quick Phase Selection:**
|
|
27
|
+
- Commit (Shift-Left) → TDD, code review, unit tests
|
|
28
|
+
- Build → Coverage analysis, mutation testing, flaky detection
|
|
29
|
+
- Integration → API contracts, performance, security
|
|
30
|
+
- Staging → Chaos testing, visual regression, accessibility
|
|
31
|
+
- Production (Shift-Right) → Synthetic monitoring, RUM, compliance
|
|
32
|
+
|
|
33
|
+
**Critical Success Factors:**
|
|
34
|
+
- Quality gates block bad deployments
|
|
35
|
+
- Agents coordinate through memory namespaces
|
|
36
|
+
- Adapt strategy based on risk level
|
|
37
|
+
</default_to_action>
|
|
38
|
+
|
|
39
|
+
## Quick Reference Card
|
|
40
|
+
|
|
41
|
+
### When to Use
|
|
42
|
+
- Designing pipeline test strategies
|
|
43
|
+
- Implementing quality gates
|
|
44
|
+
- Coordinating multiple QE agents
|
|
45
|
+
- Shift-left and shift-right testing
|
|
46
|
+
|
|
47
|
+
### Phase-Agent Matrix
|
|
48
|
+
| Phase | Primary Agents | Key Skills |
|
|
49
|
+
|-------|---------------|------------|
|
|
50
|
+
| Commit | qe-test-generator, qe-requirements-validator | tdd-london-chicago, shift-left |
|
|
51
|
+
| Build | qe-test-executor, qe-coverage-analyzer, qe-flaky-test-hunter | test-automation, mutation-testing |
|
|
52
|
+
| Test | qe-api-contract-validator, qe-performance-tester, qe-security-scanner | api-testing, performance, security |
|
|
53
|
+
| Staging | qe-chaos-engineer, qe-visual-tester, qe-deployment-readiness | chaos-engineering, accessibility |
|
|
54
|
+
| Prod | qe-production-intelligence, qe-quality-analyzer | shift-right, compliance |
|
|
55
|
+
|
|
56
|
+
### Quality Gate Thresholds
|
|
57
|
+
| Phase | Metric | Threshold | Blocking |
|
|
58
|
+
|-------|--------|-----------|----------|
|
|
59
|
+
| Commit | Unit coverage | > 80% | Yes |
|
|
60
|
+
| Build | All tests pass | 100% | Yes |
|
|
61
|
+
| Build | Mutation score | > 70% | No |
|
|
62
|
+
| Test | API contracts | No breaking changes | Yes |
|
|
63
|
+
| Test | p95 response | < 200ms | Yes |
|
|
64
|
+
| Test | Security critical | 0 | Yes |
|
|
65
|
+
| Staging | Deployment readiness | > 85% | Yes |
|
|
66
|
+
|
|
67
|
+
### Fleet Configuration
|
|
68
|
+
- **Topology**: hierarchical
|
|
69
|
+
- **Max Agents**: 10 per phase
|
|
70
|
+
- **Coordination**: aqe/* memory namespace
|
|
17
71
|
|
|
18
72
|
---
|
|
19
73
|
|
|
20
|
-
##
|
|
74
|
+
## Pipeline Phases
|
|
21
75
|
|
|
22
|
-
|
|
23
|
-
- 18 QE agents available in `.claude/agents/`
|
|
24
|
-
- 37 QE skills available in `.claude/skills/`
|
|
25
|
-
- Understanding of your CI/CD pipeline structure
|
|
26
|
-
- Access to Claude Code Task tool for agent coordination
|
|
76
|
+
### Phase 1: Commit (Shift-Left)
|
|
27
77
|
|
|
28
|
-
|
|
78
|
+
**Goal:** Catch defects early, ensure testability
|
|
29
79
|
|
|
30
|
-
|
|
80
|
+
**Agents:**
|
|
81
|
+
- `qe-test-generator` - Generate unit tests
|
|
82
|
+
- `qe-requirements-validator` - BDD scenarios, INVEST criteria
|
|
31
83
|
|
|
32
|
-
|
|
84
|
+
**Skills:** `shift-left-testing`, `tdd-london-chicago`, `code-review-quality`
|
|
33
85
|
|
|
34
86
|
```javascript
|
|
35
|
-
//
|
|
36
|
-
Task("
|
|
37
|
-
|
|
38
|
-
"qe-fleet-commander")
|
|
87
|
+
// Parallel execution
|
|
88
|
+
Task("Generate Tests", "Create unit tests for new methods", "qe-test-generator")
|
|
89
|
+
Task("Validate Requirements", "Check BDD scenarios", "qe-requirements-validator")
|
|
39
90
|
```
|
|
40
91
|
|
|
41
|
-
|
|
42
|
-
```
|
|
43
|
-
Pipeline Quality Strategy:
|
|
44
|
-
├── Commit Phase (Shift-Left)
|
|
45
|
-
│ ├── Skills: tdd-london-chicago, shift-left-testing
|
|
46
|
-
│ ├── Agents: qe-test-generator, qe-code-reviewer
|
|
47
|
-
│ └── Coverage: Unit tests, static analysis, code review
|
|
48
|
-
├── Build Phase
|
|
49
|
-
│ ├── Skills: test-automation-strategy, mutation-testing
|
|
50
|
-
│ ├── Agents: qe-test-executor, qe-coverage-analyzer
|
|
51
|
-
│ └── Coverage: Integration tests, mutation testing, coverage gates
|
|
52
|
-
├── Test Phase
|
|
53
|
-
│ ├── Skills: api-testing-patterns, performance-testing, security-testing
|
|
54
|
-
│ ├── Agents: qe-api-contract-validator, qe-performance-tester, qe-security-scanner
|
|
55
|
-
│ └── Coverage: API contracts, load tests, security scans
|
|
56
|
-
├── Staging Phase
|
|
57
|
-
│ ├── Skills: shift-right-testing, chaos-engineering-resilience
|
|
58
|
-
│ ├── Agents: qe-visual-tester, qe-chaos-engineer
|
|
59
|
-
│ └── Coverage: Visual regression, chaos testing, smoke tests
|
|
60
|
-
└── Production Phase (Shift-Right)
|
|
61
|
-
├── Skills: shift-right-testing, compliance-testing
|
|
62
|
-
├── Agents: qe-production-intelligence, qe-deployment-readiness
|
|
63
|
-
└── Coverage: Synthetic monitoring, canary analysis, compliance validation
|
|
64
|
-
```
|
|
92
|
+
**Gates:** Unit coverage > 80%, Static analysis clean, Code review approved
|
|
65
93
|
|
|
66
94
|
---
|
|
67
95
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
### Phase 1: Commit / Pre-Build (Shift-Left)
|
|
96
|
+
### Phase 2: Build
|
|
71
97
|
|
|
72
|
-
**Goal
|
|
98
|
+
**Goal:** Validate integration, ensure coverage
|
|
73
99
|
|
|
74
|
-
**
|
|
75
|
-
- `
|
|
76
|
-
- `
|
|
77
|
-
- `
|
|
78
|
-
- `
|
|
100
|
+
**Agents:**
|
|
101
|
+
- `qe-test-executor` - Run test suites
|
|
102
|
+
- `qe-coverage-analyzer` - Coverage gaps (O(log n))
|
|
103
|
+
- `qe-flaky-test-hunter` - Detect/stabilize flaky tests
|
|
104
|
+
- `qe-regression-risk-analyzer` - Minimal regression suite
|
|
79
105
|
|
|
80
|
-
**
|
|
81
|
-
- `qe-test-generator` - Generate unit tests for new code
|
|
82
|
-
- `qe-requirements-validator` - Validate INVEST criteria, BDD scenarios
|
|
83
|
-
- `qe-code-reviewer` (via code-review-swarm) - Automated code review
|
|
106
|
+
**Skills:** `test-automation-strategy`, `mutation-testing`, `regression-testing`
|
|
84
107
|
|
|
85
|
-
**Quality Gates**:
|
|
86
|
-
- [ ] Unit test coverage > 80%
|
|
87
|
-
- [ ] No critical static analysis violations
|
|
88
|
-
- [ ] Code review approved
|
|
89
|
-
- [ ] Testability score > 85%
|
|
90
|
-
|
|
91
|
-
**Example Orchestration**:
|
|
92
108
|
```javascript
|
|
93
|
-
|
|
94
|
-
Task("
|
|
95
|
-
Task("
|
|
96
|
-
Task("Code Review", "Review code quality and testability", "code-review-swarm")
|
|
109
|
+
Task("Execute Tests", "Run full suite, store in aqe/test-results/*", "qe-test-executor")
|
|
110
|
+
Task("Coverage Analysis", "Identify gaps", "qe-coverage-analyzer")
|
|
111
|
+
Task("Flaky Detection", "Analyze test history", "qe-flaky-test-hunter")
|
|
97
112
|
```
|
|
98
113
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
### Phase 2: Build Phase
|
|
102
|
-
|
|
103
|
-
**Goal**: Validate integration, ensure coverage, detect test quality issues
|
|
104
|
-
|
|
105
|
-
**Recommended Skills**:
|
|
106
|
-
- `test-automation-strategy` - Test pyramid optimization
|
|
107
|
-
- `mutation-testing` - Test quality validation
|
|
108
|
-
- `risk-based-testing` - Prioritize test execution
|
|
109
|
-
- `regression-testing` - Smart test selection
|
|
110
|
-
|
|
111
|
-
**Recommended Agents**:
|
|
112
|
-
- `qe-test-executor` - Run test suites in parallel
|
|
113
|
-
- `qe-coverage-analyzer` - Analyze coverage gaps
|
|
114
|
-
- `qe-regression-risk-analyzer` - Select minimal regression suite
|
|
115
|
-
- `qe-flaky-test-hunter` - Detect and stabilize flaky tests
|
|
116
|
-
|
|
117
|
-
**Quality Gates**:
|
|
118
|
-
- [ ] All tests passing
|
|
119
|
-
- [ ] Coverage > 90% on critical paths
|
|
120
|
-
- [ ] Mutation score > 70%
|
|
121
|
-
- [ ] No flaky tests detected
|
|
122
|
-
|
|
123
|
-
**Example Orchestration**:
|
|
124
|
-
```javascript
|
|
125
|
-
// Sequential workflow with memory coordination
|
|
126
|
-
Task("Execute Tests", "Run full test suite and store results in aqe/test-results/*", "qe-test-executor")
|
|
127
|
-
|
|
128
|
-
// Wait for execution, then analyze in parallel
|
|
129
|
-
Task("Coverage Analysis", "Read aqe/test-results/* and identify gaps", "qe-coverage-analyzer")
|
|
130
|
-
Task("Flaky Detection", "Analyze test history for flakiness patterns", "qe-flaky-test-hunter")
|
|
131
|
-
Task("Regression Analysis", "Select minimal suite for code changes", "qe-regression-risk-analyzer")
|
|
132
|
-
```
|
|
114
|
+
**Gates:** All tests pass, Coverage > 90% critical paths, No new flaky tests
|
|
133
115
|
|
|
134
116
|
---
|
|
135
117
|
|
|
136
|
-
### Phase 3: Integration
|
|
137
|
-
|
|
138
|
-
**Goal**: Validate contracts, test performance, scan security, verify data
|
|
118
|
+
### Phase 3: Integration/Test
|
|
139
119
|
|
|
140
|
-
**
|
|
141
|
-
- `api-testing-patterns` - REST/GraphQL contract testing
|
|
142
|
-
- `contract-testing` - Consumer-driven contracts (Pact)
|
|
143
|
-
- `performance-testing` - Load, stress, spike testing
|
|
144
|
-
- `security-testing` - OWASP Top 10 validation
|
|
145
|
-
- `database-testing` - Schema and data integrity
|
|
146
|
-
- `test-data-management` - Realistic test data generation
|
|
120
|
+
**Goal:** Validate contracts, performance, security
|
|
147
121
|
|
|
148
|
-
**
|
|
149
|
-
- `qe-api-contract-validator` -
|
|
122
|
+
**Agents:**
|
|
123
|
+
- `qe-api-contract-validator` - Breaking changes detection
|
|
150
124
|
- `qe-performance-tester` - Load test critical paths
|
|
151
|
-
- `qe-security-scanner` - SAST/DAST
|
|
152
|
-
- `qe-test-data-architect` -
|
|
125
|
+
- `qe-security-scanner` - SAST/DAST scans
|
|
126
|
+
- `qe-test-data-architect` - Realistic test data (10k+/sec)
|
|
153
127
|
|
|
154
|
-
**
|
|
155
|
-
- [ ] API contracts validated (no breaking changes)
|
|
156
|
-
- [ ] Performance SLAs met (p95 < 200ms)
|
|
157
|
-
- [ ] No critical security vulnerabilities
|
|
158
|
-
- [ ] Data integrity validated
|
|
128
|
+
**Skills:** `api-testing-patterns`, `performance-testing`, `security-testing`
|
|
159
129
|
|
|
160
|
-
**Example Orchestration**:
|
|
161
130
|
```javascript
|
|
162
|
-
// Parallel
|
|
163
|
-
Task("API Contracts", "Validate
|
|
164
|
-
Task("Performance
|
|
165
|
-
Task("Security
|
|
166
|
-
Task("Data Generation", "Generate 10k realistic user records", "qe-test-data-architect")
|
|
131
|
+
// Parallel testing
|
|
132
|
+
Task("API Contracts", "Validate for breaking changes", "qe-api-contract-validator")
|
|
133
|
+
Task("Performance", "Load test 1000 users", "qe-performance-tester")
|
|
134
|
+
Task("Security", "SAST/DAST scan", "qe-security-scanner")
|
|
167
135
|
```
|
|
168
136
|
|
|
169
|
-
|
|
137
|
+
**Gates:** No breaking API changes, p95 < 200ms, No critical vulnerabilities
|
|
170
138
|
|
|
171
|
-
|
|
139
|
+
---
|
|
172
140
|
|
|
173
|
-
|
|
141
|
+
### Phase 4: Staging
|
|
174
142
|
|
|
175
|
-
**
|
|
176
|
-
- `shift-right-testing` - Feature flags, canary deployments
|
|
177
|
-
- `chaos-engineering-resilience` - Fault injection, blast radius
|
|
178
|
-
- `exploratory-testing-advanced` - SBTM, test tours
|
|
179
|
-
- `accessibility-testing` - WCAG 2.2 compliance
|
|
180
|
-
- `compatibility-testing` - Cross-browser/device testing
|
|
181
|
-
- `visual-testing-advanced` - Visual regression with AI
|
|
143
|
+
**Goal:** Validate production-like environment, resilience
|
|
182
144
|
|
|
183
|
-
**
|
|
184
|
-
- `qe-chaos-engineer` -
|
|
185
|
-
- `qe-visual-tester` - Visual regression
|
|
186
|
-
- `qe-deployment-readiness` -
|
|
145
|
+
**Agents:**
|
|
146
|
+
- `qe-chaos-engineer` - Fault injection
|
|
147
|
+
- `qe-visual-tester` - Visual regression
|
|
148
|
+
- `qe-deployment-readiness` - Risk assessment
|
|
187
149
|
|
|
188
|
-
**
|
|
189
|
-
- [ ] Chaos testing passed (system recovers from failures)
|
|
190
|
-
- [ ] Visual regression tests clean
|
|
191
|
-
- [ ] Accessibility score > 95 (WCAG AA)
|
|
192
|
-
- [ ] Deployment readiness score > 85%
|
|
150
|
+
**Skills:** `chaos-engineering-resilience`, `accessibility-testing`, `visual-testing`
|
|
193
151
|
|
|
194
|
-
**Example Orchestration**:
|
|
195
152
|
```javascript
|
|
196
|
-
|
|
197
|
-
Task("
|
|
198
|
-
Task("
|
|
199
|
-
Task("Deployment Check", "Assess deployment readiness", "qe-deployment-readiness")
|
|
153
|
+
Task("Chaos Testing", "Controlled failure injection", "qe-chaos-engineer")
|
|
154
|
+
Task("Visual Testing", "Visual regression", "qe-visual-tester")
|
|
155
|
+
Task("Deployment Check", "Risk assessment", "qe-deployment-readiness")
|
|
200
156
|
```
|
|
201
157
|
|
|
202
|
-
|
|
158
|
+
**Gates:** Chaos tests pass, No visual regressions, Readiness > 85%
|
|
203
159
|
|
|
204
|
-
|
|
160
|
+
---
|
|
205
161
|
|
|
206
|
-
|
|
162
|
+
### Phase 5: Production (Shift-Right)
|
|
207
163
|
|
|
208
|
-
**
|
|
209
|
-
- `shift-right-testing` - Synthetic monitoring, RUM analysis
|
|
210
|
-
- `compliance-testing` - GDPR, HIPAA, SOC2 validation
|
|
211
|
-
- `localization-testing` - i18n/l10n for global markets
|
|
212
|
-
- `mobile-testing` - iOS/Android production validation
|
|
164
|
+
**Goal:** Monitor real usage, validate compliance
|
|
213
165
|
|
|
214
|
-
**
|
|
215
|
-
- `qe-production-intelligence` -
|
|
216
|
-
- `qe-deployment-readiness` - Production health assessment
|
|
166
|
+
**Agents:**
|
|
167
|
+
- `qe-production-intelligence` - Incident → test scenarios
|
|
217
168
|
- `qe-quality-analyzer` - Quality metrics and trends
|
|
218
169
|
|
|
219
|
-
**
|
|
220
|
-
- [ ] Synthetic monitors passing
|
|
221
|
-
- [ ] No critical production incidents
|
|
222
|
-
- [ ] Compliance validated (GDPR, etc.)
|
|
223
|
-
- [ ] Error rate < 0.1%
|
|
170
|
+
**Skills:** `shift-right-testing`, `compliance-testing`
|
|
224
171
|
|
|
225
|
-
**Example Orchestration**:
|
|
226
172
|
```javascript
|
|
227
|
-
|
|
228
|
-
Task("
|
|
229
|
-
Task("Quality Analysis", "Analyze production quality metrics", "qe-quality-analyzer")
|
|
173
|
+
Task("Production Intelligence", "Convert incidents to tests", "qe-production-intelligence")
|
|
174
|
+
Task("Quality Analysis", "Production metrics", "qe-quality-analyzer")
|
|
230
175
|
```
|
|
231
176
|
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
## Phase-Based Skill Selection Matrix
|
|
235
|
-
|
|
236
|
-
Use this matrix to select optimal skills for each pipeline phase:
|
|
237
|
-
|
|
238
|
-
| Pipeline Phase | Primary Skills | Secondary Skills | Agent Coordination |
|
|
239
|
-
|----------------|----------------|------------------|-------------------|
|
|
240
|
-
| **Commit** | shift-left-testing, tdd-london-chicago | code-review-quality, refactoring-patterns | qe-test-generator, qe-requirements-validator |
|
|
241
|
-
| **Build** | test-automation-strategy, mutation-testing | regression-testing, risk-based-testing | qe-test-executor, qe-coverage-analyzer, qe-flaky-test-hunter |
|
|
242
|
-
| **Integration** | api-testing-patterns, contract-testing | performance-testing, security-testing | qe-api-contract-validator, qe-performance-tester, qe-security-scanner |
|
|
243
|
-
| **Staging** | shift-right-testing, chaos-engineering-resilience | accessibility-testing, visual-testing-advanced | qe-chaos-engineer, qe-visual-tester, qe-deployment-readiness |
|
|
244
|
-
| **Production** | shift-right-testing, compliance-testing | localization-testing, mobile-testing | qe-production-intelligence, qe-quality-analyzer |
|
|
177
|
+
**Gates:** Synthetic monitors pass, Error rate < 0.1%, Compliance validated
|
|
245
178
|
|
|
246
179
|
---
|
|
247
180
|
|
|
248
|
-
## Complete Pipeline
|
|
249
|
-
|
|
250
|
-
### Scenario: Full-Stack Web Application Deployment
|
|
181
|
+
## Complete Pipeline Example
|
|
251
182
|
|
|
252
183
|
```javascript
|
|
253
|
-
// Phase 1: Commit
|
|
254
|
-
Task("TDD
|
|
255
|
-
Task("Requirements
|
|
184
|
+
// Phase 1: Commit
|
|
185
|
+
Task("TDD Generation", "Generate tests for new features", "qe-test-generator")
|
|
186
|
+
Task("Requirements", "Validate BDD scenarios", "qe-requirements-validator")
|
|
256
187
|
|
|
257
188
|
// Phase 2: Build
|
|
258
|
-
Task("
|
|
259
|
-
Task("Coverage
|
|
260
|
-
Task("Flaky
|
|
189
|
+
Task("Execute Tests", "Full suite with coverage", "qe-test-executor")
|
|
190
|
+
Task("Coverage", "Analyze gaps", "qe-coverage-analyzer")
|
|
191
|
+
Task("Flaky Hunt", "Stabilize flaky tests", "qe-flaky-test-hunter")
|
|
261
192
|
|
|
262
193
|
// Phase 3: Integration
|
|
263
|
-
Task("API
|
|
264
|
-
Task("Performance
|
|
265
|
-
Task("Security
|
|
194
|
+
Task("API Contracts", "Check breaking changes", "qe-api-contract-validator")
|
|
195
|
+
Task("Performance", "1000 user load test", "qe-performance-tester")
|
|
196
|
+
Task("Security", "SAST/DAST scans", "qe-security-scanner")
|
|
266
197
|
|
|
267
198
|
// Phase 4: Staging
|
|
268
|
-
Task("Chaos
|
|
269
|
-
Task("Visual
|
|
270
|
-
Task("
|
|
199
|
+
Task("Chaos", "Fault injection testing", "qe-chaos-engineer")
|
|
200
|
+
Task("Visual", "Visual regression", "qe-visual-tester")
|
|
201
|
+
Task("Readiness", "Deployment assessment", "qe-deployment-readiness")
|
|
271
202
|
|
|
272
203
|
// Phase 5: Production
|
|
273
|
-
Task("
|
|
274
|
-
Task("Quality Gate", "
|
|
204
|
+
Task("Intelligence", "Convert incidents", "qe-production-intelligence")
|
|
205
|
+
Task("Quality Gate", "Final validation", "qe-quality-gate")
|
|
275
206
|
```
|
|
276
207
|
|
|
277
208
|
---
|
|
278
209
|
|
|
279
|
-
## Adaptive Strategy
|
|
210
|
+
## Adaptive Strategy
|
|
280
211
|
|
|
281
|
-
|
|
212
|
+
### By Risk Level
|
|
213
|
+
| Risk | Strategy | Agents |
|
|
214
|
+
|------|----------|--------|
|
|
215
|
+
| Critical | All phases, manual gates | Full fleet |
|
|
216
|
+
| High | Automated gates, comprehensive | 10+ agents |
|
|
217
|
+
| Medium | Smart selection, risk-based | 5-8 agents |
|
|
218
|
+
| Low | Minimal regression, fast | 2-3 agents |
|
|
282
219
|
|
|
283
|
-
###
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
- **API**: Focus on contract-testing, api-testing-patterns, performance-testing
|
|
291
|
-
- **Web UI**: Focus on visual-testing-advanced, accessibility-testing, compatibility-testing
|
|
292
|
-
- **Mobile**: Focus on mobile-testing, performance-testing, compatibility-testing
|
|
293
|
-
- **Backend**: Focus on database-testing, performance-testing, security-testing
|
|
294
|
-
|
|
295
|
-
### Deployment Frequency
|
|
296
|
-
- **Continuous (hourly)**: Minimal regression, fast feedback, shift-right monitoring
|
|
297
|
-
- **Daily**: Smart test selection, parallel execution, automated gates
|
|
298
|
-
- **Weekly**: Comprehensive testing, exploratory sessions, manual validation
|
|
299
|
-
- **Monthly**: Full regression, compliance checks, extensive validation
|
|
220
|
+
### By Application Type
|
|
221
|
+
| Type | Focus Skills | Primary Agents |
|
|
222
|
+
|------|-------------|----------------|
|
|
223
|
+
| API | api-testing, contract, performance | api-contract-validator, performance-tester |
|
|
224
|
+
| Web UI | visual-testing, accessibility | visual-tester, accessibility |
|
|
225
|
+
| Mobile | mobile-testing, compatibility | performance-tester, visual-tester |
|
|
226
|
+
| Backend | database-testing, security | security-scanner, performance-tester |
|
|
300
227
|
|
|
301
228
|
---
|
|
302
229
|
|
|
303
|
-
##
|
|
304
|
-
|
|
305
|
-
### Memory Coordination
|
|
230
|
+
## Agent Coordination Hints
|
|
306
231
|
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
```
|
|
310
|
-
aqe/
|
|
311
|
-
├── pipeline/
|
|
312
|
-
│ ├── phase-results/ # Results from each phase
|
|
313
|
-
│ ├── quality-gates/ # Gate validation results
|
|
314
|
-
│ └── orchestration-plan/ # Selected skills and agents
|
|
315
|
-
├── test-plan/generated # Test plans from generators
|
|
316
|
-
├── coverage/gaps # Coverage analysis results
|
|
317
|
-
├── security/findings # Security scan results
|
|
318
|
-
└── performance/results # Performance test results
|
|
232
|
+
### Memory Namespace
|
|
319
233
|
```
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
Skill("chaos-engineering-resilience") // Apply chaos engineering
|
|
329
|
-
Skill("api-testing-patterns") // Implement API testing
|
|
234
|
+
aqe/pipeline/
|
|
235
|
+
├── phase-results/* - Results from each phase
|
|
236
|
+
├── quality-gates/* - Gate validation results
|
|
237
|
+
├── orchestration-plan/* - Selected skills and agents
|
|
238
|
+
├── test-plan/generated - Test plans
|
|
239
|
+
├── coverage/gaps - Coverage analysis
|
|
240
|
+
├── security/findings - Security results
|
|
241
|
+
└── performance/results - Performance data
|
|
330
242
|
```
|
|
331
243
|
|
|
332
|
-
###
|
|
333
|
-
|
|
334
|
-
Use the `qe-fleet-commander` for complex multi-agent orchestration:
|
|
335
|
-
|
|
336
|
-
```javascript
|
|
244
|
+
### Fleet Orchestration
|
|
245
|
+
```typescript
|
|
337
246
|
Task("Fleet Orchestration",
|
|
338
|
-
"Coordinate 10 agents across
|
|
247
|
+
"Coordinate 10 agents across phases: commit (2), build (3), test (3), staging (2)",
|
|
339
248
|
"qe-fleet-commander")
|
|
340
249
|
```
|
|
341
250
|
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
251
|
+
### Blackboard Events
|
|
252
|
+
| Event | Trigger | Subscribers |
|
|
253
|
+
|-------|---------|-------------|
|
|
254
|
+
| `phase:commit:complete` | Commit phase done | build agents |
|
|
255
|
+
| `coverage:gap:detected` | Gap found | test-generator |
|
|
256
|
+
| `security:finding:critical` | Vulnerability | quality-gate |
|
|
257
|
+
| `quality:gate:evaluated` | Gate decision | fleet-commander |
|
|
345
258
|
|
|
346
|
-
|
|
259
|
+
---
|
|
347
260
|
|
|
348
|
-
|
|
349
|
-
```json
|
|
350
|
-
{
|
|
351
|
-
"phase": "commit",
|
|
352
|
-
"gates": [
|
|
353
|
-
{ "metric": "unit_coverage", "threshold": 80, "blocking": true },
|
|
354
|
-
{ "metric": "static_analysis", "severity": "critical", "max_violations": 0 },
|
|
355
|
-
{ "metric": "code_review", "status": "approved", "blocking": true }
|
|
356
|
-
]
|
|
357
|
-
}
|
|
358
|
-
```
|
|
261
|
+
## Quality Gate Configuration
|
|
359
262
|
|
|
360
|
-
**Build Gate**:
|
|
361
263
|
```json
|
|
362
264
|
{
|
|
363
|
-
"
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
265
|
+
"commit": {
|
|
266
|
+
"gates": [
|
|
267
|
+
{ "metric": "unit_coverage", "threshold": 80, "blocking": true },
|
|
268
|
+
{ "metric": "static_analysis_critical", "max": 0, "blocking": true }
|
|
269
|
+
]
|
|
270
|
+
},
|
|
271
|
+
"build": {
|
|
272
|
+
"gates": [
|
|
273
|
+
{ "metric": "all_tests_passed", "threshold": 100, "blocking": true },
|
|
274
|
+
{ "metric": "mutation_score", "threshold": 70, "blocking": false }
|
|
275
|
+
]
|
|
276
|
+
},
|
|
277
|
+
"integration": {
|
|
278
|
+
"gates": [
|
|
279
|
+
{ "metric": "api_breaking_changes", "max": 0, "blocking": true },
|
|
280
|
+
{ "metric": "performance_p95_ms", "threshold": 200, "blocking": true },
|
|
281
|
+
{ "metric": "security_critical", "max": 0, "blocking": true }
|
|
282
|
+
]
|
|
283
|
+
}
|
|
369
284
|
}
|
|
370
285
|
```
|
|
371
286
|
|
|
372
|
-
**Integration Gate**:
|
|
373
|
-
```json
|
|
374
|
-
{
|
|
375
|
-
"phase": "integration",
|
|
376
|
-
"gates": [
|
|
377
|
-
{ "metric": "api_contracts", "breaking_changes": 0, "blocking": true },
|
|
378
|
-
{ "metric": "performance_p95", "threshold_ms": 200, "blocking": true },
|
|
379
|
-
{ "metric": "security_critical", "max_vulnerabilities": 0, "blocking": true }
|
|
380
|
-
]
|
|
381
|
-
}
|
|
382
|
-
```
|
|
383
|
-
|
|
384
|
-
### Gate Execution
|
|
385
|
-
|
|
386
|
-
```javascript
|
|
387
|
-
// Validate quality gates
|
|
388
|
-
Task("Quality Gate Validation",
|
|
389
|
-
"Validate all gates for integration phase: API contracts, performance SLAs, security scans",
|
|
390
|
-
"qe-quality-gate")
|
|
391
|
-
```
|
|
392
|
-
|
|
393
|
-
---
|
|
394
|
-
|
|
395
|
-
## Advanced Orchestration Patterns
|
|
396
|
-
|
|
397
|
-
### Pattern 1: Parallel Phase Execution
|
|
398
|
-
|
|
399
|
-
```javascript
|
|
400
|
-
// Execute multiple phases in parallel (for microservices)
|
|
401
|
-
Task("Service A Pipeline", "Run full pipeline for service A", "qe-fleet-commander")
|
|
402
|
-
Task("Service B Pipeline", "Run full pipeline for service B", "qe-fleet-commander")
|
|
403
|
-
Task("Service C Pipeline", "Run full pipeline for service C", "qe-fleet-commander")
|
|
404
|
-
```
|
|
405
|
-
|
|
406
|
-
### Pattern 2: Sequential with Gates
|
|
407
|
-
|
|
408
|
-
```javascript
|
|
409
|
-
// Sequential execution with quality gates
|
|
410
|
-
Task("Commit Phase", "Shift-left testing and validation", "qe-test-generator")
|
|
411
|
-
// Check gate before proceeding
|
|
412
|
-
Task("Build Phase", "Run tests if commit gate passes", "qe-test-executor")
|
|
413
|
-
// Check gate before proceeding
|
|
414
|
-
Task("Integration Phase", "API and performance tests if build gate passes", "qe-api-contract-validator")
|
|
415
|
-
```
|
|
416
|
-
|
|
417
|
-
### Pattern 3: Adaptive Selection
|
|
418
|
-
|
|
419
|
-
```javascript
|
|
420
|
-
// Adaptive strategy based on code changes
|
|
421
|
-
Task("Risk Analysis", "Analyze code changes and select minimal test suite", "qe-regression-risk-analyzer")
|
|
422
|
-
// Execute only selected tests
|
|
423
|
-
Task("Targeted Testing", "Run minimal test suite based on risk analysis", "qe-test-executor")
|
|
424
|
-
```
|
|
425
|
-
|
|
426
287
|
---
|
|
427
288
|
|
|
428
289
|
## Troubleshooting
|
|
429
290
|
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
```javascript
|
|
437
|
-
// Use batched test execution
|
|
438
|
-
Task("Batched Tests", "Run tests in batches of 10 with memory limits", "qe-test-executor")
|
|
439
|
-
```
|
|
440
|
-
|
|
441
|
-
See [Test Execution Policy](https://github.com/proffesor-for-testing/agentic-qe/blob/main/docs/policies/test-execution.md)
|
|
442
|
-
|
|
443
|
-
### Issue: Pipeline Takes Too Long
|
|
444
|
-
|
|
445
|
-
**Symptoms**: Pipeline exceeds time budget
|
|
446
|
-
**Cause**: Running comprehensive testing on every commit
|
|
447
|
-
**Solution**: Use smart test selection
|
|
448
|
-
|
|
449
|
-
```javascript
|
|
450
|
-
// Use regression risk analysis
|
|
451
|
-
Task("Smart Selection", "Select minimal test suite based on code changes", "qe-regression-risk-analyzer")
|
|
452
|
-
```
|
|
453
|
-
|
|
454
|
-
### Issue: Quality Gates Failing
|
|
455
|
-
|
|
456
|
-
**Symptoms**: Deployment blocked by quality gates
|
|
457
|
-
**Cause**: Thresholds too strict or tests not comprehensive
|
|
458
|
-
**Solution**: Review and adjust gates
|
|
459
|
-
|
|
460
|
-
```javascript
|
|
461
|
-
// Analyze quality metrics
|
|
462
|
-
Task("Quality Analysis", "Analyze trends and recommend threshold adjustments", "qe-quality-analyzer")
|
|
463
|
-
```
|
|
291
|
+
| Issue | Cause | Solution |
|
|
292
|
+
|-------|-------|----------|
|
|
293
|
+
| OOM during tests | Running all tests in parallel | Use batched execution |
|
|
294
|
+
| Pipeline too slow | Comprehensive testing every commit | Smart test selection |
|
|
295
|
+
| Gates always failing | Thresholds too strict | Analyze trends, adjust |
|
|
464
296
|
|
|
465
297
|
---
|
|
466
298
|
|
|
467
|
-
##
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
### Template 2: Monolith Pipeline
|
|
474
|
-
|
|
475
|
-
See [resources/workflows/monolith-pipeline.md](resources/workflows/monolith-pipeline.md)
|
|
476
|
-
|
|
477
|
-
### Template 3: Mobile App Pipeline
|
|
478
|
-
|
|
479
|
-
See [resources/workflows/mobile-pipeline.md](resources/workflows/mobile-pipeline.md)
|
|
299
|
+
## Related Skills
|
|
300
|
+
- [agentic-quality-engineering](../agentic-quality-engineering/) - Fleet coordination
|
|
301
|
+
- [shift-left-testing](../shift-left-testing/) - Early defect detection
|
|
302
|
+
- [shift-right-testing](../shift-right-testing/) - Production monitoring
|
|
303
|
+
- [chaos-engineering-resilience](../chaos-engineering-resilience/) - Resilience testing
|
|
480
304
|
|
|
481
305
|
---
|
|
482
306
|
|
|
483
|
-
##
|
|
484
|
-
|
|
485
|
-
### Skills Reference
|
|
486
|
-
- [All 37 QE Skills](https://github.com/proffesor-for-testing/agentic-qe/blob/main/docs/reference/skills.md)
|
|
487
|
-
- [Shift-Left Testing](../shift-left-testing/SKILL.md)
|
|
488
|
-
- [Shift-Right Testing](../shift-right-testing/SKILL.md)
|
|
489
|
-
- [Chaos Engineering](../chaos-engineering-resilience/SKILL.md)
|
|
490
|
-
|
|
491
|
-
### Agent Reference
|
|
492
|
-
- [All 18 QE Agents](https://github.com/proffesor-for-testing/agentic-qe/blob/main/docs/reference/agents.md)
|
|
493
|
-
- [Fleet Commander](https://github.com/proffesor-for-testing/agentic-qe/blob/main/docs/reference/agents.md#qe-fleet-commander)
|
|
307
|
+
## Remember
|
|
494
308
|
|
|
495
|
-
|
|
496
|
-
-
|
|
497
|
-
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
- [Continuous Testing in DevOps](https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment)
|
|
501
|
-
- [Shift-Left vs Shift-Right](https://www.dynatrace.com/news/blog/what-is-shift-left-and-shift-right/)
|
|
502
|
-
- [Quality Gates in CI/CD](https://docs.sonarqube.org/latest/user-guide/quality-gates/)
|
|
503
|
-
|
|
504
|
-
---
|
|
309
|
+
The CI/CD Pipeline QE Orchestrator provides:
|
|
310
|
+
- **Phase-based strategy** with optimal skill/agent selection
|
|
311
|
+
- **Quality gates** that block bad deployments
|
|
312
|
+
- **Adaptive strategy** based on risk and context
|
|
313
|
+
- **Full fleet coordination** through memory namespaces
|
|
505
314
|
|
|
506
|
-
**
|
|
507
|
-
**Category**: Quality Engineering Orchestration
|
|
508
|
-
**Difficulty**: Advanced
|
|
509
|
-
**Estimated Time**: 30-60 minutes for full pipeline setup
|
|
510
|
-
**Integrations**: All 37 QE skills, 18 QE agents, CI/CD platforms (GitHub Actions, Jenkins, GitLab CI)
|
|
315
|
+
**With Agents:** Use `qe-fleet-commander` for multi-agent orchestration. Coordinate through `aqe/*` memory namespace. Batch operations for efficiency.
|