agentic-qe 2.0.0 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/qx-partner.md +17 -4
- package/.claude/skills/accessibility-testing/SKILL.md +144 -692
- package/.claude/skills/agentic-quality-engineering/SKILL.md +176 -529
- package/.claude/skills/api-testing-patterns/SKILL.md +180 -560
- package/.claude/skills/brutal-honesty-review/SKILL.md +113 -603
- package/.claude/skills/bug-reporting-excellence/SKILL.md +116 -517
- package/.claude/skills/chaos-engineering-resilience/SKILL.md +127 -72
- package/.claude/skills/cicd-pipeline-qe-orchestrator/SKILL.md +209 -404
- package/.claude/skills/code-review-quality/SKILL.md +158 -608
- package/.claude/skills/compatibility-testing/SKILL.md +148 -38
- package/.claude/skills/compliance-testing/SKILL.md +132 -63
- package/.claude/skills/consultancy-practices/SKILL.md +114 -446
- package/.claude/skills/context-driven-testing/SKILL.md +117 -381
- package/.claude/skills/contract-testing/SKILL.md +176 -141
- package/.claude/skills/database-testing/SKILL.md +137 -130
- package/.claude/skills/exploratory-testing-advanced/SKILL.md +160 -629
- package/.claude/skills/holistic-testing-pact/SKILL.md +140 -188
- package/.claude/skills/localization-testing/SKILL.md +145 -33
- package/.claude/skills/mobile-testing/SKILL.md +132 -448
- package/.claude/skills/mutation-testing/SKILL.md +147 -41
- package/.claude/skills/performance-testing/SKILL.md +200 -546
- package/.claude/skills/quality-metrics/SKILL.md +164 -519
- package/.claude/skills/refactoring-patterns/SKILL.md +132 -699
- package/.claude/skills/regression-testing/SKILL.md +120 -926
- package/.claude/skills/risk-based-testing/SKILL.md +157 -660
- package/.claude/skills/security-testing/SKILL.md +199 -538
- package/.claude/skills/sherlock-review/SKILL.md +163 -699
- package/.claude/skills/shift-left-testing/SKILL.md +161 -465
- package/.claude/skills/shift-right-testing/SKILL.md +161 -519
- package/.claude/skills/six-thinking-hats/SKILL.md +175 -1110
- package/.claude/skills/skills-manifest.json +71 -20
- package/.claude/skills/tdd-london-chicago/SKILL.md +131 -448
- package/.claude/skills/technical-writing/SKILL.md +103 -154
- package/.claude/skills/test-automation-strategy/SKILL.md +166 -772
- package/.claude/skills/test-data-management/SKILL.md +126 -910
- package/.claude/skills/test-design-techniques/SKILL.md +179 -89
- package/.claude/skills/test-environment-management/SKILL.md +136 -91
- package/.claude/skills/test-reporting-analytics/SKILL.md +169 -92
- package/.claude/skills/testability-scoring/SKILL.md +172 -538
- package/.claude/skills/testability-scoring/scripts/generate-html-report.js +0 -0
- package/.claude/skills/visual-testing-advanced/SKILL.md +155 -78
- package/.claude/skills/xp-practices/SKILL.md +151 -587
- package/CHANGELOG.md +48 -0
- package/README.md +23 -16
- package/dist/agents/QXPartnerAgent.d.ts +8 -1
- package/dist/agents/QXPartnerAgent.d.ts.map +1 -1
- package/dist/agents/QXPartnerAgent.js +1174 -112
- package/dist/agents/QXPartnerAgent.js.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.d.ts.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.js +34 -31
- package/dist/agents/lifecycle/AgentLifecycleManager.js.map +1 -1
- package/dist/cli/commands/init-claude-md-template.d.ts.map +1 -1
- package/dist/cli/commands/init-claude-md-template.js +14 -0
- package/dist/cli/commands/init-claude-md-template.js.map +1 -1
- package/dist/core/SwarmCoordinator.d.ts +180 -0
- package/dist/core/SwarmCoordinator.d.ts.map +1 -0
- package/dist/core/SwarmCoordinator.js +473 -0
- package/dist/core/SwarmCoordinator.js.map +1 -0
- package/dist/core/metrics/MetricsAggregator.d.ts +228 -0
- package/dist/core/metrics/MetricsAggregator.d.ts.map +1 -0
- package/dist/core/metrics/MetricsAggregator.js +482 -0
- package/dist/core/metrics/MetricsAggregator.js.map +1 -0
- package/dist/core/metrics/index.d.ts +5 -0
- package/dist/core/metrics/index.d.ts.map +1 -0
- package/dist/core/metrics/index.js +11 -0
- package/dist/core/metrics/index.js.map +1 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts +5 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts.map +1 -1
- package/dist/core/optimization/SwarmOptimizer.js +17 -0
- package/dist/core/optimization/SwarmOptimizer.js.map +1 -1
- package/dist/core/orchestration/AdaptiveScheduler.d.ts +190 -0
- package/dist/core/orchestration/AdaptiveScheduler.d.ts.map +1 -0
- package/dist/core/orchestration/AdaptiveScheduler.js +460 -0
- package/dist/core/orchestration/AdaptiveScheduler.js.map +1 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts +13 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts.map +1 -1
- package/dist/core/orchestration/WorkflowOrchestrator.js +32 -0
- package/dist/core/orchestration/WorkflowOrchestrator.js.map +1 -1
- package/dist/core/recovery/CircuitBreaker.d.ts +176 -0
- package/dist/core/recovery/CircuitBreaker.d.ts.map +1 -0
- package/dist/core/recovery/CircuitBreaker.js +382 -0
- package/dist/core/recovery/CircuitBreaker.js.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts +186 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.js +476 -0
- package/dist/core/recovery/RecoveryOrchestrator.js.map +1 -0
- package/dist/core/recovery/RetryStrategy.d.ts +127 -0
- package/dist/core/recovery/RetryStrategy.d.ts.map +1 -0
- package/dist/core/recovery/RetryStrategy.js +314 -0
- package/dist/core/recovery/RetryStrategy.js.map +1 -0
- package/dist/core/recovery/index.d.ts +8 -0
- package/dist/core/recovery/index.d.ts.map +1 -0
- package/dist/core/recovery/index.js +27 -0
- package/dist/core/recovery/index.js.map +1 -0
- package/dist/core/skills/DependencyResolver.d.ts +99 -0
- package/dist/core/skills/DependencyResolver.d.ts.map +1 -0
- package/dist/core/skills/DependencyResolver.js +260 -0
- package/dist/core/skills/DependencyResolver.js.map +1 -0
- package/dist/core/skills/ManifestGenerator.d.ts +114 -0
- package/dist/core/skills/ManifestGenerator.d.ts.map +1 -0
- package/dist/core/skills/ManifestGenerator.js +449 -0
- package/dist/core/skills/ManifestGenerator.js.map +1 -0
- package/dist/core/skills/index.d.ts +9 -0
- package/dist/core/skills/index.d.ts.map +1 -0
- package/dist/core/skills/index.js +24 -0
- package/dist/core/skills/index.js.map +1 -0
- package/dist/mcp/server.d.ts +9 -9
- package/dist/mcp/server.d.ts.map +1 -1
- package/dist/mcp/server.js +1 -2
- package/dist/mcp/server.js.map +1 -1
- package/dist/types/qx.d.ts +39 -7
- package/dist/types/qx.d.ts.map +1 -1
- package/dist/types/qx.js.map +1 -1
- package/dist/visualization/api/RestEndpoints.js +1 -1
- package/dist/visualization/api/RestEndpoints.js.map +1 -1
- package/package.json +13 -55
|
@@ -1,416 +1,118 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
3
|
-
description: "AI-powered testability assessment using 10 principles of intrinsic testability with Playwright. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity,
|
|
2
|
+
name: testability-scoring
|
|
3
|
+
description: "AI-powered testability assessment using 10 principles of intrinsic testability with Playwright. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports."
|
|
4
|
+
category: testing-methodologies
|
|
5
|
+
priority: high
|
|
6
|
+
tokenEstimate: 1100
|
|
7
|
+
agents: [qe-quality-analyzer, qx-partner, qe-visual-tester]
|
|
8
|
+
implementation_status: optimized
|
|
9
|
+
optimization_version: 2.1
|
|
10
|
+
last_optimized: 2025-12-03
|
|
11
|
+
dependencies: []
|
|
12
|
+
quick_reference_card: true
|
|
13
|
+
tags: [testability, scoring, playwright, assessment, 10-principles, intrinsic-testability, james-bach, michael-bolton]
|
|
14
|
+
contributor: "@fndlalit"
|
|
4
15
|
---
|
|
5
16
|
|
|
6
17
|
# Testability Scoring
|
|
7
18
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
- **Quantitative Scoring**: 0-100 scale with letter grades (A-F) for each principle
|
|
16
|
-
- **AI-Enhanced Insights**: Intelligent recommendations powered by advanced analysis
|
|
17
|
-
- **Interactive HTML Reports**: Beautiful visual reports with Chart.js radar visualizations
|
|
18
|
-
- **Multi-User Comparison**: Side-by-side testability analysis across user types
|
|
19
|
-
- **Historical Tracking**: Progress measurement and improvement validation
|
|
20
|
-
- **Automated Analysis**: Hands-free assessment with error-resilient execution
|
|
21
|
-
|
|
22
|
-
## The 10 Principles of Intrinsic Testability
|
|
23
|
-
|
|
24
|
-
### 1. **Observability** (15% weight)
|
|
25
|
-
Can we see what's happening in the system?
|
|
26
|
-
- State visibility and capture
|
|
27
|
-
- Event logging and monitoring
|
|
28
|
-
- Network request tracking
|
|
29
|
-
- Error visibility
|
|
30
|
-
- Visual state inspection
|
|
31
|
-
|
|
32
|
-
### 2. **Controllability** (15% weight)
|
|
33
|
-
Can we control the application precisely?
|
|
34
|
-
- Direct input control
|
|
35
|
-
- State manipulation
|
|
36
|
-
- API accessibility
|
|
37
|
-
- Test data injection
|
|
38
|
-
- Deterministic behavior
|
|
39
|
-
|
|
40
|
-
### 3. **Algorithmic Simplicity** (10% weight)
|
|
41
|
-
Are behaviors simple and predictable?
|
|
42
|
-
- Clear input-output relationships
|
|
43
|
-
- Operation complexity
|
|
44
|
-
- Interaction patterns
|
|
45
|
-
- Behavior predictability
|
|
46
|
-
|
|
47
|
-
### 4. **Algorithmic Transparency** (10% weight)
|
|
48
|
-
Can we understand what the system does?
|
|
49
|
-
- Behavior visibility
|
|
50
|
-
- Process understanding
|
|
51
|
-
- Black box reduction
|
|
52
|
-
- Code readability indicators
|
|
53
|
-
|
|
54
|
-
### 5. **Algorithmic Stability** (10% weight)
|
|
55
|
-
Does behavior remain consistent?
|
|
56
|
-
- Change resilience
|
|
57
|
-
- Test maintainability
|
|
58
|
-
- Behavior consistency
|
|
59
|
-
- Version stability
|
|
60
|
-
|
|
61
|
-
### 6. **Explainability** (10% weight)
|
|
62
|
-
Can users and developers understand the interface?
|
|
63
|
-
- Documentation quality
|
|
64
|
-
- Code clarity
|
|
65
|
-
- Semantic structure
|
|
66
|
-
- Help text and guidance
|
|
67
|
-
- Error message clarity
|
|
68
|
-
|
|
69
|
-
### 7. **Unbugginess** (10% weight)
|
|
70
|
-
How error-free is the application?
|
|
71
|
-
- Console error tracking
|
|
72
|
-
- Page error monitoring
|
|
73
|
-
- Warning analysis
|
|
74
|
-
- Runtime stability
|
|
75
|
-
|
|
76
|
-
### 8. **Smallness** (10% weight)
|
|
77
|
-
Are components appropriately sized?
|
|
78
|
-
- Page complexity
|
|
79
|
-
- Element count
|
|
80
|
-
- Script/style bloat
|
|
81
|
-
- Component granularity
|
|
82
|
-
|
|
83
|
-
### 9. **Decomposability** (5% weight)
|
|
84
|
-
Can we test parts in isolation?
|
|
85
|
-
- Component separation
|
|
86
|
-
- Isolated testing capability
|
|
87
|
-
- Modular design
|
|
88
|
-
- Feature independence
|
|
89
|
-
|
|
90
|
-
### 10. **Similarity** (5% weight)
|
|
91
|
-
How familiar is the technology stack?
|
|
92
|
-
- Standard frameworks
|
|
93
|
-
- Common patterns
|
|
94
|
-
- Known platforms
|
|
95
|
-
- Familiar conventions
|
|
96
|
-
|
|
97
|
-
## Quick Start
|
|
98
|
-
|
|
99
|
-
### Method 1: Shell Script (Recommended)
|
|
100
|
-
```bash
|
|
101
|
-
# Run assessment on any URL (runtime override)
|
|
102
|
-
TEST_URL='https://example.com/' npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
|
|
103
|
-
|
|
104
|
-
# Or use the shell script wrapper
|
|
105
|
-
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
|
|
106
|
-
|
|
107
|
-
# With specific browser
|
|
108
|
-
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/ firefox
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
### Method 2: Update Config File (Persistent)
|
|
112
|
-
```bash
|
|
113
|
-
# Update config for repeated runs
|
|
114
|
-
echo 'module.exports = { baseURL: "https://example.com/", ... };' > tests/testability-scoring/config.js
|
|
115
|
-
|
|
116
|
-
# Run assessment
|
|
117
|
-
npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
### Method 3: Generate HTML Report Only
|
|
121
|
-
```bash
|
|
122
|
-
# From existing JSON results
|
|
123
|
-
AUTO_OPEN=false node .claude/skills/testability-scoring/scripts/generate-html-report.js tests/reports/testability-results-*.json
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
## Complete Usage Guide
|
|
127
|
-
|
|
128
|
-
### Assessment Workflow
|
|
129
|
-
|
|
130
|
-
1. **Run Assessment**: Execute tests against target URL
|
|
131
|
-
2. **JSON Generation**: Automated results saved to `tests/reports/`
|
|
132
|
-
3. **HTML Report**: Interactive report with charts automatically generated
|
|
133
|
-
4. **Browser Opening**: Report opens automatically (configurable)
|
|
134
|
-
5. **Analysis**: Review scores, charts, and recommendations
|
|
135
|
-
|
|
136
|
-
### Configuration
|
|
19
|
+
<default_to_action>
|
|
20
|
+
When assessing testability:
|
|
21
|
+
1. RUN assessment against target URL
|
|
22
|
+
2. ANALYZE all 10 principles automatically
|
|
23
|
+
3. GENERATE HTML report with radar chart
|
|
24
|
+
4. PRIORITIZE improvements by impact/effort
|
|
25
|
+
5. INTEGRATE with QX Partner for holistic view
|
|
137
26
|
|
|
138
|
-
|
|
139
|
-
Use the `TEST_URL` environment variable for one-time assessments:
|
|
27
|
+
**Quick Assessment:**
|
|
140
28
|
```bash
|
|
29
|
+
# Run assessment on any URL
|
|
141
30
|
TEST_URL='https://example.com/' npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
|
|
142
|
-
```
|
|
143
|
-
|
|
144
|
-
#### Config File Location
|
|
145
|
-
`tests/testability-scoring/config.js`
|
|
146
|
-
|
|
147
|
-
```javascript
|
|
148
|
-
module.exports = {
|
|
149
|
-
// Runtime URL override via TEST_URL environment variable
|
|
150
|
-
baseURL: process.env.TEST_URL || 'https://your-application.com/',
|
|
151
|
-
timeout: 60000, // 60 second timeout
|
|
152
|
-
networkTimeout: 15000 // 15 second network idle timeout
|
|
153
|
-
};
|
|
154
|
-
```
|
|
155
|
-
|
|
156
|
-
#### Playwright Configuration
|
|
157
|
-
Located in `playwright.config.js`:
|
|
158
|
-
- Timeout: 60s per test
|
|
159
|
-
- Workers: 1 (serial execution)
|
|
160
|
-
- Launch args: `--no-sandbox`, `--disable-dev-shm-usage`
|
|
161
|
-
- Viewport: 1280x720
|
|
162
|
-
- ignoreHTTPSErrors: true
|
|
163
31
|
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
```bash
|
|
167
|
-
# Disable automatic browser opening
|
|
168
|
-
AUTO_OPEN=false .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
|
|
169
|
-
|
|
170
|
-
# With custom timeout
|
|
171
|
-
timeout 180 .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
|
|
172
|
-
```
|
|
173
|
-
|
|
174
|
-
## Assessment Script Architecture
|
|
175
|
-
|
|
176
|
-
### Error Handling Strategy
|
|
177
|
-
|
|
178
|
-
The assessment script uses **progressive fallback** strategy:
|
|
179
|
-
|
|
180
|
-
```javascript
|
|
181
|
-
// Navigation with multi-level fallback
|
|
182
|
-
async function navigateToPage(page) {
|
|
183
|
-
try {
|
|
184
|
-
// Level 1: domcontentloaded
|
|
185
|
-
await page.goto(baseURL, {
|
|
186
|
-
waitUntil: 'domcontentloaded',
|
|
187
|
-
timeout: 45000
|
|
188
|
-
});
|
|
189
|
-
|
|
190
|
-
// Level 2: networkidle with timeout
|
|
191
|
-
await page.waitForLoadState('networkidle', {
|
|
192
|
-
timeout: 15000
|
|
193
|
-
}).catch(() => console.log('[NAV] Continuing without networkidle'));
|
|
194
|
-
|
|
195
|
-
return true;
|
|
196
|
-
} catch (error) {
|
|
197
|
-
// Level 3: commit fallback
|
|
198
|
-
await page.goto(baseURL, {
|
|
199
|
-
waitUntil: 'commit',
|
|
200
|
-
timeout: 45000
|
|
201
|
-
});
|
|
202
|
-
return true;
|
|
203
|
-
}
|
|
204
|
-
}
|
|
32
|
+
# Or use shell script wrapper
|
|
33
|
+
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
|
|
205
34
|
```
|
|
206
35
|
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
36
|
+
**The 10 Principles at a Glance:**
|
|
37
|
+
| Principle | Weight | Key Question |
|
|
38
|
+
|-----------|--------|--------------|
|
|
39
|
+
| **Observability** | 15% | Can we see what's happening? |
|
|
40
|
+
| **Controllability** | 15% | Can we control the application? |
|
|
41
|
+
| **Algorithmic Simplicity** | 10% | Are behaviors predictable? |
|
|
42
|
+
| **Algorithmic Transparency** | 10% | Can we understand what it does? |
|
|
43
|
+
| **Algorithmic Stability** | 10% | Does behavior remain consistent? |
|
|
44
|
+
| **Explainability** | 10% | Is the interface understandable? |
|
|
45
|
+
| **Unbugginess** | 10% | How error-free is it? |
|
|
46
|
+
| **Smallness** | 10% | Are components appropriately sized? |
|
|
47
|
+
| **Decomposability** | 5% | Can we test parts in isolation? |
|
|
48
|
+
| **Similarity** | 5% | Is the tech stack familiar? |
|
|
49
|
+
|
|
50
|
+
**Grade Scale:**
|
|
51
|
+
- **A (90-100)**: Excellent testability
|
|
52
|
+
- **B (80-89)**: Good testability
|
|
53
|
+
- **C (70-79)**: Adequate testability
|
|
54
|
+
- **D (60-69)**: Below average
|
|
55
|
+
- **F (0-59)**: Poor testability
|
|
56
|
+
</default_to_action>
|
|
57
|
+
|
|
58
|
+
## Quick Reference Card
|
|
59
|
+
|
|
60
|
+
### Running Assessments
|
|
61
|
+
|
|
62
|
+
| Method | Command | When to Use |
|
|
63
|
+
|--------|---------|-------------|
|
|
64
|
+
| Shell Script | `./scripts/run-assessment.sh URL` | One-time assessment |
|
|
65
|
+
| ENV Override | `TEST_URL='URL' npx playwright test...` | CI/CD integration |
|
|
66
|
+
| Config File | Update `tests/testability-scoring/config.js` | Repeated runs |
|
|
67
|
+
|
|
68
|
+
### Principle Details
|
|
69
|
+
|
|
70
|
+
#### High Weight (15% each)
|
|
71
|
+
| Principle | Measures | Indicators |
|
|
72
|
+
|-----------|----------|------------|
|
|
73
|
+
| **Observability** | State visibility, logging, monitoring | Console output, network tracking, error visibility |
|
|
74
|
+
| **Controllability** | Input control, state manipulation | API access, test data injection, determinism |
|
|
75
|
+
|
|
76
|
+
#### Medium Weight (10% each)
|
|
77
|
+
| Principle | Measures | Indicators |
|
|
78
|
+
|-----------|----------|------------|
|
|
79
|
+
| **Simplicity** | Predictable behavior | Clear I/O relationships, low complexity |
|
|
80
|
+
| **Transparency** | Understanding what system does | Visible processes, readable code |
|
|
81
|
+
| **Stability** | Consistent behavior | Change resilience, maintainability |
|
|
82
|
+
| **Explainability** | Interface understanding | Good docs, semantic structure, help text |
|
|
83
|
+
| **Unbugginess** | Error-free operation | Console errors, warnings, runtime issues |
|
|
84
|
+
| **Smallness** | Component size | Element count, script bloat, page complexity |
|
|
85
|
+
|
|
86
|
+
#### Low Weight (5% each)
|
|
87
|
+
| Principle | Measures | Indicators |
|
|
88
|
+
|-----------|----------|------------|
|
|
89
|
+
| **Decomposability** | Isolation testing | Component separation, modular design |
|
|
90
|
+
| **Similarity** | Technology familiarity | Standard frameworks, known patterns |
|
|
226
91
|
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
Each principle test follows this pattern:
|
|
230
|
-
```javascript
|
|
231
|
-
test('1. Observability Assessment', async ({ page }) => {
|
|
232
|
-
try {
|
|
233
|
-
const loaded = await navigateToPage(page);
|
|
234
|
-
if (!loaded) throw new Error('Failed to load page');
|
|
235
|
-
|
|
236
|
-
// Principle-specific analysis
|
|
237
|
-
let score = 0;
|
|
238
|
-
// ... scoring logic ...
|
|
239
|
-
|
|
240
|
-
testabilityScores.principles.observability = {
|
|
241
|
-
score: Math.min(score, 100),
|
|
242
|
-
grade: getLetterGrade(score),
|
|
243
|
-
weight: config.weights.observability
|
|
244
|
-
};
|
|
245
|
-
|
|
246
|
-
// Generate recommendations if score is low
|
|
247
|
-
if (score < 70) {
|
|
248
|
-
testabilityScores.recommendations.push({
|
|
249
|
-
principle: 'Observability',
|
|
250
|
-
severity: 'medium',
|
|
251
|
-
recommendation: 'Implement detailed event logging...',
|
|
252
|
-
impact: 15,
|
|
253
|
-
effort: 'Low (4-6 hours)'
|
|
254
|
-
});
|
|
255
|
-
}
|
|
256
|
-
} catch (error) {
|
|
257
|
-
console.error('Observability assessment failed:', error.message);
|
|
258
|
-
// Fallback to default score
|
|
259
|
-
testabilityScores.principles.observability = {
|
|
260
|
-
score: 50,
|
|
261
|
-
grade: 'F',
|
|
262
|
-
weight: config.weights.observability
|
|
263
|
-
};
|
|
264
|
-
}
|
|
265
|
-
});
|
|
266
|
-
```
|
|
92
|
+
---
|
|
267
93
|
|
|
268
|
-
##
|
|
269
|
-
|
|
270
|
-
### Interactive Elements
|
|
271
|
-
|
|
272
|
-
- **Radar Chart**: Visual representation of all 10 principles
|
|
273
|
-
- **Score Cards**: Color-coded principle scores with grades
|
|
274
|
-
- **Detailed Breakdowns**: Expandable sections for each principle
|
|
275
|
-
- **Recommendations**: Prioritized improvement suggestions
|
|
276
|
-
- **Metadata**: URL, browser, version, duration, timestamp
|
|
277
|
-
|
|
278
|
-
### Chart.js Integration
|
|
279
|
-
|
|
280
|
-
Reports use Chart.js for beautiful visualizations:
|
|
281
|
-
```javascript
|
|
282
|
-
new Chart(ctx, {
|
|
283
|
-
type: 'radar',
|
|
284
|
-
data: {
|
|
285
|
-
labels: [
|
|
286
|
-
'Observability', 'Controllability', 'Simplicity',
|
|
287
|
-
'Transparency', 'Explainability', 'Similarity',
|
|
288
|
-
'Stability', 'Unbugginess', 'Smallness', 'Decomposability'
|
|
289
|
-
],
|
|
290
|
-
datasets: [{
|
|
291
|
-
label: 'Testability Scores',
|
|
292
|
-
data: [92, 20, 75, 60, 75, 85, 70, 85, 95, 70],
|
|
293
|
-
backgroundColor: 'rgba(74, 144, 226, 0.2)',
|
|
294
|
-
borderColor: 'rgba(74, 144, 226, 1)',
|
|
295
|
-
pointBackgroundColor: 'rgba(74, 144, 226, 1)'
|
|
296
|
-
}]
|
|
297
|
-
}
|
|
298
|
-
});
|
|
299
|
-
```
|
|
94
|
+
## Assessment Workflow
|
|
300
95
|
|
|
301
|
-
### Report Sections
|
|
302
|
-
|
|
303
|
-
1. **Executive Summary**: Overall score, grade, and quick stats
|
|
304
|
-
2. **Principle Scores**: Detailed breakdown with grades
|
|
305
|
-
3. **Visual Analysis**: Radar chart and score distribution
|
|
306
|
-
4. **Recommendations**: Actionable improvements prioritized by impact
|
|
307
|
-
5. **Metadata**: Assessment details and context
|
|
308
|
-
|
|
309
|
-
## Scoring Methodology
|
|
310
|
-
|
|
311
|
-
### Grade Scale
|
|
312
|
-
- **90-100 (A)**: Excellent testability
|
|
313
|
-
- **80-89 (B)**: Good testability
|
|
314
|
-
- **70-79 (C)**: Adequate testability
|
|
315
|
-
- **60-69 (D)**: Below average testability
|
|
316
|
-
- **0-59 (F)**: Poor testability
|
|
317
|
-
|
|
318
|
-
### Weighted Calculation
|
|
319
|
-
|
|
320
|
-
Overall score uses weighted average:
|
|
321
|
-
```javascript
|
|
322
|
-
function calculateOverallScore(principles) {
|
|
323
|
-
let totalWeightedScore = 0;
|
|
324
|
-
let totalWeight = 0;
|
|
325
|
-
|
|
326
|
-
for (const [principle, data] of Object.entries(principles)) {
|
|
327
|
-
totalWeightedScore += data.score * data.weight;
|
|
328
|
-
totalWeight += data.weight;
|
|
329
|
-
}
|
|
330
|
-
|
|
331
|
-
return Math.round(totalWeightedScore / totalWeight);
|
|
332
|
-
}
|
|
333
96
|
```
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
if (score >= 60) return 'D';
|
|
342
|
-
return 'F';
|
|
343
|
-
}
|
|
97
|
+
1. Navigate to URL → 2. Collect Metrics → 3. Score Principles
|
|
98
|
+
↓
|
|
99
|
+
4. Generate JSON ← 5. Calculate Grades ← 6. Apply Weights
|
|
100
|
+
↓
|
|
101
|
+
7. Generate HTML Report with Radar Chart
|
|
102
|
+
↓
|
|
103
|
+
8. Open in Browser (auto-opens)
|
|
344
104
|
```
|
|
345
105
|
|
|
346
|
-
|
|
106
|
+
### Output Files
|
|
347
107
|
|
|
348
108
|
```
|
|
349
|
-
.claude/skills/testability-scoring/
|
|
350
|
-
├── SKILL.md # This file
|
|
351
|
-
├── README.md # Quick reference
|
|
352
|
-
├── scripts/
|
|
353
|
-
│ ├── run-assessment.sh # Main assessment runner
|
|
354
|
-
│ └── generate-html-report.js # HTML report generator
|
|
355
|
-
├── resources/
|
|
356
|
-
│ └── templates/
|
|
357
|
-
│ └── config.template.js # Config file template
|
|
358
|
-
├── examples/
|
|
359
|
-
│ ├── sample-assessment.json # Example JSON output
|
|
360
|
-
│ └── sample-report.html # Example HTML report
|
|
361
|
-
└── docs/
|
|
362
|
-
├── ARCHITECTURE.md # System architecture
|
|
363
|
-
├── SCORING-METHODOLOGY.md # Detailed scoring guide
|
|
364
|
-
└── TROUBLESHOOTING.md # Common issues and fixes
|
|
365
|
-
|
|
366
|
-
tests/testability-scoring/
|
|
367
|
-
├── testability-scoring.spec.js # Main test file
|
|
368
|
-
├── config.js # Runtime configuration
|
|
369
|
-
└── playwright.config.js # (optional) Test-specific config
|
|
370
|
-
|
|
371
109
|
tests/reports/
|
|
372
|
-
├── testability-results-<timestamp>.json #
|
|
373
|
-
├── testability-report-<timestamp>.html #
|
|
374
|
-
└── latest.json # Symlink
|
|
110
|
+
├── testability-results-<timestamp>.json # Raw data
|
|
111
|
+
├── testability-report-<timestamp>.html # Visual report
|
|
112
|
+
└── latest.json # Symlink
|
|
375
113
|
```
|
|
376
114
|
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
### Example 1: High Testability Site
|
|
380
|
-
```bash
|
|
381
|
-
$ .claude/skills/testability-scoring/scripts/run-assessment.sh https://talesoftesting.com/
|
|
382
|
-
|
|
383
|
-
Results:
|
|
384
|
-
Overall Score: 71/100 (C)
|
|
385
|
-
- Observability: 92 (A) ⭐
|
|
386
|
-
- Controllability: 20 (F) ⚠️
|
|
387
|
-
- Algorithmic Simplicity: 75 (C)
|
|
388
|
-
- Algorithmic Transparency: 60 (D)
|
|
389
|
-
- Explainability: 75 (C)
|
|
390
|
-
- Similarity: 85 (B)
|
|
391
|
-
- Algorithmic Stability: 70 (C)
|
|
392
|
-
- Unbugginess: 85 (B)
|
|
393
|
-
- Smallness: 95 (A) ⭐
|
|
394
|
-
- Decomposability: 70 (C)
|
|
395
|
-
```
|
|
396
|
-
|
|
397
|
-
### Example 2: E-Commerce Site
|
|
398
|
-
```bash
|
|
399
|
-
$ .claude/skills/testability-scoring/scripts/run-assessment.sh https://www.saucedemo.com/
|
|
400
|
-
|
|
401
|
-
Results:
|
|
402
|
-
Overall Score: 69/100 (D)
|
|
403
|
-
- Observability: 92 (A) ⭐
|
|
404
|
-
- Controllability: 20 (F) ⚠️ Critical Issue
|
|
405
|
-
- Smallness: 100 (A) ⭐ Perfect
|
|
406
|
-
- Similarity: 85 (B)
|
|
407
|
-
- Unbugginess: 85 (B)
|
|
408
|
-
- Algorithmic Simplicity: 75 (C)
|
|
409
|
-
- Algorithmic Stability: 70 (C)
|
|
410
|
-
- Algorithmic Transparency: 60 (D)
|
|
411
|
-
- Explainability: 60 (D)
|
|
412
|
-
- Decomposability: 50 (F)
|
|
413
|
-
```
|
|
115
|
+
---
|
|
414
116
|
|
|
415
117
|
## Integration Examples
|
|
416
118
|
|
|
@@ -420,7 +122,7 @@ Overall Score: 69/100 (D)
|
|
|
420
122
|
- name: Testability Assessment
|
|
421
123
|
run: |
|
|
422
124
|
timeout 180 .claude/skills/testability-scoring/scripts/run-assessment.sh ${{ env.APP_URL }}
|
|
423
|
-
|
|
125
|
+
|
|
424
126
|
- name: Upload Reports
|
|
425
127
|
uses: actions/upload-artifact@v3
|
|
426
128
|
with:
|
|
@@ -428,154 +130,94 @@ Overall Score: 69/100 (D)
|
|
|
428
130
|
path: tests/reports/testability-*.html
|
|
429
131
|
```
|
|
430
132
|
|
|
431
|
-
###
|
|
432
|
-
```
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
let output = '';
|
|
440
|
-
proc.stdout.on('data', data => output += data);
|
|
441
|
-
|
|
442
|
-
proc.on('close', code => {
|
|
443
|
-
if (code === 0) {
|
|
444
|
-
resolve(output);
|
|
445
|
-
} else {
|
|
446
|
-
reject(new Error(`Assessment failed with code ${code}`));
|
|
447
|
-
}
|
|
448
|
-
});
|
|
449
|
-
});
|
|
450
|
-
}
|
|
451
|
-
|
|
452
|
-
// Usage
|
|
453
|
-
const results = await runTestabilityAssessment('https://example.com/');
|
|
454
|
-
console.log(results);
|
|
455
|
-
```
|
|
456
|
-
|
|
457
|
-
## Best Practices
|
|
133
|
+
### QX Partner Integration
|
|
134
|
+
```typescript
|
|
135
|
+
// Combine testability with QX analysis
|
|
136
|
+
const qxAnalysis = await Task("QX Analysis", {
|
|
137
|
+
target: 'https://example.com',
|
|
138
|
+
integrateTestability: true
|
|
139
|
+
}, "qx-partner");
|
|
458
140
|
|
|
459
|
-
|
|
460
|
-
-
|
|
461
|
-
-
|
|
462
|
-
-
|
|
463
|
-
|
|
464
|
-
### 2. Focus Areas
|
|
465
|
-
- Prioritize principles with lowest scores
|
|
466
|
-
- Address "F" grades first (scores < 60)
|
|
467
|
-
- Target quick wins (low effort, high impact)
|
|
141
|
+
// Returns combined insights:
|
|
142
|
+
// - QX Score: 78/100
|
|
143
|
+
// - Testability Integration: Observability 72/100
|
|
144
|
+
// - Combined Insight: Low observability may mask UX issues
|
|
145
|
+
```
|
|
468
146
|
|
|
469
|
-
###
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
- Track progress over time
|
|
147
|
+
### Programmatic Usage
|
|
148
|
+
```typescript
|
|
149
|
+
import { runTestabilityAssessment } from './testability';
|
|
473
150
|
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
151
|
+
const results = await runTestabilityAssessment('https://example.com');
|
|
152
|
+
console.log(`Overall: ${results.overallScore}/100 (${results.grade})`);
|
|
153
|
+
console.log('Recommendations:', results.recommendations);
|
|
154
|
+
```
|
|
478
155
|
|
|
479
|
-
|
|
480
|
-
- Test with different user personas
|
|
481
|
-
- Compare accessibility across roles
|
|
482
|
-
- Identify role-specific issues
|
|
156
|
+
---
|
|
483
157
|
|
|
484
|
-
##
|
|
158
|
+
## Agent Integration
|
|
485
159
|
|
|
486
|
-
|
|
160
|
+
```typescript
|
|
161
|
+
// Run testability assessment
|
|
162
|
+
const assessment = await Task("Testability Assessment", {
|
|
163
|
+
url: 'https://example.com',
|
|
164
|
+
generateReport: true,
|
|
165
|
+
openBrowser: true
|
|
166
|
+
}, "qe-quality-analyzer");
|
|
487
167
|
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
168
|
+
// Use with QX Partner for holistic analysis
|
|
169
|
+
const qxReport = await Task("Full QX Analysis", {
|
|
170
|
+
target: 'https://example.com',
|
|
171
|
+
integrateTestability: true,
|
|
172
|
+
detectOracleProblems: true
|
|
173
|
+
}, "qx-partner");
|
|
492
174
|
```
|
|
493
175
|
|
|
494
|
-
|
|
495
|
-
- Check for JavaScript errors in console
|
|
496
|
-
- Verify network connectivity
|
|
497
|
-
- Increase timeouts in playwright.config.js
|
|
498
|
-
- Review navigation fallback strategy
|
|
176
|
+
---
|
|
499
177
|
|
|
500
|
-
|
|
501
|
-
```bash
|
|
502
|
-
# Disable auto-open and open manually
|
|
503
|
-
AUTO_OPEN=false .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
|
|
178
|
+
## Agent Coordination Hints
|
|
504
179
|
|
|
505
|
-
|
|
506
|
-
open tests/reports/testability-report-<timestamp>.html
|
|
180
|
+
### Memory Namespace
|
|
507
181
|
```
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
# Run direct playwright
|
|
515
|
-
npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
|
|
182
|
+
aqe/testability/
|
|
183
|
+
├── assessments/* - Assessment results by URL
|
|
184
|
+
├── historical/* - Historical scores for trend analysis
|
|
185
|
+
├── recommendations/* - Improvement recommendations
|
|
186
|
+
└── integration/* - QX integration data
|
|
516
187
|
```
|
|
517
188
|
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
algorithmicTransparency: 5, // Decreased importance
|
|
530
|
-
algorithmicStability: 5,
|
|
531
|
-
explainability: 10,
|
|
532
|
-
unbugginess: 10,
|
|
533
|
-
smallness: 10,
|
|
534
|
-
decomposability: 5,
|
|
535
|
-
similarity: 5
|
|
536
|
-
}
|
|
537
|
-
};
|
|
189
|
+
### Fleet Coordination
|
|
190
|
+
```typescript
|
|
191
|
+
const testabilityFleet = await FleetManager.coordinate({
|
|
192
|
+
strategy: 'testability-assessment',
|
|
193
|
+
agents: [
|
|
194
|
+
'qe-quality-analyzer', // Primary assessment
|
|
195
|
+
'qx-partner', // UX integration
|
|
196
|
+
'qe-visual-tester' // Visual validation
|
|
197
|
+
],
|
|
198
|
+
topology: 'sequential'
|
|
199
|
+
});
|
|
538
200
|
```
|
|
539
201
|
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
Add domain-specific recommendations:
|
|
543
|
-
```javascript
|
|
544
|
-
if (score < 70) {
|
|
545
|
-
testabilityScores.recommendations.push({
|
|
546
|
-
principle: 'Observability',
|
|
547
|
-
severity: 'critical',
|
|
548
|
-
recommendation: 'Implement centralized logging with Datadog',
|
|
549
|
-
impact: 25,
|
|
550
|
-
effort: 'Medium (1-2 days)',
|
|
551
|
-
priority: 'P0'
|
|
552
|
-
});
|
|
553
|
-
}
|
|
554
|
-
```
|
|
202
|
+
---
|
|
555
203
|
|
|
556
|
-
|
|
557
|
-
```bash
|
|
558
|
-
# Run on all browsers
|
|
559
|
-
for browser in chromium firefox webkit; do
|
|
560
|
-
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/ $browser
|
|
561
|
-
done
|
|
562
|
-
```
|
|
204
|
+
## Common Issues & Solutions
|
|
563
205
|
|
|
564
|
-
|
|
206
|
+
| Issue | Solution |
|
|
207
|
+
|-------|----------|
|
|
208
|
+
| Tests timing out | Increase timeout: `timeout 300 ./scripts/run-assessment.sh URL` |
|
|
209
|
+
| Partial results | Check console errors, increase network timeout |
|
|
210
|
+
| Report not opening | Use `AUTO_OPEN=false`, open manually |
|
|
211
|
+
| Config not updating | Use `TEST_URL` env var instead |
|
|
565
212
|
|
|
566
|
-
|
|
567
|
-
- **Timeout Strategy**: Progressive fallback (domcontentloaded → commit)
|
|
568
|
-
- **Network Handling**: 15s networkidle with graceful continuation
|
|
569
|
-
- **Error Resilience**: Default scores prevent incomplete reports
|
|
570
|
-
- **Memory Management**: Single worker prevents memory exhaustion
|
|
213
|
+
---
|
|
571
214
|
|
|
572
|
-
##
|
|
215
|
+
## Related Skills
|
|
216
|
+
- [accessibility-testing](../accessibility-testing/) - WCAG compliance (overlaps with Explainability)
|
|
217
|
+
- [visual-testing-advanced](../visual-testing-advanced/) - UI consistency
|
|
218
|
+
- [performance-testing](../performance-testing/) - Load time metrics
|
|
573
219
|
|
|
574
|
-
|
|
575
|
-
- **Snapshot Assessment**: Point-in-time analysis, not continuous monitoring
|
|
576
|
-
- **Heuristic-Based**: Scores are estimates based on observable patterns
|
|
577
|
-
- **Browser-Dependent**: Results may vary slightly across browsers
|
|
578
|
-
- **Static Analysis**: Cannot assess dynamic runtime behaviors fully
|
|
220
|
+
---
|
|
579
221
|
|
|
580
222
|
## Credits & References
|
|
581
223
|
|
|
@@ -584,28 +226,20 @@ done
|
|
|
584
226
|
- Available at: https://www.satisfice.com/download/heuristics-of-software-testability
|
|
585
227
|
|
|
586
228
|
### Implementation
|
|
587
|
-
- Based on https://github.com/fndlalit/testability-scorer
|
|
229
|
+
- Based on https://github.com/fndlalit/testability-scorer (contributed by [@fndlalit](https://github.com/fndlalit))
|
|
588
230
|
- Playwright v1.49.0+ with AI capabilities
|
|
589
|
-
- Chart.js for visualizations
|
|
590
|
-
|
|
591
|
-
### Inspiration
|
|
592
|
-
- Conference workshop material by Lalit Kumar
|
|
593
|
-
- Interactive demonstrations for teaching testability concepts
|
|
231
|
+
- Chart.js for radar visualizations
|
|
594
232
|
|
|
595
|
-
|
|
233
|
+
---
|
|
596
234
|
|
|
597
|
-
|
|
598
|
-
- Full architecture: `docs/ARCHITECTURE.md`
|
|
599
|
-
- Scoring details: `docs/SCORING-METHODOLOGY.md`
|
|
600
|
-
- Troubleshooting: `docs/TROUBLESHOOTING.md`
|
|
235
|
+
## Remember
|
|
601
236
|
|
|
602
|
-
|
|
603
|
-
- [Playwright Documentation](https://playwright.dev/)
|
|
604
|
-
- [Intrinsic Testability Heuristics](https://www.satisfice.com/download/heuristics-of-software-testability)
|
|
605
|
-
- [Original Repository](https://github.com/fndlalit/testability-scorer)
|
|
237
|
+
**Testability is an investment, not an afterthought.**
|
|
606
238
|
|
|
607
|
-
|
|
239
|
+
Good testability:
|
|
240
|
+
- Reduces debugging time
|
|
241
|
+
- Enables faster feedback loops
|
|
242
|
+
- Makes defects easier to find
|
|
243
|
+
- Supports continuous testing
|
|
608
244
|
|
|
609
|
-
**
|
|
610
|
-
**Last Updated**: December 2025
|
|
611
|
-
**Compatibility**: Claude Code 2.0+, Playwright 1.49.0+
|
|
245
|
+
**Low scores = High risk. Prioritize improvements by weight × impact.**
|