agentic-qe 2.0.0 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (116) hide show
  1. package/.claude/agents/qx-partner.md +17 -4
  2. package/.claude/skills/accessibility-testing/SKILL.md +144 -692
  3. package/.claude/skills/agentic-quality-engineering/SKILL.md +176 -529
  4. package/.claude/skills/api-testing-patterns/SKILL.md +180 -560
  5. package/.claude/skills/brutal-honesty-review/SKILL.md +113 -603
  6. package/.claude/skills/bug-reporting-excellence/SKILL.md +116 -517
  7. package/.claude/skills/chaos-engineering-resilience/SKILL.md +127 -72
  8. package/.claude/skills/cicd-pipeline-qe-orchestrator/SKILL.md +209 -404
  9. package/.claude/skills/code-review-quality/SKILL.md +158 -608
  10. package/.claude/skills/compatibility-testing/SKILL.md +148 -38
  11. package/.claude/skills/compliance-testing/SKILL.md +132 -63
  12. package/.claude/skills/consultancy-practices/SKILL.md +114 -446
  13. package/.claude/skills/context-driven-testing/SKILL.md +117 -381
  14. package/.claude/skills/contract-testing/SKILL.md +176 -141
  15. package/.claude/skills/database-testing/SKILL.md +137 -130
  16. package/.claude/skills/exploratory-testing-advanced/SKILL.md +160 -629
  17. package/.claude/skills/holistic-testing-pact/SKILL.md +140 -188
  18. package/.claude/skills/localization-testing/SKILL.md +145 -33
  19. package/.claude/skills/mobile-testing/SKILL.md +132 -448
  20. package/.claude/skills/mutation-testing/SKILL.md +147 -41
  21. package/.claude/skills/performance-testing/SKILL.md +200 -546
  22. package/.claude/skills/quality-metrics/SKILL.md +164 -519
  23. package/.claude/skills/refactoring-patterns/SKILL.md +132 -699
  24. package/.claude/skills/regression-testing/SKILL.md +120 -926
  25. package/.claude/skills/risk-based-testing/SKILL.md +157 -660
  26. package/.claude/skills/security-testing/SKILL.md +199 -538
  27. package/.claude/skills/sherlock-review/SKILL.md +163 -699
  28. package/.claude/skills/shift-left-testing/SKILL.md +161 -465
  29. package/.claude/skills/shift-right-testing/SKILL.md +161 -519
  30. package/.claude/skills/six-thinking-hats/SKILL.md +175 -1110
  31. package/.claude/skills/skills-manifest.json +71 -20
  32. package/.claude/skills/tdd-london-chicago/SKILL.md +131 -448
  33. package/.claude/skills/technical-writing/SKILL.md +103 -154
  34. package/.claude/skills/test-automation-strategy/SKILL.md +166 -772
  35. package/.claude/skills/test-data-management/SKILL.md +126 -910
  36. package/.claude/skills/test-design-techniques/SKILL.md +179 -89
  37. package/.claude/skills/test-environment-management/SKILL.md +136 -91
  38. package/.claude/skills/test-reporting-analytics/SKILL.md +169 -92
  39. package/.claude/skills/testability-scoring/SKILL.md +172 -538
  40. package/.claude/skills/testability-scoring/scripts/generate-html-report.js +0 -0
  41. package/.claude/skills/visual-testing-advanced/SKILL.md +155 -78
  42. package/.claude/skills/xp-practices/SKILL.md +151 -587
  43. package/CHANGELOG.md +48 -0
  44. package/README.md +23 -16
  45. package/dist/agents/QXPartnerAgent.d.ts +8 -1
  46. package/dist/agents/QXPartnerAgent.d.ts.map +1 -1
  47. package/dist/agents/QXPartnerAgent.js +1174 -112
  48. package/dist/agents/QXPartnerAgent.js.map +1 -1
  49. package/dist/agents/lifecycle/AgentLifecycleManager.d.ts.map +1 -1
  50. package/dist/agents/lifecycle/AgentLifecycleManager.js +34 -31
  51. package/dist/agents/lifecycle/AgentLifecycleManager.js.map +1 -1
  52. package/dist/cli/commands/init-claude-md-template.d.ts.map +1 -1
  53. package/dist/cli/commands/init-claude-md-template.js +14 -0
  54. package/dist/cli/commands/init-claude-md-template.js.map +1 -1
  55. package/dist/core/SwarmCoordinator.d.ts +180 -0
  56. package/dist/core/SwarmCoordinator.d.ts.map +1 -0
  57. package/dist/core/SwarmCoordinator.js +473 -0
  58. package/dist/core/SwarmCoordinator.js.map +1 -0
  59. package/dist/core/metrics/MetricsAggregator.d.ts +228 -0
  60. package/dist/core/metrics/MetricsAggregator.d.ts.map +1 -0
  61. package/dist/core/metrics/MetricsAggregator.js +482 -0
  62. package/dist/core/metrics/MetricsAggregator.js.map +1 -0
  63. package/dist/core/metrics/index.d.ts +5 -0
  64. package/dist/core/metrics/index.d.ts.map +1 -0
  65. package/dist/core/metrics/index.js +11 -0
  66. package/dist/core/metrics/index.js.map +1 -0
  67. package/dist/core/optimization/SwarmOptimizer.d.ts +5 -0
  68. package/dist/core/optimization/SwarmOptimizer.d.ts.map +1 -1
  69. package/dist/core/optimization/SwarmOptimizer.js +17 -0
  70. package/dist/core/optimization/SwarmOptimizer.js.map +1 -1
  71. package/dist/core/orchestration/AdaptiveScheduler.d.ts +190 -0
  72. package/dist/core/orchestration/AdaptiveScheduler.d.ts.map +1 -0
  73. package/dist/core/orchestration/AdaptiveScheduler.js +460 -0
  74. package/dist/core/orchestration/AdaptiveScheduler.js.map +1 -0
  75. package/dist/core/orchestration/WorkflowOrchestrator.d.ts +13 -0
  76. package/dist/core/orchestration/WorkflowOrchestrator.d.ts.map +1 -1
  77. package/dist/core/orchestration/WorkflowOrchestrator.js +32 -0
  78. package/dist/core/orchestration/WorkflowOrchestrator.js.map +1 -1
  79. package/dist/core/recovery/CircuitBreaker.d.ts +176 -0
  80. package/dist/core/recovery/CircuitBreaker.d.ts.map +1 -0
  81. package/dist/core/recovery/CircuitBreaker.js +382 -0
  82. package/dist/core/recovery/CircuitBreaker.js.map +1 -0
  83. package/dist/core/recovery/RecoveryOrchestrator.d.ts +186 -0
  84. package/dist/core/recovery/RecoveryOrchestrator.d.ts.map +1 -0
  85. package/dist/core/recovery/RecoveryOrchestrator.js +476 -0
  86. package/dist/core/recovery/RecoveryOrchestrator.js.map +1 -0
  87. package/dist/core/recovery/RetryStrategy.d.ts +127 -0
  88. package/dist/core/recovery/RetryStrategy.d.ts.map +1 -0
  89. package/dist/core/recovery/RetryStrategy.js +314 -0
  90. package/dist/core/recovery/RetryStrategy.js.map +1 -0
  91. package/dist/core/recovery/index.d.ts +8 -0
  92. package/dist/core/recovery/index.d.ts.map +1 -0
  93. package/dist/core/recovery/index.js +27 -0
  94. package/dist/core/recovery/index.js.map +1 -0
  95. package/dist/core/skills/DependencyResolver.d.ts +99 -0
  96. package/dist/core/skills/DependencyResolver.d.ts.map +1 -0
  97. package/dist/core/skills/DependencyResolver.js +260 -0
  98. package/dist/core/skills/DependencyResolver.js.map +1 -0
  99. package/dist/core/skills/ManifestGenerator.d.ts +114 -0
  100. package/dist/core/skills/ManifestGenerator.d.ts.map +1 -0
  101. package/dist/core/skills/ManifestGenerator.js +449 -0
  102. package/dist/core/skills/ManifestGenerator.js.map +1 -0
  103. package/dist/core/skills/index.d.ts +9 -0
  104. package/dist/core/skills/index.d.ts.map +1 -0
  105. package/dist/core/skills/index.js +24 -0
  106. package/dist/core/skills/index.js.map +1 -0
  107. package/dist/mcp/server.d.ts +9 -9
  108. package/dist/mcp/server.d.ts.map +1 -1
  109. package/dist/mcp/server.js +1 -2
  110. package/dist/mcp/server.js.map +1 -1
  111. package/dist/types/qx.d.ts +39 -7
  112. package/dist/types/qx.d.ts.map +1 -1
  113. package/dist/types/qx.js.map +1 -1
  114. package/dist/visualization/api/RestEndpoints.js +1 -1
  115. package/dist/visualization/api/RestEndpoints.js.map +1 -1
  116. package/package.json +13 -55
@@ -1,416 +1,118 @@
1
1
  ---
2
- name: "Testability Scoring"
3
- description: "AI-powered testability assessment using 10 principles of intrinsic testability with Playwright. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Algorithmic Transparency, Algorithmic Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports for web applications."
2
+ name: testability-scoring
3
+ description: "AI-powered testability assessment using 10 principles of intrinsic testability with Playwright. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports."
4
+ category: testing-methodologies
5
+ priority: high
6
+ tokenEstimate: 1100
7
+ agents: [qe-quality-analyzer, qx-partner, qe-visual-tester]
8
+ implementation_status: optimized
9
+ optimization_version: 2.1
10
+ last_optimized: 2025-12-03
11
+ dependencies: []
12
+ quick_reference_card: true
13
+ tags: [testability, scoring, playwright, assessment, 10-principles, intrinsic-testability, james-bach, michael-bolton]
14
+ contributor: "@fndlalit"
4
15
  ---
5
16
 
6
17
  # Testability Scoring
7
18
 
8
- ## What This Skill Does
9
-
10
- Performs comprehensive AI-powered testability assessments of web applications using Playwright and the 10 Principles of Intrinsic Testability framework developed by James Bach and Michael Bolton. Generates detailed scoring reports with visual charts, metrics breakdowns, and actionable AI-driven recommendations for improving testability.
11
-
12
- ### Core Capabilities
13
-
14
- - **10-Principle Analysis**: Complete assessment across all intrinsic testability dimensions
15
- - **Quantitative Scoring**: 0-100 scale with letter grades (A-F) for each principle
16
- - **AI-Enhanced Insights**: Intelligent recommendations powered by advanced analysis
17
- - **Interactive HTML Reports**: Beautiful visual reports with Chart.js radar visualizations
18
- - **Multi-User Comparison**: Side-by-side testability analysis across user types
19
- - **Historical Tracking**: Progress measurement and improvement validation
20
- - **Automated Analysis**: Hands-free assessment with error-resilient execution
21
-
22
- ## The 10 Principles of Intrinsic Testability
23
-
24
- ### 1. **Observability** (15% weight)
25
- Can we see what's happening in the system?
26
- - State visibility and capture
27
- - Event logging and monitoring
28
- - Network request tracking
29
- - Error visibility
30
- - Visual state inspection
31
-
32
- ### 2. **Controllability** (15% weight)
33
- Can we control the application precisely?
34
- - Direct input control
35
- - State manipulation
36
- - API accessibility
37
- - Test data injection
38
- - Deterministic behavior
39
-
40
- ### 3. **Algorithmic Simplicity** (10% weight)
41
- Are behaviors simple and predictable?
42
- - Clear input-output relationships
43
- - Operation complexity
44
- - Interaction patterns
45
- - Behavior predictability
46
-
47
- ### 4. **Algorithmic Transparency** (10% weight)
48
- Can we understand what the system does?
49
- - Behavior visibility
50
- - Process understanding
51
- - Black box reduction
52
- - Code readability indicators
53
-
54
- ### 5. **Algorithmic Stability** (10% weight)
55
- Does behavior remain consistent?
56
- - Change resilience
57
- - Test maintainability
58
- - Behavior consistency
59
- - Version stability
60
-
61
- ### 6. **Explainability** (10% weight)
62
- Can users and developers understand the interface?
63
- - Documentation quality
64
- - Code clarity
65
- - Semantic structure
66
- - Help text and guidance
67
- - Error message clarity
68
-
69
- ### 7. **Unbugginess** (10% weight)
70
- How error-free is the application?
71
- - Console error tracking
72
- - Page error monitoring
73
- - Warning analysis
74
- - Runtime stability
75
-
76
- ### 8. **Smallness** (10% weight)
77
- Are components appropriately sized?
78
- - Page complexity
79
- - Element count
80
- - Script/style bloat
81
- - Component granularity
82
-
83
- ### 9. **Decomposability** (5% weight)
84
- Can we test parts in isolation?
85
- - Component separation
86
- - Isolated testing capability
87
- - Modular design
88
- - Feature independence
89
-
90
- ### 10. **Similarity** (5% weight)
91
- How familiar is the technology stack?
92
- - Standard frameworks
93
- - Common patterns
94
- - Known platforms
95
- - Familiar conventions
96
-
97
- ## Quick Start
98
-
99
- ### Method 1: Shell Script (Recommended)
100
- ```bash
101
- # Run assessment on any URL (runtime override)
102
- TEST_URL='https://example.com/' npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
103
-
104
- # Or use the shell script wrapper
105
- .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
106
-
107
- # With specific browser
108
- .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/ firefox
109
- ```
110
-
111
- ### Method 2: Update Config File (Persistent)
112
- ```bash
113
- # Update config for repeated runs
114
- echo 'module.exports = { baseURL: "https://example.com/", ... };' > tests/testability-scoring/config.js
115
-
116
- # Run assessment
117
- npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
118
- ```
119
-
120
- ### Method 3: Generate HTML Report Only
121
- ```bash
122
- # From existing JSON results
123
- AUTO_OPEN=false node .claude/skills/testability-scoring/scripts/generate-html-report.js tests/reports/testability-results-*.json
124
- ```
125
-
126
- ## Complete Usage Guide
127
-
128
- ### Assessment Workflow
129
-
130
- 1. **Run Assessment**: Execute tests against target URL
131
- 2. **JSON Generation**: Automated results saved to `tests/reports/`
132
- 3. **HTML Report**: Interactive report with charts automatically generated
133
- 4. **Browser Opening**: Report opens automatically (configurable)
134
- 5. **Analysis**: Review scores, charts, and recommendations
135
-
136
- ### Configuration
19
+ <default_to_action>
20
+ When assessing testability:
21
+ 1. RUN assessment against target URL
22
+ 2. ANALYZE all 10 principles automatically
23
+ 3. GENERATE HTML report with radar chart
24
+ 4. PRIORITIZE improvements by impact/effort
25
+ 5. INTEGRATE with QX Partner for holistic view
137
26
 
138
- #### Runtime URL Override (Recommended)
139
- Use the `TEST_URL` environment variable for one-time assessments:
27
+ **Quick Assessment:**
140
28
  ```bash
29
+ # Run assessment on any URL
141
30
  TEST_URL='https://example.com/' npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
142
- ```
143
-
144
- #### Config File Location
145
- `tests/testability-scoring/config.js`
146
-
147
- ```javascript
148
- module.exports = {
149
- // Runtime URL override via TEST_URL environment variable
150
- baseURL: process.env.TEST_URL || 'https://your-application.com/',
151
- timeout: 60000, // 60 second timeout
152
- networkTimeout: 15000 // 15 second network idle timeout
153
- };
154
- ```
155
-
156
- #### Playwright Configuration
157
- Located in `playwright.config.js`:
158
- - Timeout: 60s per test
159
- - Workers: 1 (serial execution)
160
- - Launch args: `--no-sandbox`, `--disable-dev-shm-usage`
161
- - Viewport: 1280x720
162
- - ignoreHTTPSErrors: true
163
31
 
164
- ### Environment Variables
165
-
166
- ```bash
167
- # Disable automatic browser opening
168
- AUTO_OPEN=false .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
169
-
170
- # With custom timeout
171
- timeout 180 .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
172
- ```
173
-
174
- ## Assessment Script Architecture
175
-
176
- ### Error Handling Strategy
177
-
178
- The assessment script uses **progressive fallback** strategy:
179
-
180
- ```javascript
181
- // Navigation with multi-level fallback
182
- async function navigateToPage(page) {
183
- try {
184
- // Level 1: domcontentloaded
185
- await page.goto(baseURL, {
186
- waitUntil: 'domcontentloaded',
187
- timeout: 45000
188
- });
189
-
190
- // Level 2: networkidle with timeout
191
- await page.waitForLoadState('networkidle', {
192
- timeout: 15000
193
- }).catch(() => console.log('[NAV] Continuing without networkidle'));
194
-
195
- return true;
196
- } catch (error) {
197
- // Level 3: commit fallback
198
- await page.goto(baseURL, {
199
- waitUntil: 'commit',
200
- timeout: 45000
201
- });
202
- return true;
203
- }
204
- }
32
+ # Or use shell script wrapper
33
+ .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
205
34
  ```
206
35
 
207
- ### Default Scoring
208
-
209
- All principles initialized with default scores (50/F) before testing:
210
- ```javascript
211
- function initializeDefaultScores() {
212
- testabilityScores.principles = {
213
- observability: { score: 50, grade: 'F', weight: 15 },
214
- controllability: { score: 50, grade: 'F', weight: 15 },
215
- algorithmicSimplicity: { score: 50, grade: 'F', weight: 10 },
216
- algorithmicTransparency: { score: 50, grade: 'F', weight: 10 },
217
- algorithmicStability: { score: 50, grade: 'F', weight: 10 },
218
- explainability: { score: 50, grade: 'F', weight: 10 },
219
- unbugginess: { score: 50, grade: 'F', weight: 10 },
220
- smallness: { score: 50, grade: 'F', weight: 10 },
221
- decomposability: { score: 50, grade: 'F', weight: 5 },
222
- similarity: { score: 50, grade: 'F', weight: 5 }
223
- };
224
- }
225
- ```
36
+ **The 10 Principles at a Glance:**
37
+ | Principle | Weight | Key Question |
38
+ |-----------|--------|--------------|
39
+ | **Observability** | 15% | Can we see what's happening? |
40
+ | **Controllability** | 15% | Can we control the application? |
41
+ | **Algorithmic Simplicity** | 10% | Are behaviors predictable? |
42
+ | **Algorithmic Transparency** | 10% | Can we understand what it does? |
43
+ | **Algorithmic Stability** | 10% | Does behavior remain consistent? |
44
+ | **Explainability** | 10% | Is the interface understandable? |
45
+ | **Unbugginess** | 10% | How error-free is it? |
46
+ | **Smallness** | 10% | Are components appropriately sized? |
47
+ | **Decomposability** | 5% | Can we test parts in isolation? |
48
+ | **Similarity** | 5% | Is the tech stack familiar? |
49
+
50
+ **Grade Scale:**
51
+ - **A (90-100)**: Excellent testability
52
+ - **B (80-89)**: Good testability
53
+ - **C (70-79)**: Adequate testability
54
+ - **D (60-69)**: Below average
55
+ - **F (0-59)**: Poor testability
56
+ </default_to_action>
57
+
58
+ ## Quick Reference Card
59
+
60
+ ### Running Assessments
61
+
62
+ | Method | Command | When to Use |
63
+ |--------|---------|-------------|
64
+ | Shell Script | `./scripts/run-assessment.sh URL` | One-time assessment |
65
+ | ENV Override | `TEST_URL='URL' npx playwright test...` | CI/CD integration |
66
+ | Config File | Update `tests/testability-scoring/config.js` | Repeated runs |
67
+
68
+ ### Principle Details
69
+
70
+ #### High Weight (15% each)
71
+ | Principle | Measures | Indicators |
72
+ |-----------|----------|------------|
73
+ | **Observability** | State visibility, logging, monitoring | Console output, network tracking, error visibility |
74
+ | **Controllability** | Input control, state manipulation | API access, test data injection, determinism |
75
+
76
+ #### Medium Weight (10% each)
77
+ | Principle | Measures | Indicators |
78
+ |-----------|----------|------------|
79
+ | **Simplicity** | Predictable behavior | Clear I/O relationships, low complexity |
80
+ | **Transparency** | Understanding what system does | Visible processes, readable code |
81
+ | **Stability** | Consistent behavior | Change resilience, maintainability |
82
+ | **Explainability** | Interface understanding | Good docs, semantic structure, help text |
83
+ | **Unbugginess** | Error-free operation | Console errors, warnings, runtime issues |
84
+ | **Smallness** | Component size | Element count, script bloat, page complexity |
85
+
86
+ #### Low Weight (5% each)
87
+ | Principle | Measures | Indicators |
88
+ |-----------|----------|------------|
89
+ | **Decomposability** | Isolation testing | Component separation, modular design |
90
+ | **Similarity** | Technology familiarity | Standard frameworks, known patterns |
226
91
 
227
- ### Test Execution Pattern
228
-
229
- Each principle test follows this pattern:
230
- ```javascript
231
- test('1. Observability Assessment', async ({ page }) => {
232
- try {
233
- const loaded = await navigateToPage(page);
234
- if (!loaded) throw new Error('Failed to load page');
235
-
236
- // Principle-specific analysis
237
- let score = 0;
238
- // ... scoring logic ...
239
-
240
- testabilityScores.principles.observability = {
241
- score: Math.min(score, 100),
242
- grade: getLetterGrade(score),
243
- weight: config.weights.observability
244
- };
245
-
246
- // Generate recommendations if score is low
247
- if (score < 70) {
248
- testabilityScores.recommendations.push({
249
- principle: 'Observability',
250
- severity: 'medium',
251
- recommendation: 'Implement detailed event logging...',
252
- impact: 15,
253
- effort: 'Low (4-6 hours)'
254
- });
255
- }
256
- } catch (error) {
257
- console.error('Observability assessment failed:', error.message);
258
- // Fallback to default score
259
- testabilityScores.principles.observability = {
260
- score: 50,
261
- grade: 'F',
262
- weight: config.weights.observability
263
- };
264
- }
265
- });
266
- ```
92
+ ---
267
93
 
268
- ## HTML Report Features
269
-
270
- ### Interactive Elements
271
-
272
- - **Radar Chart**: Visual representation of all 10 principles
273
- - **Score Cards**: Color-coded principle scores with grades
274
- - **Detailed Breakdowns**: Expandable sections for each principle
275
- - **Recommendations**: Prioritized improvement suggestions
276
- - **Metadata**: URL, browser, version, duration, timestamp
277
-
278
- ### Chart.js Integration
279
-
280
- Reports use Chart.js for beautiful visualizations:
281
- ```javascript
282
- new Chart(ctx, {
283
- type: 'radar',
284
- data: {
285
- labels: [
286
- 'Observability', 'Controllability', 'Simplicity',
287
- 'Transparency', 'Explainability', 'Similarity',
288
- 'Stability', 'Unbugginess', 'Smallness', 'Decomposability'
289
- ],
290
- datasets: [{
291
- label: 'Testability Scores',
292
- data: [92, 20, 75, 60, 75, 85, 70, 85, 95, 70],
293
- backgroundColor: 'rgba(74, 144, 226, 0.2)',
294
- borderColor: 'rgba(74, 144, 226, 1)',
295
- pointBackgroundColor: 'rgba(74, 144, 226, 1)'
296
- }]
297
- }
298
- });
299
- ```
94
+ ## Assessment Workflow
300
95
 
301
- ### Report Sections
302
-
303
- 1. **Executive Summary**: Overall score, grade, and quick stats
304
- 2. **Principle Scores**: Detailed breakdown with grades
305
- 3. **Visual Analysis**: Radar chart and score distribution
306
- 4. **Recommendations**: Actionable improvements prioritized by impact
307
- 5. **Metadata**: Assessment details and context
308
-
309
- ## Scoring Methodology
310
-
311
- ### Grade Scale
312
- - **90-100 (A)**: Excellent testability
313
- - **80-89 (B)**: Good testability
314
- - **70-79 (C)**: Adequate testability
315
- - **60-69 (D)**: Below average testability
316
- - **0-59 (F)**: Poor testability
317
-
318
- ### Weighted Calculation
319
-
320
- Overall score uses weighted average:
321
- ```javascript
322
- function calculateOverallScore(principles) {
323
- let totalWeightedScore = 0;
324
- let totalWeight = 0;
325
-
326
- for (const [principle, data] of Object.entries(principles)) {
327
- totalWeightedScore += data.score * data.weight;
328
- totalWeight += data.weight;
329
- }
330
-
331
- return Math.round(totalWeightedScore / totalWeight);
332
- }
333
96
  ```
334
-
335
- ### Letter Grade Assignment
336
- ```javascript
337
- function getLetterGrade(score) {
338
- if (score >= 90) return 'A';
339
- if (score >= 80) return 'B';
340
- if (score >= 70) return 'C';
341
- if (score >= 60) return 'D';
342
- return 'F';
343
- }
97
+ 1. Navigate to URL → 2. Collect Metrics → 3. Score Principles
98
+
99
+ 4. Generate JSON ← 5. Calculate Grades ← 6. Apply Weights
100
+
101
+ 7. Generate HTML Report with Radar Chart
102
+
103
+ 8. Open in Browser (auto-opens)
344
104
  ```
345
105
 
346
- ## File Structure
106
+ ### Output Files
347
107
 
348
108
  ```
349
- .claude/skills/testability-scoring/
350
- ├── SKILL.md # This file
351
- ├── README.md # Quick reference
352
- ├── scripts/
353
- │ ├── run-assessment.sh # Main assessment runner
354
- │ └── generate-html-report.js # HTML report generator
355
- ├── resources/
356
- │ └── templates/
357
- │ └── config.template.js # Config file template
358
- ├── examples/
359
- │ ├── sample-assessment.json # Example JSON output
360
- │ └── sample-report.html # Example HTML report
361
- └── docs/
362
- ├── ARCHITECTURE.md # System architecture
363
- ├── SCORING-METHODOLOGY.md # Detailed scoring guide
364
- └── TROUBLESHOOTING.md # Common issues and fixes
365
-
366
- tests/testability-scoring/
367
- ├── testability-scoring.spec.js # Main test file
368
- ├── config.js # Runtime configuration
369
- └── playwright.config.js # (optional) Test-specific config
370
-
371
109
  tests/reports/
372
- ├── testability-results-<timestamp>.json # JSON results
373
- ├── testability-report-<timestamp>.html # HTML reports
374
- └── latest.json # Symlink to latest results
110
+ ├── testability-results-<timestamp>.json # Raw data
111
+ ├── testability-report-<timestamp>.html # Visual report
112
+ └── latest.json # Symlink
375
113
  ```
376
114
 
377
- ## Example Assessments
378
-
379
- ### Example 1: High Testability Site
380
- ```bash
381
- $ .claude/skills/testability-scoring/scripts/run-assessment.sh https://talesoftesting.com/
382
-
383
- Results:
384
- Overall Score: 71/100 (C)
385
- - Observability: 92 (A) ⭐
386
- - Controllability: 20 (F) ⚠️
387
- - Algorithmic Simplicity: 75 (C)
388
- - Algorithmic Transparency: 60 (D)
389
- - Explainability: 75 (C)
390
- - Similarity: 85 (B)
391
- - Algorithmic Stability: 70 (C)
392
- - Unbugginess: 85 (B)
393
- - Smallness: 95 (A) ⭐
394
- - Decomposability: 70 (C)
395
- ```
396
-
397
- ### Example 2: E-Commerce Site
398
- ```bash
399
- $ .claude/skills/testability-scoring/scripts/run-assessment.sh https://www.saucedemo.com/
400
-
401
- Results:
402
- Overall Score: 69/100 (D)
403
- - Observability: 92 (A) ⭐
404
- - Controllability: 20 (F) ⚠️ Critical Issue
405
- - Smallness: 100 (A) ⭐ Perfect
406
- - Similarity: 85 (B)
407
- - Unbugginess: 85 (B)
408
- - Algorithmic Simplicity: 75 (C)
409
- - Algorithmic Stability: 70 (C)
410
- - Algorithmic Transparency: 60 (D)
411
- - Explainability: 60 (D)
412
- - Decomposability: 50 (F)
413
- ```
115
+ ---
414
116
 
415
117
  ## Integration Examples
416
118
 
@@ -420,7 +122,7 @@ Overall Score: 69/100 (D)
420
122
  - name: Testability Assessment
421
123
  run: |
422
124
  timeout 180 .claude/skills/testability-scoring/scripts/run-assessment.sh ${{ env.APP_URL }}
423
-
125
+
424
126
  - name: Upload Reports
425
127
  uses: actions/upload-artifact@v3
426
128
  with:
@@ -428,154 +130,94 @@ Overall Score: 69/100 (D)
428
130
  path: tests/reports/testability-*.html
429
131
  ```
430
132
 
431
- ### Programmatic Usage
432
- ```javascript
433
- const { spawn } = require('child_process');
434
-
435
- function runTestabilityAssessment(url) {
436
- return new Promise((resolve, reject) => {
437
- const proc = spawn('.claude/skills/testability-scoring/scripts/run-assessment.sh', [url]);
438
-
439
- let output = '';
440
- proc.stdout.on('data', data => output += data);
441
-
442
- proc.on('close', code => {
443
- if (code === 0) {
444
- resolve(output);
445
- } else {
446
- reject(new Error(`Assessment failed with code ${code}`));
447
- }
448
- });
449
- });
450
- }
451
-
452
- // Usage
453
- const results = await runTestabilityAssessment('https://example.com/');
454
- console.log(results);
455
- ```
456
-
457
- ## Best Practices
133
+ ### QX Partner Integration
134
+ ```typescript
135
+ // Combine testability with QX analysis
136
+ const qxAnalysis = await Task("QX Analysis", {
137
+ target: 'https://example.com',
138
+ integrateTestability: true
139
+ }, "qx-partner");
458
140
 
459
- ### 1. Baseline Establishment
460
- - Run initial assessment to establish baseline
461
- - Document scores before any changes
462
- - Create historical comparison point
463
-
464
- ### 2. Focus Areas
465
- - Prioritize principles with lowest scores
466
- - Address "F" grades first (scores < 60)
467
- - Target quick wins (low effort, high impact)
141
+ // Returns combined insights:
142
+ // - QX Score: 78/100
143
+ // - Testability Integration: Observability 72/100
144
+ // - Combined Insight: Low observability may mask UX issues
145
+ ```
468
146
 
469
- ### 3. Iterative Improvement
470
- - Make targeted changes
471
- - Re-run assessment to validate
472
- - Track progress over time
147
+ ### Programmatic Usage
148
+ ```typescript
149
+ import { runTestabilityAssessment } from './testability';
473
150
 
474
- ### 4. Continuous Monitoring
475
- - Integrate into CI/CD pipeline
476
- - Set minimum score thresholds
477
- - Alert on score degradation
151
+ const results = await runTestabilityAssessment('https://example.com');
152
+ console.log(`Overall: ${results.overallScore}/100 (${results.grade})`);
153
+ console.log('Recommendations:', results.recommendations);
154
+ ```
478
155
 
479
- ### 5. Multi-User Testing
480
- - Test with different user personas
481
- - Compare accessibility across roles
482
- - Identify role-specific issues
156
+ ---
483
157
 
484
- ## Troubleshooting
158
+ ## Agent Integration
485
159
 
486
- ### Common Issues
160
+ ```typescript
161
+ // Run testability assessment
162
+ const assessment = await Task("Testability Assessment", {
163
+ url: 'https://example.com',
164
+ generateReport: true,
165
+ openBrowser: true
166
+ }, "qe-quality-analyzer");
487
167
 
488
- #### Tests Timing Out
489
- ```bash
490
- # Increase timeout
491
- timeout 300 .claude/skills/testability-scoring/scripts/run-assessment.sh https://slow-site.com/
168
+ // Use with QX Partner for holistic analysis
169
+ const qxReport = await Task("Full QX Analysis", {
170
+ target: 'https://example.com',
171
+ integrateTestability: true,
172
+ detectOracleProblems: true
173
+ }, "qx-partner");
492
174
  ```
493
175
 
494
- #### Partial Results (Not All 10 Principles)
495
- - Check for JavaScript errors in console
496
- - Verify network connectivity
497
- - Increase timeouts in playwright.config.js
498
- - Review navigation fallback strategy
176
+ ---
499
177
 
500
- #### HTML Report Not Opening
501
- ```bash
502
- # Disable auto-open and open manually
503
- AUTO_OPEN=false .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
178
+ ## Agent Coordination Hints
504
179
 
505
- # Then open manually
506
- open tests/reports/testability-report-<timestamp>.html
180
+ ### Memory Namespace
507
181
  ```
508
-
509
- #### Config Not Updating
510
- ```bash
511
- # Manually update config
512
- echo 'module.exports = { baseURL: "https://example.com/" };' > tests/testability-scoring/config.js
513
-
514
- # Run direct playwright
515
- npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
182
+ aqe/testability/
183
+ ├── assessments/* - Assessment results by URL
184
+ ├── historical/* - Historical scores for trend analysis
185
+ ├── recommendations/* - Improvement recommendations
186
+ └── integration/* - QX integration data
516
187
  ```
517
188
 
518
- ## Advanced Features
519
-
520
- ### Custom Scoring Weights
521
-
522
- Modify weights in test spec file:
523
- ```javascript
524
- const config = {
525
- weights: {
526
- observability: 20, // Increased importance
527
- controllability: 20, // Increased importance
528
- algorithmicSimplicity: 10,
529
- algorithmicTransparency: 5, // Decreased importance
530
- algorithmicStability: 5,
531
- explainability: 10,
532
- unbugginess: 10,
533
- smallness: 10,
534
- decomposability: 5,
535
- similarity: 5
536
- }
537
- };
189
+ ### Fleet Coordination
190
+ ```typescript
191
+ const testabilityFleet = await FleetManager.coordinate({
192
+ strategy: 'testability-assessment',
193
+ agents: [
194
+ 'qe-quality-analyzer', // Primary assessment
195
+ 'qx-partner', // UX integration
196
+ 'qe-visual-tester' // Visual validation
197
+ ],
198
+ topology: 'sequential'
199
+ });
538
200
  ```
539
201
 
540
- ### Custom Recommendations
541
-
542
- Add domain-specific recommendations:
543
- ```javascript
544
- if (score < 70) {
545
- testabilityScores.recommendations.push({
546
- principle: 'Observability',
547
- severity: 'critical',
548
- recommendation: 'Implement centralized logging with Datadog',
549
- impact: 25,
550
- effort: 'Medium (1-2 days)',
551
- priority: 'P0'
552
- });
553
- }
554
- ```
202
+ ---
555
203
 
556
- ### Multi-Browser Comparison
557
- ```bash
558
- # Run on all browsers
559
- for browser in chromium firefox webkit; do
560
- .claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/ $browser
561
- done
562
- ```
204
+ ## Common Issues & Solutions
563
205
 
564
- ## Performance Considerations
206
+ | Issue | Solution |
207
+ |-------|----------|
208
+ | Tests timing out | Increase timeout: `timeout 300 ./scripts/run-assessment.sh URL` |
209
+ | Partial results | Check console errors, increase network timeout |
210
+ | Report not opening | Use `AUTO_OPEN=false`, open manually |
211
+ | Config not updating | Use `TEST_URL` env var instead |
565
212
 
566
- - **Serial Execution**: Tests run with `--workers=1` for consistent results
567
- - **Timeout Strategy**: Progressive fallback (domcontentloaded → commit)
568
- - **Network Handling**: 15s networkidle with graceful continuation
569
- - **Error Resilience**: Default scores prevent incomplete reports
570
- - **Memory Management**: Single worker prevents memory exhaustion
213
+ ---
571
214
 
572
- ## Limitations
215
+ ## Related Skills
216
+ - [accessibility-testing](../accessibility-testing/) - WCAG compliance (overlaps with Explainability)
217
+ - [visual-testing-advanced](../visual-testing-advanced/) - UI consistency
218
+ - [performance-testing](../performance-testing/) - Load time metrics
573
219
 
574
- - **Client-Side Only**: Assesses UI/frontend, not backend APIs directly
575
- - **Snapshot Assessment**: Point-in-time analysis, not continuous monitoring
576
- - **Heuristic-Based**: Scores are estimates based on observable patterns
577
- - **Browser-Dependent**: Results may vary slightly across browsers
578
- - **Static Analysis**: Cannot assess dynamic runtime behaviors fully
220
+ ---
579
221
 
580
222
  ## Credits & References
581
223
 
@@ -584,28 +226,20 @@ done
584
226
  - Available at: https://www.satisfice.com/download/heuristics-of-software-testability
585
227
 
586
228
  ### Implementation
587
- - Based on https://github.com/fndlalit/testability-scorer
229
+ - Based on https://github.com/fndlalit/testability-scorer (contributed by [@fndlalit](https://github.com/fndlalit))
588
230
  - Playwright v1.49.0+ with AI capabilities
589
- - Chart.js for visualizations
590
-
591
- ### Inspiration
592
- - Conference workshop material by Lalit Kumar
593
- - Interactive demonstrations for teaching testability concepts
231
+ - Chart.js for radar visualizations
594
232
 
595
- ## Support
233
+ ---
596
234
 
597
- ### Documentation
598
- - Full architecture: `docs/ARCHITECTURE.md`
599
- - Scoring details: `docs/SCORING-METHODOLOGY.md`
600
- - Troubleshooting: `docs/TROUBLESHOOTING.md`
235
+ ## Remember
601
236
 
602
- ### Resources
603
- - [Playwright Documentation](https://playwright.dev/)
604
- - [Intrinsic Testability Heuristics](https://www.satisfice.com/download/heuristics-of-software-testability)
605
- - [Original Repository](https://github.com/fndlalit/testability-scorer)
237
+ **Testability is an investment, not an afterthought.**
606
238
 
607
- ---
239
+ Good testability:
240
+ - Reduces debugging time
241
+ - Enables faster feedback loops
242
+ - Makes defects easier to find
243
+ - Supports continuous testing
608
244
 
609
- **Version**: 1.0.0
610
- **Last Updated**: December 2025
611
- **Compatibility**: Claude Code 2.0+, Playwright 1.49.0+
245
+ **Low scores = High risk. Prioritize improvements by weight × impact.**