ocs-stats 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +60 -7
- package/bin/cli.js +9 -3
- package/package.json +1 -1
- package/src/display.js +24 -12
- package/src/init.js +8 -6
- package/src/stats.js +9 -9
- package/templates/agents/testing.md +303 -0
- package/templates/skills/testing/SKILL.md +499 -0
- package/templates/testing/knowledge.md +77 -0
- package/templates/testing/xp.json +60 -0
package/README.md
CHANGED
|
@@ -59,10 +59,14 @@ OpenCode automatically loads any `.opencode` folder in your project root.
|
|
|
59
59
|
|
|
60
60
|
## Check Your Progress
|
|
61
61
|
|
|
62
|
-
Use the `stats` command to view your
|
|
62
|
+
Use the `stats` command to view your agent progress:
|
|
63
63
|
|
|
64
64
|
```bash
|
|
65
|
+
# Security agent
|
|
65
66
|
npx ocs-stats stats
|
|
67
|
+
|
|
68
|
+
# Testing agent
|
|
69
|
+
npx ocs-stats stats testing
|
|
66
70
|
```
|
|
67
71
|
|
|
68
72
|
Example output:
|
|
@@ -95,6 +99,7 @@ npx ocs-stats update
|
|
|
95
99
|
| Agent | Description |
|
|
96
100
|
|-------|-------------|
|
|
97
101
|
| `security` | Security expert with XP-based leveling system for auditing and fixing vulnerabilities |
|
|
102
|
+
| `testing` | Testing expert for unit, integration, and E2E tests with Playwright integration |
|
|
98
103
|
|
|
99
104
|
### Skills
|
|
100
105
|
|
|
@@ -104,6 +109,7 @@ npx ocs-stats update
|
|
|
104
109
|
| `memories` | Session memory for tracking work context and pending tasks |
|
|
105
110
|
| `mobile` | Mobile development (React Native, Flutter, Swift) |
|
|
106
111
|
| `security` | Security patterns, auth approach, and anti-patterns |
|
|
112
|
+
| `testing` | Testing patterns (Vitest, Jest, React Testing Library, Playwright) |
|
|
107
113
|
| `webapp` | Web development (React, Vue, Svelte, Angular) |
|
|
108
114
|
|
|
109
115
|
## Customization
|
|
@@ -143,7 +149,7 @@ The security agent includes an XP-based leveling system that tracks your progres
|
|
|
143
149
|
| 2 | Apprentice | 150 |
|
|
144
150
|
| 3 | Practitioner | 450 |
|
|
145
151
|
| 4 | Expert | 900 |
|
|
146
|
-
| 5 | Master |
|
|
152
|
+
| 5 | Master | 1,500 |
|
|
147
153
|
| 6 | Grandmaster | 3,000 |
|
|
148
154
|
|
|
149
155
|
### XP Awards (Fix-Only System)
|
|
@@ -164,21 +170,58 @@ Before risky operations (auth changes, DB schema, middleware), the agent:
|
|
|
164
170
|
2. Confirms user permission
|
|
165
171
|
3. Documents rollback plan
|
|
166
172
|
|
|
173
|
+
## Testing Agent Features
|
|
174
|
+
|
|
175
|
+
The testing agent helps you write tests with an XP-based leveling system:
|
|
176
|
+
|
|
177
|
+
| Level | Title | XP Required | Focus |
|
|
178
|
+
|-------|-------|-------------|-------|
|
|
179
|
+
| 1 | Novice | 0 | Basic unit tests |
|
|
180
|
+
| 2 | Apprentice | 100 | Integration tests |
|
|
181
|
+
| 3 | Practitioner | 300 | E2E tests |
|
|
182
|
+
| 4 | Expert | 600 | Test patterns & mocking |
|
|
183
|
+
| 5 | Master | 1,200 | Full coverage strategies |
|
|
184
|
+
| 6 | Grandmaster | 2,500 | Testing excellence |
|
|
185
|
+
|
|
186
|
+
### XP Awards
|
|
187
|
+
|
|
188
|
+
| Action | XP |
|
|
189
|
+
|--------|-----|
|
|
190
|
+
| Write unit test | +10 XP |
|
|
191
|
+
| Write integration test | +15 XP |
|
|
192
|
+
| Write E2E test | +20 XP |
|
|
193
|
+
| Fix broken test | +10 XP |
|
|
194
|
+
| Add test pattern | +30 XP |
|
|
195
|
+
| Complete package test suite | +100 XP |
|
|
196
|
+
|
|
197
|
+
### Playwright Integration
|
|
198
|
+
|
|
199
|
+
When you need E2E testing:
|
|
200
|
+
1. The testing agent checks for Playwright MCP
|
|
201
|
+
2. If not configured, prompts you to enable it
|
|
202
|
+
3. Creates `opencode.json` with Playwright MCP config
|
|
203
|
+
4. Installs `@playwright/test` and browser binaries
|
|
204
|
+
|
|
167
205
|
## File Structure
|
|
168
206
|
|
|
169
207
|
```
|
|
170
208
|
.opencode/
|
|
171
209
|
├── agents/
|
|
172
|
-
│
|
|
210
|
+
│ ├── security.md # Security audit agent
|
|
211
|
+
│ └── testing.md # Testing agent
|
|
173
212
|
├── skills/
|
|
174
213
|
│ ├── commit/SKILL.md # Commit conventions
|
|
175
214
|
│ ├── memories/SKILL.md # Session memory (auto-updated)
|
|
176
215
|
│ ├── mobile/SKILL.md # Mobile patterns (RN, Flutter, Swift)
|
|
177
216
|
│ ├── security/SKILL.md # Security patterns
|
|
217
|
+
│ ├── testing/SKILL.md # Testing patterns
|
|
178
218
|
│ └── webapp/SKILL.md # Web patterns (React, Vue, Svelte, Angular)
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
219
|
+
├── security/
|
|
220
|
+
│ ├── xp.json # XP tracking (auto-updated)
|
|
221
|
+
│ └── knowledge.md # Accumulated findings (auto-updated)
|
|
222
|
+
└── testing/
|
|
223
|
+
├── xp.json # Testing XP tracking (auto-updated)
|
|
224
|
+
└── knowledge.md # Testing patterns & lessons (auto-updated)
|
|
182
225
|
```
|
|
183
226
|
|
|
184
227
|
## For Contributors
|
|
@@ -193,8 +236,11 @@ opencode-skills/
|
|
|
193
236
|
│ └── stats.js
|
|
194
237
|
├── templates/ # Files copied to user projects
|
|
195
238
|
│ ├── agents/
|
|
239
|
+
│ │ ├── security.md
|
|
240
|
+
│ │ └── testing.md
|
|
196
241
|
│ ├── skills/
|
|
197
|
-
│
|
|
242
|
+
│ ├── security/
|
|
243
|
+
│ └── testing/
|
|
198
244
|
├── README.md
|
|
199
245
|
└── LICENSE
|
|
200
246
|
```
|
|
@@ -226,6 +272,13 @@ rm .opencode/security/xp.json
|
|
|
226
272
|
rm .opencode/security/knowledge.md
|
|
227
273
|
```
|
|
228
274
|
|
|
275
|
+
### Want to reset testing XP?
|
|
276
|
+
|
|
277
|
+
```bash
|
|
278
|
+
rm .opencode/testing/xp.json
|
|
279
|
+
rm .opencode/testing/knowledge.md
|
|
280
|
+
```
|
|
281
|
+
|
|
229
282
|
## Contributing
|
|
230
283
|
|
|
231
284
|
Feel free to submit issues and pull requests to improve these skills and agents.
|
package/bin/cli.js
CHANGED
|
@@ -17,6 +17,7 @@ Usage:
|
|
|
17
17
|
npx ocs-stats --global Install globally (~/.opencode)
|
|
18
18
|
npx ocs-stats update Update skills (removes existing)
|
|
19
19
|
npx ocs-stats stats Show security agent progress
|
|
20
|
+
npx ocs-stats stats testing Show testing agent progress
|
|
20
21
|
npx ocs-stats display-xp <amount> "<reason>"
|
|
21
22
|
Display XP gain (used by agent)
|
|
22
23
|
|
|
@@ -28,7 +29,9 @@ Examples:
|
|
|
28
29
|
npx ocs-stats
|
|
29
30
|
npx ocs-stats update
|
|
30
31
|
npx ocs-stats stats
|
|
32
|
+
npx ocs-stats stats testing
|
|
31
33
|
npx ocs-stats display-xp 35 "Fixed high issue"
|
|
34
|
+
npx ocs-stats display-xp 80 "Wrote 8 unit tests [testing]"
|
|
32
35
|
`);
|
|
33
36
|
process.exit(0);
|
|
34
37
|
}
|
|
@@ -42,14 +45,17 @@ if (command === 'update') {
|
|
|
42
45
|
}
|
|
43
46
|
|
|
44
47
|
if (command === 'stats') {
|
|
45
|
-
|
|
48
|
+
const category = args[1] || 'security';
|
|
49
|
+
stats(category);
|
|
46
50
|
process.exit(0);
|
|
47
51
|
}
|
|
48
52
|
|
|
49
53
|
if (command === 'display-xp') {
|
|
50
54
|
const amount = args[1];
|
|
51
|
-
const reason = args
|
|
52
|
-
|
|
55
|
+
const reason = args.slice(2).join(' ') || 'XP earned';
|
|
56
|
+
const category = reason.includes('[testing]') ? 'testing' : 'security';
|
|
57
|
+
const cleanReason = reason.replace(/\[testing\]/g, '').trim();
|
|
58
|
+
displayXp(amount, cleanReason, category);
|
|
53
59
|
process.exit(0);
|
|
54
60
|
}
|
|
55
61
|
|
package/package.json
CHANGED
package/src/display.js
CHANGED
|
@@ -44,7 +44,7 @@ function emptyLine() {
|
|
|
44
44
|
return '║' + ' '.repeat(CONTENT_WIDTH) + '║';
|
|
45
45
|
}
|
|
46
46
|
|
|
47
|
-
export function showXpGain(amount, reason, data) {
|
|
47
|
+
export function showXpGain(amount, reason, data, category = 'security') {
|
|
48
48
|
const { xp, level, title } = data;
|
|
49
49
|
const nextLevelXp = getNextLevelXp(level);
|
|
50
50
|
|
|
@@ -69,20 +69,21 @@ export function showXpGain(amount, reason, data) {
|
|
|
69
69
|
console.log('');
|
|
70
70
|
}
|
|
71
71
|
|
|
72
|
-
export function showStats(data) {
|
|
73
|
-
const { xp, level, title, issuesFixed, totalAudits, patternsAdded, mistakes } = data;
|
|
72
|
+
export function showStats(data, category = 'security') {
|
|
73
|
+
const { xp, level, title, issuesFixed, totalAudits, patternsAdded, mistakes, testsWritten, testsFixed, totalTests } = data;
|
|
74
74
|
const nextLevelXp = getNextLevelXp(level);
|
|
75
75
|
|
|
76
|
-
const
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
76
|
+
const categoryTitle = category === 'testing' ? 'TESTING AGENT' : 'SECURITY AGENT';
|
|
77
|
+
|
|
78
|
+
const totalFixed = category === 'security'
|
|
79
|
+
? ((issuesFixed?.critical || 0) + (issuesFixed?.high || 0) + (issuesFixed?.medium || 0) + (issuesFixed?.low || 0))
|
|
80
|
+
: (testsFixed || 0);
|
|
80
81
|
|
|
81
82
|
const totalPenalty = mistakes?.totalPenaltyXP || 0;
|
|
82
83
|
|
|
83
84
|
console.log('');
|
|
84
85
|
console.log(topBorder());
|
|
85
|
-
console.log(line(
|
|
86
|
+
console.log(line(` ${categoryTitle}`));
|
|
86
87
|
console.log(divider());
|
|
87
88
|
console.log(line(` Level ${level} - ${title}`));
|
|
88
89
|
|
|
@@ -98,10 +99,21 @@ export function showStats(data) {
|
|
|
98
99
|
|
|
99
100
|
console.log(emptyLine());
|
|
100
101
|
console.log(line(' Stats:'));
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
102
|
+
|
|
103
|
+
if (category === 'testing') {
|
|
104
|
+
console.log(line(` * Tests Written: ${totalTests || 0}`));
|
|
105
|
+
console.log(line(` * Unit Tests: ${testsWritten?.unit || 0}`));
|
|
106
|
+
console.log(line(` * Integration: ${testsWritten?.integration || 0}`));
|
|
107
|
+
console.log(line(` * E2E Tests: ${testsWritten?.e2e || 0}`));
|
|
108
|
+
console.log(line(` * Tests Fixed: ${testsFixed || 0}`));
|
|
109
|
+
console.log(line(` * Patterns Added: ${patternsAdded || 0}`));
|
|
110
|
+
} else {
|
|
111
|
+
console.log(line(` * Issues Fixed: ${totalFixed}`));
|
|
112
|
+
console.log(line(` * Audits Done: ${totalAudits || 0}`));
|
|
113
|
+
console.log(line(` * Patterns Added: ${patternsAdded || 0}`));
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
console.log(line(` * XP Penalties: ${totalPenalty}`));
|
|
105
117
|
console.log(bottomBorder());
|
|
106
118
|
console.log('');
|
|
107
119
|
}
|
package/src/init.js
CHANGED
|
@@ -30,9 +30,10 @@ export async function init({ isGlobal = false } = {}) {
|
|
|
30
30
|
|
|
31
31
|
console.log('Installed successfully!\n');
|
|
32
32
|
console.log('What was installed:');
|
|
33
|
-
console.log(' * Agents: security');
|
|
34
|
-
console.log(' * Skills: commit, memories, mobile, security, webapp');
|
|
35
|
-
console.log(' * Security: XP tracking, knowledge base
|
|
33
|
+
console.log(' * Agents: security, testing');
|
|
34
|
+
console.log(' * Skills: commit, memories, mobile, security, testing, webapp');
|
|
35
|
+
console.log(' * Security: XP tracking, knowledge base');
|
|
36
|
+
console.log(' * Testing: XP tracking, knowledge base\n');
|
|
36
37
|
|
|
37
38
|
if (!isGlobal) {
|
|
38
39
|
console.log('Next steps:');
|
|
@@ -64,9 +65,10 @@ export async function update({ isGlobal = false } = {}) {
|
|
|
64
65
|
|
|
65
66
|
console.log('Updated successfully!\n');
|
|
66
67
|
console.log('What was installed:');
|
|
67
|
-
console.log(' * Agents: security');
|
|
68
|
-
console.log(' * Skills: commit, memories, mobile, security, webapp');
|
|
69
|
-
console.log(' * Security: XP tracking, knowledge base
|
|
68
|
+
console.log(' * Agents: security, testing');
|
|
69
|
+
console.log(' * Skills: commit, memories, mobile, security, testing, webapp');
|
|
70
|
+
console.log(' * Security: XP tracking, knowledge base');
|
|
71
|
+
console.log(' * Testing: XP tracking, knowledge base\n');
|
|
70
72
|
}
|
|
71
73
|
|
|
72
74
|
function copyDir(src, dest) {
|
package/src/stats.js
CHANGED
|
@@ -2,8 +2,8 @@ import fs from 'fs';
|
|
|
2
2
|
import path from 'path';
|
|
3
3
|
import { showStats, showXpGain } from './display.js';
|
|
4
4
|
|
|
5
|
-
function findXpJson() {
|
|
6
|
-
const cwdPath = path.join(process.cwd(), '.opencode',
|
|
5
|
+
function findXpJson(category = 'security') {
|
|
6
|
+
const cwdPath = path.join(process.cwd(), '.opencode', category, 'xp.json');
|
|
7
7
|
|
|
8
8
|
if (fs.existsSync(cwdPath)) {
|
|
9
9
|
return cwdPath;
|
|
@@ -12,12 +12,12 @@ function findXpJson() {
|
|
|
12
12
|
return null;
|
|
13
13
|
}
|
|
14
14
|
|
|
15
|
-
export function stats() {
|
|
16
|
-
const xpPath = findXpJson();
|
|
15
|
+
export function stats(category = 'security') {
|
|
16
|
+
const xpPath = findXpJson(category);
|
|
17
17
|
|
|
18
18
|
if (!xpPath) {
|
|
19
19
|
console.log('');
|
|
20
|
-
console.log(
|
|
20
|
+
console.log(` No .opencode/${category}/xp.json found in this project.`);
|
|
21
21
|
console.log('');
|
|
22
22
|
console.log(' Make sure you\'re in a project with opencode-skills installed.');
|
|
23
23
|
console.log(' Run: npx create-opencode-skills');
|
|
@@ -26,11 +26,11 @@ export function stats() {
|
|
|
26
26
|
}
|
|
27
27
|
|
|
28
28
|
const data = JSON.parse(fs.readFileSync(xpPath, 'utf-8'));
|
|
29
|
-
showStats(data);
|
|
29
|
+
showStats(data, category);
|
|
30
30
|
}
|
|
31
31
|
|
|
32
|
-
export function displayXp(amount, reason) {
|
|
33
|
-
const xpPath = findXpJson();
|
|
32
|
+
export function displayXp(amount, reason, category = 'security') {
|
|
33
|
+
const xpPath = findXpJson(category);
|
|
34
34
|
|
|
35
35
|
if (!xpPath) {
|
|
36
36
|
console.log('No xp.json found');
|
|
@@ -44,5 +44,5 @@ export function displayXp(amount, reason) {
|
|
|
44
44
|
|
|
45
45
|
const data = JSON.parse(fs.readFileSync(xpPath, 'utf-8'));
|
|
46
46
|
const reasonText = reason || 'XP earned';
|
|
47
|
-
showXpGain(parseInt(amount, 10), reasonText, data);
|
|
47
|
+
showXpGain(parseInt(amount, 10), reasonText, data, category);
|
|
48
48
|
}
|
|
@@ -0,0 +1,303 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Testing expert agent for writing unit, integration, and E2E tests
|
|
3
|
+
mode: subagent
|
|
4
|
+
tools:
|
|
5
|
+
write: true
|
|
6
|
+
edit: true
|
|
7
|
+
bash: true
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
You are a testing expert agent specialized in writing, fixing, and improving tests. You have a leveling system that tracks your experience and growth.
|
|
11
|
+
|
|
12
|
+
## Current Status
|
|
13
|
+
|
|
14
|
+
Your current status is stored in `.opencode/testing/xp.json`:
|
|
15
|
+
- Level: {READ from .opencode/testing/xp.json}
|
|
16
|
+
- XP: {READ from .opencode/testing/xp.json}
|
|
17
|
+
- Title: {READ from .opencode/testing/xp.json}
|
|
18
|
+
|
|
19
|
+
## Level System
|
|
20
|
+
|
|
21
|
+
### XP Awards
|
|
22
|
+
|
|
23
|
+
| Action | XP |
|
|
24
|
+
|--------|-----|
|
|
25
|
+
| Write passing unit test | +10 XP |
|
|
26
|
+
| Write passing integration test | +15 XP |
|
|
27
|
+
| Write passing E2E test | +20 XP |
|
|
28
|
+
| Fix broken/flaky test | +10 XP |
|
|
29
|
+
| Add new test pattern to skill | +30 XP |
|
|
30
|
+
| Complete test suite (single file) | +20 XP |
|
|
31
|
+
| Complete test suite (package) | +100 XP |
|
|
32
|
+
|
|
33
|
+
### Deduplication
|
|
34
|
+
|
|
35
|
+
- Same pattern in multiple tests: **80% XP reduction** (only 20% XP awarded)
|
|
36
|
+
- Track seen patterns in `.opencode/testing/xp.json` under `seenPatterns`
|
|
37
|
+
|
|
38
|
+
### Penalty System
|
|
39
|
+
|
|
40
|
+
| Mistake | XP Penalty |
|
|
41
|
+
|---------|------------|
|
|
42
|
+
| Introduce flaky test | **-25 XP** |
|
|
43
|
+
| Repeat a previous mistake | **-15 XP** |
|
|
44
|
+
|
|
45
|
+
### Mistake Tracking
|
|
46
|
+
|
|
47
|
+
All mistakes are recorded in:
|
|
48
|
+
- `xp.json` → `mistakes` object and `mistakeHistory` array
|
|
49
|
+
- `knowledge.md` → `## Lessons Learned` section
|
|
50
|
+
|
|
51
|
+
**Before writing tests, ALWAYS check `Lessons Learned` to avoid repeating mistakes.**
|
|
52
|
+
|
|
53
|
+
### Level Thresholds
|
|
54
|
+
|
|
55
|
+
| Level | Title | XP Required | Focus |
|
|
56
|
+
|-------|-------|-------------|-------|
|
|
57
|
+
| 1 | Novice | 0 | Basic unit tests |
|
|
58
|
+
| 2 | Apprentice | 100 | Integration tests |
|
|
59
|
+
| 3 | Practitioner | 300 | E2E tests |
|
|
60
|
+
| 4 | Expert | 600 | Test patterns & mocking |
|
|
61
|
+
| 5 | Master | 1200 | Full coverage strategies |
|
|
62
|
+
| 6 | Grandmaster | 2500 | Testing excellence |
|
|
63
|
+
|
|
64
|
+
## Level-Specific Focus
|
|
65
|
+
|
|
66
|
+
### Level 1 - Novice (Current)
|
|
67
|
+
Focus on:
|
|
68
|
+
- Basic unit tests with AAA pattern
|
|
69
|
+
- Simple function testing
|
|
70
|
+
- Common matchers
|
|
71
|
+
|
|
72
|
+
### Level 2 - Apprentice (100 XP)
|
|
73
|
+
Adds:
|
|
74
|
+
- Integration tests
|
|
75
|
+
- API testing with mocked context
|
|
76
|
+
- Database testing patterns
|
|
77
|
+
|
|
78
|
+
### Level 3 - Practitioner (300 XP)
|
|
79
|
+
Adds:
|
|
80
|
+
- E2E testing with Playwright
|
|
81
|
+
- Browser automation
|
|
82
|
+
- User flow testing
|
|
83
|
+
|
|
84
|
+
### Level 4 - Expert (600 XP)
|
|
85
|
+
Adds:
|
|
86
|
+
- Advanced mocking patterns
|
|
87
|
+
- Test utilities and factories
|
|
88
|
+
- Test organization
|
|
89
|
+
|
|
90
|
+
### Level 5 - Master (1200 XP)
|
|
91
|
+
Adds:
|
|
92
|
+
- Coverage strategies
|
|
93
|
+
- Flaky test prevention
|
|
94
|
+
- Performance testing
|
|
95
|
+
|
|
96
|
+
### Level 6 - Grandmaster (2500 XP)
|
|
97
|
+
Adds:
|
|
98
|
+
- Testing architecture
|
|
99
|
+
- CI/CD integration
|
|
100
|
+
- Custom test frameworks
|
|
101
|
+
|
|
102
|
+
## Available Resources
|
|
103
|
+
|
|
104
|
+
You have access to:
|
|
105
|
+
- `.opencode/skills/testing/SKILL.md` - Core testing patterns
|
|
106
|
+
- `.opencode/testing/xp.json` - Your XP and level
|
|
107
|
+
- `.opencode/testing/knowledge.md` - Accumulated testing knowledge
|
|
108
|
+
- `opencode.json` - MCP configuration (created on-demand when needed)
|
|
109
|
+
|
|
110
|
+
## Playwright Integration
|
|
111
|
+
|
|
112
|
+
### MCP Setup Flow
|
|
113
|
+
|
|
114
|
+
When user asks for E2E tests or browser automation:
|
|
115
|
+
|
|
116
|
+
1. **Check if opencode.json exists:**
|
|
117
|
+
- Try to read `opencode.json` from project root
|
|
118
|
+
|
|
119
|
+
2. **If opencode.json does NOT exist, create it with Playwright MCP:**
|
|
120
|
+
```json
|
|
121
|
+
{
|
|
122
|
+
"$schema": "https://opencode.ai/config.json",
|
|
123
|
+
"mcp": {
|
|
124
|
+
"playwright": {
|
|
125
|
+
"type": "local",
|
|
126
|
+
"command": ["npx", "-y", "@playwright/mcp-server"],
|
|
127
|
+
"enabled": true
|
|
128
|
+
}
|
|
129
|
+
}
|
|
130
|
+
}
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
3. **If opencode.json exists but NO Playwright MCP, add it:**
|
|
134
|
+
- Read existing opencode.json
|
|
135
|
+
- Add playwright to mcp section
|
|
136
|
+
- Write back the updated config
|
|
137
|
+
|
|
138
|
+
4. **If already configured, skip setup**
|
|
139
|
+
|
|
140
|
+
5. **Prompt user:**
|
|
141
|
+
```
|
|
142
|
+
Enable Playwright for E2E testing? This will:
|
|
143
|
+
- Create opencode.json with Playwright MCP (for AI-driven browser automation)
|
|
144
|
+
- Install @playwright/test in your project (for CLI tests)
|
|
145
|
+
- Run: npx playwright install (browser binaries)
|
|
146
|
+
|
|
147
|
+
This enables both AI-assisted testing and direct Playwright usage.
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
6. **If user agrees, install dependencies:**
|
|
151
|
+
```bash
|
|
152
|
+
npm install -D @playwright/test
|
|
153
|
+
npx playwright install
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
7. **Restart OpenCode** to load the MCP server
|
|
157
|
+
|
|
158
|
+
8. **If already available, use directly:**
|
|
159
|
+
- MCP provides browser automation tools
|
|
160
|
+
- Also use `@playwright/test` for CLI test runs
|
|
161
|
+
|
|
162
|
+
## Workflow
|
|
163
|
+
|
|
164
|
+
1. **Read your current status**: Read `.opencode/testing/xp.json` to know your level
|
|
165
|
+
2. **Read knowledge base**: Check `.opencode/testing/knowledge.md` for known patterns AND lessons learned
|
|
166
|
+
3. **Detect frameworks**: Check package.json for test framework (Vitest, Jest, Playwright)
|
|
167
|
+
4. **Identify code to test**: Look at unimplemented test files or code needing coverage
|
|
168
|
+
5. **Write tests**: Apply patterns from `.opencode/skills/testing/SKILL.md`
|
|
169
|
+
6. **Run tests**: Verify tests pass before claiming XP
|
|
170
|
+
7. **Check Playwright need**: If E2E needed, follow MCP setup flow above
|
|
171
|
+
8. **Award XP**: Only after tests pass successfully
|
|
172
|
+
9. **Update knowledge**: Add any new patterns discovered
|
|
173
|
+
10. **If mistake made**: Record in `mistakeHistory` (xp.json) and `Lessons Learned` (knowledge.md)
|
|
174
|
+
|
|
175
|
+
## Test Execution
|
|
176
|
+
|
|
177
|
+
### Run Tests
|
|
178
|
+
|
|
179
|
+
```bash
|
|
180
|
+
# Unit/Integration tests
|
|
181
|
+
npm test
|
|
182
|
+
# or
|
|
183
|
+
npm run test:watch
|
|
184
|
+
|
|
185
|
+
# Playwright E2E
|
|
186
|
+
npx playwright test
|
|
187
|
+
|
|
188
|
+
# Specific file
|
|
189
|
+
npx playwright test tests/login.e2e.ts
|
|
190
|
+
|
|
191
|
+
# With UI
|
|
192
|
+
npx playwright test --ui
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
### Debug Failed Tests
|
|
196
|
+
|
|
197
|
+
```bash
|
|
198
|
+
# Show console logs
|
|
199
|
+
npx playwright test --reporter=line
|
|
200
|
+
|
|
201
|
+
# Debug mode
|
|
202
|
+
npx playwright test --debug
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
## Output Format
|
|
206
|
+
|
|
207
|
+
### Test Writing Phase (No XP Yet)
|
|
208
|
+
|
|
209
|
+
```
|
|
210
|
+
## Test Plan
|
|
211
|
+
|
|
212
|
+
### Files to Create/Modify
|
|
213
|
+
|
|
214
|
+
1. `src/utils/__tests__/calculate.test.ts`
|
|
215
|
+
- Test: calculateTotal with tax
|
|
216
|
+
- Test: calculateTotal with discounts
|
|
217
|
+
- Test: calculateTotal edge cases
|
|
218
|
+
|
|
219
|
+
2. `src/utils/__tests__/format.test.ts`
|
|
220
|
+
- Test: formatCurrency
|
|
221
|
+
- Test: formatDate
|
|
222
|
+
|
|
223
|
+
### Estimated Tests
|
|
224
|
+
- Unit tests: 8
|
|
225
|
+
- Expected XP: 80 XP (after passing)
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
### After Tests Pass
|
|
229
|
+
|
|
230
|
+
```
|
|
231
|
+
## Test Results
|
|
232
|
+
|
|
233
|
+
### Tests Created
|
|
234
|
+
- `src/utils/__tests__/calculate.test.ts` - 5 tests
|
|
235
|
+
- `src/utils/__tests__/format.test.ts` - 3 tests
|
|
236
|
+
|
|
237
|
+
### XP Earned
|
|
238
|
+
- Unit tests: 8 × 10 XP = 80 XP
|
|
239
|
+
- Total: 80 XP
|
|
240
|
+
|
|
241
|
+
### Level Progress
|
|
242
|
+
- Current: Level 1 - Novice
|
|
243
|
+
- XP: 80 / 100
|
|
244
|
+
- Next: Level 2 (Apprentice) at 100 XP
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Display XP gain:
|
|
248
|
+
```bash
|
|
249
|
+
npx ocs-stats display-xp 80 "Wrote 8 unit tests [testing]"
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
## Mistake Recording
|
|
253
|
+
|
|
254
|
+
If you introduce a flaky test or make a mistake:
|
|
255
|
+
|
|
256
|
+
### 1. Record in xp.json
|
|
257
|
+
|
|
258
|
+
Update the `mistakes` object and add to `mistakeHistory`:
|
|
259
|
+
|
|
260
|
+
```json
|
|
261
|
+
{
|
|
262
|
+
"mistakes": {
|
|
263
|
+
"flakyTests": 1,
|
|
264
|
+
"repeatedMistakes": 0,
|
|
265
|
+
"totalPenaltyXP": -25
|
|
266
|
+
},
|
|
267
|
+
"mistakeHistory": [
|
|
268
|
+
{
|
|
269
|
+
"date": "2025-02-25",
|
|
270
|
+
"type": "flaky_test",
|
|
271
|
+
"description": "Test depends on random value without seed",
|
|
272
|
+
"file": "src/utils/random.test.ts:15",
|
|
273
|
+
"xpPenalty": -25,
|
|
274
|
+
"lesson": "Always seed random values or use deterministic test data"
|
|
275
|
+
}
|
|
276
|
+
]
|
|
277
|
+
}
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
### 2. Record in knowledge.md
|
|
281
|
+
|
|
282
|
+
Add to `## Lessons Learned` table:
|
|
283
|
+
|
|
284
|
+
| Date | Mistake | Severity | Lesson Learned | Fixed In |
|
|
285
|
+
|------|---------|----------|----------------|----------|
|
|
286
|
+
| 2025-02-25 | Test depends on random value | High | Always seed random values or use deterministic test data | src/utils/random.test.ts:15 |
|
|
287
|
+
|
|
288
|
+
### 3. Before Writing Similar Tests
|
|
289
|
+
|
|
290
|
+
Always check `mistakeHistory` and `Lessons Learned` to ensure you're not repeating a pattern that caused issues before.
|
|
291
|
+
|
|
292
|
+
## Important Rules
|
|
293
|
+
|
|
294
|
+
1. ALWAYS read your current level from `.opencode/testing/xp.json` at the start
|
|
295
|
+
2. NEVER award XP for failing tests - only for passing ones
|
|
296
|
+
3. ALWAYS check `.opencode/testing/knowledge.md` for duplicates before claiming XP
|
|
297
|
+
4. ALWAYS check `Lessons Learned` before writing tests to avoid repeating mistakes
|
|
298
|
+
5. ALWAYS run tests to verify they pass before claiming XP
|
|
299
|
+
6. For E2E tests, follow the Playwright MCP setup flow
|
|
300
|
+
7. NEVER write flaky tests (random values, timing-dependent, external APIs)
|
|
301
|
+
8. ALWAYS use deterministic test data
|
|
302
|
+
9. Record mistakes in both `xp.json` and `knowledge.md` if you introduce a flaky test
|
|
303
|
+
10. Repeated mistakes incur additional -15 XP penalty
|
|
@@ -0,0 +1,499 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: testing
|
|
3
|
+
description: Testing patterns, frameworks, and best practices for unit, integration, and E2E tests
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Testing Patterns
|
|
7
|
+
|
|
8
|
+
## Framework Detection
|
|
9
|
+
|
|
10
|
+
| Indicator | Framework |
|
|
11
|
+
|-----------|-----------|
|
|
12
|
+
| `vitest` in package.json | Vitest |
|
|
13
|
+
| `jest` in package.json | Jest |
|
|
14
|
+
| `@testing-library/react` | React Testing Library |
|
|
15
|
+
| `@testing-library/vue` | Vue Testing Library |
|
|
16
|
+
| `@playwright/test` | Playwright |
|
|
17
|
+
| `cypress` | Cypress |
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## 1. Unit Testing (Vitest/Jest)
|
|
22
|
+
|
|
23
|
+
### AAA Pattern
|
|
24
|
+
|
|
25
|
+
```typescript
|
|
26
|
+
describe('calculateTotal', () => {
|
|
27
|
+
it('should calculate total with tax', () => {
|
|
28
|
+
// Arrange
|
|
29
|
+
const items = [{ price: 100 }, { price: 50 }];
|
|
30
|
+
const taxRate = 0. Act
|
|
31
|
+
const1;
|
|
32
|
+
|
|
33
|
+
// result = calculateTotal(items, taxRate);
|
|
34
|
+
|
|
35
|
+
// Assert
|
|
36
|
+
expect(result).toBe(165);
|
|
37
|
+
});
|
|
38
|
+
});
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
### Test File Naming
|
|
42
|
+
|
|
43
|
+
| Type | Pattern | Example |
|
|
44
|
+
|------|---------|---------|
|
|
45
|
+
| Unit | `*.test.ts` or `*.spec.ts` | `utils.test.ts` |
|
|
46
|
+
| Component | `*.test.tsx` or `*.spec.tsx` | `Button.test.tsx` |
|
|
47
|
+
| Integration | `*.integration.test.ts` | `api.integration.test.ts` |
|
|
48
|
+
| E2E | `*.e2e.test.ts` | `login.e2e.test.ts` |
|
|
49
|
+
|
|
50
|
+
### Common Matchers
|
|
51
|
+
|
|
52
|
+
```typescript
|
|
53
|
+
// Equality
|
|
54
|
+
expect(value).toBe(42);
|
|
55
|
+
expect(value).toEqual({ name: 'test' });
|
|
56
|
+
|
|
57
|
+
// Truthiness
|
|
58
|
+
expect(value).toBeTruthy();
|
|
59
|
+
expect(value).toBeFalsy();
|
|
60
|
+
expect(value).toBeNull();
|
|
61
|
+
expect(value).toBeUndefined();
|
|
62
|
+
|
|
63
|
+
// Numbers
|
|
64
|
+
expect(value).toBeGreaterThan(10);
|
|
65
|
+
expect(value).toBeLessThanOrEqual(100);
|
|
66
|
+
expect(value).toBeCloseTo(3.14, 2);
|
|
67
|
+
|
|
68
|
+
// Strings
|
|
69
|
+
expect(text).toMatch(/regex/);
|
|
70
|
+
expect(text).toContain('substring');
|
|
71
|
+
|
|
72
|
+
// Arrays
|
|
73
|
+
expect(array).toContain(item);
|
|
74
|
+
expect(array).toHaveLength(3);
|
|
75
|
+
|
|
76
|
+
// Objects
|
|
77
|
+
expect(obj).toHaveProperty('key');
|
|
78
|
+
expect(obj).toMatchObject({ key: 'value' });
|
|
79
|
+
|
|
80
|
+
// Exceptions
|
|
81
|
+
expect(() => throwError()).toThrow();
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
## 2. tRPC/API Testing
|
|
87
|
+
|
|
88
|
+
### Testing Routers with Context
|
|
89
|
+
|
|
90
|
+
```typescript
|
|
91
|
+
import { appRouter } from '../src/routers/_app';
|
|
92
|
+
import { createTRPCContext } from '../src/server/trpc';
|
|
93
|
+
|
|
94
|
+
describe('userRouter', () => {
|
|
95
|
+
const createMockContext = (overrides = {}) => {
|
|
96
|
+
return createTRPCContext({
|
|
97
|
+
req: {} as Request,
|
|
98
|
+
res: {} as Response,
|
|
99
|
+
...overrides,
|
|
100
|
+
});
|
|
101
|
+
};
|
|
102
|
+
|
|
103
|
+
it('should get user profile', async () => {
|
|
104
|
+
const ctx = createMockContext();
|
|
105
|
+
const caller = appRouter.createCaller(ctx);
|
|
106
|
+
|
|
107
|
+
const result = await caller.user.getProfile({ userId: '123' });
|
|
108
|
+
|
|
109
|
+
expect(result).toHaveProperty('id');
|
|
110
|
+
});
|
|
111
|
+
|
|
112
|
+
it('should throw on invalid input', async () => {
|
|
113
|
+
const ctx = createMockContext();
|
|
114
|
+
const caller = appRouter.createCaller(ctx);
|
|
115
|
+
|
|
116
|
+
await expect(
|
|
117
|
+
caller.user.getProfile({ userId: '' })
|
|
118
|
+
).rejects.toThrow();
|
|
119
|
+
});
|
|
120
|
+
});
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
### Testing Input Validation
|
|
124
|
+
|
|
125
|
+
```typescript
|
|
126
|
+
it('should reject invalid email', async () => {
|
|
127
|
+
const ctx = createMockContext();
|
|
128
|
+
const caller = appRouter.createCaller(ctx);
|
|
129
|
+
|
|
130
|
+
await expect(
|
|
131
|
+
caller.user.create({
|
|
132
|
+
email: 'not-an-email',
|
|
133
|
+
username: 'test'
|
|
134
|
+
})
|
|
135
|
+
).rejects.toThrow('Invalid email');
|
|
136
|
+
});
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
### Testing Error Handling
|
|
140
|
+
|
|
141
|
+
```typescript
|
|
142
|
+
it('should throw NOT_FOUND for non-existent user', async () => {
|
|
143
|
+
const ctx = createMockContext();
|
|
144
|
+
const caller = appRouter.createCaller(ctx);
|
|
145
|
+
|
|
146
|
+
await expect(
|
|
147
|
+
caller.user.getProfile({ userId: 'non-existent' })
|
|
148
|
+
).rejects.toMatchObject({
|
|
149
|
+
code: 'NOT_FOUND',
|
|
150
|
+
message: expect.stringContaining('not found')
|
|
151
|
+
});
|
|
152
|
+
});
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## 3. React Component Testing (Testing Library)
|
|
158
|
+
|
|
159
|
+
### Query Priority
|
|
160
|
+
|
|
161
|
+
Use queries in this order (most to least preferred):
|
|
162
|
+
|
|
163
|
+
1. **`getByRole`** - Most accessible
|
|
164
|
+
```typescript
|
|
165
|
+
expect(screen.getByRole('button', { name: /submit/i })).toBeDisabled();
|
|
166
|
+
expect(screen.getByRole('textbox', { name: /email/i })).toHaveValue('test@test.com');
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
2. **`getByLabelText`** - For form fields
|
|
170
|
+
```typescript
|
|
171
|
+
expect(screen.getByLabelText(/email/i)).toHaveValue('test@test.com');
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
3. **`getByPlaceholderText`** - If no label
|
|
175
|
+
```typescript
|
|
176
|
+
screen.getByPlaceholderText('Enter your email');
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
4. **`getByText`** - For non-interactive elements
|
|
180
|
+
```typescript
|
|
181
|
+
expect(screen.getByText(/welcome back/i)).toBeInTheDocument();
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
5. **`getByTestId`** - Last resort
|
|
185
|
+
```typescript
|
|
186
|
+
<div data-testid="custom-element" />
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
### User Events (Preferred over fireEvent)
|
|
190
|
+
|
|
191
|
+
```typescript
|
|
192
|
+
import userEvent from '@testing-library/user-event';
|
|
193
|
+
|
|
194
|
+
it('should submit form', async () => {
|
|
195
|
+
const user = userEvent.setup();
|
|
196
|
+
|
|
197
|
+
await user.type(screen.getByLabelText(/email/i), 'test@test.com');
|
|
198
|
+
await user.click(screen.getByRole('button', { name: /submit/i }));
|
|
199
|
+
|
|
200
|
+
expect(screen.getByText(/submitted/i)).toBeInTheDocument();
|
|
201
|
+
});
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
### Testing Forms
|
|
205
|
+
|
|
206
|
+
```typescript
|
|
207
|
+
it('should show validation errors', async () => {
|
|
208
|
+
const user = userEvent.setup();
|
|
209
|
+
|
|
210
|
+
const submitButton = screen.getByRole('button', { name: /submit/i });
|
|
211
|
+
await user.click(submitButton);
|
|
212
|
+
|
|
213
|
+
expect(screen.getByText(/email is required/i)).toBeInTheDocument();
|
|
214
|
+
expect(submitButton).toBeDisabled();
|
|
215
|
+
});
|
|
216
|
+
|
|
217
|
+
it('should submit with valid data', async () => {
|
|
218
|
+
const user = userEvent.setup();
|
|
219
|
+
const onSubmit = vi.fn();
|
|
220
|
+
|
|
221
|
+
await user.type(screen.getByLabelText(/email/i), 'test@test.com');
|
|
222
|
+
await user.click(screen.getByRole('button', { name: /submit/i }));
|
|
223
|
+
|
|
224
|
+
expect(onSubmit).toHaveBeenCalledWith({
|
|
225
|
+
email: 'test@test.com'
|
|
226
|
+
});
|
|
227
|
+
});
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
### Testing Async Components
|
|
231
|
+
|
|
232
|
+
```typescript
|
|
233
|
+
it('should show loading then data', async () => {
|
|
234
|
+
render(<UserProfile userId="123" />);
|
|
235
|
+
|
|
236
|
+
expect(screen.getByText(/loading/i)).toBeInTheDocument();
|
|
237
|
+
|
|
238
|
+
await waitFor(() => {
|
|
239
|
+
expect(screen.getByText(/john doe/i)).toBeInTheDocument();
|
|
240
|
+
});
|
|
241
|
+
});
|
|
242
|
+
|
|
243
|
+
it('should handle error state', async () => {
|
|
244
|
+
server.use(...mockErrorResponse);
|
|
245
|
+
|
|
246
|
+
render(<UserProfile userId="123" />);
|
|
247
|
+
|
|
248
|
+
await waitFor(() => {
|
|
249
|
+
expect(screen.getByText(/error loading user/i)).toBeInTheDocument();
|
|
250
|
+
});
|
|
251
|
+
});
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
---
|
|
255
|
+
|
|
256
|
+
## 4. Playwright E2E Testing
|
|
257
|
+
|
|
258
|
+
### Setup
|
|
259
|
+
|
|
260
|
+
```typescript
|
|
261
|
+
import { test, expect } from '@playwright/test';
|
|
262
|
+
|
|
263
|
+
test.describe('Login Flow', () => {
|
|
264
|
+
test.beforeEach(async ({ page }) => {
|
|
265
|
+
await page.goto('/login');
|
|
266
|
+
});
|
|
267
|
+
|
|
268
|
+
test('should login successfully', async ({ page }) => {
|
|
269
|
+
await page.fill('[name="email"]', 'test@example.com');
|
|
270
|
+
await page.fill('[name="password"]', 'password123');
|
|
271
|
+
await page.click('button[type="submit"]');
|
|
272
|
+
|
|
273
|
+
await expect(page).toHaveURL('/dashboard');
|
|
274
|
+
await expect(page.locator('[data-testid="welcome"]')).toContainText('Welcome');
|
|
275
|
+
});
|
|
276
|
+
|
|
277
|
+
test('should show error on invalid credentials', async ({ page }) => {
|
|
278
|
+
await page.fill('[name="email"]', 'wrong@example.com');
|
|
279
|
+
await page.fill('[name="password"]', 'wrongpass');
|
|
280
|
+
await page.click('button[type="submit"]');
|
|
281
|
+
|
|
282
|
+
await expect(page.locator('[role="alert"]')).toContainText('Invalid credentials');
|
|
283
|
+
});
|
|
284
|
+
});
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
### Playwright MCP Integration
|
|
288
|
+
|
|
289
|
+
When user needs E2E testing assistance:
|
|
290
|
+
|
|
291
|
+
1. **Check if Playwright MCP is configured:**
|
|
292
|
+
- Read `opencode.json` and check for `mcp.playwright` configuration
|
|
293
|
+
|
|
294
|
+
2. **If not configured, prompt user:**
|
|
295
|
+
```
|
|
296
|
+
Enable Playwright for E2E testing? This will:
|
|
297
|
+
- Add Playwright MCP to opencode.json (for AI-driven testing)
|
|
298
|
+
- Install @playwright/test as dev dependency
|
|
299
|
+
- Install browser binaries
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
3. **If user agrees, setup:**
|
|
303
|
+
```bash
|
|
304
|
+
# Add to opencode.json
|
|
305
|
+
npm install -D @playwright/test
|
|
306
|
+
npx playwright install
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
4. **If already available, use directly**
|
|
310
|
+
|
|
311
|
+
### Best Practices
|
|
312
|
+
|
|
313
|
+
```typescript
|
|
314
|
+
// ✅ GOOD - Use locators
|
|
315
|
+
const submitButton = page.getByRole('button', { name: /submit/i });
|
|
316
|
+
await submitButton.click();
|
|
317
|
+
|
|
318
|
+
// ✅ GOOD - Wait for assertions
|
|
319
|
+
await expect(page.locator('.data-loaded')).toBeVisible();
|
|
320
|
+
|
|
321
|
+
// ✅ GOOD - Use test hooks
|
|
322
|
+
test.beforeEach(async ({ page }) => {
|
|
323
|
+
await page.goto('/reset-state');
|
|
324
|
+
});
|
|
325
|
+
|
|
326
|
+
// ❌ BAD - Race conditions
|
|
327
|
+
await page.click('button');
|
|
328
|
+
await expect(page.locator('.success')).toBeVisible();
|
|
329
|
+
|
|
330
|
+
// ✅ GOOD - Proper waiting
|
|
331
|
+
await page.click('button');
|
|
332
|
+
await expect(page.locator('.success')).toBeVisible({ timeout: 10000 });
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
---
|
|
336
|
+
|
|
337
|
+
## 5. Mocking Patterns
|
|
338
|
+
|
|
339
|
+
### Mocking tRPC Procedures
|
|
340
|
+
|
|
341
|
+
```typescript
|
|
342
|
+
import { vi } from 'vitest';
|
|
343
|
+
|
|
344
|
+
vi.mock('../trpc', () => ({
|
|
345
|
+
createTRPCContext: vi.fn(() => ({
|
|
346
|
+
prisma: mockPrisma,
|
|
347
|
+
user: null,
|
|
348
|
+
})),
|
|
349
|
+
}));
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
### Mocking Prisma
|
|
353
|
+
|
|
354
|
+
```typescript
|
|
355
|
+
const mockPrisma = {
|
|
356
|
+
user: {
|
|
357
|
+
findUnique: vi.fn(),
|
|
358
|
+
create: vi.fn(),
|
|
359
|
+
update: vi.fn(),
|
|
360
|
+
delete: vi.fn(),
|
|
361
|
+
},
|
|
362
|
+
// ... other models
|
|
363
|
+
};
|
|
364
|
+
|
|
365
|
+
beforeEach(() => {
|
|
366
|
+
vi.clearAllMocks();
|
|
367
|
+
mockPrisma.user.findUnique.mockResolvedValue(null);
|
|
368
|
+
});
|
|
369
|
+
```
|
|
370
|
+
|
|
371
|
+
### Mocking External APIs
|
|
372
|
+
|
|
373
|
+
```typescript
|
|
374
|
+
import { http, HttpResponse } from 'msw';
|
|
375
|
+
import { setupServer } from 'msw/node';
|
|
376
|
+
|
|
377
|
+
const server = setupServer(
|
|
378
|
+
http.get('/api/users', () => {
|
|
379
|
+
return HttpResponse.json([
|
|
380
|
+
{ id: '1', name: 'John' }
|
|
381
|
+
]);
|
|
382
|
+
})
|
|
383
|
+
);
|
|
384
|
+
|
|
385
|
+
beforeAll(() => server.listen());
|
|
386
|
+
afterEach(() => server.resetHandlers());
|
|
387
|
+
afterAll(() => server.close());
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
---
|
|
391
|
+
|
|
392
|
+
## 6. Anti-Patterns
|
|
393
|
+
|
|
394
|
+
### Testing Implementation Details
|
|
395
|
+
|
|
396
|
+
```typescript
|
|
397
|
+
// ❌ BAD - Testing internal state
|
|
398
|
+
const instance = new MyClass();
|
|
399
|
+
instance['privateMethod']();
|
|
400
|
+
|
|
401
|
+
// ✅ GOOD - Testing behavior
|
|
402
|
+
expect(instance.calculate(2, 3)).toBe(5);
|
|
403
|
+
```
|
|
404
|
+
|
|
405
|
+
### Using fireEvent (Prefer userEvent)
|
|
406
|
+
|
|
407
|
+
```typescript
|
|
408
|
+
// ❌ BAD
|
|
409
|
+
fireEvent.change(input, { target: { value: 'test' } });
|
|
410
|
+
|
|
411
|
+
// ✅ GOOD
|
|
412
|
+
await userEvent.type(input, 'test');
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
### Index as Key
|
|
416
|
+
|
|
417
|
+
```typescript
|
|
418
|
+
// ❌ BAD
|
|
419
|
+
items.map((item, index) => <div key={index}>...</div>);
|
|
420
|
+
|
|
421
|
+
// ✅ GOOD
|
|
422
|
+
items.map(item => <div key={item.id}>...</div>);
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
### Missing Cleanup
|
|
426
|
+
|
|
427
|
+
```typescript
|
|
428
|
+
// ❌ BAD
|
|
429
|
+
it('test', () => {
|
|
430
|
+
const instance = new Class();
|
|
431
|
+
});
|
|
432
|
+
|
|
433
|
+
// ✅ GOOD
|
|
434
|
+
let instance;
|
|
435
|
+
beforeEach(() => instance = new Class());
|
|
436
|
+
afterEach(() => instance = null);
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
### Hardcoded Time/Dates
|
|
440
|
+
|
|
441
|
+
```typescript
|
|
442
|
+
// ❌ BAD
|
|
443
|
+
expect(new Date().toISOString()).toBe('2024-01-01T00:00:00.000Z');
|
|
444
|
+
|
|
445
|
+
// ✅ GOOD - Use fake timers
|
|
446
|
+
vi.useFakeTimers();
|
|
447
|
+
vi.setSystemTime(new Date('2024-01-01'));
|
|
448
|
+
```
|
|
449
|
+
|
|
450
|
+
---
|
|
451
|
+
|
|
452
|
+
## 7. Coverage Guidelines
|
|
453
|
+
|
|
454
|
+
### Prioritize Testing
|
|
455
|
+
|
|
456
|
+
| Priority | What to Test |
|
|
457
|
+
|----------|-------------|
|
|
458
|
+
| **High** | Business logic, calculations, transformations |
|
|
459
|
+
| **High** | Edge cases, boundary conditions |
|
|
460
|
+
| **High** | Error handling, exceptions |
|
|
461
|
+
| **Medium** | Component rendering, user interactions |
|
|
462
|
+
| **Medium** | API endpoints, integrations |
|
|
463
|
+
| **Low** | Simple getters/setters |
|
|
464
|
+
| **Low** | Boilerplate, types only |
|
|
465
|
+
|
|
466
|
+
### What NOT to Test
|
|
467
|
+
|
|
468
|
+
- TypeScript types (already enforced by compiler)
|
|
469
|
+
- Simple utility functions that just pass through
|
|
470
|
+
- Third-party library internals
|
|
471
|
+
- Implementation details
|
|
472
|
+
|
|
473
|
+
---
|
|
474
|
+
|
|
475
|
+
## 8. Fix-First Testing Process
|
|
476
|
+
|
|
477
|
+
### Process
|
|
478
|
+
|
|
479
|
+
1. Identify code that needs tests
|
|
480
|
+
2. Write tests (NO XP awarded yet)
|
|
481
|
+
3. Run tests to verify they pass
|
|
482
|
+
4. Document any patterns discovered
|
|
483
|
+
5. Award XP for passing tests
|
|
484
|
+
6. Update knowledge.md
|
|
485
|
+
|
|
486
|
+
### XP Awards
|
|
487
|
+
|
|
488
|
+
- **Unit test:** +10 XP
|
|
489
|
+
- **Integration test:** +15 XP
|
|
490
|
+
- **E2E test:** +20 XP
|
|
491
|
+
- **Fix broken test:** +10 XP
|
|
492
|
+
- **Add pattern to skill:** +30 XP
|
|
493
|
+
|
|
494
|
+
### Rules
|
|
495
|
+
|
|
496
|
+
- XP only awarded for PASSING tests
|
|
497
|
+
- Track seen patterns to avoid duplicate XP
|
|
498
|
+
- Record mistakes in knowledge.md
|
|
499
|
+
- Always run tests before claiming XP
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Testing Agent Knowledge Base
|
|
2
|
+
|
|
3
|
+
This file stores accumulated testing patterns, lessons learned, and test knowledge.
|
|
4
|
+
|
|
5
|
+
## Version History
|
|
6
|
+
|
|
7
|
+
- v1.0 (Level 1 - Novice): Initial knowledge base
|
|
8
|
+
- v1.1: Added Lessons Learned section for mistake tracking
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## Lessons Learned (Mistakes to Avoid)
|
|
13
|
+
|
|
14
|
+
This section tracks mistakes the agent has made to prevent repeating them.
|
|
15
|
+
|
|
16
|
+
| Date | Mistake | Severity | Lesson Learned | Fixed In |
|
|
17
|
+
|------|---------|----------|----------------|----------|
|
|
18
|
+
| _None yet_ | - | - | - | - |
|
|
19
|
+
|
|
20
|
+
### How to Use This Section
|
|
21
|
+
|
|
22
|
+
When the agent writes a flaky test or makes a testing mistake:
|
|
23
|
+
1. Record the mistake with date, description, and severity
|
|
24
|
+
2. Document the lesson learned (what should have been done)
|
|
25
|
+
3. Reference the file where it was fixed
|
|
26
|
+
4. **Always check this section before writing similar tests**
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Known Test Patterns
|
|
31
|
+
|
|
32
|
+
### Unit Testing Patterns
|
|
33
|
+
|
|
34
|
+
| Pattern | Description | Status | Example |
|
|
35
|
+
|---------|-------------|--------|---------|
|
|
36
|
+
| _None documented yet_ | - | - | - |
|
|
37
|
+
|
|
38
|
+
### Integration Testing Patterns
|
|
39
|
+
|
|
40
|
+
| Pattern | Description | Status | Example |
|
|
41
|
+
|---------|-------------|--------|---------|
|
|
42
|
+
| _None documented yet_ | - | - | - |
|
|
43
|
+
|
|
44
|
+
### E2E Testing Patterns
|
|
45
|
+
|
|
46
|
+
| Pattern | Description | Status | Example |
|
|
47
|
+
|---------|-------------|--------|---------|
|
|
48
|
+
| _None documented yet_ | - | - | - |
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## Fixes Applied
|
|
53
|
+
|
|
54
|
+
_No fixes documented yet._
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Test Suites Completed
|
|
59
|
+
|
|
60
|
+
_No test suites completed yet._
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Anti-Patterns to Avoid
|
|
65
|
+
|
|
66
|
+
| Anti-Pattern | Why Bad | Correct Approach |
|
|
67
|
+
|--------------|---------|------------------|
|
|
68
|
+
| _None documented yet_ | - | - |
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## Notes
|
|
73
|
+
|
|
74
|
+
- Level 1: Focus on basic unit test patterns
|
|
75
|
+
- Higher levels unlock integration and E2E testing
|
|
76
|
+
- XP awarded for writing tests AND fixing broken ones
|
|
77
|
+
- Mistakes tracked to prevent repetition
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
{
|
|
2
|
+
"xp": 0,
|
|
3
|
+
"level": 1,
|
|
4
|
+
"title": "Novice",
|
|
5
|
+
"totalTests": 0,
|
|
6
|
+
"testsWritten": {
|
|
7
|
+
"unit": 0,
|
|
8
|
+
"integration": 0,
|
|
9
|
+
"e2e": 0
|
|
10
|
+
},
|
|
11
|
+
"testsFixed": 0,
|
|
12
|
+
"patternsAdded": 0,
|
|
13
|
+
"completedSuites": [],
|
|
14
|
+
"seenPatterns": [],
|
|
15
|
+
"mistakes": {
|
|
16
|
+
"flakyTests": 0,
|
|
17
|
+
"repeatedMistakes": 0,
|
|
18
|
+
"totalPenaltyXP": 0
|
|
19
|
+
},
|
|
20
|
+
"mistakeHistory": [],
|
|
21
|
+
"levelHistory": [
|
|
22
|
+
{
|
|
23
|
+
"level": 1,
|
|
24
|
+
"title": "Novice",
|
|
25
|
+
"xpRequired": 0,
|
|
26
|
+
"unlockedAt": null
|
|
27
|
+
}
|
|
28
|
+
],
|
|
29
|
+
"xpTable": {
|
|
30
|
+
"tests": {
|
|
31
|
+
"unit": 10,
|
|
32
|
+
"integration": 15,
|
|
33
|
+
"e2e": 20
|
|
34
|
+
},
|
|
35
|
+
"fixes": {
|
|
36
|
+
"brokenTest": 10,
|
|
37
|
+
"flakyTest": 15
|
|
38
|
+
},
|
|
39
|
+
"knowledge": {
|
|
40
|
+
"addPattern": 30,
|
|
41
|
+
"documentPattern": 20
|
|
42
|
+
},
|
|
43
|
+
"completion": {
|
|
44
|
+
"singleFile": 20,
|
|
45
|
+
"package": 100
|
|
46
|
+
},
|
|
47
|
+
"penalty": {
|
|
48
|
+
"introduceFlaky": -25,
|
|
49
|
+
"repeatMistake": -15
|
|
50
|
+
}
|
|
51
|
+
},
|
|
52
|
+
"levelThresholds": [
|
|
53
|
+
{ "level": 1, "title": "Novice", "xpRequired": 0, "focus": "Basic unit tests" },
|
|
54
|
+
{ "level": 2, "title": "Apprentice", "xpRequired": 100, "focus": "Integration tests" },
|
|
55
|
+
{ "level": 3, "title": "Practitioner", "xpRequired": 300, "focus": "E2E tests" },
|
|
56
|
+
{ "level": 4, "title": "Expert", "xpRequired": 600, "focus": "Test patterns & mocking" },
|
|
57
|
+
{ "level": 5, "title": "Master", "xpRequired": 1200, "focus": "Full coverage strategies" },
|
|
58
|
+
{ "level": 6, "title": "Grandmaster", "xpRequired": 2500, "focus": "Testing excellence" }
|
|
59
|
+
]
|
|
60
|
+
}
|