claudecode-omc 4.7.4 → 4.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/README.md +50 -0
- package/agents/test-engineer.md +74 -0
- package/bridge/cli.cjs +9335 -117
- package/dist/cli/index.js +201 -0
- package/dist/cli/index.js.map +1 -1
- package/dist/testing/analyzers/complexity.d.ts +18 -0
- package/dist/testing/analyzers/complexity.d.ts.map +1 -0
- package/dist/testing/analyzers/complexity.js +121 -0
- package/dist/testing/analyzers/complexity.js.map +1 -0
- package/dist/testing/analyzers/coverage.d.ts +13 -0
- package/dist/testing/analyzers/coverage.d.ts.map +1 -0
- package/dist/testing/analyzers/coverage.js +99 -0
- package/dist/testing/analyzers/coverage.js.map +1 -0
- package/dist/testing/analyzers/quality-scorer.d.ts +8 -0
- package/dist/testing/analyzers/quality-scorer.d.ts.map +1 -0
- package/dist/testing/analyzers/quality-scorer.js +128 -0
- package/dist/testing/analyzers/quality-scorer.js.map +1 -0
- package/dist/testing/analyzers/types.d.ts +56 -0
- package/dist/testing/analyzers/types.d.ts.map +1 -0
- package/dist/testing/analyzers/types.js +2 -0
- package/dist/testing/analyzers/types.js.map +1 -0
- package/dist/testing/cli/agent-integration.d.ts +20 -0
- package/dist/testing/cli/agent-integration.d.ts.map +1 -0
- package/dist/testing/cli/agent-integration.js +60 -0
- package/dist/testing/cli/agent-integration.js.map +1 -0
- package/dist/testing/cli/commands.d.ts +100 -0
- package/dist/testing/cli/commands.d.ts.map +1 -0
- package/dist/testing/cli/commands.js +250 -0
- package/dist/testing/cli/commands.js.map +1 -0
- package/dist/testing/cli/ultraqa-integration.d.ts +13 -0
- package/dist/testing/cli/ultraqa-integration.d.ts.map +1 -0
- package/dist/testing/cli/ultraqa-integration.js +68 -0
- package/dist/testing/cli/ultraqa-integration.js.map +1 -0
- package/dist/testing/detectors/go.d.ts +3 -0
- package/dist/testing/detectors/go.d.ts.map +1 -0
- package/dist/testing/detectors/go.js +38 -0
- package/dist/testing/detectors/go.js.map +1 -0
- package/dist/testing/detectors/index.d.ts +8 -0
- package/dist/testing/detectors/index.d.ts.map +1 -0
- package/dist/testing/detectors/index.js +46 -0
- package/dist/testing/detectors/index.js.map +1 -0
- package/dist/testing/detectors/package-json.d.ts +3 -0
- package/dist/testing/detectors/package-json.d.ts.map +1 -0
- package/dist/testing/detectors/package-json.js +52 -0
- package/dist/testing/detectors/package-json.js.map +1 -0
- package/dist/testing/detectors/python.d.ts +3 -0
- package/dist/testing/detectors/python.d.ts.map +1 -0
- package/dist/testing/detectors/python.js +37 -0
- package/dist/testing/detectors/python.js.map +1 -0
- package/dist/testing/detectors/rust.d.ts +3 -0
- package/dist/testing/detectors/rust.d.ts.map +1 -0
- package/dist/testing/detectors/rust.js +39 -0
- package/dist/testing/detectors/rust.js.map +1 -0
- package/dist/testing/generators/contract.d.ts +14 -0
- package/dist/testing/generators/contract.d.ts.map +1 -0
- package/dist/testing/generators/contract.js +163 -0
- package/dist/testing/generators/contract.js.map +1 -0
- package/dist/testing/generators/e2e.d.ts +34 -0
- package/dist/testing/generators/e2e.d.ts.map +1 -0
- package/dist/testing/generators/e2e.js +74 -0
- package/dist/testing/generators/e2e.js.map +1 -0
- package/dist/testing/generators/go.d.ts +12 -0
- package/dist/testing/generators/go.d.ts.map +1 -0
- package/dist/testing/generators/go.js +144 -0
- package/dist/testing/generators/go.js.map +1 -0
- package/dist/testing/generators/nodejs.d.ts +12 -0
- package/dist/testing/generators/nodejs.d.ts.map +1 -0
- package/dist/testing/generators/nodejs.js +37 -0
- package/dist/testing/generators/nodejs.js.map +1 -0
- package/dist/testing/generators/python.d.ts +12 -0
- package/dist/testing/generators/python.d.ts.map +1 -0
- package/dist/testing/generators/python.js +163 -0
- package/dist/testing/generators/python.js.map +1 -0
- package/dist/testing/generators/react.d.ts +12 -0
- package/dist/testing/generators/react.d.ts.map +1 -0
- package/dist/testing/generators/react.js +31 -0
- package/dist/testing/generators/react.js.map +1 -0
- package/dist/testing/generators/rust.d.ts +11 -0
- package/dist/testing/generators/rust.d.ts.map +1 -0
- package/dist/testing/generators/rust.js +138 -0
- package/dist/testing/generators/rust.js.map +1 -0
- package/dist/testing/index.d.ts +6 -0
- package/dist/testing/index.d.ts.map +1 -0
- package/dist/testing/index.js +11 -0
- package/dist/testing/index.js.map +1 -0
- package/dist/testing/integrations/autopilot.d.ts +42 -0
- package/dist/testing/integrations/autopilot.d.ts.map +1 -0
- package/dist/testing/integrations/autopilot.js +55 -0
- package/dist/testing/integrations/autopilot.js.map +1 -0
- package/dist/testing/integrations/cicd.d.ts +26 -0
- package/dist/testing/integrations/cicd.d.ts.map +1 -0
- package/dist/testing/integrations/cicd.js +162 -0
- package/dist/testing/integrations/cicd.js.map +1 -0
- package/dist/testing/integrations/giskard/behavioral-tests.d.ts +4 -0
- package/dist/testing/integrations/giskard/behavioral-tests.d.ts.map +1 -0
- package/dist/testing/integrations/giskard/behavioral-tests.js +66 -0
- package/dist/testing/integrations/giskard/behavioral-tests.js.map +1 -0
- package/dist/testing/integrations/giskard/types.d.ts +35 -0
- package/dist/testing/integrations/giskard/types.d.ts.map +1 -0
- package/dist/testing/integrations/giskard/types.js +2 -0
- package/dist/testing/integrations/giskard/types.js.map +1 -0
- package/dist/testing/integrations/promptfoo/config-generator.d.ts +5 -0
- package/dist/testing/integrations/promptfoo/config-generator.d.ts.map +1 -0
- package/dist/testing/integrations/promptfoo/config-generator.js +44 -0
- package/dist/testing/integrations/promptfoo/config-generator.js.map +1 -0
- package/dist/testing/integrations/promptfoo/types.d.ts +36 -0
- package/dist/testing/integrations/promptfoo/types.d.ts.map +1 -0
- package/dist/testing/integrations/promptfoo/types.js +2 -0
- package/dist/testing/integrations/promptfoo/types.js.map +1 -0
- package/dist/testing/integrations/ralph.d.ts +65 -0
- package/dist/testing/integrations/ralph.d.ts.map +1 -0
- package/dist/testing/integrations/ralph.js +69 -0
- package/dist/testing/integrations/ralph.js.map +1 -0
- package/dist/testing/performance/cache-manager.d.ts +16 -0
- package/dist/testing/performance/cache-manager.d.ts.map +1 -0
- package/dist/testing/performance/cache-manager.js +39 -0
- package/dist/testing/performance/cache-manager.js.map +1 -0
- package/dist/testing/performance/parallel-generator.d.ts +23 -0
- package/dist/testing/performance/parallel-generator.d.ts.map +1 -0
- package/dist/testing/performance/parallel-generator.js +31 -0
- package/dist/testing/performance/parallel-generator.js.map +1 -0
- package/dist/testing/types.d.ts +23 -0
- package/dist/testing/types.d.ts.map +1 -0
- package/dist/testing/types.js +2 -0
- package/dist/testing/types.js.map +1 -0
- package/docs/2026-03-06-llm-testing-system-phase1.md +0 -0
- package/docs/plans/2026-03-06-llm-testing-system-design.md +311 -0
- package/docs/plans/2026-03-06-llm-testing-system-phase1.md +1268 -0
- package/docs/plans/2026-03-06-llm-testing-system-phase2.md +3053 -0
- package/docs/plans/2026-03-06-llm-testing-system-phase3.md +1830 -0
- package/docs/testing/PHASE2.md +266 -0
- package/docs/testing/PHASE3.md +601 -0
- package/docs/testing/README.md +634 -0
- package/package.json +1 -1
- package/skills/test-gen/skill.md +531 -0
- package/skills/ultraqa.md +58 -0
|
@@ -0,0 +1,1268 @@
|
|
|
1
|
+
# LLM Testing System - Phase 1 Implementation Plan
|
|
2
|
+
|
|
3
|
+
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
|
4
|
+
|
|
5
|
+
**Goal:** Build core testing infrastructure with tech stack detection, basic test generation for React + Node.js, and `/test-gen` skill integration.
|
|
6
|
+
|
|
7
|
+
**Architecture:** Extend oh-my-claudecode with a new `src/testing/` module containing CLI commands, test generators, and tech stack detection. Integrate via new skill that delegates to enhanced test-engineer agent.
|
|
8
|
+
|
|
9
|
+
**Tech Stack:** TypeScript, Node.js, Promptfoo, Vitest, Jest, React Testing Library
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Task 1: Project Structure Setup
|
|
14
|
+
|
|
15
|
+
**Files:**
|
|
16
|
+
- Create: `src/testing/index.ts`
|
|
17
|
+
- Create: `src/testing/cli/index.ts`
|
|
18
|
+
- Create: `src/testing/types.ts`
|
|
19
|
+
- Create: `tests/testing/setup.test.ts`
|
|
20
|
+
|
|
21
|
+
**Step 1: Write the failing test**
|
|
22
|
+
|
|
23
|
+
Create `tests/testing/setup.test.ts`:
|
|
24
|
+
|
|
25
|
+
```typescript
|
|
26
|
+
import { describe, it, expect } from 'vitest';
|
|
27
|
+
import { TestingModule } from '../../src/testing';
|
|
28
|
+
|
|
29
|
+
describe('TestingModule', () => {
|
|
30
|
+
it('should export core testing functions', () => {
|
|
31
|
+
expect(TestingModule).toBeDefined();
|
|
32
|
+
expect(TestingModule.detectStack).toBeDefined();
|
|
33
|
+
expect(TestingModule.generateTests).toBeDefined();
|
|
34
|
+
});
|
|
35
|
+
});
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
**Step 2: Run test to verify it fails**
|
|
39
|
+
|
|
40
|
+
Run: `pnpm test tests/testing/setup.test.ts`
|
|
41
|
+
Expected: FAIL with "Cannot find module '../../src/testing'"
|
|
42
|
+
|
|
43
|
+
**Step 3: Create basic module structure**
|
|
44
|
+
|
|
45
|
+
Create `src/testing/types.ts`:
|
|
46
|
+
|
|
47
|
+
```typescript
|
|
48
|
+
export interface TechStack {
|
|
49
|
+
frontend?: {
|
|
50
|
+
framework: 'react' | 'vue' | 'svelte' | 'none';
|
|
51
|
+
testFramework?: 'vitest' | 'jest' | 'none';
|
|
52
|
+
};
|
|
53
|
+
backend?: {
|
|
54
|
+
language: 'nodejs' | 'python' | 'go' | 'rust';
|
|
55
|
+
testFramework?: string;
|
|
56
|
+
};
|
|
57
|
+
databases?: string[];
|
|
58
|
+
apis?: ('rest' | 'graphql' | 'grpc')[];
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
export interface TestGenerationOptions {
|
|
62
|
+
filePath: string;
|
|
63
|
+
techStack: TechStack;
|
|
64
|
+
complexity?: 'simple' | 'complex';
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
export interface TestGenerationResult {
|
|
68
|
+
testFilePath: string;
|
|
69
|
+
testCode: string;
|
|
70
|
+
framework: string;
|
|
71
|
+
}
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Create `src/testing/index.ts`:
|
|
75
|
+
|
|
76
|
+
```typescript
|
|
77
|
+
export * from './types';
|
|
78
|
+
|
|
79
|
+
export const TestingModule = {
|
|
80
|
+
detectStack: async () => {
|
|
81
|
+
throw new Error('Not implemented');
|
|
82
|
+
},
|
|
83
|
+
generateTests: async () => {
|
|
84
|
+
throw new Error('Not implemented');
|
|
85
|
+
},
|
|
86
|
+
};
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
**Step 4: Run test to verify it passes**
|
|
90
|
+
|
|
91
|
+
Run: `pnpm test tests/testing/setup.test.ts`
|
|
92
|
+
Expected: PASS
|
|
93
|
+
|
|
94
|
+
**Step 5: Commit**
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
git add src/testing/ tests/testing/
|
|
98
|
+
git commit -m "feat(testing): add basic module structure and types
|
|
99
|
+
|
|
100
|
+
- Add TechStack and TestGenerationOptions types
|
|
101
|
+
- Create TestingModule with placeholder functions
|
|
102
|
+
- Add initial test suite
|
|
103
|
+
|
|
104
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## Task 2: Tech Stack Detection
|
|
110
|
+
|
|
111
|
+
**Files:**
|
|
112
|
+
- Create: `src/testing/detectors/index.ts`
|
|
113
|
+
- Create: `src/testing/detectors/package-json.ts`
|
|
114
|
+
- Create: `tests/testing/detectors/package-json.test.ts`
|
|
115
|
+
|
|
116
|
+
**Step 1: Write the failing test**
|
|
117
|
+
|
|
118
|
+
Create `tests/testing/detectors/package-json.test.ts`:
|
|
119
|
+
|
|
120
|
+
```typescript
|
|
121
|
+
import { describe, it, expect } from 'vitest';
|
|
122
|
+
import { detectFromPackageJson } from '../../../src/testing/detectors/package-json';
|
|
123
|
+
|
|
124
|
+
describe('detectFromPackageJson', () => {
|
|
125
|
+
it('should detect React + Vitest stack', async () => {
|
|
126
|
+
const mockPackageJson = {
|
|
127
|
+
dependencies: { react: '^18.0.0' },
|
|
128
|
+
devDependencies: { vitest: '^1.0.0', '@testing-library/react': '^14.0.0' },
|
|
129
|
+
};
|
|
130
|
+
|
|
131
|
+
const result = await detectFromPackageJson(mockPackageJson);
|
|
132
|
+
|
|
133
|
+
expect(result.frontend?.framework).toBe('react');
|
|
134
|
+
expect(result.frontend?.testFramework).toBe('vitest');
|
|
135
|
+
});
|
|
136
|
+
|
|
137
|
+
it('should detect Node.js backend', async () => {
|
|
138
|
+
const mockPackageJson = {
|
|
139
|
+
dependencies: { express: '^4.18.0' },
|
|
140
|
+
devDependencies: { jest: '^29.0.0' },
|
|
141
|
+
};
|
|
142
|
+
|
|
143
|
+
const result = await detectFromPackageJson(mockPackageJson);
|
|
144
|
+
|
|
145
|
+
expect(result.backend?.language).toBe('nodejs');
|
|
146
|
+
expect(result.backend?.testFramework).toBe('jest');
|
|
147
|
+
});
|
|
148
|
+
});
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
**Step 2: Run test to verify it fails**
|
|
152
|
+
|
|
153
|
+
Run: `pnpm test tests/testing/detectors/package-json.test.ts`
|
|
154
|
+
Expected: FAIL with "Cannot find module"
|
|
155
|
+
|
|
156
|
+
**Step 3: Implement package.json detection**
|
|
157
|
+
|
|
158
|
+
Create `src/testing/detectors/package-json.ts`:
|
|
159
|
+
|
|
160
|
+
```typescript
|
|
161
|
+
import type { TechStack } from '../types';
|
|
162
|
+
|
|
163
|
+
export async function detectFromPackageJson(packageJson: any): Promise<TechStack> {
|
|
164
|
+
const deps = { ...packageJson.dependencies, ...packageJson.devDependencies };
|
|
165
|
+
const stack: TechStack = {};
|
|
166
|
+
|
|
167
|
+
// Detect frontend framework
|
|
168
|
+
if (deps.react) {
|
|
169
|
+
stack.frontend = {
|
|
170
|
+
framework: 'react',
|
|
171
|
+
testFramework: deps.vitest ? 'vitest' : deps.jest ? 'jest' : 'none',
|
|
172
|
+
};
|
|
173
|
+
} else if (deps.vue) {
|
|
174
|
+
stack.frontend = {
|
|
175
|
+
framework: 'vue',
|
|
176
|
+
testFramework: deps.vitest ? 'vitest' : 'none',
|
|
177
|
+
};
|
|
178
|
+
} else if (deps.svelte) {
|
|
179
|
+
stack.frontend = {
|
|
180
|
+
framework: 'svelte',
|
|
181
|
+
testFramework: deps.vitest ? 'vitest' : 'none',
|
|
182
|
+
};
|
|
183
|
+
}
|
|
184
|
+
|
|
185
|
+
// Detect backend
|
|
186
|
+
if (deps.express || deps.fastify || deps.koa) {
|
|
187
|
+
stack.backend = {
|
|
188
|
+
language: 'nodejs',
|
|
189
|
+
testFramework: deps.vitest ? 'vitest' : deps.jest ? 'jest' : undefined,
|
|
190
|
+
};
|
|
191
|
+
}
|
|
192
|
+
|
|
193
|
+
// Detect databases
|
|
194
|
+
const databases: string[] = [];
|
|
195
|
+
if (deps.pg || deps.postgres) databases.push('postgresql');
|
|
196
|
+
if (deps.mysql || deps.mysql2) databases.push('mysql');
|
|
197
|
+
if (deps.mongodb || deps.mongoose) databases.push('mongodb');
|
|
198
|
+
if (databases.length > 0) stack.databases = databases;
|
|
199
|
+
|
|
200
|
+
// Detect API types
|
|
201
|
+
const apis: ('rest' | 'graphql' | 'grpc')[] = [];
|
|
202
|
+
if (deps.express || deps.fastify) apis.push('rest');
|
|
203
|
+
if (deps.graphql || deps['@apollo/server']) apis.push('graphql');
|
|
204
|
+
if (deps['@grpc/grpc-js']) apis.push('grpc');
|
|
205
|
+
if (apis.length > 0) stack.apis = apis;
|
|
206
|
+
|
|
207
|
+
return stack;
|
|
208
|
+
}
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
Create `src/testing/detectors/index.ts`:
|
|
212
|
+
|
|
213
|
+
```typescript
|
|
214
|
+
import fs from 'fs/promises';
|
|
215
|
+
import path from 'path';
|
|
216
|
+
import { detectFromPackageJson } from './package-json';
|
|
217
|
+
import type { TechStack } from '../types';
|
|
218
|
+
|
|
219
|
+
export async function detectTechStack(projectRoot: string): Promise<TechStack> {
|
|
220
|
+
const packageJsonPath = path.join(projectRoot, 'package.json');
|
|
221
|
+
|
|
222
|
+
try {
|
|
223
|
+
const content = await fs.readFile(packageJsonPath, 'utf-8');
|
|
224
|
+
const packageJson = JSON.parse(content);
|
|
225
|
+
return await detectFromPackageJson(packageJson);
|
|
226
|
+
} catch (error) {
|
|
227
|
+
return {};
|
|
228
|
+
}
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
export { detectFromPackageJson };
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
**Step 4: Run test to verify it passes**
|
|
235
|
+
|
|
236
|
+
Run: `pnpm test tests/testing/detectors/package-json.test.ts`
|
|
237
|
+
Expected: PASS
|
|
238
|
+
|
|
239
|
+
**Step 5: Commit**
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
git add src/testing/detectors/ tests/testing/detectors/
|
|
243
|
+
git commit -m "feat(testing): add tech stack detection from package.json
|
|
244
|
+
|
|
245
|
+
- Detect React/Vue/Svelte frontend frameworks
|
|
246
|
+
- Detect Node.js backend with Express/Fastify/Koa
|
|
247
|
+
- Detect test frameworks (Vitest/Jest)
|
|
248
|
+
- Detect databases (PostgreSQL/MySQL/MongoDB)
|
|
249
|
+
- Detect API types (REST/GraphQL/gRPC)
|
|
250
|
+
|
|
251
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
---
|
|
255
|
+
|
|
256
|
+
## Task 3: React Test Generator
|
|
257
|
+
|
|
258
|
+
**Files:**
|
|
259
|
+
- Create: `src/testing/generators/react.ts`
|
|
260
|
+
- Create: `tests/testing/generators/react.test.ts`
|
|
261
|
+
|
|
262
|
+
**Step 1: Write the failing test**
|
|
263
|
+
|
|
264
|
+
Create `tests/testing/generators/react.test.ts`:
|
|
265
|
+
|
|
266
|
+
```typescript
|
|
267
|
+
import { describe, it, expect } from 'vitest';
|
|
268
|
+
import { generateReactTest } from '../../../src/testing/generators/react';
|
|
269
|
+
|
|
270
|
+
describe('generateReactTest', () => {
|
|
271
|
+
it('should generate test for simple React component', async () => {
|
|
272
|
+
const componentCode = `
|
|
273
|
+
export function Button({ onClick, children }) {
|
|
274
|
+
return <button onClick={onClick}>{children}</button>;
|
|
275
|
+
}
|
|
276
|
+
`;
|
|
277
|
+
|
|
278
|
+
const result = await generateReactTest({
|
|
279
|
+
filePath: 'src/components/Button.tsx',
|
|
280
|
+
code: componentCode,
|
|
281
|
+
testFramework: 'vitest',
|
|
282
|
+
});
|
|
283
|
+
|
|
284
|
+
expect(result.testCode).toContain('import { render, screen }');
|
|
285
|
+
expect(result.testCode).toContain('describe(\'Button\'');
|
|
286
|
+
expect(result.testCode).toContain('it(\'renders children\'');
|
|
287
|
+
expect(result.testFilePath).toBe('src/components/Button.test.tsx');
|
|
288
|
+
});
|
|
289
|
+
});
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
**Step 2: Run test to verify it fails**
|
|
293
|
+
|
|
294
|
+
Run: `pnpm test tests/testing/generators/react.test.ts`
|
|
295
|
+
Expected: FAIL with "Cannot find module"
|
|
296
|
+
|
|
297
|
+
**Step 3: Implement React test generator**
|
|
298
|
+
|
|
299
|
+
Create `src/testing/generators/react.ts`:
|
|
300
|
+
|
|
301
|
+
```typescript
|
|
302
|
+
interface ReactTestOptions {
|
|
303
|
+
filePath: string;
|
|
304
|
+
code: string;
|
|
305
|
+
testFramework: 'vitest' | 'jest';
|
|
306
|
+
}
|
|
307
|
+
|
|
308
|
+
interface ReactTestResult {
|
|
309
|
+
testFilePath: string;
|
|
310
|
+
testCode: string;
|
|
311
|
+
}
|
|
312
|
+
|
|
313
|
+
export async function generateReactTest(options: ReactTestOptions): Promise<ReactTestResult> {
|
|
314
|
+
const { filePath, code, testFramework } = options;
|
|
315
|
+
|
|
316
|
+
// Extract component name from file path
|
|
317
|
+
const fileName = filePath.split('/').pop()?.replace(/\.(tsx?|jsx?)$/, '') || 'Component';
|
|
318
|
+
|
|
319
|
+
// Generate test file path
|
|
320
|
+
const testFilePath = filePath.replace(/\.(tsx?|jsx?)$/, '.test.$1');
|
|
321
|
+
|
|
322
|
+
// Parse component to understand props and behavior
|
|
323
|
+
const hasOnClick = code.includes('onClick');
|
|
324
|
+
const hasChildren = code.includes('children');
|
|
325
|
+
|
|
326
|
+
// Generate test code
|
|
327
|
+
const testCode = `import { describe, it, expect${testFramework === 'vitest' ? ', vi' : ''} } from '${testFramework}';
|
|
328
|
+
import { render, screen${hasOnClick ? ', fireEvent' : ''} } from '@testing-library/react';
|
|
329
|
+
import { ${fileName} } from './${fileName}';
|
|
330
|
+
|
|
331
|
+
describe('${fileName}', () => {
|
|
332
|
+
it('renders children', () => {
|
|
333
|
+
render(<${fileName}>Click me</${fileName}>);
|
|
334
|
+
expect(screen.getByText('Click me')).toBeInTheDocument();
|
|
335
|
+
});
|
|
336
|
+
${hasOnClick ? `
|
|
337
|
+
it('calls onClick when clicked', () => {
|
|
338
|
+
const handleClick = ${testFramework === 'vitest' ? 'vi.fn()' : 'jest.fn()'};
|
|
339
|
+
render(<${fileName} onClick={handleClick}>Click me</${fileName}>);
|
|
340
|
+
fireEvent.click(screen.getByText('Click me'));
|
|
341
|
+
expect(handleClick).toHaveBeenCalledTimes(1);
|
|
342
|
+
});
|
|
343
|
+
` : ''}
|
|
344
|
+
});
|
|
345
|
+
`;
|
|
346
|
+
|
|
347
|
+
return { testFilePath, testCode };
|
|
348
|
+
}
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
**Step 4: Run test to verify it passes**
|
|
352
|
+
|
|
353
|
+
Run: `pnpm test tests/testing/generators/react.test.ts`
|
|
354
|
+
Expected: PASS
|
|
355
|
+
|
|
356
|
+
**Step 5: Commit**
|
|
357
|
+
|
|
358
|
+
```bash
|
|
359
|
+
git add src/testing/generators/ tests/testing/generators/
|
|
360
|
+
git commit -m "feat(testing): add React component test generator
|
|
361
|
+
|
|
362
|
+
- Generate tests for React components
|
|
363
|
+
- Support Vitest and Jest frameworks
|
|
364
|
+
- Auto-detect onClick handlers and children props
|
|
365
|
+
- Generate appropriate test file paths
|
|
366
|
+
|
|
367
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
---
|
|
371
|
+
|
|
372
|
+
## Task 4: Node.js Test Generator
|
|
373
|
+
|
|
374
|
+
**Files:**
|
|
375
|
+
- Create: `src/testing/generators/nodejs.ts`
|
|
376
|
+
- Create: `tests/testing/generators/nodejs.test.ts`
|
|
377
|
+
|
|
378
|
+
**Step 1: Write the failing test**
|
|
379
|
+
|
|
380
|
+
Create `tests/testing/generators/nodejs.test.ts`:
|
|
381
|
+
|
|
382
|
+
```typescript
|
|
383
|
+
import { describe, it, expect } from 'vitest';
|
|
384
|
+
import { generateNodeJsTest } from '../../../src/testing/generators/nodejs';
|
|
385
|
+
|
|
386
|
+
describe('generateNodeJsTest', () => {
|
|
387
|
+
it('should generate test for simple function', async () => {
|
|
388
|
+
const functionCode = `
|
|
389
|
+
export function add(a: number, b: number): number {
|
|
390
|
+
return a + b;
|
|
391
|
+
}
|
|
392
|
+
`;
|
|
393
|
+
|
|
394
|
+
const result = await generateNodeJsTest({
|
|
395
|
+
filePath: 'src/utils/math.ts',
|
|
396
|
+
code: functionCode,
|
|
397
|
+
testFramework: 'vitest',
|
|
398
|
+
});
|
|
399
|
+
|
|
400
|
+
expect(result.testCode).toContain('import { describe, it, expect }');
|
|
401
|
+
expect(result.testCode).toContain('describe(\'add\'');
|
|
402
|
+
expect(result.testCode).toContain('it(\'adds two numbers\'');
|
|
403
|
+
expect(result.testFilePath).toBe('src/utils/math.test.ts');
|
|
404
|
+
});
|
|
405
|
+
});
|
|
406
|
+
```
|
|
407
|
+
|
|
408
|
+
**Step 2: Run test to verify it fails**
|
|
409
|
+
|
|
410
|
+
Run: `pnpm test tests/testing/generators/nodejs.test.ts`
|
|
411
|
+
Expected: FAIL with "Cannot find module"
|
|
412
|
+
|
|
413
|
+
**Step 3: Implement Node.js test generator**
|
|
414
|
+
|
|
415
|
+
Create `src/testing/generators/nodejs.ts`:
|
|
416
|
+
|
|
417
|
+
```typescript
|
|
418
|
+
interface NodeJsTestOptions {
|
|
419
|
+
filePath: string;
|
|
420
|
+
code: string;
|
|
421
|
+
testFramework: 'vitest' | 'jest';
|
|
422
|
+
}
|
|
423
|
+
|
|
424
|
+
interface NodeJsTestResult {
|
|
425
|
+
testFilePath: string;
|
|
426
|
+
testCode: string;
|
|
427
|
+
}
|
|
428
|
+
|
|
429
|
+
export async function generateNodeJsTest(options: NodeJsTestOptions): Promise<NodeJsTestResult> {
|
|
430
|
+
const { filePath, code, testFramework } = options;
|
|
431
|
+
|
|
432
|
+
// Extract function names from code
|
|
433
|
+
const functionMatches = code.matchAll(/export\s+(?:async\s+)?function\s+(\w+)/g);
|
|
434
|
+
const functions = Array.from(functionMatches).map(match => match[1]);
|
|
435
|
+
|
|
436
|
+
// Generate test file path
|
|
437
|
+
const testFilePath = filePath.replace(/\.ts$/, '.test.ts');
|
|
438
|
+
|
|
439
|
+
// Extract file name for import
|
|
440
|
+
const fileName = filePath.split('/').pop()?.replace(/\.ts$/, '') || 'module';
|
|
441
|
+
|
|
442
|
+
// Generate test code for each function
|
|
443
|
+
const testCases = functions.map(funcName => {
|
|
444
|
+
// Simple heuristic: if function name suggests math operation, generate math test
|
|
445
|
+
if (funcName === 'add') {
|
|
446
|
+
return ` it('adds two numbers', () => {
|
|
447
|
+
expect(add(2, 3)).toBe(5);
|
|
448
|
+
expect(add(-1, 1)).toBe(0);
|
|
449
|
+
expect(add(0, 0)).toBe(0);
|
|
450
|
+
});`;
|
|
451
|
+
}
|
|
452
|
+
|
|
453
|
+
// Generic test template
|
|
454
|
+
return ` it('${funcName} works correctly', () => {
|
|
455
|
+
// TODO: Add specific test cases for ${funcName}
|
|
456
|
+
expect(${funcName}).toBeDefined();
|
|
457
|
+
});`;
|
|
458
|
+
}).join('\n\n');
|
|
459
|
+
|
|
460
|
+
const testCode = `import { describe, it, expect } from '${testFramework}';
|
|
461
|
+
import { ${functions.join(', ')} } from './${fileName}';
|
|
462
|
+
|
|
463
|
+
describe('${fileName}', () => {
|
|
464
|
+
${testCases}
|
|
465
|
+
});
|
|
466
|
+
`;
|
|
467
|
+
|
|
468
|
+
return { testFilePath, testCode };
|
|
469
|
+
}
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
**Step 4: Run test to verify it passes**
|
|
473
|
+
|
|
474
|
+
Run: `pnpm test tests/testing/generators/nodejs.test.ts`
|
|
475
|
+
Expected: PASS
|
|
476
|
+
|
|
477
|
+
**Step 5: Commit**
|
|
478
|
+
|
|
479
|
+
```bash
|
|
480
|
+
git add src/testing/generators/nodejs.ts tests/testing/generators/nodejs.test.ts
|
|
481
|
+
git commit -m "feat(testing): add Node.js function test generator
|
|
482
|
+
|
|
483
|
+
- Generate tests for exported functions
|
|
484
|
+
- Support Vitest and Jest frameworks
|
|
485
|
+
- Auto-detect function names from code
|
|
486
|
+
- Generate appropriate test cases
|
|
487
|
+
|
|
488
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
489
|
+
```
|
|
490
|
+
|
|
491
|
+
---
|
|
492
|
+
|
|
493
|
+
## Task 5: CLI Commands
|
|
494
|
+
|
|
495
|
+
**Files:**
|
|
496
|
+
- Create: `src/testing/cli/commands.ts`
|
|
497
|
+
- Modify: `src/cli/index.ts` (add test subcommands)
|
|
498
|
+
- Create: `tests/testing/cli/commands.test.ts`
|
|
499
|
+
|
|
500
|
+
**Step 1: Write the failing test**
|
|
501
|
+
|
|
502
|
+
Create `tests/testing/cli/commands.test.ts`:
|
|
503
|
+
|
|
504
|
+
```typescript
|
|
505
|
+
import { describe, it, expect, vi } from 'vitest';
|
|
506
|
+
import { testGenCommand, testDetectStackCommand } from '../../../src/testing/cli/commands';
|
|
507
|
+
|
|
508
|
+
describe('CLI Commands', () => {
|
|
509
|
+
it('testGenCommand should generate tests for a file', async () => {
|
|
510
|
+
const mockFs = vi.fn();
|
|
511
|
+
const result = await testGenCommand({
|
|
512
|
+
filePath: 'src/components/Button.tsx',
|
|
513
|
+
output: 'src/components/Button.test.tsx',
|
|
514
|
+
});
|
|
515
|
+
|
|
516
|
+
expect(result.success).toBe(true);
|
|
517
|
+
expect(result.testFilePath).toBeDefined();
|
|
518
|
+
});
|
|
519
|
+
|
|
520
|
+
it('testDetectStackCommand should detect tech stack', async () => {
|
|
521
|
+
const result = await testDetectStackCommand({
|
|
522
|
+
projectRoot: process.cwd(),
|
|
523
|
+
});
|
|
524
|
+
|
|
525
|
+
expect(result.stack).toBeDefined();
|
|
526
|
+
});
|
|
527
|
+
});
|
|
528
|
+
```
|
|
529
|
+
|
|
530
|
+
**Step 2: Run test to verify it fails**
|
|
531
|
+
|
|
532
|
+
Run: `pnpm test tests/testing/cli/commands.test.ts`
|
|
533
|
+
Expected: FAIL with "Cannot find module"
|
|
534
|
+
|
|
535
|
+
**Step 3: Implement CLI commands**
|
|
536
|
+
|
|
537
|
+
Create `src/testing/cli/commands.ts`:
|
|
538
|
+
|
|
539
|
+
```typescript
|
|
540
|
+
import fs from 'fs/promises';
|
|
541
|
+
import path from 'path';
|
|
542
|
+
import { detectTechStack } from '../detectors';
|
|
543
|
+
import { generateReactTest } from '../generators/react';
|
|
544
|
+
import { generateNodeJsTest } from '../generators/nodejs';
|
|
545
|
+
import type { TechStack } from '../types';
|
|
546
|
+
|
|
547
|
+
interface TestGenOptions {
|
|
548
|
+
filePath: string;
|
|
549
|
+
output?: string;
|
|
550
|
+
}
|
|
551
|
+
|
|
552
|
+
interface TestGenResult {
|
|
553
|
+
success: boolean;
|
|
554
|
+
testFilePath?: string;
|
|
555
|
+
error?: string;
|
|
556
|
+
}
|
|
557
|
+
|
|
558
|
+
export async function testGenCommand(options: TestGenOptions): Promise<TestGenResult> {
|
|
559
|
+
try {
|
|
560
|
+
const { filePath, output } = options;
|
|
561
|
+
|
|
562
|
+
// Read source file
|
|
563
|
+
const code = await fs.readFile(filePath, 'utf-8');
|
|
564
|
+
|
|
565
|
+
// Detect tech stack
|
|
566
|
+
const projectRoot = process.cwd();
|
|
567
|
+
const stack = await detectTechStack(projectRoot);
|
|
568
|
+
|
|
569
|
+
// Determine generator based on file type and stack
|
|
570
|
+
let result;
|
|
571
|
+
if (filePath.match(/\.(tsx|jsx)$/) && stack.frontend?.framework === 'react') {
|
|
572
|
+
result = await generateReactTest({
|
|
573
|
+
filePath,
|
|
574
|
+
code,
|
|
575
|
+
testFramework: stack.frontend.testFramework || 'vitest',
|
|
576
|
+
});
|
|
577
|
+
} else if (filePath.match(/\.ts$/)) {
|
|
578
|
+
result = await generateNodeJsTest({
|
|
579
|
+
filePath,
|
|
580
|
+
code,
|
|
581
|
+
testFramework: stack.backend?.testFramework || 'vitest',
|
|
582
|
+
});
|
|
583
|
+
} else {
|
|
584
|
+
return {
|
|
585
|
+
success: false,
|
|
586
|
+
error: 'Unsupported file type',
|
|
587
|
+
};
|
|
588
|
+
}
|
|
589
|
+
|
|
590
|
+
// Write test file
|
|
591
|
+
const testFilePath = output || result.testFilePath;
|
|
592
|
+
await fs.writeFile(testFilePath, result.testCode, 'utf-8');
|
|
593
|
+
|
|
594
|
+
return {
|
|
595
|
+
success: true,
|
|
596
|
+
testFilePath,
|
|
597
|
+
};
|
|
598
|
+
} catch (error) {
|
|
599
|
+
return {
|
|
600
|
+
success: false,
|
|
601
|
+
error: error instanceof Error ? error.message : 'Unknown error',
|
|
602
|
+
};
|
|
603
|
+
}
|
|
604
|
+
}
|
|
605
|
+
|
|
606
|
+
interface DetectStackOptions {
|
|
607
|
+
projectRoot: string;
|
|
608
|
+
}
|
|
609
|
+
|
|
610
|
+
interface DetectStackResult {
|
|
611
|
+
stack: TechStack;
|
|
612
|
+
}
|
|
613
|
+
|
|
614
|
+
export async function testDetectStackCommand(options: DetectStackOptions): Promise<DetectStackResult> {
|
|
615
|
+
const stack = await detectTechStack(options.projectRoot);
|
|
616
|
+
return { stack };
|
|
617
|
+
}
|
|
618
|
+
```
|
|
619
|
+
|
|
620
|
+
**Step 4: Add CLI integration**
|
|
621
|
+
|
|
622
|
+
Modify `src/cli/index.ts` to add test subcommands (exact location depends on existing CLI structure):
|
|
623
|
+
|
|
624
|
+
```typescript
|
|
625
|
+
// Add to existing CLI command structure
|
|
626
|
+
program
|
|
627
|
+
.command('test')
|
|
628
|
+
.description('Testing utilities')
|
|
629
|
+
.command('gen <file>')
|
|
630
|
+
.description('Generate tests for a file')
|
|
631
|
+
.option('-o, --output <path>', 'Output test file path')
|
|
632
|
+
.action(async (file, options) => {
|
|
633
|
+
const { testGenCommand } = await import('../testing/cli/commands');
|
|
634
|
+
const result = await testGenCommand({
|
|
635
|
+
filePath: file,
|
|
636
|
+
output: options.output,
|
|
637
|
+
});
|
|
638
|
+
|
|
639
|
+
if (result.success) {
|
|
640
|
+
console.log(`✅ Test generated: ${result.testFilePath}`);
|
|
641
|
+
} else {
|
|
642
|
+
console.error(`❌ Error: ${result.error}`);
|
|
643
|
+
process.exit(1);
|
|
644
|
+
}
|
|
645
|
+
});
|
|
646
|
+
|
|
647
|
+
program
|
|
648
|
+
.command('test')
|
|
649
|
+
.command('detect-stack')
|
|
650
|
+
.description('Detect project tech stack')
|
|
651
|
+
.action(async () => {
|
|
652
|
+
const { testDetectStackCommand } = await import('../testing/cli/commands');
|
|
653
|
+
const result = await testDetectStackCommand({
|
|
654
|
+
projectRoot: process.cwd(),
|
|
655
|
+
});
|
|
656
|
+
|
|
657
|
+
console.log('📊 Detected Tech Stack:');
|
|
658
|
+
console.log(JSON.stringify(result.stack, null, 2));
|
|
659
|
+
});
|
|
660
|
+
```
|
|
661
|
+
|
|
662
|
+
**Step 5: Run test to verify it passes**
|
|
663
|
+
|
|
664
|
+
Run: `pnpm test tests/testing/cli/commands.test.ts`
|
|
665
|
+
Expected: PASS
|
|
666
|
+
|
|
667
|
+
**Step 6: Test CLI manually**
|
|
668
|
+
|
|
669
|
+
Run: `pnpm build && omc test detect-stack`
|
|
670
|
+
Expected: Output showing detected tech stack
|
|
671
|
+
|
|
672
|
+
**Step 7: Commit**
|
|
673
|
+
|
|
674
|
+
```bash
|
|
675
|
+
git add src/testing/cli/ src/cli/index.ts tests/testing/cli/
|
|
676
|
+
git commit -m "feat(testing): add CLI commands for test generation
|
|
677
|
+
|
|
678
|
+
- Add 'omc test gen' command to generate tests
|
|
679
|
+
- Add 'omc test detect-stack' command
|
|
680
|
+
- Integrate with existing CLI structure
|
|
681
|
+
- Support React and Node.js test generation
|
|
682
|
+
|
|
683
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
684
|
+
```
|
|
685
|
+
|
|
686
|
+
---
|
|
687
|
+
|
|
688
|
+
## Task 6: Test-Gen Skill
|
|
689
|
+
|
|
690
|
+
**Files:**
|
|
691
|
+
- Create: `skills/test-gen.md`
|
|
692
|
+
- Create: `tests/skills/test-gen.test.ts`
|
|
693
|
+
|
|
694
|
+
**Step 1: Write the skill document**
|
|
695
|
+
|
|
696
|
+
Create `skills/test-gen.md`:
|
|
697
|
+
|
|
698
|
+
```markdown
|
|
699
|
+
---
|
|
700
|
+
name: test-gen
|
|
701
|
+
description: Generate tests for code files using LLM-driven test generation. Detects tech stack, analyzes code complexity, and generates appropriate tests. Use when user asks to "generate tests", "add tests", "create test coverage", or explicitly invokes /test-gen.
|
|
702
|
+
---
|
|
703
|
+
|
|
704
|
+
# Test Generation Skill
|
|
705
|
+
|
|
706
|
+
Generate comprehensive tests for code files with automatic tech stack detection and complexity analysis.
|
|
707
|
+
|
|
708
|
+
## Usage
|
|
709
|
+
|
|
710
|
+
```
|
|
711
|
+
/test-gen <file-path>
|
|
712
|
+
/test-gen src/components/Button.tsx
|
|
713
|
+
/test-gen src/utils/math.ts --coverage-analysis
|
|
714
|
+
```
|
|
715
|
+
|
|
716
|
+
## Workflow
|
|
717
|
+
|
|
718
|
+
### Step 1: Detect Tech Stack
|
|
719
|
+
|
|
720
|
+
Use the `omc test detect-stack` command to identify:
|
|
721
|
+
- Frontend framework (React/Vue/Svelte)
|
|
722
|
+
- Backend language (Node.js/Python/Go/Rust)
|
|
723
|
+
- Test frameworks (Vitest/Jest/pytest)
|
|
724
|
+
- Databases and API types
|
|
725
|
+
|
|
726
|
+
### Step 2: Analyze Code Complexity
|
|
727
|
+
|
|
728
|
+
Determine if code is "simple" or "complex":
|
|
729
|
+
|
|
730
|
+
**Simple (auto-generate tests)**:
|
|
731
|
+
- Function lines < 50
|
|
732
|
+
- Cyclomatic complexity < 10
|
|
733
|
+
- Nesting level < 3
|
|
734
|
+
- No external dependencies
|
|
735
|
+
|
|
736
|
+
**Complex (generate framework + prompt for details)**:
|
|
737
|
+
- Payment/auth logic
|
|
738
|
+
- Multi-step workflows
|
|
739
|
+
- External service dependencies
|
|
740
|
+
- Async/concurrent scenarios
|
|
741
|
+
|
|
742
|
+
### Step 3: Generate Tests
|
|
743
|
+
|
|
744
|
+
For simple code:
|
|
745
|
+
- Call `omc test gen <file>` directly
|
|
746
|
+
- Review generated tests
|
|
747
|
+
- Commit
|
|
748
|
+
|
|
749
|
+
For complex code:
|
|
750
|
+
- Generate test framework with `omc test gen <file>`
|
|
751
|
+
- Delegate to enhanced `test-engineer` agent for detailed test cases
|
|
752
|
+
- Prompt user for business logic validation
|
|
753
|
+
|
|
754
|
+
### Step 4: Coverage Analysis (Optional)
|
|
755
|
+
|
|
756
|
+
If `--coverage-analysis` flag is provided:
|
|
757
|
+
- Run coverage tools
|
|
758
|
+
- Identify missing test scenarios
|
|
759
|
+
- Generate supplementary tests
|
|
760
|
+
|
|
761
|
+
## Integration with Other Skills
|
|
762
|
+
|
|
763
|
+
- Works with `/tdd` for test-first development
|
|
764
|
+
- Integrates with `/ultraqa` for test-verify-fix loops
|
|
765
|
+
- Called automatically by `/autopilot` when generating code
|
|
766
|
+
|
|
767
|
+
## Examples
|
|
768
|
+
|
|
769
|
+
### Example 1: Simple React Component
|
|
770
|
+
|
|
771
|
+
```
|
|
772
|
+
User: /test-gen src/components/Button.tsx
|
|
773
|
+
Agent: Detecting tech stack...
|
|
774
|
+
✅ Detected: React + Vitest
|
|
775
|
+
|
|
776
|
+
Agent: Analyzing complexity...
|
|
777
|
+
✅ Simple component (25 lines, no external deps)
|
|
778
|
+
|
|
779
|
+
Agent: Generating tests...
|
|
780
|
+
✅ Generated: src/components/Button.test.tsx
|
|
781
|
+
|
|
782
|
+
Tests include:
|
|
783
|
+
- Renders children correctly
|
|
784
|
+
- Handles onClick events
|
|
785
|
+
- Accessibility checks
|
|
786
|
+
|
|
787
|
+
Would you like me to commit these tests?
|
|
788
|
+
```
|
|
789
|
+
|
|
790
|
+
### Example 2: Complex Payment Service
|
|
791
|
+
|
|
792
|
+
```
|
|
793
|
+
User: /test-gen src/services/payment.ts
|
|
794
|
+
|
|
795
|
+
Agent: Detecting tech stack...
|
|
796
|
+
✅ Detected: Node.js + Express + PostgreSQL + Vitest
|
|
797
|
+
|
|
798
|
+
Agent: Analyzing complexity...
|
|
799
|
+
⚠️ Complex code detected:
|
|
800
|
+
- Payment processing logic
|
|
801
|
+
- External Stripe API calls
|
|
802
|
+
- Database transactions
|
|
803
|
+
- Error handling for multiple scenarios
|
|
804
|
+
|
|
805
|
+
Agent: Generating test framework...
|
|
806
|
+
✅ Generated: src/services/payment.test.ts (framework)
|
|
807
|
+
|
|
808
|
+
Agent: Delegating to test-engineer for detailed test cases...
|
|
809
|
+
|
|
810
|
+
Test-Engineer: I'll need some information:
|
|
811
|
+
1. What are the expected payment flows? (success, failure, retry)
|
|
812
|
+
2. Should I mock Stripe API calls?
|
|
813
|
+
3. What database states should I test?
|
|
814
|
+
4. Are there specific edge cases to cover?
|
|
815
|
+
|
|
816
|
+
[User provides details]
|
|
817
|
+
|
|
818
|
+
Test-Engineer: Generating comprehensive test suite...
|
|
819
|
+
✅ Generated 15 test cases covering:
|
|
820
|
+
- Happy path payment processing
|
|
821
|
+
- Stripe API failure scenarios
|
|
822
|
+
- Database transaction rollbacks
|
|
823
|
+
- Idempotency checks
|
|
824
|
+
- Concurrent payment handling
|
|
825
|
+
|
|
826
|
+
Would you like me to commit these tests?
|
|
827
|
+
```
|
|
828
|
+
|
|
829
|
+
### Example 3: Coverage Analysis
|
|
830
|
+
|
|
831
|
+
```
|
|
832
|
+
User: /test-gen src/utils/validation.ts --coverage-analysis
|
|
833
|
+
|
|
834
|
+
Agent: Generating initial tests...
|
|
835
|
+
✅ Generated: src/utils/validation.test.ts
|
|
836
|
+
|
|
837
|
+
Agent: Running coverage analysis...
|
|
838
|
+
📊 Current coverage: 65%
|
|
839
|
+
|
|
840
|
+
Missing coverage:
|
|
841
|
+
- Line 42-48: Error handling for invalid email formats
|
|
842
|
+
- Line 67-72: Edge case for international phone numbers
|
|
843
|
+
- Line 89: Null check branch
|
|
844
|
+
|
|
845
|
+
Agent: Generating supplementary tests...
|
|
846
|
+
✅ Added 5 additional test cases
|
|
847
|
+
|
|
848
|
+
Agent: Re-running coverage...
|
|
849
|
+
📊 New coverage: 92%
|
|
850
|
+
|
|
851
|
+
Would you like me to commit these tests?
|
|
852
|
+
```
|
|
853
|
+
|
|
854
|
+
## Integration Notes
|
|
855
|
+
|
|
856
|
+
### With Test-Engineer Agent
|
|
857
|
+
|
|
858
|
+
The `/test-gen` skill enhances the existing `test-engineer` agent by:
|
|
859
|
+
|
|
860
|
+
1. **Automatic invocation**: When user asks to "generate tests", invoke `/test-gen` instead of directly calling `test-engineer`
|
|
861
|
+
2. **Context enrichment**: Pass detected tech stack and complexity analysis to `test-engineer`
|
|
862
|
+
3. **Workflow guidance**: Provide structured workflow for test generation
|
|
863
|
+
|
|
864
|
+
Update `agents/test-engineer.md` to reference `/test-gen`:
|
|
865
|
+
|
|
866
|
+
```markdown
|
|
867
|
+
## When to Use
|
|
868
|
+
|
|
869
|
+
- User explicitly asks for test generation → Use `/test-gen` skill first
|
|
870
|
+
- User asks for test strategy/planning → Use test-engineer directly
|
|
871
|
+
- User asks for test debugging → Use test-engineer directly
|
|
872
|
+
```
|
|
873
|
+
|
|
874
|
+
### With Autopilot
|
|
875
|
+
|
|
876
|
+
Update `skills/autopilot.md` to include test generation:
|
|
877
|
+
|
|
878
|
+
```markdown
|
|
879
|
+
## Phase 4: Testing
|
|
880
|
+
|
|
881
|
+
After implementation:
|
|
882
|
+
1. Run `/test-gen` on all new source files
|
|
883
|
+
2. Verify tests pass
|
|
884
|
+
3. Check coverage meets threshold (default: 80%)
|
|
885
|
+
```
|
|
886
|
+
|
|
887
|
+
### With TDD Skill
|
|
888
|
+
|
|
889
|
+
Update `skills/tdd.md` to integrate:
|
|
890
|
+
|
|
891
|
+
```markdown
|
|
892
|
+
## Red Phase
|
|
893
|
+
|
|
894
|
+
Instead of manually writing tests, use:
|
|
895
|
+
```
|
|
896
|
+
/test-gen <file> --tdd-mode
|
|
897
|
+
```
|
|
898
|
+
|
|
899
|
+
This generates failing tests based on function signatures.
|
|
900
|
+
```
|
|
901
|
+
|
|
902
|
+
## Success Criteria
|
|
903
|
+
|
|
904
|
+
- [ ] Tech stack detection works for React + Node.js projects
|
|
905
|
+
- [ ] Test generation produces valid, runnable tests
|
|
906
|
+
- [ ] CLI commands (`omc test gen`, `omc test detect-stack`) work correctly
|
|
907
|
+
- [ ] `/test-gen` skill integrates with existing agent workflows
|
|
908
|
+
- [ ] All tests pass: `pnpm test tests/testing/**`
|
|
909
|
+
- [ ] Documentation is complete and accurate
|
|
910
|
+
|
|
911
|
+
**Step 2: Write integration test**
|
|
912
|
+
|
|
913
|
+
Create `tests/skills/test-gen.test.ts`:
|
|
914
|
+
|
|
915
|
+
```typescript
|
|
916
|
+
import { describe, it, expect, vi } from 'vitest';
|
|
917
|
+
import { execSync } from 'child_process';
|
|
918
|
+
import fs from 'fs/promises';
|
|
919
|
+
import path from 'path';
|
|
920
|
+
|
|
921
|
+
describe('test-gen skill integration', () => {
|
|
922
|
+
it('should generate tests via CLI', async () => {
|
|
923
|
+
// Create a temporary test file
|
|
924
|
+
const tmpDir = path.join(process.cwd(), 'tmp-test-gen');
|
|
925
|
+
await fs.mkdir(tmpDir, { recursive: true });
|
|
926
|
+
|
|
927
|
+
const componentPath = path.join(tmpDir, 'Button.tsx');
|
|
928
|
+
await fs.writeFile(componentPath, `
|
|
929
|
+
export function Button({ onClick, children }) {
|
|
930
|
+
return <button onClick={onClick}>{children}</button>;
|
|
931
|
+
}
|
|
932
|
+
`);
|
|
933
|
+
|
|
934
|
+
// Run CLI command
|
|
935
|
+
const result = execSync(`pnpm omc test gen ${componentPath}`, {
|
|
936
|
+
encoding: 'utf-8',
|
|
937
|
+
});
|
|
938
|
+
|
|
939
|
+
expect(result).toContain('✅ Test generated');
|
|
940
|
+
|
|
941
|
+
// Verify test file exists
|
|
942
|
+
const testPath = path.join(tmpDir, 'Button.test.tsx');
|
|
943
|
+
const testContent = await fs.readFile(testPath, 'utf-8');
|
|
944
|
+
expect(testContent).toContain('describe(\'Button\'');
|
|
945
|
+
|
|
946
|
+
// Cleanup
|
|
947
|
+
await fs.rm(tmpDir, { recursive: true });
|
|
948
|
+
});
|
|
949
|
+
});
|
|
950
|
+
```
|
|
951
|
+
|
|
952
|
+
**Step 3: Run test to verify it fails**
|
|
953
|
+
|
|
954
|
+
Run: `pnpm test tests/skills/test-gen.test.ts`
|
|
955
|
+
Expected: FAIL (skill not yet integrated)
|
|
956
|
+
|
|
957
|
+
**Step 4: Verify skill document is complete**
|
|
958
|
+
|
|
959
|
+
The skill document should be complete and ready for use.
|
|
960
|
+
|
|
961
|
+
**Step 5: Commit**
|
|
962
|
+
|
|
963
|
+
```bash
|
|
964
|
+
git add skills/test-gen.md tests/skills/test-gen.test.ts
|
|
965
|
+
git commit -m "feat(skills): add test-gen skill for LLM-driven test generation
|
|
966
|
+
|
|
967
|
+
- Add /test-gen skill with tech stack detection
|
|
968
|
+
- Support simple and complex code analysis
|
|
969
|
+
- Integrate with test-engineer agent
|
|
970
|
+
- Add coverage analysis support
|
|
971
|
+
- Include comprehensive examples
|
|
972
|
+
|
|
973
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
974
|
+
```
|
|
975
|
+
|
|
976
|
+
---
|
|
977
|
+
|
|
978
|
+
## Task 7: Documentation and Integration
|
|
979
|
+
|
|
980
|
+
**Files:**
|
|
981
|
+
- Create: `docs/testing/README.md`
|
|
982
|
+
- Update: `README.md` (add testing section)
|
|
983
|
+
- Update: `agents/test-engineer.md` (reference test-gen)
|
|
984
|
+
|
|
985
|
+
**Step 1: Create testing documentation**
|
|
986
|
+
|
|
987
|
+
Create `docs/testing/README.md`:
|
|
988
|
+
|
|
989
|
+
```markdown
|
|
990
|
+
# LLM Testing System
|
|
991
|
+
|
|
992
|
+
Intelligent test generation system for oh-my-claudecode.
|
|
993
|
+
|
|
994
|
+
## Quick Start
|
|
995
|
+
|
|
996
|
+
Generate tests for a file:
|
|
997
|
+
|
|
998
|
+
```bash
|
|
999
|
+
omc test gen src/components/Button.tsx
|
|
1000
|
+
```
|
|
1001
|
+
|
|
1002
|
+
Or use the skill:
|
|
1003
|
+
|
|
1004
|
+
```bash
|
|
1005
|
+
/test-gen src/components/Button.tsx
|
|
1006
|
+
```
|
|
1007
|
+
|
|
1008
|
+
## Features
|
|
1009
|
+
|
|
1010
|
+
- **Automatic tech stack detection**: Identifies React, Vue, Svelte, Node.js, and more
|
|
1011
|
+
- **Complexity analysis**: Determines if code is simple or complex
|
|
1012
|
+
- **Smart test generation**: Creates appropriate tests based on code patterns
|
|
1013
|
+
- **Coverage analysis**: Identifies missing test scenarios
|
|
1014
|
+
- **Framework support**: Vitest, Jest, React Testing Library
|
|
1015
|
+
|
|
1016
|
+
## Architecture
|
|
1017
|
+
|
|
1018
|
+
```
|
|
1019
|
+
src/testing/
|
|
1020
|
+
├── index.ts # Main module exports
|
|
1021
|
+
├── types.ts # TypeScript interfaces
|
|
1022
|
+
├── cli/
|
|
1023
|
+
│ └── commands.ts # CLI command implementations
|
|
1024
|
+
├── detectors/
|
|
1025
|
+
│ ├── index.ts # Tech stack detection orchestrator
|
|
1026
|
+
│ └── package-json.ts # package.json parser
|
|
1027
|
+
└── generators/
|
|
1028
|
+
├── react.ts # React component test generator
|
|
1029
|
+
└── nodejs.ts # Node.js function test generator
|
|
1030
|
+
```
|
|
1031
|
+
|
|
1032
|
+
## CLI Commands
|
|
1033
|
+
|
|
1034
|
+
### Generate Tests
|
|
1035
|
+
|
|
1036
|
+
```bash
|
|
1037
|
+
omc test gen <file> [--output <path>]
|
|
1038
|
+
```
|
|
1039
|
+
|
|
1040
|
+
Generates tests for the specified file.
|
|
1041
|
+
|
|
1042
|
+
### Detect Tech Stack
|
|
1043
|
+
|
|
1044
|
+
```bash
|
|
1045
|
+
omc test detect-stack
|
|
1046
|
+
```
|
|
1047
|
+
|
|
1048
|
+
Displays the detected tech stack for the current project.
|
|
1049
|
+
|
|
1050
|
+
## Skill Usage
|
|
1051
|
+
|
|
1052
|
+
The `/test-gen` skill provides a higher-level interface:
|
|
1053
|
+
|
|
1054
|
+
```bash
|
|
1055
|
+
/test-gen src/components/Button.tsx
|
|
1056
|
+
/test-gen src/services/payment.ts --coverage-analysis
|
|
1057
|
+
```
|
|
1058
|
+
|
|
1059
|
+
## Supported Tech Stacks
|
|
1060
|
+
|
|
1061
|
+
### Frontend
|
|
1062
|
+
- React (with Vitest or Jest)
|
|
1063
|
+
- Vue (with Vitest)
|
|
1064
|
+
- Svelte (with Vitest)
|
|
1065
|
+
|
|
1066
|
+
### Backend
|
|
1067
|
+
- Node.js (with Vitest or Jest)
|
|
1068
|
+
|
|
1069
|
+
### Coming Soon (Phase 2)
|
|
1070
|
+
- Python (pytest)
|
|
1071
|
+
- Go (testing package)
|
|
1072
|
+
- Rust (cargo test)
|
|
1073
|
+
|
|
1074
|
+
## Examples
|
|
1075
|
+
|
|
1076
|
+
See `skills/test-gen.md` for detailed examples.
|
|
1077
|
+
|
|
1078
|
+
## Development
|
|
1079
|
+
|
|
1080
|
+
Run tests:
|
|
1081
|
+
|
|
1082
|
+
```bash
|
|
1083
|
+
pnpm test tests/testing/**
|
|
1084
|
+
```
|
|
1085
|
+
|
|
1086
|
+
Build:
|
|
1087
|
+
|
|
1088
|
+
```bash
|
|
1089
|
+
pnpm build
|
|
1090
|
+
```
|
|
1091
|
+
```
|
|
1092
|
+
|
|
1093
|
+
**Step 2: Update main README**
|
|
1094
|
+
|
|
1095
|
+
Add to `README.md` (in appropriate section):
|
|
1096
|
+
|
|
1097
|
+
```markdown
|
|
1098
|
+
## Testing
|
|
1099
|
+
|
|
1100
|
+
oh-my-claudecode includes an intelligent test generation system:
|
|
1101
|
+
|
|
1102
|
+
```bash
|
|
1103
|
+
# Generate tests for a file
|
|
1104
|
+
omc test gen src/components/Button.tsx
|
|
1105
|
+
|
|
1106
|
+
# Detect project tech stack
|
|
1107
|
+
omc test detect-stack
|
|
1108
|
+
|
|
1109
|
+
# Use the skill for advanced features
|
|
1110
|
+
/test-gen src/services/payment.ts --coverage-analysis
|
|
1111
|
+
```
|
|
1112
|
+
|
|
1113
|
+
See [Testing Documentation](docs/testing/README.md) for details.
|
|
1114
|
+
```
|
|
1115
|
+
|
|
1116
|
+
**Step 3: Update test-engineer agent**
|
|
1117
|
+
|
|
1118
|
+
Update `agents/test-engineer.md` to add:
|
|
1119
|
+
|
|
1120
|
+
```markdown
|
|
1121
|
+
## Integration with Test-Gen Skill
|
|
1122
|
+
|
|
1123
|
+
When user asks to "generate tests" or "add tests", prefer using the `/test-gen` skill:
|
|
1124
|
+
|
|
1125
|
+
- `/test-gen` handles tech stack detection and complexity analysis automatically
|
|
1126
|
+
- For complex code, `/test-gen` will delegate to test-engineer with enriched context
|
|
1127
|
+
- For test strategy and planning, use test-engineer directly
|
|
1128
|
+
```
|
|
1129
|
+
|
|
1130
|
+
**Step 4: Commit**
|
|
1131
|
+
|
|
1132
|
+
```bash
|
|
1133
|
+
git add docs/testing/ README.md agents/test-engineer.md
|
|
1134
|
+
git commit -m "docs(testing): add comprehensive testing documentation
|
|
1135
|
+
|
|
1136
|
+
- Add testing system README with architecture overview
|
|
1137
|
+
- Update main README with testing section
|
|
1138
|
+
- Integrate test-gen skill with test-engineer agent
|
|
1139
|
+
- Include usage examples and CLI reference
|
|
1140
|
+
|
|
1141
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
1142
|
+
```
|
|
1143
|
+
|
|
1144
|
+
---
|
|
1145
|
+
|
|
1146
|
+
## Phase 1 Summary
|
|
1147
|
+
|
|
1148
|
+
### Deliverables Checklist
|
|
1149
|
+
|
|
1150
|
+
- [x] **Core Infrastructure**
|
|
1151
|
+
- [x] Project structure (`src/testing/`)
|
|
1152
|
+
- [x] TypeScript types and interfaces
|
|
1153
|
+
- [x] Module exports and organization
|
|
1154
|
+
|
|
1155
|
+
- [x] **Tech Stack Detection**
|
|
1156
|
+
- [x] package.json parser
|
|
1157
|
+
- [x] Frontend framework detection (React/Vue/Svelte)
|
|
1158
|
+
- [x] Backend language detection (Node.js)
|
|
1159
|
+
- [x] Test framework detection (Vitest/Jest)
|
|
1160
|
+
- [x] Database and API detection
|
|
1161
|
+
|
|
1162
|
+
- [x] **Test Generators**
|
|
1163
|
+
- [x] React component test generator
|
|
1164
|
+
- [x] Node.js function test generator
|
|
1165
|
+
- [x] Support for Vitest and Jest
|
|
1166
|
+
- [x] Automatic test file path generation
|
|
1167
|
+
|
|
1168
|
+
- [x] **CLI Integration**
|
|
1169
|
+
- [x] `omc test gen` command
|
|
1170
|
+
- [x] `omc test detect-stack` command
|
|
1171
|
+
- [x] CLI command tests
|
|
1172
|
+
|
|
1173
|
+
- [x] **Skill Integration**
|
|
1174
|
+
- [x] `/test-gen` skill document
|
|
1175
|
+
- [x] Workflow for simple vs complex code
|
|
1176
|
+
- [x] Integration with test-engineer agent
|
|
1177
|
+
- [x] Coverage analysis support
|
|
1178
|
+
|
|
1179
|
+
- [x] **Documentation**
|
|
1180
|
+
- [x] Testing system README
|
|
1181
|
+
- [x] Main README updates
|
|
1182
|
+
- [x] Agent integration docs
|
|
1183
|
+
- [x] Usage examples
|
|
1184
|
+
|
|
1185
|
+
- [x] **Testing**
|
|
1186
|
+
- [x] Unit tests for all modules
|
|
1187
|
+
- [x] Integration tests for CLI
|
|
1188
|
+
- [x] Skill integration tests
|
|
1189
|
+
|
|
1190
|
+
### Estimated Timeline
|
|
1191
|
+
|
|
1192
|
+
- **Task 1**: Project Structure Setup - 30 minutes
|
|
1193
|
+
- **Task 2**: Tech Stack Detection - 1 hour
|
|
1194
|
+
- **Task 3**: React Test Generator - 1.5 hours
|
|
1195
|
+
- **Task 4**: Node.js Test Generator - 1 hour
|
|
1196
|
+
- **Task 5**: CLI Commands - 1.5 hours
|
|
1197
|
+
- **Task 6**: Test-Gen Skill - 1 hour
|
|
1198
|
+
- **Task 7**: Documentation - 1 hour
|
|
1199
|
+
|
|
1200
|
+
**Total**: ~7.5 hours
|
|
1201
|
+
|
|
1202
|
+
### Next Steps
|
|
1203
|
+
|
|
1204
|
+
**Phase 2: Advanced Features**
|
|
1205
|
+
1. Python test generation (pytest)
|
|
1206
|
+
2. Go test generation (testing package)
|
|
1207
|
+
3. Rust test generation (cargo test)
|
|
1208
|
+
4. Integration test generation
|
|
1209
|
+
5. E2E test generation (Playwright/Cypress)
|
|
1210
|
+
6. Advanced complexity analysis with AST parsing
|
|
1211
|
+
7. LLM-powered test case suggestions
|
|
1212
|
+
8. Test quality scoring
|
|
1213
|
+
|
|
1214
|
+
**Phase 3: Promptfoo Integration**
|
|
1215
|
+
1. Promptfoo configuration generation
|
|
1216
|
+
2. LLM evaluation test generation
|
|
1217
|
+
3. Regression test suite for prompts
|
|
1218
|
+
4. Performance benchmarking
|
|
1219
|
+
5. Multi-model comparison
|
|
1220
|
+
|
|
1221
|
+
**Immediate Actions**
|
|
1222
|
+
1. Review this plan with team
|
|
1223
|
+
2. Create GitHub issues for each task
|
|
1224
|
+
3. Set up project board
|
|
1225
|
+
4. Begin Task 1 implementation
|
|
1226
|
+
|
|
1227
|
+
---
|
|
1228
|
+
|
|
1229
|
+
## Implementation Notes
|
|
1230
|
+
|
|
1231
|
+
### TDD Approach
|
|
1232
|
+
|
|
1233
|
+
Each task follows strict TDD:
|
|
1234
|
+
1. Write failing test
|
|
1235
|
+
2. Run test to verify failure
|
|
1236
|
+
3. Implement minimal code to pass
|
|
1237
|
+
4. Run test to verify pass
|
|
1238
|
+
5. Commit with descriptive message
|
|
1239
|
+
|
|
1240
|
+
### Code Quality
|
|
1241
|
+
|
|
1242
|
+
- All code must pass TypeScript type checking
|
|
1243
|
+
- All tests must pass before committing
|
|
1244
|
+
- Follow existing code style and conventions
|
|
1245
|
+
- Use meaningful variable and function names
|
|
1246
|
+
- Add comments for complex logic
|
|
1247
|
+
|
|
1248
|
+
### Git Workflow
|
|
1249
|
+
|
|
1250
|
+
- Create feature branch from `dev`: `git checkout -b feature/llm-testing-phase1 dev`
|
|
1251
|
+
- Commit after each task completion
|
|
1252
|
+
- Use conventional commit format: `feat(testing): description`
|
|
1253
|
+
- Include co-author tag: `Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>`
|
|
1254
|
+
- Create PR targeting `dev` branch when complete
|
|
1255
|
+
|
|
1256
|
+
### Testing Strategy
|
|
1257
|
+
|
|
1258
|
+
- Unit tests for all functions
|
|
1259
|
+
- Integration tests for CLI commands
|
|
1260
|
+
- Mock external dependencies (file system, APIs)
|
|
1261
|
+
- Use Vitest for all tests
|
|
1262
|
+
- Aim for >80% code coverage
|
|
1263
|
+
|
|
1264
|
+
---
|
|
1265
|
+
|
|
1266
|
+
**Plan Status**: ✅ Complete and ready for implementation
|
|
1267
|
+
|
|
1268
|
+
**Next Action**: Begin Task 1 - Project Structure Setup
|