claudecode-omc 4.7.4 → 4.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/README.md +50 -0
- package/agents/test-engineer.md +74 -0
- package/bridge/cli.cjs +9335 -117
- package/dist/cli/index.js +201 -0
- package/dist/cli/index.js.map +1 -1
- package/dist/testing/analyzers/complexity.d.ts +18 -0
- package/dist/testing/analyzers/complexity.d.ts.map +1 -0
- package/dist/testing/analyzers/complexity.js +121 -0
- package/dist/testing/analyzers/complexity.js.map +1 -0
- package/dist/testing/analyzers/coverage.d.ts +13 -0
- package/dist/testing/analyzers/coverage.d.ts.map +1 -0
- package/dist/testing/analyzers/coverage.js +99 -0
- package/dist/testing/analyzers/coverage.js.map +1 -0
- package/dist/testing/analyzers/quality-scorer.d.ts +8 -0
- package/dist/testing/analyzers/quality-scorer.d.ts.map +1 -0
- package/dist/testing/analyzers/quality-scorer.js +128 -0
- package/dist/testing/analyzers/quality-scorer.js.map +1 -0
- package/dist/testing/analyzers/types.d.ts +56 -0
- package/dist/testing/analyzers/types.d.ts.map +1 -0
- package/dist/testing/analyzers/types.js +2 -0
- package/dist/testing/analyzers/types.js.map +1 -0
- package/dist/testing/cli/agent-integration.d.ts +20 -0
- package/dist/testing/cli/agent-integration.d.ts.map +1 -0
- package/dist/testing/cli/agent-integration.js +60 -0
- package/dist/testing/cli/agent-integration.js.map +1 -0
- package/dist/testing/cli/commands.d.ts +100 -0
- package/dist/testing/cli/commands.d.ts.map +1 -0
- package/dist/testing/cli/commands.js +250 -0
- package/dist/testing/cli/commands.js.map +1 -0
- package/dist/testing/cli/ultraqa-integration.d.ts +13 -0
- package/dist/testing/cli/ultraqa-integration.d.ts.map +1 -0
- package/dist/testing/cli/ultraqa-integration.js +68 -0
- package/dist/testing/cli/ultraqa-integration.js.map +1 -0
- package/dist/testing/detectors/go.d.ts +3 -0
- package/dist/testing/detectors/go.d.ts.map +1 -0
- package/dist/testing/detectors/go.js +38 -0
- package/dist/testing/detectors/go.js.map +1 -0
- package/dist/testing/detectors/index.d.ts +8 -0
- package/dist/testing/detectors/index.d.ts.map +1 -0
- package/dist/testing/detectors/index.js +46 -0
- package/dist/testing/detectors/index.js.map +1 -0
- package/dist/testing/detectors/package-json.d.ts +3 -0
- package/dist/testing/detectors/package-json.d.ts.map +1 -0
- package/dist/testing/detectors/package-json.js +52 -0
- package/dist/testing/detectors/package-json.js.map +1 -0
- package/dist/testing/detectors/python.d.ts +3 -0
- package/dist/testing/detectors/python.d.ts.map +1 -0
- package/dist/testing/detectors/python.js +37 -0
- package/dist/testing/detectors/python.js.map +1 -0
- package/dist/testing/detectors/rust.d.ts +3 -0
- package/dist/testing/detectors/rust.d.ts.map +1 -0
- package/dist/testing/detectors/rust.js +39 -0
- package/dist/testing/detectors/rust.js.map +1 -0
- package/dist/testing/generators/contract.d.ts +14 -0
- package/dist/testing/generators/contract.d.ts.map +1 -0
- package/dist/testing/generators/contract.js +163 -0
- package/dist/testing/generators/contract.js.map +1 -0
- package/dist/testing/generators/e2e.d.ts +34 -0
- package/dist/testing/generators/e2e.d.ts.map +1 -0
- package/dist/testing/generators/e2e.js +74 -0
- package/dist/testing/generators/e2e.js.map +1 -0
- package/dist/testing/generators/go.d.ts +12 -0
- package/dist/testing/generators/go.d.ts.map +1 -0
- package/dist/testing/generators/go.js +144 -0
- package/dist/testing/generators/go.js.map +1 -0
- package/dist/testing/generators/nodejs.d.ts +12 -0
- package/dist/testing/generators/nodejs.d.ts.map +1 -0
- package/dist/testing/generators/nodejs.js +37 -0
- package/dist/testing/generators/nodejs.js.map +1 -0
- package/dist/testing/generators/python.d.ts +12 -0
- package/dist/testing/generators/python.d.ts.map +1 -0
- package/dist/testing/generators/python.js +163 -0
- package/dist/testing/generators/python.js.map +1 -0
- package/dist/testing/generators/react.d.ts +12 -0
- package/dist/testing/generators/react.d.ts.map +1 -0
- package/dist/testing/generators/react.js +31 -0
- package/dist/testing/generators/react.js.map +1 -0
- package/dist/testing/generators/rust.d.ts +11 -0
- package/dist/testing/generators/rust.d.ts.map +1 -0
- package/dist/testing/generators/rust.js +138 -0
- package/dist/testing/generators/rust.js.map +1 -0
- package/dist/testing/index.d.ts +6 -0
- package/dist/testing/index.d.ts.map +1 -0
- package/dist/testing/index.js +11 -0
- package/dist/testing/index.js.map +1 -0
- package/dist/testing/integrations/autopilot.d.ts +42 -0
- package/dist/testing/integrations/autopilot.d.ts.map +1 -0
- package/dist/testing/integrations/autopilot.js +55 -0
- package/dist/testing/integrations/autopilot.js.map +1 -0
- package/dist/testing/integrations/cicd.d.ts +26 -0
- package/dist/testing/integrations/cicd.d.ts.map +1 -0
- package/dist/testing/integrations/cicd.js +162 -0
- package/dist/testing/integrations/cicd.js.map +1 -0
- package/dist/testing/integrations/giskard/behavioral-tests.d.ts +4 -0
- package/dist/testing/integrations/giskard/behavioral-tests.d.ts.map +1 -0
- package/dist/testing/integrations/giskard/behavioral-tests.js +66 -0
- package/dist/testing/integrations/giskard/behavioral-tests.js.map +1 -0
- package/dist/testing/integrations/giskard/types.d.ts +35 -0
- package/dist/testing/integrations/giskard/types.d.ts.map +1 -0
- package/dist/testing/integrations/giskard/types.js +2 -0
- package/dist/testing/integrations/giskard/types.js.map +1 -0
- package/dist/testing/integrations/promptfoo/config-generator.d.ts +5 -0
- package/dist/testing/integrations/promptfoo/config-generator.d.ts.map +1 -0
- package/dist/testing/integrations/promptfoo/config-generator.js +44 -0
- package/dist/testing/integrations/promptfoo/config-generator.js.map +1 -0
- package/dist/testing/integrations/promptfoo/types.d.ts +36 -0
- package/dist/testing/integrations/promptfoo/types.d.ts.map +1 -0
- package/dist/testing/integrations/promptfoo/types.js +2 -0
- package/dist/testing/integrations/promptfoo/types.js.map +1 -0
- package/dist/testing/integrations/ralph.d.ts +65 -0
- package/dist/testing/integrations/ralph.d.ts.map +1 -0
- package/dist/testing/integrations/ralph.js +69 -0
- package/dist/testing/integrations/ralph.js.map +1 -0
- package/dist/testing/performance/cache-manager.d.ts +16 -0
- package/dist/testing/performance/cache-manager.d.ts.map +1 -0
- package/dist/testing/performance/cache-manager.js +39 -0
- package/dist/testing/performance/cache-manager.js.map +1 -0
- package/dist/testing/performance/parallel-generator.d.ts +23 -0
- package/dist/testing/performance/parallel-generator.d.ts.map +1 -0
- package/dist/testing/performance/parallel-generator.js +31 -0
- package/dist/testing/performance/parallel-generator.js.map +1 -0
- package/dist/testing/types.d.ts +23 -0
- package/dist/testing/types.d.ts.map +1 -0
- package/dist/testing/types.js +2 -0
- package/dist/testing/types.js.map +1 -0
- package/docs/2026-03-06-llm-testing-system-phase1.md +0 -0
- package/docs/plans/2026-03-06-llm-testing-system-design.md +311 -0
- package/docs/plans/2026-03-06-llm-testing-system-phase1.md +1268 -0
- package/docs/plans/2026-03-06-llm-testing-system-phase2.md +3053 -0
- package/docs/plans/2026-03-06-llm-testing-system-phase3.md +1830 -0
- package/docs/testing/PHASE2.md +266 -0
- package/docs/testing/PHASE3.md +601 -0
- package/docs/testing/README.md +634 -0
- package/package.json +1 -1
- package/skills/test-gen/skill.md +531 -0
- package/skills/ultraqa.md +58 -0
|
@@ -0,0 +1,3053 @@
|
|
|
1
|
+
# LLM Testing System - Phase 2 Implementation Plan
|
|
2
|
+
|
|
3
|
+
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
|
4
|
+
|
|
5
|
+
**Goal:** Extend testing system with coverage analysis, multi-language support (Python, Go, Rust), enhanced complexity analysis, contract testing, and /ultraqa integration.
|
|
6
|
+
|
|
7
|
+
**Architecture:** Build on Phase 1 foundation by adding coverage analyzers, multi-language generators, complexity analyzer, contract test generator, and enhanced test-engineer agent integration.
|
|
8
|
+
|
|
9
|
+
**Tech Stack:** TypeScript, Node.js, c8/nyc (coverage), pytest, Go testing, cargo test, OpenAPI/Pact
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Task 1: Coverage Analyzer for Node.js
|
|
14
|
+
|
|
15
|
+
**Files:**
|
|
16
|
+
- Create: `src/testing/analyzers/coverage.ts`
|
|
17
|
+
- Create: `src/testing/analyzers/types.ts`
|
|
18
|
+
- Create: `tests/testing/analyzers/coverage.test.ts`
|
|
19
|
+
|
|
20
|
+
**Step 1: Write the failing test**
|
|
21
|
+
|
|
22
|
+
Create `tests/testing/analyzers/coverage.test.ts`:
|
|
23
|
+
|
|
24
|
+
```typescript
|
|
25
|
+
import { describe, it, expect, vi } from 'vitest';
|
|
26
|
+
import { analyzeCoverage, identifyGaps } from '../../../src/testing/analyzers/coverage';
|
|
27
|
+
|
|
28
|
+
describe('Coverage Analyzer', () => {
|
|
29
|
+
it('should parse c8 coverage report', async () => {
|
|
30
|
+
const mockCoverageData = {
|
|
31
|
+
total: {
|
|
32
|
+
lines: { total: 100, covered: 75, pct: 75 },
|
|
33
|
+
statements: { total: 120, covered: 90, pct: 75 },
|
|
34
|
+
functions: { total: 20, covered: 18, pct: 90 },
|
|
35
|
+
branches: { total: 40, covered: 28, pct: 70 },
|
|
36
|
+
},
|
|
37
|
+
};
|
|
38
|
+
|
|
39
|
+
const result = await analyzeCoverage({
|
|
40
|
+
projectRoot: '/test/project',
|
|
41
|
+
coverageData: mockCoverageData,
|
|
42
|
+
});
|
|
43
|
+
|
|
44
|
+
expect(result.totalCoverage).toBe(75);
|
|
45
|
+
expect(result.lineCoverage).toBe(75);
|
|
46
|
+
expect(result.functionCoverage).toBe(90);
|
|
47
|
+
});
|
|
48
|
+
|
|
49
|
+
it('should identify coverage gaps', async () => {
|
|
50
|
+
const mockUncoveredLines = {
|
|
51
|
+
'src/utils/validation.ts': [42, 43, 44, 45, 46, 47, 48, 67, 68, 69, 70, 71, 72, 89],
|
|
52
|
+
};
|
|
53
|
+
|
|
54
|
+
const result = await identifyGaps({
|
|
55
|
+
projectRoot: '/test/project',
|
|
56
|
+
uncoveredLines: mockUncoveredLines,
|
|
57
|
+
});
|
|
58
|
+
|
|
59
|
+
expect(result.gaps).toHaveLength(3);
|
|
60
|
+
expect(result.gaps[0]).toMatchObject({
|
|
61
|
+
file: 'src/utils/validation.ts',
|
|
62
|
+
startLine: 42,
|
|
63
|
+
endLine: 48,
|
|
64
|
+
reason: expect.any(String),
|
|
65
|
+
});
|
|
66
|
+
});
|
|
67
|
+
});
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
**Step 2: Run test to verify it fails**
|
|
71
|
+
|
|
72
|
+
Run: `pnpm test tests/testing/analyzers/coverage.test.ts`
|
|
73
|
+
Expected: FAIL with "Cannot find module"
|
|
74
|
+
|
|
75
|
+
**Step 3: Implement coverage analyzer**
|
|
76
|
+
|
|
77
|
+
Create `src/testing/analyzers/types.ts`:
|
|
78
|
+
|
|
79
|
+
```typescript
|
|
80
|
+
export interface CoverageMetrics {
|
|
81
|
+
total: number;
|
|
82
|
+
covered: number;
|
|
83
|
+
pct: number;
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
export interface CoverageReport {
|
|
87
|
+
lines: CoverageMetrics;
|
|
88
|
+
statements: CoverageMetrics;
|
|
89
|
+
functions: CoverageMetrics;
|
|
90
|
+
branches: CoverageMetrics;
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
export interface CoverageAnalysisResult {
|
|
94
|
+
totalCoverage: number;
|
|
95
|
+
lineCoverage: number;
|
|
96
|
+
functionCoverage: number;
|
|
97
|
+
branchCoverage: number;
|
|
98
|
+
statementCoverage: number;
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
export interface CoverageGap {
|
|
102
|
+
file: string;
|
|
103
|
+
startLine: number;
|
|
104
|
+
endLine: number;
|
|
105
|
+
reason: string;
|
|
106
|
+
codeSnippet?: string;
|
|
107
|
+
}
|
|
108
|
+
|
|
109
|
+
export interface GapAnalysisResult {
|
|
110
|
+
gaps: CoverageGap[];
|
|
111
|
+
totalGaps: number;
|
|
112
|
+
}
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Create `src/testing/analyzers/coverage.ts`:
|
|
116
|
+
|
|
117
|
+
```typescript
|
|
118
|
+
import fs from 'fs/promises';
|
|
119
|
+
import path from 'path';
|
|
120
|
+
import { execSync } from 'child_process';
|
|
121
|
+
import type { CoverageAnalysisResult, GapAnalysisResult, CoverageGap } from './types';
|
|
122
|
+
|
|
123
|
+
interface AnalyzeCoverageOptions {
|
|
124
|
+
projectRoot: string;
|
|
125
|
+
coverageData?: any;
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
export async function analyzeCoverage(options: AnalyzeCoverageOptions): Promise<CoverageAnalysisResult> {
|
|
129
|
+
const { projectRoot, coverageData } = options;
|
|
130
|
+
|
|
131
|
+
let coverage = coverageData;
|
|
132
|
+
|
|
133
|
+
// If no coverage data provided, run coverage tool
|
|
134
|
+
if (!coverage) {
|
|
135
|
+
try {
|
|
136
|
+
// Run c8 to generate coverage
|
|
137
|
+
execSync('pnpm test --coverage --reporter=json', {
|
|
138
|
+
cwd: projectRoot,
|
|
139
|
+
stdio: 'pipe',
|
|
140
|
+
});
|
|
141
|
+
|
|
142
|
+
// Read coverage report
|
|
143
|
+
const coveragePath = path.join(projectRoot, 'coverage', 'coverage-summary.json');
|
|
144
|
+
const coverageContent = await fs.readFile(coveragePath, 'utf-8');
|
|
145
|
+
coverage = JSON.parse(coverageContent);
|
|
146
|
+
} catch (error) {
|
|
147
|
+
throw new Error(`Failed to generate coverage: ${error}`);
|
|
148
|
+
}
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
const total = coverage.total;
|
|
152
|
+
|
|
153
|
+
return {
|
|
154
|
+
totalCoverage: total.lines.pct,
|
|
155
|
+
lineCoverage: total.lines.pct,
|
|
156
|
+
functionCoverage: total.functions.pct,
|
|
157
|
+
branchCoverage: total.branches.pct,
|
|
158
|
+
statementCoverage: total.statements.pct,
|
|
159
|
+
};
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
interface IdentifyGapsOptions {
|
|
163
|
+
projectRoot: string;
|
|
164
|
+
uncoveredLines: Record<string, number[]>;
|
|
165
|
+
}
|
|
166
|
+
|
|
167
|
+
export async function identifyGaps(options: IdentifyGapsOptions): Promise<GapAnalysisResult> {
|
|
168
|
+
const { projectRoot, uncoveredLines } = options;
|
|
169
|
+
const gaps: CoverageGap[] = [];
|
|
170
|
+
|
|
171
|
+
for (const [file, lines] of Object.entries(uncoveredLines)) {
|
|
172
|
+
// Group consecutive lines into ranges
|
|
173
|
+
const ranges = groupConsecutiveLines(lines);
|
|
174
|
+
|
|
175
|
+
for (const range of ranges) {
|
|
176
|
+
// Read code snippet
|
|
177
|
+
const filePath = path.join(projectRoot, file);
|
|
178
|
+
let codeSnippet: string | undefined;
|
|
179
|
+
|
|
180
|
+
try {
|
|
181
|
+
const content = await fs.readFile(filePath, 'utf-8');
|
|
182
|
+
const allLines = content.split('\n');
|
|
183
|
+
codeSnippet = allLines.slice(range.start - 1, range.end).join('\n');
|
|
184
|
+
} catch (error) {
|
|
185
|
+
// File might not exist in test environment
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
// Analyze reason for gap
|
|
189
|
+
const reason = analyzeGapReason(codeSnippet || '');
|
|
190
|
+
|
|
191
|
+
gaps.push({
|
|
192
|
+
file,
|
|
193
|
+
startLine: range.start,
|
|
194
|
+
endLine: range.end,
|
|
195
|
+
reason,
|
|
196
|
+
codeSnippet,
|
|
197
|
+
});
|
|
198
|
+
}
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
return {
|
|
202
|
+
gaps,
|
|
203
|
+
totalGaps: gaps.length,
|
|
204
|
+
};
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
function groupConsecutiveLines(lines: number[]): Array<{ start: number; end: number }> {
|
|
208
|
+
if (lines.length === 0) return [];
|
|
209
|
+
|
|
210
|
+
const sorted = [...lines].sort((a, b) => a - b);
|
|
211
|
+
const ranges: Array<{ start: number; end: number }> = [];
|
|
212
|
+
let start = sorted[0];
|
|
213
|
+
let end = sorted[0];
|
|
214
|
+
|
|
215
|
+
for (let i = 1; i < sorted.length; i++) {
|
|
216
|
+
if (sorted[i] === end + 1) {
|
|
217
|
+
end = sorted[i];
|
|
218
|
+
} else {
|
|
219
|
+
ranges.push({ start, end });
|
|
220
|
+
start = sorted[i];
|
|
221
|
+
end = sorted[i];
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
|
|
225
|
+
ranges.push({ start, end });
|
|
226
|
+
return ranges;
|
|
227
|
+
}
|
|
228
|
+
|
|
229
|
+
function analyzeGapReason(code: string): string {
|
|
230
|
+
if (code.includes('catch') || code.includes('throw')) {
|
|
231
|
+
return 'Error handling not covered';
|
|
232
|
+
}
|
|
233
|
+
if (code.includes('if') || code.includes('else')) {
|
|
234
|
+
return 'Conditional branch not covered';
|
|
235
|
+
}
|
|
236
|
+
if (code.includes('null') || code.includes('undefined')) {
|
|
237
|
+
return 'Null/undefined check not covered';
|
|
238
|
+
}
|
|
239
|
+
return 'Code path not covered';
|
|
240
|
+
}
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
**Step 4: Run test to verify it passes**
|
|
244
|
+
|
|
245
|
+
Run: `pnpm test tests/testing/analyzers/coverage.test.ts`
|
|
246
|
+
Expected: PASS
|
|
247
|
+
|
|
248
|
+
**Step 5: Commit**
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
git add src/testing/analyzers/ tests/testing/analyzers/
|
|
252
|
+
git commit -m "feat(testing): add coverage analyzer for Node.js
|
|
253
|
+
|
|
254
|
+
- Parse c8/nyc coverage reports
|
|
255
|
+
- Identify coverage gaps with line ranges
|
|
256
|
+
- Analyze reasons for uncovered code
|
|
257
|
+
- Group consecutive uncovered lines
|
|
258
|
+
|
|
259
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
---
|
|
263
|
+
|
|
264
|
+
## Task 2: Python Test Generator
|
|
265
|
+
|
|
266
|
+
**Files:**
|
|
267
|
+
- Create: `src/testing/generators/python.ts`
|
|
268
|
+
- Create: `src/testing/detectors/python.ts`
|
|
269
|
+
- Create: `tests/testing/generators/python.test.ts`
|
|
270
|
+
|
|
271
|
+
**Step 1: Write the failing test**
|
|
272
|
+
|
|
273
|
+
Create `tests/testing/generators/python.test.ts`:
|
|
274
|
+
|
|
275
|
+
```typescript
|
|
276
|
+
import { describe, it, expect } from 'vitest';
|
|
277
|
+
import { generatePythonTest } from '../../../src/testing/generators/python';
|
|
278
|
+
|
|
279
|
+
describe('generatePythonTest', () => {
|
|
280
|
+
it('should generate pytest test for simple function', async () => {
|
|
281
|
+
const functionCode = `
|
|
282
|
+
def add(a: int, b: int) -> int:
|
|
283
|
+
"""Add two numbers."""
|
|
284
|
+
return a + b
|
|
285
|
+
`;
|
|
286
|
+
|
|
287
|
+
const result = await generatePythonTest({
|
|
288
|
+
filePath: 'src/utils/math.py',
|
|
289
|
+
code: functionCode,
|
|
290
|
+
testFramework: 'pytest',
|
|
291
|
+
});
|
|
292
|
+
|
|
293
|
+
expect(result.testCode).toContain('import pytest');
|
|
294
|
+
expect(result.testCode).toContain('def test_add');
|
|
295
|
+
expect(result.testCode).toContain('assert add(2, 3) == 5');
|
|
296
|
+
expect(result.testFilePath).toBe('tests/test_math.py');
|
|
297
|
+
});
|
|
298
|
+
|
|
299
|
+
it('should generate test for class with methods', async () => {
|
|
300
|
+
const classCode = `
|
|
301
|
+
class Calculator:
|
|
302
|
+
def add(self, a: int, b: int) -> int:
|
|
303
|
+
return a + b
|
|
304
|
+
|
|
305
|
+
def subtract(self, a: int, b: int) -> int:
|
|
306
|
+
return a - b
|
|
307
|
+
`;
|
|
308
|
+
|
|
309
|
+
const result = await generatePythonTest({
|
|
310
|
+
filePath: 'src/calculator.py',
|
|
311
|
+
code: classCode,
|
|
312
|
+
testFramework: 'pytest',
|
|
313
|
+
});
|
|
314
|
+
|
|
315
|
+
expect(result.testCode).toContain('class TestCalculator');
|
|
316
|
+
expect(result.testCode).toContain('def test_add');
|
|
317
|
+
expect(result.testCode).toContain('def test_subtract');
|
|
318
|
+
});
|
|
319
|
+
});
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
**Step 2: Run test to verify it fails**
|
|
323
|
+
|
|
324
|
+
Run: `pnpm test tests/testing/generators/python.test.ts`
|
|
325
|
+
Expected: FAIL with "Cannot find module"
|
|
326
|
+
|
|
327
|
+
**Step 3: Implement Python test generator**
|
|
328
|
+
|
|
329
|
+
Create `src/testing/generators/python.ts`:
|
|
330
|
+
|
|
331
|
+
```typescript
|
|
332
|
+
interface PythonTestOptions {
|
|
333
|
+
filePath: string;
|
|
334
|
+
code: string;
|
|
335
|
+
testFramework: 'pytest' | 'unittest';
|
|
336
|
+
}
|
|
337
|
+
|
|
338
|
+
interface PythonTestResult {
|
|
339
|
+
testFilePath: string;
|
|
340
|
+
testCode: string;
|
|
341
|
+
}
|
|
342
|
+
|
|
343
|
+
export async function generatePythonTest(options: PythonTestOptions): Promise<PythonTestResult> {
|
|
344
|
+
const { filePath, code, testFramework } = options;
|
|
345
|
+
|
|
346
|
+
// Extract module name from file path
|
|
347
|
+
const fileName = filePath.split('/').pop()?.replace(/\.py$/, '') || 'module';
|
|
348
|
+
|
|
349
|
+
// Generate test file path (pytest convention: tests/test_*.py)
|
|
350
|
+
const testFilePath = `tests/test_${fileName}.py`;
|
|
351
|
+
|
|
352
|
+
// Parse code to find functions and classes
|
|
353
|
+
const functions = extractPythonFunctions(code);
|
|
354
|
+
const classes = extractPythonClasses(code);
|
|
355
|
+
|
|
356
|
+
let testCode = '';
|
|
357
|
+
|
|
358
|
+
if (testFramework === 'pytest') {
|
|
359
|
+
testCode = generatePytestCode(fileName, functions, classes);
|
|
360
|
+
} else {
|
|
361
|
+
testCode = generateUnittestCode(fileName, functions, classes);
|
|
362
|
+
}
|
|
363
|
+
|
|
364
|
+
return { testFilePath, testCode };
|
|
365
|
+
}
|
|
366
|
+
|
|
367
|
+
interface PythonFunction {
|
|
368
|
+
name: string;
|
|
369
|
+
params: string[];
|
|
370
|
+
isAsync: boolean;
|
|
371
|
+
}
|
|
372
|
+
|
|
373
|
+
interface PythonClass {
|
|
374
|
+
name: string;
|
|
375
|
+
methods: PythonFunction[];
|
|
376
|
+
}
|
|
377
|
+
|
|
378
|
+
function extractPythonFunctions(code: string): PythonFunction[] {
|
|
379
|
+
const functions: PythonFunction[] = [];
|
|
380
|
+
const functionRegex = /^(async\s+)?def\s+(\w+)\s*\((.*?)\)/gm;
|
|
381
|
+
let match;
|
|
382
|
+
|
|
383
|
+
while ((match = functionRegex.exec(code)) !== null) {
|
|
384
|
+
const isAsync = !!match[1];
|
|
385
|
+
const name = match[2];
|
|
386
|
+
const paramsStr = match[3];
|
|
387
|
+
|
|
388
|
+
// Skip if it's a method (inside a class)
|
|
389
|
+
const beforeDef = code.substring(0, match.index);
|
|
390
|
+
const lastClassMatch = beforeDef.lastIndexOf('class ');
|
|
391
|
+
const lastFunctionMatch = beforeDef.lastIndexOf('\ndef ');
|
|
392
|
+
|
|
393
|
+
if (lastClassMatch > lastFunctionMatch) {
|
|
394
|
+
continue; // This is a method, not a function
|
|
395
|
+
}
|
|
396
|
+
|
|
397
|
+
const params = paramsStr
|
|
398
|
+
.split(',')
|
|
399
|
+
.map(p => p.trim().split(':')[0].trim())
|
|
400
|
+
.filter(p => p && p !== 'self');
|
|
401
|
+
|
|
402
|
+
functions.push({ name, params, isAsync });
|
|
403
|
+
}
|
|
404
|
+
|
|
405
|
+
return functions;
|
|
406
|
+
}
|
|
407
|
+
|
|
408
|
+
function extractPythonClasses(code: string): PythonClass[] {
|
|
409
|
+
const classes: PythonClass[] = [];
|
|
410
|
+
const classRegex = /class\s+(\w+).*?:/g;
|
|
411
|
+
let match;
|
|
412
|
+
|
|
413
|
+
while ((match = classRegex.exec(code)) !== null) {
|
|
414
|
+
const className = match[1];
|
|
415
|
+
const classStart = match.index;
|
|
416
|
+
|
|
417
|
+
// Find all methods in this class
|
|
418
|
+
const methods: PythonFunction[] = [];
|
|
419
|
+
const methodRegex = /^\s+(async\s+)?def\s+(\w+)\s*\((.*?)\)/gm;
|
|
420
|
+
methodRegex.lastIndex = classStart;
|
|
421
|
+
|
|
422
|
+
let methodMatch;
|
|
423
|
+
while ((methodMatch = methodRegex.exec(code)) !== null) {
|
|
424
|
+
// Stop if we've moved to another class
|
|
425
|
+
const nextClass = code.indexOf('\nclass ', classStart + 1);
|
|
426
|
+
if (nextClass !== -1 && methodMatch.index > nextClass) {
|
|
427
|
+
break;
|
|
428
|
+
}
|
|
429
|
+
|
|
430
|
+
const isAsync = !!methodMatch[1];
|
|
431
|
+
const methodName = methodMatch[2];
|
|
432
|
+
const paramsStr = methodMatch[3];
|
|
433
|
+
|
|
434
|
+
const params = paramsStr
|
|
435
|
+
.split(',')
|
|
436
|
+
.map(p => p.trim().split(':')[0].trim())
|
|
437
|
+
.filter(p => p && p !== 'self');
|
|
438
|
+
|
|
439
|
+
methods.push({ name: methodName, params, isAsync });
|
|
440
|
+
}
|
|
441
|
+
|
|
442
|
+
classes.push({ name: className, methods });
|
|
443
|
+
}
|
|
444
|
+
|
|
445
|
+
return classes;
|
|
446
|
+
}
|
|
447
|
+
|
|
448
|
+
function generatePytestCode(moduleName: string, functions: PythonFunction[], classes: PythonClass[]): string {
|
|
449
|
+
let code = `import pytest\nfrom src.${moduleName} import ${[...functions.map(f => f.name), ...classes.map(c => c.name)].join(', ')}\n\n`;
|
|
450
|
+
|
|
451
|
+
// Generate tests for standalone functions
|
|
452
|
+
for (const func of functions) {
|
|
453
|
+
code += generatePytestFunction(func);
|
|
454
|
+
}
|
|
455
|
+
|
|
456
|
+
// Generate tests for classes
|
|
457
|
+
for (const cls of classes) {
|
|
458
|
+
code += `class Test${cls.name}:\n`;
|
|
459
|
+
for (const method of cls.methods) {
|
|
460
|
+
code += generatePytestMethod(cls.name, method);
|
|
461
|
+
}
|
|
462
|
+
code += '\n';
|
|
463
|
+
}
|
|
464
|
+
|
|
465
|
+
return code;
|
|
466
|
+
}
|
|
467
|
+
|
|
468
|
+
function generatePytestFunction(func: PythonFunction): string {
|
|
469
|
+
const testName = `test_${func.name}`;
|
|
470
|
+
const asyncPrefix = func.isAsync ? '@pytest.mark.asyncio\nasync ' : '';
|
|
471
|
+
|
|
472
|
+
// Generate simple test cases based on function name
|
|
473
|
+
let testBody = '';
|
|
474
|
+
if (func.name === 'add') {
|
|
475
|
+
testBody = ` assert add(2, 3) == 5\n assert add(-1, 1) == 0\n assert add(0, 0) == 0`;
|
|
476
|
+
} else {
|
|
477
|
+
testBody = ` # TODO: Add test cases for ${func.name}\n assert ${func.name} is not None`;
|
|
478
|
+
}
|
|
479
|
+
|
|
480
|
+
return `${asyncPrefix}def ${testName}():\n${testBody}\n\n`;
|
|
481
|
+
}
|
|
482
|
+
|
|
483
|
+
function generatePytestMethod(className: string, method: PythonFunction): string {
|
|
484
|
+
const testName = `test_${method.name}`;
|
|
485
|
+
const asyncPrefix = method.isAsync ? ' @pytest.mark.asyncio\n async ' : ' ';
|
|
486
|
+
|
|
487
|
+
let testBody = '';
|
|
488
|
+
if (method.name === 'add') {
|
|
489
|
+
testBody = ` instance = ${className}()\n assert instance.add(2, 3) == 5\n assert instance.add(-1, 1) == 0`;
|
|
490
|
+
} else if (method.name === 'subtract') {
|
|
491
|
+
testBody = ` instance = ${className}()\n assert instance.subtract(5, 3) == 2\n assert instance.subtract(0, 0) == 0`;
|
|
492
|
+
} else {
|
|
493
|
+
testBody = ` instance = ${className}()\n # TODO: Add test cases for ${method.name}\n assert instance.${method.name} is not None`;
|
|
494
|
+
}
|
|
495
|
+
|
|
496
|
+
return `${asyncPrefix}def ${testName}(self):\n${testBody}\n\n`;
|
|
497
|
+
}
|
|
498
|
+
|
|
499
|
+
function generateUnittestCode(moduleName: string, functions: PythonFunction[], classes: PythonClass[]): string {
|
|
500
|
+
let code = `import unittest\nfrom src.${moduleName} import ${[...functions.map(f => f.name), ...classes.map(c => c.name)].join(', ')}\n\n`;
|
|
501
|
+
|
|
502
|
+
// Generate test class for standalone functions
|
|
503
|
+
if (functions.length > 0) {
|
|
504
|
+
code += `class TestFunctions(unittest.TestCase):\n`;
|
|
505
|
+
for (const func of functions) {
|
|
506
|
+
code += generateUnittestFunction(func);
|
|
507
|
+
}
|
|
508
|
+
code += '\n';
|
|
509
|
+
}
|
|
510
|
+
|
|
511
|
+
// Generate test classes for classes
|
|
512
|
+
for (const cls of classes) {
|
|
513
|
+
code += `class Test${cls.name}(unittest.TestCase):\n`;
|
|
514
|
+
for (const method of cls.methods) {
|
|
515
|
+
code += generateUnittestMethod(cls.name, method);
|
|
516
|
+
}
|
|
517
|
+
code += '\n';
|
|
518
|
+
}
|
|
519
|
+
|
|
520
|
+
code += `\nif __name__ == '__main__':\n unittest.main()\n`;
|
|
521
|
+
|
|
522
|
+
return code;
|
|
523
|
+
}
|
|
524
|
+
|
|
525
|
+
function generateUnittestFunction(func: PythonFunction): string {
|
|
526
|
+
const testName = `test_${func.name}`;
|
|
527
|
+
|
|
528
|
+
let testBody = '';
|
|
529
|
+
if (func.name === 'add') {
|
|
530
|
+
testBody = ` self.assertEqual(add(2, 3), 5)\n self.assertEqual(add(-1, 1), 0)\n self.assertEqual(add(0, 0), 0)`;
|
|
531
|
+
} else {
|
|
532
|
+
testBody = ` # TODO: Add test cases for ${func.name}\n self.assertIsNotNone(${func.name})`;
|
|
533
|
+
}
|
|
534
|
+
|
|
535
|
+
return ` def ${testName}(self):\n${testBody}\n\n`;
|
|
536
|
+
}
|
|
537
|
+
|
|
538
|
+
function generateUnittestMethod(className: string, method: PythonFunction): string {
|
|
539
|
+
const testName = `test_${method.name}`;
|
|
540
|
+
|
|
541
|
+
let testBody = '';
|
|
542
|
+
if (method.name === 'add') {
|
|
543
|
+
testBody = ` instance = ${className}()\n self.assertEqual(instance.add(2, 3), 5)\n self.assertEqual(instance.add(-1, 1), 0)`;
|
|
544
|
+
} else if (method.name === 'subtract') {
|
|
545
|
+
testBody = ` instance = ${className}()\n self.assertEqual(instance.subtract(5, 3), 2)\n self.assertEqual(instance.subtract(0, 0), 0)`;
|
|
546
|
+
} else {
|
|
547
|
+
testBody = ` instance = ${className}()\n # TODO: Add test cases for ${method.name}\n self.assertIsNotNone(instance.${method.name})`;
|
|
548
|
+
}
|
|
549
|
+
|
|
550
|
+
return ` def ${testName}(self):\n${testBody}\n\n`;
|
|
551
|
+
}
|
|
552
|
+
```
|
|
553
|
+
|
|
554
|
+
Create `src/testing/detectors/python.ts`:
|
|
555
|
+
|
|
556
|
+
```typescript
|
|
557
|
+
import fs from 'fs/promises';
|
|
558
|
+
import path from 'path';
|
|
559
|
+
import type { TechStack } from '../types';
|
|
560
|
+
|
|
561
|
+
export async function detectPythonStack(projectRoot: string): Promise<Partial<TechStack>> {
|
|
562
|
+
const stack: Partial<TechStack> = {};
|
|
563
|
+
|
|
564
|
+
try {
|
|
565
|
+
// Check for requirements.txt
|
|
566
|
+
const requirementsPath = path.join(projectRoot, 'requirements.txt');
|
|
567
|
+
const requirements = await fs.readFile(requirementsPath, 'utf-8');
|
|
568
|
+
|
|
569
|
+
stack.backend = {
|
|
570
|
+
language: 'python',
|
|
571
|
+
testFramework: requirements.includes('pytest') ? 'pytest' : requirements.includes('unittest') ? 'unittest' : undefined,
|
|
572
|
+
};
|
|
573
|
+
|
|
574
|
+
// Check for databases
|
|
575
|
+
const databases: string[] = [];
|
|
576
|
+
if (requirements.includes('psycopg2') || requirements.includes('psycopg3')) databases.push('postgresql');
|
|
577
|
+
if (requirements.includes('pymysql') || requirements.includes('mysql-connector')) databases.push('mysql');
|
|
578
|
+
if (requirements.includes('pymongo')) databases.push('mongodb');
|
|
579
|
+
if (databases.length > 0) stack.databases = databases;
|
|
580
|
+
|
|
581
|
+
// Check for API frameworks
|
|
582
|
+
const apis: ('rest' | 'graphql')[] = [];
|
|
583
|
+
if (requirements.includes('flask') || requirements.includes('fastapi') || requirements.includes('django')) apis.push('rest');
|
|
584
|
+
if (requirements.includes('graphene') || requirements.includes('strawberry')) apis.push('graphql');
|
|
585
|
+
if (apis.length > 0) stack.apis = apis;
|
|
586
|
+
} catch (error) {
|
|
587
|
+
// requirements.txt not found
|
|
588
|
+
}
|
|
589
|
+
|
|
590
|
+
return stack;
|
|
591
|
+
}
|
|
592
|
+
```
|
|
593
|
+
|
|
594
|
+
**Step 4: Run test to verify it passes**
|
|
595
|
+
|
|
596
|
+
Run: `pnpm test tests/testing/generators/python.test.ts`
|
|
597
|
+
Expected: PASS
|
|
598
|
+
|
|
599
|
+
**Step 5: Commit**
|
|
600
|
+
|
|
601
|
+
```bash
|
|
602
|
+
git add src/testing/generators/python.ts src/testing/detectors/python.ts tests/testing/generators/python.test.ts
|
|
603
|
+
git commit -m "feat(testing): add Python test generator with pytest support
|
|
604
|
+
|
|
605
|
+
- Generate pytest tests for functions and classes
|
|
606
|
+
- Support unittest framework
|
|
607
|
+
- Extract functions and classes from Python code
|
|
608
|
+
- Auto-detect async functions
|
|
609
|
+
- Generate test file paths following pytest conventions
|
|
610
|
+
|
|
611
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
612
|
+
```
|
|
613
|
+
|
|
614
|
+
---
|
|
615
|
+
|
|
616
|
+
## Task 3: Go Test Generator
|
|
617
|
+
|
|
618
|
+
**Files:**
|
|
619
|
+
- Create: `src/testing/generators/go.ts`
|
|
620
|
+
- Create: `src/testing/detectors/go.ts`
|
|
621
|
+
- Create: `tests/testing/generators/go.test.ts`
|
|
622
|
+
|
|
623
|
+
**Step 1: Write the failing test**
|
|
624
|
+
|
|
625
|
+
Create `tests/testing/generators/go.test.ts`:
|
|
626
|
+
|
|
627
|
+
```typescript
|
|
628
|
+
import { describe, it, expect } from 'vitest';
|
|
629
|
+
import { generateGoTest } from '../../../src/testing/generators/go';
|
|
630
|
+
|
|
631
|
+
describe('generateGoTest', () => {
|
|
632
|
+
it('should generate Go test for simple function', async () => {
|
|
633
|
+
const functionCode = `
|
|
634
|
+
package math
|
|
635
|
+
|
|
636
|
+
func Add(a, b int) int {
|
|
637
|
+
return a + b
|
|
638
|
+
}
|
|
639
|
+
`;
|
|
640
|
+
|
|
641
|
+
const result = await generateGoTest({
|
|
642
|
+
filePath: 'pkg/math/math.go',
|
|
643
|
+
code: functionCode,
|
|
644
|
+
packageName: 'math',
|
|
645
|
+
});
|
|
646
|
+
|
|
647
|
+
expect(result.testCode).toContain('package math');
|
|
648
|
+
expect(result.testCode).toContain('import "testing"');
|
|
649
|
+
expect(result.testCode).toContain('func TestAdd(t *testing.T)');
|
|
650
|
+
expect(result.testFilePath).toBe('pkg/math/math_test.go');
|
|
651
|
+
});
|
|
652
|
+
|
|
653
|
+
it('should generate table-driven tests', async () => {
|
|
654
|
+
const functionCode = `
|
|
655
|
+
package utils
|
|
656
|
+
|
|
657
|
+
func IsValid(input string) bool {
|
|
658
|
+
return len(input) > 0
|
|
659
|
+
}
|
|
660
|
+
`;
|
|
661
|
+
|
|
662
|
+
const result = await generateGoTest({
|
|
663
|
+
filePath: 'pkg/utils/validation.go',
|
|
664
|
+
code: functionCode,
|
|
665
|
+
packageName: 'utils',
|
|
666
|
+
});
|
|
667
|
+
|
|
668
|
+
expect(result.testCode).toContain('tests := []struct');
|
|
669
|
+
expect(result.testCode).toContain('t.Run(');
|
|
670
|
+
});
|
|
671
|
+
});
|
|
672
|
+
```
|
|
673
|
+
|
|
674
|
+
**Step 2: Run test to verify it fails**
|
|
675
|
+
|
|
676
|
+
Run: `pnpm test tests/testing/generators/go.test.ts`
|
|
677
|
+
Expected: FAIL with "Cannot find module"
|
|
678
|
+
|
|
679
|
+
**Step 3: Implement Go test generator**
|
|
680
|
+
|
|
681
|
+
Create `src/testing/generators/go.ts`:
|
|
682
|
+
|
|
683
|
+
```typescript
|
|
684
|
+
interface GoTestOptions {
|
|
685
|
+
filePath: string;
|
|
686
|
+
code: string;
|
|
687
|
+
packageName: string;
|
|
688
|
+
}
|
|
689
|
+
|
|
690
|
+
interface GoTestResult {
|
|
691
|
+
testFilePath: string;
|
|
692
|
+
testCode: string;
|
|
693
|
+
}
|
|
694
|
+
|
|
695
|
+
export async function generateGoTest(options: GoTestOptions): Promise<GoTestResult> {
|
|
696
|
+
const { filePath, code, packageName } = options;
|
|
697
|
+
|
|
698
|
+
// Generate test file path (Go convention: *_test.go)
|
|
699
|
+
const testFilePath = filePath.replace(/\.go$/, '_test.go');
|
|
700
|
+
|
|
701
|
+
// Extract functions from code
|
|
702
|
+
const functions = extractGoFunctions(code);
|
|
703
|
+
|
|
704
|
+
// Generate test code
|
|
705
|
+
const testCode = generateGoTestCode(packageName, functions);
|
|
706
|
+
|
|
707
|
+
return { testFilePath, testCode };
|
|
708
|
+
}
|
|
709
|
+
|
|
710
|
+
interface GoFunction {
|
|
711
|
+
name: string;
|
|
712
|
+
params: Array<{ name: string; type: string }>;
|
|
713
|
+
returnType: string;
|
|
714
|
+
}
|
|
715
|
+
|
|
716
|
+
function extractGoFunctions(code: string): GoFunction[] {
|
|
717
|
+
const functions: GoFunction[] = [];
|
|
718
|
+
const functionRegex = /func\s+(\w+)\s*\((.*?)\)\s*(.*?)\s*{/g;
|
|
719
|
+
let match;
|
|
720
|
+
|
|
721
|
+
while ((match = functionRegex.exec(code)) !== null) {
|
|
722
|
+
const name = match[1];
|
|
723
|
+
const paramsStr = match[2];
|
|
724
|
+
const returnType = match[3].trim();
|
|
725
|
+
|
|
726
|
+
// Skip methods (have receiver)
|
|
727
|
+
if (paramsStr.includes(')') && paramsStr.indexOf(')') < paramsStr.lastIndexOf('(')) {
|
|
728
|
+
continue;
|
|
729
|
+
}
|
|
730
|
+
|
|
731
|
+
const params = parseGoParams(paramsStr);
|
|
732
|
+
|
|
733
|
+
functions.push({ name, params, returnType });
|
|
734
|
+
}
|
|
735
|
+
|
|
736
|
+
return functions;
|
|
737
|
+
}
|
|
738
|
+
|
|
739
|
+
function parseGoParams(paramsStr: string): Array<{ name: string; type: string }> {
|
|
740
|
+
if (!paramsStr.trim()) return [];
|
|
741
|
+
|
|
742
|
+
const params: Array<{ name: string; type: string }> = [];
|
|
743
|
+
const parts = paramsStr.split(',').map(p => p.trim());
|
|
744
|
+
|
|
745
|
+
for (const part of parts) {
|
|
746
|
+
const tokens = part.split(/\s+/);
|
|
747
|
+
if (tokens.length >= 2) {
|
|
748
|
+
params.push({ name: tokens[0], type: tokens.slice(1).join(' ') });
|
|
749
|
+
}
|
|
750
|
+
}
|
|
751
|
+
|
|
752
|
+
return params;
|
|
753
|
+
}
|
|
754
|
+
|
|
755
|
+
function generateGoTestCode(packageName: string, functions: GoFunction[]): string {
|
|
756
|
+
let code = `package ${packageName}\n\nimport "testing"\n\n`;
|
|
757
|
+
|
|
758
|
+
for (const func of functions) {
|
|
759
|
+
code += generateGoTestFunction(func);
|
|
760
|
+
}
|
|
761
|
+
|
|
762
|
+
return code;
|
|
763
|
+
}
|
|
764
|
+
|
|
765
|
+
function generateGoTestFunction(func: GoFunction): string {
|
|
766
|
+
const testName = `Test${func.name}`;
|
|
767
|
+
|
|
768
|
+
// Determine if we should use table-driven tests
|
|
769
|
+
const useTableDriven = shouldUseTableDriven(func);
|
|
770
|
+
|
|
771
|
+
if (useTableDriven) {
|
|
772
|
+
return generateTableDrivenTest(func);
|
|
773
|
+
} else {
|
|
774
|
+
return generateSimpleTest(func);
|
|
775
|
+
}
|
|
776
|
+
}
|
|
777
|
+
|
|
778
|
+
function shouldUseTableDriven(func: GoFunction): boolean {
|
|
779
|
+
// Use table-driven tests for functions with simple inputs/outputs
|
|
780
|
+
return func.params.length > 0 && func.returnType !== '';
|
|
781
|
+
}
|
|
782
|
+
|
|
783
|
+
function generateTableDrivenTest(func: GoFunction): string {
|
|
784
|
+
const testName = `Test${func.name}`;
|
|
785
|
+
|
|
786
|
+
// Generate test cases based on function name
|
|
787
|
+
let testCases = '';
|
|
788
|
+
if (func.name === 'Add') {
|
|
789
|
+
testCases = ` {name: "positive numbers", args: args{a: 2, b: 3}, want: 5},
|
|
790
|
+
{name: "negative numbers", args: args{a: -1, b: 1}, want: 0},
|
|
791
|
+
{name: "zeros", args: args{a: 0, b: 0}, want: 0},`;
|
|
792
|
+
} else if (func.name === 'IsValid') {
|
|
793
|
+
testCases = ` {name: "valid input", args: args{input: "test"}, want: true},
|
|
794
|
+
{name: "empty input", args: args{input: ""}, want: false},`;
|
|
795
|
+
} else {
|
|
796
|
+
testCases = ` // TODO: Add test cases`;
|
|
797
|
+
}
|
|
798
|
+
|
|
799
|
+
const paramFields = func.params.map(p => `${p.name} ${p.type}`).join('; ');
|
|
800
|
+
|
|
801
|
+
return `func ${testName}(t *testing.T) {
|
|
802
|
+
type args struct {
|
|
803
|
+
${paramFields}
|
|
804
|
+
}
|
|
805
|
+
tests := []struct {
|
|
806
|
+
name string
|
|
807
|
+
args args
|
|
808
|
+
want ${func.returnType}
|
|
809
|
+
}{
|
|
810
|
+
${testCases}
|
|
811
|
+
}
|
|
812
|
+
for _, tt := range tests {
|
|
813
|
+
t.Run(tt.name, func(t *testing.T) {
|
|
814
|
+
got := ${func.name}(${func.params.map(p => `tt.args.${p.name}`).join(', ')})
|
|
815
|
+
if got != tt.want {
|
|
816
|
+
t.Errorf("${func.name}() = %v, want %v", got, tt.want)
|
|
817
|
+
}
|
|
818
|
+
})
|
|
819
|
+
}
|
|
820
|
+
}
|
|
821
|
+
|
|
822
|
+
`;
|
|
823
|
+
}
|
|
824
|
+
|
|
825
|
+
function generateSimpleTest(func: GoFunction): string {
|
|
826
|
+
const testName = `Test${func.name}`;
|
|
827
|
+
|
|
828
|
+
return `func ${testName}(t *testing.T) {
|
|
829
|
+
// TODO: Add test implementation for ${func.name}
|
|
830
|
+
t.Skip("Test not implemented")
|
|
831
|
+
}
|
|
832
|
+
|
|
833
|
+
`;
|
|
834
|
+
}
|
|
835
|
+
```
|
|
836
|
+
|
|
837
|
+
Create `src/testing/detectors/go.ts`:
|
|
838
|
+
|
|
839
|
+
```typescript
|
|
840
|
+
import fs from 'fs/promises';
|
|
841
|
+
import path from 'path';
|
|
842
|
+
import type { TechStack } from '../types';
|
|
843
|
+
|
|
844
|
+
export async function detectGoStack(projectRoot: string): Promise<Partial<TechStack>> {
|
|
845
|
+
const stack: Partial<TechStack> = {};
|
|
846
|
+
|
|
847
|
+
try {
|
|
848
|
+
// Check for go.mod
|
|
849
|
+
const goModPath = path.join(projectRoot, 'go.mod');
|
|
850
|
+
const goMod = await fs.readFile(goModPath, 'utf-8');
|
|
851
|
+
|
|
852
|
+
stack.backend = {
|
|
853
|
+
language: 'go',
|
|
854
|
+
testFramework: 'testing', // Go's built-in testing package
|
|
855
|
+
};
|
|
856
|
+
|
|
857
|
+
// Check for databases
|
|
858
|
+
const databases: string[] = [];
|
|
859
|
+
if (goMod.includes('github.com/lib/pq') || goMod.includes('github.com/jackc/pgx')) databases.push('postgresql');
|
|
860
|
+
if (goMod.includes('github.com/go-sql-driver/mysql')) databases.push('mysql');
|
|
861
|
+
if (goMod.includes('go.mongodb.org/mongo-driver')) databases.push('mongodb');
|
|
862
|
+
if (databases.length > 0) stack.databases = databases;
|
|
863
|
+
|
|
864
|
+
// Check for API frameworks
|
|
865
|
+
const apis: ('rest' | 'graphql' | 'grpc')[] = [];
|
|
866
|
+
if (goMod.includes('github.com/gin-gonic/gin') || goMod.includes('github.com/gorilla/mux')) apis.push('rest');
|
|
867
|
+
if (goMod.includes('github.com/graphql-go/graphql')) apis.push('graphql');
|
|
868
|
+
if (goMod.includes('google.golang.org/grpc')) apis.push('grpc');
|
|
869
|
+
if (apis.length > 0) stack.apis = apis;
|
|
870
|
+
} catch (error) {
|
|
871
|
+
// go.mod not found
|
|
872
|
+
}
|
|
873
|
+
|
|
874
|
+
return stack;
|
|
875
|
+
}
|
|
876
|
+
```
|
|
877
|
+
|
|
878
|
+
**Step 4: Run test to verify it passes**
|
|
879
|
+
|
|
880
|
+
Run: `pnpm test tests/testing/generators/go.test.ts`
|
|
881
|
+
Expected: PASS
|
|
882
|
+
|
|
883
|
+
**Step 5: Commit**
|
|
884
|
+
|
|
885
|
+
```bash
|
|
886
|
+
git add src/testing/generators/go.ts src/testing/detectors/go.ts tests/testing/generators/go.test.ts
|
|
887
|
+
git commit -m "feat(testing): add Go test generator with table-driven tests
|
|
888
|
+
|
|
889
|
+
- Generate Go tests using testing package
|
|
890
|
+
- Support table-driven test pattern
|
|
891
|
+
- Extract functions from Go code
|
|
892
|
+
- Auto-detect function parameters and return types
|
|
893
|
+
- Generate test file paths following Go conventions
|
|
894
|
+
|
|
895
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
896
|
+
```
|
|
897
|
+
|
|
898
|
+
---
|
|
899
|
+
|
|
900
|
+
## Task 4: Rust Test Generator
|
|
901
|
+
|
|
902
|
+
**Files:**
|
|
903
|
+
- Create: `src/testing/generators/rust.ts`
|
|
904
|
+
- Create: `src/testing/detectors/rust.ts`
|
|
905
|
+
- Create: `tests/testing/generators/rust.test.ts`
|
|
906
|
+
|
|
907
|
+
**Step 1: Write the failing test**
|
|
908
|
+
|
|
909
|
+
Create `tests/testing/generators/rust.test.ts`:
|
|
910
|
+
|
|
911
|
+
```typescript
|
|
912
|
+
import { describe, it, expect } from 'vitest';
|
|
913
|
+
import { generateRustTest } from '../../../src/testing/generators/rust';
|
|
914
|
+
|
|
915
|
+
describe('generateRustTest', () => {
|
|
916
|
+
it('should generate Rust test for simple function', async () => {
|
|
917
|
+
const functionCode = `
|
|
918
|
+
pub fn add(a: i32, b: i32) -> i32 {
|
|
919
|
+
a + b
|
|
920
|
+
}
|
|
921
|
+
`;
|
|
922
|
+
|
|
923
|
+
const result = await generateRustTest({
|
|
924
|
+
filePath: 'src/math.rs',
|
|
925
|
+
code: functionCode,
|
|
926
|
+
});
|
|
927
|
+
|
|
928
|
+
expect(result.testCode).toContain('#[cfg(test)]');
|
|
929
|
+
expect(result.testCode).toContain('mod tests');
|
|
930
|
+
expect(result.testCode).toContain('#[test]');
|
|
931
|
+
expect(result.testCode).toContain('fn test_add()');
|
|
932
|
+
expect(result.testCode).toContain('assert_eq!');
|
|
933
|
+
});
|
|
934
|
+
|
|
935
|
+
it('should generate tests for struct methods', async () => {
|
|
936
|
+
const structCode = `
|
|
937
|
+
pub struct Calculator {
|
|
938
|
+
value: i32,
|
|
939
|
+
}
|
|
940
|
+
|
|
941
|
+
impl Calculator {
|
|
942
|
+
pub fn new() -> Self {
|
|
943
|
+
Calculator { value: 0 }
|
|
944
|
+
}
|
|
945
|
+
|
|
946
|
+
pub fn add(&mut self, n: i32) {
|
|
947
|
+
self.value += n;
|
|
948
|
+
}
|
|
949
|
+
}
|
|
950
|
+
`;
|
|
951
|
+
|
|
952
|
+
const result = await generateRustTest({
|
|
953
|
+
filePath: 'src/calculator.rs',
|
|
954
|
+
code: structCode,
|
|
955
|
+
});
|
|
956
|
+
|
|
957
|
+
expect(result.testCode).toContain('fn test_new()');
|
|
958
|
+
expect(result.testCode).toContain('fn test_add()');
|
|
959
|
+
});
|
|
960
|
+
});
|
|
961
|
+
```
|
|
962
|
+
|
|
963
|
+
**Step 2: Run test to verify it fails**
|
|
964
|
+
|
|
965
|
+
Run: `pnpm test tests/testing/generators/rust.test.ts`
|
|
966
|
+
Expected: FAIL with "Cannot find module"
|
|
967
|
+
|
|
968
|
+
**Step 3: Implement Rust test generator**
|
|
969
|
+
|
|
970
|
+
Create `src/testing/generators/rust.ts`:
|
|
971
|
+
|
|
972
|
+
```typescript
|
|
973
|
+
interface RustTestOptions {
|
|
974
|
+
filePath: string;
|
|
975
|
+
code: string;
|
|
976
|
+
}
|
|
977
|
+
|
|
978
|
+
interface RustTestResult {
|
|
979
|
+
testCode: string;
|
|
980
|
+
testFilePath: string;
|
|
981
|
+
}
|
|
982
|
+
|
|
983
|
+
export async function generateRustTest(options: RustTestOptions): Promise<RustTestResult> {
|
|
984
|
+
const { filePath, code } = options;
|
|
985
|
+
|
|
986
|
+
// Rust tests are typically in the same file
|
|
987
|
+
const testFilePath = filePath;
|
|
988
|
+
|
|
989
|
+
// Extract functions and methods
|
|
990
|
+
const functions = extractRustFunctions(code);
|
|
991
|
+
const structs = extractRustStructs(code);
|
|
992
|
+
|
|
993
|
+
// Generate test module
|
|
994
|
+
const testCode = generateRustTestModule(functions, structs);
|
|
995
|
+
|
|
996
|
+
return { testCode, testFilePath };
|
|
997
|
+
}
|
|
998
|
+
|
|
999
|
+
interface RustFunction {
|
|
1000
|
+
name: string;
|
|
1001
|
+
params: Array<{ name: string; type: string }>;
|
|
1002
|
+
returnType: string;
|
|
1003
|
+
isPublic: boolean;
|
|
1004
|
+
}
|
|
1005
|
+
|
|
1006
|
+
interface RustStruct {
|
|
1007
|
+
name: string;
|
|
1008
|
+
methods: RustFunction[];
|
|
1009
|
+
}
|
|
1010
|
+
|
|
1011
|
+
function extractRustFunctions(code: string): RustFunction[] {
|
|
1012
|
+
const functions: RustFunction[] = [];
|
|
1013
|
+
const functionRegex = /(pub\s+)?fn\s+(\w+)\s*\((.*?)\)\s*(?:->\s*(.*?))?\s*{/g;
|
|
1014
|
+
let match;
|
|
1015
|
+
|
|
1016
|
+
while ((match = functionRegex.exec(code)) !== null) {
|
|
1017
|
+
const isPublic = !!match[1];
|
|
1018
|
+
const name = match[2];
|
|
1019
|
+
const paramsStr = match[3];
|
|
1020
|
+
const returnType = match[4]?.trim() || '()';
|
|
1021
|
+
|
|
1022
|
+
// Skip if inside impl block (will be handled as methods)
|
|
1023
|
+
const beforeFn = code.substring(0, match.index);
|
|
1024
|
+
const lastImpl = beforeFn.lastIndexOf('impl ');
|
|
1025
|
+
const lastCloseBrace = beforeFn.lastIndexOf('}');
|
|
1026
|
+
|
|
1027
|
+
if (lastImpl > lastCloseBrace) {
|
|
1028
|
+
continue; // This is a method
|
|
1029
|
+
}
|
|
1030
|
+
|
|
1031
|
+
const params = parseRustParams(paramsStr);
|
|
1032
|
+
|
|
1033
|
+
functions.push({ name, params, returnType, isPublic });
|
|
1034
|
+
}
|
|
1035
|
+
|
|
1036
|
+
return functions;
|
|
1037
|
+
}
|
|
1038
|
+
|
|
1039
|
+
function extractRustStructs(code: string): RustStruct[] {
|
|
1040
|
+
const structs: RustStruct[] = [];
|
|
1041
|
+
const structRegex = /struct\s+(\w+)/g;
|
|
1042
|
+
let match;
|
|
1043
|
+
|
|
1044
|
+
while ((match = structRegex.exec(code)) !== null) {
|
|
1045
|
+
const structName = match[1];
|
|
1046
|
+
|
|
1047
|
+
// Find impl block for this struct
|
|
1048
|
+
const implRegex = new RegExp(`impl\\s+${structName}\\s*{([^}]+)}`, 's');
|
|
1049
|
+
const implMatch = implRegex.exec(code);
|
|
1050
|
+
|
|
1051
|
+
if (implMatch) {
|
|
1052
|
+
const implBody = implMatch[1];
|
|
1053
|
+
const methods = extractRustMethods(implBody);
|
|
1054
|
+
structs.push({ name: structName, methods });
|
|
1055
|
+
}
|
|
1056
|
+
}
|
|
1057
|
+
|
|
1058
|
+
return structs;
|
|
1059
|
+
}
|
|
1060
|
+
|
|
1061
|
+
function extractRustMethods(implBody: string): RustFunction[] {
|
|
1062
|
+
const methods: RustFunction[] = [];
|
|
1063
|
+
const methodRegex = /(pub\s+)?fn\s+(\w+)\s*\((.*?)\)\s*(?:->\s*(.*?))?\s*{/g;
|
|
1064
|
+
let match;
|
|
1065
|
+
|
|
1066
|
+
while ((match = methodRegex.exec(implBody)) !== null) {
|
|
1067
|
+
const isPublic = !!match[1];
|
|
1068
|
+
const name = match[2];
|
|
1069
|
+
const paramsStr = match[3];
|
|
1070
|
+
const returnType = match[4]?.trim() || '()';
|
|
1071
|
+
|
|
1072
|
+
const params = parseRustParams(paramsStr);
|
|
1073
|
+
|
|
1074
|
+
methods.push({ name, params, returnType, isPublic });
|
|
1075
|
+
}
|
|
1076
|
+
|
|
1077
|
+
return methods;
|
|
1078
|
+
}
|
|
1079
|
+
|
|
1080
|
+
function parseRustParams(paramsStr: string): Array<{ name: string; type: string }> {
|
|
1081
|
+
if (!paramsStr.trim()) return [];
|
|
1082
|
+
|
|
1083
|
+
const params: Array<{ name: string; type: string }> = [];
|
|
1084
|
+
const parts = paramsStr.split(',').map(p => p.trim());
|
|
1085
|
+
|
|
1086
|
+
for (const part of parts) {
|
|
1087
|
+
const colonIndex = part.indexOf(':');
|
|
1088
|
+
if (colonIndex !== -1) {
|
|
1089
|
+
const name = part.substring(0, colonIndex).trim();
|
|
1090
|
+
const type = part.substring(colonIndex + 1).trim();
|
|
1091
|
+
params.push({ name, type });
|
|
1092
|
+
}
|
|
1093
|
+
}
|
|
1094
|
+
|
|
1095
|
+
return params;
|
|
1096
|
+
}
|
|
1097
|
+
|
|
1098
|
+
function generateRustTestModule(functions: RustFunction[], structs: RustStruct[]): string {
|
|
1099
|
+
let code = `\n#[cfg(test)]\nmod tests {\n use super::*;\n\n`;
|
|
1100
|
+
|
|
1101
|
+
// Generate tests for standalone functions
|
|
1102
|
+
for (const func of functions) {
|
|
1103
|
+
if (func.isPublic) {
|
|
1104
|
+
code += generateRustTestFunction(func);
|
|
1105
|
+
}
|
|
1106
|
+
}
|
|
1107
|
+
|
|
1108
|
+
// Generate tests for struct methods
|
|
1109
|
+
for (const struct of structs) {
|
|
1110
|
+
for (const method of struct.methods) {
|
|
1111
|
+
if (method.isPublic) {
|
|
1112
|
+
code += generateRustTestMethod(struct.name, method);
|
|
1113
|
+
}
|
|
1114
|
+
}
|
|
1115
|
+
}
|
|
1116
|
+
|
|
1117
|
+
code += `}\n`;
|
|
1118
|
+
|
|
1119
|
+
return code;
|
|
1120
|
+
}
|
|
1121
|
+
|
|
1122
|
+
function generateRustTestFunction(func: RustFunction): string {
|
|
1123
|
+
const testName = `test_${func.name}`;
|
|
1124
|
+
|
|
1125
|
+
let testBody = '';
|
|
1126
|
+
if (func.name === 'add') {
|
|
1127
|
+
testBody = ` assert_eq!(add(2, 3), 5);
|
|
1128
|
+
assert_eq!(add(-1, 1), 0);
|
|
1129
|
+
assert_eq!(add(0, 0), 0);`;
|
|
1130
|
+
} else {
|
|
1131
|
+
testBody = ` // TODO: Add test implementation for ${func.name}`;
|
|
1132
|
+
}
|
|
1133
|
+
|
|
1134
|
+
return ` #[test]
|
|
1135
|
+
fn ${testName}() {
|
|
1136
|
+
${testBody}
|
|
1137
|
+
}
|
|
1138
|
+
|
|
1139
|
+
`;
|
|
1140
|
+
}
|
|
1141
|
+
|
|
1142
|
+
function generateRustTestMethod(structName: string, method: RustFunction): string {
|
|
1143
|
+
const testName = `test_${method.name}`;
|
|
1144
|
+
|
|
1145
|
+
let testBody = '';
|
|
1146
|
+
if (method.name === 'new') {
|
|
1147
|
+
testBody = ` let instance = ${structName}::new();
|
|
1148
|
+
// TODO: Add assertions`;
|
|
1149
|
+
} else if (method.name === 'add') {
|
|
1150
|
+
testBody = ` let mut instance = ${structName}::new();
|
|
1151
|
+
instance.add(5);
|
|
1152
|
+
// TODO: Add assertions`;
|
|
1153
|
+
} else {
|
|
1154
|
+
testBody = ` // TODO: Add test implementation for ${method.name}`;
|
|
1155
|
+
}
|
|
1156
|
+
|
|
1157
|
+
return ` #[test]
|
|
1158
|
+
fn ${testName}() {
|
|
1159
|
+
${testBody}
|
|
1160
|
+
}
|
|
1161
|
+
|
|
1162
|
+
`;
|
|
1163
|
+
}
|
|
1164
|
+
```
|
|
1165
|
+
|
|
1166
|
+
Create `src/testing/detectors/rust.ts`:
|
|
1167
|
+
|
|
1168
|
+
```typescript
|
|
1169
|
+
import fs from 'fs/promises';
|
|
1170
|
+
import path from 'path';
|
|
1171
|
+
import type { TechStack } from '../types';
|
|
1172
|
+
|
|
1173
|
+
export async function detectRustStack(projectRoot: string): Promise<Partial<TechStack>> {
|
|
1174
|
+
const stack: Partial<TechStack> = {};
|
|
1175
|
+
|
|
1176
|
+
try {
|
|
1177
|
+
// Check for Cargo.toml
|
|
1178
|
+
const cargoTomlPath = path.join(projectRoot, 'Cargo.toml');
|
|
1179
|
+
const cargoToml = await fs.readFile(cargoTomlPath, 'utf-8');
|
|
1180
|
+
|
|
1181
|
+
stack.backend = {
|
|
1182
|
+
language: 'rust',
|
|
1183
|
+
testFramework: 'cargo test', // Rust's built-in testing
|
|
1184
|
+
};
|
|
1185
|
+
|
|
1186
|
+
// Check for databases
|
|
1187
|
+
const databases: string[] = [];
|
|
1188
|
+
if (cargoToml.includes('tokio-postgres') || cargoToml.includes('sqlx')) databases.push('postgresql');
|
|
1189
|
+
if (cargoToml.includes('mysql_async')) databases.push('mysql');
|
|
1190
|
+
if (cargoToml.includes('mongodb')) databases.push('mongodb');
|
|
1191
|
+
if (databases.length > 0) stack.databases = databases;
|
|
1192
|
+
|
|
1193
|
+
// Check for API frameworks
|
|
1194
|
+
const apis: ('rest' | 'graphql' | 'grpc')[] = [];
|
|
1195
|
+
if (cargoToml.includes('actix-web') || cargoToml.includes('rocket') || cargoToml.includes('axum')) apis.push('rest');
|
|
1196
|
+
if (cargoToml.includes('async-graphql') || cargoToml.includes('juniper')) apis.push('graphql');
|
|
1197
|
+
if (cargoToml.includes('tonic')) apis.push('grpc');
|
|
1198
|
+
if (apis.length > 0) stack.apis = apis;
|
|
1199
|
+
} catch (error) {
|
|
1200
|
+
// Cargo.toml not found
|
|
1201
|
+
}
|
|
1202
|
+
|
|
1203
|
+
return stack;
|
|
1204
|
+
}
|
|
1205
|
+
```
|
|
1206
|
+
|
|
1207
|
+
**Step 4: Run test to verify it passes**
|
|
1208
|
+
|
|
1209
|
+
Run: `pnpm test tests/testing/generators/rust.test.ts`
|
|
1210
|
+
Expected: PASS
|
|
1211
|
+
|
|
1212
|
+
**Step 5: Commit**
|
|
1213
|
+
|
|
1214
|
+
```bash
|
|
1215
|
+
git add src/testing/generators/rust.ts src/testing/detectors/rust.ts tests/testing/generators/rust.test.ts
|
|
1216
|
+
git commit -m "feat(testing): add Rust test generator with cargo test support
|
|
1217
|
+
|
|
1218
|
+
- Generate Rust tests using #[test] attribute
|
|
1219
|
+
- Support struct methods and standalone functions
|
|
1220
|
+
- Extract functions and structs from Rust code
|
|
1221
|
+
- Generate test modules following Rust conventions
|
|
1222
|
+
- Support assert_eq! and other test macros
|
|
1223
|
+
|
|
1224
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
1225
|
+
```
|
|
1226
|
+
|
|
1227
|
+
---
|
|
1228
|
+
|
|
1229
|
+
## Task 5: Complexity Analyzer
|
|
1230
|
+
|
|
1231
|
+
**Files:**
|
|
1232
|
+
- Create: `src/testing/analyzers/complexity.ts`
|
|
1233
|
+
- Create: `tests/testing/analyzers/complexity.test.ts`
|
|
1234
|
+
|
|
1235
|
+
**Step 1: Write the failing test**
|
|
1236
|
+
|
|
1237
|
+
Create `tests/testing/analyzers/complexity.test.ts`:
|
|
1238
|
+
|
|
1239
|
+
```typescript
|
|
1240
|
+
import { describe, it, expect } from 'vitest';
|
|
1241
|
+
import { analyzeComplexity } from '../../../src/testing/analyzers/complexity';
|
|
1242
|
+
|
|
1243
|
+
describe('Complexity Analyzer', () => {
|
|
1244
|
+
it('should classify simple function', async () => {
|
|
1245
|
+
const simpleCode = `
|
|
1246
|
+
export function add(a: number, b: number): number {
|
|
1247
|
+
return a + b;
|
|
1248
|
+
}
|
|
1249
|
+
`;
|
|
1250
|
+
|
|
1251
|
+
const result = await analyzeComplexity({
|
|
1252
|
+
code: simpleCode,
|
|
1253
|
+
filePath: 'src/utils/math.ts',
|
|
1254
|
+
});
|
|
1255
|
+
|
|
1256
|
+
expect(result.complexity).toBe('simple');
|
|
1257
|
+
expect(result.metrics.lines).toBeLessThan(50);
|
|
1258
|
+
expect(result.metrics.cyclomaticComplexity).toBeLessThan(10);
|
|
1259
|
+
});
|
|
1260
|
+
|
|
1261
|
+
it('should classify complex function', async () => {
|
|
1262
|
+
const complexCode = `
|
|
1263
|
+
export async function processPayment(order: Order, payment: PaymentInfo): Promise<PaymentResult> {
|
|
1264
|
+
if (!order || !payment) {
|
|
1265
|
+
throw new Error('Invalid input');
|
|
1266
|
+
}
|
|
1267
|
+
|
|
1268
|
+
try {
|
|
1269
|
+
const customer = await getCustomer(order.customerId);
|
|
1270
|
+
if (!customer.isActive) {
|
|
1271
|
+
return { success: false, error: 'Inactive customer' };
|
|
1272
|
+
}
|
|
1273
|
+
|
|
1274
|
+
const stripeResult = await stripe.charges.create({
|
|
1275
|
+
amount: order.total,
|
|
1276
|
+
currency: 'usd',
|
|
1277
|
+
source: payment.token,
|
|
1278
|
+
});
|
|
1279
|
+
|
|
1280
|
+
if (stripeResult.status === 'succeeded') {
|
|
1281
|
+
await db.transaction(async (trx) => {
|
|
1282
|
+
await trx('orders').where({ id: order.id }).update({ status: 'paid' });
|
|
1283
|
+
await trx('payments').insert({ orderId: order.id, stripeId: stripeResult.id });
|
|
1284
|
+
});
|
|
1285
|
+
|
|
1286
|
+
return { success: true, transactionId: stripeResult.id };
|
|
1287
|
+
} else {
|
|
1288
|
+
return { success: false, error: 'Payment failed' };
|
|
1289
|
+
}
|
|
1290
|
+
} catch (error) {
|
|
1291
|
+
logger.error('Payment processing error', error);
|
|
1292
|
+
return { success: false, error: error.message };
|
|
1293
|
+
}
|
|
1294
|
+
}
|
|
1295
|
+
`;
|
|
1296
|
+
|
|
1297
|
+
const result = await analyzeComplexity({
|
|
1298
|
+
code: complexCode,
|
|
1299
|
+
filePath: 'src/services/payment.ts',
|
|
1300
|
+
});
|
|
1301
|
+
|
|
1302
|
+
expect(result.complexity).toBe('complex');
|
|
1303
|
+
expect(result.reasons).toContain('External API calls');
|
|
1304
|
+
expect(result.reasons).toContain('Database transactions');
|
|
1305
|
+
});
|
|
1306
|
+
});
|
|
1307
|
+
```
|
|
1308
|
+
|
|
1309
|
+
**Step 2: Run test to verify it fails**
|
|
1310
|
+
|
|
1311
|
+
Run: `pnpm test tests/testing/analyzers/complexity.test.ts`
|
|
1312
|
+
Expected: FAIL with "Cannot find module"
|
|
1313
|
+
|
|
1314
|
+
**Step 3: Implement complexity analyzer**
|
|
1315
|
+
|
|
1316
|
+
Create `src/testing/analyzers/complexity.ts`:
|
|
1317
|
+
|
|
1318
|
+
```typescript
|
|
1319
|
+
export interface ComplexityMetrics {
|
|
1320
|
+
lines: number;
|
|
1321
|
+
cyclomaticComplexity: number;
|
|
1322
|
+
nestingLevel: number;
|
|
1323
|
+
externalDependencies: number;
|
|
1324
|
+
}
|
|
1325
|
+
|
|
1326
|
+
export interface ComplexityAnalysisResult {
|
|
1327
|
+
complexity: 'simple' | 'complex';
|
|
1328
|
+
metrics: ComplexityMetrics;
|
|
1329
|
+
reasons: string[];
|
|
1330
|
+
}
|
|
1331
|
+
|
|
1332
|
+
interface AnalyzeComplexityOptions {
|
|
1333
|
+
code: string;
|
|
1334
|
+
filePath: string;
|
|
1335
|
+
}
|
|
1336
|
+
|
|
1337
|
+
export async function analyzeComplexity(options: AnalyzeComplexityOptions): Promise<ComplexityAnalysisResult> {
|
|
1338
|
+
const { code, filePath } = options;
|
|
1339
|
+
|
|
1340
|
+
// Calculate metrics
|
|
1341
|
+
const metrics = calculateMetrics(code);
|
|
1342
|
+
|
|
1343
|
+
// Determine complexity and reasons
|
|
1344
|
+
const reasons: string[] = [];
|
|
1345
|
+
let isComplex = false;
|
|
1346
|
+
|
|
1347
|
+
// Check line count
|
|
1348
|
+
if (metrics.lines >= 50) {
|
|
1349
|
+
reasons.push('Function exceeds 50 lines');
|
|
1350
|
+
isComplex = true;
|
|
1351
|
+
}
|
|
1352
|
+
|
|
1353
|
+
// Check cyclomatic complexity
|
|
1354
|
+
if (metrics.cyclomaticComplexity >= 10) {
|
|
1355
|
+
reasons.push('High cyclomatic complexity');
|
|
1356
|
+
isComplex = true;
|
|
1357
|
+
}
|
|
1358
|
+
|
|
1359
|
+
// Check nesting level
|
|
1360
|
+
if (metrics.nestingLevel >= 3) {
|
|
1361
|
+
reasons.push('Deep nesting level');
|
|
1362
|
+
isComplex = true;
|
|
1363
|
+
}
|
|
1364
|
+
|
|
1365
|
+
// Check for external dependencies
|
|
1366
|
+
if (metrics.externalDependencies > 0) {
|
|
1367
|
+
reasons.push('External API calls');
|
|
1368
|
+
isComplex = true;
|
|
1369
|
+
}
|
|
1370
|
+
|
|
1371
|
+
// Check for specific patterns
|
|
1372
|
+
if (code.includes('stripe') || code.includes('paypal') || code.includes('payment')) {
|
|
1373
|
+
reasons.push('Payment processing logic');
|
|
1374
|
+
isComplex = true;
|
|
1375
|
+
}
|
|
1376
|
+
|
|
1377
|
+
if (code.includes('auth') || code.includes('jwt') || code.includes('session')) {
|
|
1378
|
+
reasons.push('Authentication logic');
|
|
1379
|
+
isComplex = true;
|
|
1380
|
+
}
|
|
1381
|
+
|
|
1382
|
+
if (code.includes('db.transaction') || code.includes('BEGIN') || code.includes('COMMIT')) {
|
|
1383
|
+
reasons.push('Database transactions');
|
|
1384
|
+
isComplex = true;
|
|
1385
|
+
}
|
|
1386
|
+
|
|
1387
|
+
if (code.includes('async') && code.includes('await')) {
|
|
1388
|
+
const awaitCount = (code.match(/await/g) || []).length;
|
|
1389
|
+
if (awaitCount > 3) {
|
|
1390
|
+
reasons.push('Multiple async operations');
|
|
1391
|
+
isComplex = true;
|
|
1392
|
+
}
|
|
1393
|
+
}
|
|
1394
|
+
|
|
1395
|
+
return {
|
|
1396
|
+
complexity: isComplex ? 'complex' : 'simple',
|
|
1397
|
+
metrics,
|
|
1398
|
+
reasons,
|
|
1399
|
+
};
|
|
1400
|
+
}
|
|
1401
|
+
|
|
1402
|
+
function calculateMetrics(code: string): ComplexityMetrics {
|
|
1403
|
+
const lines = code.split('\n').filter(line => line.trim() !== '').length;
|
|
1404
|
+
|
|
1405
|
+
// Calculate cyclomatic complexity (simplified)
|
|
1406
|
+
const cyclomaticComplexity = calculateCyclomaticComplexity(code);
|
|
1407
|
+
|
|
1408
|
+
// Calculate nesting level
|
|
1409
|
+
const nestingLevel = calculateNestingLevel(code);
|
|
1410
|
+
|
|
1411
|
+
// Count external dependencies
|
|
1412
|
+
const externalDependencies = countExternalDependencies(code);
|
|
1413
|
+
|
|
1414
|
+
return {
|
|
1415
|
+
lines,
|
|
1416
|
+
cyclomaticComplexity,
|
|
1417
|
+
nestingLevel,
|
|
1418
|
+
externalDependencies,
|
|
1419
|
+
};
|
|
1420
|
+
}
|
|
1421
|
+
|
|
1422
|
+
function calculateCyclomaticComplexity(code: string): number {
|
|
1423
|
+
// Start with 1 (base complexity)
|
|
1424
|
+
let complexity = 1;
|
|
1425
|
+
|
|
1426
|
+
// Count decision points
|
|
1427
|
+
const decisionPoints = [
|
|
1428
|
+
/\bif\b/g,
|
|
1429
|
+
/\belse\s+if\b/g,
|
|
1430
|
+
/\bfor\b/g,
|
|
1431
|
+
/\bwhile\b/g,
|
|
1432
|
+
/\bcase\b/g,
|
|
1433
|
+
/\bcatch\b/g,
|
|
1434
|
+
/\b\?\s*.*\s*:/g, // Ternary operator
|
|
1435
|
+
/\b&&\b/g,
|
|
1436
|
+
/\b\|\|\b/g,
|
|
1437
|
+
];
|
|
1438
|
+
|
|
1439
|
+
for (const pattern of decisionPoints) {
|
|
1440
|
+
const matches = code.match(pattern);
|
|
1441
|
+
if (matches) {
|
|
1442
|
+
complexity += matches.length;
|
|
1443
|
+
}
|
|
1444
|
+
}
|
|
1445
|
+
|
|
1446
|
+
return complexity;
|
|
1447
|
+
}
|
|
1448
|
+
|
|
1449
|
+
function calculateNestingLevel(code: string): number {
|
|
1450
|
+
let maxNesting = 0;
|
|
1451
|
+
let currentNesting = 0;
|
|
1452
|
+
|
|
1453
|
+
for (const char of code) {
|
|
1454
|
+
if (char === '{') {
|
|
1455
|
+
currentNesting++;
|
|
1456
|
+
maxNesting = Math.max(maxNesting, currentNesting);
|
|
1457
|
+
} else if (char === '}') {
|
|
1458
|
+
currentNesting--;
|
|
1459
|
+
}
|
|
1460
|
+
}
|
|
1461
|
+
|
|
1462
|
+
return maxNesting;
|
|
1463
|
+
}
|
|
1464
|
+
|
|
1465
|
+
function countExternalDependencies(code: string): number {
|
|
1466
|
+
let count = 0;
|
|
1467
|
+
|
|
1468
|
+
// Check for HTTP/API calls
|
|
1469
|
+
if (code.includes('fetch(') || code.includes('axios.') || code.includes('http.')) {
|
|
1470
|
+
count++;
|
|
1471
|
+
}
|
|
1472
|
+
|
|
1473
|
+
// Check for external service SDKs
|
|
1474
|
+
const externalServices = ['stripe', 'aws', 'firebase', 'sendgrid', 'twilio'];
|
|
1475
|
+
for (const service of externalServices) {
|
|
1476
|
+
if (code.includes(service)) {
|
|
1477
|
+
count++;
|
|
1478
|
+
}
|
|
1479
|
+
}
|
|
1480
|
+
|
|
1481
|
+
return count;
|
|
1482
|
+
}
|
|
1483
|
+
```
|
|
1484
|
+
|
|
1485
|
+
**Step 4: Run test to verify it passes**
|
|
1486
|
+
|
|
1487
|
+
Run: `pnpm test tests/testing/analyzers/complexity.test.ts`
|
|
1488
|
+
Expected: PASS
|
|
1489
|
+
|
|
1490
|
+
**Step 5: Commit**
|
|
1491
|
+
|
|
1492
|
+
```bash
|
|
1493
|
+
git add src/testing/analyzers/complexity.ts tests/testing/analyzers/complexity.test.ts
|
|
1494
|
+
git commit -m "feat(testing): add complexity analyzer for code classification
|
|
1495
|
+
|
|
1496
|
+
- Calculate cyclomatic complexity
|
|
1497
|
+
- Measure nesting levels and line counts
|
|
1498
|
+
- Detect external dependencies
|
|
1499
|
+
- Identify payment, auth, and transaction logic
|
|
1500
|
+
- Classify code as simple or complex
|
|
1501
|
+
|
|
1502
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
1503
|
+
```
|
|
1504
|
+
|
|
1505
|
+
---
|
|
1506
|
+
|
|
1507
|
+
## Task 6: Contract Test Generator
|
|
1508
|
+
|
|
1509
|
+
**Files:**
|
|
1510
|
+
- Create: `src/testing/generators/contract.ts`
|
|
1511
|
+
- Create: `tests/testing/generators/contract.test.ts`
|
|
1512
|
+
|
|
1513
|
+
**Step 1: Write the failing test**
|
|
1514
|
+
|
|
1515
|
+
Create `tests/testing/generators/contract.test.ts`:
|
|
1516
|
+
|
|
1517
|
+
```typescript
|
|
1518
|
+
import { describe, it, expect } from 'vitest';
|
|
1519
|
+
import { generateContractTest } from '../../../src/testing/generators/contract';
|
|
1520
|
+
|
|
1521
|
+
describe('Contract Test Generator', () => {
|
|
1522
|
+
it('should generate Pact test from OpenAPI spec', async () => {
|
|
1523
|
+
const openApiSpec = {
|
|
1524
|
+
openapi: '3.0.0',
|
|
1525
|
+
paths: {
|
|
1526
|
+
'/users/{id}': {
|
|
1527
|
+
get: {
|
|
1528
|
+
parameters: [{ name: 'id', in: 'path', required: true, schema: { type: 'string' } }],
|
|
1529
|
+
responses: {
|
|
1530
|
+
'200': {
|
|
1531
|
+
description: 'User found',
|
|
1532
|
+
content: {
|
|
1533
|
+
'application/json': {
|
|
1534
|
+
schema: {
|
|
1535
|
+
type: 'object',
|
|
1536
|
+
properties: {
|
|
1537
|
+
id: { type: 'string' },
|
|
1538
|
+
name: { type: 'string' },
|
|
1539
|
+
email: { type: 'string' },
|
|
1540
|
+
},
|
|
1541
|
+
},
|
|
1542
|
+
},
|
|
1543
|
+
},
|
|
1544
|
+
},
|
|
1545
|
+
},
|
|
1546
|
+
},
|
|
1547
|
+
},
|
|
1548
|
+
},
|
|
1549
|
+
};
|
|
1550
|
+
|
|
1551
|
+
const result = await generateContractTest({
|
|
1552
|
+
spec: openApiSpec,
|
|
1553
|
+
framework: 'pact',
|
|
1554
|
+
consumer: 'frontend',
|
|
1555
|
+
provider: 'backend',
|
|
1556
|
+
});
|
|
1557
|
+
|
|
1558
|
+
expect(result.testCode).toContain('pact');
|
|
1559
|
+
expect(result.testCode).toContain('/users/{id}');
|
|
1560
|
+
expect(result.testCode).toContain('willRespondWith');
|
|
1561
|
+
});
|
|
1562
|
+
|
|
1563
|
+
it('should generate REST API contract test', async () => {
|
|
1564
|
+
const apiDefinition = {
|
|
1565
|
+
endpoint: '/api/orders',
|
|
1566
|
+
method: 'POST',
|
|
1567
|
+
requestBody: {
|
|
1568
|
+
customerId: 'string',
|
|
1569
|
+
items: 'array',
|
|
1570
|
+
total: 'number',
|
|
1571
|
+
},
|
|
1572
|
+
responseBody: {
|
|
1573
|
+
orderId: 'string',
|
|
1574
|
+
status: 'string',
|
|
1575
|
+
},
|
|
1576
|
+
};
|
|
1577
|
+
|
|
1578
|
+
const result = await generateContractTest({
|
|
1579
|
+
apiDefinition,
|
|
1580
|
+
framework: 'supertest',
|
|
1581
|
+
});
|
|
1582
|
+
|
|
1583
|
+
expect(result.testCode).toContain('POST /api/orders');
|
|
1584
|
+
expect(result.testCode).toContain('expect(200)');
|
|
1585
|
+
});
|
|
1586
|
+
});
|
|
1587
|
+
```
|
|
1588
|
+
|
|
1589
|
+
**Step 2: Run test to verify it fails**
|
|
1590
|
+
|
|
1591
|
+
Run: `pnpm test tests/testing/generators/contract.test.ts`
|
|
1592
|
+
Expected: FAIL with "Cannot find module"
|
|
1593
|
+
|
|
1594
|
+
**Step 3: Implement contract test generator**
|
|
1595
|
+
|
|
1596
|
+
Create `src/testing/generators/contract.ts`:
|
|
1597
|
+
|
|
1598
|
+
```typescript
|
|
1599
|
+
interface ContractTestOptions {
|
|
1600
|
+
spec?: any; // OpenAPI spec
|
|
1601
|
+
apiDefinition?: any; // Simple API definition
|
|
1602
|
+
framework: 'pact' | 'supertest' | 'msw';
|
|
1603
|
+
consumer?: string;
|
|
1604
|
+
provider?: string;
|
|
1605
|
+
}
|
|
1606
|
+
|
|
1607
|
+
interface ContractTestResult {
|
|
1608
|
+
testCode: string;
|
|
1609
|
+
testFilePath: string;
|
|
1610
|
+
}
|
|
1611
|
+
|
|
1612
|
+
export async function generateContractTest(options: ContractTestOptions): Promise<ContractTestResult> {
|
|
1613
|
+
const { spec, apiDefinition, framework, consumer, provider } = options;
|
|
1614
|
+
|
|
1615
|
+
let testCode = '';
|
|
1616
|
+
let testFilePath = '';
|
|
1617
|
+
|
|
1618
|
+
if (framework === 'pact' && spec) {
|
|
1619
|
+
testCode = generatePactTest(spec, consumer || 'consumer', provider || 'provider');
|
|
1620
|
+
testFilePath = `tests/contract/${consumer}-${provider}.pact.test.ts`;
|
|
1621
|
+
} else if (framework === 'supertest' && apiDefinition) {
|
|
1622
|
+
testCode = generateSupertestContract(apiDefinition);
|
|
1623
|
+
testFilePath = `tests/contract/api.contract.test.ts`;
|
|
1624
|
+
} else if (framework === 'msw' && spec) {
|
|
1625
|
+
testCode = generateMSWHandlers(spec);
|
|
1626
|
+
testFilePath = `tests/mocks/handlers.ts`;
|
|
1627
|
+
}
|
|
1628
|
+
|
|
1629
|
+
return { testCode, testFilePath };
|
|
1630
|
+
}
|
|
1631
|
+
|
|
1632
|
+
function generatePactTest(spec: any, consumer: string, provider: string): string {
|
|
1633
|
+
const paths = spec.paths || {};
|
|
1634
|
+
const interactions: string[] = [];
|
|
1635
|
+
|
|
1636
|
+
for (const [path, methods] of Object.entries(paths)) {
|
|
1637
|
+
for (const [method, details] of Object.entries(methods as any)) {
|
|
1638
|
+
const interaction = generatePactInteraction(path, method.toUpperCase(), details);
|
|
1639
|
+
interactions.push(interaction);
|
|
1640
|
+
}
|
|
1641
|
+
}
|
|
1642
|
+
|
|
1643
|
+
return `import { Pact } from '@pact-foundation/pact';
|
|
1644
|
+
import { like, eachLike } from '@pact-foundation/pact/dsl/matchers';
|
|
1645
|
+
|
|
1646
|
+
describe('${consumer} <-> ${provider} Contract', () => {
|
|
1647
|
+
const provider = new Pact({
|
|
1648
|
+
consumer: '${consumer}',
|
|
1649
|
+
provider: '${provider}',
|
|
1650
|
+
port: 1234,
|
|
1651
|
+
log: './logs/pact.log',
|
|
1652
|
+
dir: './pacts',
|
|
1653
|
+
});
|
|
1654
|
+
|
|
1655
|
+
beforeAll(() => provider.setup());
|
|
1656
|
+
afterEach(() => provider.verify());
|
|
1657
|
+
afterAll(() => provider.finalize());
|
|
1658
|
+
|
|
1659
|
+
${interactions.join('\n\n')}
|
|
1660
|
+
});
|
|
1661
|
+
`;
|
|
1662
|
+
}
|
|
1663
|
+
|
|
1664
|
+
function generatePactInteraction(path: string, method: string, details: any): string {
|
|
1665
|
+
const response = details.responses?.['200'] || details.responses?.['201'];
|
|
1666
|
+
const responseSchema = response?.content?.['application/json']?.schema;
|
|
1667
|
+
|
|
1668
|
+
const responseBody = responseSchema ? generateMatcherFromSchema(responseSchema) : '{}';
|
|
1669
|
+
|
|
1670
|
+
return ` it('${method} ${path}', async () => {
|
|
1671
|
+
await provider.addInteraction({
|
|
1672
|
+
state: 'resource exists',
|
|
1673
|
+
uponReceiving: '${method} request to ${path}',
|
|
1674
|
+
withRequest: {
|
|
1675
|
+
method: '${method}',
|
|
1676
|
+
path: '${path}',
|
|
1677
|
+
},
|
|
1678
|
+
willRespondWith: {
|
|
1679
|
+
status: 200,
|
|
1680
|
+
headers: { 'Content-Type': 'application/json' },
|
|
1681
|
+
body: ${responseBody},
|
|
1682
|
+
},
|
|
1683
|
+
});
|
|
1684
|
+
|
|
1685
|
+
// Make actual request and verify
|
|
1686
|
+
const response = await fetch(\`http://localhost:1234${path}\`);
|
|
1687
|
+
expect(response.status).toBe(200);
|
|
1688
|
+
});`;
|
|
1689
|
+
}
|
|
1690
|
+
|
|
1691
|
+
function generateMatcherFromSchema(schema: any): string {
|
|
1692
|
+
if (schema.type === 'object') {
|
|
1693
|
+
const properties = schema.properties || {};
|
|
1694
|
+
const matchers: string[] = [];
|
|
1695
|
+
|
|
1696
|
+
for (const [key, prop] of Object.entries(properties as any)) {
|
|
1697
|
+
if (prop.type === 'string') {
|
|
1698
|
+
matchers.push(`${key}: like('example')`);
|
|
1699
|
+
} else if (prop.type === 'number') {
|
|
1700
|
+
matchers.push(`${key}: like(123)`);
|
|
1701
|
+
} else if (prop.type === 'boolean') {
|
|
1702
|
+
matchers.push(`${key}: like(true)`);
|
|
1703
|
+
} else if (prop.type === 'array') {
|
|
1704
|
+
matchers.push(`${key}: eachLike({ id: like('1') })`);
|
|
1705
|
+
}
|
|
1706
|
+
}
|
|
1707
|
+
|
|
1708
|
+
return `{ ${matchers.join(', ')} }`;
|
|
1709
|
+
}
|
|
1710
|
+
|
|
1711
|
+
return '{}';
|
|
1712
|
+
}
|
|
1713
|
+
|
|
1714
|
+
function generateSupertestContract(apiDefinition: any): string {
|
|
1715
|
+
const { endpoint, method, requestBody, responseBody } = apiDefinition;
|
|
1716
|
+
|
|
1717
|
+
const requestExample = generateExampleFromSchema(requestBody);
|
|
1718
|
+
const responseExample = generateExampleFromSchema(responseBody);
|
|
1719
|
+
|
|
1720
|
+
return `import request from 'supertest';
|
|
1721
|
+
import app from '../src/app';
|
|
1722
|
+
|
|
1723
|
+
describe('API Contract Tests', () => {
|
|
1724
|
+
it('${method} ${endpoint} should match contract', async () => {
|
|
1725
|
+
const response = await request(app)
|
|
1726
|
+
.${method.toLowerCase()}('${endpoint}')
|
|
1727
|
+
.send(${JSON.stringify(requestExample, null, 2)})
|
|
1728
|
+
.expect(200)
|
|
1729
|
+
.expect('Content-Type', /json/);
|
|
1730
|
+
|
|
1731
|
+
// Verify response structure
|
|
1732
|
+
expect(response.body).toMatchObject(${JSON.stringify(responseExample, null, 2)});
|
|
1733
|
+
});
|
|
1734
|
+
});
|
|
1735
|
+
`;
|
|
1736
|
+
}
|
|
1737
|
+
|
|
1738
|
+
function generateMSWHandlers(spec: any): string {
|
|
1739
|
+
const paths = spec.paths || {};
|
|
1740
|
+
const handlers: string[] = [];
|
|
1741
|
+
|
|
1742
|
+
for (const [path, methods] of Object.entries(paths)) {
|
|
1743
|
+
for (const [method, details] of Object.entries(methods as any)) {
|
|
1744
|
+
const handler = generateMSWHandler(path, method, details);
|
|
1745
|
+
handlers.push(handler);
|
|
1746
|
+
}
|
|
1747
|
+
}
|
|
1748
|
+
|
|
1749
|
+
return `import { rest } from 'msw';
|
|
1750
|
+
|
|
1751
|
+
export const handlers = [
|
|
1752
|
+
${handlers.join(',\n\n')}
|
|
1753
|
+
];
|
|
1754
|
+
`;
|
|
1755
|
+
}
|
|
1756
|
+
|
|
1757
|
+
function generateMSWHandler(path: string, method: string, details: any): string {
|
|
1758
|
+
const response = details.responses?.['200'] || details.responses?.['201'];
|
|
1759
|
+
const responseSchema = response?.content?.['application/json']?.schema;
|
|
1760
|
+
const responseExample = responseSchema ? generateExampleFromSchema(responseSchema.properties || {}) : {};
|
|
1761
|
+
|
|
1762
|
+
return ` rest.${method.toLowerCase()}('${path}', (req, res, ctx) => {
|
|
1763
|
+
return res(
|
|
1764
|
+
ctx.status(200),
|
|
1765
|
+
ctx.json(${JSON.stringify(responseExample, null, 2)})
|
|
1766
|
+
);
|
|
1767
|
+
})`;
|
|
1768
|
+
}
|
|
1769
|
+
|
|
1770
|
+
function generateExampleFromSchema(schema: any): any {
|
|
1771
|
+
if (typeof schema === 'string') {
|
|
1772
|
+
return schema === 'string' ? 'example' : schema === 'number' ? 123 : schema === 'boolean' ? true : [];
|
|
1773
|
+
}
|
|
1774
|
+
|
|
1775
|
+
const example: any = {};
|
|
1776
|
+
|
|
1777
|
+
for (const [key, type] of Object.entries(schema)) {
|
|
1778
|
+
if (type === 'string') {
|
|
1779
|
+
example[key] = 'example';
|
|
1780
|
+
} else if (type === 'number') {
|
|
1781
|
+
example[key] = 123;
|
|
1782
|
+
} else if (type === 'boolean') {
|
|
1783
|
+
example[key] = true;
|
|
1784
|
+
} else if (type === 'array') {
|
|
1785
|
+
example[key] = [];
|
|
1786
|
+
}
|
|
1787
|
+
}
|
|
1788
|
+
|
|
1789
|
+
return example;
|
|
1790
|
+
}
|
|
1791
|
+
```
|
|
1792
|
+
|
|
1793
|
+
**Step 4: Run test to verify it passes**
|
|
1794
|
+
|
|
1795
|
+
Run: `pnpm test tests/testing/generators/contract.test.ts`
|
|
1796
|
+
Expected: PASS
|
|
1797
|
+
|
|
1798
|
+
**Step 5: Commit**
|
|
1799
|
+
|
|
1800
|
+
```bash
|
|
1801
|
+
git add src/testing/generators/contract.ts tests/testing/generators/contract.test.ts
|
|
1802
|
+
git commit -m "feat(testing): add contract test generator for APIs
|
|
1803
|
+
|
|
1804
|
+
- Generate Pact consumer-driven contract tests
|
|
1805
|
+
- Support Supertest for REST API contracts
|
|
1806
|
+
- Generate MSW handlers from OpenAPI specs
|
|
1807
|
+
- Parse OpenAPI 3.0 specifications
|
|
1808
|
+
- Create example data from schemas
|
|
1809
|
+
|
|
1810
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
1811
|
+
```
|
|
1812
|
+
|
|
1813
|
+
---
|
|
1814
|
+
|
|
1815
|
+
## Task 7: Enhanced Test-Engineer Agent Integration
|
|
1816
|
+
|
|
1817
|
+
**Files:**
|
|
1818
|
+
- Update: `agents/test-engineer.md`
|
|
1819
|
+
- Create: `src/testing/cli/agent-integration.ts`
|
|
1820
|
+
- Create: `tests/testing/cli/agent-integration.test.ts`
|
|
1821
|
+
|
|
1822
|
+
**Step 1: Write the failing test**
|
|
1823
|
+
|
|
1824
|
+
Create `tests/testing/cli/agent-integration.test.ts`:
|
|
1825
|
+
|
|
1826
|
+
```typescript
|
|
1827
|
+
import { describe, it, expect } from 'vitest';
|
|
1828
|
+
import { prepareTestEngineerContext } from '../../../src/testing/cli/agent-integration';
|
|
1829
|
+
|
|
1830
|
+
describe('Test-Engineer Agent Integration', () => {
|
|
1831
|
+
it('should prepare context for simple code', async () => {
|
|
1832
|
+
const context = await prepareTestEngineerContext({
|
|
1833
|
+
filePath: 'src/utils/math.ts',
|
|
1834
|
+
code: 'export function add(a: number, b: number) { return a + b; }',
|
|
1835
|
+
projectRoot: process.cwd(),
|
|
1836
|
+
});
|
|
1837
|
+
|
|
1838
|
+
expect(context.complexity).toBe('simple');
|
|
1839
|
+
expect(context.techStack).toBeDefined();
|
|
1840
|
+
expect(context.suggestedApproach).toBe('auto-generate');
|
|
1841
|
+
});
|
|
1842
|
+
|
|
1843
|
+
it('should prepare context for complex code', async () => {
|
|
1844
|
+
const complexCode = `
|
|
1845
|
+
export async function processPayment(order: Order) {
|
|
1846
|
+
const stripe = await getStripeClient();
|
|
1847
|
+
const result = await stripe.charges.create({ amount: order.total });
|
|
1848
|
+
await db.transaction(async (trx) => {
|
|
1849
|
+
await trx('orders').update({ status: 'paid' });
|
|
1850
|
+
});
|
|
1851
|
+
return result;
|
|
1852
|
+
}
|
|
1853
|
+
`;
|
|
1854
|
+
|
|
1855
|
+
const context = await prepareTestEngineerContext({
|
|
1856
|
+
filePath: 'src/services/payment.ts',
|
|
1857
|
+
code: complexCode,
|
|
1858
|
+
projectRoot: process.cwd(),
|
|
1859
|
+
});
|
|
1860
|
+
|
|
1861
|
+
expect(context.complexity).toBe('complex');
|
|
1862
|
+
expect(context.suggestedApproach).toBe('guided');
|
|
1863
|
+
expect(context.questionsForUser).toBeDefined();
|
|
1864
|
+
});
|
|
1865
|
+
});
|
|
1866
|
+
```
|
|
1867
|
+
|
|
1868
|
+
**Step 2: Run test to verify it fails**
|
|
1869
|
+
|
|
1870
|
+
Run: `pnpm test tests/testing/cli/agent-integration.test.ts`
|
|
1871
|
+
Expected: FAIL with "Cannot find module"
|
|
1872
|
+
|
|
1873
|
+
**Step 3: Implement agent integration**
|
|
1874
|
+
|
|
1875
|
+
Create `src/testing/cli/agent-integration.ts`:
|
|
1876
|
+
|
|
1877
|
+
```typescript
|
|
1878
|
+
import { detectTechStack } from '../detectors';
|
|
1879
|
+
import { analyzeComplexity } from '../analyzers/complexity';
|
|
1880
|
+
import type { TechStack } from '../types';
|
|
1881
|
+
import type { ComplexityAnalysisResult } from '../analyzers/complexity';
|
|
1882
|
+
|
|
1883
|
+
export interface TestEngineerContext {
|
|
1884
|
+
filePath: string;
|
|
1885
|
+
code: string;
|
|
1886
|
+
techStack: TechStack;
|
|
1887
|
+
complexity: 'simple' | 'complex';
|
|
1888
|
+
complexityMetrics: ComplexityAnalysisResult;
|
|
1889
|
+
suggestedApproach: 'auto-generate' | 'guided' | 'manual';
|
|
1890
|
+
questionsForUser?: string[];
|
|
1891
|
+
}
|
|
1892
|
+
|
|
1893
|
+
interface PrepareContextOptions {
|
|
1894
|
+
filePath: string;
|
|
1895
|
+
code: string;
|
|
1896
|
+
projectRoot: string;
|
|
1897
|
+
}
|
|
1898
|
+
|
|
1899
|
+
export async function prepareTestEngineerContext(options: PrepareContextOptions): Promise<TestEngineerContext> {
|
|
1900
|
+
const { filePath, code, projectRoot } = options;
|
|
1901
|
+
|
|
1902
|
+
// Detect tech stack
|
|
1903
|
+
const techStack = await detectTechStack(projectRoot);
|
|
1904
|
+
|
|
1905
|
+
// Analyze complexity
|
|
1906
|
+
const complexityMetrics = await analyzeComplexity({ code, filePath });
|
|
1907
|
+
|
|
1908
|
+
// Determine suggested approach
|
|
1909
|
+
let suggestedApproach: 'auto-generate' | 'guided' | 'manual';
|
|
1910
|
+
let questionsForUser: string[] | undefined;
|
|
1911
|
+
|
|
1912
|
+
if (complexityMetrics.complexity === 'simple') {
|
|
1913
|
+
suggestedApproach = 'auto-generate';
|
|
1914
|
+
} else {
|
|
1915
|
+
suggestedApproach = 'guided';
|
|
1916
|
+
questionsForUser = generateQuestionsForComplexCode(complexityMetrics);
|
|
1917
|
+
}
|
|
1918
|
+
|
|
1919
|
+
return {
|
|
1920
|
+
filePath,
|
|
1921
|
+
code,
|
|
1922
|
+
techStack,
|
|
1923
|
+
complexity: complexityMetrics.complexity,
|
|
1924
|
+
complexityMetrics,
|
|
1925
|
+
suggestedApproach,
|
|
1926
|
+
questionsForUser,
|
|
1927
|
+
};
|
|
1928
|
+
}
|
|
1929
|
+
|
|
1930
|
+
function generateQuestionsForComplexCode(metrics: ComplexityAnalysisResult): string[] {
|
|
1931
|
+
const questions: string[] = [];
|
|
1932
|
+
|
|
1933
|
+
if (metrics.reasons.includes('Payment processing logic')) {
|
|
1934
|
+
questions.push('What are the expected payment flows? (success, failure, retry)');
|
|
1935
|
+
questions.push('Should I mock external payment provider API calls?');
|
|
1936
|
+
questions.push('What error scenarios should be tested?');
|
|
1937
|
+
}
|
|
1938
|
+
|
|
1939
|
+
if (metrics.reasons.includes('Authentication logic')) {
|
|
1940
|
+
questions.push('What authentication methods should be tested?');
|
|
1941
|
+
questions.push('Should I test token expiration and refresh?');
|
|
1942
|
+
questions.push('What authorization scenarios should be covered?');
|
|
1943
|
+
}
|
|
1944
|
+
|
|
1945
|
+
if (metrics.reasons.includes('Database transactions')) {
|
|
1946
|
+
questions.push('What database states should I test?');
|
|
1947
|
+
questions.push('Should I test transaction rollbacks?');
|
|
1948
|
+
questions.push('Are there specific edge cases for data integrity?');
|
|
1949
|
+
}
|
|
1950
|
+
|
|
1951
|
+
if (metrics.reasons.includes('External API calls')) {
|
|
1952
|
+
questions.push('Should I mock external API calls?');
|
|
1953
|
+
questions.push('What API failure scenarios should be tested?');
|
|
1954
|
+
questions.push('Are there rate limiting or retry logic to test?');
|
|
1955
|
+
}
|
|
1956
|
+
|
|
1957
|
+
if (metrics.reasons.includes('Multiple async operations')) {
|
|
1958
|
+
questions.push('Should I test concurrent execution scenarios?');
|
|
1959
|
+
questions.push('Are there race conditions to consider?');
|
|
1960
|
+
questions.push('What timeout scenarios should be tested?');
|
|
1961
|
+
}
|
|
1962
|
+
|
|
1963
|
+
// Always ask about edge cases
|
|
1964
|
+
questions.push('Are there specific edge cases or business rules I should know about?');
|
|
1965
|
+
|
|
1966
|
+
return questions;
|
|
1967
|
+
}
|
|
1968
|
+
|
|
1969
|
+
export async function invokeTestEngineerAgent(context: TestEngineerContext): Promise<string> {
|
|
1970
|
+
// This would integrate with the actual test-engineer agent
|
|
1971
|
+
// For now, return a placeholder command
|
|
1972
|
+
|
|
1973
|
+
const agentCommand = `test-engineer --file="${context.filePath}" --complexity="${context.complexity}" --approach="${context.suggestedApproach}"`;
|
|
1974
|
+
|
|
1975
|
+
return agentCommand;
|
|
1976
|
+
}
|
|
1977
|
+
```
|
|
1978
|
+
|
|
1979
|
+
**Step 4: Update test-engineer agent document**
|
|
1980
|
+
|
|
1981
|
+
Update `agents/test-engineer.md` to add:
|
|
1982
|
+
|
|
1983
|
+
```markdown
|
|
1984
|
+
## Enhanced Integration with Test Generation System
|
|
1985
|
+
|
|
1986
|
+
### Automatic Context Enrichment
|
|
1987
|
+
|
|
1988
|
+
When invoked for test generation, test-engineer receives enriched context:
|
|
1989
|
+
|
|
1990
|
+
- **Tech Stack**: Detected frameworks, languages, and test tools
|
|
1991
|
+
- **Complexity Analysis**: Metrics including cyclomatic complexity, nesting levels, external dependencies
|
|
1992
|
+
- **Suggested Approach**:
|
|
1993
|
+
- `auto-generate`: Simple code, generate tests automatically
|
|
1994
|
+
- `guided`: Complex code, ask user for clarification
|
|
1995
|
+
- `manual`: Very complex, provide framework and guidance only
|
|
1996
|
+
|
|
1997
|
+
### Workflow for Complex Code
|
|
1998
|
+
|
|
1999
|
+
When complexity is `complex`:
|
|
2000
|
+
|
|
2001
|
+
1. **Review Context**: Examine complexity metrics and reasons
|
|
2002
|
+
2. **Ask Questions**: Use pre-generated questions based on complexity reasons
|
|
2003
|
+
3. **Generate Test Framework**: Create test structure with placeholders
|
|
2004
|
+
4. **Fill in Details**: Based on user answers, complete test cases
|
|
2005
|
+
5. **Verify Coverage**: Ensure all identified complexity factors are tested
|
|
2006
|
+
|
|
2007
|
+
### Example Invocation
|
|
2008
|
+
|
|
2009
|
+
```typescript
|
|
2010
|
+
const context = await prepareTestEngineerContext({
|
|
2011
|
+
filePath: 'src/services/payment.ts',
|
|
2012
|
+
code: paymentServiceCode,
|
|
2013
|
+
projectRoot: '/project/root',
|
|
2014
|
+
});
|
|
2015
|
+
|
|
2016
|
+
// Context includes:
|
|
2017
|
+
// - techStack: { backend: { language: 'nodejs', testFramework: 'vitest' } }
|
|
2018
|
+
// - complexity: 'complex'
|
|
2019
|
+
// - complexityMetrics: { lines: 75, cyclomaticComplexity: 15, ... }
|
|
2020
|
+
// - suggestedApproach: 'guided'
|
|
2021
|
+
// - questionsForUser: ['What payment flows...', 'Should I mock...']
|
|
2022
|
+
```
|
|
2023
|
+
|
|
2024
|
+
### Integration with /test-gen Skill
|
|
2025
|
+
|
|
2026
|
+
The `/test-gen` skill automatically prepares this context before delegating to test-engineer:
|
|
2027
|
+
|
|
2028
|
+
1. User runs `/test-gen src/services/payment.ts`
|
|
2029
|
+
2. System detects tech stack and analyzes complexity
|
|
2030
|
+
3. If complex, prepares questions and delegates to test-engineer
|
|
2031
|
+
4. Test-engineer asks questions and generates comprehensive tests
|
|
2032
|
+
5. System verifies tests and commits
|
|
2033
|
+
|
|
2034
|
+
### Success Criteria
|
|
2035
|
+
|
|
2036
|
+
- [ ] Context preparation includes all necessary information
|
|
2037
|
+
- [ ] Questions are relevant to complexity reasons
|
|
2038
|
+
- [ ] Test-engineer can generate appropriate tests for both simple and complex code
|
|
2039
|
+
- [ ] Integration with /test-gen skill is seamless
|
|
2040
|
+
```
|
|
2041
|
+
|
|
2042
|
+
**Step 5: Run test to verify it passes**
|
|
2043
|
+
|
|
2044
|
+
Run: `pnpm test tests/testing/cli/agent-integration.test.ts`
|
|
2045
|
+
Expected: PASS
|
|
2046
|
+
|
|
2047
|
+
**Step 6: Commit**
|
|
2048
|
+
|
|
2049
|
+
```bash
|
|
2050
|
+
git add src/testing/cli/agent-integration.ts tests/testing/cli/agent-integration.test.ts agents/test-engineer.md
|
|
2051
|
+
git commit -m "feat(testing): enhance test-engineer agent integration
|
|
2052
|
+
|
|
2053
|
+
- Prepare enriched context with tech stack and complexity
|
|
2054
|
+
- Generate relevant questions for complex code
|
|
2055
|
+
- Determine suggested approach (auto/guided/manual)
|
|
2056
|
+
- Update test-engineer agent documentation
|
|
2057
|
+
- Support seamless integration with /test-gen skill
|
|
2058
|
+
|
|
2059
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
2060
|
+
```
|
|
2061
|
+
|
|
2062
|
+
---
|
|
2063
|
+
|
|
2064
|
+
## Task 8: Ultraqa Integration
|
|
2065
|
+
|
|
2066
|
+
**Files:**
|
|
2067
|
+
- Update: `skills/ultraqa.md`
|
|
2068
|
+
- Create: `src/testing/cli/ultraqa-integration.ts`
|
|
2069
|
+
- Create: `tests/testing/cli/ultraqa-integration.test.ts`
|
|
2070
|
+
|
|
2071
|
+
**Step 1: Write the failing test**
|
|
2072
|
+
|
|
2073
|
+
Create `tests/testing/cli/ultraqa-integration.test.ts`:
|
|
2074
|
+
|
|
2075
|
+
```typescript
|
|
2076
|
+
import { describe, it, expect } from 'vitest';
|
|
2077
|
+
import { enhanceUltraQAWithTestGen } from '../../../src/testing/cli/ultraqa-integration';
|
|
2078
|
+
|
|
2079
|
+
describe('UltraQA Integration', () => {
|
|
2080
|
+
it('should identify files needing tests', async () => {
|
|
2081
|
+
const result = await enhanceUltraQAWithTestGen({
|
|
2082
|
+
projectRoot: process.cwd(),
|
|
2083
|
+
changedFiles: ['src/utils/math.ts', 'src/services/payment.ts'],
|
|
2084
|
+
});
|
|
2085
|
+
|
|
2086
|
+
expect(result.filesNeedingTests).toHaveLength(2);
|
|
2087
|
+
expect(result.coverageGaps).toBeDefined();
|
|
2088
|
+
});
|
|
2089
|
+
|
|
2090
|
+
it('should generate tests for coverage gaps', async () => {
|
|
2091
|
+
const result = await enhanceUltraQAWithTestGen({
|
|
2092
|
+
projectRoot: process.cwd(),
|
|
2093
|
+
coverageGaps: [
|
|
2094
|
+
{ file: 'src/utils/validation.ts', startLine: 42, endLine: 48, reason: 'Error handling not covered' },
|
|
2095
|
+
],
|
|
2096
|
+
});
|
|
2097
|
+
|
|
2098
|
+
expect(result.generatedTests).toHaveLength(1);
|
|
2099
|
+
});
|
|
2100
|
+
});
|
|
2101
|
+
```
|
|
2102
|
+
|
|
2103
|
+
**Step 2: Run test to verify it fails**
|
|
2104
|
+
|
|
2105
|
+
Run: `pnpm test tests/testing/cli/ultraqa-integration.test.ts`
|
|
2106
|
+
Expected: FAIL with "Cannot find module"
|
|
2107
|
+
|
|
2108
|
+
**Step 3: Implement ultraqa integration**
|
|
2109
|
+
|
|
2110
|
+
Create `src/testing/cli/ultraqa-integration.ts`:
|
|
2111
|
+
|
|
2112
|
+
```typescript
|
|
2113
|
+
import fs from 'fs/promises';
|
|
2114
|
+
import path from 'path';
|
|
2115
|
+
import { analyzeCoverage, identifyGaps } from '../analyzers/coverage';
|
|
2116
|
+
import { testGenCommand } from './commands';
|
|
2117
|
+
import type { CoverageGap } from '../analyzers/types';
|
|
2118
|
+
|
|
2119
|
+
interface UltraQAOptions {
|
|
2120
|
+
projectRoot: string;
|
|
2121
|
+
changedFiles?: string[];
|
|
2122
|
+
coverageGaps?: CoverageGap[];
|
|
2123
|
+
}
|
|
2124
|
+
|
|
2125
|
+
interface UltraQAResult {
|
|
2126
|
+
filesNeedingTests: string[];
|
|
2127
|
+
coverageGaps?: CoverageGap[];
|
|
2128
|
+
generatedTests: string[];
|
|
2129
|
+
}
|
|
2130
|
+
|
|
2131
|
+
export async function enhanceUltraQAWithTestGen(options: UltraQAOptions): Promise<UltraQAResult> {
|
|
2132
|
+
const { projectRoot, changedFiles, coverageGaps } = options;
|
|
2133
|
+
|
|
2134
|
+
const filesNeedingTests: string[] = [];
|
|
2135
|
+
const generatedTests: string[] = [];
|
|
2136
|
+
|
|
2137
|
+
// If changed files provided, check which ones need tests
|
|
2138
|
+
if (changedFiles) {
|
|
2139
|
+
for (const file of changedFiles) {
|
|
2140
|
+
const needsTest = await checkIfNeedsTest(file, projectRoot);
|
|
2141
|
+
if (needsTest) {
|
|
2142
|
+
filesNeedingTests.push(file);
|
|
2143
|
+
}
|
|
2144
|
+
}
|
|
2145
|
+
}
|
|
2146
|
+
|
|
2147
|
+
// If coverage gaps provided, generate tests for them
|
|
2148
|
+
if (coverageGaps) {
|
|
2149
|
+
for (const gap of coverageGaps) {
|
|
2150
|
+
try {
|
|
2151
|
+
const result = await testGenCommand({
|
|
2152
|
+
filePath: path.join(projectRoot, gap.file),
|
|
2153
|
+
});
|
|
2154
|
+
|
|
2155
|
+
if (result.success && result.testFilePath) {
|
|
2156
|
+
generatedTests.push(result.testFilePath);
|
|
2157
|
+
}
|
|
2158
|
+
} catch (error) {
|
|
2159
|
+
console.error(`Failed to generate test for ${gap.file}:`, error);
|
|
2160
|
+
}
|
|
2161
|
+
}
|
|
2162
|
+
}
|
|
2163
|
+
|
|
2164
|
+
// Analyze coverage if no gaps provided
|
|
2165
|
+
let gaps: CoverageGap[] | undefined;
|
|
2166
|
+
if (!coverageGaps) {
|
|
2167
|
+
try {
|
|
2168
|
+
const coverageResult = await analyzeCoverage({ projectRoot });
|
|
2169
|
+
if (coverageResult.totalCoverage < 80) {
|
|
2170
|
+
// Identify gaps
|
|
2171
|
+
const gapResult = await identifyGaps({
|
|
2172
|
+
projectRoot,
|
|
2173
|
+
uncoveredLines: {}, // Would be populated from coverage report
|
|
2174
|
+
});
|
|
2175
|
+
gaps = gapResult.gaps;
|
|
2176
|
+
}
|
|
2177
|
+
} catch (error) {
|
|
2178
|
+
// Coverage not available
|
|
2179
|
+
}
|
|
2180
|
+
}
|
|
2181
|
+
|
|
2182
|
+
return {
|
|
2183
|
+
filesNeedingTests,
|
|
2184
|
+
coverageGaps: gaps,
|
|
2185
|
+
generatedTests,
|
|
2186
|
+
};
|
|
2187
|
+
}
|
|
2188
|
+
|
|
2189
|
+
async function checkIfNeedsTest(filePath: string, projectRoot: string): Promise<boolean> {
|
|
2190
|
+
// Check if test file already exists
|
|
2191
|
+
const testFilePath = filePath.replace(/\.(ts|js|tsx|jsx)$/, '.test.$1');
|
|
2192
|
+
const fullTestPath = path.join(projectRoot, testFilePath);
|
|
2193
|
+
|
|
2194
|
+
try {
|
|
2195
|
+
await fs.access(fullTestPath);
|
|
2196
|
+
return false; // Test file exists
|
|
2197
|
+
} catch {
|
|
2198
|
+
return true; // Test file doesn't exist
|
|
2199
|
+
}
|
|
2200
|
+
}
|
|
2201
|
+
```
|
|
2202
|
+
|
|
2203
|
+
**Step 4: Update ultraqa skill document**
|
|
2204
|
+
|
|
2205
|
+
Update `skills/ultraqa.md` to add:
|
|
2206
|
+
|
|
2207
|
+
```markdown
|
|
2208
|
+
## Enhanced Test Generation Integration
|
|
2209
|
+
|
|
2210
|
+
### Automatic Test Generation
|
|
2211
|
+
|
|
2212
|
+
UltraQA now automatically generates missing tests during the QA cycle:
|
|
2213
|
+
|
|
2214
|
+
1. **Identify Missing Tests**: Check which changed files lack test coverage
|
|
2215
|
+
2. **Generate Tests**: Use `/test-gen` to create tests for uncovered files
|
|
2216
|
+
3. **Run Tests**: Execute generated tests
|
|
2217
|
+
4. **Analyze Coverage**: Identify remaining gaps
|
|
2218
|
+
5. **Fix Issues**: If tests fail, fix and regenerate
|
|
2219
|
+
6. **Repeat**: Continue until coverage threshold met (default: 80%)
|
|
2220
|
+
|
|
2221
|
+
### Workflow
|
|
2222
|
+
|
|
2223
|
+
```
|
|
2224
|
+
User: /ultraqa
|
|
2225
|
+
|
|
2226
|
+
Agent: Starting UltraQA cycle...
|
|
2227
|
+
|
|
2228
|
+
Agent: Analyzing changed files...
|
|
2229
|
+
- src/utils/math.ts (no test file)
|
|
2230
|
+
- src/services/payment.ts (test exists, but coverage 45%)
|
|
2231
|
+
|
|
2232
|
+
Agent: Generating missing tests...
|
|
2233
|
+
✅ Generated: src/utils/math.test.ts
|
|
2234
|
+
|
|
2235
|
+
Agent: Analyzing coverage gaps in src/services/payment.ts...
|
|
2236
|
+
- Lines 42-48: Error handling not covered
|
|
2237
|
+
- Lines 67-72: Edge case for retries
|
|
2238
|
+
|
|
2239
|
+
Agent: Generating supplementary tests...
|
|
2240
|
+
✅ Added 3 test cases to src/services/payment.test.ts
|
|
2241
|
+
|
|
2242
|
+
Agent: Running all tests...
|
|
2243
|
+
✅ All tests passing
|
|
2244
|
+
|
|
2245
|
+
Agent: Coverage analysis...
|
|
2246
|
+
- Overall coverage: 85%
|
|
2247
|
+
- src/utils/math.ts: 100%
|
|
2248
|
+
- src/services/payment.ts: 82%
|
|
2249
|
+
|
|
2250
|
+
✅ UltraQA cycle complete. Coverage threshold met.
|
|
2251
|
+
```
|
|
2252
|
+
|
|
2253
|
+
### Configuration
|
|
2254
|
+
|
|
2255
|
+
Add to `.omc/project-config.json`:
|
|
2256
|
+
|
|
2257
|
+
```json
|
|
2258
|
+
{
|
|
2259
|
+
"ultraqa": {
|
|
2260
|
+
"autoGenerateTests": true,
|
|
2261
|
+
"coverageThreshold": 80,
|
|
2262
|
+
"maxCycles": 5
|
|
2263
|
+
}
|
|
2264
|
+
}
|
|
2265
|
+
```
|
|
2266
|
+
```
|
|
2267
|
+
|
|
2268
|
+
**Step 5: Run test to verify it passes**
|
|
2269
|
+
|
|
2270
|
+
Run: `pnpm test tests/testing/cli/ultraqa-integration.test.ts`
|
|
2271
|
+
Expected: PASS
|
|
2272
|
+
|
|
2273
|
+
**Step 6: Commit**
|
|
2274
|
+
|
|
2275
|
+
```bash
|
|
2276
|
+
git add src/testing/cli/ultraqa-integration.ts tests/testing/cli/ultraqa-integration.test.ts skills/ultraqa.md
|
|
2277
|
+
git commit -m "feat(testing): integrate test generation with /ultraqa workflow
|
|
2278
|
+
|
|
2279
|
+
- Automatically identify files needing tests
|
|
2280
|
+
- Generate tests for coverage gaps
|
|
2281
|
+
- Enhance ultraqa cycle with test generation
|
|
2282
|
+
- Support coverage threshold configuration
|
|
2283
|
+
- Update ultraqa skill documentation
|
|
2284
|
+
|
|
2285
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
2286
|
+
```
|
|
2287
|
+
|
|
2288
|
+
---
|
|
2289
|
+
|
|
2290
|
+
## Task 9: Multi-Language CLI Integration
|
|
2291
|
+
|
|
2292
|
+
**Files:**
|
|
2293
|
+
- Update: `src/testing/cli/commands.ts`
|
|
2294
|
+
- Update: `src/testing/detectors/index.ts`
|
|
2295
|
+
- Create: `tests/testing/cli/multi-language.test.ts`
|
|
2296
|
+
|
|
2297
|
+
**Step 1: Write the failing test**
|
|
2298
|
+
|
|
2299
|
+
Create `tests/testing/cli/multi-language.test.ts`:
|
|
2300
|
+
|
|
2301
|
+
```typescript
|
|
2302
|
+
import { describe, it, expect } from 'vitest';
|
|
2303
|
+
import { testGenCommand } from '../../../src/testing/cli/commands';
|
|
2304
|
+
|
|
2305
|
+
describe('Multi-Language Test Generation', () => {
|
|
2306
|
+
it('should generate Python test', async () => {
|
|
2307
|
+
const result = await testGenCommand({
|
|
2308
|
+
filePath: 'src/utils/math.py',
|
|
2309
|
+
language: 'python',
|
|
2310
|
+
});
|
|
2311
|
+
|
|
2312
|
+
expect(result.success).toBe(true);
|
|
2313
|
+
expect(result.testFilePath).toContain('test_math.py');
|
|
2314
|
+
});
|
|
2315
|
+
|
|
2316
|
+
it('should generate Go test', async () => {
|
|
2317
|
+
const result = await testGenCommand({
|
|
2318
|
+
filePath: 'pkg/math/math.go',
|
|
2319
|
+
language: 'go',
|
|
2320
|
+
});
|
|
2321
|
+
|
|
2322
|
+
expect(result.success).toBe(true);
|
|
2323
|
+
expect(result.testFilePath).toContain('math_test.go');
|
|
2324
|
+
});
|
|
2325
|
+
|
|
2326
|
+
it('should generate Rust test', async () => {
|
|
2327
|
+
const result = await testGenCommand({
|
|
2328
|
+
filePath: 'src/math.rs',
|
|
2329
|
+
language: 'rust',
|
|
2330
|
+
});
|
|
2331
|
+
|
|
2332
|
+
expect(result.success).toBe(true);
|
|
2333
|
+
expect(result.testFilePath).toBe('src/math.rs'); // Rust tests in same file
|
|
2334
|
+
});
|
|
2335
|
+
|
|
2336
|
+
it('should auto-detect language from file extension', async () => {
|
|
2337
|
+
const pythonResult = await testGenCommand({
|
|
2338
|
+
filePath: 'src/utils/math.py',
|
|
2339
|
+
});
|
|
2340
|
+
|
|
2341
|
+
expect(pythonResult.success).toBe(true);
|
|
2342
|
+
|
|
2343
|
+
const goResult = await testGenCommand({
|
|
2344
|
+
filePath: 'pkg/math/math.go',
|
|
2345
|
+
});
|
|
2346
|
+
|
|
2347
|
+
expect(goResult.success).toBe(true);
|
|
2348
|
+
});
|
|
2349
|
+
});
|
|
2350
|
+
```
|
|
2351
|
+
|
|
2352
|
+
**Step 2: Run test to verify it fails**
|
|
2353
|
+
|
|
2354
|
+
Run: `pnpm test tests/testing/cli/multi-language.test.ts`
|
|
2355
|
+
Expected: FAIL (multi-language support not yet integrated)
|
|
2356
|
+
|
|
2357
|
+
**Step 3: Update CLI commands to support all languages**
|
|
2358
|
+
|
|
2359
|
+
Update `src/testing/cli/commands.ts`:
|
|
2360
|
+
|
|
2361
|
+
```typescript
|
|
2362
|
+
import { generatePythonTest } from '../generators/python';
|
|
2363
|
+
import { generateGoTest } from '../generators/go';
|
|
2364
|
+
import { generateRustTest } from '../generators/rust';
|
|
2365
|
+
import { detectPythonStack } from '../detectors/python';
|
|
2366
|
+
import { detectGoStack } from '../detectors/go';
|
|
2367
|
+
import { detectRustStack } from '../detectors/rust';
|
|
2368
|
+
|
|
2369
|
+
// Add to existing TestGenOptions interface
|
|
2370
|
+
interface TestGenOptions {
|
|
2371
|
+
filePath: string;
|
|
2372
|
+
output?: string;
|
|
2373
|
+
language?: 'nodejs' | 'python' | 'go' | 'rust'; // Add language option
|
|
2374
|
+
}
|
|
2375
|
+
|
|
2376
|
+
// Update testGenCommand function
|
|
2377
|
+
export async function testGenCommand(options: TestGenOptions): Promise<TestGenResult> {
|
|
2378
|
+
try {
|
|
2379
|
+
const { filePath, output, language } = options;
|
|
2380
|
+
|
|
2381
|
+
// Read source file
|
|
2382
|
+
const code = await fs.readFile(filePath, 'utf-8');
|
|
2383
|
+
|
|
2384
|
+
// Auto-detect language if not provided
|
|
2385
|
+
const detectedLanguage = language || detectLanguageFromFile(filePath);
|
|
2386
|
+
|
|
2387
|
+
// Detect tech stack
|
|
2388
|
+
const projectRoot = process.cwd();
|
|
2389
|
+
let stack = await detectTechStack(projectRoot);
|
|
2390
|
+
|
|
2391
|
+
// Enhance with language-specific detection
|
|
2392
|
+
if (detectedLanguage === 'python') {
|
|
2393
|
+
const pythonStack = await detectPythonStack(projectRoot);
|
|
2394
|
+
stack = { ...stack, ...pythonStack };
|
|
2395
|
+
} else if (detectedLanguage === 'go') {
|
|
2396
|
+
const goStack = await detectGoStack(projectRoot);
|
|
2397
|
+
stack = { ...stack, ...goStack };
|
|
2398
|
+
} else if (detectedLanguage === 'rust') {
|
|
2399
|
+
const rustStack = await detectRustStack(projectRoot);
|
|
2400
|
+
stack = { ...stack, ...rustStack };
|
|
2401
|
+
}
|
|
2402
|
+
|
|
2403
|
+
// Generate test based on language
|
|
2404
|
+
let result;
|
|
2405
|
+
|
|
2406
|
+
if (detectedLanguage === 'python') {
|
|
2407
|
+
result = await generatePythonTest({
|
|
2408
|
+
filePath,
|
|
2409
|
+
code,
|
|
2410
|
+
testFramework: stack.backend?.testFramework || 'pytest',
|
|
2411
|
+
});
|
|
2412
|
+
} else if (detectedLanguage === 'go') {
|
|
2413
|
+
// Extract package name from file
|
|
2414
|
+
const packageMatch = code.match(/package\s+(\w+)/);
|
|
2415
|
+
const packageName = packageMatch ? packageMatch[1] : 'main';
|
|
2416
|
+
|
|
2417
|
+
result = await generateGoTest({
|
|
2418
|
+
filePath,
|
|
2419
|
+
code,
|
|
2420
|
+
packageName,
|
|
2421
|
+
});
|
|
2422
|
+
} else if (detectedLanguage === 'rust') {
|
|
2423
|
+
result = await generateRustTest({
|
|
2424
|
+
filePath,
|
|
2425
|
+
code,
|
|
2426
|
+
});
|
|
2427
|
+
} else if (filePath.match(/\.(tsx|jsx)$/) && stack.frontend?.framework === 'react') {
|
|
2428
|
+
result = await generateReactTest({
|
|
2429
|
+
filePath,
|
|
2430
|
+
code,
|
|
2431
|
+
testFramework: stack.frontend.testFramework || 'vitest',
|
|
2432
|
+
});
|
|
2433
|
+
} else if (filePath.match(/\.ts$/)) {
|
|
2434
|
+
result = await generateNodeJsTest({
|
|
2435
|
+
filePath,
|
|
2436
|
+
code,
|
|
2437
|
+
testFramework: stack.backend?.testFramework || 'vitest',
|
|
2438
|
+
});
|
|
2439
|
+
} else {
|
|
2440
|
+
return {
|
|
2441
|
+
success: false,
|
|
2442
|
+
error: `Unsupported file type or language: ${detectedLanguage}`,
|
|
2443
|
+
};
|
|
2444
|
+
}
|
|
2445
|
+
|
|
2446
|
+
// Write test file
|
|
2447
|
+
const testFilePath = output || result.testFilePath;
|
|
2448
|
+
await fs.writeFile(testFilePath, result.testCode, 'utf-8');
|
|
2449
|
+
|
|
2450
|
+
return {
|
|
2451
|
+
success: true,
|
|
2452
|
+
testFilePath,
|
|
2453
|
+
};
|
|
2454
|
+
} catch (error) {
|
|
2455
|
+
return {
|
|
2456
|
+
success: false,
|
|
2457
|
+
error: error instanceof Error ? error.message : 'Unknown error',
|
|
2458
|
+
};
|
|
2459
|
+
}
|
|
2460
|
+
}
|
|
2461
|
+
|
|
2462
|
+
function detectLanguageFromFile(filePath: string): 'nodejs' | 'python' | 'go' | 'rust' {
|
|
2463
|
+
if (filePath.endsWith('.py')) return 'python';
|
|
2464
|
+
if (filePath.endsWith('.go')) return 'go';
|
|
2465
|
+
if (filePath.endsWith('.rs')) return 'rust';
|
|
2466
|
+
return 'nodejs';
|
|
2467
|
+
}
|
|
2468
|
+
```
|
|
2469
|
+
|
|
2470
|
+
Update `src/testing/detectors/index.ts`:
|
|
2471
|
+
|
|
2472
|
+
```typescript
|
|
2473
|
+
import { detectFromPackageJson } from './package-json';
|
|
2474
|
+
import { detectPythonStack } from './python';
|
|
2475
|
+
import { detectGoStack } from './go';
|
|
2476
|
+
import { detectRustStack } from './rust';
|
|
2477
|
+
import type { TechStack } from '../types';
|
|
2478
|
+
|
|
2479
|
+
export async function detectTechStack(projectRoot: string): Promise<TechStack> {
|
|
2480
|
+
let stack: TechStack = {};
|
|
2481
|
+
|
|
2482
|
+
// Try Node.js detection
|
|
2483
|
+
try {
|
|
2484
|
+
const nodeStack = await detectFromPackageJson(projectRoot);
|
|
2485
|
+
stack = { ...stack, ...nodeStack };
|
|
2486
|
+
} catch (error) {
|
|
2487
|
+
// package.json not found
|
|
2488
|
+
}
|
|
2489
|
+
|
|
2490
|
+
// Try Python detection
|
|
2491
|
+
try {
|
|
2492
|
+
const pythonStack = await detectPythonStack(projectRoot);
|
|
2493
|
+
stack = { ...stack, ...pythonStack };
|
|
2494
|
+
} catch (error) {
|
|
2495
|
+
// requirements.txt not found
|
|
2496
|
+
}
|
|
2497
|
+
|
|
2498
|
+
// Try Go detection
|
|
2499
|
+
try {
|
|
2500
|
+
const goStack = await detectGoStack(projectRoot);
|
|
2501
|
+
stack = { ...stack, ...goStack };
|
|
2502
|
+
} catch (error) {
|
|
2503
|
+
// go.mod not found
|
|
2504
|
+
}
|
|
2505
|
+
|
|
2506
|
+
// Try Rust detection
|
|
2507
|
+
try {
|
|
2508
|
+
const rustStack = await detectRustStack(projectRoot);
|
|
2509
|
+
stack = { ...stack, ...rustStack };
|
|
2510
|
+
} catch (error) {
|
|
2511
|
+
// Cargo.toml not found
|
|
2512
|
+
}
|
|
2513
|
+
|
|
2514
|
+
return stack;
|
|
2515
|
+
}
|
|
2516
|
+
|
|
2517
|
+
export { detectFromPackageJson, detectPythonStack, detectGoStack, detectRustStack };
|
|
2518
|
+
```
|
|
2519
|
+
|
|
2520
|
+
**Step 4: Run test to verify it passes**
|
|
2521
|
+
|
|
2522
|
+
Run: `pnpm test tests/testing/cli/multi-language.test.ts`
|
|
2523
|
+
Expected: PASS
|
|
2524
|
+
|
|
2525
|
+
**Step 5: Commit**
|
|
2526
|
+
|
|
2527
|
+
```bash
|
|
2528
|
+
git add src/testing/cli/commands.ts src/testing/detectors/index.ts tests/testing/cli/multi-language.test.ts
|
|
2529
|
+
git commit -m "feat(testing): add multi-language support to CLI commands
|
|
2530
|
+
|
|
2531
|
+
- Support Python, Go, and Rust test generation
|
|
2532
|
+
- Auto-detect language from file extension
|
|
2533
|
+
- Integrate language-specific detectors
|
|
2534
|
+
- Update CLI to route to appropriate generator
|
|
2535
|
+
- Support all languages in omc test gen command
|
|
2536
|
+
|
|
2537
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
2538
|
+
```
|
|
2539
|
+
|
|
2540
|
+
---
|
|
2541
|
+
|
|
2542
|
+
## Task 10: Documentation and Phase 2 Summary
|
|
2543
|
+
|
|
2544
|
+
**Files:**
|
|
2545
|
+
- Update: `docs/testing/README.md`
|
|
2546
|
+
- Create: `docs/testing/PHASE2.md`
|
|
2547
|
+
- Update: `README.md`
|
|
2548
|
+
|
|
2549
|
+
**Step 1: Create Phase 2 documentation**
|
|
2550
|
+
|
|
2551
|
+
Create `docs/testing/PHASE2.md`:
|
|
2552
|
+
|
|
2553
|
+
```markdown
|
|
2554
|
+
# LLM Testing System - Phase 2 Features
|
|
2555
|
+
|
|
2556
|
+
Phase 2 extends the testing system with advanced features including coverage analysis, multi-language support, complexity analysis, contract testing, and workflow integration.
|
|
2557
|
+
|
|
2558
|
+
## New Features
|
|
2559
|
+
|
|
2560
|
+
### 1. Coverage Analysis
|
|
2561
|
+
|
|
2562
|
+
Analyze test coverage and identify gaps:
|
|
2563
|
+
|
|
2564
|
+
```bash
|
|
2565
|
+
omc test analyze
|
|
2566
|
+
```
|
|
2567
|
+
|
|
2568
|
+
Features:
|
|
2569
|
+
- Parse c8/nyc coverage reports
|
|
2570
|
+
- Identify uncovered code ranges
|
|
2571
|
+
- Analyze reasons for gaps (error handling, branches, etc.)
|
|
2572
|
+
- Generate supplementary tests for gaps
|
|
2573
|
+
|
|
2574
|
+
### 2. Multi-Language Support
|
|
2575
|
+
|
|
2576
|
+
Generate tests for Python, Go, and Rust:
|
|
2577
|
+
|
|
2578
|
+
```bash
|
|
2579
|
+
# Python (pytest)
|
|
2580
|
+
omc test gen src/utils/math.py
|
|
2581
|
+
|
|
2582
|
+
# Go (testing package)
|
|
2583
|
+
omc test gen pkg/math/math.go
|
|
2584
|
+
|
|
2585
|
+
# Rust (cargo test)
|
|
2586
|
+
omc test gen src/math.rs
|
|
2587
|
+
```
|
|
2588
|
+
|
|
2589
|
+
Supported frameworks:
|
|
2590
|
+
- **Python**: pytest, unittest
|
|
2591
|
+
- **Go**: testing package with table-driven tests
|
|
2592
|
+
- **Rust**: cargo test with #[test] attributes
|
|
2593
|
+
|
|
2594
|
+
### 3. Complexity Analysis
|
|
2595
|
+
|
|
2596
|
+
Automatically classify code as simple or complex:
|
|
2597
|
+
|
|
2598
|
+
```typescript
|
|
2599
|
+
const analysis = await analyzeComplexity({ code, filePath });
|
|
2600
|
+
// Returns: { complexity: 'simple' | 'complex', metrics, reasons }
|
|
2601
|
+
```
|
|
2602
|
+
|
|
2603
|
+
Metrics:
|
|
2604
|
+
- Lines of code
|
|
2605
|
+
- Cyclomatic complexity
|
|
2606
|
+
- Nesting levels
|
|
2607
|
+
- External dependencies
|
|
2608
|
+
|
|
2609
|
+
Complexity indicators:
|
|
2610
|
+
- Payment/auth logic
|
|
2611
|
+
- Database transactions
|
|
2612
|
+
- External API calls
|
|
2613
|
+
- Multiple async operations
|
|
2614
|
+
|
|
2615
|
+
### 4. Contract Testing
|
|
2616
|
+
|
|
2617
|
+
Generate API contract tests from OpenAPI specs:
|
|
2618
|
+
|
|
2619
|
+
```bash
|
|
2620
|
+
omc test contract api/openapi.yaml
|
|
2621
|
+
```
|
|
2622
|
+
|
|
2623
|
+
Supported frameworks:
|
|
2624
|
+
- **Pact**: Consumer-driven contract testing
|
|
2625
|
+
- **Supertest**: REST API contract tests
|
|
2626
|
+
- **MSW**: Mock Service Worker handlers
|
|
2627
|
+
|
|
2628
|
+
### 5. Enhanced Test-Engineer Agent
|
|
2629
|
+
|
|
2630
|
+
The test-engineer agent now receives enriched context:
|
|
2631
|
+
|
|
2632
|
+
- Tech stack detection
|
|
2633
|
+
- Complexity analysis
|
|
2634
|
+
- Suggested approach (auto/guided/manual)
|
|
2635
|
+
- Pre-generated questions for complex code
|
|
2636
|
+
|
|
2637
|
+
### 6. UltraQA Integration
|
|
2638
|
+
|
|
2639
|
+
UltraQA now includes automatic test generation:
|
|
2640
|
+
|
|
2641
|
+
```bash
|
|
2642
|
+
/ultraqa
|
|
2643
|
+
```
|
|
2644
|
+
|
|
2645
|
+
Workflow:
|
|
2646
|
+
1. Identify files needing tests
|
|
2647
|
+
2. Generate missing tests
|
|
2648
|
+
3. Run tests and analyze coverage
|
|
2649
|
+
4. Generate supplementary tests for gaps
|
|
2650
|
+
5. Repeat until coverage threshold met
|
|
2651
|
+
|
|
2652
|
+
## Usage Examples
|
|
2653
|
+
|
|
2654
|
+
### Example 1: Python Test Generation
|
|
2655
|
+
|
|
2656
|
+
```bash
|
|
2657
|
+
omc test gen src/calculator.py
|
|
2658
|
+
```
|
|
2659
|
+
|
|
2660
|
+
Output:
|
|
2661
|
+
```
|
|
2662
|
+
✅ Detected: Python + pytest
|
|
2663
|
+
✅ Generated: tests/test_calculator.py
|
|
2664
|
+
|
|
2665
|
+
Tests include:
|
|
2666
|
+
- test_add
|
|
2667
|
+
- test_subtract
|
|
2668
|
+
- test_multiply
|
|
2669
|
+
- test_divide
|
|
2670
|
+
```
|
|
2671
|
+
|
|
2672
|
+
### Example 2: Coverage Analysis
|
|
2673
|
+
|
|
2674
|
+
```bash
|
|
2675
|
+
omc test analyze
|
|
2676
|
+
```
|
|
2677
|
+
|
|
2678
|
+
Output:
|
|
2679
|
+
```
|
|
2680
|
+
📊 Coverage Analysis:
|
|
2681
|
+
- Overall: 75%
|
|
2682
|
+
- Lines: 75%
|
|
2683
|
+
- Functions: 90%
|
|
2684
|
+
- Branches: 70%
|
|
2685
|
+
|
|
2686
|
+
Coverage Gaps:
|
|
2687
|
+
1. src/utils/validation.ts (lines 42-48)
|
|
2688
|
+
Reason: Error handling not covered
|
|
2689
|
+
|
|
2690
|
+
2. src/services/payment.ts (lines 67-72)
|
|
2691
|
+
Reason: Edge case for retries
|
|
2692
|
+
|
|
2693
|
+
Generate tests for gaps? (y/n)
|
|
2694
|
+
```
|
|
2695
|
+
|
|
2696
|
+
### Example 3: Contract Testing
|
|
2697
|
+
|
|
2698
|
+
```bash
|
|
2699
|
+
omc test contract api/openapi.yaml --framework=pact
|
|
2700
|
+
```
|
|
2701
|
+
|
|
2702
|
+
Output:
|
|
2703
|
+
```
|
|
2704
|
+
✅ Generated: tests/contract/frontend-backend.pact.test.ts
|
|
2705
|
+
|
|
2706
|
+
Contract tests:
|
|
2707
|
+
- GET /users/{id}
|
|
2708
|
+
- POST /users
|
|
2709
|
+
- PUT /users/{id}
|
|
2710
|
+
- DELETE /users/{id}
|
|
2711
|
+
```
|
|
2712
|
+
|
|
2713
|
+
### Example 4: Complex Code with Test-Engineer
|
|
2714
|
+
|
|
2715
|
+
```bash
|
|
2716
|
+
/test-gen src/services/payment.ts
|
|
2717
|
+
```
|
|
2718
|
+
|
|
2719
|
+
Output:
|
|
2720
|
+
```
|
|
2721
|
+
Agent: Detecting tech stack...
|
|
2722
|
+
✅ Detected: Node.js + Express + PostgreSQL + Vitest
|
|
2723
|
+
|
|
2724
|
+
Agent: Analyzing complexity...
|
|
2725
|
+
⚠️ Complex code detected:
|
|
2726
|
+
- Payment processing logic
|
|
2727
|
+
- External Stripe API calls
|
|
2728
|
+
- Database transactions
|
|
2729
|
+
- Multiple async operations
|
|
2730
|
+
|
|
2731
|
+
Agent: Delegating to test-engineer for detailed test cases...
|
|
2732
|
+
|
|
2733
|
+
Test-Engineer: I'll need some information:
|
|
2734
|
+
1. What are the expected payment flows? (success, failure, retry)
|
|
2735
|
+
2. Should I mock Stripe API calls?
|
|
2736
|
+
3. What database states should I test?
|
|
2737
|
+
4. Are there specific edge cases to cover?
|
|
2738
|
+
|
|
2739
|
+
[User provides details]
|
|
2740
|
+
|
|
2741
|
+
Test-Engineer: Generating comprehensive test suite...
|
|
2742
|
+
✅ Generated 12 test cases covering:
|
|
2743
|
+
- Happy path payment processing
|
|
2744
|
+
- Stripe API failure scenarios
|
|
2745
|
+
- Database transaction rollbacks
|
|
2746
|
+
- Idempotency checks
|
|
2747
|
+
- Concurrent payment handling
|
|
2748
|
+
```
|
|
2749
|
+
|
|
2750
|
+
## Configuration
|
|
2751
|
+
|
|
2752
|
+
Add to `.omc/project-config.json`:
|
|
2753
|
+
|
|
2754
|
+
```json
|
|
2755
|
+
{
|
|
2756
|
+
"testing": {
|
|
2757
|
+
"coverageThreshold": 80,
|
|
2758
|
+
"complexityThresholds": {
|
|
2759
|
+
"lines": 50,
|
|
2760
|
+
"cyclomaticComplexity": 10,
|
|
2761
|
+
"nestingLevel": 3
|
|
2762
|
+
},
|
|
2763
|
+
"autoGenerateTests": true,
|
|
2764
|
+
"languages": ["nodejs", "python", "go", "rust"]
|
|
2765
|
+
}
|
|
2766
|
+
}
|
|
2767
|
+
```
|
|
2768
|
+
|
|
2769
|
+
## Architecture
|
|
2770
|
+
|
|
2771
|
+
```
|
|
2772
|
+
src/testing/
|
|
2773
|
+
├── analyzers/
|
|
2774
|
+
│ ├── coverage.ts # Coverage analysis
|
|
2775
|
+
│ ├── complexity.ts # Complexity analysis
|
|
2776
|
+
│ └── types.ts # Analyzer types
|
|
2777
|
+
├── generators/
|
|
2778
|
+
│ ├── react.ts # React component tests
|
|
2779
|
+
│ ├── nodejs.ts # Node.js function tests
|
|
2780
|
+
│ ├── python.ts # Python pytest tests
|
|
2781
|
+
│ ├── go.ts # Go table-driven tests
|
|
2782
|
+
│ ├── rust.ts # Rust cargo tests
|
|
2783
|
+
│ └── contract.ts # API contract tests
|
|
2784
|
+
├── detectors/
|
|
2785
|
+
│ ├── index.ts # Multi-language detection
|
|
2786
|
+
│ ├── package-json.ts # Node.js detection
|
|
2787
|
+
│ ├── python.ts # Python detection
|
|
2788
|
+
│ ├── go.ts # Go detection
|
|
2789
|
+
│ └── rust.ts # Rust detection
|
|
2790
|
+
└── cli/
|
|
2791
|
+
├── commands.ts # CLI command implementations
|
|
2792
|
+
├── agent-integration.ts # Test-engineer integration
|
|
2793
|
+
└── ultraqa-integration.ts # UltraQA integration
|
|
2794
|
+
```
|
|
2795
|
+
|
|
2796
|
+
## Next Steps (Phase 3)
|
|
2797
|
+
|
|
2798
|
+
- Giskard integration for behavior testing
|
|
2799
|
+
- E2E test generation with Playwright
|
|
2800
|
+
- CI/CD integration
|
|
2801
|
+
- Ralph mode test loops
|
|
2802
|
+
- Autopilot automatic testing
|
|
2803
|
+
- Performance optimization
|
|
2804
|
+
|
|
2805
|
+
## Success Metrics
|
|
2806
|
+
|
|
2807
|
+
Phase 2 Achievements:
|
|
2808
|
+
- ✅ Multi-language support (Python, Go, Rust)
|
|
2809
|
+
- ✅ Coverage analysis and gap identification
|
|
2810
|
+
- ✅ Complexity analysis for smart test generation
|
|
2811
|
+
- ✅ Contract testing for APIs
|
|
2812
|
+
- ✅ Enhanced test-engineer agent
|
|
2813
|
+
- ✅ UltraQA integration
|
|
2814
|
+
|
|
2815
|
+
Target Metrics:
|
|
2816
|
+
- Test coverage: 80%+
|
|
2817
|
+
- Test generation time: < 30 seconds/file
|
|
2818
|
+
- Multi-language support: 4 languages
|
|
2819
|
+
- Complexity classification accuracy: > 90%
|
|
2820
|
+
```
|
|
2821
|
+
|
|
2822
|
+
**Step 2: Update main testing README**
|
|
2823
|
+
|
|
2824
|
+
Update `docs/testing/README.md` to add Phase 2 section:
|
|
2825
|
+
|
|
2826
|
+
```markdown
|
|
2827
|
+
## Phase 2 Features (NEW)
|
|
2828
|
+
|
|
2829
|
+
Phase 2 adds advanced capabilities:
|
|
2830
|
+
|
|
2831
|
+
- **Coverage Analysis**: Identify and fill coverage gaps
|
|
2832
|
+
- **Multi-Language**: Python, Go, Rust support
|
|
2833
|
+
- **Complexity Analysis**: Smart classification of code complexity
|
|
2834
|
+
- **Contract Testing**: API contract tests from OpenAPI specs
|
|
2835
|
+
- **Enhanced Agent**: Test-engineer with enriched context
|
|
2836
|
+
- **UltraQA Integration**: Automatic test generation in QA cycles
|
|
2837
|
+
|
|
2838
|
+
See [Phase 2 Documentation](./PHASE2.md) for details.
|
|
2839
|
+
|
|
2840
|
+
## Supported Languages
|
|
2841
|
+
|
|
2842
|
+
- **Node.js**: Vitest, Jest
|
|
2843
|
+
- **Python**: pytest, unittest
|
|
2844
|
+
- **Go**: testing package
|
|
2845
|
+
- **Rust**: cargo test
|
|
2846
|
+
- **React**: Vitest + Testing Library
|
|
2847
|
+
```
|
|
2848
|
+
|
|
2849
|
+
**Step 3: Update main README**
|
|
2850
|
+
|
|
2851
|
+
Update `README.md` to highlight Phase 2 features:
|
|
2852
|
+
|
|
2853
|
+
```markdown
|
|
2854
|
+
## Testing (Phase 2 - NEW)
|
|
2855
|
+
|
|
2856
|
+
oh-my-claudecode now supports advanced test generation across multiple languages:
|
|
2857
|
+
|
|
2858
|
+
```bash
|
|
2859
|
+
# Node.js/TypeScript
|
|
2860
|
+
omc test gen src/utils/math.ts
|
|
2861
|
+
|
|
2862
|
+
# Python
|
|
2863
|
+
omc test gen src/utils/math.py
|
|
2864
|
+
|
|
2865
|
+
# Go
|
|
2866
|
+
omc test gen pkg/math/math.go
|
|
2867
|
+
|
|
2868
|
+
# Rust
|
|
2869
|
+
omc test gen src/math.rs
|
|
2870
|
+
|
|
2871
|
+
# Coverage analysis
|
|
2872
|
+
omc test analyze
|
|
2873
|
+
|
|
2874
|
+
# Contract testing
|
|
2875
|
+
omc test contract api/openapi.yaml
|
|
2876
|
+
```
|
|
2877
|
+
|
|
2878
|
+
Features:
|
|
2879
|
+
- Multi-language support (Node.js, Python, Go, Rust)
|
|
2880
|
+
- Coverage analysis and gap identification
|
|
2881
|
+
- Complexity-based test generation
|
|
2882
|
+
- API contract testing
|
|
2883
|
+
- Integrated with /ultraqa workflow
|
|
2884
|
+
|
|
2885
|
+
See [Testing Documentation](docs/testing/README.md) for details.
|
|
2886
|
+
```
|
|
2887
|
+
|
|
2888
|
+
**Step 4: Commit**
|
|
2889
|
+
|
|
2890
|
+
```bash
|
|
2891
|
+
git add docs/testing/ README.md
|
|
2892
|
+
git commit -m "docs(testing): add Phase 2 documentation and examples
|
|
2893
|
+
|
|
2894
|
+
- Create comprehensive Phase 2 feature documentation
|
|
2895
|
+
- Add usage examples for all new features
|
|
2896
|
+
- Update main README with Phase 2 highlights
|
|
2897
|
+
- Document multi-language support
|
|
2898
|
+
- Include configuration examples
|
|
2899
|
+
|
|
2900
|
+
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>"
|
|
2901
|
+
```
|
|
2902
|
+
|
|
2903
|
+
---
|
|
2904
|
+
|
|
2905
|
+
## Phase 2 Summary
|
|
2906
|
+
|
|
2907
|
+
### Deliverables Checklist
|
|
2908
|
+
|
|
2909
|
+
- [x] **Coverage Analysis**
|
|
2910
|
+
- [x] Coverage analyzer for Node.js (c8/nyc)
|
|
2911
|
+
- [x] Gap identification with line ranges
|
|
2912
|
+
- [x] Reason analysis for uncovered code
|
|
2913
|
+
|
|
2914
|
+
- [x] **Multi-Language Support**
|
|
2915
|
+
- [x] Python test generator (pytest/unittest)
|
|
2916
|
+
- [x] Go test generator (table-driven tests)
|
|
2917
|
+
- [x] Rust test generator (cargo test)
|
|
2918
|
+
- [x] Language-specific tech stack detectors
|
|
2919
|
+
|
|
2920
|
+
- [x] **Complexity Analysis**
|
|
2921
|
+
- [x] Cyclomatic complexity calculation
|
|
2922
|
+
- [x] Nesting level measurement
|
|
2923
|
+
- [x] External dependency detection
|
|
2924
|
+
- [x] Pattern-based complexity indicators
|
|
2925
|
+
|
|
2926
|
+
- [x] **Contract Testing**
|
|
2927
|
+
- [x] Pact consumer-driven contract tests
|
|
2928
|
+
- [x] Supertest REST API contracts
|
|
2929
|
+
- [x] MSW handler generation
|
|
2930
|
+
- [x] OpenAPI spec parsing
|
|
2931
|
+
|
|
2932
|
+
- [x] **Enhanced Test-Engineer Agent**
|
|
2933
|
+
- [x] Context preparation with tech stack
|
|
2934
|
+
- [x] Complexity-based question generation
|
|
2935
|
+
- [x] Suggested approach determination
|
|
2936
|
+
- [x] Agent documentation updates
|
|
2937
|
+
|
|
2938
|
+
- [x] **UltraQA Integration**
|
|
2939
|
+
- [x] Automatic test generation in QA cycles
|
|
2940
|
+
- [x] Coverage gap identification
|
|
2941
|
+
- [x] Test generation for changed files
|
|
2942
|
+
- [x] Skill documentation updates
|
|
2943
|
+
|
|
2944
|
+
- [x] **CLI Integration**
|
|
2945
|
+
- [x] Multi-language command routing
|
|
2946
|
+
- [x] Auto-detection of language from file
|
|
2947
|
+
- [x] Unified test generation interface
|
|
2948
|
+
|
|
2949
|
+
- [x] **Documentation**
|
|
2950
|
+
- [x] Phase 2 feature documentation
|
|
2951
|
+
- [x] Usage examples for all features
|
|
2952
|
+
- [x] Architecture documentation
|
|
2953
|
+
- [x] Configuration examples
|
|
2954
|
+
|
|
2955
|
+
### Estimated Timeline
|
|
2956
|
+
|
|
2957
|
+
- **Task 1**: Coverage Analyzer - 1.5 hours
|
|
2958
|
+
- **Task 2**: Python Test Generator - 2 hours
|
|
2959
|
+
- **Task 3**: Go Test Generator - 1.5 hours
|
|
2960
|
+
- **Task 4**: Rust Test Generator - 1.5 hours
|
|
2961
|
+
- **Task 5**: Complexity Analyzer - 1.5 hours
|
|
2962
|
+
- **Task 6**: Contract Test Generator - 2 hours
|
|
2963
|
+
- **Task 7**: Test-Engineer Integration - 1.5 hours
|
|
2964
|
+
- **Task 8**: UltraQA Integration - 1.5 hours
|
|
2965
|
+
- **Task 9**: Multi-Language CLI - 1 hour
|
|
2966
|
+
- **Task 10**: Documentation - 1 hour
|
|
2967
|
+
|
|
2968
|
+
**Total**: ~15 hours
|
|
2969
|
+
|
|
2970
|
+
### Success Criteria
|
|
2971
|
+
|
|
2972
|
+
- [ ] All tests pass: `pnpm test tests/testing/**`
|
|
2973
|
+
- [ ] Coverage analyzer works for Node.js projects
|
|
2974
|
+
- [ ] Python, Go, and Rust test generation produces valid tests
|
|
2975
|
+
- [ ] Complexity analyzer correctly classifies simple vs complex code
|
|
2976
|
+
- [ ] Contract tests generate from OpenAPI specs
|
|
2977
|
+
- [ ] Test-engineer receives enriched context
|
|
2978
|
+
- [ ] UltraQA automatically generates missing tests
|
|
2979
|
+
- [ ] CLI supports all languages with auto-detection
|
|
2980
|
+
- [ ] Documentation is complete and accurate
|
|
2981
|
+
|
|
2982
|
+
### Next Steps (Phase 3)
|
|
2983
|
+
|
|
2984
|
+
1. **Giskard Integration**
|
|
2985
|
+
- Behavior testing (perturbations, robustness)
|
|
2986
|
+
- Fairness metrics
|
|
2987
|
+
- LLM-specific test scenarios
|
|
2988
|
+
|
|
2989
|
+
2. **E2E Test Generation**
|
|
2990
|
+
- Playwright test generation
|
|
2991
|
+
- User flow testing
|
|
2992
|
+
- Cross-stack integration tests
|
|
2993
|
+
|
|
2994
|
+
3. **CI/CD Integration**
|
|
2995
|
+
- GitHub Actions workflow
|
|
2996
|
+
- Automatic test generation on PR
|
|
2997
|
+
- Coverage reporting
|
|
2998
|
+
|
|
2999
|
+
4. **Ralph Mode Integration**
|
|
3000
|
+
- Test-fix-verify loops
|
|
3001
|
+
- Automatic test regeneration on failure
|
|
3002
|
+
- Continuous improvement
|
|
3003
|
+
|
|
3004
|
+
5. **Autopilot Integration**
|
|
3005
|
+
- Automatic test generation during code creation
|
|
3006
|
+
- Test-first development mode
|
|
3007
|
+
- Coverage threshold enforcement
|
|
3008
|
+
|
|
3009
|
+
6. **Performance Optimization**
|
|
3010
|
+
- Parallel test generation
|
|
3011
|
+
- Caching of tech stack detection
|
|
3012
|
+
- Incremental coverage analysis
|
|
3013
|
+
|
|
3014
|
+
### Implementation Notes
|
|
3015
|
+
|
|
3016
|
+
#### TDD Approach
|
|
3017
|
+
|
|
3018
|
+
Each task follows strict TDD:
|
|
3019
|
+
1. Write failing test
|
|
3020
|
+
2. Run test to verify failure
|
|
3021
|
+
3. Implement minimal code to pass
|
|
3022
|
+
4. Run test to verify pass
|
|
3023
|
+
5. Commit with descriptive message
|
|
3024
|
+
|
|
3025
|
+
#### Code Quality
|
|
3026
|
+
|
|
3027
|
+
- All code must pass TypeScript type checking
|
|
3028
|
+
- All tests must pass before committing
|
|
3029
|
+
- Follow existing code style and conventions
|
|
3030
|
+
- Use meaningful variable and function names
|
|
3031
|
+
- Add comments for complex logic
|
|
3032
|
+
|
|
3033
|
+
#### Git Workflow
|
|
3034
|
+
|
|
3035
|
+
- Work on feature branch: `git checkout -b feature/llm-testing-phase2 dev`
|
|
3036
|
+
- Commit after each task completion
|
|
3037
|
+
- Use conventional commit format: `feat(testing): description`
|
|
3038
|
+
- Include co-author tag: `Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>`
|
|
3039
|
+
- Create PR targeting `dev` branch when complete
|
|
3040
|
+
|
|
3041
|
+
#### Testing Strategy
|
|
3042
|
+
|
|
3043
|
+
- Unit tests for all functions
|
|
3044
|
+
- Integration tests for CLI commands
|
|
3045
|
+
- Mock external dependencies (file system, APIs)
|
|
3046
|
+
- Use Vitest for all tests
|
|
3047
|
+
- Aim for >80% code coverage
|
|
3048
|
+
|
|
3049
|
+
---
|
|
3050
|
+
|
|
3051
|
+
**Plan Status**: ✅ Complete and ready for implementation
|
|
3052
|
+
|
|
3053
|
+
**Next Action**: Begin Task 1 - Coverage Analyzer for Node.js
|