ai-unit-test-generator 1.3.0

package/README.md

@@ -0,0 +1,432 @@
+# ai-test-generator
+
+> AI-powered unit test generator with smart priority scoring
+
+[![npm version](https://img.shields.io/npm/v/ai-test-generator.svg)](https://www.npmjs.com/package/ai-test-generator)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+## 🎯 Features
+
+- **Layered Architecture**: Scores based on code layer (Foundation, Business Logic, State Management, UI)
+- **AI-Optimized**: Designed for AI-powered test generation workflows
+- **Batch Test Generation**: Built-in tools for generating tests with AI (ChatGPT, Claude, Cursor Agent, etc.)
+- **Multiple Metrics**: Combines testability, complexity, dependency count, and more
+- **Flexible Configuration**: JSON-based config with presets for different project types
+- **CLI & Programmatic API**: Use as a command-line tool or integrate into your workflow
+- **Rich Reports**: Generates Markdown and CSV reports with detailed scoring
+- **Progress Tracking**: Mark functions as DONE/TODO/SKIP to track testing progress
+
+## 📦 Installation
+
+### Global Installation (CLI)
+
+```bash
+npm install -g ai-test-generator
+```
+
+### Project Installation
+
+```bash
+npm install --save-dev ai-test-generator
+```
+
+## 🚀 Quick Start
+
+### 1. Initialize Configuration
+
+```bash
+ai-test init
+```
+
+This creates `ut_scoring_config.json` with default settings.
+
+### 2. Scan and Score
+
+```bash
+ai-test scan
+```
+
+This will:
+1. Scan your codebase for testable targets
+2. Analyze Git history for error signals
+3. Calculate priority scores
+4. Generate reports in `reports/`
+
+### 3. View Report
+
+```bash
+# View complete report
+cat reports/ut_scores.md
+
+# View P0 targets only
+grep "| P0 |" reports/ut_scores.md
+
+# Export P0 targets to file
+grep "| P0 |" reports/ut_scores.md | awk -F'|' '{print $2}' > p0_targets.txt
+```
+
+### 4. Generate Tests with AI
+
+```bash
+# Generate prompt for AI
+ai-test gen-prompt -p P0 -n 10 > prompt.txt
+
+# Copy prompt.txt to your AI tool (ChatGPT, Claude, Cursor, etc.)
+# Get the response, save as response.txt
+
+# Extract test files from AI response
+ai-test extract-tests response.txt
+
+# Run tests
+npm test
+
+# Mark as done
+ai-test mark-done func1 func2 func3
+```
+
+## 📖 CLI Commands
+
+### `ai-test init`
+
+Initialize UT scoring configuration.
+
+```bash
+ai-test init [options]
+
+Options:
+  -p, --preset <type>   Use preset config (default)
+  -o, --output <path>   Output config file path (default: ut_scoring_config.json)
+```
+
+### `ai-test scan`
+
+Scan code and generate UT priority scoring report.
+
+```bash
+ai-test scan [options]
+
+Options:
+  -c, --config <path>   Config file path (default: ut_scoring_config.json)
+  -o, --output <dir>    Output directory (default: reports)
+  --skip-git            Skip Git signals generation
+```
+
+**Output Files:**
+- `reports/targets.json` - List of all scanned targets
+- `reports/git_signals.json` - Git analysis data
+- `reports/ut_scores.md` - Priority scoring report (Markdown, sorted by score)
+- `reports/ut_scores.csv` - Priority scoring report (CSV)
+
+### `ai-test gen-prompt`
+
+Generate AI prompt for batch test generation.
+
+```bash
+ai-test gen-prompt [options] > prompt.txt
+
+Options:
+  -p, --priority <level>     Filter by priority (P0, P1, P2, P3)
+  -l, --layer <name>         Filter by layer (Foundation, Business, State, UI)
+  -n, --limit <count>        Limit number of targets
+      --skip <count>         Skip first N targets (for pagination)
+      --min-score <score>    Minimum score threshold
+      --report <path>        Report file path (default: reports/ut_scores.md)
+      --framework <name>     Project framework (default: React + TypeScript)
+      --hints <text>         Inject failure hints text into prompt
+      --hints-file <path>    Inject failure hints from file (e.g., reports/hints.txt)
+      --only-paths <csv>     Restrict to specific source paths (comma-separated)
+```
+
+**Example:**
+```bash
+# Generate prompt for 10 P0 functions
+ai-test gen-prompt -p P0 -n 10 > prompt.txt
+
+# Generate prompt for Foundation layer with high scores
+ai-test gen-prompt -l Foundation --min-score 7.5 -n 5 > prompt.txt
+
+# Generate next batch (skip first 10)
+ai-test gen-prompt -p P0 --skip 10 -n 10 > prompt.txt
+```
+
+### `ai-test extract-tests`
+
+Extract test files from AI response.
+
+```bash
+ai-test extract-tests <response-file> [options]
+
+Options:
+  --overwrite    Overwrite existing test files
+  --dry-run      Show what would be created without actually creating files
+```
+
+**Example:**
+```bash
+# Extract tests from AI response
+ai-test extract-tests response.txt
+
+# Dry-run to see what would be created
+ai-test extract-tests response.txt --dry-run
+
+# Overwrite existing tests
+ai-test extract-tests response.txt --overwrite
+```
+
+### `ai-test mark-done`
+
+Mark functions as done in the scoring report.
+
+```bash
+ai-test mark-done <function-names...> [options]
+
+Options:
+  --report <path>       Report file path (default: reports/ut_scores.md)
+  --status <status>     Mark status: DONE or SKIP (default: DONE)
+  --dry-run             Show what would be changed without actually changing
+```
+
+**Example:**
+```bash
+# Mark functions as done
+ai-test mark-done disableDragBack getMediumScale
+
+# Mark as skipped
+ai-test mark-done complexFunction --status SKIP
+
+# Dry-run
+ai-test mark-done func1 func2 --dry-run
+```
+
+## ⚙️ Configuration
+
+### Basic Configuration
+
+```json
+{
+  "scoringMode": "layered",
+  "layers": {
+    "foundation": {
+      "name": "Foundation (基础工具层)",
+      "patterns": ["utils/**", "helpers/**", "constants/**"],
+      "weights": {
+        "testability": 0.50,
+        "dependencyCount": 0.30,
+        "complexity": 0.20
+      },
+      "thresholds": {
+        "P0": 7.5,
+        "P1": 6.0,
+        "P2": 4.0
+      }
+    }
+  }
+}
+```
+
+### Presets
+
+- **default**: Balanced scoring for general projects
+- **react**: Optimized for React applications
+- **node**: Optimized for Node.js projects
+
+## 📊 Priority Levels
+
+| Priority | Score Range | Description | Action |
+|----------|-------------|-------------|--------|
+| **P0** | ≥7.5 | Must test | Generate immediately with AI |
+| **P1** | 6.0-7.4 | High priority | Batch generate |
+| **P2** | 4.0-5.9 | Medium priority | Generate with careful review |
+| **P3** | <4.0 | Low priority | Optional coverage |
+
+## 🏗️ Architecture
+
+### Layered Scoring System
+
+1. **Foundation Layer**: Utils, helpers, constants
+   - High testability weight (50%)
+   - Focus on pure functions
+   
+2. **Business Logic Layer**: Services, APIs, data processing
+   - Balanced metrics
+   - Critical for correctness
+
+3. **State Management Layer**: Stores, contexts, reducers
+   - Error risk weight emphasized
+   
+4. **UI Components Layer**: React components, views
+   - Complexity and error risk balanced
+
+## 🔧 Programmatic Usage
+
+If you need to integrate into your Node.js workflow:
+
+```javascript
+import { exec } from 'child_process'
+import { promisify } from 'util'
+
+const execAsync = promisify(exec)
+
+// Run scoring workflow
+const { stdout } = await execAsync('ai-test scan')
+console.log(stdout)
+```
+
+**Recommended**: Use the CLI directly for simplicity.
+
+## 🤖 AI Integration
+
+This tool is designed for AI-powered test generation workflows:
+
+1. **Run scoring** to get prioritized target list
+   ```bash
+   ai-test scan
+   ```
+
+2. **Extract P0 targets**
+   ```bash
+   grep "| P0 |" reports/ut_scores.md | awk -F'|' '{print $2}' > p0_targets.txt
+   ```
+
+3. **Batch Automation with Cursor Agent**
+  ```bash
+  # One batch
+  ai-test run-batch -p P0 -n 10 --skip 0
+
+  # Run all batches until TODO=0
+  ai-test run-all -p P0 -n 10
+  ```
+
+4. **Failure Feedback Loop**
+  - After each batch, we parse Jest JSON report and write hints into `reports/hints.txt`
+  - Next batch will inject hints via `--hints-file reports/hints.txt`
+
+### ✅ Quality Assurance (Default Enhanced Mode)
+
+To balance quality and speed, ai-test-generator enables a light assurance layer by default (inspired by Meta's TestGen-LLM deployment at scale).
+
+- Stability reruns (P0 only):
+  - `run-batch` reruns Jest once for P0 targets to reduce flakiness (runInBand)
+- Coverage-aware prioritization:
+  - `score-ut.mjs` supports coverageBoost to prioritize low-coverage files
+- Failure hints loop:
+  - `jest-failure-analyzer.mjs` produces actionable hints (import path, type errors, timers, selectors)
+  - Hints are injected to the next prompt automatically
+
+Notes:
+- Defaults are conservative (P0 strong, others light) to avoid runtime explosion.
+- You can further tune coverageBoost and thresholds in `ut_scoring_config.json`.
+
+## 📈 Metrics Explained
+
+### Testability (50% weight in Foundation layer)
+- Pure functions: 10/10
+- Simple mocks: 8-9/10
+- Complex dependencies: 4-6/10
+
+### Dependency Count (30% weight in Foundation layer)
+- ≥10 modules depend on it: 10/10
+- 5-9 modules: 10/10
+- 3-4 modules: 9/10
+- 1-2 modules: 7/10
+- Not referenced: 5/10
+
+### Complexity (20% weight in Foundation layer)
+- Cyclomatic complexity: 11-15: 10/10
+- Cognitive complexity from ESLint
+- Adjustments for nesting depth, branch count
+
+## 🛠️ Development
+
+### Project Structure
+
+```
+ut-priority-scorer/
+├── bin/
+│   └── ut-score.js           # CLI entry point
+├── lib/
+│   ├── index.js              # Main exports
+│   ├── gen-targets.mjs           # Target generation
+│   ├── gen-git-signals.mjs       # Git analysis
+│   ├── score-ut.mjs              # Scoring engine (supports coverage boost)
+│   ├── gen-test-prompt.mjs       # Build batch prompt (JSON manifest + code blocks)
+│   ├── extract-tests.mjs         # Extract tests (manifest-first, fallback headings)
+│   ├── ai-generate.mjs           # Call cursor-agent chat
+│   ├── jest-runner.mjs           # Run Jest and collect reports/coverage
+│   ├── jest-failure-analyzer.mjs # Parse Jest JSON and produce hints
+│   ├── run-batch.mjs             # Orchestrate one batch
+│   └── run-all.mjs               # Loop through all batches
+├── templates/
+│   ├── default.config.json   # Default preset
+│   ├── react.config.json     # React preset
+│   └── node.config.json      # Node.js preset
+└── docs/
+    ├── workflow.md           # Detailed workflow
+    └── architecture.md       # Architecture design
+```
+
+## 📝 Examples
+
+### Example: Scoring a React Project
+
+```bash
+# Initialize with default config
+ai-test init
+
+# Customize config if needed
+vim ut_scoring_config.json
+
+# Run scoring
+ai-test scan
+
+# View P0 targets
+grep "| P0 |" reports/ut_scores.md
+```
+
+### Example: npm Scripts Integration
+
+Add to your `package.json`:
+
+```json
+{
+  "scripts": {
+    "test:priority": "ai-test scan",
+    "test:p0": "grep '| P0 |' reports/ut_scores.md"
+  }
+}
+```
+
+## 🤝 Contributing
+
+Contributions are welcome! Please read our contributing guidelines.
+
+## 📄 License
+
+MIT © YuhengZhou
+
+## 🔗 Links
+
+- [GitHub Repository](https://github.com/temptrip/ai-test-generator)
+- [npm Package](https://www.npmjs.com/package/ai-test-generator)
+- [Issue Tracker](https://github.com/temptrip/ai-test-generator/issues)
+
+## 📚 Inspiration & Prior Art
+
+We learned from and adapted ideas in the community:
+
+- Meta TestGen-LLM — assurance filters and at-scale practices: [Automated Unit Test Improvement using Large Language Models at Meta](https://arxiv.org/abs/2402.09171)
+- ByteDance Midscene.js (NL to UI Test) — natural language interface and stability practices: `https://github.com/web-infra-dev/midscene`
+- Airbnb — large-scale LLM-aided migration and batching: `https://medium.com/airbnb-engineering/accelerating-large-scale-test-migration-with-llms-9565c208023b`
+- TestART (paper) — iterative generation and template repairs: `https://arxiv.org/abs/2408.03095`
+
+In ai-test-generator, these ideas shape:
+- Strict output protocol (JSON manifest + per-file code blocks)
+- Failure feedback loop (Jest JSON → actionable hints → next prompt)
+- Batch processing with progress tracking (TODO/DONE/SKIP)
+- Optional coverage-aware prioritization
+
+## 💬 Support
+
+- GitHub Issues: [Report bugs](https://github.com/temptrip/ut-priority-scorer/issues)
+- Documentation: [Read the docs](https://github.com/temptrip/ut-priority-scorer/tree/main/docs)
+

package/bin/cli.js

@@ -0,0 +1,217 @@
+#!/usr/bin/env node
+
+import { program } from 'commander'
+import { fileURLToPath } from 'url'
+import { dirname, join, resolve } from 'path'
+import { existsSync, copyFileSync, mkdirSync, readFileSync } from 'fs'
+import { spawn } from 'child_process'
+
+const __filename = fileURLToPath(import.meta.url)
+const __dirname = dirname(__filename)
+const PKG_ROOT = resolve(__dirname, '..')
+
+// 读取 package.json 版本
+const pkgJson = JSON.parse(readFileSync(join(PKG_ROOT, 'package.json'), 'utf-8'))
+
+program
+  .name('ai-test')
+  .description('AI-powered unit test generator with smart priority scoring')
+  .version(pkgJson.version)
+
+// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+// 命令 1: scan - 扫描代码并生成优先级报告
+// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+program
+  .command('scan')
+  .description('Scan code and generate priority scoring report')
+  .option('-c, --config <path>', 'Config file path', 'ut_scoring_config.json')
+  .option('-o, --output <dir>', 'Output directory', 'reports')
+  .option('--skip-git', 'Skip Git signals generation')
+  .action(async (options) => {
+    const { config, output, skipGit } = options
+
+    // 自动初始化配置（如果不存在）
+    if (!existsSync(config)) {
+      console.log('⚙️  Config not found, creating default config...')
+      const templatePath = join(PKG_ROOT, 'templates', 'default.config.json')
+      copyFileSync(templatePath, config)
+      console.log(`✅ Config created: ${config}\n`)
+    }
+
+    // 创建输出目录
+    if (!existsSync(output)) {
+      mkdirSync(output, { recursive: true })
+    }
+
+    console.log('🚀 Starting code scan...\n')
+
+    try {
+      // Step 1: 生成目标列表
+      console.log('📋 Step 1: Scanning targets...')
+      await runScript('core/scanner.mjs', [
+        '--config', config,
+        '--out', join(output, 'targets.json')
+      ])
+
+      // Step 2: 生成 Git 信号 (可选)
+      if (!skipGit) {
+        console.log('\n📊 Step 2: Analyzing Git history...')
+        await runScript('core/git-analyzer.mjs', [
+          '--targets', join(output, 'targets.json'),
+          '--out', join(output, 'git_signals.json')
+        ])
+      }
+
+      // Step 3: 运行评分（保留现有状态）
+      console.log('\n🎯 Step 3: Scoring targets...')
+      const scoreArgs = [
+        '--targets', join(output, 'targets.json'),
+        '--config', config,
+        '--out-md', join(output, 'ut_scores.md'),
+        '--out-csv', join(output, 'ut_scores.csv')
+      ]
+      if (!skipGit && existsSync(join(output, 'git_signals.json'))) {
+        scoreArgs.push('--git', join(output, 'git_signals.json'))
+      }
+      await runScript('core/scorer.mjs', scoreArgs)
+
+      // 统计 TODO/DONE
+      const reportPath = join(output, 'ut_scores.md')
+      if (existsSync(reportPath)) {
+        const content = readFileSync(reportPath, 'utf-8')
+        const todoCount = (content.match(/\| TODO \|/g) || []).length
+        const doneCount = (content.match(/\| DONE \|/g) || []).length
+        
+        console.log('\n✅ Scan completed!')
+        console.log(`\n📊 Status:`)
+        console.log(`   TODO: ${todoCount}`)
+        console.log(`   DONE: ${doneCount}`)
+        console.log(`   Total: ${todoCount + doneCount}`)
+        console.log(`\n📄 Report: ${reportPath}`)
+        console.log(`\n💡 Next: ai-test generate`)
+      }
+    } catch (err) {
+      console.error('❌ Scan failed:', err.message)
+      process.exit(1)
+    }
+  })
+
+// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+// 命令 2: generate - 生成测试
+// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+program
+  .command('generate')
+  .description('Generate tests (default: 10 TODO functions)')
+  .option('-n, --count <number>', 'Number of functions to generate', parseInt, 10)
+  .option('-p, --priority <level>', 'Priority filter (P0, P1, P2, P3)', 'P0')
+  .option('--all', 'Generate all remaining TODO functions')
+  .option('--report <path>', 'Report file path', 'reports/ut_scores.md')
+  .action(async (options) => {
+    const { count, priority, all, report } = options
+
+    // 检查报告是否存在
+    if (!existsSync(report)) {
+      console.error(`❌ Report not found: ${report}`)
+      console.log(`   Run: ai-test scan`)
+      process.exit(1)
+    }
+
+    if (all) {
+      // 持续生成直到所有 TODO 完成
+      console.log(`🚀 Generating all ${priority} TODO functions...\n`)
+      
+      let batchNum = 1
+      let totalGenerated = 0
+      let totalPassed = 0
+      
+      while (true) {
+        // 检查还有多少 TODO
+        const content = readFileSync(report, 'utf-8')
+        const lines = content.split('\n')
+        const todoLines = lines.filter(line => 
+          line.includes('| TODO |') && line.includes(`| ${priority} |`)
+        )
+        
+        if (todoLines.length === 0) {
+          console.log(`\n✅ All ${priority} functions completed!`)
+          console.log(`   Total generated: ${totalGenerated}`)
+          console.log(`   Total passed: ${totalPassed}`)
+          break
+        }
+        
+        console.log(`\n━━━━ Batch ${batchNum} (${todoLines.length} TODO remaining) ━━━━`)
+        
+        try {
+          const result = await generateBatch(priority, Math.min(count, todoLines.length), 0, report)
+          totalGenerated += result.generated
+          totalPassed += result.passed
+          batchNum++
+        } catch (err) {
+          console.error(`❌ Batch ${batchNum} failed:`, err.message)
+          break
+        }
+      }
+    } else {
+      // 生成指定数量
+      console.log(`🚀 Generating ${count} ${priority} functions...\n`)
+      try {
+        await generateBatch(priority, count, 0, report)
+      } catch (err) {
+        console.error('❌ Generation failed:', err.message)
+        process.exit(1)
+      }
+    }
+  })
+
+// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+// 辅助函数
+// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+/**
+ * 生成单批测试
+ */
+async function generateBatch(priority, count, skip, report) {
+  const batchScript = join(PKG_ROOT, 'lib/workflows/batch.mjs')
+  
+  return new Promise((resolve, reject) => {
+    const child = spawn('node', [batchScript, priority, String(count), String(skip), report], {
+      stdio: 'inherit',
+      cwd: process.cwd()
+    })
+    
+    child.on('close', (code) => {
+      if (code === 0) {
+        resolve({ generated: count, passed: count }) // TODO: 从 batch 输出获取实际数字
+      } else {
+        reject(new Error(`Batch generation failed with code ${code}`))
+      }
+    })
+    
+    child.on('error', reject)
+  })
+}
+
+/**
+ * 运行脚本
+ */
+function runScript(scriptPath, args) {
+  return new Promise((resolve, reject) => {
+    const fullPath = join(PKG_ROOT, 'lib', scriptPath)
+    const child = spawn('node', [fullPath, ...args], {
+      stdio: 'inherit',
+      cwd: process.cwd()
+    })
+
+    child.on('close', (code) => {
+      if (code === 0) {
+        resolve()
+      } else {
+        reject(new Error(`Script ${scriptPath} exited with code ${code}`))
+      }
+    })
+
+    child.on('error', reject)
+  })
+}
+
+program.parse()