test-genie-mcp 2.0.2 → 3.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (93) hide show
  1. package/CHANGELOG.md +224 -0
  2. package/LICENSE +21 -0
  3. package/README.md +334 -169
  4. package/SECURITY.md +150 -0
  5. package/dist/analyzers/performanceAnalyzer.d.ts.map +1 -1
  6. package/dist/analyzers/performanceAnalyzer.js +0 -3
  7. package/dist/analyzers/performanceAnalyzer.js.map +1 -1
  8. package/dist/analyzers/raceConditionAnalyzer.d.ts +39 -0
  9. package/dist/analyzers/raceConditionAnalyzer.d.ts.map +1 -0
  10. package/dist/analyzers/raceConditionAnalyzer.js +412 -0
  11. package/dist/analyzers/raceConditionAnalyzer.js.map +1 -0
  12. package/dist/analyzers/securityAnalyzer.d.ts +45 -0
  13. package/dist/analyzers/securityAnalyzer.d.ts.map +1 -0
  14. package/dist/analyzers/securityAnalyzer.js +457 -0
  15. package/dist/analyzers/securityAnalyzer.js.map +1 -0
  16. package/dist/core/mcpServerFactory.d.ts +20 -0
  17. package/dist/core/mcpServerFactory.d.ts.map +1 -0
  18. package/dist/core/mcpServerFactory.js +133 -0
  19. package/dist/core/mcpServerFactory.js.map +1 -0
  20. package/dist/core/prompts.d.ts +38 -0
  21. package/dist/core/prompts.d.ts.map +1 -0
  22. package/dist/core/prompts.js +158 -0
  23. package/dist/core/prompts.js.map +1 -0
  24. package/dist/core/resources.d.ts +47 -0
  25. package/dist/core/resources.d.ts.map +1 -0
  26. package/dist/core/resources.js +116 -0
  27. package/dist/core/resources.js.map +1 -0
  28. package/dist/core/subprocess.d.ts +66 -0
  29. package/dist/core/subprocess.d.ts.map +1 -0
  30. package/dist/core/subprocess.js +147 -0
  31. package/dist/core/subprocess.js.map +1 -0
  32. package/dist/core/syntaxValidator.d.ts +50 -0
  33. package/dist/core/syntaxValidator.d.ts.map +1 -0
  34. package/dist/core/syntaxValidator.js +296 -0
  35. package/dist/core/syntaxValidator.js.map +1 -0
  36. package/dist/core/toolRegistry.d.ts +30 -0
  37. package/dist/core/toolRegistry.d.ts.map +1 -0
  38. package/dist/core/toolRegistry.js +655 -0
  39. package/dist/core/toolRegistry.js.map +1 -0
  40. package/dist/index.d.ts +10 -0
  41. package/dist/index.d.ts.map +1 -1
  42. package/dist/index.js +17 -670
  43. package/dist/index.js.map +1 -1
  44. package/dist/llm/index.d.ts +54 -0
  45. package/dist/llm/index.d.ts.map +1 -0
  46. package/dist/llm/index.js +190 -0
  47. package/dist/llm/index.js.map +1 -0
  48. package/dist/platforms/android/index.d.ts +2 -2
  49. package/dist/platforms/android/index.d.ts.map +1 -1
  50. package/dist/platforms/android/index.js +189 -164
  51. package/dist/platforms/android/index.js.map +1 -1
  52. package/dist/platforms/flutter/index.d.ts +1 -1
  53. package/dist/platforms/flutter/index.d.ts.map +1 -1
  54. package/dist/platforms/flutter/index.js +133 -193
  55. package/dist/platforms/flutter/index.js.map +1 -1
  56. package/dist/platforms/ios/index.d.ts.map +1 -1
  57. package/dist/platforms/ios/index.js +101 -102
  58. package/dist/platforms/ios/index.js.map +1 -1
  59. package/dist/platforms/react-native/index.d.ts +1 -1
  60. package/dist/platforms/react-native/index.d.ts.map +1 -1
  61. package/dist/platforms/react-native/index.js +122 -240
  62. package/dist/platforms/react-native/index.js.map +1 -1
  63. package/dist/platforms/web/index.d.ts.map +1 -1
  64. package/dist/platforms/web/index.js +45 -28
  65. package/dist/platforms/web/index.js.map +1 -1
  66. package/dist/security.d.ts +81 -0
  67. package/dist/security.d.ts.map +1 -0
  68. package/dist/security.js +197 -0
  69. package/dist/security.js.map +1 -0
  70. package/dist/storage/index.d.ts +35 -0
  71. package/dist/storage/index.d.ts.map +1 -1
  72. package/dist/storage/index.js +33 -1
  73. package/dist/storage/index.js.map +1 -1
  74. package/dist/tools/automation/diagnoseAutoFix.d.ts +131 -0
  75. package/dist/tools/automation/diagnoseAutoFix.d.ts.map +1 -0
  76. package/dist/tools/automation/diagnoseAutoFix.js +366 -0
  77. package/dist/tools/automation/diagnoseAutoFix.js.map +1 -0
  78. package/dist/tools/automation/diagnoseProject.d.ts +57 -0
  79. package/dist/tools/automation/diagnoseProject.d.ts.map +1 -0
  80. package/dist/tools/automation/diagnoseProject.js +399 -0
  81. package/dist/tools/automation/diagnoseProject.js.map +1 -0
  82. package/dist/tools/automation/runIterativeFixLoop.d.ts +92 -0
  83. package/dist/tools/automation/runIterativeFixLoop.d.ts.map +1 -0
  84. package/dist/tools/automation/runIterativeFixLoop.js +474 -0
  85. package/dist/tools/automation/runIterativeFixLoop.js.map +1 -0
  86. package/dist/tools/fixing/applyFix.d.ts.map +1 -1
  87. package/dist/tools/fixing/applyFix.js +11 -37
  88. package/dist/tools/fixing/applyFix.js.map +1 -1
  89. package/dist/tools/fixing/llmFixSuggester.d.ts +29 -0
  90. package/dist/tools/fixing/llmFixSuggester.d.ts.map +1 -0
  91. package/dist/tools/fixing/llmFixSuggester.js +128 -0
  92. package/dist/tools/fixing/llmFixSuggester.js.map +1 -0
  93. package/package.json +29 -17
package/README.md CHANGED
@@ -1,219 +1,384 @@
1
- # Test Genie MCP
1
+ # test-genie-mcp
2
2
 
3
- AI-powered App Test Automation MCP Server
3
+ **Built for vibe coders: one command, get a prioritized list of what's actually broken about your project.**
4
4
 
5
- 테스트 자동화를 위한 MCP 서버입니다. 시나리오 생성부터 테스트 실행, 문제 검출, 수정 제안 적용까지 전체 테스트 파이프라인을 자동화합니다.
5
+ Self-healing test automation for iOS, Android, Flutter, React Native and Web apps as an MCP server.
6
6
 
7
- ## Features
7
+ [![npm version](https://img.shields.io/npm/v/test-genie-mcp.svg)](https://www.npmjs.com/package/test-genie-mcp)
8
+ [![CI](https://img.shields.io/github/actions/workflow/status/MUSE-CODE-SPACE/test-genie-mcp/ci.yml?branch=main)](https://github.com/MUSE-CODE-SPACE/test-genie-mcp/actions)
9
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
10
+ [![MCP](https://img.shields.io/badge/MCP-1.29-blue)](https://modelcontextprotocol.io)
8
11
 
9
- ### Phase 1: Analysis & Scenario Generation
10
- - **analyze_app_structure**: 앱 코드베이스 분석 (화면, 컴포넌트, API, 상태관리)
11
- - **generate_scenarios**: AI 기반 테스트 시나리오 자동 생성
12
- - **create_test_plan**: 테스트 계획 수립 및 스케줄링
12
+ > **v3.1.1 — vibe-check + honest auto-fix.** One MCP call, ~30 seconds: race conditions + security issues + memory leaks + logic errors + perf smells, prioritized. Stays on your machine, no telemetry. Pass `autoFix: true` for the small, safe mechanical fixes (weak-hash, simple `Math.random` assignment) — backup + syntax-validate + rollback-on-syntax-fail. For test-verified application of harder fixes, use [v3.0.0](#how-the-iterate-fix-loop-works)'s iterate-fix loop.
13
13
 
14
- ### Phase 2: Test Execution
15
- - **run_scenario_test**: 개별 시나리오 테스트 실행
16
- - **run_simulation**: 사용자 행동 시뮬레이션 (랜덤/패턴 기반)
17
- - **run_stress_test**: 스트레스/부하 테스트
14
+ ---
18
15
 
19
- ### Phase 3: Issue Detection
20
- - **detect_memory_leaks**: 메모리 릭 감지 (힙 분석, 순환 참조)
21
- - **detect_logic_errors**: 논리적 오류 검출 (레이스 컨디션, 상태 불일치)
16
+ ## Vibe coders quickstart
22
17
 
23
- ### Phase 4: Fix Suggestions & Application
24
- - **suggest_fixes**: AI 기반 수정 방안 제안
25
- - **confirm_fix**: 수정 사항 사용자 확인
26
- - **apply_fix**: 확인된 수정 사항 적용
27
- - **rollback_fix**: 적용된 수정 롤백
18
+ You don't read the docs. You open the project, talk to Claude, and want a verdict. Here it is:
28
19
 
29
- ### Phase 5: Full Automation
30
- - **run_full_automation**: 전체 파이프라인 자동 실행
31
- - **generate_report**: 상세 보고서 생성 (Markdown, HTML, JSON)
20
+ In Claude (with test-genie-mcp installed — [setup](#5-minute-quickstart)):
32
21
 
33
- ### Enhanced Analysis (v2.0)
34
- - **analyze_performance**: 성능 심층 분석 (렌더링, 연산, 네트워크, 번들)
35
- - **analyze_code_deep**: AST 기반 코드 분석 (복잡도, 훅, 의존성)
36
- - **generate_cicd_config**: CI/CD 설정 자동 생성 (GitHub Actions, Jenkins, GitLab CI)
22
+ ```
23
+ /vibe-check /Users/me/my-app
24
+ ```
37
25
 
38
- ## Supported Platforms
26
+ Claude calls `diagnose_project` under the hood. ~30 seconds later you see:
39
27
 
40
- | Platform | Language | Test Framework |
41
- |----------|----------|----------------|
42
- | iOS | Swift, Objective-C | XCTest, XCUITest |
43
- | Android | Kotlin, Java | Espresso, UI Automator |
44
- | Flutter | Dart | flutter_test |
45
- | React Native | TypeScript, JavaScript | Detox, Jest |
46
- | Web | TypeScript, JavaScript | Playwright, Puppeteer |
28
+ ```text
29
+ # vibe-check report
47
30
 
48
- ## Installation
31
+ - Project: /Users/me/my-app
32
+ - Platform: web
33
+ - Findings: 11 total — 4 critical, 4 high, 1 medium, 1 low
34
+ - Estimated fix time: ~85 min
49
35
 
50
- ```bash
51
- npm install -g test-genie-mcp
36
+ ## Top 5 issues
37
+
38
+ ### 1. [CRIT] Hardcoded AWS access key id found in source
39
+ - File: `server.js:7`
40
+ - Category: security / secret (CWE-798)
41
+ - Confidence: 95%
42
+ - Fix: Move the value to an env var, gitignore the config, rotate the leaked key.
43
+
44
+ ### 2. [CRIT] SQL string built by concatenating user input
45
+ - File: `server.js:21`
46
+ - Category: security / injection (CWE-89)
47
+ - Fix: Use parameterized queries (`db.query("... WHERE id = ?", [id])`).
48
+
49
+ ### 3. [HIGH] useState setter called after await without mount guard
50
+ - File: `UserProfile.tsx:16`
51
+ - Category: race-condition / react-setstate-after-await (CWE-362)
52
+ - Confidence: 78%
53
+ - Fix: Use AbortController and check signal.aborted before calling setters.
54
+
55
+ … (top 5 shown — full list at output: "detailed")
56
+
57
+ ## Next steps
58
+ 1. Address the critical / high findings above.
59
+ 2. Re-run diagnose_project after fixing to confirm convergence.
60
+ 3. Use run_iterative_fix_loop for test-driven verification of each fix.
52
61
  ```
53
62
 
54
- Or install from source:
63
+ If any finding is `autoFixable: true` and is at `high`/`critical` severity, the `diagnose_project` call accepts `autoFix: true` to apply the mechanical replacement directly (with backup + syntax validation — see [SAFETY.md](SAFETY.md) for the exact guards). The v3.1.1 honest scope is narrow: weak hash (`createHash('md5'|'sha1')` → `createHash('sha256')`) and standalone `Math.random()` in security-sensitive files. For broader/structural fixes (race conditions, eval, exec injection) run `run_iterative_fix_loop` separately — it re-runs tests and auto-rolls-back on regression.
55
64
 
56
- ```bash
57
- git clone https://github.com/MUSE-CODE-SPACE/test-genie-mcp.git
58
- cd test-genie-mcp
59
- npm install
60
- npm run build
65
+ ---
66
+
67
+ ## Why test-genie?
68
+
69
+ The bottleneck in mobile + cross-platform test automation isn't writing tests — it's the loop *between* a failing test and a passing test. test-genie closes that loop:
70
+
71
+ ```
72
+ failing test → analyzer flags issue → fix proposed → dry-run + syntax check →
73
+ applied with backup → affected tests re-run → regression check → loop or stop
61
74
  ```
62
75
 
63
- ## Usage with Claude Desktop
76
+ This full loop is the `run_iterative_fix_loop` tool. The `diagnose_project autoFix: true` path in v3.1.1 covers a strict subset — backup + dry-run + syntax-validate + apply, **without** re-running tests (so no test-regression rollback in that path). Use the right tool for the job — and see [SAFETY.md](SAFETY.md) for the exact guards on each.
64
77
 
65
- Add to your Claude Desktop config (`~/.config/claude/claude_desktop_config.json`):
78
+ Other tools (Detox, Maestro, Playwright, `xcodebuild test`) run tests. test-genie **runs tests *and* drives the fix until the bar is met or it can no longer make progress** — without you scrubbing through stack traces.
66
79
 
67
- ```json
80
+ ---
81
+
82
+ ## 5-minute Quickstart
83
+
84
+ ```bash
85
+ # 1. Install
86
+ npm install -g test-genie-mcp
87
+
88
+ # 2. Add to Claude Desktop config (~/.config/claude/claude_desktop_config.json)
68
89
  {
69
90
  "mcpServers": {
70
91
  "test-genie": {
71
92
  "command": "npx",
72
- "args": ["test-genie-mcp"]
93
+ "args": ["test-genie-mcp"],
94
+ "env": {
95
+ "TEST_GENIE_ALLOWED_ROOT": "/path/to/your/project"
96
+ }
73
97
  }
74
98
  }
75
99
  }
100
+
101
+ # 3. Restart Claude Desktop. From a chat:
102
+ # "Run the iterate-fix loop on /Users/me/my-rn-app with autoApply=false"
76
103
  ```
77
104
 
78
- ## Example Usage
105
+ Expected output (truncated):
79
106
 
80
- ### Full Automation
81
- ```
82
- User: "자동화 테스트해줘"
83
-
84
- Claude will:
85
- 1. Analyze your app structure
86
- 2. Generate test scenarios
87
- 3. Execute tests
88
- 4. Detect issues (memory leaks, logic errors)
89
- 5. Suggest fixes
90
- 6. Wait for your confirmation
91
- 7. Apply approved fixes
92
- 8. Generate report
93
107
  ```
108
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
109
+ Iterative fix loop f8b3… — PAUSED-FOR-CONFIRMATION
110
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
94
111
 
95
- ### Step-by-Step
112
+ Iterations completed: 1
113
+ Fixes applied: 0
114
+ Regressions rolled back: 0
115
+ Final tests: 7/10 passing (3 failing)
96
116
 
117
+ Pending confirmations (3):
118
+ - 71fbe…: Fix: useEffect missing cleanup for setInterval (confidence: 85)
119
+ - 92ad1…: Fix: Force-unwrap on possibly-undefined name (confidence: 85)
120
+ - …
121
+
122
+ Resume token: f8b3…
97
123
  ```
98
- # Analyze app
99
- analyze_app_structure(projectPath: "/path/to/app")
100
124
 
101
- # Generate scenarios
102
- generate_scenarios(projectPath: "/path/to/app", testTypes: ["e2e", "unit"])
125
+ Re-call with `autoApply: true` (or `resumeToken: "f8b3…"`) to actually patch the files.
103
126
 
104
- # Detect memory leaks
105
- detect_memory_leaks(projectPath: "/path/to/app")
127
+ ---
106
128
 
107
- # Get fix suggestions
108
- suggest_fixes(projectPath: "/path/to/app")
129
+ ## Real use cases
109
130
 
110
- # Confirm and apply fix
111
- confirm_fix(fixId: "xxx", action: "approve")
112
- apply_fix(fixId: "xxx")
113
- ```
131
+ > The flows below describe the **`run_iterative_fix_loop` path** (v3.0
132
+ > headline) full detect → propose → dry-run → apply-with-backup →
133
+ > re-run-tests → rollback-on-regression. The `diagnose_project autoFix`
134
+ > path in v3.1.1 is the narrower mechanical-replacement-only path; see
135
+ > [SAFETY.md](SAFETY.md) §4 for what that one actually touches.
136
+
137
+ ### 1. React Native memory-leak self-healing
138
+
139
+ A team adds `setInterval(...)` in a `useEffect` and forgets cleanup. test-genie's `detect_memory_leaks` flags it, `suggest_fixes` proposes `return () => clearInterval(id)` (`src/tools/fixing/suggestFixes.ts:169-179`), the loop dry-runs the patch through the TS compiler, applies with backup, re-runs only the affected snapshot test, confirms 100% pass, stops. **Before:** 1 failing snapshot. **After:** 0 failing, 1 fix applied, 1 backup at `.test-genie-backups/`.
140
+
141
+ ### 2. Flutter widget `dispose()` automation
142
+
143
+ `AnimationController` left undisposed. test-genie sees the missing `dispose()` override, generates a Dart `@override dispose() { controller.dispose(); super.dispose(); }` block (`suggestFixes.ts:214-217`), runs `dart analyze` on the patched file, applies, re-runs `flutter test`, converges.
114
144
 
115
- ## Fix Confirmation Workflow
145
+ ### 3. iOS retain-cycle (closure capture)
116
146
 
117
- When issues are detected, Test Genie will:
147
+ `self.timer = Timer.scheduledTimer(...) { _ in self.tick() }` — rule-based detector flags closure self-capture, fixer rewrites to `[weak self] _ in guard let self = self else { return }; self.tick()` (`suggestFixes.ts:239-242`). If `swiftc` is on PATH the syntax check is real; otherwise test-genie reports "downgraded validation" so you know.
118
148
 
119
- 1. **Suggest Fixes**: Generate AI-powered fix suggestions with confidence scores
120
- 2. **Show Diff**: Display exactly what will change
121
- 3. **Await Confirmation**: Wait for user approval
122
- 4. **Apply Fix**: Only apply after explicit approval
123
- 5. **Backup**: Automatically create backups before applying
149
+ ---
124
150
 
125
- Example confirmation prompt:
151
+ ## How the iterate-fix loop works
126
152
 
127
153
  ```
128
- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
129
- 🔧 Fix Suggestion #abc123
130
- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
131
-
132
- 📁 File: src/screens/HomeScreen.tsx:45
133
- 📝 Title: Fix: useEffect missing cleanup
134
- 🎯 Confidence: 90%
135
-
136
- 📄 Current Code:
137
- ┌──────────────────────────────────────────
138
- useEffect(() => {
139
- │ const subscription = api.subscribe();
140
- }, []);
141
- └──────────────────────────────────────────
142
-
143
- ✨ Suggested Fix:
144
- ┌──────────────────────────────────────────
145
- │ useEffect(() => {
146
- const subscription = api.subscribe();
147
- │ return () => subscription.unsubscribe();
148
- }, []);
149
- └──────────────────────────────────────────
150
-
151
- Actions: [✅ Approve] [❌ Reject] [✏️ Modify]
152
- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
154
+ ┌────────────────────┐
155
+ │ collect tests │ (run_scenario_test / supplied list)
156
+ └─────────┬──────────┘
157
+
158
+ pass-rate threshold? ── yes ──▶ SUCCESS
159
+ no
160
+
161
+ ┌────────────────────┐
162
+ │ detect issues │ memory + logic analyzers
163
+ └─────────┬──────────┘
164
+
165
+ ┌────────────────────┐
166
+ suggest fixes │ rule-based (default) → LLM (hybrid, optional)
167
+ └─────────┬──────────┘
168
+
169
+ ┌────────────────────┐
170
+ │ dry-run + syntax │ TS compiler API / platform compiler / brace check
171
+ └─────────┬──────────┘
172
+
173
+ ┌────────────────────┐
174
+ apply with backup │ per-file `.test-genie-backups/`
175
+ └─────────┬──────────┘
176
+
177
+ ┌────────────────────┐
178
+ │ re-run tests │ regression? yes → auto-rollback
179
+ └─────────┬──────────┘
180
+
181
+
182
+ loop (≤ maxIterations, ≤ totalTimeout)
153
183
  ```
154
184
 
155
- ## Tools Reference
156
-
157
- | Tool | Description |
158
- |------|-------------|
159
- | `analyze_app_structure` | Analyze codebase structure |
160
- | `generate_scenarios` | Generate test scenarios |
161
- | `create_test_plan` | Create test plan |
162
- | `run_scenario_test` | Run single scenario |
163
- | `run_simulation` | Run user simulation |
164
- | `run_stress_test` | Run stress test |
165
- | `detect_memory_leaks` | Detect memory leaks |
166
- | `detect_logic_errors` | Detect logic errors |
167
- | `suggest_fixes` | Generate fix suggestions |
168
- | `confirm_fix` | Confirm/reject fix |
169
- | `apply_fix` | Apply confirmed fix |
170
- | `rollback_fix` | Rollback applied fix |
171
- | `run_full_automation` | Run full automation pipeline |
172
- | `generate_report` | Generate test report |
173
- | `get_pending_fixes` | Get pending confirmations |
174
- | `get_test_history` | Get test history |
175
- | `analyze_performance` | Deep performance analysis |
176
- | `analyze_code_deep` | AST-based code analysis |
177
- | `generate_cicd_config` | Generate CI/CD configuration |
178
-
179
- ## Platform-Specific Features
180
-
181
- ### iOS
182
- - XCTest, XCUITest integration
183
- - Instruments profiling (Time Profiler, Allocations, Leaks)
184
- - Simulator management
185
- - Screenshot & video recording
186
-
187
- ### Android
188
- - Gradle test integration
189
- - Espresso, UI Automator support
190
- - Android Profiler (CPU, Memory, Network)
191
- - LeakCanary integration
192
- - ADB device management
193
-
194
- ### Flutter
195
- - flutter_test, integration_test
196
- - Golden tests (snapshot testing)
197
- - Performance profiling
198
- - Memory analysis
199
-
200
- ### React Native
201
- - Jest, React Native Testing Library
202
- - Detox E2E testing
203
- - Performance monitoring
204
- - Memory leak detection
205
-
206
- ### Web
207
- - Playwright, Cypress support
208
- - Lighthouse performance audits
209
- - Accessibility testing (axe-core)
210
- - Visual regression testing
211
- - Load testing (K6 integration)
185
+ See **[docs/ITERATE_FIX_LOOP.md](docs/ITERATE_FIX_LOOP.md)** for a sequence diagram and the full safety-guard list.
186
+
187
+ ---
188
+
189
+ ## Tools (23)
190
+
191
+ | # | Tool | Mode |
192
+ |---|------|------|
193
+ | 1 | `analyze_app_structure` | real |
194
+ | 2 | `generate_scenarios` | real |
195
+ | 3 | `create_test_plan` | real |
196
+ | 4 | `run_scenario_test` | hybrid |
197
+ | 5 | `run_simulation` | simulated |
198
+ | 6 | `run_stress_test` | hybrid |
199
+ | 7 | `detect_memory_leaks` | real |
200
+ | 8 | `detect_logic_errors` | real |
201
+ | 9 | `suggest_fixes` | real |
202
+ | 10 | `confirm_fix` | real |
203
+ | 11 | `apply_fix` | real |
204
+ | 12 | `rollback_fix` | real |
205
+ | 13 | `run_full_automation` | hybrid |
206
+ | 14 | `run_iterative_fix_loop` (v3.0 headline) | hybrid |
207
+ | 15 | `generate_report` | real |
208
+ | 16 | `get_pending_fixes` | real |
209
+ | 17 | `get_test_history` | real |
210
+ | 18 | `analyze_performance` | real |
211
+ | 19 | `analyze_code_deep` | real |
212
+ | 20 | `generate_cicd_config` | real |
213
+ | 21 | **`diagnose_project`** (v3.1 headline vibe-check) | real |
214
+ | 22 | `detect_race_conditions` | real |
215
+ | 23 | `detect_security_issues` | real |
216
+
217
+ `mode` legend in **[docs/SIMULATION_VS_REAL.md](docs/SIMULATION_VS_REAL.md)**.
218
+
219
+ Plus 4 resources (`test-genie://iteration-logs`, `…/test-history/{path}`, `…/iteration-logs/{loopId}`, `…/applied-fixes/{path}`) and 3 prompts (`full-test-pipeline`, `diagnose-failure`, `vibe-check`).
220
+
221
+ ---
222
+
223
+ ## What vibe-check catches
224
+
225
+ Race conditions (`detect_race_conditions` / `diagnose_project`):
226
+
227
+ | Pattern | Language | Severity | Auto-fixable (v3.1.1) |
228
+ |---|---|---|---|
229
+ | `useState` setter called after `await` without mount guard | TS/JS/React | high | no (structural) |
230
+ | `useEffect` with async fetch, no AbortController/cleanup | TS/JS/React | high | no (structural) |
231
+ | `arr.forEach(async ...)` (silent fire-and-forget) | TS/JS | medium | no (ordering-sensitive) |
232
+ | Adjacent fetches without `Promise.all` / sequencing | TS/JS | medium | no |
233
+ | TOCTOU: `existsSync` then `readFileSync` without lock | TS/JS Node | medium | no |
234
+ | Non-atomic counter increment in async context | TS/JS | low | no |
235
+ | `@Published` mutation outside `@MainActor` | Swift | medium | no |
236
+ | Concurrent `DispatchQueue` writes without `.barrier` | Swift | medium | no |
237
+ | `MutableStateFlow` mutated off `Dispatchers.Main` | Kotlin | medium | no |
238
+ | `Flow` collected without `flowOn` | Kotlin | low | no |
239
+ | Goroutine + shared map without `sync.Mutex` | Go | high | no |
240
+
241
+ > v3.1.1 honesty audit: `useEffect-no-abort` and `forEach-await` were
242
+ > previously advertised as auto-fixable. They are not — wrapping with
243
+ > `AbortController` or rewriting to `Promise.all(arr.map(...))` changes
244
+ > behavior we can't verify statically. They are now report-only. See
245
+ > [SAFETY.md](SAFETY.md).
246
+
247
+ Security (`detect_security_issues` / `diagnose_project`):
248
+
249
+ | Pattern | Severity | CWE | Auto-fixable (v3.1.1) |
250
+ |---|---|---|---|
251
+ | Hardcoded AWS / Stripe / GitHub / Google / Slack token | critical / high | CWE-798 | no (rotate) |
252
+ | Hardcoded JWT secret literal | high | CWE-798 | no |
253
+ | API token in URL query string | high | CWE-200 | no |
254
+ | `.env` file present but not gitignored | high | CWE-538 | no (rotation must follow) |
255
+ | SQL string concat with `req.params` / `req.body` | critical | CWE-89 | no |
256
+ | `innerHTML` / `dangerouslySetInnerHTML` with dynamic value | high | CWE-79 | no |
257
+ | `eval()` / `new Function()` with non-literal | critical | CWE-95 | no |
258
+ | `Math.random()` in security-sensitive file, **standalone assignment** | high | CWE-338 | **yes** (`crypto.randomInt`) |
259
+ | `Math.random()` mixed into arithmetic | high | CWE-338 | no (semantic) |
260
+ | `createHash('md5'\|'sha1')` in security-keyword file | high | CWE-327 | **yes** (`'sha256'`) |
261
+ | `createHash('md5'\|'sha1')` elsewhere | medium | CWE-327 | no (below severity floor) |
262
+ | `child_process.exec` with user-input template literal | critical | CWE-78 | no |
263
+ | `fetch(req.query.url)` (SSRF) | high | CWE-918 | no |
264
+ | CORS `*` origin + `Allow-Credentials: true` | high | CWE-942 | no |
265
+ | Cookie set without `httpOnly` / `secure` / `sameSite` | low | CWE-1004 | no |
266
+ | `yaml.load` without safe schema | medium | CWE-502 | no |
267
+
268
+ > v3.1.1 honesty audit: `.env`/`Math.random` (general)/`yaml.load` were
269
+ > previously advertised as auto-fixable. They were either too risky to
270
+ > rewrite blindly or no strategy shipped — flipped to report-only. See
271
+ > [SAFETY.md](SAFETY.md) §5.
272
+
273
+ ---
274
+
275
+ ## What vibe-check misses (honest list)
276
+
277
+ This is a "catch the obvious stuff in 30s" filter, not Snyk / Semgrep / a full SAST tool. We don't catch:
278
+
279
+ - **Cross-file data-flow.** If user input flows through three files before reaching a `db.query`, the regex won't connect the dots. A real SAST traces taint across the call graph. Roadmap: ts-morph reference walking for top-N entry points.
280
+ - **Vulnerable transitive deps.** We don't query npm advisories — that's `npm audit`'s job, and bundling a stale advisory list would lie. Run `npm audit --json` in parallel if you want dep-CVE coverage.
281
+ - **Race conditions across processes.** We catch in-process JS / Swift / Kotlin / Go races. Distributed races (lock ordering across services, DB transactions) need different tooling.
282
+ - **Type-correct but logic-broken code.** The analyzer is syntactic, not semantic. A `Math.random()` named `getNonce` won't fool us; a properly-named `crypto.randomBytes` used with a tiny entropy budget will.
283
+ - **Custom secret formats.** Internal company tokens with unique prefixes need a regex you can add to `securityAnalyzer.SECRET_PATTERNS`. PR welcome.
284
+ - **Real-time / dynamic issues.** Memory leaks under load, network timeouts, slow renders mid-interaction — those need `run_stress_test` / `run_simulation`, not static analysis.
285
+
286
+ If you want deeper coverage on top of vibe-check: feed the findings into `run_iterative_fix_loop` for test-verified application, or escalate to Snyk / Semgrep / GitHub Advanced Security for compliance use cases.
287
+
288
+ ---
289
+
290
+ ## vibe-check vs alternatives
291
+
292
+ | | vibe-check (test-genie) | Snyk | Semgrep | GitHub Advanced Security |
293
+ |------------------------|-------------------------|------|---------|--------------------------|
294
+ | Runs locally | yes | hybrid (cloud) | yes | no (cloud) |
295
+ | Telemetry-free | yes (zero network calls) | no | partial | no |
296
+ | Fix loop integration | yes (`run_iterative_fix_loop`) | no | no | no |
297
+ | Race-condition detection | yes (JS/Swift/Kotlin/Go) | no | partial | partial |
298
+ | Cross-file taint flow | no (roadmap) | yes | yes | yes |
299
+ | Setup time | none (already installed if test-genie is installed) | account + auth | install + ruleset | repo-level enable |
212
300
 
213
- ## License
301
+ If your goal is "before I commit, what's broken?", vibe-check wins on latency. If your goal is "compliance + supply chain audit", use the dedicated tools.
302
+
303
+ ---
304
+
305
+ ## When NOT to use test-genie
306
+
307
+ - **Production-gate test runs.** test-genie is built for the *development* feedback loop. For shipping decisions, use a proper CI that you control end-to-end.
308
+ - **Code your team must hand-review every line of.** The loop's job is to *propose and apply* fixes; if every fix needs a human eye, leave `autoApply: false` (the default) and use it as a fix-proposal generator only.
309
+ - **No backup / no version control situations.** test-genie's auto-rollback is best-effort and requires the per-file backup to exist. Always run inside a git working tree.
310
+
311
+ ---
312
+
313
+ ## Comparison
314
+
315
+ | | test-genie | Detox | Maestro | xcodebuild test |
316
+ |---|---|---|---|---|
317
+ | Runs E2E / unit tests | ✅ (via Jest/Detox/etc.) | ✅ | ✅ | ✅ |
318
+ | Detects code issues | ✅ rule + LLM | ❌ | ❌ | ❌ |
319
+ | **Iterative fix loop** | **✅** (`run_iterative_fix_loop`) | ❌ | ❌ | ❌ |
320
+ | Auto-rollback on test regression | ✅ inside `run_iterative_fix_loop` only | ❌ | ❌ | ❌ |
321
+ | Auto-rollback on syntax failure | ✅ all apply paths | ❌ | ❌ | ❌ |
322
+ | MCP-native (talks to Claude / agents) | ✅ | ❌ | ❌ | ❌ |
323
+ | Multi-platform | iOS+Android+Web+Flutter+RN | iOS+Android | iOS+Android | iOS only |
324
+
325
+ > Scope note: `diagnose_project autoFix: true` rolls back on syntax-validate
326
+ > failure (`applyFix.ts:185-202`) but does **not** re-run tests, so it
327
+ > cannot detect test regressions. For test-driven rollback use
328
+ > `run_iterative_fix_loop`. See [SAFETY.md](SAFETY.md) §2.4.
329
+
330
+ test-genie *uses* tools like Jest, Detox, and `xcodebuild test` under the hood — it sits at the orchestration layer, not the test-runner layer.
331
+
332
+ ---
333
+
334
+ ## Known limitations
335
+
336
+ - **Platform syntax check downgrade.** For Swift/Kotlin/Java/Dart we try the platform compiler in `-typecheck` mode. If the compiler isn't on PATH, we fall back to brace-balance validation and surface `downgraded: true` in the result. Install `swiftc` / `kotlinc` / `javac` / `dart` for real validation.
337
+ - **LLM is optional and gated.** `strategy: 'hybrid'` only kicks LLM in when rule-based confidence is below threshold. Without an API key the loop is rule-based-only — no failure.
338
+ - **Storage is per-machine.** Test history / iteration logs live under `$TEST_GENIE_STORAGE_DIR` (defaults to `~/.test-genie-mcp`). Not synced across machines.
339
+ - **Simulated mode is "simulation," not magic.** `run_simulation` returns *plausible* anomalies, not real ones. Use `run_scenario_test` (hybrid) for real-device runs.
340
+
341
+ ---
214
342
 
215
- MIT
343
+ ## Configuration
216
344
 
217
- ## Author
345
+ | Env var | Default | Purpose |
346
+ |---|---|---|
347
+ | `TEST_GENIE_ALLOWED_ROOT` | `cwd` | Capability-based path safety — server refuses to read/write outside this root. |
348
+ | `TEST_GENIE_STORAGE_DIR` | `~/.test-genie-mcp` | Where scenarios / results / iteration logs live. |
349
+ | `TEST_GENIE_LLM_PROVIDER` | auto-detect | `anthropic` / `openai` / `none`. |
350
+ | `ANTHROPIC_API_KEY` | — | Used when provider = `anthropic`. |
351
+ | `OPENAI_API_KEY` | — | Used when provider = `openai`. |
352
+ | `TEST_GENIE_ANTHROPIC_MODEL` | `claude-haiku-4-5` | Override Anthropic model. |
353
+ | `TEST_GENIE_OPENAI_MODEL` | `gpt-4o-mini` | Override OpenAI model. |
354
+
355
+ ---
356
+
357
+ ## Migrating from v2.x
358
+
359
+ - `run_full_automation` still works. The `confirmMode` / `autoFix` options are kept for compatibility but **`autoApply: boolean` is the new way** — `autoApply: true` is equivalent to `confirmMode: 'auto'`.
360
+ - Subprocess hardening means platform tools now reject scheme / device / package-name arguments that contain shell metacharacters. If your CI was passing weird-looking values, sanitize them first.
361
+ - See **[CHANGELOG.md](CHANGELOG.md)** for the full breaking-change list + migration recipes.
362
+
363
+ ---
364
+
365
+ ## Roadmap
366
+
367
+ - LLM-based fix-proposal **voting** (multiple proposals → pick the best by syntax + retest delta)
368
+ - Multi-repo sync (run the loop across N repos in parallel from one MCP call)
369
+ - A "watch mode" that runs the loop on file save
370
+ - Better Detox / Maestro artifact ingestion (link videos into iteration logs)
371
+
372
+ ---
373
+
374
+ ## Contributing
375
+
376
+ Issues, PRs, and ideas welcome — see **[CONTRIBUTING.md](CONTRIBUTING.md)** (TODO). Code lives under `src/`, tests under `tests/`. Run `npm test` before sending a PR.
377
+
378
+ ## Maintainer
379
+
380
+ [@MUSE-CODE-SPACE](https://github.com/MUSE-CODE-SPACE) — Yoonkyoung Gong.
381
+
382
+ ## License
218
383
 
219
- Yoonkyoung Gong
384
+ MIT — see [LICENSE](LICENSE).