@kakarot-ci/core 0.7.0 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,2 +1,457 @@
1
- # kakarot-ci
2
- The core package for kakarot-ci
1
+ <div align="center">
2
+ <img src="assets/logo.svg" alt="Kakarot CI Logo" width="200">
3
+
4
+ # Kakarot CI
5
+
6
+ [![npm version](https://img.shields.io/npm/v/@kakarot-ci/core.svg)](https://www.npmjs.com/package/@kakarot-ci/core)
7
+ [![Node.js version](https://img.shields.io/node/v/@kakarot-ci/core.svg)](https://nodejs.org/)
8
+ </div>
9
+
10
+ > AI-powered unit test generation for TypeScript and JavaScript
11
+
12
+ > **⚠️ Beta Software**: This project is currently in beta. While functional, expect occasional bugs and breaking changes as we iterate and improve. Please report issues and provide feedback!
13
+
14
+ Kakarot CI automatically generates comprehensive unit tests using AI. While optimized for pull request workflows, it can be used in various scenarios: analyzing PR changes, generating tests for specific files, or creating test suites for entire codebases. It analyzes code, generates test files, runs them, and can automatically commit results back to your repository.
15
+
16
+ ## Features
17
+
18
+ - 🤖 **AI-Powered Test Generation**: Uses LLMs (OpenAI, Anthropic, Google) to generate comprehensive unit tests
19
+ - 🔍 **Smart Code Analysis**: Analyzes AST to extract functions and understand code structure
20
+ - 🎯 **Targeted Testing**: Generates tests for specific functions, files, or entire codebases
21
+ - 🔄 **Auto-Fix Loop**: Automatically fixes failing tests with multiple retry attempts
22
+ - 📊 **Coverage Reports**: Optional test coverage analysis and summaries
23
+ - 🚀 **GitHub Integration**: Seamlessly integrates with GitHub Actions and PR workflows (optional)
24
+ - ⚙️ **Flexible Configuration**: Supports Jest and Vitest, configurable test locations and patterns
25
+ - 📝 **PR Comments**: Automatically posts test generation summaries to pull requests
26
+
27
+ ## Installation
28
+
29
+ ```bash
30
+ npm install --save-dev @kakarot-ci/core
31
+ ```
32
+
33
+ ## Usage
34
+
35
+ ### Pull Request Workflow (Primary Use Case)
36
+
37
+ The simplest way to use Kakarot CI is via the command-line interface for pull requests:
38
+
39
+ ```bash
40
+ npx kakarot-ci --pr 123 --owner myorg --repo myrepo --token ghp_xxxxx
41
+ ```
42
+
43
+ Or in a GitHub Actions workflow:
44
+
45
+ ```yaml
46
+ - name: Generate Tests
47
+ run: npx kakarot-ci
48
+ env:
49
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
50
+ KAKAROT_API_KEY: ${{ secrets.KAKAROT_API_KEY }}
51
+ ```
52
+
53
+ The CLI automatically detects:
54
+ - PR number from `GITHUB_EVENT_PATH` (GitHub Actions)
55
+ - Repository owner/repo from `GITHUB_REPOSITORY` or git remote
56
+ - GitHub token from `GITHUB_TOKEN` environment variable
57
+
58
+ ### Programmatic API
59
+
60
+ #### Pull Request Processing
61
+
62
+ ```typescript
63
+ import { runPullRequest } from '@kakarot-ci/core';
64
+
65
+ const summary = await runPullRequest({
66
+ prNumber: 123,
67
+ owner: 'myorg',
68
+ repo: 'myrepo',
69
+ githubToken: 'ghp_xxxxx',
70
+ });
71
+
72
+ console.log(`Generated ${summary.testsGenerated} tests`);
73
+ ```
74
+
75
+ #### Custom Workflows
76
+
77
+ You can also use the lower-level APIs for custom workflows:
78
+
79
+ ```typescript
80
+ import {
81
+ TestGenerator,
82
+ extractTestTargets,
83
+ analyzeFile,
84
+ writeTestFiles
85
+ } from '@kakarot-ci/core';
86
+
87
+ // Generate tests for specific files
88
+ const targets = await analyzeFile('src/utils.ts', 'main', githubClient, config);
89
+ const generator = new TestGenerator({ apiKey, provider: 'openai' });
90
+
91
+ for (const target of targets) {
92
+ const test = await generator.generateTest({
93
+ target,
94
+ framework: 'vitest',
95
+ });
96
+ // Write test file...
97
+ }
98
+
99
+ // Or extract targets from file diffs
100
+ const prFiles = [...]; // Your file changes
101
+ const targets = await extractTestTargets(prFiles, githubClient, 'sha', config);
102
+ ```
103
+
104
+ ## Configuration
105
+
106
+ Kakarot CI can be configured via:
107
+
108
+ 1. **Config file**: `kakarot.config.js`, `.kakarot-ci.config.js`, or `.kakarot-ci.config.json`
109
+ 2. **package.json**: Add a `kakarotCi` field
110
+ 3. **Environment variables**: `KAKAROT_API_KEY`, `GITHUB_TOKEN`, etc.
111
+
112
+ **Note**: TypeScript config files (`.ts`) are not supported in the compiled package. Use JavaScript (`.js`) or JSON (`.json`) config files instead.
113
+
114
+ ### Configuration Options
115
+
116
+ #### Required
117
+
118
+ - **`apiKey`** (string, required)
119
+ - Your LLM provider API key
120
+ - Can also be set via `KAKAROT_API_KEY` environment variable
121
+ - Example: `"sk-xxxxx"` (OpenAI) or `"sk-ant-xxxxx"` (Anthropic)
122
+
123
+ #### LLM Provider Settings
124
+
125
+ - **`provider`** (string, optional, default: `"openai"`)
126
+ - LLM provider to use: `"openai"`, `"anthropic"`, or `"google"`
127
+ - Can also be set via `PROVIDER` environment variable
128
+ - Example: `"anthropic"`
129
+
130
+ - **`model`** (string, optional)
131
+ - Specific model to use (provider-specific)
132
+ - Can also be set via `MODEL` environment variable
133
+ - Defaults to provider's recommended model
134
+ - Examples: `"gpt-4"`, `"claude-3-opus-20240229"`, `"gemini-pro"`
135
+
136
+ - **`maxTokens`** (number, optional, default: provider default)
137
+ - Maximum tokens in response (1-100000)
138
+ - Example: `4000`
139
+
140
+ - **`temperature`** (number, optional, default: `0.2`)
141
+ - Temperature for test generation (0-2)
142
+ - Lower values = more consistent, higher = more creative
143
+ - Example: `0.2`
144
+
145
+ - **`fixTemperature`** (number, optional, default: `0.2`)
146
+ - Temperature for test fixing attempts (0-2)
147
+ - Example: `0.3`
148
+
149
+ - **`maxFixAttempts`** (number, optional, default: `3`)
150
+ - Maximum number of attempts to fix failing tests (0-5)
151
+ - Example: `5`
152
+
153
+ #### Test Framework
154
+
155
+ - **`framework`** (string, required)
156
+ - Test framework: `"jest"` or `"vitest"`
157
+ - Example: `"vitest"`
158
+
159
+ #### Test File Organization
160
+
161
+ - **`testLocation`** (string, optional, default: `"separate"`)
162
+ - Where to place test files: `"separate"` or `"co-located"`
163
+ - `"separate"`: Tests in a dedicated test directory
164
+ - `"co-located"`: Tests next to source files
165
+ - Example: `"co-located"`
166
+
167
+ - **`testDirectory`** (string, optional, default: `"__tests__"`)
168
+ - Directory name for test files (when `testLocation` is `"separate"`)
169
+ - Example: `"tests"` or `"__tests__"`
170
+
171
+ - **`testFilePattern`** (string, optional, default: `"*.test.ts"`)
172
+ - Glob pattern for test file names
173
+ - Example: `"*.spec.ts"` or `"*.test.js"`
174
+
175
+ #### File Filtering
176
+
177
+ - **`includePatterns`** (string[], optional, default: `["**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx"]`)
178
+ - Glob patterns for files to include when scanning for changes
179
+ - Example: `["src/**/*.ts", "lib/**/*.ts"]`
180
+
181
+ - **`excludePatterns`** (string[], optional, default: `["**/*.test.ts", "**/*.spec.ts", "**/*.test.js", "**/*.spec.js", "**/node_modules/**"]`)
182
+ - Glob patterns for files to exclude
183
+ - Example: `["**/*.test.ts", "**/vendor/**"]`
184
+
185
+ #### Limits
186
+
187
+ - **`maxTestsPerPR`** (number, optional, default: `50`)
188
+ - Maximum number of test targets to process per PR
189
+ - Prevents excessive API usage on large PRs
190
+ - Example: `100`
191
+
192
+ #### GitHub Integration
193
+
194
+ - **`githubToken`** (string, optional)
195
+ - GitHub personal access token
196
+ - Can also be set via `GITHUB_TOKEN` environment variable
197
+ - Required for GitHub operations (commits, PR comments)
198
+ - Example: `"ghp_xxxxx"`
199
+
200
+ - **`githubOwner`** (string, optional)
201
+ - Repository owner (can be auto-detected from git remote)
202
+ - Example: `"myorg"`
203
+
204
+ - **`githubRepo`** (string, optional)
205
+ - Repository name (can be auto-detected from git remote)
206
+ - Example: `"myrepo"`
207
+
208
+ #### Commit Strategy
209
+
210
+ - **`enableAutoCommit`** (boolean, optional, default: `true`)
211
+ - Automatically commit generated tests
212
+ - Example: `false` (to review tests before committing)
213
+
214
+ - **`commitStrategy`** (string, optional, default: `"direct"`)
215
+ - How to commit tests: `"direct"` or `"branch-pr"`
216
+ - `"direct"`: Commit directly to PR branch
217
+ - `"branch-pr"`: Create a new branch and open a PR
218
+ - Example: `"branch-pr"`
219
+
220
+ - **`enablePRComments`** (boolean, optional, default: `true`)
221
+ - Post test generation summary as PR comment
222
+ - Example: `true`
223
+
224
+ #### Coverage
225
+
226
+ - **`enableCoverage`** (boolean, optional, default: `false`)
227
+ - Enable test coverage collection and reporting
228
+ - Example: `true`
229
+
230
+ #### Debugging
231
+
232
+ - **`debug`** (boolean, optional, default: `false`)
233
+ - Enable debug logging
234
+ - Can also be set via `KAKAROT_DEBUG=true` environment variable
235
+ - Example: `true`
236
+
237
+ ### Example Configuration
238
+
239
+ **`kakarot.config.js`**:
240
+
241
+ ```javascript
242
+ /** @type {import('@kakarot-ci/core').KakarotConfig} */
243
+ const config = {
244
+ // Required: API key (can also be set via KAKAROT_API_KEY env var)
245
+ apiKey: process.env.OPENAI_API_KEY,
246
+
247
+ // Required: Test framework
248
+ framework: 'vitest',
249
+
250
+ // Optional: LLM provider settings (can also be set via PROVIDER and MODEL env vars)
251
+ provider: 'openai', // 'openai' | 'anthropic' | 'google'
252
+ model: 'gpt-4',
253
+ temperature: 0.2,
254
+ fixTemperature: 0.2,
255
+ maxTokens: 4000,
256
+
257
+ // File filtering
258
+ includePatterns: ['src/**/*.ts'],
259
+ excludePatterns: [
260
+ '**/*.test.ts',
261
+ '**/*.spec.ts',
262
+ '**/node_modules/**',
263
+ '**/dist/**',
264
+ '**/coverage/**',
265
+ ],
266
+
267
+ // Test file organization
268
+ testLocation: 'co-located', // 'separate' | 'co-located'
269
+ testDirectory: '__tests__', // Only used when testLocation is 'separate'
270
+ testFilePattern: '*.test.ts',
271
+
272
+ // Limits and behavior
273
+ maxTestsPerPR: 50,
274
+ maxFixAttempts: 3,
275
+
276
+ // GitHub integration
277
+ enableAutoCommit: true,
278
+ commitStrategy: 'branch-pr', // 'direct' | 'branch-pr'
279
+ enablePRComments: true,
280
+
281
+ // Optional features
282
+ enableCoverage: false,
283
+ debug: true,
284
+ };
285
+
286
+ module.exports = config;
287
+ ```
288
+
289
+ **`.kakarot-ci.config.json`** (minimal example):
290
+
291
+ ```json
292
+ {
293
+ "framework": "vitest",
294
+ "includePatterns": ["src/**/*.ts"],
295
+ "excludePatterns": [
296
+ "**/*.test.ts",
297
+ "**/*.spec.ts",
298
+ "**/node_modules/**",
299
+ "**/dist/**"
300
+ ],
301
+ "maxFixAttempts": 3,
302
+ "enableAutoCommit": true,
303
+ "commitStrategy": "branch-pr",
304
+ "enablePRComments": true
305
+ }
306
+ ```
307
+
308
+ **Note**: The `apiKey` should be provided via the `KAKAROT_API_KEY` environment variable for security.
309
+
310
+ **`package.json`**:
311
+
312
+ ```json
313
+ {
314
+ "kakarotCi": {
315
+ "framework": "vitest",
316
+ "testLocation": "co-located",
317
+ "maxTestsPerPR": 50
318
+ }
319
+ }
320
+ ```
321
+
322
+ ## GitHub Actions Integration
323
+
324
+ Create a workflow file (`.github/workflows/kakarot-ci.yml`):
325
+
326
+ ```yaml
327
+ name: Kakarot CI
328
+
329
+ on:
330
+ pull_request:
331
+ types: [opened, synchronize, reopened]
332
+
333
+ jobs:
334
+ generate-tests:
335
+ runs-on: ubuntu-latest
336
+ permissions:
337
+ contents: write # Required to push commits directly to branch (commitStrategy: 'direct')
338
+ pull-requests: write # Required to create PRs (commitStrategy: 'branch-pr') and post comments
339
+ steps:
340
+ - uses: actions/checkout@v4
341
+ with:
342
+ fetch-depth: 0
343
+ token: ${{ secrets.GITHUB_TOKEN }}
344
+
345
+ - name: Setup Node.js
346
+ uses: actions/setup-node@v4
347
+ with:
348
+ node-version: '20'
349
+
350
+ - name: Install dependencies
351
+ run: npm ci
352
+
353
+ - name: Generate tests
354
+ run: npx kakarot-ci
355
+ env:
356
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
357
+ KAKAROT_API_KEY: ${{ secrets.KAKAROT_API_KEY }}
358
+ # Optional: Override provider/model via env vars
359
+ # PROVIDER: openai
360
+ # MODEL: gpt-4
361
+
362
+ ```
363
+
364
+ **Note on permissions:**
365
+ - `contents: write` is required to push commits directly to the PR branch (when using `commitStrategy: 'direct'`, the default)
366
+ - `pull-requests: write` is required to:
367
+ - Create pull requests (when using `commitStrategy: 'branch-pr'`)
368
+ - Post PR comments (when `enablePRComments: true`)
369
+ - Both permissions are recommended for full functionality
370
+
371
+ **Using a Personal Access Token (PAT):**
372
+
373
+ If your organization restricts `GITHUB_TOKEN` permissions, you'll need to use a Personal Access Token:
374
+
375
+ ```yaml
376
+ - uses: actions/checkout@v4
377
+ with:
378
+ fetch-depth: 0
379
+ token: ${{ secrets.GH_PAT }} # Use PAT instead of GITHUB_TOKEN
380
+
381
+ - name: Generate tests
382
+ run: npx kakarot-ci
383
+ env:
384
+ GITHUB_TOKEN: ${{ secrets.GH_PAT }} # Use PAT instead of GITHUB_TOKEN
385
+ KAKAROT_API_KEY: ${{ secrets.KAKAROT_API_KEY }}
386
+ # Optional: Override provider/model via env vars
387
+ # PROVIDER: openai
388
+ # MODEL: gpt-4
389
+ ```
390
+
391
+ Create a PAT with `repo` scope and add it as a repository secret named `GH_PAT`.
392
+
393
+ ## How It Works
394
+
395
+ ### Pull Request Workflow
396
+
397
+ 1. **Analyze PR Changes**: Scans the pull request diff to identify changed files
398
+ 2. **Extract Functions**: Uses AST analysis to find functions, methods, and classes that were modified
399
+ 3. **Generate Tests**: Sends function code to LLM with carefully crafted prompts to generate comprehensive tests
400
+ 4. **Run Tests**: Executes the generated tests using your configured test framework
401
+ 5. **Fix Failures**: Automatically attempts to fix failing tests (up to `maxFixAttempts` times)
402
+ 6. **Commit Results**: Commits generated tests back to the PR (if `enableAutoCommit` is true)
403
+ 7. **Post Summary**: Posts a summary comment to the PR with test generation results
404
+
405
+ ### Core Components
406
+
407
+ The tool is built with modular components that can be used independently:
408
+
409
+ - **AST Analysis**: Extracts functions, methods, and classes from TypeScript/JavaScript files
410
+ - **Test Generation**: Uses LLM prompts optimized for generating accurate, comprehensive tests
411
+ - **Test Execution**: Runs tests using Jest or Vitest
412
+ - **Auto-Fix Loop**: Iteratively fixes failing tests using LLM feedback
413
+ - **GitHub Integration**: Optional integration for PR workflows, commits, and comments
414
+
415
+ You can use these components programmatically for custom workflows beyond pull requests.
416
+
417
+ ## Requirements
418
+
419
+ - Node.js >= 18.0.0
420
+ - A test framework (Jest or Vitest) already set up in your project
421
+ - An API key for your chosen LLM provider (OpenAI, Anthropic, or Google)
422
+ - GitHub token with appropriate permissions (required only for GitHub integration features)
423
+
424
+ ## CLI Options
425
+
426
+ ```bash
427
+ kakarot-ci [options]
428
+
429
+ Options:
430
+ --pr <number> Pull request number
431
+ --owner <string> Repository owner
432
+ --repo <string> Repository name
433
+ --token <string> GitHub token (or use GITHUB_TOKEN env var)
434
+ -V, --version Show version number
435
+ -h, --help Display help
436
+ ```
437
+
438
+ ## Environment Variables
439
+
440
+ - `KAKAROT_API_KEY`: LLM provider API key (required, can also be set in config file)
441
+ - `PROVIDER`: LLM provider (`openai`, `anthropic`, or `google`, can also be set in config file)
442
+ - `MODEL`: LLM model name (e.g., `gpt-4`, `claude-3-opus-20240229`, can also be set in config file)
443
+ - `GITHUB_TOKEN`: GitHub personal access token (required for GitHub operations)
444
+ - `GITHUB_REPOSITORY`: Repository in format `owner/repo` (auto-detected in GitHub Actions)
445
+ - `GITHUB_EVENT_PATH`: Path to GitHub event JSON (auto-set in GitHub Actions)
446
+ - `PR_NUMBER`: Pull request number (alternative to `--pr` flag)
447
+ - `KAKAROT_DEBUG`: Enable debug logging (`true`/`false`)
448
+
449
+ ## Contributing
450
+
451
+ Contributions are welcome! Please feel free to submit a Pull Request.
452
+
453
+ **Note**: This project is maintained by a single person as a side project. While I do my best to respond to issues and review PRs, response times may vary. Your patience and understanding are appreciated!
454
+
455
+ ## License
456
+
457
+ [BUSL-1.1](LICENSE)
package/dist/cli/index.js CHANGED
@@ -276,6 +276,7 @@ var GitHubClient = class {
276
276
  async fileExists(ref, path) {
277
277
  const originalError = console.error;
278
278
  const originalWarn = console.warn;
279
+ const originalLog = console.log;
279
280
  const suppress404 = (...args) => {
280
281
  const message = String(args[0] || "");
281
282
  if (message.includes("404") || message.includes("Not Found")) {
@@ -290,9 +291,17 @@ var GitHubClient = class {
290
291
  }
291
292
  originalWarn(...args);
292
293
  };
294
+ const suppress404Log = (...args) => {
295
+ const message = String(args[0] || "");
296
+ if (message.includes("404") || message.includes("Not Found")) {
297
+ return;
298
+ }
299
+ originalLog(...args);
300
+ };
293
301
  try {
294
302
  console.error = suppress404;
295
303
  console.warn = suppress404Warn;
304
+ console.log = suppress404Log;
296
305
  await this.octokit.rest.repos.getContent({
297
306
  owner: this.owner,
298
307
  repo: this.repo,
@@ -315,6 +324,7 @@ var GitHubClient = class {
315
324
  } finally {
316
325
  console.error = originalError;
317
326
  console.warn = originalWarn;
327
+ console.log = originalLog;
318
328
  }
319
329
  }
320
330
  /**
@@ -374,24 +384,11 @@ async function findProjectRoot(startPath) {
374
384
  async function loadConfig() {
375
385
  const explorer = cosmiconfig("kakarot", {
376
386
  searchPlaces: [
377
- "kakarot.config.ts",
378
387
  "kakarot.config.js",
379
- ".kakarot-ci.config.ts",
380
388
  ".kakarot-ci.config.js",
381
389
  ".kakarot-ci.config.json",
382
390
  "package.json"
383
- ],
384
- loaders: {
385
- ".ts": async (filepath) => {
386
- try {
387
- const configModule = await import(filepath);
388
- return configModule.default || configModule.config || null;
389
- } catch (err) {
390
- error(`Failed to load TypeScript config: ${err instanceof Error ? err.message : String(err)}`);
391
- return null;
392
- }
393
- }
394
- }
391
+ ]
395
392
  });
396
393
  try {
397
394
  const result = await explorer.search();
@@ -428,7 +425,7 @@ async function loadConfig() {
428
425
  } catch (err) {
429
426
  if (err instanceof Error && err.message.includes("apiKey")) {
430
427
  error(
431
- "Missing required apiKey. Provide it via:\n - Config file (kakarot.config.ts, .kakarot-ci.config.js/json, or package.json)\n - Environment variable: KAKAROT_API_KEY"
428
+ "Missing required apiKey. Provide it via:\n - Config file (kakarot.config.js, .kakarot-ci.config.js/json, or package.json)\n - Environment variable: KAKAROT_API_KEY"
432
429
  );
433
430
  }
434
431
  throw err;
@@ -1068,15 +1065,49 @@ function buildSystemPrompt(framework) {
1068
1065
  const importStatement = framework === "jest" ? "import { describe, it, expect } from 'jest';" : "import { describe, it, expect } from 'vitest';";
1069
1066
  return `You are an expert ${frameworkName} test writer. Your task is to generate comprehensive unit tests for TypeScript/JavaScript functions.
1070
1067
 
1068
+ CRITICAL: Test the ACTUAL behavior of the code, not assumed behavior.
1069
+
1071
1070
  Requirements:
1072
1071
  1. Generate complete, runnable ${frameworkName} test code
1073
1072
  2. Use ${frameworkName} syntax and best practices
1074
- 3. Test edge cases, error conditions, and normal operation
1075
- 4. Use descriptive test names that explain what is being tested
1076
- 5. Include proper setup/teardown if needed
1077
- 6. Mock external dependencies appropriately
1078
- 7. Test both success and failure scenarios
1079
- 8. Follow the existing test file structure if one exists
1073
+ 3. Analyze the function code to determine its ACTUAL runtime behavior before writing tests
1074
+ 4. Test edge cases and normal operation based on what the code actually does
1075
+ 5. Only test for errors/exceptions if the code actually throws them (check for try/catch, throw statements, or validation logic)
1076
+ 6. Match JavaScript/TypeScript runtime semantics:
1077
+ - Arithmetic operations (+, -, *, /, %, **) do NOT throw errors for invalid inputs; they return NaN or Infinity
1078
+ - Division by zero returns Infinity (not an error)
1079
+ - Modulo by zero returns NaN (not an error)
1080
+ - Bitwise operations convert values to 32-bit integers
1081
+ - Operations with null/undefined may coerce to numbers or return NaN
1082
+ - TypeScript types are compile-time only; runtime behavior follows JavaScript rules
1083
+ 7. For async functions:
1084
+ - Use async/await or .then()/.catch() appropriately
1085
+ - Test both resolved and rejected promises
1086
+ - Use resolves and rejects matchers when appropriate
1087
+ - Await async function calls in tests
1088
+ 8. For functions with side effects or external dependencies:
1089
+ - Mock external dependencies (APIs, file system, databases, etc.)
1090
+ - Mock imported modules using ${framework === "jest" ? "jest.mock()" : "vi.mock()"}
1091
+ - Reset mocks between tests to ensure test isolation
1092
+ - Verify mock calls if the function's behavior depends on them
1093
+ 9. For functions that modify state:
1094
+ - Test state before and after function calls
1095
+ - Reset state between tests if needed
1096
+ - Test state mutations, not just return values
1097
+ 10. For error testing:
1098
+ - Only test for errors if the function actually throws them
1099
+ - Match actual error types and messages (use toThrow() with specific error types/messages)
1100
+ - For async errors, use rejects matcher
1101
+ 11. Import handling:
1102
+ - Use the same import paths as the source file
1103
+ - Import types correctly (type imports for TypeScript types)
1104
+ - Mock dependencies at the module level, not the function level
1105
+ 12. Test isolation:
1106
+ - Each test should be independent and not rely on other tests
1107
+ - Use beforeEach/afterEach for setup/teardown when needed
1108
+ - Don't share mutable state between tests
1109
+ 13. Use descriptive test names that explain what is being tested
1110
+ 14. Follow the existing test file structure if one exists
1080
1111
 
1081
1112
  Output format:
1082
1113
  - Return ONLY the test code, no explanations or markdown code blocks
@@ -1146,18 +1177,38 @@ ${existingTestFile}
1146
1177
 
1147
1178
  `;
1148
1179
  }
1149
- prompt += `Generate comprehensive unit tests for ${target.functionName}. Include:
1180
+ prompt += `Generate comprehensive unit tests for ${target.functionName}. IMPORTANT: Analyze the function code above to determine its ACTUAL behavior before writing tests.
1181
+
1182
+ `;
1183
+ prompt += `Include:
1150
1184
  `;
1151
1185
  prompt += `- Tests for normal operation with various inputs
1152
1186
  `;
1153
- prompt += `- Tests for edge cases (null, undefined, empty arrays, etc.)
1187
+ prompt += `- Tests for edge cases based on what the code actually does (null, undefined, empty arrays, etc.)
1154
1188
  `;
1155
- prompt += `- Tests for error conditions if applicable
1189
+ prompt += `- Tests for error conditions ONLY if the code actually throws errors (check for throw statements, validation, or error handling)
1156
1190
  `;
1157
1191
  prompt += `- Tests for boundary conditions
1158
1192
  `;
1159
1193
  prompt += `- Proper mocking of dependencies if needed
1160
1194
 
1195
+ `;
1196
+ prompt += `Key considerations:
1197
+ `;
1198
+ prompt += `- If the function is async, use async/await in tests and test both success and error cases
1199
+ `;
1200
+ prompt += `- If the function uses external dependencies (imports), mock them appropriately
1201
+ `;
1202
+ prompt += `- If the function modifies state, test the state changes
1203
+ `;
1204
+ prompt += `- Match actual return types and values, not assumed types
1205
+ `;
1206
+ prompt += `- TypeScript types are compile-time only; test runtime behavior
1207
+ `;
1208
+ prompt += `- JavaScript/TypeScript arithmetic and bitwise operations do NOT throw errors for invalid inputs. They return NaN, Infinity, or perform type coercion. Only test for errors if the function code explicitly throws them.
1209
+ `;
1210
+ prompt += `- Use the same import paths as the source file for consistency
1211
+
1161
1212
  `;
1162
1213
  prompt += `Return ONLY the test code, no explanations or markdown formatting.`;
1163
1214
  return prompt;
@@ -1182,13 +1233,34 @@ Context:
1182
1233
  - The test code failed to run or produced incorrect results
1183
1234
  - You need to analyze the error and fix the test code
1184
1235
 
1236
+ CRITICAL: Tests must match the ACTUAL behavior of the code being tested, not assumed behavior.
1237
+
1185
1238
  Requirements:
1186
- 1. Fix the test code to make it pass
1187
- 2. Maintain the original test intent
1188
- 3. Use proper ${frameworkName} syntax
1189
- 4. Ensure all imports and dependencies are correct
1190
- 5. Fix any syntax errors, type errors, or logical errors
1191
- 6. If the original code being tested has issues, note that but focus on fixing the test
1239
+ 1. Analyze the original function code to understand its ACTUAL runtime behavior
1240
+ 2. Fix the test code to match what the function actually does, not what it "should" do
1241
+ 3. Only expect errors/exceptions if the function code actually throws them
1242
+ 4. Match JavaScript/TypeScript runtime semantics:
1243
+ - Arithmetic operations do NOT throw errors; they return NaN or Infinity
1244
+ - Division by zero returns Infinity (not an error)
1245
+ - Modulo by zero returns NaN (not an error)
1246
+ - Bitwise operations convert values to 32-bit integers
1247
+ - TypeScript types are compile-time only; runtime behavior follows JavaScript rules
1248
+ 5. For async functions:
1249
+ - Ensure tests properly await async calls
1250
+ - Use resolves/rejects matchers appropriately
1251
+ - Test both success and error cases for async functions
1252
+ 6. For functions with dependencies:
1253
+ - Ensure mocks are properly set up
1254
+ - Verify import paths match the source file
1255
+ - Reset mocks if needed for test isolation
1256
+ 7. For error testing:
1257
+ - Only expect errors if the function actually throws them
1258
+ - Match actual error types and messages
1259
+ - Use appropriate matchers (toThrow, rejects, etc.)
1260
+ 8. Maintain the original test intent where possible, but prioritize correctness
1261
+ 9. Use proper ${frameworkName} syntax
1262
+ 10. Ensure all imports and dependencies are correct
1263
+ 11. Fix any syntax errors, type errors, or logical errors
1192
1264
 
1193
1265
  Output format:
1194
1266
  - Return ONLY the fixed test code, no explanations or markdown code blocks