@kakarot-ci/core 0.7.0 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +457 -2
- package/dist/cli/index.js +102 -30
- package/dist/cli/index.js.map +2 -2
- package/dist/index.cjs +102 -30
- package/dist/index.cjs.map +2 -2
- package/dist/index.js +102 -30
- package/dist/index.js.map +2 -2
- package/dist/src/github/client.d.ts.map +1 -1
- package/dist/src/utils/config-loader.d.ts.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,2 +1,457 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
1
|
+
<div align="center">
|
|
2
|
+
<img src="assets/logo.svg" alt="Kakarot CI Logo" width="200">
|
|
3
|
+
|
|
4
|
+
# Kakarot CI
|
|
5
|
+
|
|
6
|
+
[](https://www.npmjs.com/package/@kakarot-ci/core)
|
|
7
|
+
[](https://nodejs.org/)
|
|
8
|
+
</div>
|
|
9
|
+
|
|
10
|
+
> AI-powered unit test generation for TypeScript and JavaScript
|
|
11
|
+
|
|
12
|
+
> **⚠️ Beta Software**: This project is currently in beta. While functional, expect occasional bugs and breaking changes as we iterate and improve. Please report issues and provide feedback!
|
|
13
|
+
|
|
14
|
+
Kakarot CI automatically generates comprehensive unit tests using AI. While optimized for pull request workflows, it can be used in various scenarios: analyzing PR changes, generating tests for specific files, or creating test suites for entire codebases. It analyzes code, generates test files, runs them, and can automatically commit results back to your repository.
|
|
15
|
+
|
|
16
|
+
## Features
|
|
17
|
+
|
|
18
|
+
- 🤖 **AI-Powered Test Generation**: Uses LLMs (OpenAI, Anthropic, Google) to generate comprehensive unit tests
|
|
19
|
+
- 🔍 **Smart Code Analysis**: Analyzes AST to extract functions and understand code structure
|
|
20
|
+
- 🎯 **Targeted Testing**: Generates tests for specific functions, files, or entire codebases
|
|
21
|
+
- 🔄 **Auto-Fix Loop**: Automatically fixes failing tests with multiple retry attempts
|
|
22
|
+
- 📊 **Coverage Reports**: Optional test coverage analysis and summaries
|
|
23
|
+
- 🚀 **GitHub Integration**: Seamlessly integrates with GitHub Actions and PR workflows (optional)
|
|
24
|
+
- ⚙️ **Flexible Configuration**: Supports Jest and Vitest, configurable test locations and patterns
|
|
25
|
+
- 📝 **PR Comments**: Automatically posts test generation summaries to pull requests
|
|
26
|
+
|
|
27
|
+
## Installation
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
npm install --save-dev @kakarot-ci/core
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Usage
|
|
34
|
+
|
|
35
|
+
### Pull Request Workflow (Primary Use Case)
|
|
36
|
+
|
|
37
|
+
The simplest way to use Kakarot CI is via the command-line interface for pull requests:
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
npx kakarot-ci --pr 123 --owner myorg --repo myrepo --token ghp_xxxxx
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Or in a GitHub Actions workflow:
|
|
44
|
+
|
|
45
|
+
```yaml
|
|
46
|
+
- name: Generate Tests
|
|
47
|
+
run: npx kakarot-ci
|
|
48
|
+
env:
|
|
49
|
+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
50
|
+
KAKAROT_API_KEY: ${{ secrets.KAKAROT_API_KEY }}
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
The CLI automatically detects:
|
|
54
|
+
- PR number from `GITHUB_EVENT_PATH` (GitHub Actions)
|
|
55
|
+
- Repository owner/repo from `GITHUB_REPOSITORY` or git remote
|
|
56
|
+
- GitHub token from `GITHUB_TOKEN` environment variable
|
|
57
|
+
|
|
58
|
+
### Programmatic API
|
|
59
|
+
|
|
60
|
+
#### Pull Request Processing
|
|
61
|
+
|
|
62
|
+
```typescript
|
|
63
|
+
import { runPullRequest } from '@kakarot-ci/core';
|
|
64
|
+
|
|
65
|
+
const summary = await runPullRequest({
|
|
66
|
+
prNumber: 123,
|
|
67
|
+
owner: 'myorg',
|
|
68
|
+
repo: 'myrepo',
|
|
69
|
+
githubToken: 'ghp_xxxxx',
|
|
70
|
+
});
|
|
71
|
+
|
|
72
|
+
console.log(`Generated ${summary.testsGenerated} tests`);
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
#### Custom Workflows
|
|
76
|
+
|
|
77
|
+
You can also use the lower-level APIs for custom workflows:
|
|
78
|
+
|
|
79
|
+
```typescript
|
|
80
|
+
import {
|
|
81
|
+
TestGenerator,
|
|
82
|
+
extractTestTargets,
|
|
83
|
+
analyzeFile,
|
|
84
|
+
writeTestFiles
|
|
85
|
+
} from '@kakarot-ci/core';
|
|
86
|
+
|
|
87
|
+
// Generate tests for specific files
|
|
88
|
+
const targets = await analyzeFile('src/utils.ts', 'main', githubClient, config);
|
|
89
|
+
const generator = new TestGenerator({ apiKey, provider: 'openai' });
|
|
90
|
+
|
|
91
|
+
for (const target of targets) {
|
|
92
|
+
const test = await generator.generateTest({
|
|
93
|
+
target,
|
|
94
|
+
framework: 'vitest',
|
|
95
|
+
});
|
|
96
|
+
// Write test file...
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
// Or extract targets from file diffs
|
|
100
|
+
const prFiles = [...]; // Your file changes
|
|
101
|
+
const targets = await extractTestTargets(prFiles, githubClient, 'sha', config);
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## Configuration
|
|
105
|
+
|
|
106
|
+
Kakarot CI can be configured via:
|
|
107
|
+
|
|
108
|
+
1. **Config file**: `kakarot.config.js`, `.kakarot-ci.config.js`, or `.kakarot-ci.config.json`
|
|
109
|
+
2. **package.json**: Add a `kakarotCi` field
|
|
110
|
+
3. **Environment variables**: `KAKAROT_API_KEY`, `GITHUB_TOKEN`, etc.
|
|
111
|
+
|
|
112
|
+
**Note**: TypeScript config files (`.ts`) are not supported in the compiled package. Use JavaScript (`.js`) or JSON (`.json`) config files instead.
|
|
113
|
+
|
|
114
|
+
### Configuration Options
|
|
115
|
+
|
|
116
|
+
#### Required
|
|
117
|
+
|
|
118
|
+
- **`apiKey`** (string, required)
|
|
119
|
+
- Your LLM provider API key
|
|
120
|
+
- Can also be set via `KAKAROT_API_KEY` environment variable
|
|
121
|
+
- Example: `"sk-xxxxx"` (OpenAI) or `"sk-ant-xxxxx"` (Anthropic)
|
|
122
|
+
|
|
123
|
+
#### LLM Provider Settings
|
|
124
|
+
|
|
125
|
+
- **`provider`** (string, optional, default: `"openai"`)
|
|
126
|
+
- LLM provider to use: `"openai"`, `"anthropic"`, or `"google"`
|
|
127
|
+
- Can also be set via `PROVIDER` environment variable
|
|
128
|
+
- Example: `"anthropic"`
|
|
129
|
+
|
|
130
|
+
- **`model`** (string, optional)
|
|
131
|
+
- Specific model to use (provider-specific)
|
|
132
|
+
- Can also be set via `MODEL` environment variable
|
|
133
|
+
- Defaults to provider's recommended model
|
|
134
|
+
- Examples: `"gpt-4"`, `"claude-3-opus-20240229"`, `"gemini-pro"`
|
|
135
|
+
|
|
136
|
+
- **`maxTokens`** (number, optional, default: provider default)
|
|
137
|
+
- Maximum tokens in response (1-100000)
|
|
138
|
+
- Example: `4000`
|
|
139
|
+
|
|
140
|
+
- **`temperature`** (number, optional, default: `0.2`)
|
|
141
|
+
- Temperature for test generation (0-2)
|
|
142
|
+
- Lower values = more consistent, higher = more creative
|
|
143
|
+
- Example: `0.2`
|
|
144
|
+
|
|
145
|
+
- **`fixTemperature`** (number, optional, default: `0.2`)
|
|
146
|
+
- Temperature for test fixing attempts (0-2)
|
|
147
|
+
- Example: `0.3`
|
|
148
|
+
|
|
149
|
+
- **`maxFixAttempts`** (number, optional, default: `3`)
|
|
150
|
+
- Maximum number of attempts to fix failing tests (0-5)
|
|
151
|
+
- Example: `5`
|
|
152
|
+
|
|
153
|
+
#### Test Framework
|
|
154
|
+
|
|
155
|
+
- **`framework`** (string, required)
|
|
156
|
+
- Test framework: `"jest"` or `"vitest"`
|
|
157
|
+
- Example: `"vitest"`
|
|
158
|
+
|
|
159
|
+
#### Test File Organization
|
|
160
|
+
|
|
161
|
+
- **`testLocation`** (string, optional, default: `"separate"`)
|
|
162
|
+
- Where to place test files: `"separate"` or `"co-located"`
|
|
163
|
+
- `"separate"`: Tests in a dedicated test directory
|
|
164
|
+
- `"co-located"`: Tests next to source files
|
|
165
|
+
- Example: `"co-located"`
|
|
166
|
+
|
|
167
|
+
- **`testDirectory`** (string, optional, default: `"__tests__"`)
|
|
168
|
+
- Directory name for test files (when `testLocation` is `"separate"`)
|
|
169
|
+
- Example: `"tests"` or `"__tests__"`
|
|
170
|
+
|
|
171
|
+
- **`testFilePattern`** (string, optional, default: `"*.test.ts"`)
|
|
172
|
+
- Glob pattern for test file names
|
|
173
|
+
- Example: `"*.spec.ts"` or `"*.test.js"`
|
|
174
|
+
|
|
175
|
+
#### File Filtering
|
|
176
|
+
|
|
177
|
+
- **`includePatterns`** (string[], optional, default: `["**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx"]`)
|
|
178
|
+
- Glob patterns for files to include when scanning for changes
|
|
179
|
+
- Example: `["src/**/*.ts", "lib/**/*.ts"]`
|
|
180
|
+
|
|
181
|
+
- **`excludePatterns`** (string[], optional, default: `["**/*.test.ts", "**/*.spec.ts", "**/*.test.js", "**/*.spec.js", "**/node_modules/**"]`)
|
|
182
|
+
- Glob patterns for files to exclude
|
|
183
|
+
- Example: `["**/*.test.ts", "**/vendor/**"]`
|
|
184
|
+
|
|
185
|
+
#### Limits
|
|
186
|
+
|
|
187
|
+
- **`maxTestsPerPR`** (number, optional, default: `50`)
|
|
188
|
+
- Maximum number of test targets to process per PR
|
|
189
|
+
- Prevents excessive API usage on large PRs
|
|
190
|
+
- Example: `100`
|
|
191
|
+
|
|
192
|
+
#### GitHub Integration
|
|
193
|
+
|
|
194
|
+
- **`githubToken`** (string, optional)
|
|
195
|
+
- GitHub personal access token
|
|
196
|
+
- Can also be set via `GITHUB_TOKEN` environment variable
|
|
197
|
+
- Required for GitHub operations (commits, PR comments)
|
|
198
|
+
- Example: `"ghp_xxxxx"`
|
|
199
|
+
|
|
200
|
+
- **`githubOwner`** (string, optional)
|
|
201
|
+
- Repository owner (can be auto-detected from git remote)
|
|
202
|
+
- Example: `"myorg"`
|
|
203
|
+
|
|
204
|
+
- **`githubRepo`** (string, optional)
|
|
205
|
+
- Repository name (can be auto-detected from git remote)
|
|
206
|
+
- Example: `"myrepo"`
|
|
207
|
+
|
|
208
|
+
#### Commit Strategy
|
|
209
|
+
|
|
210
|
+
- **`enableAutoCommit`** (boolean, optional, default: `true`)
|
|
211
|
+
- Automatically commit generated tests
|
|
212
|
+
- Example: `false` (to review tests before committing)
|
|
213
|
+
|
|
214
|
+
- **`commitStrategy`** (string, optional, default: `"direct"`)
|
|
215
|
+
- How to commit tests: `"direct"` or `"branch-pr"`
|
|
216
|
+
- `"direct"`: Commit directly to PR branch
|
|
217
|
+
- `"branch-pr"`: Create a new branch and open a PR
|
|
218
|
+
- Example: `"branch-pr"`
|
|
219
|
+
|
|
220
|
+
- **`enablePRComments`** (boolean, optional, default: `true`)
|
|
221
|
+
- Post test generation summary as PR comment
|
|
222
|
+
- Example: `true`
|
|
223
|
+
|
|
224
|
+
#### Coverage
|
|
225
|
+
|
|
226
|
+
- **`enableCoverage`** (boolean, optional, default: `false`)
|
|
227
|
+
- Enable test coverage collection and reporting
|
|
228
|
+
- Example: `true`
|
|
229
|
+
|
|
230
|
+
#### Debugging
|
|
231
|
+
|
|
232
|
+
- **`debug`** (boolean, optional, default: `false`)
|
|
233
|
+
- Enable debug logging
|
|
234
|
+
- Can also be set via `KAKAROT_DEBUG=true` environment variable
|
|
235
|
+
- Example: `true`
|
|
236
|
+
|
|
237
|
+
### Example Configuration
|
|
238
|
+
|
|
239
|
+
**`kakarot.config.js`**:
|
|
240
|
+
|
|
241
|
+
```javascript
|
|
242
|
+
/** @type {import('@kakarot-ci/core').KakarotConfig} */
|
|
243
|
+
const config = {
|
|
244
|
+
// Required: API key (can also be set via KAKAROT_API_KEY env var)
|
|
245
|
+
apiKey: process.env.OPENAI_API_KEY,
|
|
246
|
+
|
|
247
|
+
// Required: Test framework
|
|
248
|
+
framework: 'vitest',
|
|
249
|
+
|
|
250
|
+
// Optional: LLM provider settings (can also be set via PROVIDER and MODEL env vars)
|
|
251
|
+
provider: 'openai', // 'openai' | 'anthropic' | 'google'
|
|
252
|
+
model: 'gpt-4',
|
|
253
|
+
temperature: 0.2,
|
|
254
|
+
fixTemperature: 0.2,
|
|
255
|
+
maxTokens: 4000,
|
|
256
|
+
|
|
257
|
+
// File filtering
|
|
258
|
+
includePatterns: ['src/**/*.ts'],
|
|
259
|
+
excludePatterns: [
|
|
260
|
+
'**/*.test.ts',
|
|
261
|
+
'**/*.spec.ts',
|
|
262
|
+
'**/node_modules/**',
|
|
263
|
+
'**/dist/**',
|
|
264
|
+
'**/coverage/**',
|
|
265
|
+
],
|
|
266
|
+
|
|
267
|
+
// Test file organization
|
|
268
|
+
testLocation: 'co-located', // 'separate' | 'co-located'
|
|
269
|
+
testDirectory: '__tests__', // Only used when testLocation is 'separate'
|
|
270
|
+
testFilePattern: '*.test.ts',
|
|
271
|
+
|
|
272
|
+
// Limits and behavior
|
|
273
|
+
maxTestsPerPR: 50,
|
|
274
|
+
maxFixAttempts: 3,
|
|
275
|
+
|
|
276
|
+
// GitHub integration
|
|
277
|
+
enableAutoCommit: true,
|
|
278
|
+
commitStrategy: 'branch-pr', // 'direct' | 'branch-pr'
|
|
279
|
+
enablePRComments: true,
|
|
280
|
+
|
|
281
|
+
// Optional features
|
|
282
|
+
enableCoverage: false,
|
|
283
|
+
debug: true,
|
|
284
|
+
};
|
|
285
|
+
|
|
286
|
+
module.exports = config;
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
**`.kakarot-ci.config.json`** (minimal example):
|
|
290
|
+
|
|
291
|
+
```json
|
|
292
|
+
{
|
|
293
|
+
"framework": "vitest",
|
|
294
|
+
"includePatterns": ["src/**/*.ts"],
|
|
295
|
+
"excludePatterns": [
|
|
296
|
+
"**/*.test.ts",
|
|
297
|
+
"**/*.spec.ts",
|
|
298
|
+
"**/node_modules/**",
|
|
299
|
+
"**/dist/**"
|
|
300
|
+
],
|
|
301
|
+
"maxFixAttempts": 3,
|
|
302
|
+
"enableAutoCommit": true,
|
|
303
|
+
"commitStrategy": "branch-pr",
|
|
304
|
+
"enablePRComments": true
|
|
305
|
+
}
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
**Note**: The `apiKey` should be provided via the `KAKAROT_API_KEY` environment variable for security.
|
|
309
|
+
|
|
310
|
+
**`package.json`**:
|
|
311
|
+
|
|
312
|
+
```json
|
|
313
|
+
{
|
|
314
|
+
"kakarotCi": {
|
|
315
|
+
"framework": "vitest",
|
|
316
|
+
"testLocation": "co-located",
|
|
317
|
+
"maxTestsPerPR": 50
|
|
318
|
+
}
|
|
319
|
+
}
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
## GitHub Actions Integration
|
|
323
|
+
|
|
324
|
+
Create a workflow file (`.github/workflows/kakarot-ci.yml`):
|
|
325
|
+
|
|
326
|
+
```yaml
|
|
327
|
+
name: Kakarot CI
|
|
328
|
+
|
|
329
|
+
on:
|
|
330
|
+
pull_request:
|
|
331
|
+
types: [opened, synchronize, reopened]
|
|
332
|
+
|
|
333
|
+
jobs:
|
|
334
|
+
generate-tests:
|
|
335
|
+
runs-on: ubuntu-latest
|
|
336
|
+
permissions:
|
|
337
|
+
contents: write # Required to push commits directly to branch (commitStrategy: 'direct')
|
|
338
|
+
pull-requests: write # Required to create PRs (commitStrategy: 'branch-pr') and post comments
|
|
339
|
+
steps:
|
|
340
|
+
- uses: actions/checkout@v4
|
|
341
|
+
with:
|
|
342
|
+
fetch-depth: 0
|
|
343
|
+
token: ${{ secrets.GITHUB_TOKEN }}
|
|
344
|
+
|
|
345
|
+
- name: Setup Node.js
|
|
346
|
+
uses: actions/setup-node@v4
|
|
347
|
+
with:
|
|
348
|
+
node-version: '20'
|
|
349
|
+
|
|
350
|
+
- name: Install dependencies
|
|
351
|
+
run: npm ci
|
|
352
|
+
|
|
353
|
+
- name: Generate tests
|
|
354
|
+
run: npx kakarot-ci
|
|
355
|
+
env:
|
|
356
|
+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
357
|
+
KAKAROT_API_KEY: ${{ secrets.KAKAROT_API_KEY }}
|
|
358
|
+
# Optional: Override provider/model via env vars
|
|
359
|
+
# PROVIDER: openai
|
|
360
|
+
# MODEL: gpt-4
|
|
361
|
+
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
**Note on permissions:**
|
|
365
|
+
- `contents: write` is required to push commits directly to the PR branch (when using `commitStrategy: 'direct'`, the default)
|
|
366
|
+
- `pull-requests: write` is required to:
|
|
367
|
+
- Create pull requests (when using `commitStrategy: 'branch-pr'`)
|
|
368
|
+
- Post PR comments (when `enablePRComments: true`)
|
|
369
|
+
- Both permissions are recommended for full functionality
|
|
370
|
+
|
|
371
|
+
**Using a Personal Access Token (PAT):**
|
|
372
|
+
|
|
373
|
+
If your organization restricts `GITHUB_TOKEN` permissions, you'll need to use a Personal Access Token:
|
|
374
|
+
|
|
375
|
+
```yaml
|
|
376
|
+
- uses: actions/checkout@v4
|
|
377
|
+
with:
|
|
378
|
+
fetch-depth: 0
|
|
379
|
+
token: ${{ secrets.GH_PAT }} # Use PAT instead of GITHUB_TOKEN
|
|
380
|
+
|
|
381
|
+
- name: Generate tests
|
|
382
|
+
run: npx kakarot-ci
|
|
383
|
+
env:
|
|
384
|
+
GITHUB_TOKEN: ${{ secrets.GH_PAT }} # Use PAT instead of GITHUB_TOKEN
|
|
385
|
+
KAKAROT_API_KEY: ${{ secrets.KAKAROT_API_KEY }}
|
|
386
|
+
# Optional: Override provider/model via env vars
|
|
387
|
+
# PROVIDER: openai
|
|
388
|
+
# MODEL: gpt-4
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
Create a PAT with `repo` scope and add it as a repository secret named `GH_PAT`.
|
|
392
|
+
|
|
393
|
+
## How It Works
|
|
394
|
+
|
|
395
|
+
### Pull Request Workflow
|
|
396
|
+
|
|
397
|
+
1. **Analyze PR Changes**: Scans the pull request diff to identify changed files
|
|
398
|
+
2. **Extract Functions**: Uses AST analysis to find functions, methods, and classes that were modified
|
|
399
|
+
3. **Generate Tests**: Sends function code to LLM with carefully crafted prompts to generate comprehensive tests
|
|
400
|
+
4. **Run Tests**: Executes the generated tests using your configured test framework
|
|
401
|
+
5. **Fix Failures**: Automatically attempts to fix failing tests (up to `maxFixAttempts` times)
|
|
402
|
+
6. **Commit Results**: Commits generated tests back to the PR (if `enableAutoCommit` is true)
|
|
403
|
+
7. **Post Summary**: Posts a summary comment to the PR with test generation results
|
|
404
|
+
|
|
405
|
+
### Core Components
|
|
406
|
+
|
|
407
|
+
The tool is built with modular components that can be used independently:
|
|
408
|
+
|
|
409
|
+
- **AST Analysis**: Extracts functions, methods, and classes from TypeScript/JavaScript files
|
|
410
|
+
- **Test Generation**: Uses LLM prompts optimized for generating accurate, comprehensive tests
|
|
411
|
+
- **Test Execution**: Runs tests using Jest or Vitest
|
|
412
|
+
- **Auto-Fix Loop**: Iteratively fixes failing tests using LLM feedback
|
|
413
|
+
- **GitHub Integration**: Optional integration for PR workflows, commits, and comments
|
|
414
|
+
|
|
415
|
+
You can use these components programmatically for custom workflows beyond pull requests.
|
|
416
|
+
|
|
417
|
+
## Requirements
|
|
418
|
+
|
|
419
|
+
- Node.js >= 18.0.0
|
|
420
|
+
- A test framework (Jest or Vitest) already set up in your project
|
|
421
|
+
- An API key for your chosen LLM provider (OpenAI, Anthropic, or Google)
|
|
422
|
+
- GitHub token with appropriate permissions (required only for GitHub integration features)
|
|
423
|
+
|
|
424
|
+
## CLI Options
|
|
425
|
+
|
|
426
|
+
```bash
|
|
427
|
+
kakarot-ci [options]
|
|
428
|
+
|
|
429
|
+
Options:
|
|
430
|
+
--pr <number> Pull request number
|
|
431
|
+
--owner <string> Repository owner
|
|
432
|
+
--repo <string> Repository name
|
|
433
|
+
--token <string> GitHub token (or use GITHUB_TOKEN env var)
|
|
434
|
+
-V, --version Show version number
|
|
435
|
+
-h, --help Display help
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
## Environment Variables
|
|
439
|
+
|
|
440
|
+
- `KAKAROT_API_KEY`: LLM provider API key (required, can also be set in config file)
|
|
441
|
+
- `PROVIDER`: LLM provider (`openai`, `anthropic`, or `google`, can also be set in config file)
|
|
442
|
+
- `MODEL`: LLM model name (e.g., `gpt-4`, `claude-3-opus-20240229`, can also be set in config file)
|
|
443
|
+
- `GITHUB_TOKEN`: GitHub personal access token (required for GitHub operations)
|
|
444
|
+
- `GITHUB_REPOSITORY`: Repository in format `owner/repo` (auto-detected in GitHub Actions)
|
|
445
|
+
- `GITHUB_EVENT_PATH`: Path to GitHub event JSON (auto-set in GitHub Actions)
|
|
446
|
+
- `PR_NUMBER`: Pull request number (alternative to `--pr` flag)
|
|
447
|
+
- `KAKAROT_DEBUG`: Enable debug logging (`true`/`false`)
|
|
448
|
+
|
|
449
|
+
## Contributing
|
|
450
|
+
|
|
451
|
+
Contributions are welcome! Please feel free to submit a Pull Request.
|
|
452
|
+
|
|
453
|
+
**Note**: This project is maintained by a single person as a side project. While I do my best to respond to issues and review PRs, response times may vary. Your patience and understanding are appreciated!
|
|
454
|
+
|
|
455
|
+
## License
|
|
456
|
+
|
|
457
|
+
[BUSL-1.1](LICENSE)
|
package/dist/cli/index.js
CHANGED
|
@@ -276,6 +276,7 @@ var GitHubClient = class {
|
|
|
276
276
|
async fileExists(ref, path) {
|
|
277
277
|
const originalError = console.error;
|
|
278
278
|
const originalWarn = console.warn;
|
|
279
|
+
const originalLog = console.log;
|
|
279
280
|
const suppress404 = (...args) => {
|
|
280
281
|
const message = String(args[0] || "");
|
|
281
282
|
if (message.includes("404") || message.includes("Not Found")) {
|
|
@@ -290,9 +291,17 @@ var GitHubClient = class {
|
|
|
290
291
|
}
|
|
291
292
|
originalWarn(...args);
|
|
292
293
|
};
|
|
294
|
+
const suppress404Log = (...args) => {
|
|
295
|
+
const message = String(args[0] || "");
|
|
296
|
+
if (message.includes("404") || message.includes("Not Found")) {
|
|
297
|
+
return;
|
|
298
|
+
}
|
|
299
|
+
originalLog(...args);
|
|
300
|
+
};
|
|
293
301
|
try {
|
|
294
302
|
console.error = suppress404;
|
|
295
303
|
console.warn = suppress404Warn;
|
|
304
|
+
console.log = suppress404Log;
|
|
296
305
|
await this.octokit.rest.repos.getContent({
|
|
297
306
|
owner: this.owner,
|
|
298
307
|
repo: this.repo,
|
|
@@ -315,6 +324,7 @@ var GitHubClient = class {
|
|
|
315
324
|
} finally {
|
|
316
325
|
console.error = originalError;
|
|
317
326
|
console.warn = originalWarn;
|
|
327
|
+
console.log = originalLog;
|
|
318
328
|
}
|
|
319
329
|
}
|
|
320
330
|
/**
|
|
@@ -374,24 +384,11 @@ async function findProjectRoot(startPath) {
|
|
|
374
384
|
async function loadConfig() {
|
|
375
385
|
const explorer = cosmiconfig("kakarot", {
|
|
376
386
|
searchPlaces: [
|
|
377
|
-
"kakarot.config.ts",
|
|
378
387
|
"kakarot.config.js",
|
|
379
|
-
".kakarot-ci.config.ts",
|
|
380
388
|
".kakarot-ci.config.js",
|
|
381
389
|
".kakarot-ci.config.json",
|
|
382
390
|
"package.json"
|
|
383
|
-
]
|
|
384
|
-
loaders: {
|
|
385
|
-
".ts": async (filepath) => {
|
|
386
|
-
try {
|
|
387
|
-
const configModule = await import(filepath);
|
|
388
|
-
return configModule.default || configModule.config || null;
|
|
389
|
-
} catch (err) {
|
|
390
|
-
error(`Failed to load TypeScript config: ${err instanceof Error ? err.message : String(err)}`);
|
|
391
|
-
return null;
|
|
392
|
-
}
|
|
393
|
-
}
|
|
394
|
-
}
|
|
391
|
+
]
|
|
395
392
|
});
|
|
396
393
|
try {
|
|
397
394
|
const result = await explorer.search();
|
|
@@ -428,7 +425,7 @@ async function loadConfig() {
|
|
|
428
425
|
} catch (err) {
|
|
429
426
|
if (err instanceof Error && err.message.includes("apiKey")) {
|
|
430
427
|
error(
|
|
431
|
-
"Missing required apiKey. Provide it via:\n - Config file (kakarot.config.
|
|
428
|
+
"Missing required apiKey. Provide it via:\n - Config file (kakarot.config.js, .kakarot-ci.config.js/json, or package.json)\n - Environment variable: KAKAROT_API_KEY"
|
|
432
429
|
);
|
|
433
430
|
}
|
|
434
431
|
throw err;
|
|
@@ -1068,15 +1065,49 @@ function buildSystemPrompt(framework) {
|
|
|
1068
1065
|
const importStatement = framework === "jest" ? "import { describe, it, expect } from 'jest';" : "import { describe, it, expect } from 'vitest';";
|
|
1069
1066
|
return `You are an expert ${frameworkName} test writer. Your task is to generate comprehensive unit tests for TypeScript/JavaScript functions.
|
|
1070
1067
|
|
|
1068
|
+
CRITICAL: Test the ACTUAL behavior of the code, not assumed behavior.
|
|
1069
|
+
|
|
1071
1070
|
Requirements:
|
|
1072
1071
|
1. Generate complete, runnable ${frameworkName} test code
|
|
1073
1072
|
2. Use ${frameworkName} syntax and best practices
|
|
1074
|
-
3.
|
|
1075
|
-
4.
|
|
1076
|
-
5.
|
|
1077
|
-
6.
|
|
1078
|
-
|
|
1079
|
-
|
|
1073
|
+
3. Analyze the function code to determine its ACTUAL runtime behavior before writing tests
|
|
1074
|
+
4. Test edge cases and normal operation based on what the code actually does
|
|
1075
|
+
5. Only test for errors/exceptions if the code actually throws them (check for try/catch, throw statements, or validation logic)
|
|
1076
|
+
6. Match JavaScript/TypeScript runtime semantics:
|
|
1077
|
+
- Arithmetic operations (+, -, *, /, %, **) do NOT throw errors for invalid inputs; they return NaN or Infinity
|
|
1078
|
+
- Division by zero returns Infinity (not an error)
|
|
1079
|
+
- Modulo by zero returns NaN (not an error)
|
|
1080
|
+
- Bitwise operations convert values to 32-bit integers
|
|
1081
|
+
- Operations with null/undefined may coerce to numbers or return NaN
|
|
1082
|
+
- TypeScript types are compile-time only; runtime behavior follows JavaScript rules
|
|
1083
|
+
7. For async functions:
|
|
1084
|
+
- Use async/await or .then()/.catch() appropriately
|
|
1085
|
+
- Test both resolved and rejected promises
|
|
1086
|
+
- Use resolves and rejects matchers when appropriate
|
|
1087
|
+
- Await async function calls in tests
|
|
1088
|
+
8. For functions with side effects or external dependencies:
|
|
1089
|
+
- Mock external dependencies (APIs, file system, databases, etc.)
|
|
1090
|
+
- Mock imported modules using ${framework === "jest" ? "jest.mock()" : "vi.mock()"}
|
|
1091
|
+
- Reset mocks between tests to ensure test isolation
|
|
1092
|
+
- Verify mock calls if the function's behavior depends on them
|
|
1093
|
+
9. For functions that modify state:
|
|
1094
|
+
- Test state before and after function calls
|
|
1095
|
+
- Reset state between tests if needed
|
|
1096
|
+
- Test state mutations, not just return values
|
|
1097
|
+
10. For error testing:
|
|
1098
|
+
- Only test for errors if the function actually throws them
|
|
1099
|
+
- Match actual error types and messages (use toThrow() with specific error types/messages)
|
|
1100
|
+
- For async errors, use rejects matcher
|
|
1101
|
+
11. Import handling:
|
|
1102
|
+
- Use the same import paths as the source file
|
|
1103
|
+
- Import types correctly (type imports for TypeScript types)
|
|
1104
|
+
- Mock dependencies at the module level, not the function level
|
|
1105
|
+
12. Test isolation:
|
|
1106
|
+
- Each test should be independent and not rely on other tests
|
|
1107
|
+
- Use beforeEach/afterEach for setup/teardown when needed
|
|
1108
|
+
- Don't share mutable state between tests
|
|
1109
|
+
13. Use descriptive test names that explain what is being tested
|
|
1110
|
+
14. Follow the existing test file structure if one exists
|
|
1080
1111
|
|
|
1081
1112
|
Output format:
|
|
1082
1113
|
- Return ONLY the test code, no explanations or markdown code blocks
|
|
@@ -1146,18 +1177,38 @@ ${existingTestFile}
|
|
|
1146
1177
|
|
|
1147
1178
|
`;
|
|
1148
1179
|
}
|
|
1149
|
-
prompt += `Generate comprehensive unit tests for ${target.functionName}.
|
|
1180
|
+
prompt += `Generate comprehensive unit tests for ${target.functionName}. IMPORTANT: Analyze the function code above to determine its ACTUAL behavior before writing tests.
|
|
1181
|
+
|
|
1182
|
+
`;
|
|
1183
|
+
prompt += `Include:
|
|
1150
1184
|
`;
|
|
1151
1185
|
prompt += `- Tests for normal operation with various inputs
|
|
1152
1186
|
`;
|
|
1153
|
-
prompt += `- Tests for edge cases (null, undefined, empty arrays, etc.)
|
|
1187
|
+
prompt += `- Tests for edge cases based on what the code actually does (null, undefined, empty arrays, etc.)
|
|
1154
1188
|
`;
|
|
1155
|
-
prompt += `- Tests for error conditions if
|
|
1189
|
+
prompt += `- Tests for error conditions ONLY if the code actually throws errors (check for throw statements, validation, or error handling)
|
|
1156
1190
|
`;
|
|
1157
1191
|
prompt += `- Tests for boundary conditions
|
|
1158
1192
|
`;
|
|
1159
1193
|
prompt += `- Proper mocking of dependencies if needed
|
|
1160
1194
|
|
|
1195
|
+
`;
|
|
1196
|
+
prompt += `Key considerations:
|
|
1197
|
+
`;
|
|
1198
|
+
prompt += `- If the function is async, use async/await in tests and test both success and error cases
|
|
1199
|
+
`;
|
|
1200
|
+
prompt += `- If the function uses external dependencies (imports), mock them appropriately
|
|
1201
|
+
`;
|
|
1202
|
+
prompt += `- If the function modifies state, test the state changes
|
|
1203
|
+
`;
|
|
1204
|
+
prompt += `- Match actual return types and values, not assumed types
|
|
1205
|
+
`;
|
|
1206
|
+
prompt += `- TypeScript types are compile-time only; test runtime behavior
|
|
1207
|
+
`;
|
|
1208
|
+
prompt += `- JavaScript/TypeScript arithmetic and bitwise operations do NOT throw errors for invalid inputs. They return NaN, Infinity, or perform type coercion. Only test for errors if the function code explicitly throws them.
|
|
1209
|
+
`;
|
|
1210
|
+
prompt += `- Use the same import paths as the source file for consistency
|
|
1211
|
+
|
|
1161
1212
|
`;
|
|
1162
1213
|
prompt += `Return ONLY the test code, no explanations or markdown formatting.`;
|
|
1163
1214
|
return prompt;
|
|
@@ -1182,13 +1233,34 @@ Context:
|
|
|
1182
1233
|
- The test code failed to run or produced incorrect results
|
|
1183
1234
|
- You need to analyze the error and fix the test code
|
|
1184
1235
|
|
|
1236
|
+
CRITICAL: Tests must match the ACTUAL behavior of the code being tested, not assumed behavior.
|
|
1237
|
+
|
|
1185
1238
|
Requirements:
|
|
1186
|
-
1.
|
|
1187
|
-
2.
|
|
1188
|
-
3.
|
|
1189
|
-
4.
|
|
1190
|
-
|
|
1191
|
-
|
|
1239
|
+
1. Analyze the original function code to understand its ACTUAL runtime behavior
|
|
1240
|
+
2. Fix the test code to match what the function actually does, not what it "should" do
|
|
1241
|
+
3. Only expect errors/exceptions if the function code actually throws them
|
|
1242
|
+
4. Match JavaScript/TypeScript runtime semantics:
|
|
1243
|
+
- Arithmetic operations do NOT throw errors; they return NaN or Infinity
|
|
1244
|
+
- Division by zero returns Infinity (not an error)
|
|
1245
|
+
- Modulo by zero returns NaN (not an error)
|
|
1246
|
+
- Bitwise operations convert values to 32-bit integers
|
|
1247
|
+
- TypeScript types are compile-time only; runtime behavior follows JavaScript rules
|
|
1248
|
+
5. For async functions:
|
|
1249
|
+
- Ensure tests properly await async calls
|
|
1250
|
+
- Use resolves/rejects matchers appropriately
|
|
1251
|
+
- Test both success and error cases for async functions
|
|
1252
|
+
6. For functions with dependencies:
|
|
1253
|
+
- Ensure mocks are properly set up
|
|
1254
|
+
- Verify import paths match the source file
|
|
1255
|
+
- Reset mocks if needed for test isolation
|
|
1256
|
+
7. For error testing:
|
|
1257
|
+
- Only expect errors if the function actually throws them
|
|
1258
|
+
- Match actual error types and messages
|
|
1259
|
+
- Use appropriate matchers (toThrow, rejects, etc.)
|
|
1260
|
+
8. Maintain the original test intent where possible, but prioritize correctness
|
|
1261
|
+
9. Use proper ${frameworkName} syntax
|
|
1262
|
+
10. Ensure all imports and dependencies are correct
|
|
1263
|
+
11. Fix any syntax errors, type errors, or logical errors
|
|
1192
1264
|
|
|
1193
1265
|
Output format:
|
|
1194
1266
|
- Return ONLY the fixed test code, no explanations or markdown code blocks
|