@zibby/cli 0.1.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +926 -0
- package/bin/zibby.js +266 -0
- package/package.json +65 -0
- package/src/auth/cli-login.js +406 -0
- package/src/commands/analyze-graph.js +334 -0
- package/src/commands/ci-setup.js +65 -0
- package/src/commands/implement.js +664 -0
- package/src/commands/init.js +736 -0
- package/src/commands/list-projects.js +78 -0
- package/src/commands/memory.js +171 -0
- package/src/commands/run.js +926 -0
- package/src/commands/setup-scripts.js +101 -0
- package/src/commands/upload.js +163 -0
- package/src/commands/video.js +30 -0
- package/src/commands/workflow.js +369 -0
- package/src/config/config.js +117 -0
- package/src/config/environments.js +145 -0
- package/src/utils/execution-context.js +25 -0
- package/src/utils/progress-reporter.js +155 -0
package/README.md
ADDED
|
@@ -0,0 +1,926 @@
|
|
|
1
|
+
# @zibby/cli
|
|
2
|
+
|
|
3
|
+
Command-line interface for Zibby Test Automation with Rails-like generators.
|
|
4
|
+
|
|
5
|
+
## Installation
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
# Global installation
|
|
9
|
+
npm install -g @zibby/cli
|
|
10
|
+
|
|
11
|
+
# Or use npx
|
|
12
|
+
npx @zibby/cli init my-project
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Authentication
|
|
18
|
+
|
|
19
|
+
Zibby CLI requires **two-layer authentication** for cloud uploads:
|
|
20
|
+
|
|
21
|
+
### Setup Options
|
|
22
|
+
|
|
23
|
+
**Option 1: Local Development (Interactive)**
|
|
24
|
+
```bash
|
|
25
|
+
# Step 1: Login with OAuth
|
|
26
|
+
zibby login
|
|
27
|
+
# Opens browser → Login → Saves to ~/.zibby/config.json
|
|
28
|
+
|
|
29
|
+
# Step 2: Set project token in .env
|
|
30
|
+
# ZIBBY_API_KEY=zby_xxx
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
**Option 2: CI/CD (Personal Access Token)**
|
|
34
|
+
```bash
|
|
35
|
+
# Step 1: Generate token at https://zibby.app/settings/tokens
|
|
36
|
+
|
|
37
|
+
# Step 2: Set environment variables
|
|
38
|
+
export ZIBBY_USER_TOKEN=zby_pat_xxxxx
|
|
39
|
+
export ZIBBY_PROJECT_ID=zby_xxxxx
|
|
40
|
+
|
|
41
|
+
# Step 3: Run tests
|
|
42
|
+
zibby run tests/**/*.spec.js
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
### Running Tests with Cloud Sync
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
# Both project token + user token are required
|
|
49
|
+
zibby run test.spec.js --sync
|
|
50
|
+
|
|
51
|
+
# Or with explicit project override
|
|
52
|
+
zibby run test.spec.js --project zby_xxx --sync
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**Backend validates:**
|
|
56
|
+
- ✅ Project token is valid (`zby_xxx`)
|
|
57
|
+
- ✅ User token is valid (JWT from `zibby login` OR PAT from env var)
|
|
58
|
+
- ✅ User is a member of the project
|
|
59
|
+
|
|
60
|
+
### Authentication Commands
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
# Interactive login (local development)
|
|
64
|
+
zibby login
|
|
65
|
+
|
|
66
|
+
# Check current status
|
|
67
|
+
zibby status
|
|
68
|
+
|
|
69
|
+
# Logout (clears session)
|
|
70
|
+
zibby logout
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### Personal Access Tokens (CI/CD)
|
|
74
|
+
|
|
75
|
+
For automated pipelines, use Personal Access Tokens:
|
|
76
|
+
|
|
77
|
+
1. **Generate token:** https://zibby.app/settings/tokens
|
|
78
|
+
2. **Add to CI secrets:**
|
|
79
|
+
```yaml
|
|
80
|
+
# GitHub Actions
|
|
81
|
+
env:
|
|
82
|
+
ZIBBY_USER_TOKEN: ${{ secrets.ZIBBY_USER_TOKEN }}
|
|
83
|
+
ZIBBY_PROJECT_ID: zby_xxxxx
|
|
84
|
+
```
|
|
85
|
+
3. **Run tests:**
|
|
86
|
+
```bash
|
|
87
|
+
npm install -g @zibby/cli
|
|
88
|
+
zibby run tests/**/*.spec.js
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
**Token features:**
|
|
92
|
+
- 📅 Configurable expiration (30/60/90 days, or never)
|
|
93
|
+
- 🔄 Revoke anytime from dashboard
|
|
94
|
+
- 📊 Track last used timestamp
|
|
95
|
+
- 🔐 Secure: hashed storage, shown once
|
|
96
|
+
|
|
97
|
+
### Benefits of Two-Layer Auth
|
|
98
|
+
|
|
99
|
+
| Benefit | Description |
|
|
100
|
+
|---------|-------------|
|
|
101
|
+
| 🔒 **Enhanced Security** | Both project token AND user token required |
|
|
102
|
+
| 📊 **Audit Trail** | Tracks WHO uploaded each test execution |
|
|
103
|
+
| 👥 **Team Access Control** | Revoke users without rotating project token |
|
|
104
|
+
| 🚫 **Project Membership** | Can't upload to projects you're not a member of |
|
|
105
|
+
| 🤖 **CI/CD Ready** | Personal Access Tokens for automation |
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## Commands
|
|
110
|
+
|
|
111
|
+
### `zibby init` - Project Generator
|
|
112
|
+
|
|
113
|
+
Initialize a new test automation project (like `rails new`):
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
# Create new project
|
|
117
|
+
zibby init my-tests
|
|
118
|
+
|
|
119
|
+
# Or initialize in current directory
|
|
120
|
+
zibby init
|
|
121
|
+
|
|
122
|
+
# Options
|
|
123
|
+
zibby init my-tests --agent=cursor --no-cloud --skip-install
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
**What it does:**
|
|
127
|
+
- ✅ Creates project structure
|
|
128
|
+
- ✅ Generates `.zibby.config.js`
|
|
129
|
+
- ✅ Generates `.env.example`
|
|
130
|
+
- ✅ Creates example test specs
|
|
131
|
+
- ✅ Installs dependencies
|
|
132
|
+
- ✅ Installs Playwright browsers
|
|
133
|
+
- ✅ Generates README
|
|
134
|
+
|
|
135
|
+
### `zibby run` - Run Test Specification
|
|
136
|
+
|
|
137
|
+
Execute a test specification:
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
# Basic usage
|
|
141
|
+
zibby run test-specs/auth/login.txt
|
|
142
|
+
|
|
143
|
+
# With options
|
|
144
|
+
zibby run test-specs/auth/login.txt --agent=cursor --headless
|
|
145
|
+
|
|
146
|
+
# Open results in browser after completion
|
|
147
|
+
zibby run test-specs/auth/login.txt --project zby_xxx --collection "My Tests" --open
|
|
148
|
+
|
|
149
|
+
# For CI/CD
|
|
150
|
+
zibby run test-specs/auth/login.txt --agent=cursor --auto-approve --headless
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
**Options:**
|
|
154
|
+
- `--agent <type>` - Agent to use (claude, cursor)
|
|
155
|
+
- `--headless` - Run browser in headless mode
|
|
156
|
+
- `--config <path>` - Path to config file (default: `.zibby.config.js`)
|
|
157
|
+
- `--auto-approve` - Auto-approve MCP tools for CI/CD
|
|
158
|
+
- `-o, --open` - Open test results in browser after upload completes
|
|
159
|
+
- `--project <id>` - Project API key for cloud sync (or use ZIBBY_API_KEY env)
|
|
160
|
+
- `--collection <name>` - Collection to upload to
|
|
161
|
+
- `--sync` / `--no-sync` - Force enable/disable cloud sync
|
|
162
|
+
|
|
163
|
+
### `zibby video` - Organize Test Videos
|
|
164
|
+
|
|
165
|
+
Move test videos next to their test files:
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
zibby video
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
**Before:**
|
|
172
|
+
```
|
|
173
|
+
test-results/auth-login-Login-Functionality-chromium/video.webm
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
**After:**
|
|
177
|
+
```
|
|
178
|
+
tests/auth/login.spec.webm
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
### `zibby ci-setup` - CI/CD Setup
|
|
182
|
+
|
|
183
|
+
Setup Cursor Agent for CI/CD:
|
|
184
|
+
|
|
185
|
+
```bash
|
|
186
|
+
# Patch cursor-agent for auto-approval
|
|
187
|
+
zibby ci-setup
|
|
188
|
+
|
|
189
|
+
# Get approval keys
|
|
190
|
+
zibby ci-setup --get-keys
|
|
191
|
+
|
|
192
|
+
# Save approval keys to project
|
|
193
|
+
zibby ci-setup --get-keys --save
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
## Generated Project Structure
|
|
197
|
+
|
|
198
|
+
When you run `zibby init`, it creates:
|
|
199
|
+
|
|
200
|
+
```
|
|
201
|
+
my-project/
|
|
202
|
+
├── .zibby.config.js # Configuration (50 lines)
|
|
203
|
+
├── .env.example # API key templates
|
|
204
|
+
├── .gitignore # Proper ignores
|
|
205
|
+
├── package.json # With zibby dependencies
|
|
206
|
+
├── playwright.config.js # Auto-configured
|
|
207
|
+
├── README.md # Full instructions
|
|
208
|
+
├── test-specs/ # Test specifications
|
|
209
|
+
│ └── examples/
|
|
210
|
+
│ └── google-search.txt
|
|
211
|
+
├── tests/ # Generated tests (created on first run)
|
|
212
|
+
└── test-results/ # Test artifacts
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
## Configuration
|
|
216
|
+
|
|
217
|
+
The `.zibby.config.js` file:
|
|
218
|
+
|
|
219
|
+
```javascript
|
|
220
|
+
export default {
|
|
221
|
+
agent: {
|
|
222
|
+
provider: 'claude', // or 'cursor'
|
|
223
|
+
|
|
224
|
+
claude: {
|
|
225
|
+
model: 'claude-sonnet-4-20250514',
|
|
226
|
+
maxTokens: 4096,
|
|
227
|
+
},
|
|
228
|
+
},
|
|
229
|
+
|
|
230
|
+
paths: {
|
|
231
|
+
specs: 'test-specs',
|
|
232
|
+
generated: 'tests',
|
|
233
|
+
},
|
|
234
|
+
|
|
235
|
+
context: {
|
|
236
|
+
discovery: {
|
|
237
|
+
global: 'CONTEXT.md',
|
|
238
|
+
pathBased: 'test-specs/{segment}/CONTEXT.md',
|
|
239
|
+
env: `env-${process.env.ENV || 'local'}.js`,
|
|
240
|
+
}
|
|
241
|
+
},
|
|
242
|
+
|
|
243
|
+
// Video recording (affects playwright.config.js generation)
|
|
244
|
+
video: 'on', // Options: 'off', 'on', 'retain-on-failure', 'on-first-retry'
|
|
245
|
+
|
|
246
|
+
// Browser viewport size for video recording and testing
|
|
247
|
+
// Default: 1280x720 (works well on most screens)
|
|
248
|
+
// For larger displays: { width: 1920, height: 1080 }
|
|
249
|
+
// For mobile testing: { width: 375, height: 667 } (iPhone SE)
|
|
250
|
+
viewport: { width: 1280, height: 720 },
|
|
251
|
+
|
|
252
|
+
// Playwright artifacts (screenshots, traces, videos)
|
|
253
|
+
// Set to false to disable, or a path like 'test-results/playwright-artifacts' to enable
|
|
254
|
+
playwrightArtifacts: false,
|
|
255
|
+
|
|
256
|
+
// Cloud sync - auto-upload test results & videos
|
|
257
|
+
cloudSync: false
|
|
258
|
+
};
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
## Examples
|
|
262
|
+
|
|
263
|
+
### Generate Project and Run Test
|
|
264
|
+
|
|
265
|
+
```bash
|
|
266
|
+
# Generate project
|
|
267
|
+
zibby init my-tests
|
|
268
|
+
cd my-tests
|
|
269
|
+
|
|
270
|
+
# Add API key
|
|
271
|
+
cp .env.example .env
|
|
272
|
+
# Edit .env: ANTHROPIC_API_KEY=sk-ant-...
|
|
273
|
+
|
|
274
|
+
# Run example test
|
|
275
|
+
zibby run test-specs/examples/google-search.txt
|
|
276
|
+
|
|
277
|
+
# Run with cloud sync and open results
|
|
278
|
+
zibby run test-specs/examples/google-search.txt \
|
|
279
|
+
--project zby_xxx \
|
|
280
|
+
--collection "My Tests" \
|
|
281
|
+
--open
|
|
282
|
+
|
|
283
|
+
# Organize videos
|
|
284
|
+
zibby video
|
|
285
|
+
|
|
286
|
+
# Run generated test
|
|
287
|
+
npx playwright test tests/examples/google-search.spec.js
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
### CI/CD Usage
|
|
291
|
+
|
|
292
|
+
#### With Claude Agent (Simple)
|
|
293
|
+
|
|
294
|
+
```bash
|
|
295
|
+
# Just set API key in CI environment
|
|
296
|
+
export ANTHROPIC_API_KEY=sk-ant-...
|
|
297
|
+
|
|
298
|
+
# Run tests
|
|
299
|
+
zibby run test-specs/auth/login.txt --agent=claude --headless
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
#### With Cursor Agent (Requires Setup)
|
|
303
|
+
|
|
304
|
+
```bash
|
|
305
|
+
# 1. Install cursor-agent (in CI container/runner)
|
|
306
|
+
curl https://cursor.com/install -fsS | bash
|
|
307
|
+
|
|
308
|
+
# 2. Set Cursor API key
|
|
309
|
+
export CURSOR_API_KEY=your-cursor-token-here
|
|
310
|
+
|
|
311
|
+
# 3. Patch cursor-agent for auto-approval (one-time setup)
|
|
312
|
+
zibby ci-setup
|
|
313
|
+
|
|
314
|
+
# 4. Get and save approval keys (optional, for faster subsequent runs)
|
|
315
|
+
zibby ci-setup --get-keys --save
|
|
316
|
+
|
|
317
|
+
# 5. Run tests
|
|
318
|
+
zibby run test-specs/auth/login.txt \
|
|
319
|
+
--agent=cursor \
|
|
320
|
+
--auto-approve \
|
|
321
|
+
--headless
|
|
322
|
+
|
|
323
|
+
# 6. Upload results to cloud (optional)
|
|
324
|
+
zibby run test-specs/auth/login.txt \
|
|
325
|
+
--agent=cursor \
|
|
326
|
+
--auto-approve \
|
|
327
|
+
--headless \
|
|
328
|
+
--sync \
|
|
329
|
+
--collection="CI Tests"
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
**GitHub Actions Example:**
|
|
333
|
+
|
|
334
|
+
```yaml
|
|
335
|
+
name: Test with Cursor Agent
|
|
336
|
+
|
|
337
|
+
on: [push]
|
|
338
|
+
|
|
339
|
+
jobs:
|
|
340
|
+
test:
|
|
341
|
+
runs-on: ubuntu-latest
|
|
342
|
+
steps:
|
|
343
|
+
- uses: actions/checkout@v3
|
|
344
|
+
|
|
345
|
+
- name: Setup Node.js
|
|
346
|
+
uses: actions/setup-node@v3
|
|
347
|
+
with:
|
|
348
|
+
node-version: '18'
|
|
349
|
+
|
|
350
|
+
- name: Install dependencies
|
|
351
|
+
run: npm install
|
|
352
|
+
|
|
353
|
+
- name: Install cursor-agent
|
|
354
|
+
run: curl https://cursor.com/install -fsS | bash
|
|
355
|
+
|
|
356
|
+
- name: Setup Cursor Agent for CI
|
|
357
|
+
run: npx zibby ci-setup
|
|
358
|
+
env:
|
|
359
|
+
CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }}
|
|
360
|
+
|
|
361
|
+
- name: Run tests
|
|
362
|
+
run: |
|
|
363
|
+
npx zibby run test-specs/auth/login.txt \
|
|
364
|
+
--agent=cursor \
|
|
365
|
+
--auto-approve \
|
|
366
|
+
--headless \
|
|
367
|
+
--sync
|
|
368
|
+
env:
|
|
369
|
+
CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }}
|
|
370
|
+
ZIBBY_API_KEY: ${{ secrets.ZIBBY_API_KEY }}
|
|
371
|
+
```
|
|
372
|
+
|
|
373
|
+
## Using the Cursor Agent
|
|
374
|
+
|
|
375
|
+
The Cursor Agent integrates with the cursor-agent CLI to leverage Cursor's AI capabilities for test generation. This is ideal for teams already using Cursor IDE.
|
|
376
|
+
|
|
377
|
+
### Setup for Cursor Agent
|
|
378
|
+
|
|
379
|
+
#### Local Development (with Cursor IDE)
|
|
380
|
+
|
|
381
|
+
If you have Cursor IDE installed, no additional setup is needed! The CLI will use your stored Cursor credentials automatically.
|
|
382
|
+
|
|
383
|
+
```bash
|
|
384
|
+
# 1. Initialize project with cursor agent
|
|
385
|
+
zibby init my-tests --agent=cursor
|
|
386
|
+
|
|
387
|
+
# 2. Run tests
|
|
388
|
+
cd my-tests
|
|
389
|
+
zibby run test-specs/examples/google-search.txt --agent=cursor
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
#### CI/CD Setup
|
|
393
|
+
|
|
394
|
+
For CI/CD environments, you need to:
|
|
395
|
+
|
|
396
|
+
1. **Install cursor-agent CLI:**
|
|
397
|
+
```bash
|
|
398
|
+
curl https://cursor.com/install -fsS | bash
|
|
399
|
+
# or
|
|
400
|
+
npm install -g cursor-agent
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
2. **Get your Cursor API token:**
|
|
404
|
+
- Visit https://cursor.com/settings
|
|
405
|
+
- Copy your API token
|
|
406
|
+
|
|
407
|
+
3. **Set environment variable:**
|
|
408
|
+
```bash
|
|
409
|
+
export CURSOR_API_KEY=your-cursor-token-here
|
|
410
|
+
```
|
|
411
|
+
|
|
412
|
+
4. **Patch cursor-agent for auto-approval:**
|
|
413
|
+
```bash
|
|
414
|
+
zibby ci-setup
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
This patches the cursor-agent binary to automatically approve MCP tool calls, which is required for automated pipelines.
|
|
418
|
+
|
|
419
|
+
### Cursor Agent Examples
|
|
420
|
+
|
|
421
|
+
#### Basic Test Generation
|
|
422
|
+
|
|
423
|
+
```bash
|
|
424
|
+
# Generate test using Cursor Agent
|
|
425
|
+
zibby run test-specs/auth/login.txt --agent=cursor
|
|
426
|
+
|
|
427
|
+
# With options and auto-open results
|
|
428
|
+
zibby run test-specs/auth/login.txt \
|
|
429
|
+
--agent=cursor \
|
|
430
|
+
--headless \
|
|
431
|
+
--collection="Auth Tests" \
|
|
432
|
+
--open
|
|
433
|
+
|
|
434
|
+
# Upload to specific project
|
|
435
|
+
zibby run test-specs/auth/login.txt \
|
|
436
|
+
--agent=cursor \
|
|
437
|
+
--project zby_your_project_id \
|
|
438
|
+
--collection="My Tests" \
|
|
439
|
+
-o
|
|
440
|
+
```
|
|
441
|
+
|
|
442
|
+
#### Running in CI/CD
|
|
443
|
+
|
|
444
|
+
```bash
|
|
445
|
+
# Complete CI/CD workflow
|
|
446
|
+
zibby run test-specs/auth/login.txt \
|
|
447
|
+
--agent=cursor \
|
|
448
|
+
--auto-approve \
|
|
449
|
+
--headless \
|
|
450
|
+
--sync
|
|
451
|
+
```
|
|
452
|
+
|
|
453
|
+
#### Running Specific Workflow Nodes
|
|
454
|
+
|
|
455
|
+
```bash
|
|
456
|
+
# Run only live execution (useful for debugging)
|
|
457
|
+
zibby run test-specs/auth/login.txt \
|
|
458
|
+
--agent=cursor \
|
|
459
|
+
--node=execute_live
|
|
460
|
+
|
|
461
|
+
# Resume from last session and run script generation
|
|
462
|
+
zibby run test-specs/auth/login.txt \
|
|
463
|
+
--agent=cursor \
|
|
464
|
+
--node=generate_script \
|
|
465
|
+
--session=last
|
|
466
|
+
```
|
|
467
|
+
|
|
468
|
+
#### Using Custom Workflows
|
|
469
|
+
|
|
470
|
+
```bash
|
|
471
|
+
# Download workflow from cloud
|
|
472
|
+
zibby workflow download --type=run_test --include-default
|
|
473
|
+
|
|
474
|
+
# Modify .zibby/workflow-run_test.json as needed
|
|
475
|
+
|
|
476
|
+
# Run with custom workflow
|
|
477
|
+
zibby run test-specs/auth/login.txt \
|
|
478
|
+
--agent=cursor \
|
|
479
|
+
--workflow=CustomWorkflow
|
|
480
|
+
```
|
|
481
|
+
|
|
482
|
+
### Cursor Agent vs Claude Agent
|
|
483
|
+
|
|
484
|
+
| Feature | Cursor Agent | Claude Agent |
|
|
485
|
+
|---------|--------------|--------------|
|
|
486
|
+
| Setup | Requires cursor-agent CLI | Just API key |
|
|
487
|
+
| Local Dev | Uses IDE credentials | Requires API key |
|
|
488
|
+
| CI/CD | Requires patching + token | Just API key |
|
|
489
|
+
| Speed | Depends on IDE connection | Direct API (faster) |
|
|
490
|
+
| Cost | Uses Cursor subscription | Pay per token |
|
|
491
|
+
| Best For | Teams using Cursor IDE | Pure CLI workflows |
|
|
492
|
+
|
|
493
|
+
### Approval Keys
|
|
494
|
+
|
|
495
|
+
Cursor Agent uses MCP tool approval keys to determine which tools can be auto-approved. You can manage these keys:
|
|
496
|
+
|
|
497
|
+
```bash
|
|
498
|
+
# Get current approval keys
|
|
499
|
+
zibby ci-setup --get-keys
|
|
500
|
+
|
|
501
|
+
# Save to project
|
|
502
|
+
zibby ci-setup --get-keys --save
|
|
503
|
+
|
|
504
|
+
# Keys are saved to:
|
|
505
|
+
# - .mcp/approval-keys.json (project-specific)
|
|
506
|
+
# - ~/.cursor-agent/approval-keys.json (global fallback)
|
|
507
|
+
```
|
|
508
|
+
|
|
509
|
+
### Troubleshooting Cursor Agent
|
|
510
|
+
|
|
511
|
+
#### "cursor-agent not found"
|
|
512
|
+
|
|
513
|
+
```bash
|
|
514
|
+
# Install cursor-agent
|
|
515
|
+
curl https://cursor.com/install -fsS | bash
|
|
516
|
+
|
|
517
|
+
# Verify installation
|
|
518
|
+
cursor-agent --version
|
|
519
|
+
```
|
|
520
|
+
|
|
521
|
+
#### "CURSOR_API_KEY not set" (CI/CD only)
|
|
522
|
+
|
|
523
|
+
```bash
|
|
524
|
+
# Get token from https://cursor.com/settings
|
|
525
|
+
export CURSOR_API_KEY=your-token-here
|
|
526
|
+
```
|
|
527
|
+
|
|
528
|
+
#### "Tool approval required" (CI/CD)
|
|
529
|
+
|
|
530
|
+
```bash
|
|
531
|
+
# Patch cursor-agent for auto-approval
|
|
532
|
+
zibby ci-setup
|
|
533
|
+
|
|
534
|
+
# Verify patch
|
|
535
|
+
zibby ci-setup --get-keys
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
#### Tests hang or timeout
|
|
539
|
+
|
|
540
|
+
```bash
|
|
541
|
+
# Run in headed mode to see what's happening
|
|
542
|
+
zibby run test-specs/auth/login.txt --agent=cursor
|
|
543
|
+
|
|
544
|
+
# Check session logs
|
|
545
|
+
ls -la test-results/sessions/
|
|
546
|
+
```
|
|
547
|
+
|
|
548
|
+
#### MCP connection errors
|
|
549
|
+
|
|
550
|
+
```bash
|
|
551
|
+
# Ensure Playwright MCP is set up
|
|
552
|
+
zibby setup-playwright --headed
|
|
553
|
+
|
|
554
|
+
# Check MCP config
|
|
555
|
+
cat .mcp/config.json
|
|
556
|
+
```
|
|
557
|
+
|
|
558
|
+
## Features
|
|
559
|
+
|
|
560
|
+
✅ **Interactive Prompts** - Choose agent, MCP, cloud sync
|
|
561
|
+
✅ **Zero Configuration** - Works out of the box
|
|
562
|
+
✅ **Beautiful CLI** - Colorful output with progress indicators
|
|
563
|
+
✅ **CI/CD Ready** - Auto-approval for automated pipelines
|
|
564
|
+
✅ **Video Organization** - Keep videos next to tests
|
|
565
|
+
✅ **Multi-Agent Support** - Claude (API) or Cursor (CLI)
|
|
566
|
+
|
|
567
|
+
## Environment Variables
|
|
568
|
+
|
|
569
|
+
### Agent Configuration
|
|
570
|
+
|
|
571
|
+
```bash
|
|
572
|
+
# ============================================
|
|
573
|
+
# Claude Agent (Anthropic API)
|
|
574
|
+
# ============================================
|
|
575
|
+
ANTHROPIC_API_KEY=sk-ant-...
|
|
576
|
+
# Required: Always
|
|
577
|
+
# Get from: https://console.anthropic.com/
|
|
578
|
+
|
|
579
|
+
# ============================================
|
|
580
|
+
# Cursor Agent (cursor-agent CLI)
|
|
581
|
+
# ============================================
|
|
582
|
+
CURSOR_API_KEY=your-cursor-token-here
|
|
583
|
+
# Required: Only in CI/CD (not needed locally with Cursor IDE)
|
|
584
|
+
# Get from: https://cursor.com/settings
|
|
585
|
+
# Install cursor-agent: curl https://cursor.com/install -fsS | bash
|
|
586
|
+
|
|
587
|
+
# ============================================
|
|
588
|
+
# Optional: Cloud Sync
|
|
589
|
+
# ============================================
|
|
590
|
+
ZIBBY_API_KEY=zby_live_...
|
|
591
|
+
# Optional: For uploading test results and videos to Zibby Cloud
|
|
592
|
+
# Get from: https://app.zibby.com/projects (copy project API key)
|
|
593
|
+
|
|
594
|
+
ZIBBY_PROJECT_ID=proj_...
|
|
595
|
+
# Optional: Specific project ID (auto-detected from API key if not set)
|
|
596
|
+
|
|
597
|
+
ZIBBY_TENANT_ID=org_...
|
|
598
|
+
# Optional: Organization/tenant ID
|
|
599
|
+
```
|
|
600
|
+
|
|
601
|
+
### Complete .env.example
|
|
602
|
+
|
|
603
|
+
```bash
|
|
604
|
+
# Copy this to .env and fill in your keys
|
|
605
|
+
|
|
606
|
+
# ===========================================
|
|
607
|
+
# AI AGENT (choose one)
|
|
608
|
+
# ===========================================
|
|
609
|
+
|
|
610
|
+
# Option 1: Claude Agent (recommended for CI/CD)
|
|
611
|
+
ANTHROPIC_API_KEY=sk-ant-your-key-here
|
|
612
|
+
|
|
613
|
+
# Option 2: Cursor Agent (for teams using Cursor IDE)
|
|
614
|
+
# Local: Not needed (uses IDE credentials)
|
|
615
|
+
# CI/CD: Required
|
|
616
|
+
# CURSOR_API_KEY=your-cursor-token-here
|
|
617
|
+
|
|
618
|
+
# ===========================================
|
|
619
|
+
# CLOUD SYNC (optional)
|
|
620
|
+
# ===========================================
|
|
621
|
+
|
|
622
|
+
# Upload test results to Zibby Cloud
|
|
623
|
+
# ZIBBY_API_KEY=zby_live_your-project-key
|
|
624
|
+
# ZIBBY_PROJECT_ID=proj_your-project-id
|
|
625
|
+
|
|
626
|
+
# ===========================================
|
|
627
|
+
# ENVIRONMENT (optional)
|
|
628
|
+
# ===========================================
|
|
629
|
+
|
|
630
|
+
# For environment-specific configs
|
|
631
|
+
# NODE_ENV=development|production|staging
|
|
632
|
+
```
|
|
633
|
+
|
|
634
|
+
## Advanced Cursor Agent Usage
|
|
635
|
+
|
|
636
|
+
### Running Multiple Tests
|
|
637
|
+
|
|
638
|
+
```bash
|
|
639
|
+
# Run all tests in a folder
|
|
640
|
+
for spec in test-specs/auth/*.txt; do
|
|
641
|
+
zibby run "$spec" --agent=cursor --headless --sync
|
|
642
|
+
done
|
|
643
|
+
|
|
644
|
+
# Or use a workflow system
|
|
645
|
+
zibby workflow download --type=run_test
|
|
646
|
+
# Edit workflow to run multiple specs
|
|
647
|
+
zibby run test-specs/auth/login.txt --workflow=BatchRunner
|
|
648
|
+
```
|
|
649
|
+
|
|
650
|
+
### Custom Approval Keys
|
|
651
|
+
|
|
652
|
+
By default, cursor-agent requires approval for each MCP tool call. For CI/CD, you can configure which tools are auto-approved:
|
|
653
|
+
|
|
654
|
+
```bash
|
|
655
|
+
# 1. Get current keys
|
|
656
|
+
zibby ci-setup --get-keys
|
|
657
|
+
|
|
658
|
+
# 2. Edit approval-keys.json
|
|
659
|
+
cat > .mcp/approval-keys.json << 'EOF'
|
|
660
|
+
{
|
|
661
|
+
"playwright": {
|
|
662
|
+
"browser_navigate": true,
|
|
663
|
+
"browser_click": true,
|
|
664
|
+
"browser_type": true,
|
|
665
|
+
"browser_screenshot": true,
|
|
666
|
+
"browser_close": true
|
|
667
|
+
}
|
|
668
|
+
}
|
|
669
|
+
EOF
|
|
670
|
+
|
|
671
|
+
# 3. Use in CI
|
|
672
|
+
zibby run test-specs/auth/login.txt --agent=cursor --auto-approve
|
|
673
|
+
```
|
|
674
|
+
|
|
675
|
+
### Session Management
|
|
676
|
+
|
|
677
|
+
Cursor Agent creates session folders for each run. You can reuse sessions for debugging:
|
|
678
|
+
|
|
679
|
+
```bash
|
|
680
|
+
# 1. Run test (creates session folder)
|
|
681
|
+
zibby run test-specs/auth/login.txt --agent=cursor
|
|
682
|
+
|
|
683
|
+
# Session saved to: test-results/sessions/1709567890/
|
|
684
|
+
|
|
685
|
+
# 2. View session files
|
|
686
|
+
ls test-results/sessions/1709567890/execute_live/
|
|
687
|
+
# events.json - All MCP tool calls and responses
|
|
688
|
+
# videos/ - Recorded video(s)
|
|
689
|
+
# title.txt - Generated test name
|
|
690
|
+
|
|
691
|
+
# 3. Resume from specific session
|
|
692
|
+
zibby run test-specs/auth/login.txt \
|
|
693
|
+
--agent=cursor \
|
|
694
|
+
--node=generate_script \
|
|
695
|
+
--session=1709567890
|
|
696
|
+
|
|
697
|
+
# 4. Or use "last" to resume from most recent
|
|
698
|
+
zibby run test-specs/auth/login.txt \
|
|
699
|
+
--agent=cursor \
|
|
700
|
+
--node=generate_script \
|
|
701
|
+
--session=last
|
|
702
|
+
```
|
|
703
|
+
|
|
704
|
+
### Debugging Cursor Agent
|
|
705
|
+
|
|
706
|
+
```bash
|
|
707
|
+
# 1. Run in headed mode (see browser)
|
|
708
|
+
zibby run test-specs/auth/login.txt --agent=cursor
|
|
709
|
+
|
|
710
|
+
# 2. Check cursor-agent logs
|
|
711
|
+
tail -f ~/.cursor-agent/logs/agent.log
|
|
712
|
+
|
|
713
|
+
# 3. Inspect MCP tool calls
|
|
714
|
+
cat test-results/sessions/*/execute_live/events.json | jq '.[] | select(.type == "tool_call")'
|
|
715
|
+
|
|
716
|
+
# 4. Verify MCP setup
|
|
717
|
+
cat .mcp/config.json
|
|
718
|
+
|
|
719
|
+
# 5. Test MCP connection
|
|
720
|
+
cursor-agent --test-mcp
|
|
721
|
+
```
|
|
722
|
+
|
|
723
|
+
### Performance Optimization
|
|
724
|
+
|
|
725
|
+
```bash
|
|
726
|
+
# Use headless mode (faster)
|
|
727
|
+
zibby run test-specs/auth/login.txt --agent=cursor --headless
|
|
728
|
+
|
|
729
|
+
# Skip video recording
|
|
730
|
+
# Edit .zibby.config.js:
|
|
731
|
+
# video: 'off'
|
|
732
|
+
|
|
733
|
+
# Run specific nodes only
|
|
734
|
+
zibby run test-specs/auth/login.txt \
|
|
735
|
+
--agent=cursor \
|
|
736
|
+
--node=execute_live # Skip script generation & verification
|
|
737
|
+
|
|
738
|
+
# Reuse existing session
|
|
739
|
+
zibby run test-specs/auth/login.txt \
|
|
740
|
+
--agent=cursor \
|
|
741
|
+
--node=generate_script \
|
|
742
|
+
--session=last # No need to execute live again
|
|
743
|
+
```
|
|
744
|
+
|
|
745
|
+
### Best Practices for Cursor Agent
|
|
746
|
+
|
|
747
|
+
1. **Local Development**: Use Cursor IDE credentials (no CURSOR_API_KEY needed)
|
|
748
|
+
2. **CI/CD**: Always set CURSOR_API_KEY and patch cursor-agent
|
|
749
|
+
3. **Auto-Approval**: Use `--auto-approve` flag in CI/CD only
|
|
750
|
+
4. **Headless Mode**: Always use `--headless` in CI/CD for performance
|
|
751
|
+
5. **Session Reuse**: Use `--session=last` to debug without re-running
|
|
752
|
+
6. **Cloud Sync**: Use `--sync` to automatically upload results
|
|
753
|
+
7. **Collections**: Use `--collection` to organize tests by feature/module
|
|
754
|
+
|
|
755
|
+
### Comparing Agents
|
|
756
|
+
|
|
757
|
+
When to use **Cursor Agent**:
|
|
758
|
+
- ✅ Your team uses Cursor IDE
|
|
759
|
+
- ✅ You want to leverage Cursor's AI capabilities
|
|
760
|
+
- ✅ You're okay with CLI-based execution
|
|
761
|
+
- ✅ You have a Cursor subscription
|
|
762
|
+
|
|
763
|
+
When to use **Claude Agent**:
|
|
764
|
+
- ✅ You want the simplest setup
|
|
765
|
+
- ✅ You prefer direct API access
|
|
766
|
+
- ✅ You need faster execution
|
|
767
|
+
- ✅ You're running in pure CI/CD (no IDE)
|
|
768
|
+
|
|
769
|
+
**Recommendation**: Start with Claude Agent for simplicity. Switch to Cursor Agent if your team already uses Cursor IDE and wants to leverage its AI capabilities.
|
|
770
|
+
|
|
771
|
+
## Opening Test Results
|
|
772
|
+
|
|
773
|
+
The `--open` (or `-o`) flag automatically opens test results in your browser after upload completes:
|
|
774
|
+
|
|
775
|
+
```bash
|
|
776
|
+
# Opens browser to test results page
|
|
777
|
+
zibby run test-specs/auth/login.txt \
|
|
778
|
+
--project zby_xxx \
|
|
779
|
+
--collection "My Tests" \
|
|
780
|
+
--open
|
|
781
|
+
|
|
782
|
+
# Short form
|
|
783
|
+
zibby run test-specs/auth/login.txt --project zby_xxx -o
|
|
784
|
+
```
|
|
785
|
+
|
|
786
|
+
**Environment-aware URLs:**
|
|
787
|
+
- Local: `http://localhost:3000/projects/{projectId}/runs/{executionId}`
|
|
788
|
+
- Staging: `https://app-staging.zibby.app/projects/{projectId}/runs/{executionId}`
|
|
789
|
+
- Production: `https://app.zibby.app/projects/{projectId}/runs/{executionId}`
|
|
790
|
+
|
|
791
|
+
**Environment configuration:**
|
|
792
|
+
```bash
|
|
793
|
+
# Default: local
|
|
794
|
+
ZIBBY_ENV=local
|
|
795
|
+
|
|
796
|
+
# Staging
|
|
797
|
+
ZIBBY_ENV=staging
|
|
798
|
+
ZIBBY_STAGING_FRONTEND_URL=https://app-staging.zibby.app
|
|
799
|
+
|
|
800
|
+
# Production
|
|
801
|
+
ZIBBY_ENV=prod
|
|
802
|
+
ZIBBY_PROD_FRONTEND_URL=https://app.zibby.app
|
|
803
|
+
```
|
|
804
|
+
|
|
805
|
+
**Note:** The `--open` flag only works when uploading to cloud (requires `--project` or `--sync` flag). It will not open anything for local-only runs.
|
|
806
|
+
|
|
807
|
+
## Quick Reference - Cursor Agent Commands
|
|
808
|
+
|
|
809
|
+
```bash
|
|
810
|
+
# ============================================
|
|
811
|
+
# SETUP
|
|
812
|
+
# ============================================
|
|
813
|
+
|
|
814
|
+
# Install cursor-agent (CI/CD only)
|
|
815
|
+
curl https://cursor.com/install -fsS | bash
|
|
816
|
+
|
|
817
|
+
# Patch for CI/CD
|
|
818
|
+
zibby ci-setup
|
|
819
|
+
|
|
820
|
+
# Get approval keys
|
|
821
|
+
zibby ci-setup --get-keys --save
|
|
822
|
+
|
|
823
|
+
# Setup Playwright MCP
|
|
824
|
+
zibby setup-playwright --headed
|
|
825
|
+
|
|
826
|
+
|
|
827
|
+
# ============================================
|
|
828
|
+
# RUN TESTS
|
|
829
|
+
# ============================================
|
|
830
|
+
|
|
831
|
+
# Basic run (local)
|
|
832
|
+
zibby run test-specs/auth/login.txt --agent=cursor
|
|
833
|
+
|
|
834
|
+
# With options and open results
|
|
835
|
+
zibby run test-specs/auth/login.txt \
|
|
836
|
+
--agent=cursor \
|
|
837
|
+
--headless \
|
|
838
|
+
--collection="Auth Tests" \
|
|
839
|
+
--open
|
|
840
|
+
|
|
841
|
+
# Quick open with short flag
|
|
842
|
+
zibby run test-specs/auth/login.txt \
|
|
843
|
+
--project zby_xxx \
|
|
844
|
+
-o
|
|
845
|
+
|
|
846
|
+
# CI/CD run
|
|
847
|
+
zibby run test-specs/auth/login.txt \
|
|
848
|
+
--agent=cursor \
|
|
849
|
+
--auto-approve \
|
|
850
|
+
--headless \
|
|
851
|
+
--sync
|
|
852
|
+
|
|
853
|
+
|
|
854
|
+
# ============================================
|
|
855
|
+
# DEBUGGING
|
|
856
|
+
# ============================================
|
|
857
|
+
|
|
858
|
+
# Run in headed mode (see browser)
|
|
859
|
+
zibby run test-specs/auth/login.txt --agent=cursor
|
|
860
|
+
|
|
861
|
+
# Run specific node
|
|
862
|
+
zibby run test-specs/auth/login.txt \
|
|
863
|
+
--agent=cursor \
|
|
864
|
+
--node=execute_live
|
|
865
|
+
|
|
866
|
+
# Resume from last session
|
|
867
|
+
zibby run test-specs/auth/login.txt \
|
|
868
|
+
--agent=cursor \
|
|
869
|
+
--node=generate_script \
|
|
870
|
+
--session=last
|
|
871
|
+
|
|
872
|
+
# Check logs
|
|
873
|
+
tail -f ~/.cursor-agent/logs/agent.log
|
|
874
|
+
cat test-results/sessions/*/execute_live/events.json
|
|
875
|
+
|
|
876
|
+
|
|
877
|
+
# ============================================
|
|
878
|
+
# WORKFLOWS
|
|
879
|
+
# ============================================
|
|
880
|
+
|
|
881
|
+
# Download workflow
|
|
882
|
+
zibby workflow download --type=run_test
|
|
883
|
+
|
|
884
|
+
# Run with custom workflow
|
|
885
|
+
zibby run test-specs/auth/login.txt \
|
|
886
|
+
--agent=cursor \
|
|
887
|
+
--workflow=CustomWorkflow
|
|
888
|
+
|
|
889
|
+
# List workflows
|
|
890
|
+
zibby workflow list
|
|
891
|
+
|
|
892
|
+
|
|
893
|
+
# ============================================
|
|
894
|
+
# ENVIRONMENT SETUP
|
|
895
|
+
# ============================================
|
|
896
|
+
|
|
897
|
+
# Local development (.env)
|
|
898
|
+
ANTHROPIC_API_KEY=sk-ant-... # For Claude
|
|
899
|
+
# No CURSOR_API_KEY needed if Cursor IDE installed
|
|
900
|
+
|
|
901
|
+
# CI/CD (.env or secrets)
|
|
902
|
+
CURSOR_API_KEY=your-token # Required in CI/CD
|
|
903
|
+
ZIBBY_API_KEY=zby_live_... # For cloud sync
|
|
904
|
+
ZIBBY_PROJECT_ID=proj_... # Optional
|
|
905
|
+
```
|
|
906
|
+
|
|
907
|
+
## Troubleshooting Reference
|
|
908
|
+
|
|
909
|
+
| Error | Solution |
|
|
910
|
+
|-------|----------|
|
|
911
|
+
| `cursor-agent not found` | Install: `curl https://cursor.com/install -fsS \| bash` |
|
|
912
|
+
| `CURSOR_API_KEY not set` | Set env var from https://cursor.com/settings |
|
|
913
|
+
| `Tool approval required` | Run `zibby ci-setup` |
|
|
914
|
+
| Tests hang | Run in headed mode to see browser |
|
|
915
|
+
| MCP connection errors | Run `zibby setup-playwright --headed` |
|
|
916
|
+
| Session files missing | Check `test-results/sessions/` folder |
|
|
917
|
+
| Video not uploaded | Ensure `--sync` flag and ZIBBY_API_KEY set |
|
|
918
|
+
| Browser doesn't open with `--open` | Check ZIBBY_ENV and frontend URL settings |
|
|
919
|
+
| `--open` does nothing | Ensure upload succeeded (requires `--project` or `--sync`) |
|
|
920
|
+
| `Not a member of this project` | Run `zibby login` to authenticate as project member |
|
|
921
|
+
| `Invalid user token` | User JWT expired, run `zibby login` again |
|
|
922
|
+
|
|
923
|
+
## License
|
|
924
|
+
|
|
925
|
+
MIT
|
|
926
|
+
|