runhuman 1.3.1 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +320 -244
- package/dist/index.js +108 -199
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -28,19 +28,39 @@ npx runhuman --help
|
|
|
28
28
|
## Quick Start
|
|
29
29
|
|
|
30
30
|
```bash
|
|
31
|
-
# 1. Login
|
|
31
|
+
# 1. Login (opens browser for OAuth)
|
|
32
32
|
runhuman login
|
|
33
33
|
|
|
34
34
|
# 2. Create your first test
|
|
35
|
-
runhuman create https://
|
|
35
|
+
runhuman create https://google.com \
|
|
36
|
+
-d "Search for 'recursion' and confirm the tester experiences a joke about recursion"
|
|
36
37
|
|
|
37
38
|
# 3. Check the status
|
|
38
|
-
runhuman status
|
|
39
|
+
runhuman status job_abc123
|
|
39
40
|
|
|
40
41
|
# 4. Get results
|
|
41
|
-
runhuman results
|
|
42
|
+
runhuman results job_abc123
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Truncated IDs
|
|
46
|
+
|
|
47
|
+
All commands accept truncated ID prefixes, similar to git short hashes. Instead of typing a full ID, you can use just the first few characters:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
# Full ID
|
|
51
|
+
runhuman status 712e42ad-8f3b-4a1c-9d5e-1234567890ab
|
|
52
|
+
|
|
53
|
+
# Truncated — resolves to the same job
|
|
54
|
+
runhuman status 712e
|
|
55
|
+
|
|
56
|
+
# Hyphens are ignored during matching
|
|
57
|
+
runhuman status 712e42ad8f3b
|
|
42
58
|
```
|
|
43
59
|
|
|
60
|
+
If a prefix matches multiple resources, you'll be prompted to provide more characters. Matching is case-insensitive and hyphen-insensitive.
|
|
61
|
+
|
|
62
|
+
**Destructive operations** (`projects delete`, `keys delete`, `projects transfer --to-org`) require the full ID to prevent accidental mismatches.
|
|
63
|
+
|
|
44
64
|
## Commands
|
|
45
65
|
|
|
46
66
|
### Jobs
|
|
@@ -51,26 +71,41 @@ Create a new QA test job.
|
|
|
51
71
|
|
|
52
72
|
```bash
|
|
53
73
|
# Basic usage
|
|
54
|
-
runhuman create https://
|
|
74
|
+
runhuman create https://google.com \
|
|
75
|
+
-d "Search for 'recursion' and confirm the tester experiences a joke about recursion"
|
|
55
76
|
|
|
56
77
|
# With template
|
|
57
|
-
runhuman create https://
|
|
78
|
+
runhuman create https://google.com --template tmpl_abc123
|
|
58
79
|
|
|
59
80
|
# Synchronous (wait for result)
|
|
60
|
-
runhuman create https://
|
|
81
|
+
runhuman create https://google.com -d "Test search" --sync
|
|
61
82
|
|
|
62
83
|
# With custom schema
|
|
63
|
-
runhuman create https://
|
|
84
|
+
runhuman create https://google.com -d "Test search" --schema ./schema.json
|
|
85
|
+
|
|
86
|
+
# With tester pool requirements
|
|
87
|
+
runhuman create https://google.com -d "Test mobile layout" \
|
|
88
|
+
--required-devices ios,android \
|
|
89
|
+
--required-languages english
|
|
64
90
|
```
|
|
65
91
|
|
|
66
92
|
**Options:**
|
|
67
93
|
- `-d, --description <text>` - Test instructions for the human tester
|
|
68
|
-
-
|
|
69
|
-
- `--duration <
|
|
70
|
-
- `--
|
|
71
|
-
-
|
|
72
|
-
- `--
|
|
73
|
-
- `--json
|
|
94
|
+
- `-t, --template <name>` - Use a pre-defined template
|
|
95
|
+
- `--duration <minutes>` - Target duration (1-60 minutes)
|
|
96
|
+
- `--device-class <class>` - Device class: desktop or mobile
|
|
97
|
+
- `-S, --schema <file>` - Path to JSON schema for structured output
|
|
98
|
+
- `--schema-inline <json>` - Inline JSON schema string
|
|
99
|
+
- `--metadata <json>` - Metadata for tracking (JSON string)
|
|
100
|
+
- `--github-repo <owner/repo>` - GitHub repo for context
|
|
101
|
+
- `--required-devices <devices>` - Required tester devices (comma-separated: ios, android, pc, mac)
|
|
102
|
+
- `--required-languages <languages>` - Required tester languages (comma-separated: english, spanish)
|
|
103
|
+
- `--require-social-videos` - Require tester to create social videos
|
|
104
|
+
- `-s, --sync` - Wait for result before exiting
|
|
105
|
+
- `--wait <seconds>` - Max wait time in sync mode (default: 300)
|
|
106
|
+
- `-p, --project <id>` - Project ID (or use default from config)
|
|
107
|
+
- `-j, --json` - Output as JSON
|
|
108
|
+
- `-q, --quiet` - Minimal output (only job ID)
|
|
74
109
|
|
|
75
110
|
#### `status <jobId>`
|
|
76
111
|
|
|
@@ -78,11 +113,12 @@ Check the status of a test job.
|
|
|
78
113
|
|
|
79
114
|
```bash
|
|
80
115
|
runhuman status job_abc123
|
|
116
|
+
runhuman status 712e # truncated ID works
|
|
81
117
|
```
|
|
82
118
|
|
|
83
119
|
#### `wait <jobId>`
|
|
84
120
|
|
|
85
|
-
Wait for a job to complete
|
|
121
|
+
Wait for a job to complete with live status updates.
|
|
86
122
|
|
|
87
123
|
```bash
|
|
88
124
|
runhuman wait job_abc123
|
|
@@ -91,6 +127,7 @@ runhuman wait job_abc123 --timeout 300
|
|
|
91
127
|
|
|
92
128
|
**Options:**
|
|
93
129
|
- `--timeout <seconds>` - Maximum wait time (default: 600)
|
|
130
|
+
- `-j, --json` - Output as JSON
|
|
94
131
|
|
|
95
132
|
#### `results <jobId>`
|
|
96
133
|
|
|
@@ -99,31 +136,100 @@ Display detailed test results.
|
|
|
99
136
|
```bash
|
|
100
137
|
runhuman results job_abc123
|
|
101
138
|
runhuman results job_abc123 --json
|
|
139
|
+
runhuman results job_abc123 --schema-only # Only extracted schema data
|
|
140
|
+
runhuman results job_abc123 --raw # Raw tester response
|
|
102
141
|
```
|
|
103
142
|
|
|
104
|
-
|
|
143
|
+
**Options:**
|
|
144
|
+
- `--schema-only` - Show only extracted schema data
|
|
145
|
+
- `--raw` - Show raw tester response without processing
|
|
146
|
+
- `-j, --json` - Output as JSON
|
|
105
147
|
|
|
106
|
-
|
|
148
|
+
#### `list [filter]`
|
|
149
|
+
|
|
150
|
+
List all your test jobs with optional status filtering.
|
|
107
151
|
|
|
108
152
|
```bash
|
|
109
153
|
# List all jobs
|
|
110
154
|
runhuman list
|
|
111
155
|
|
|
112
|
-
# Filter by status
|
|
113
|
-
runhuman list
|
|
156
|
+
# Filter by status (positional argument)
|
|
157
|
+
runhuman list completed
|
|
158
|
+
runhuman list pending
|
|
114
159
|
|
|
115
160
|
# Filter by project
|
|
116
161
|
runhuman list --project proj_abc123
|
|
117
162
|
|
|
118
|
-
#
|
|
119
|
-
runhuman list --limit 20
|
|
163
|
+
# Pagination
|
|
164
|
+
runhuman list --limit 20 --offset 40
|
|
120
165
|
```
|
|
121
166
|
|
|
167
|
+
**Arguments:**
|
|
168
|
+
- `[filter]` - Status filter: all, pending, claimed, in_progress, completed, failed, timeout
|
|
169
|
+
|
|
122
170
|
**Options:**
|
|
123
|
-
-
|
|
124
|
-
-
|
|
125
|
-
- `--
|
|
126
|
-
- `--
|
|
171
|
+
- `-p, --project <id>` - Filter by project
|
|
172
|
+
- `-n, --limit <number>` - Maximum number of jobs (default: 20)
|
|
173
|
+
- `--offset <number>` - Pagination offset (default: 0)
|
|
174
|
+
- `--format <type>` - Output format: table, json, compact
|
|
175
|
+
- `-j, --json` - Output as JSON
|
|
176
|
+
|
|
177
|
+
#### `delete <jobId>`
|
|
178
|
+
|
|
179
|
+
Delete a job permanently.
|
|
180
|
+
|
|
181
|
+
```bash
|
|
182
|
+
runhuman delete job_abc123 --force
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
**Options:**
|
|
186
|
+
- `-f, --force` - Skip confirmation prompt
|
|
187
|
+
- `-j, --json` - Output as JSON
|
|
188
|
+
|
|
189
|
+
### Organizations
|
|
190
|
+
|
|
191
|
+
#### `orgs list`
|
|
192
|
+
|
|
193
|
+
List all organizations you belong to.
|
|
194
|
+
|
|
195
|
+
```bash
|
|
196
|
+
runhuman orgs list
|
|
197
|
+
runhuman orgs ls # alias
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
#### `orgs show <organizationId>`
|
|
201
|
+
|
|
202
|
+
Show organization details.
|
|
203
|
+
|
|
204
|
+
```bash
|
|
205
|
+
runhuman orgs show org_abc123
|
|
206
|
+
runhuman orgs info org_abc123 # alias
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
#### `orgs balance <organizationId>`
|
|
210
|
+
|
|
211
|
+
Check billing balance.
|
|
212
|
+
|
|
213
|
+
```bash
|
|
214
|
+
runhuman orgs balance org_abc123
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
#### `orgs projects <organizationId>`
|
|
218
|
+
|
|
219
|
+
List all projects in an organization.
|
|
220
|
+
|
|
221
|
+
```bash
|
|
222
|
+
runhuman orgs projects org_abc123
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
#### `orgs switch <organizationId>`
|
|
226
|
+
|
|
227
|
+
Set the default organization for CLI commands.
|
|
228
|
+
|
|
229
|
+
```bash
|
|
230
|
+
runhuman orgs switch org_abc123
|
|
231
|
+
runhuman orgs switch org_abc123 --global # all directories (default)
|
|
232
|
+
```
|
|
127
233
|
|
|
128
234
|
### Projects
|
|
129
235
|
|
|
@@ -133,139 +239,212 @@ List all your projects.
|
|
|
133
239
|
|
|
134
240
|
```bash
|
|
135
241
|
runhuman projects list
|
|
136
|
-
runhuman projects ls
|
|
242
|
+
runhuman projects ls # alias
|
|
137
243
|
```
|
|
138
244
|
|
|
139
245
|
#### `projects create <name>`
|
|
140
246
|
|
|
141
|
-
Create a new project.
|
|
247
|
+
Create a new project in an organization.
|
|
142
248
|
|
|
143
249
|
```bash
|
|
144
|
-
runhuman projects create "My App"
|
|
145
|
-
runhuman projects create "My App"
|
|
250
|
+
runhuman projects create "My App" --organization org_abc123
|
|
251
|
+
runhuman projects create "My App" --organization org_abc123 --set-default
|
|
146
252
|
```
|
|
147
253
|
|
|
148
254
|
**Options:**
|
|
149
|
-
- `-
|
|
150
|
-
- `--default-url <url>` - Default URL
|
|
255
|
+
- `-o, --organization <id>` - Organization ID (required)
|
|
256
|
+
- `--default-url <url>` - Default URL for tests
|
|
151
257
|
- `--github-repo <owner/repo>` - Link GitHub repository
|
|
258
|
+
- `--set-default` - Set as default project after creation
|
|
152
259
|
|
|
153
260
|
#### `projects show <projectId>`
|
|
154
261
|
|
|
155
|
-
Show details
|
|
262
|
+
Show project details including organization and balance.
|
|
156
263
|
|
|
157
264
|
```bash
|
|
158
265
|
runhuman projects show proj_abc123
|
|
159
|
-
runhuman projects info proj_abc123
|
|
266
|
+
runhuman projects info proj_abc123 # alias
|
|
267
|
+
runhuman projects get proj_abc123 # alias
|
|
160
268
|
```
|
|
161
269
|
|
|
162
270
|
#### `projects update <projectId>`
|
|
163
271
|
|
|
164
|
-
Update
|
|
272
|
+
Update project settings.
|
|
165
273
|
|
|
166
274
|
```bash
|
|
167
275
|
runhuman projects update proj_abc123 --name "New Name"
|
|
168
|
-
runhuman projects update proj_abc123 --
|
|
276
|
+
runhuman projects update proj_abc123 --default-url https://google.com
|
|
169
277
|
```
|
|
170
278
|
|
|
171
279
|
**Options:**
|
|
172
|
-
-
|
|
173
|
-
- `-
|
|
174
|
-
-
|
|
175
|
-
|
|
280
|
+
- `-n, --name <text>` - New project name
|
|
281
|
+
- `-u, --default-url <url>` - New default URL
|
|
282
|
+
- `-g, --github-repo <owner/repo>` - New GitHub repository
|
|
283
|
+
|
|
284
|
+
#### `projects switch <projectId>`
|
|
285
|
+
|
|
286
|
+
Set the default project for CLI commands.
|
|
287
|
+
|
|
288
|
+
```bash
|
|
289
|
+
runhuman projects switch proj_abc123
|
|
290
|
+
runhuman projects use proj_abc123 # alias
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
**Options:**
|
|
294
|
+
- `-g, --global` - Set as global default instead of local (default: true)
|
|
295
|
+
|
|
296
|
+
#### `projects transfer <projectId>`
|
|
297
|
+
|
|
298
|
+
Transfer a project to another organization.
|
|
299
|
+
|
|
300
|
+
```bash
|
|
301
|
+
runhuman projects transfer proj_abc123 --to-org org_target123
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
**Options:**
|
|
305
|
+
- `--to-org <organizationId>` - Target organization ID (required, full ID required)
|
|
306
|
+
- `-f, --force` - Skip confirmation prompt
|
|
176
307
|
|
|
177
308
|
#### `projects delete <projectId>`
|
|
178
309
|
|
|
179
|
-
Delete a project permanently.
|
|
310
|
+
Delete a project permanently. Requires the full project ID.
|
|
180
311
|
|
|
181
312
|
```bash
|
|
182
313
|
runhuman projects delete proj_abc123 --force
|
|
183
314
|
```
|
|
184
315
|
|
|
185
316
|
**Options:**
|
|
186
|
-
-
|
|
317
|
+
- `-f, --force` - Skip confirmation prompt
|
|
318
|
+
|
|
319
|
+
### Transfers
|
|
320
|
+
|
|
321
|
+
#### `transfers list`
|
|
322
|
+
|
|
323
|
+
List pending and outgoing project transfers.
|
|
324
|
+
|
|
325
|
+
```bash
|
|
326
|
+
runhuman transfers list
|
|
327
|
+
runhuman transfers ls # alias
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
#### `transfers accept <transferId>`
|
|
331
|
+
|
|
332
|
+
Accept an incoming transfer.
|
|
333
|
+
|
|
334
|
+
```bash
|
|
335
|
+
runhuman transfers accept xfer_abc123
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
#### `transfers reject <transferId>`
|
|
339
|
+
|
|
340
|
+
Reject an incoming transfer.
|
|
341
|
+
|
|
342
|
+
```bash
|
|
343
|
+
runhuman transfers reject xfer_abc123
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
#### `transfers cancel <transferId>`
|
|
347
|
+
|
|
348
|
+
Cancel an outgoing transfer you initiated.
|
|
349
|
+
|
|
350
|
+
```bash
|
|
351
|
+
runhuman transfers cancel xfer_abc123
|
|
352
|
+
```
|
|
187
353
|
|
|
188
354
|
### API Keys
|
|
189
355
|
|
|
190
356
|
#### `keys list`
|
|
191
357
|
|
|
192
|
-
List all API keys for
|
|
358
|
+
List all API keys for an organization.
|
|
193
359
|
|
|
194
360
|
```bash
|
|
195
|
-
runhuman keys list --
|
|
196
|
-
runhuman keys ls --
|
|
361
|
+
runhuman keys list --organization org_abc123
|
|
362
|
+
runhuman keys ls --organization org_abc123 # alias
|
|
363
|
+
runhuman keys list --organization org_abc123 --show-keys
|
|
197
364
|
```
|
|
198
365
|
|
|
199
366
|
**Options:**
|
|
200
|
-
-
|
|
367
|
+
- `-o, --organization <id>` - Organization ID (required)
|
|
201
368
|
- `--show-keys` - Show full API keys (default: masked)
|
|
369
|
+
- `-j, --json` - Output as JSON
|
|
202
370
|
|
|
203
371
|
#### `keys create <name>`
|
|
204
372
|
|
|
205
|
-
Create a new API key.
|
|
373
|
+
Create a new API key for an organization.
|
|
206
374
|
|
|
207
375
|
```bash
|
|
208
|
-
runhuman keys create "CI/CD Key" --
|
|
209
|
-
runhuman keys new "CI/CD Key" --
|
|
376
|
+
runhuman keys create "CI/CD Key" --organization org_abc123
|
|
377
|
+
runhuman keys new "CI/CD Key" --organization org_abc123 # alias
|
|
210
378
|
```
|
|
211
379
|
|
|
212
380
|
**Options:**
|
|
213
|
-
-
|
|
381
|
+
- `-o, --organization <id>` - Organization ID (required)
|
|
214
382
|
- `--copy` - Copy key to clipboard
|
|
215
383
|
|
|
384
|
+
#### `keys show <keyId>`
|
|
385
|
+
|
|
386
|
+
Show details of a specific API key.
|
|
387
|
+
|
|
388
|
+
```bash
|
|
389
|
+
runhuman keys show key_abc123
|
|
390
|
+
runhuman keys show key_abc123 --show-key
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
**Options:**
|
|
394
|
+
- `--show-key` - Show full API key (default: masked)
|
|
395
|
+
|
|
216
396
|
#### `keys delete <keyId>`
|
|
217
397
|
|
|
218
|
-
Delete an API key permanently.
|
|
398
|
+
Delete an API key permanently. Requires the full key ID.
|
|
219
399
|
|
|
220
400
|
```bash
|
|
221
401
|
runhuman keys delete key_abc123 --force
|
|
222
|
-
runhuman keys rm key_abc123 --force
|
|
402
|
+
runhuman keys rm key_abc123 --force # alias
|
|
403
|
+
runhuman keys revoke key_abc123 --force # alias
|
|
223
404
|
```
|
|
224
405
|
|
|
225
406
|
**Options:**
|
|
226
|
-
-
|
|
407
|
+
- `-f, --force` - Skip confirmation prompt
|
|
227
408
|
|
|
228
409
|
### Templates
|
|
229
410
|
|
|
230
411
|
#### `templates list`
|
|
231
412
|
|
|
232
|
-
List all
|
|
413
|
+
List all templates for a project.
|
|
233
414
|
|
|
234
415
|
```bash
|
|
235
416
|
runhuman templates list --project proj_abc123
|
|
236
|
-
runhuman templates ls --project proj_abc123
|
|
417
|
+
runhuman templates ls --project proj_abc123 # alias
|
|
237
418
|
```
|
|
238
419
|
|
|
239
420
|
**Options:**
|
|
240
|
-
-
|
|
421
|
+
- `-p, --project <id>` - Project ID (required, or use default from config)
|
|
241
422
|
|
|
242
423
|
#### `templates create <name>`
|
|
243
424
|
|
|
244
|
-
Create a new
|
|
425
|
+
Create a new template.
|
|
245
426
|
|
|
246
427
|
```bash
|
|
247
|
-
runhuman templates create "
|
|
248
|
-
|
|
249
|
-
-d "Test login functionality" \
|
|
428
|
+
runhuman templates create "Search Flow Test" --project proj_abc123 \
|
|
429
|
+
-d "Search for 'recursion' and confirm the joke appears" \
|
|
250
430
|
--duration 180 \
|
|
251
|
-
--
|
|
431
|
+
--device-class desktop \
|
|
252
432
|
--schema ./schema.json
|
|
253
433
|
```
|
|
254
434
|
|
|
255
435
|
**Options:**
|
|
256
|
-
-
|
|
436
|
+
- `-p, --project <id>` - Project ID (required, or use default from config)
|
|
257
437
|
- `-d, --description <text>` - Template description
|
|
258
|
-
- `--duration <seconds>` - Target test duration
|
|
259
|
-
- `--
|
|
260
|
-
-
|
|
438
|
+
- `--duration <seconds>` - Target test duration in seconds
|
|
439
|
+
- `--device-class <class>` - Default device class (desktop/mobile)
|
|
440
|
+
- `-S, --schema <path>` - Path to JSON schema file
|
|
261
441
|
|
|
262
442
|
#### `templates show <templateId>`
|
|
263
443
|
|
|
264
|
-
Show details
|
|
444
|
+
Show template details.
|
|
265
445
|
|
|
266
446
|
```bash
|
|
267
|
-
runhuman templates show tmpl_abc123
|
|
268
|
-
runhuman templates info tmpl_abc123 # alias
|
|
447
|
+
runhuman templates show tmpl_abc123 --project proj_abc123
|
|
269
448
|
```
|
|
270
449
|
|
|
271
450
|
#### `templates update <templateId>`
|
|
@@ -273,29 +452,26 @@ runhuman templates info tmpl_abc123 # alias
|
|
|
273
452
|
Update a template.
|
|
274
453
|
|
|
275
454
|
```bash
|
|
276
|
-
runhuman templates update tmpl_abc123 --name "New Name"
|
|
277
|
-
runhuman templates update tmpl_abc123 --description "New description"
|
|
278
|
-
runhuman templates update tmpl_abc123 --schema ./new-schema.json
|
|
455
|
+
runhuman templates update tmpl_abc123 --project proj_abc123 --name "New Name"
|
|
279
456
|
```
|
|
280
457
|
|
|
281
458
|
**Options:**
|
|
282
|
-
-
|
|
459
|
+
- `-n, --name <name>` - New template name
|
|
283
460
|
- `-d, --description <text>` - New description
|
|
284
461
|
- `--duration <seconds>` - New target duration
|
|
285
|
-
- `--
|
|
286
|
-
-
|
|
462
|
+
- `--device-class <class>` - New default device class
|
|
463
|
+
- `-S, --schema <path>` - Path to new JSON schema file
|
|
287
464
|
|
|
288
465
|
#### `templates delete <templateId>`
|
|
289
466
|
|
|
290
|
-
Delete a template
|
|
467
|
+
Delete a template.
|
|
291
468
|
|
|
292
469
|
```bash
|
|
293
|
-
runhuman templates delete tmpl_abc123 --force
|
|
294
|
-
runhuman templates rm tmpl_abc123 --force # alias
|
|
470
|
+
runhuman templates delete tmpl_abc123 --project proj_abc123 --force
|
|
295
471
|
```
|
|
296
472
|
|
|
297
473
|
**Options:**
|
|
298
|
-
-
|
|
474
|
+
- `-f, --force` - Skip confirmation prompt
|
|
299
475
|
|
|
300
476
|
### GitHub Integration
|
|
301
477
|
|
|
@@ -305,23 +481,23 @@ Link a GitHub repository to your project.
|
|
|
305
481
|
|
|
306
482
|
```bash
|
|
307
483
|
runhuman github link owner/repo --project proj_abc123
|
|
308
|
-
runhuman gh link volter-ai/runhuman --project proj_abc123 # alias
|
|
309
484
|
```
|
|
310
485
|
|
|
311
486
|
**Options:**
|
|
312
|
-
-
|
|
487
|
+
- `-p, --project <id>` - Project ID (required)
|
|
313
488
|
|
|
314
489
|
#### `github repos`
|
|
315
490
|
|
|
316
|
-
List
|
|
491
|
+
List GitHub repositories accessible to an organization.
|
|
317
492
|
|
|
318
493
|
```bash
|
|
319
|
-
runhuman github repos
|
|
320
|
-
runhuman github repos --
|
|
494
|
+
runhuman github repos --organization org_abc123
|
|
495
|
+
runhuman github repos --organization org_abc123 --search "my-app"
|
|
321
496
|
```
|
|
322
497
|
|
|
323
498
|
**Options:**
|
|
324
|
-
-
|
|
499
|
+
- `-o, --organization <id>` - Organization ID (required)
|
|
500
|
+
- `-s, --search <query>` - Filter by repository name
|
|
325
501
|
|
|
326
502
|
#### `github issues <repo>`
|
|
327
503
|
|
|
@@ -334,65 +510,62 @@ runhuman github issues owner/repo --labels "bug,needs-qa"
|
|
|
334
510
|
```
|
|
335
511
|
|
|
336
512
|
**Options:**
|
|
337
|
-
-
|
|
338
|
-
-
|
|
513
|
+
- `-s, --state <state>` - Filter by state (open/closed/all, default: open)
|
|
514
|
+
- `-l, --labels <labels>` - Filter by comma-separated labels
|
|
339
515
|
|
|
340
516
|
#### `github test <issueNumber>`
|
|
341
517
|
|
|
342
518
|
Create a QA test job for a GitHub issue.
|
|
343
519
|
|
|
344
520
|
```bash
|
|
345
|
-
runhuman github test 123
|
|
346
|
-
runhuman github test 123
|
|
521
|
+
runhuman github test 123 -r owner/repo -u https://google.com
|
|
522
|
+
runhuman github test 123 -u https://google.com --sync
|
|
347
523
|
```
|
|
348
524
|
|
|
349
525
|
**Options:**
|
|
350
|
-
-
|
|
351
|
-
-
|
|
352
|
-
-
|
|
353
|
-
-
|
|
526
|
+
- `-r, --repo <owner/repo>` - Repository (or use default from config)
|
|
527
|
+
- `-u, --url <url>` - URL to test (required)
|
|
528
|
+
- `-t, --template <id>` - Template ID to use
|
|
529
|
+
- `-s, --sync` - Wait for result before exiting
|
|
354
530
|
|
|
355
531
|
#### `github bulk-test`
|
|
356
532
|
|
|
357
533
|
Create QA test jobs for multiple GitHub issues.
|
|
358
534
|
|
|
359
535
|
```bash
|
|
360
|
-
|
|
361
|
-
runhuman github bulk-test
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
runhuman github bulk-test --repo owner/repo --url https://myapp.com \
|
|
365
|
-
--labels "bug,needs-qa" \
|
|
366
|
-
--limit 5
|
|
536
|
+
runhuman github bulk-test -r owner/repo -u https://google.com
|
|
537
|
+
runhuman github bulk-test -r owner/repo -u https://google.com \
|
|
538
|
+
-l "bug,needs-qa" \
|
|
539
|
+
-n 5
|
|
367
540
|
```
|
|
368
541
|
|
|
369
542
|
**Options:**
|
|
370
|
-
-
|
|
371
|
-
-
|
|
372
|
-
-
|
|
373
|
-
-
|
|
374
|
-
-
|
|
375
|
-
-
|
|
543
|
+
- `-r, --repo <owner/repo>` - Repository (or use default from config)
|
|
544
|
+
- `-u, --url <url>` - URL to test (required)
|
|
545
|
+
- `-l, --labels <labels>` - Filter issues by comma-separated labels
|
|
546
|
+
- `-s, --state <state>` - Filter by state (open/closed/all, default: open)
|
|
547
|
+
- `-t, --template <id>` - Template ID to use
|
|
548
|
+
- `-n, --limit <number>` - Maximum number of jobs to create (default: 10)
|
|
376
549
|
|
|
377
550
|
### Authentication
|
|
378
551
|
|
|
379
552
|
#### `login`
|
|
380
553
|
|
|
381
|
-
Login to Runhuman
|
|
554
|
+
Login to Runhuman. Opens your browser for OAuth by default.
|
|
382
555
|
|
|
383
556
|
```bash
|
|
384
557
|
runhuman login
|
|
385
|
-
# Prompts for API key
|
|
386
558
|
|
|
387
|
-
#
|
|
388
|
-
runhuman login --api-key
|
|
389
|
-
```
|
|
559
|
+
# Login with API key (skip browser)
|
|
560
|
+
runhuman login --token <your-api-key>
|
|
390
561
|
|
|
391
|
-
|
|
562
|
+
# Print auth URL without opening browser
|
|
563
|
+
runhuman login --no-browser
|
|
564
|
+
```
|
|
392
565
|
|
|
393
566
|
#### `logout`
|
|
394
567
|
|
|
395
|
-
|
|
568
|
+
Clear saved credentials.
|
|
396
569
|
|
|
397
570
|
```bash
|
|
398
571
|
runhuman logout
|
|
@@ -400,7 +573,7 @@ runhuman logout
|
|
|
400
573
|
|
|
401
574
|
#### `whoami`
|
|
402
575
|
|
|
403
|
-
Display current user info
|
|
576
|
+
Display current user info.
|
|
404
577
|
|
|
405
578
|
```bash
|
|
406
579
|
runhuman whoami
|
|
@@ -422,12 +595,9 @@ runhuman config get project
|
|
|
422
595
|
Set a configuration value.
|
|
423
596
|
|
|
424
597
|
```bash
|
|
425
|
-
# Set globally
|
|
426
|
-
runhuman config set apiUrl https://api.runhuman.com --global
|
|
427
|
-
|
|
428
|
-
# Set for current project
|
|
429
598
|
runhuman config set project proj_abc123
|
|
430
|
-
runhuman config set
|
|
599
|
+
runhuman config set color false
|
|
600
|
+
runhuman config set project proj_abc123 --global
|
|
431
601
|
```
|
|
432
602
|
|
|
433
603
|
**Options:**
|
|
@@ -439,27 +609,20 @@ List all configuration values.
|
|
|
439
609
|
|
|
440
610
|
```bash
|
|
441
611
|
runhuman config list
|
|
612
|
+
runhuman config list --show-secrets
|
|
442
613
|
```
|
|
443
614
|
|
|
444
|
-
|
|
445
|
-
-
|
|
446
|
-
- Project config (`.runhumanrc`)
|
|
447
|
-
- Environment variables
|
|
448
|
-
- Effective (merged) config
|
|
615
|
+
**Options:**
|
|
616
|
+
- `--show-secrets` - Show API keys (default: masked)
|
|
449
617
|
|
|
450
618
|
#### `config reset`
|
|
451
619
|
|
|
452
620
|
Reset configuration to defaults.
|
|
453
621
|
|
|
454
622
|
```bash
|
|
455
|
-
|
|
456
|
-
runhuman config reset
|
|
457
|
-
|
|
458
|
-
# Reset global config
|
|
459
|
-
runhuman config reset global
|
|
460
|
-
|
|
461
|
-
# Reset all
|
|
462
|
-
runhuman config reset all
|
|
623
|
+
runhuman config reset --project --force
|
|
624
|
+
runhuman config reset --global --force
|
|
625
|
+
runhuman config reset --all --force
|
|
463
626
|
```
|
|
464
627
|
|
|
465
628
|
#### `init`
|
|
@@ -468,17 +631,16 @@ Initialize a new Runhuman project with interactive prompts.
|
|
|
468
631
|
|
|
469
632
|
```bash
|
|
470
633
|
runhuman init
|
|
634
|
+
runhuman init --name "My App" --url https://google.com --yes
|
|
471
635
|
```
|
|
472
636
|
|
|
473
|
-
Creates a `.runhumanrc` file in the current directory
|
|
637
|
+
Creates a `.runhumanrc` file in the current directory.
|
|
474
638
|
|
|
475
|
-
## Configuration
|
|
476
|
-
|
|
477
|
-
### Configuration Hierarchy
|
|
639
|
+
## Configuration Hierarchy
|
|
478
640
|
|
|
479
641
|
Configuration is loaded from multiple sources with the following priority (highest to lowest):
|
|
480
642
|
|
|
481
|
-
1. **CLI flags** - `--api-key`, `--
|
|
643
|
+
1. **CLI flags** - `--api-key`, `--project`, etc.
|
|
482
644
|
2. **Environment variables** - `RUNHUMAN_API_KEY`, `RUNHUMAN_API_URL`, etc.
|
|
483
645
|
3. **Project config** - `.runhumanrc` in current directory
|
|
484
646
|
4. **Global config** - `~/.config/runhuman/config.json`
|
|
@@ -487,70 +649,28 @@ Configuration is loaded from multiple sources with the following priority (highe
|
|
|
487
649
|
### Environment Variables
|
|
488
650
|
|
|
489
651
|
```bash
|
|
490
|
-
RUNHUMAN_API_KEY
|
|
491
|
-
RUNHUMAN_API_URL
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
RUNHUMAN_DEFAULT_DURATION # Default test duration in seconds
|
|
495
|
-
RUNHUMAN_DEFAULT_SCREEN_SIZE # Default screen size (desktop/mobile/tablet)
|
|
496
|
-
RUNHUMAN_OUTPUT_FORMAT # Output format (pretty/json/compact)
|
|
497
|
-
RUNHUMAN_NO_COLOR=1 # Disable colored output
|
|
498
|
-
```
|
|
499
|
-
|
|
500
|
-
### Project Config (`.runhumanrc`)
|
|
501
|
-
|
|
502
|
-
Create a `.runhumanrc` file in your project root:
|
|
503
|
-
|
|
504
|
-
```json
|
|
505
|
-
{
|
|
506
|
-
"project": "proj_abc123",
|
|
507
|
-
"defaultUrl": "https://myapp.com",
|
|
508
|
-
"defaultDuration": 300,
|
|
509
|
-
"defaultScreenSize": "desktop",
|
|
510
|
-
"githubRepo": "owner/repo",
|
|
511
|
-
"outputFormat": "pretty",
|
|
512
|
-
"color": true
|
|
513
|
-
}
|
|
514
|
-
```
|
|
515
|
-
|
|
516
|
-
### Global Config
|
|
517
|
-
|
|
518
|
-
Stored at `~/.config/runhuman/config.json`:
|
|
519
|
-
|
|
520
|
-
```json
|
|
521
|
-
{
|
|
522
|
-
"apiUrl": "https://runhuman.com",
|
|
523
|
-
"apiKey": "qa_live_xxxxxxxxxxxxx",
|
|
524
|
-
"color": true
|
|
525
|
-
}
|
|
652
|
+
RUNHUMAN_API_KEY # API key for authentication
|
|
653
|
+
RUNHUMAN_API_URL # API base URL (default: https://runhuman.com)
|
|
654
|
+
RUNHUMAN_PROJECT_ID # Default project ID
|
|
655
|
+
RUNHUMAN_NO_COLOR=true # Disable colored output
|
|
526
656
|
```
|
|
527
657
|
|
|
528
658
|
## JSON Output
|
|
529
659
|
|
|
530
|
-
All commands support
|
|
660
|
+
All commands support `-j, --json` for machine-readable output:
|
|
531
661
|
|
|
532
662
|
```bash
|
|
533
|
-
runhuman create https://
|
|
663
|
+
runhuman create https://google.com -d "Test search" --json
|
|
534
664
|
runhuman status job_abc123 --json
|
|
535
665
|
runhuman list --json
|
|
536
666
|
```
|
|
537
667
|
|
|
538
|
-
JSON output format:
|
|
539
|
-
|
|
540
|
-
```json
|
|
541
|
-
{
|
|
542
|
-
"success": true,
|
|
543
|
-
"data": { ... },
|
|
544
|
-
"error": null
|
|
545
|
-
}
|
|
546
|
-
```
|
|
547
|
-
|
|
548
668
|
## Exit Codes
|
|
549
669
|
|
|
550
670
|
- `0` - Success
|
|
551
|
-
- `1` - General error
|
|
671
|
+
- `1` - General error / ambiguous ID / full ID required
|
|
552
672
|
- `2` - Authentication error
|
|
553
|
-
- `3` - Not found
|
|
673
|
+
- `3` - Not found / no match for ID prefix
|
|
554
674
|
- `4` - Validation error
|
|
555
675
|
- `5` - Timeout error
|
|
556
676
|
|
|
@@ -566,68 +686,34 @@ runhuman login
|
|
|
566
686
|
runhuman init
|
|
567
687
|
|
|
568
688
|
# 3. Create a test
|
|
569
|
-
runhuman create https://
|
|
689
|
+
runhuman create https://google.com \
|
|
690
|
+
-d "Search for 'recursion' and confirm the tester experiences a joke about recursion"
|
|
570
691
|
|
|
571
|
-
# 4. Wait for results
|
|
572
|
-
runhuman wait
|
|
692
|
+
# 4. Wait for results (truncated ID)
|
|
693
|
+
runhuman wait 712e
|
|
573
694
|
|
|
574
695
|
# 5. View detailed results
|
|
575
|
-
runhuman results
|
|
696
|
+
runhuman results 712e
|
|
576
697
|
```
|
|
577
698
|
|
|
578
699
|
### CI/CD Integration
|
|
579
700
|
|
|
580
701
|
```bash
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
-d "Smoke test on staging" \
|
|
702
|
+
runhuman create https://staging.google.com \
|
|
703
|
+
-d "Search for 'recursion' and confirm the joke appears" \
|
|
584
704
|
--sync \
|
|
585
705
|
--json > result.json
|
|
586
|
-
|
|
587
|
-
# Exit code will be non-zero if test fails
|
|
588
|
-
```
|
|
589
|
-
|
|
590
|
-
### GitHub Integration Workflow
|
|
591
|
-
|
|
592
|
-
```bash
|
|
593
|
-
# 1. Link your repo
|
|
594
|
-
runhuman github link owner/repo --project proj_abc123
|
|
595
|
-
|
|
596
|
-
# 2. List issues needing QA
|
|
597
|
-
runhuman github issues owner/repo --labels "needs-qa"
|
|
598
|
-
|
|
599
|
-
# 3. Test a specific issue
|
|
600
|
-
runhuman github test 42 --url https://myapp.com --sync
|
|
601
|
-
|
|
602
|
-
# 4. Bulk test multiple issues
|
|
603
|
-
runhuman github bulk-test --url https://myapp.com --labels "needs-qa" --limit 5
|
|
604
|
-
```
|
|
605
|
-
|
|
606
|
-
### Using Templates
|
|
607
|
-
|
|
608
|
-
```bash
|
|
609
|
-
# 1. Create a template
|
|
610
|
-
runhuman templates create "Login Flow Test" --project proj_abc123 \
|
|
611
|
-
-d "Test login with valid and invalid credentials" \
|
|
612
|
-
--duration 180 \
|
|
613
|
-
--schema ./schemas/login-test.json
|
|
614
|
-
|
|
615
|
-
# 2. Use the template
|
|
616
|
-
runhuman create https://myapp.com --template tmpl_abc123
|
|
617
|
-
|
|
618
|
-
# 3. List all templates
|
|
619
|
-
runhuman templates list --project proj_abc123
|
|
620
706
|
```
|
|
621
707
|
|
|
622
|
-
##
|
|
623
|
-
|
|
624
|
-
If you're using AI agents (Claude, GPT, etc.), check out our MCP server package:
|
|
708
|
+
## How It Works
|
|
625
709
|
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
|
|
710
|
+
1. **You create a job** - Define test instructions and optional output schema
|
|
711
|
+
2. **Job posted to testers** - Request goes to human tester pool
|
|
712
|
+
3. **Human performs test** - Real person tests with video/screenshot recording
|
|
713
|
+
4. **AI extracts data** - AI processes feedback into structured JSON
|
|
714
|
+
5. **You get results** - Clean, typed data ready for automation
|
|
629
715
|
|
|
630
|
-
|
|
716
|
+
**Pricing:** Pay-per-second ($0.0085/sec of tester time). No monthly fees, no minimums.
|
|
631
717
|
|
|
632
718
|
## Learn More
|
|
633
719
|
|
|
@@ -640,16 +726,6 @@ See [@runhuman/mcp](https://www.npmjs.com/package/@runhuman/mcp) for details.
|
|
|
640
726
|
- **Email:** hey@runhuman.com
|
|
641
727
|
- **Issues:** [GitHub Issues](https://github.com/volter-ai/runhuman/issues)
|
|
642
728
|
|
|
643
|
-
## How It Works
|
|
644
|
-
|
|
645
|
-
1. **You create a job** - Define test instructions and optional output schema
|
|
646
|
-
2. **Job posted to testers** - Request goes to human tester pool
|
|
647
|
-
3. **Human performs test** - Real person tests with video/screenshot recording
|
|
648
|
-
4. **AI extracts data** - GPT-4o processes feedback into structured JSON
|
|
649
|
-
5. **You get results** - Clean, typed data ready for automation
|
|
650
|
-
|
|
651
|
-
**Pricing:** Pay-per-second ($0.0085/sec of tester time). No monthly fees, no minimums.
|
|
652
|
-
|
|
653
729
|
---
|
|
654
730
|
|
|
655
|
-
**Built
|
|
731
|
+
**Built by the [Volter AI](https://volter.ai) team**
|