@eigenpal/cli 0.6.9 → 0.6.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/README.md +2 -2
- package/dist/cli.js +1310 -1274
- package/dist/skill/SKILL.md +60 -57
- package/dist/skill/reference/cli/agents.md +143 -556
- package/dist/skill/reference/cli/init.md +1 -1
- package/dist/skill/reference/cli/runs.md +481 -0
- package/dist/skill/reference/cli/surface.md +135 -0
- package/dist/skill/reference/cli/workflow.md +35 -149
- package/dist/skill/reference/dataset-format.md +1 -1
- package/dist/skill/reference/debugging.md +23 -22
- package/dist/skill/reference/step-exec.md +3 -3
- package/dist/skill/reference/workflow-yaml.md +2 -2
- package/dist/templates/blank/README.md +1 -1
- package/dist/templates/pdf-extraction/README.md +1 -1
- package/dist/templates/text-classification/README.md +1 -1
- package/package.json +1 -1
package/dist/skill/SKILL.md
CHANGED
|
@@ -23,10 +23,9 @@ Use long command names in generated commands and documentation. They are more
|
|
|
23
23
|
readable and cost very little for agents to type. Some short aliases exist for
|
|
24
24
|
interactive users; understand them if you see them, but do not prefer them:
|
|
25
25
|
|
|
26
|
-
- `
|
|
27
|
-
- `
|
|
28
|
-
- `
|
|
29
|
-
- `workflow exec` = `workflow execution`
|
|
26
|
+
- `runs fb` = `runs feedback`
|
|
27
|
+
- `runs artifact` = `runs artifacts`
|
|
28
|
+
- `workflow run` = `workflow runution`
|
|
30
29
|
- `workflow exp` = `workflow experiment`
|
|
31
30
|
- `ls` = `list` anywhere in the CLI
|
|
32
31
|
- `diff` = `compare` anywhere in the CLI
|
|
@@ -35,15 +34,20 @@ For exact flags and defaults, prefer the generated references:
|
|
|
35
34
|
|
|
36
35
|
- [`reference/cli/workflow.md`](reference/cli/workflow.md)
|
|
37
36
|
- [`reference/cli/agents.md`](reference/cli/agents.md)
|
|
37
|
+
- [`reference/cli/runs.md`](reference/cli/runs.md)
|
|
38
38
|
- [`reference/cli/git.md`](reference/cli/git.md)
|
|
39
39
|
|
|
40
40
|
## Pick The Surface
|
|
41
41
|
|
|
42
42
|
Choose `workflow` when the work is about a workflow definition, dataset,
|
|
43
|
-
evaluator, experiment, or workflow
|
|
43
|
+
evaluator, experiment, or creating workflow runutions from local examples.
|
|
44
44
|
|
|
45
45
|
Choose `agents` when the work is about an agent workspace, agent file, trigger,
|
|
46
|
-
|
|
46
|
+
or starting an agent run.
|
|
47
|
+
|
|
48
|
+
Choose `runs` when the work is about inspecting, watching, comparing,
|
|
49
|
+
cancelling, rerunning, downloading artifacts for, or attaching feedback/expected
|
|
50
|
+
artifacts to any workflow or agent run.
|
|
47
51
|
|
|
48
52
|
Agents and workflows are related, but their iteration loops are different:
|
|
49
53
|
workflow experiments evaluate datasets with evaluators; agent workflows do not
|
|
@@ -66,9 +70,9 @@ eigenpal workflow validate ./workflow.yaml
|
|
|
66
70
|
eigenpal workflow push --file workflow.yaml
|
|
67
71
|
|
|
68
72
|
# 4. Run one example while iterating.
|
|
69
|
-
eigenpal workflow
|
|
70
|
-
eigenpal
|
|
71
|
-
eigenpal
|
|
73
|
+
eigenpal workflow run <workflow-id> <example-name>
|
|
74
|
+
eigenpal runs watch <execution-id>
|
|
75
|
+
eigenpal runs get <execution-id> --json
|
|
72
76
|
|
|
73
77
|
# 5. Manage the dataset and evaluators when ready to evaluate.
|
|
74
78
|
eigenpal workflow dataset validate ./dataset
|
|
@@ -88,7 +92,7 @@ also accept the workflow slug for convenience, but use the id in scripts.
|
|
|
88
92
|
Useful workflow ids are not interchangeable:
|
|
89
93
|
|
|
90
94
|
- `<workflow-id>` identifies a workflow (`wf_...`).
|
|
91
|
-
- `<execution-id>` identifies one workflow
|
|
95
|
+
- `<execution-id>` identifies one workflow runution.
|
|
92
96
|
- `<batch-id>` identifies one experiment batch.
|
|
93
97
|
- `<example-id>` identifies one dataset example.
|
|
94
98
|
|
|
@@ -163,9 +167,9 @@ eigenpal agents save -m "Improve <agent> behavior"
|
|
|
163
167
|
SOURCE_REF="$(eigenpal git -- rev-parse HEAD)"
|
|
164
168
|
eigenpal agents run agents.<slug>@"$SOURCE_REF" --input-json '{"text":"hello"}' --wait
|
|
165
169
|
eigenpal agents run agents.<slug>@"$SOURCE_REF" --example example-name --wait
|
|
166
|
-
eigenpal
|
|
167
|
-
eigenpal
|
|
168
|
-
eigenpal
|
|
170
|
+
eigenpal runs get <run-id> --json | jq '.run | {status,schemaValid,error,requestedSourceRef,resolvedGitRef,resolvedGitTag,resolvedCommitSha,cost}'
|
|
171
|
+
eigenpal runs artifacts list <run-id>
|
|
172
|
+
eigenpal runs trace <run-id> --out ./review/<run-id>/trace.jsonl
|
|
169
173
|
|
|
170
174
|
# 6. Release and sync only when ready.
|
|
171
175
|
eigenpal agents release patch agents/<slug>
|
|
@@ -173,7 +177,7 @@ eigenpal agents sync agents.<slug>
|
|
|
173
177
|
|
|
174
178
|
# 7. Verify the released package.
|
|
175
179
|
eigenpal agents run agents.<slug>@latest --example example-name --wait --json
|
|
176
|
-
eigenpal
|
|
180
|
+
eigenpal runs get <released-run-id> --json | jq '.run | {status,schemaValid,error,resolvedGitTag,resolvedCommitSha,cost}'
|
|
177
181
|
```
|
|
178
182
|
|
|
179
183
|
File inputs must use schema field names when there is more than one file input:
|
|
@@ -223,7 +227,7 @@ eigenpal agents sync agents.<dependent-slug>
|
|
|
223
227
|
```
|
|
224
228
|
|
|
225
229
|
After a shared dependency rollout, run one affected agent and inspect
|
|
226
|
-
`eigenpal.lock` via `
|
|
230
|
+
`eigenpal.lock` via `runs artifacts fetch` to confirm the dependency
|
|
227
231
|
version and commit that inference actually installed.
|
|
228
232
|
|
|
229
233
|
### Agent Datasets And Examples
|
|
@@ -260,18 +264,18 @@ Run a persisted example with `eigenpal agents run agents.<slug>@<ref> --example
|
|
|
260
264
|
### Debug Agent Runs
|
|
261
265
|
|
|
262
266
|
```bash
|
|
263
|
-
eigenpal
|
|
264
|
-
eigenpal
|
|
265
|
-
eigenpal
|
|
266
|
-
eigenpal
|
|
267
|
-
eigenpal
|
|
268
|
-
eigenpal
|
|
269
|
-
eigenpal
|
|
270
|
-
eigenpal
|
|
267
|
+
eigenpal runs list agents.<slug> --compact
|
|
268
|
+
eigenpal runs list agents.<slug> --type agent --status failed --compact
|
|
269
|
+
eigenpal runs artifacts list <agent-execution-id>
|
|
270
|
+
eigenpal runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
|
|
271
|
+
eigenpal runs trace <agent-execution-id> --out ./review/<agent-execution-id>/trace.jsonl
|
|
272
|
+
eigenpal runs rerun <agent-execution-id> --wait
|
|
273
|
+
eigenpal runs rerun <agent-execution-id> --source-ref original --wait
|
|
274
|
+
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
|
|
271
275
|
--baseline \
|
|
272
276
|
--normalize-dates
|
|
273
277
|
|
|
274
|
-
eigenpal
|
|
278
|
+
eigenpal runs feedback resolve <source-agent-execution-id> \
|
|
275
279
|
--message "Fixed and verified in <new-agent-execution-id>."
|
|
276
280
|
```
|
|
277
281
|
|
|
@@ -282,7 +286,7 @@ targets show all source refs; add `@<ref>` only when you want a specific
|
|
|
282
286
|
release, branch, tag, semver range, or commit. `runs list --json` returns
|
|
283
287
|
`{ runs, total, limit, offset }`; `runs get --json` returns `{ run }`.
|
|
284
288
|
|
|
285
|
-
`
|
|
289
|
+
`runs rerun <run-id>` reuses the previous input snapshot with `latest` source
|
|
286
290
|
by default. Use `--source-ref original` only when you need to reproduce the
|
|
287
291
|
previous resolved version exactly.
|
|
288
292
|
|
|
@@ -353,12 +357,12 @@ eigenpal workflow dataset example delete <workflow-id> <example-id> --yes
|
|
|
353
357
|
### Run And Compare Workflow Executions
|
|
354
358
|
|
|
355
359
|
```bash
|
|
356
|
-
eigenpal workflow
|
|
357
|
-
eigenpal
|
|
358
|
-
eigenpal
|
|
359
|
-
eigenpal
|
|
360
|
-
eigenpal
|
|
361
|
-
eigenpal
|
|
360
|
+
eigenpal workflow run <workflow-id> <example-name>
|
|
361
|
+
eigenpal runs watch <execution-id>
|
|
362
|
+
eigenpal runs get <execution-id> --include input,output,error --json
|
|
363
|
+
eigenpal runs list <workflow-id> --type workflow --status failed --limit 10
|
|
364
|
+
eigenpal runs compare <execution-id-a> <execution-id-b>
|
|
365
|
+
eigenpal runs cancel <execution-id> --yes
|
|
362
366
|
```
|
|
363
367
|
|
|
364
368
|
### Run And Compare Experiments
|
|
@@ -400,11 +404,10 @@ and `agents sync`. Live file commands are read/compare tools.
|
|
|
400
404
|
### Inspect Agent Executions
|
|
401
405
|
|
|
402
406
|
```bash
|
|
403
|
-
eigenpal
|
|
404
|
-
eigenpal
|
|
405
|
-
eigenpal
|
|
406
|
-
eigenpal
|
|
407
|
-
eigenpal agents runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
|
|
407
|
+
eigenpal runs list <agent-id-or-slug> --type agent --status failed --limit 10
|
|
408
|
+
eigenpal runs get <agent-execution-id> --include feedback,expected,files,trace,issues --json | jq '.run'
|
|
409
|
+
eigenpal runs artifacts list <agent-execution-id>
|
|
410
|
+
eigenpal runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
|
|
408
411
|
```
|
|
409
412
|
|
|
410
413
|
Use `--compact` on execution lists when you only need triage rows.
|
|
@@ -412,9 +415,9 @@ Use `--compact` on execution lists when you only need triage rows.
|
|
|
412
415
|
### Download Agent Traces
|
|
413
416
|
|
|
414
417
|
```bash
|
|
415
|
-
eigenpal
|
|
416
|
-
eigenpal
|
|
417
|
-
eigenpal
|
|
418
|
+
eigenpal runs trace <agent-execution-id> --out trace.jsonl
|
|
419
|
+
eigenpal runs trace <agent-execution-id> | jq -r 'select(.toolName? or .tool_name? or .tool?)'
|
|
420
|
+
eigenpal runs trace <agent-execution-id> | rg 'error|tool'
|
|
418
421
|
```
|
|
419
422
|
|
|
420
423
|
`trace` streams the raw JSONL to stdout by default. Use shell tools such as
|
|
@@ -423,18 +426,18 @@ eigenpal agents runs trace <agent-execution-id> | rg 'error|tool'
|
|
|
423
426
|
### Manage Agent Feedback And Expected Artifacts
|
|
424
427
|
|
|
425
428
|
```bash
|
|
426
|
-
eigenpal
|
|
429
|
+
eigenpal runs feedback update <agent-execution-id> \
|
|
427
430
|
--status open \
|
|
428
431
|
--rating fail \
|
|
429
432
|
--message "Expected the filing date to be extracted."
|
|
430
|
-
eigenpal
|
|
433
|
+
eigenpal runs feedback resolve <agent-execution-id> \
|
|
431
434
|
--message "Fixed and verified."
|
|
432
|
-
eigenpal
|
|
435
|
+
eigenpal runs feedback clear <agent-execution-id> --yes
|
|
433
436
|
|
|
434
|
-
eigenpal
|
|
435
|
-
eigenpal
|
|
436
|
-
eigenpal
|
|
437
|
-
eigenpal
|
|
437
|
+
eigenpal runs expected list <agent-execution-id>
|
|
438
|
+
eigenpal runs expected upload <agent-execution-id> expected.json --name expected.json
|
|
439
|
+
eigenpal runs expected copy-output <agent-execution-id> result.json --name expected.json
|
|
440
|
+
eigenpal runs expected pull <agent-execution-id> --out expected/
|
|
438
441
|
```
|
|
439
442
|
|
|
440
443
|
Feedback is attached to one execution. Expected artifacts are the references
|
|
@@ -443,10 +446,10 @@ used when comparing future runs.
|
|
|
443
446
|
### Rerun And Compare Agent Executions
|
|
444
447
|
|
|
445
448
|
```bash
|
|
446
|
-
eigenpal
|
|
447
|
-
eigenpal
|
|
448
|
-
eigenpal
|
|
449
|
-
eigenpal
|
|
449
|
+
eigenpal runs rerun <agent-execution-id> --wait
|
|
450
|
+
eigenpal runs rerun <agent-execution-id> --source-ref original --wait
|
|
451
|
+
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id>
|
|
452
|
+
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
|
|
450
453
|
--baseline \
|
|
451
454
|
--normalize-dates \
|
|
452
455
|
--fail-on-diff
|
|
@@ -463,9 +466,9 @@ Most `list`, `get`, and mutating commands support `--json`. Prefer JSON when a
|
|
|
463
466
|
script or agent needs to inspect the result with `jq`.
|
|
464
467
|
|
|
465
468
|
```bash
|
|
466
|
-
eigenpal
|
|
467
|
-
eigenpal
|
|
468
|
-
eigenpal
|
|
469
|
+
eigenpal runs list <workflow-id> --type workflow --json | jq '.runs[0].id'
|
|
470
|
+
eigenpal runs list <agent-id-or-slug> --type agent --json | jq '.runs[0].id'
|
|
471
|
+
eigenpal runs get <agent-execution-id> --json | jq '.run.id'
|
|
469
472
|
```
|
|
470
473
|
|
|
471
474
|
General exit-code convention:
|
|
@@ -488,7 +491,7 @@ Workflow schemas:
|
|
|
488
491
|
- [`reference/step-exec.md`](reference/step-exec.md) — single-step execution command
|
|
489
492
|
- [`reference/dataset-format.md`](reference/dataset-format.md) — dataset folder format
|
|
490
493
|
- [`reference/evaluators.md`](reference/evaluators.md) — evaluator configuration
|
|
491
|
-
- [`reference/debugging.md`](reference/debugging.md) — workflow
|
|
494
|
+
- [`reference/debugging.md`](reference/debugging.md) — workflow runution debugging
|
|
492
495
|
|
|
493
496
|
CLI command references:
|
|
494
497
|
|
|
@@ -566,8 +569,8 @@ Agent command roles:
|
|
|
566
569
|
- `agents release` — create immutable package release tags.
|
|
567
570
|
- `agents sync` — apply latest released agent manifest/triggers to the server.
|
|
568
571
|
- `agents run` — start a run from a source ref (`@latest`, version, branch, commit).
|
|
569
|
-
- `
|
|
570
|
-
- `
|
|
571
|
-
- `
|
|
572
|
+
- `runs rerun` — reuse a previous input snapshot; defaults to latest source.
|
|
573
|
+
- `runs list/get/watch/cancel/compare/trace` — inspect and manage executions.
|
|
574
|
+
- `runs artifacts list/fetch` — inventory and download run artifacts by path.
|
|
572
575
|
- `agents dataset` — manage persisted agent examples; runtime data, not Git source.
|
|
573
576
|
- `agents file` — live read/diff inspection only.
|