@eigenpal/cli 0.6.9 → 0.6.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -23,10 +23,9 @@ Use long command names in generated commands and documentation. They are more
23
23
  readable and cost very little for agents to type. Some short aliases exist for
24
24
  interactive users; understand them if you see them, but do not prefer them:
25
25
 
26
- - `agents exec` = `agents runs`
27
- - `agents runs fb` = `agents runs feedback`
28
- - `agents runs artifact` = `agents runs artifacts`
29
- - `workflow exec` = `workflow execution`
26
+ - `runs fb` = `runs feedback`
27
+ - `runs artifact` = `runs artifacts`
28
+ - `workflow run` = `workflow runution`
30
29
  - `workflow exp` = `workflow experiment`
31
30
  - `ls` = `list` anywhere in the CLI
32
31
  - `diff` = `compare` anywhere in the CLI
@@ -35,15 +34,20 @@ For exact flags and defaults, prefer the generated references:
35
34
 
36
35
  - [`reference/cli/workflow.md`](reference/cli/workflow.md)
37
36
  - [`reference/cli/agents.md`](reference/cli/agents.md)
37
+ - [`reference/cli/runs.md`](reference/cli/runs.md)
38
38
  - [`reference/cli/git.md`](reference/cli/git.md)
39
39
 
40
40
  ## Pick The Surface
41
41
 
42
42
  Choose `workflow` when the work is about a workflow definition, dataset,
43
- evaluator, experiment, or workflow execution.
43
+ evaluator, experiment, or creating workflow runutions from local examples.
44
44
 
45
45
  Choose `agents` when the work is about an agent workspace, agent file, trigger,
46
- agent execution, trace, feedback entry, expected artifact, or rerun.
46
+ or starting an agent run.
47
+
48
+ Choose `runs` when the work is about inspecting, watching, comparing,
49
+ cancelling, rerunning, downloading artifacts for, or attaching feedback/expected
50
+ artifacts to any workflow or agent run.
47
51
 
48
52
  Agents and workflows are related, but their iteration loops are different:
49
53
  workflow experiments evaluate datasets with evaluators; agent workflows do not
@@ -66,9 +70,9 @@ eigenpal workflow validate ./workflow.yaml
66
70
  eigenpal workflow push --file workflow.yaml
67
71
 
68
72
  # 4. Run one example while iterating.
69
- eigenpal workflow execution run <workflow-id> <example-name>
70
- eigenpal workflow execution watch <execution-id>
71
- eigenpal workflow execution get <execution-id> --json
73
+ eigenpal workflow run <workflow-id> <example-name>
74
+ eigenpal runs watch <execution-id>
75
+ eigenpal runs get <execution-id> --json
72
76
 
73
77
  # 5. Manage the dataset and evaluators when ready to evaluate.
74
78
  eigenpal workflow dataset validate ./dataset
@@ -88,7 +92,7 @@ also accept the workflow slug for convenience, but use the id in scripts.
88
92
  Useful workflow ids are not interchangeable:
89
93
 
90
94
  - `<workflow-id>` identifies a workflow (`wf_...`).
91
- - `<execution-id>` identifies one workflow execution.
95
+ - `<execution-id>` identifies one workflow runution.
92
96
  - `<batch-id>` identifies one experiment batch.
93
97
  - `<example-id>` identifies one dataset example.
94
98
 
@@ -163,9 +167,9 @@ eigenpal agents save -m "Improve <agent> behavior"
163
167
  SOURCE_REF="$(eigenpal git -- rev-parse HEAD)"
164
168
  eigenpal agents run agents.<slug>@"$SOURCE_REF" --input-json '{"text":"hello"}' --wait
165
169
  eigenpal agents run agents.<slug>@"$SOURCE_REF" --example example-name --wait
166
- eigenpal agents runs get <run-id> --json | jq '.run | {status,schemaValid,error,requestedSourceRef,resolvedGitRef,resolvedGitTag,resolvedCommitSha,cost}'
167
- eigenpal agents runs artifacts list <run-id>
168
- eigenpal agents runs trace <run-id> --out ./review/<run-id>/trace.jsonl
170
+ eigenpal runs get <run-id> --json | jq '.run | {status,schemaValid,error,requestedSourceRef,resolvedGitRef,resolvedGitTag,resolvedCommitSha,cost}'
171
+ eigenpal runs artifacts list <run-id>
172
+ eigenpal runs trace <run-id> --out ./review/<run-id>/trace.jsonl
169
173
 
170
174
  # 6. Release and sync only when ready.
171
175
  eigenpal agents release patch agents/<slug>
@@ -173,7 +177,7 @@ eigenpal agents sync agents.<slug>
173
177
 
174
178
  # 7. Verify the released package.
175
179
  eigenpal agents run agents.<slug>@latest --example example-name --wait --json
176
- eigenpal agents runs get <released-run-id> --json | jq '.run | {status,schemaValid,error,resolvedGitTag,resolvedCommitSha,cost}'
180
+ eigenpal runs get <released-run-id> --json | jq '.run | {status,schemaValid,error,resolvedGitTag,resolvedCommitSha,cost}'
177
181
  ```
178
182
 
179
183
  File inputs must use schema field names when there is more than one file input:
@@ -223,7 +227,7 @@ eigenpal agents sync agents.<dependent-slug>
223
227
  ```
224
228
 
225
229
  After a shared dependency rollout, run one affected agent and inspect
226
- `eigenpal.lock` via `agents runs artifacts fetch` to confirm the dependency
230
+ `eigenpal.lock` via `runs artifacts fetch` to confirm the dependency
227
231
  version and commit that inference actually installed.
228
232
 
229
233
  ### Agent Datasets And Examples
@@ -260,18 +264,18 @@ Run a persisted example with `eigenpal agents run agents.<slug>@<ref> --example
260
264
  ### Debug Agent Runs
261
265
 
262
266
  ```bash
263
- eigenpal agents runs list agents.<slug> --compact
264
- eigenpal agents runs list agents.<slug> --status failed --compact
265
- eigenpal agents runs artifacts list <agent-execution-id>
266
- eigenpal agents runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
267
- eigenpal agents runs trace <agent-execution-id> --out ./review/<agent-execution-id>/trace.jsonl
268
- eigenpal agents rerun <agent-execution-id> --wait
269
- eigenpal agents rerun <agent-execution-id> --source-ref original --wait
270
- eigenpal agents runs compare <source-agent-execution-id> <new-agent-execution-id> \
267
+ eigenpal runs list agents.<slug> --compact
268
+ eigenpal runs list agents.<slug> --type agent --status failed --compact
269
+ eigenpal runs artifacts list <agent-execution-id>
270
+ eigenpal runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
271
+ eigenpal runs trace <agent-execution-id> --out ./review/<agent-execution-id>/trace.jsonl
272
+ eigenpal runs rerun <agent-execution-id> --wait
273
+ eigenpal runs rerun <agent-execution-id> --source-ref original --wait
274
+ eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
271
275
  --baseline \
272
276
  --normalize-dates
273
277
 
274
- eigenpal agents runs feedback resolve <source-agent-execution-id> \
278
+ eigenpal runs feedback resolve <source-agent-execution-id> \
275
279
  --message "Fixed and verified in <new-agent-execution-id>."
276
280
  ```
277
281
 
@@ -282,7 +286,7 @@ targets show all source refs; add `@<ref>` only when you want a specific
282
286
  release, branch, tag, semver range, or commit. `runs list --json` returns
283
287
  `{ runs, total, limit, offset }`; `runs get --json` returns `{ run }`.
284
288
 
285
- `agents rerun <run-id>` reuses the previous input snapshot with `latest` source
289
+ `runs rerun <run-id>` reuses the previous input snapshot with `latest` source
286
290
  by default. Use `--source-ref original` only when you need to reproduce the
287
291
  previous resolved version exactly.
288
292
 
@@ -353,12 +357,12 @@ eigenpal workflow dataset example delete <workflow-id> <example-id> --yes
353
357
  ### Run And Compare Workflow Executions
354
358
 
355
359
  ```bash
356
- eigenpal workflow execution run <workflow-id> <example-name>
357
- eigenpal workflow execution watch <execution-id>
358
- eigenpal workflow execution get <execution-id> --include input,output,error --json
359
- eigenpal workflow execution list <workflow-id> --status failed --limit 10
360
- eigenpal workflow execution compare <execution-id-a> <execution-id-b>
361
- eigenpal workflow execution cancel <execution-id> --yes
360
+ eigenpal workflow run <workflow-id> <example-name>
361
+ eigenpal runs watch <execution-id>
362
+ eigenpal runs get <execution-id> --include input,output,error --json
363
+ eigenpal runs list <workflow-id> --type workflow --status failed --limit 10
364
+ eigenpal runs compare <execution-id-a> <execution-id-b>
365
+ eigenpal runs cancel <execution-id> --yes
362
366
  ```
363
367
 
364
368
  ### Run And Compare Experiments
@@ -400,11 +404,10 @@ and `agents sync`. Live file commands are read/compare tools.
400
404
  ### Inspect Agent Executions
401
405
 
402
406
  ```bash
403
- eigenpal agents runs list <agent-id-or-slug> --status failed --limit 10
404
- eigenpal agents runs list <agent-id-or-slug> --feedback-rating fail --include feedback,expected
405
- eigenpal agents runs get <agent-execution-id> --include feedback,expected,files,trace,issues --json | jq '.run'
406
- eigenpal agents runs artifacts list <agent-execution-id>
407
- eigenpal agents runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
407
+ eigenpal runs list <agent-id-or-slug> --type agent --status failed --limit 10
408
+ eigenpal runs get <agent-execution-id> --include feedback,expected,files,trace,issues --json | jq '.run'
409
+ eigenpal runs artifacts list <agent-execution-id>
410
+ eigenpal runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
408
411
  ```
409
412
 
410
413
  Use `--compact` on execution lists when you only need triage rows.
@@ -412,9 +415,9 @@ Use `--compact` on execution lists when you only need triage rows.
412
415
  ### Download Agent Traces
413
416
 
414
417
  ```bash
415
- eigenpal agents runs trace <agent-execution-id> --out trace.jsonl
416
- eigenpal agents runs trace <agent-execution-id> | jq -r 'select(.toolName? or .tool_name? or .tool?)'
417
- eigenpal agents runs trace <agent-execution-id> | rg 'error|tool'
418
+ eigenpal runs trace <agent-execution-id> --out trace.jsonl
419
+ eigenpal runs trace <agent-execution-id> | jq -r 'select(.toolName? or .tool_name? or .tool?)'
420
+ eigenpal runs trace <agent-execution-id> | rg 'error|tool'
418
421
  ```
419
422
 
420
423
  `trace` streams the raw JSONL to stdout by default. Use shell tools such as
@@ -423,18 +426,18 @@ eigenpal agents runs trace <agent-execution-id> | rg 'error|tool'
423
426
  ### Manage Agent Feedback And Expected Artifacts
424
427
 
425
428
  ```bash
426
- eigenpal agents runs feedback update <agent-execution-id> \
429
+ eigenpal runs feedback update <agent-execution-id> \
427
430
  --status open \
428
431
  --rating fail \
429
432
  --message "Expected the filing date to be extracted."
430
- eigenpal agents runs feedback resolve <agent-execution-id> \
433
+ eigenpal runs feedback resolve <agent-execution-id> \
431
434
  --message "Fixed and verified."
432
- eigenpal agents runs feedback clear <agent-execution-id> --yes
435
+ eigenpal runs feedback clear <agent-execution-id> --yes
433
436
 
434
- eigenpal agents runs expected list <agent-execution-id>
435
- eigenpal agents runs expected upload <agent-execution-id> expected.json --name expected.json
436
- eigenpal agents runs expected copy-output <agent-execution-id> result.json --name expected.json
437
- eigenpal agents runs expected pull <agent-execution-id> --out expected/
437
+ eigenpal runs expected list <agent-execution-id>
438
+ eigenpal runs expected upload <agent-execution-id> expected.json --name expected.json
439
+ eigenpal runs expected copy-output <agent-execution-id> result.json --name expected.json
440
+ eigenpal runs expected pull <agent-execution-id> --out expected/
438
441
  ```
439
442
 
440
443
  Feedback is attached to one execution. Expected artifacts are the references
@@ -443,10 +446,10 @@ used when comparing future runs.
443
446
  ### Rerun And Compare Agent Executions
444
447
 
445
448
  ```bash
446
- eigenpal agents rerun <agent-execution-id> --wait
447
- eigenpal agents rerun <agent-execution-id> --source-ref original --wait
448
- eigenpal agents runs compare <source-agent-execution-id> <new-agent-execution-id>
449
- eigenpal agents runs compare <source-agent-execution-id> <new-agent-execution-id> \
449
+ eigenpal runs rerun <agent-execution-id> --wait
450
+ eigenpal runs rerun <agent-execution-id> --source-ref original --wait
451
+ eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id>
452
+ eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
450
453
  --baseline \
451
454
  --normalize-dates \
452
455
  --fail-on-diff
@@ -463,9 +466,9 @@ Most `list`, `get`, and mutating commands support `--json`. Prefer JSON when a
463
466
  script or agent needs to inspect the result with `jq`.
464
467
 
465
468
  ```bash
466
- eigenpal workflow execution list <workflow-id> --json | jq '.data[0].id'
467
- eigenpal agents runs list <agent-id-or-slug> --json | jq '.runs[0].id'
468
- eigenpal agents runs get <agent-execution-id> --json | jq '.run.id'
469
+ eigenpal runs list <workflow-id> --type workflow --json | jq '.runs[0].id'
470
+ eigenpal runs list <agent-id-or-slug> --type agent --json | jq '.runs[0].id'
471
+ eigenpal runs get <agent-execution-id> --json | jq '.run.id'
469
472
  ```
470
473
 
471
474
  General exit-code convention:
@@ -488,7 +491,7 @@ Workflow schemas:
488
491
  - [`reference/step-exec.md`](reference/step-exec.md) — single-step execution command
489
492
  - [`reference/dataset-format.md`](reference/dataset-format.md) — dataset folder format
490
493
  - [`reference/evaluators.md`](reference/evaluators.md) — evaluator configuration
491
- - [`reference/debugging.md`](reference/debugging.md) — workflow execution debugging
494
+ - [`reference/debugging.md`](reference/debugging.md) — workflow runution debugging
492
495
 
493
496
  CLI command references:
494
497
 
@@ -566,8 +569,8 @@ Agent command roles:
566
569
  - `agents release` — create immutable package release tags.
567
570
  - `agents sync` — apply latest released agent manifest/triggers to the server.
568
571
  - `agents run` — start a run from a source ref (`@latest`, version, branch, commit).
569
- - `agents rerun` — reuse a previous input snapshot; defaults to latest source.
570
- - `agents runs list/get/watch/cancel/compare/trace` — inspect and manage executions.
571
- - `agents runs artifacts list/fetch` — inventory and download run artifacts by path.
572
+ - `runs rerun` — reuse a previous input snapshot; defaults to latest source.
573
+ - `runs list/get/watch/cancel/compare/trace` — inspect and manage executions.
574
+ - `runs artifacts list/fetch` — inventory and download run artifacts by path.
572
575
  - `agents dataset` — manage persisted agent examples; runtime data, not Git source.
573
576
  - `agents file` — live read/diff inspection only.