@eigenpal/cli 0.6.17 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +10 -0
- package/README.md +9 -8
- package/dist/cli.js +644 -150
- package/dist/skill/SKILL.md +12 -11
- package/dist/skill/reference/cli/runs.md +22 -20
- package/dist/skill/reference/dataset-format.md +1 -1
- package/dist/skill/reference/debugging.md +19 -16
- package/dist/skill/reference/evaluators.md +3 -3
- package/dist/skill/reference/step-exec.md +6 -6
- package/dist/skill/reference/workflow-yaml.md +4 -2
- package/package.json +1 -1
package/dist/skill/SKILL.md
CHANGED
|
@@ -167,7 +167,7 @@ eigenpal agents save -m "Improve <agent> behavior"
|
|
|
167
167
|
SOURCE_REF="$(eigenpal git -- rev-parse HEAD)"
|
|
168
168
|
eigenpal run agents.<slug>@"$SOURCE_REF" --input-json '{"text":"hello"}' --wait
|
|
169
169
|
eigenpal run agents.<slug>@"$SOURCE_REF" --example example-name --wait
|
|
170
|
-
eigenpal runs get <run-id> --json | jq '
|
|
170
|
+
eigenpal runs get <run-id> --json | jq '{status, schemaValid, error, requestedSourceRef, resolvedGitRef, resolvedGitTag, resolvedCommitSha, cost}'
|
|
171
171
|
eigenpal runs artifacts list <run-id>
|
|
172
172
|
eigenpal runs trace <run-id> --out ./review/<run-id>/trace.jsonl
|
|
173
173
|
|
|
@@ -177,7 +177,7 @@ eigenpal agents sync agents.<slug>
|
|
|
177
177
|
|
|
178
178
|
# 7. Verify the released package.
|
|
179
179
|
eigenpal run agents.<slug> --example example-name --wait --json
|
|
180
|
-
eigenpal runs get <released-run-id> --json | jq '
|
|
180
|
+
eigenpal runs get <released-run-id> --json | jq '{status, schemaValid, error, resolvedGitTag, resolvedCommitSha, cost}'
|
|
181
181
|
```
|
|
182
182
|
|
|
183
183
|
File inputs must use schema field names when there is more than one file input:
|
|
@@ -270,7 +270,7 @@ eigenpal runs artifacts list <agent-execution-id>
|
|
|
270
270
|
eigenpal runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
|
|
271
271
|
eigenpal runs trace <agent-execution-id> --out ./review/<agent-execution-id>/trace.jsonl
|
|
272
272
|
eigenpal runs rerun <agent-execution-id> --wait
|
|
273
|
-
eigenpal runs rerun <agent-execution-id> --
|
|
273
|
+
eigenpal runs rerun <agent-execution-id> --version original --wait
|
|
274
274
|
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
|
|
275
275
|
--baseline \
|
|
276
276
|
--normalize-dates
|
|
@@ -284,10 +284,10 @@ take an agent execution id. Artifact fetches and comparisons write review
|
|
|
284
284
|
artifacts under `.eigenpal/artifacts/...` by default. Unqualified run-list
|
|
285
285
|
targets show all source refs; add `@<ref>` only when you want a specific
|
|
286
286
|
release, branch, tag, semver range, or commit. `runs list --json` returns
|
|
287
|
-
`{ runs, total, limit, offset }`; `runs get --json` returns
|
|
287
|
+
`{ runs, total, limit, offset }`; `runs get --json` returns the flattened run object at the top level.
|
|
288
288
|
|
|
289
289
|
`runs rerun <run-id>` reuses the previous input snapshot with `latest` source
|
|
290
|
-
by default. Use `--
|
|
290
|
+
by default. Use `--version original` only when you need to reproduce the
|
|
291
291
|
previous resolved version exactly.
|
|
292
292
|
|
|
293
293
|
Use source refs such as `latest`, `main`, exact versions/tags (`1.2.3`),
|
|
@@ -359,12 +359,14 @@ eigenpal workflow dataset example delete <workflow-id> <example-id> --yes
|
|
|
359
359
|
```bash
|
|
360
360
|
eigenpal run workflows.<workflow-id> --example <example-name>
|
|
361
361
|
eigenpal runs watch <execution-id>
|
|
362
|
-
eigenpal runs get <execution-id> --
|
|
362
|
+
eigenpal runs get <execution-id> --expand input,execution --json
|
|
363
363
|
eigenpal runs list <workflow-id> --type workflow --status failed --limit 10
|
|
364
364
|
eigenpal runs compare <execution-id-a> <execution-id-b>
|
|
365
365
|
eigenpal runs cancel <execution-id> --yes
|
|
366
366
|
```
|
|
367
367
|
|
|
368
|
+
`runs get --json` returns the CLI's flattened run view at the top level (`id`, `finished`, `output`, `files`, `error`, plus legacy aliases like `stepExecutions`). Prefer `--expand input,execution` for server-backed sections; `--include` only adds local workflow step projection fields on top of the default expand set.
|
|
369
|
+
|
|
368
370
|
### Run And Compare Experiments
|
|
369
371
|
|
|
370
372
|
```bash
|
|
@@ -405,7 +407,7 @@ and `agents sync`. Live file commands are read/compare tools.
|
|
|
405
407
|
|
|
406
408
|
```bash
|
|
407
409
|
eigenpal runs list <agent-id-or-slug> --type agent --status failed --limit 10
|
|
408
|
-
eigenpal runs get <agent-execution-id> --
|
|
410
|
+
eigenpal runs get <agent-execution-id> --expand input,execution --json | jq '{id, output, files, error, feedback: .feedback, expected: .expected}'
|
|
409
411
|
eigenpal runs artifacts list <agent-execution-id>
|
|
410
412
|
eigenpal runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
|
|
411
413
|
```
|
|
@@ -447,7 +449,7 @@ used when comparing future runs.
|
|
|
447
449
|
|
|
448
450
|
```bash
|
|
449
451
|
eigenpal runs rerun <agent-execution-id> --wait
|
|
450
|
-
eigenpal runs rerun <agent-execution-id> --
|
|
452
|
+
eigenpal runs rerun <agent-execution-id> --version original --wait
|
|
451
453
|
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id>
|
|
452
454
|
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
|
|
453
455
|
--baseline \
|
|
@@ -458,7 +460,7 @@ eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
|
|
|
458
460
|
By default, compare a new output against the reference execution's expected
|
|
459
461
|
artifacts. Use `--baseline` to compare actual outputs from both executions.
|
|
460
462
|
By default, rerun uses the previous inputs with latest source. Use
|
|
461
|
-
`--
|
|
463
|
+
`--version original` for provenance reproduction.
|
|
462
464
|
|
|
463
465
|
## Output And Exit Codes
|
|
464
466
|
|
|
@@ -468,7 +470,7 @@ script or agent needs to inspect the result with `jq`.
|
|
|
468
470
|
```bash
|
|
469
471
|
eigenpal runs list <workflow-id> --type workflow --json | jq '.runs[0].id'
|
|
470
472
|
eigenpal runs list <agent-id-or-slug> --type agent --json | jq '.runs[0].id'
|
|
471
|
-
eigenpal runs get <agent-execution-id> --json | jq '.
|
|
473
|
+
eigenpal runs get <agent-execution-id> --json | jq '.id'
|
|
472
474
|
```
|
|
473
475
|
|
|
474
476
|
General exit-code convention:
|
|
@@ -521,7 +523,6 @@ eigenpal
|
|
|
521
523
|
│ ├── clear-local
|
|
522
524
|
│ ├── dataset
|
|
523
525
|
│ ├── evaluators
|
|
524
|
-
│ ├── execution
|
|
525
526
|
│ ├── experiment
|
|
526
527
|
│ ├── versions
|
|
527
528
|
│ ├── step-type
|
|
@@ -73,10 +73,10 @@ runs
|
|
|
73
73
|
|
|
74
74
|
### Artifacts
|
|
75
75
|
|
|
76
|
-
| Command | Description
|
|
77
|
-
| --------------------------------------------------------------- |
|
|
78
|
-
| `eigenpal runs artifacts\|artifact list\|ls [options] <run-id>` | List available
|
|
79
|
-
| `eigenpal runs artifacts\|artifact fetch [options] <run-id>` | Download
|
|
76
|
+
| Command | Description |
|
|
77
|
+
| --------------------------------------------------------------- | ------------------------------------------------------ |
|
|
78
|
+
| `eigenpal runs artifacts\|artifact list\|ls [options] <run-id>` | List available run artifacts without downloading them. |
|
|
79
|
+
| `eigenpal runs artifacts\|artifact fetch [options] <run-id>` | Download run artifacts by canonical artifact path. |
|
|
80
80
|
|
|
81
81
|
### Feedback
|
|
82
82
|
|
|
@@ -145,12 +145,13 @@ Get one run.
|
|
|
145
145
|
|
|
146
146
|
### Options
|
|
147
147
|
|
|
148
|
-
| Flag
|
|
149
|
-
|
|
|
150
|
-
| `--base-url <url>`
|
|
151
|
-
| `--json`
|
|
152
|
-
| `--step <name>`
|
|
153
|
-
| `--
|
|
148
|
+
| Flag | Required | Default | Description |
|
|
149
|
+
| --------------------- | -------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
150
|
+
| `--base-url <url>` | no | | Server base URL |
|
|
151
|
+
| `--json` | no | | Output the raw server response as JSON |
|
|
152
|
+
| `--step <name>` | no | | For workflow runs, show only this step (or comma-separated list) |
|
|
153
|
+
| `--expand <sections>` | no | | Comma-separated server expand sections for GET /api/v1/runs/:id: input, usage, execution, debug. Use execution for status/schemaValid; completed runs include top-level output/files/error. |
|
|
154
|
+
| `--include <parts>` | no | `"feedback"` | Comma-separated local workflow step projection fields (error, duration, inputRef, …). Legacy expand names are mapped when possible; use --expand for server sections. |
|
|
154
155
|
|
|
155
156
|
### `eigenpal runs compare|diff [options] <reference-run-id> <run-id>`
|
|
156
157
|
|
|
@@ -187,14 +188,15 @@ Create a new run from a previous run's stored input snapshot.
|
|
|
187
188
|
|
|
188
189
|
### Options
|
|
189
190
|
|
|
190
|
-
| Flag | Required | Default | Description
|
|
191
|
-
| ---------------------- | -------- | ------- |
|
|
192
|
-
| `--base-url <url>` | no | | Server base URL
|
|
193
|
-
| `--json` | no | | Output the raw server response as JSON
|
|
194
|
-
| `--
|
|
195
|
-
| `--
|
|
196
|
-
| `--
|
|
197
|
-
| `--
|
|
191
|
+
| Flag | Required | Default | Description |
|
|
192
|
+
| ---------------------- | -------- | ------- | ----------------------------------------------------------------------------------------------------------------- |
|
|
193
|
+
| `--base-url <url>` | no | | Server base URL |
|
|
194
|
+
| `--json` | no | | Output the raw server response as JSON |
|
|
195
|
+
| `--version <version>` | no | | Version/source ref for the new run: latest (default), original, or an explicit ref (same grammar as eigenpal run) |
|
|
196
|
+
| `--source-ref <ref>` | no | | Alias for --version |
|
|
197
|
+
| `--wait` | no | | Poll until the rerun reaches a terminal status |
|
|
198
|
+
| `--interval <seconds>` | no | `2` | Polling interval in seconds |
|
|
199
|
+
| `--max-wait <seconds>` | no | `1800` | Maximum wait before exiting 2 |
|
|
198
200
|
|
|
199
201
|
### `eigenpal runs trace [options] <run-id>`
|
|
200
202
|
|
|
@@ -252,7 +254,7 @@ Cancel a run.
|
|
|
252
254
|
|
|
253
255
|
### `eigenpal runs artifacts|artifact list|ls [options] <run-id>`
|
|
254
256
|
|
|
255
|
-
List available
|
|
257
|
+
List available run artifacts without downloading them.
|
|
256
258
|
|
|
257
259
|
### Arguments
|
|
258
260
|
|
|
@@ -269,7 +271,7 @@ List available agent run artifacts without downloading them.
|
|
|
269
271
|
|
|
270
272
|
### `eigenpal runs artifacts|artifact fetch [options] <run-id>`
|
|
271
273
|
|
|
272
|
-
Download
|
|
274
|
+
Download run artifacts by canonical artifact path.
|
|
273
275
|
|
|
274
276
|
### Arguments
|
|
275
277
|
|
|
@@ -270,7 +270,7 @@ Pretty mode prints labelled `Inputs` / `Expected` / `Metadata` sections;
|
|
|
270
270
|
|
|
271
271
|
```bash
|
|
272
272
|
# 1. Look at what the workflow actually returned for the example
|
|
273
|
-
eigenpal runs get exec_… --json | jq '.
|
|
273
|
+
eigenpal runs get exec_… --json | jq '.output.data' > /tmp/correct.json
|
|
274
274
|
|
|
275
275
|
# 2. Patch the example's expected output in place. PATCH semantics:
|
|
276
276
|
# every flag you omit is left alone.
|
|
@@ -35,12 +35,15 @@ eigenpal runs get exec_…
|
|
|
35
35
|
eigenpal runs get <exec-id>
|
|
36
36
|
```
|
|
37
37
|
|
|
38
|
-
The payload includes:
|
|
39
|
-
|
|
40
|
-
- `
|
|
41
|
-
- `
|
|
42
|
-
- `
|
|
43
|
-
- `
|
|
38
|
+
The grouped run payload includes:
|
|
39
|
+
|
|
40
|
+
- `id`, `type`, `finished` — run identity and whether it reached a terminal status
|
|
41
|
+
- `timing` — created/started/completed timestamps and duration
|
|
42
|
+
- `execution.status` — `pending` | `running` | `completed` | `failed` | `cancelled` | `skipped`
|
|
43
|
+
- `output` — the workflow's final return value when `finished` and completed (or partial when failed)
|
|
44
|
+
- `error` — terminal failure message when the run failed or was cancelled
|
|
45
|
+
- `input` — the input the workflow was called with (included on detail fetches)
|
|
46
|
+
- `execution.steps` — one entry per step when you pass `--expand execution` (the CLI also mirrors this as `stepExecutions`):
|
|
44
47
|
- `stepName` — handle from `workflow.yaml`
|
|
45
48
|
- `status` — same set as the top-level. `skipped` means the step's
|
|
46
49
|
`if:` evaluated false, OR a `control.foreach` was passed an empty
|
|
@@ -61,21 +64,21 @@ The payload includes:
|
|
|
61
64
|
Quick triage with `jq`:
|
|
62
65
|
|
|
63
66
|
```bash
|
|
64
|
-
eigenpal runs get <exec-id> --json \
|
|
65
|
-
| jq '.
|
|
67
|
+
eigenpal runs get <exec-id> --expand execution --json \
|
|
68
|
+
| jq '.stepExecutions[] | { step: .stepName, status, error }'
|
|
66
69
|
```
|
|
67
70
|
|
|
68
71
|
Narrow to a single step:
|
|
69
72
|
|
|
70
73
|
```bash
|
|
71
|
-
eigenpal runs get <exec-id> --include input,output,error --json \
|
|
72
|
-
| jq '.
|
|
74
|
+
eigenpal runs get <exec-id> --expand execution --include input,output,error --json \
|
|
75
|
+
| jq '.stepExecutions[] | select(.stepName == "extract")'
|
|
73
76
|
```
|
|
74
77
|
|
|
75
78
|
Drill into one field:
|
|
76
79
|
|
|
77
80
|
```bash
|
|
78
|
-
eigenpal runs get <exec-id> --json | jq '.
|
|
81
|
+
eigenpal runs get <exec-id> --expand execution --json | jq '.stepExecutions[2].output.totalAmount'
|
|
79
82
|
```
|
|
80
83
|
|
|
81
84
|
## 3. Common failure modes
|
|
@@ -256,15 +259,15 @@ steps were faked. Re-import the dataset (`workflow dataset push
|
|
|
256
259
|
## 9. Watching a long batch
|
|
257
260
|
|
|
258
261
|
```bash
|
|
259
|
-
# Kicks off, prints { batchId,
|
|
262
|
+
# Kicks off, prints { batchId, total }
|
|
260
263
|
eigenpal workflow experiment run <workflow-id>
|
|
261
264
|
|
|
262
265
|
# --watch polls until every execution is terminal, prints a per-example
|
|
263
266
|
# failure summary, and exits non-zero if any failed/cancelled.
|
|
264
|
-
eigenpal workflow experiment status <workflow-id>
|
|
267
|
+
eigenpal workflow experiment status <workflow-id> <batch-id> --watch
|
|
265
268
|
|
|
266
269
|
# Without --watch: one-shot snapshot, no polling. Prefer --watch in CI / agents.
|
|
267
|
-
eigenpal workflow experiment status <workflow-id>
|
|
270
|
+
eigenpal workflow experiment status <workflow-id> <batch-id>
|
|
268
271
|
```
|
|
269
272
|
|
|
270
273
|
The terminal `--watch` output looks like:
|
|
@@ -289,8 +292,8 @@ returned before erroring) follow the suggested command:
|
|
|
289
292
|
|
|
290
293
|
```bash
|
|
291
294
|
eigenpal runs get exec_… # full payload
|
|
292
|
-
eigenpal runs get exec_… --
|
|
293
|
-
eigenpal runs get exec_… --json | jq -r '.
|
|
295
|
+
eigenpal runs get exec_… --expand execution --json | jq '.stepExecutions[] | select(.stepName == "<name>")'
|
|
296
|
+
eigenpal runs get exec_… --expand execution --json | jq -r '.stepExecutions[] | select(.error != null) | .error' | head -1
|
|
294
297
|
```
|
|
295
298
|
|
|
296
299
|
For per-execution live view, `eigenpal runs watch <exec-id>` does
|
|
@@ -210,17 +210,17 @@ Reports field-level issues:
|
|
|
210
210
|
eigenpal workflow evaluators push <workflow-id> --file evaluators.yaml
|
|
211
211
|
|
|
212
212
|
# Kick off a server-side batch eval over the whole pushed dataset.
|
|
213
|
-
# Returns { batchId,
|
|
213
|
+
# Returns { batchId, total }.
|
|
214
214
|
eigenpal workflow experiment run <workflow-id>
|
|
215
215
|
|
|
216
216
|
# Or restrict to a single example:
|
|
217
217
|
eigenpal workflow experiment run <workflow-id> --example-id evx_…
|
|
218
218
|
|
|
219
219
|
# Watch progress.
|
|
220
|
-
eigenpal workflow experiment status <workflow-id>
|
|
220
|
+
eigenpal workflow experiment status <workflow-id> <batch-id>
|
|
221
221
|
|
|
222
222
|
# Pull results when done.
|
|
223
|
-
eigenpal workflow experiment results <workflow-id>
|
|
223
|
+
eigenpal workflow experiment results <workflow-id> <batch-id> \
|
|
224
224
|
--format csv --out results.csv
|
|
225
225
|
```
|
|
226
226
|
|
|
@@ -11,8 +11,8 @@
|
|
|
11
11
|
>
|
|
12
12
|
> Calling `eigenpal workflow step exec <type>` today exits 2 with a
|
|
13
13
|
> redirect message. The server-side replacement is tracked in EIG-104:
|
|
14
|
-
> a
|
|
15
|
-
>
|
|
14
|
+
> a future server-routed endpoint that runs the same code path executions
|
|
15
|
+
> take.
|
|
16
16
|
|
|
17
17
|
## What to use instead
|
|
18
18
|
|
|
@@ -36,12 +36,12 @@ To inspect what a step actually produced (input / output / resolved
|
|
|
36
36
|
config), use:
|
|
37
37
|
|
|
38
38
|
```bash
|
|
39
|
-
eigenpal runs get <executionId> --json --include input,output,config
|
|
39
|
+
eigenpal runs get <executionId> --expand execution --json --include input,output,config
|
|
40
40
|
```
|
|
41
41
|
|
|
42
|
-
The
|
|
43
|
-
step
|
|
44
|
-
|
|
42
|
+
The CLI requests the run detail with `expand=execution` and projects
|
|
43
|
+
per-step `inputData` / `outputData` / `resolvedConfig` under shorter names
|
|
44
|
+
(`input` / `output` / `config`).
|
|
45
45
|
|
|
46
46
|
## Exit codes (current placeholder)
|
|
47
47
|
|
|
@@ -42,8 +42,10 @@ Required. Each entry is an object that adds an invocation surface:
|
|
|
42
42
|
- `{ type: manual }` — the workflow can be started manually
|
|
43
43
|
- `{ type: email, whitelist?: ... }` — inbound email
|
|
44
44
|
|
|
45
|
-
If `{ type: api }` is missing, `eigenpal run workflows.<slug>` rejects with
|
|
46
|
-
403 /
|
|
45
|
+
If `{ type: api }` is missing, `eigenpal run workflows.<slug>` rejects with
|
|
46
|
+
`403` / issue code `api_trigger_disabled` — enable the API trigger in the
|
|
47
|
+
workflow editor or dashboard settings. Runtime gates read the
|
|
48
|
+
`automation_triggers` projection (synced on workflow publish).
|
|
47
49
|
|
|
48
50
|
### `inputs`
|
|
49
51
|
|