@eigenpal/cli 0.7.2 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +89 -23
- package/README.md +8 -8
- package/dist/cli.js +9024 -8865
- package/dist/skill/SKILL.md +87 -415
- package/dist/skill/reference/cli/agents.md +25 -25
- package/dist/skill/reference/cli/runs.md +21 -0
- package/dist/skill/reference/cli/surface.md +1 -2
- package/dist/skill/reference/cli/workflow.md +10 -10
- package/dist/skill/reference/dataset-format.md +6 -6
- package/dist/skill/reference/debugging.md +5 -5
- package/dist/skill/reference/evaluators.md +7 -7
- package/dist/skill/reference/step-exec.md +2 -2
- package/dist/skill/reference/step-types.md +5 -5
- package/dist/skill/reference/workflow-yaml.md +4 -4
- package/dist/templates/blank/README.md +1 -1
- package/package.json +1 -1
- package/dist/skill/reference/cli/completion.md +0 -9
- package/dist/skill/reference/cli/git.md +0 -9
package/dist/skill/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: eigenpal
|
|
3
|
-
description:
|
|
3
|
+
description: Use the Eigenpal CLI to author workflows, manage agent source, run executions, inspect results, and download artifacts.
|
|
4
4
|
license: Apache-2.0
|
|
5
5
|
compatibility: Requires eigenpal CLI.
|
|
6
6
|
metadata:
|
|
@@ -9,161 +9,108 @@ metadata:
|
|
|
9
9
|
generatedBy: 'eigenpal@__CLI_VERSION__'
|
|
10
10
|
---
|
|
11
11
|
|
|
12
|
-
# Eigenpal
|
|
12
|
+
# Eigenpal CLI
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
Use long command names in generated commands and documentation. Short aliases may
|
|
15
|
+
appear in user examples, but do not prefer them.
|
|
15
16
|
|
|
16
|
-
|
|
17
|
-
datasets, evaluators, executions, experiments, versions, and schema
|
|
18
|
-
introspection.
|
|
19
|
-
- `eigenpal agents` — build and operate agent workspaces: agent files, triggers,
|
|
20
|
-
runs, traces, feedback, expected artifacts, and reruns.
|
|
17
|
+
## Choose A Command
|
|
21
18
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
19
|
+
- `eigenpal workflow` manages YAML workflow definitions, datasets, evaluators,
|
|
20
|
+
experiments, versions, and step schemas.
|
|
21
|
+
- `eigenpal agents` manages Git-backed agent source packages, shared resources,
|
|
22
|
+
releases, sync, secrets, datasets, and live file inspection.
|
|
23
|
+
- `eigenpal run` starts a workflow or agent run from a target.
|
|
24
|
+
- `eigenpal runs` inspects, watches, compares, reruns, cancels, and downloads
|
|
25
|
+
artifacts for workflow or agent runs.
|
|
25
26
|
|
|
26
|
-
|
|
27
|
-
- `runs artifact` = `runs artifacts`
|
|
28
|
-
- `run` = start a workflow or agent run
|
|
29
|
-
- `workflow exp` = `workflow experiment`
|
|
30
|
-
- `ls` = `list` anywhere in the CLI
|
|
31
|
-
- `diff` = `compare` anywhere in the CLI
|
|
27
|
+
Use stable ids in scripts:
|
|
32
28
|
|
|
33
|
-
|
|
29
|
+
- Workflow id: `wf_...`
|
|
30
|
+
- Workflow execution id: `evx_...`
|
|
31
|
+
- Agent execution id: `aex_...`
|
|
32
|
+
- Experiment batch id: `evb_...`
|
|
33
|
+
- Dataset example id: `ex_...`
|
|
34
34
|
|
|
35
|
-
|
|
36
|
-
- [`reference/cli/agents.md`](reference/cli/agents.md)
|
|
37
|
-
- [`reference/cli/runs.md`](reference/cli/runs.md)
|
|
38
|
-
- [`reference/cli/git.md`](reference/cli/git.md)
|
|
39
|
-
|
|
40
|
-
## Pick The Surface
|
|
41
|
-
|
|
42
|
-
Choose `workflow` when the work is about a workflow definition, dataset,
|
|
43
|
-
evaluator, or experiment. Choose root `run` to create workflow runs from local examples.
|
|
44
|
-
|
|
45
|
-
Choose `agents` when the work is about an agent workspace, agent file, trigger,
|
|
46
|
-
or starting an agent run.
|
|
47
|
-
|
|
48
|
-
Choose `runs` when the work is about inspecting, watching, comparing,
|
|
49
|
-
cancelling, rerunning, downloading artifacts for, or attaching feedback/expected
|
|
50
|
-
artifacts to any workflow or agent run.
|
|
51
|
-
|
|
52
|
-
Agents and workflows are related, but their iteration loops are different:
|
|
53
|
-
workflow experiments evaluate datasets with evaluators; agent workflows do not
|
|
54
|
-
currently expose that same experiment/evaluator loop through the CLI.
|
|
55
|
-
|
|
56
|
-
## Workflow Iteration Loop
|
|
35
|
+
## Workflow Loop
|
|
57
36
|
|
|
58
37
|
```bash
|
|
59
|
-
#
|
|
38
|
+
# Create or edit a workflow project.
|
|
60
39
|
eigenpal init workflow invoices --template pdf-extraction
|
|
61
40
|
cd invoices
|
|
62
41
|
$EDITOR workflow.yaml
|
|
63
42
|
|
|
64
|
-
#
|
|
43
|
+
# Inspect step schemas before editing config.
|
|
65
44
|
eigenpal workflow step-type list --search extract
|
|
66
45
|
eigenpal workflow step-type get ai.extract --json | jq '.configSchema.properties'
|
|
67
46
|
|
|
68
|
-
#
|
|
47
|
+
# Validate and push.
|
|
69
48
|
eigenpal workflow validate ./workflow.yaml
|
|
70
49
|
eigenpal workflow push --file workflow.yaml
|
|
71
50
|
|
|
72
|
-
#
|
|
51
|
+
# Run one example.
|
|
73
52
|
eigenpal run workflows.<workflow-id> --example <example-name>
|
|
74
53
|
eigenpal runs watch <execution-id>
|
|
75
54
|
eigenpal runs get <execution-id> --json
|
|
76
55
|
|
|
77
|
-
#
|
|
56
|
+
# Push dataset and evaluators.
|
|
78
57
|
eigenpal workflow dataset validate ./dataset
|
|
79
58
|
eigenpal workflow dataset push <workflow-id> --file ./dataset --mode replace
|
|
80
59
|
eigenpal workflow evaluators validate ./evaluators.yaml
|
|
81
60
|
eigenpal workflow evaluators push <workflow-id> --file ./evaluators.yaml
|
|
82
61
|
|
|
83
|
-
#
|
|
62
|
+
# Run an experiment.
|
|
84
63
|
eigenpal workflow experiment run <workflow-id>
|
|
85
64
|
eigenpal workflow experiment watch <workflow-id> <batch-id>
|
|
86
65
|
eigenpal workflow experiment results <workflow-id> <batch-id> --format csv --out results.csv
|
|
87
66
|
```
|
|
88
67
|
|
|
89
|
-
|
|
90
|
-
also accept the workflow slug for convenience, but use the id in scripts.
|
|
91
|
-
|
|
92
|
-
Useful workflow ids are not interchangeable:
|
|
93
|
-
|
|
94
|
-
- `<workflow-id>` identifies a workflow (`wf_...`).
|
|
95
|
-
- `<execution-id>` identifies one workflow run.
|
|
96
|
-
- `<batch-id>` identifies one experiment batch.
|
|
97
|
-
- `<example-id>` identifies one dataset example.
|
|
98
|
-
|
|
99
|
-
### Organize workflows into folders
|
|
100
|
-
|
|
101
|
-
Folders live on the server, not in `workflow.yaml`. `workflow push` and
|
|
102
|
-
`workflow pull` do not read or write folder placement. Use `workflow move`
|
|
103
|
-
to move workflows between dashboard folders:
|
|
68
|
+
Folders are server metadata, not `workflow.yaml` content:
|
|
104
69
|
|
|
105
70
|
```bash
|
|
106
71
|
eigenpal workflow move <workflow-id> --folder billing/invoices
|
|
107
72
|
eigenpal workflow move <workflow-id> --folder /
|
|
108
73
|
```
|
|
109
74
|
|
|
110
|
-
|
|
111
|
-
Missing folders in the path are created automatically.
|
|
112
|
-
Default to creating workflows at the top level. Do not move a workflow into
|
|
113
|
-
a folder unless the user specifically asks for that placement.
|
|
75
|
+
## Agent Source Loop
|
|
114
76
|
|
|
115
|
-
|
|
77
|
+
Agents are Git-backed source packages. Edit source in Git, save the branch, test
|
|
78
|
+
the exact commit, then release and sync.
|
|
116
79
|
|
|
117
|
-
|
|
118
|
-
from the organization source repository; editing live files is not the normal
|
|
119
|
-
workflow. Prefer the Eigenpal CLI over raw provider/API calls because it handles
|
|
120
|
-
validation, source refs, release tags, and server sync consistently. Cloned
|
|
121
|
-
source repos also work with raw Git and IDE source-control panels: `agents clone`
|
|
122
|
-
and `eigenpal git -- ...` configure a repo-local credential helper for the
|
|
123
|
-
Eigenpal Git remote, so `git status`, `git log`, `git pull`, and `git push`
|
|
124
|
-
can authenticate without copying API keys into `.git/config`.
|
|
125
|
-
|
|
126
|
-
Core mental model:
|
|
80
|
+
Key rules:
|
|
127
81
|
|
|
128
82
|
- `agents/<slug>/` is one agent package.
|
|
129
83
|
- `resources/skills/<slug>/`, `resources/knowledge/<slug>/`,
|
|
130
84
|
`resources/templates/<slug>/`, and `resources/rules/<slug>/` are shared
|
|
131
|
-
packages
|
|
132
|
-
- `eigenpal agents save` validates
|
|
133
|
-
|
|
134
|
-
- `eigenpal agents
|
|
135
|
-
|
|
136
|
-
- `
|
|
137
|
-
manifest to the server DB, including triggers.
|
|
138
|
-
- `@latest` means latest released package, not your current branch.
|
|
139
|
-
- To test current branch work before release, run the exact commit SHA from
|
|
140
|
-
`eigenpal git -- rev-parse HEAD`.
|
|
141
|
-
|
|
142
|
-
Use branches. Do not casually edit `main` unless the user explicitly wants that:
|
|
85
|
+
packages.
|
|
86
|
+
- `eigenpal agents save` validates, commits if dirty, and pushes the branch.
|
|
87
|
+
- `eigenpal agents release patch agents/<slug>` creates an immutable release.
|
|
88
|
+
- `eigenpal agents sync agents.<slug>` applies the latest released manifest and
|
|
89
|
+
triggers to the server.
|
|
90
|
+
- `@latest` means latest released package, not the current branch.
|
|
143
91
|
|
|
144
92
|
```bash
|
|
145
|
-
#
|
|
93
|
+
# Clone source and create a branch.
|
|
146
94
|
eigenpal agents clone --out ./source
|
|
147
95
|
cd ./source
|
|
148
96
|
eigenpal git -- switch -c <short-task-branch>
|
|
149
97
|
|
|
150
|
-
#
|
|
98
|
+
# Inspect source state.
|
|
151
99
|
eigenpal agents status
|
|
152
100
|
eigenpal agents show agents.<slug> --json
|
|
153
101
|
eigenpal agents versions agents.<slug> --json
|
|
154
102
|
|
|
155
|
-
#
|
|
103
|
+
# Edit source.
|
|
156
104
|
$EDITOR agents/<slug>/AGENT.md
|
|
157
105
|
$EDITOR agents/<slug>/eigenpal.yaml
|
|
158
106
|
$EDITOR agents/<slug>/input-schema.json agents/<slug>/output-schema.json
|
|
159
|
-
$EDITOR agents/<slug>/skills/<skill>/SKILL.md agents/<slug>/skills/<skill>/run.py
|
|
160
107
|
|
|
161
|
-
#
|
|
108
|
+
# Validate and save.
|
|
162
109
|
eigenpal agents validate agents/<slug>
|
|
163
110
|
eigenpal agents deps --dir agents/<slug>
|
|
164
111
|
eigenpal agents save -m "Improve <agent> behavior"
|
|
165
112
|
|
|
166
|
-
#
|
|
113
|
+
# Test the saved commit.
|
|
167
114
|
SOURCE_REF="$(eigenpal git -- rev-parse HEAD)"
|
|
168
115
|
eigenpal run agents.<slug>@"$SOURCE_REF" --input-json '{"text":"hello"}' --wait
|
|
169
116
|
eigenpal run agents.<slug>@"$SOURCE_REF" --example example-name --wait
|
|
@@ -171,22 +118,15 @@ eigenpal runs get <run-id> --json | jq '{status, schemaValid, error, requestedSo
|
|
|
171
118
|
eigenpal runs artifacts list <run-id>
|
|
172
119
|
eigenpal runs trace <run-id> --out ./review/<run-id>/trace.jsonl
|
|
173
120
|
|
|
174
|
-
#
|
|
121
|
+
# Release and sync.
|
|
175
122
|
eigenpal agents release patch agents/<slug>
|
|
176
123
|
eigenpal agents sync agents.<slug>
|
|
177
|
-
|
|
178
|
-
# 7. Verify the released package.
|
|
179
|
-
eigenpal run agents.<slug> --example example-name --wait --json
|
|
180
|
-
eigenpal runs get <released-run-id> --json | jq '{status, schemaValid, error, resolvedGitTag, resolvedCommitSha, cost}'
|
|
181
124
|
```
|
|
182
125
|
|
|
183
|
-
|
|
126
|
+
Use schema field names for file inputs when an agent has more than one file
|
|
127
|
+
field:
|
|
184
128
|
|
|
185
129
|
```bash
|
|
186
|
-
eigenpal run agents.<slug> \
|
|
187
|
-
--input-file start_info_doc=./ZFRP.docx \
|
|
188
|
-
--wait
|
|
189
|
-
|
|
190
130
|
eigenpal run agents.<slug> \
|
|
191
131
|
--input-file zfzal_doc=./ZFZAL.pdf \
|
|
192
132
|
--input-file lv_doc=./list_vlastnictva.pdf \
|
|
@@ -194,31 +134,24 @@ eigenpal run agents.<slug> \
|
|
|
194
134
|
--wait
|
|
195
135
|
```
|
|
196
136
|
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
`agents release` + `agents sync`.
|
|
137
|
+
`agents file list/get/diff` are read/compare commands only. Change agent files
|
|
138
|
+
through Git source, then use `agents save`, `agents release`, and `agents sync`.
|
|
200
139
|
|
|
201
|
-
|
|
140
|
+
## Shared Resource Updates
|
|
202
141
|
|
|
203
|
-
|
|
204
|
-
|
|
142
|
+
Do not move every dependent agent automatically. Find dependents, ask which ones
|
|
143
|
+
to update, then release and sync only those agents.
|
|
205
144
|
|
|
206
145
|
```bash
|
|
207
|
-
# 1. Edit and validate the shared package.
|
|
208
146
|
$EDITOR resources/skills/<slug>/SKILL.md resources/skills/<slug>/run.py
|
|
209
147
|
eigenpal agents validate resources/skills/<slug>
|
|
210
148
|
|
|
211
|
-
# 2. Find dependent agents/resources.
|
|
212
149
|
rg '<slug>|resources/skills/<slug>|resources/knowledge/<slug>' agents/ resources/
|
|
213
150
|
eigenpal agents deps --dir agents/<dependent-slug>
|
|
214
151
|
|
|
215
|
-
# 3. Ask the user which dependents should move to the new resource version.
|
|
216
|
-
|
|
217
|
-
# 4. Release the shared package.
|
|
218
152
|
eigenpal agents save -m "Improve shared <slug>"
|
|
219
153
|
eigenpal agents release patch resources/skills/<slug>
|
|
220
154
|
|
|
221
|
-
# 5. Bump selected agents' eigenpal.yaml dependency refs, then release/sync them.
|
|
222
155
|
$EDITOR agents/<dependent-slug>/eigenpal.yaml
|
|
223
156
|
eigenpal agents validate agents/<dependent-slug>
|
|
224
157
|
eigenpal agents save -m "Bump <slug> dependency"
|
|
@@ -226,206 +159,43 @@ eigenpal agents release patch agents/<dependent-slug>
|
|
|
226
159
|
eigenpal agents sync agents.<dependent-slug>
|
|
227
160
|
```
|
|
228
161
|
|
|
229
|
-
After
|
|
230
|
-
|
|
231
|
-
version and commit that inference actually installed.
|
|
162
|
+
After rollout, run one affected agent and inspect `eigenpal.lock` from run
|
|
163
|
+
artifacts to confirm the dependency version.
|
|
232
164
|
|
|
233
|
-
|
|
165
|
+
## Agent Datasets
|
|
234
166
|
|
|
235
|
-
Agent
|
|
236
|
-
|
|
237
|
-
dataset CLI:
|
|
167
|
+
Agent examples are runtime data, not Git source. `agents save` does not persist
|
|
168
|
+
them.
|
|
238
169
|
|
|
239
170
|
```bash
|
|
240
|
-
# Builder sandboxes use /workspace/evals; local projects often use ./dataset.
|
|
241
171
|
eigenpal agents dataset validate /workspace/evals --agent-dir agents/<slug>
|
|
242
172
|
eigenpal agents dataset push agents.<slug> --file /workspace/evals
|
|
243
173
|
eigenpal agents dataset list agents.<slug>
|
|
244
174
|
eigenpal agents dataset pull agents.<slug> --out ./dataset
|
|
245
175
|
```
|
|
246
176
|
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
```text
|
|
250
|
-
evals/<name>/
|
|
251
|
-
input.json # scalar/object args
|
|
252
|
-
input/<field>.* # file input, when needed
|
|
253
|
-
expected.json # partial expected output: only stable fields
|
|
254
|
-
expected/<file> # golden file outputs
|
|
255
|
-
feedback.md # optional notes
|
|
256
|
-
```
|
|
257
|
-
|
|
258
|
-
For `expected.json`, include only fields that must be identical on every correct
|
|
259
|
-
run. Omit non-deterministic values such as timestamps, random IDs, and free-form
|
|
260
|
-
LLM text. Validate the actual agent output with the output schema; validate the
|
|
261
|
-
expected file as a partial assertion.
|
|
262
|
-
Run a persisted example with `eigenpal run agents.<slug>@<ref> --example <name> --wait`.
|
|
263
|
-
|
|
264
|
-
### Debug Agent Runs
|
|
265
|
-
|
|
266
|
-
```bash
|
|
267
|
-
eigenpal runs list agents.<slug> --compact
|
|
268
|
-
eigenpal runs list agents.<slug> --type agent --status failed --compact
|
|
269
|
-
eigenpal runs artifacts list <agent-execution-id>
|
|
270
|
-
eigenpal runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
|
|
271
|
-
eigenpal runs trace <agent-execution-id> --out ./review/<agent-execution-id>/trace.jsonl
|
|
272
|
-
eigenpal runs rerun <agent-execution-id> --wait
|
|
273
|
-
eigenpal runs rerun <agent-execution-id> --version original --wait
|
|
274
|
-
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
|
|
275
|
-
--baseline \
|
|
276
|
-
--normalize-dates
|
|
277
|
-
|
|
278
|
-
eigenpal runs feedback resolve <source-agent-execution-id> \
|
|
279
|
-
--message "Fixed and verified in <new-agent-execution-id>."
|
|
280
|
-
```
|
|
281
|
-
|
|
282
|
-
Agent commands commonly accept `<agent-id-or-slug>`. Agent execution commands
|
|
283
|
-
take an agent execution id. Artifact fetches and comparisons write review
|
|
284
|
-
artifacts under `.eigenpal/artifacts/...` by default. Unqualified run-list
|
|
285
|
-
targets show all source refs; add `@<ref>` only when you want a specific
|
|
286
|
-
release, branch, tag, semver range, or commit. `runs list --json` returns
|
|
287
|
-
`{ runs, total, limit, offset }`; `runs get --json` returns the flattened run object at the top level.
|
|
288
|
-
|
|
289
|
-
`runs rerun <run-id>` reuses the previous input snapshot with `latest` source
|
|
290
|
-
by default. Use `--version original` only when you need to reproduce the
|
|
291
|
-
previous resolved version exactly.
|
|
292
|
-
|
|
293
|
-
Use source refs such as `latest`, `main`, exact versions/tags (`1.2.3`),
|
|
294
|
-
semver ranges (`1.2.x`, `1.x`), or exact commit SHAs when you need provenance.
|
|
295
|
-
|
|
296
|
-
### Git-backed secrets
|
|
297
|
-
|
|
298
|
-
Set secrets with the authenticated CLI — plaintext goes to
|
|
299
|
-
`POST /api/v1/source/secrets/encrypt`; only ciphertext is written to
|
|
300
|
-
`secrets.enc.yaml`. Organization decrypt keys never leave the server.
|
|
301
|
-
|
|
302
|
-
```bash
|
|
303
|
-
echo -n "$VALUE" | eigenpal agents secret set OPENAI_API_KEY --stdin
|
|
304
|
-
eigenpal agents secret import ./local.env
|
|
305
|
-
eigenpal agents save -m "Add API key"
|
|
306
|
-
```
|
|
307
|
-
|
|
308
|
-
Runtime sandboxes decrypt via `eigenpal agents env pull` →
|
|
309
|
-
`POST /api/v1/source/secrets/decrypt` (same API key auth).
|
|
310
|
-
|
|
311
|
-
## Workflow Recipes
|
|
312
|
-
|
|
313
|
-
### Author And Validate A Workflow
|
|
314
|
-
|
|
315
|
-
```bash
|
|
316
|
-
eigenpal workflow step-type list
|
|
317
|
-
eigenpal workflow step-type get <step-type> --json
|
|
318
|
-
eigenpal workflow validate ./workflow.yaml
|
|
319
|
-
eigenpal workflow push --file workflow.yaml
|
|
320
|
-
```
|
|
321
|
-
|
|
322
|
-
The validator reports structured field-path errors. If the error hints at a
|
|
323
|
-
step type, inspect that type with `eigenpal workflow step-type get <type>`.
|
|
324
|
-
|
|
325
|
-
### Manage A Dataset
|
|
326
|
-
|
|
327
|
-
```bash
|
|
328
|
-
eigenpal workflow dataset validate ./dataset
|
|
329
|
-
eigenpal workflow dataset push <workflow-id> --file ./dataset --mode append
|
|
330
|
-
eigenpal workflow dataset list <workflow-id>
|
|
331
|
-
eigenpal workflow dataset pull <workflow-id> --out dataset.zip
|
|
332
|
-
```
|
|
333
|
-
|
|
334
|
-
Dataset archives use this folder shape:
|
|
335
|
-
|
|
336
|
-
```text
|
|
337
|
-
dataset/
|
|
338
|
-
└── examples/
|
|
339
|
-
└── <example-name>/
|
|
340
|
-
├── input/
|
|
341
|
-
│ ├── arguments.json
|
|
342
|
-
│ └── <file-argument-name>/<file>
|
|
343
|
-
├── expected/output.json
|
|
344
|
-
└── meta.json
|
|
345
|
-
```
|
|
177
|
+
Expected outputs should include stable fields only. Omit timestamps, random IDs,
|
|
178
|
+
and open-ended LLM text.
|
|
346
179
|
|
|
347
|
-
|
|
348
|
-
whole dataset:
|
|
180
|
+
## Runs And Artifacts
|
|
349
181
|
|
|
350
182
|
```bash
|
|
351
|
-
eigenpal
|
|
352
|
-
eigenpal
|
|
353
|
-
eigenpal workflow dataset example create <workflow-id> --name edge-case --input-json '{"language":"en"}'
|
|
354
|
-
eigenpal workflow dataset example delete <workflow-id> <example-id> --yes
|
|
355
|
-
```
|
|
356
|
-
|
|
357
|
-
### Run And Compare Workflow Executions
|
|
358
|
-
|
|
359
|
-
```bash
|
|
360
|
-
eigenpal run workflows.<workflow-id> --example <example-name>
|
|
183
|
+
eigenpal runs list <workflow-id> --type workflow --status failed --limit 10
|
|
184
|
+
eigenpal runs list <agent-id-or-slug> --type agent --status failed --compact
|
|
361
185
|
eigenpal runs watch <execution-id>
|
|
362
186
|
eigenpal runs get <execution-id> --expand input,execution --json
|
|
363
|
-
eigenpal runs list <
|
|
364
|
-
eigenpal runs
|
|
365
|
-
eigenpal runs
|
|
187
|
+
eigenpal runs artifacts list <execution-id>
|
|
188
|
+
eigenpal runs artifacts fetch <execution-id> --include all --out ./review/<execution-id>
|
|
189
|
+
eigenpal runs trace <execution-id> --out ./review/<execution-id>/trace.jsonl
|
|
190
|
+
eigenpal runs rerun <execution-id> --wait
|
|
191
|
+
eigenpal runs rerun <execution-id> --version original --wait
|
|
192
|
+
eigenpal runs compare <source-execution-id> <new-execution-id> --baseline --normalize-dates
|
|
366
193
|
```
|
|
367
194
|
|
|
368
|
-
`runs
|
|
195
|
+
`runs rerun` reuses the previous input snapshot with latest source by default.
|
|
196
|
+
Use `--version original` only to reproduce the previous resolved version.
|
|
369
197
|
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
```bash
|
|
373
|
-
eigenpal workflow experiment run <workflow-id>
|
|
374
|
-
eigenpal workflow experiment status <workflow-id> <batch-id> --json
|
|
375
|
-
eigenpal workflow experiment watch <workflow-id> <batch-id> --format json
|
|
376
|
-
eigenpal workflow experiment results <workflow-id> <batch-id> --format csv --out results.csv
|
|
377
|
-
eigenpal workflow experiment compare <batch-id-a> <batch-id-b> --regression-threshold 0.05
|
|
378
|
-
```
|
|
379
|
-
|
|
380
|
-
`workflow experiment compare` resolves the owning workflow from the batch ids;
|
|
381
|
-
both batches must belong to the same workflow.
|
|
382
|
-
|
|
383
|
-
## Agent Recipes
|
|
384
|
-
|
|
385
|
-
### Inspect Agent Source And Live Files
|
|
386
|
-
|
|
387
|
-
```bash
|
|
388
|
-
eigenpal agents list
|
|
389
|
-
eigenpal agents show agents.<slug> --json
|
|
390
|
-
eigenpal agents versions agents.<slug> --json
|
|
391
|
-
eigenpal agents clone --out ./source
|
|
392
|
-
cd ./source
|
|
393
|
-
eigenpal agents status
|
|
394
|
-
eigenpal agents deps --dir agents/<slug>
|
|
395
|
-
|
|
396
|
-
# Live file inspection only. Do not edit through this surface.
|
|
397
|
-
eigenpal agents file list <agent-id-or-slug>
|
|
398
|
-
eigenpal agents file list <agent-id-or-slug> --path agent/knowledge
|
|
399
|
-
eigenpal agents file get <agent-id-or-slug> agent/instructions.md --out instructions.md
|
|
400
|
-
eigenpal agents file diff <agent-id-or-slug> agent/instructions.md instructions.md
|
|
401
|
-
```
|
|
402
|
-
|
|
403
|
-
Edit `agents/<slug>/...` in Git source, then `agents save`, `agents release`,
|
|
404
|
-
and `agents sync`. Live file commands are read/compare tools.
|
|
405
|
-
|
|
406
|
-
### Inspect Agent Executions
|
|
407
|
-
|
|
408
|
-
```bash
|
|
409
|
-
eigenpal runs list <agent-id-or-slug> --type agent --status failed --limit 10
|
|
410
|
-
eigenpal runs get <agent-execution-id> --expand input,execution --json | jq '{id, output, files, error, feedback: .feedback, expected: .expected}'
|
|
411
|
-
eigenpal runs artifacts list <agent-execution-id>
|
|
412
|
-
eigenpal runs artifacts fetch <agent-execution-id> --include all --out ./review/<agent-execution-id>
|
|
413
|
-
```
|
|
414
|
-
|
|
415
|
-
Use `--compact` on execution lists when you only need triage rows.
|
|
416
|
-
|
|
417
|
-
### Download Agent Traces
|
|
418
|
-
|
|
419
|
-
```bash
|
|
420
|
-
eigenpal runs trace <agent-execution-id> --out trace.jsonl
|
|
421
|
-
eigenpal runs trace <agent-execution-id> | jq -r 'select(.toolName? or .tool_name? or .tool?)'
|
|
422
|
-
eigenpal runs trace <agent-execution-id> | rg 'error|tool'
|
|
423
|
-
```
|
|
424
|
-
|
|
425
|
-
`trace` streams the raw JSONL to stdout by default. Use shell tools such as
|
|
426
|
-
`jq`, `rg`, or `awk` for filtering; use `--out` when you want to save the file.
|
|
427
|
-
|
|
428
|
-
### Manage Agent Feedback And Expected Artifacts
|
|
198
|
+
## Feedback And Expected Artifacts
|
|
429
199
|
|
|
430
200
|
```bash
|
|
431
201
|
eigenpal runs feedback update <agent-execution-id> \
|
|
@@ -442,136 +212,38 @@ eigenpal runs expected copy-output <agent-execution-id> result.json --name expec
|
|
|
442
212
|
eigenpal runs expected pull <agent-execution-id> --out expected/
|
|
443
213
|
```
|
|
444
214
|
|
|
445
|
-
|
|
446
|
-
used when comparing future runs.
|
|
447
|
-
|
|
448
|
-
### Rerun And Compare Agent Executions
|
|
449
|
-
|
|
450
|
-
```bash
|
|
451
|
-
eigenpal runs rerun <agent-execution-id> --wait
|
|
452
|
-
eigenpal runs rerun <agent-execution-id> --version original --wait
|
|
453
|
-
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id>
|
|
454
|
-
eigenpal runs compare <source-agent-execution-id> <new-agent-execution-id> \
|
|
455
|
-
--baseline \
|
|
456
|
-
--normalize-dates \
|
|
457
|
-
--fail-on-diff
|
|
458
|
-
```
|
|
459
|
-
|
|
460
|
-
By default, compare a new output against the reference execution's expected
|
|
461
|
-
artifacts. Use `--baseline` to compare actual outputs from both executions.
|
|
462
|
-
By default, rerun uses the previous inputs with latest source. Use
|
|
463
|
-
`--version original` for provenance reproduction.
|
|
215
|
+
## Output
|
|
464
216
|
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
Most `list`, `get`, and mutating commands support `--json`. Prefer JSON when a
|
|
468
|
-
script or agent needs to inspect the result with `jq`.
|
|
217
|
+
Prefer `--json` for scripts and agent work.
|
|
469
218
|
|
|
470
219
|
```bash
|
|
471
220
|
eigenpal runs list <workflow-id> --type workflow --json | jq '.runs[0].id'
|
|
472
|
-
eigenpal runs
|
|
473
|
-
eigenpal runs get <agent-execution-id> --json | jq '.id'
|
|
221
|
+
eigenpal runs get <execution-id> --json | jq '.id'
|
|
474
222
|
```
|
|
475
223
|
|
|
476
|
-
|
|
224
|
+
Exit codes:
|
|
477
225
|
|
|
478
226
|
| Code | Meaning |
|
|
479
227
|
| --- | --- |
|
|
480
|
-
| 0 | Success
|
|
481
|
-
| 1 | Runtime or command failure
|
|
482
|
-
| 2 | Invalid invocation, unsupported option, or
|
|
483
|
-
|
|
484
|
-
Status and progress messages go to stderr. Data intended for piping goes to
|
|
485
|
-
stdout.
|
|
486
|
-
|
|
487
|
-
## Schema And Reference Files
|
|
228
|
+
| 0 | Success |
|
|
229
|
+
| 1 | Runtime or command failure |
|
|
230
|
+
| 2 | Invalid invocation, unsupported option, timeout, or deadline |
|
|
488
231
|
|
|
489
|
-
|
|
232
|
+
Status messages go to stderr. Pipeable data goes to stdout.
|
|
490
233
|
|
|
491
|
-
|
|
492
|
-
- [`reference/step-types.md`](reference/step-types.md) — step-type catalog
|
|
493
|
-
- [`reference/step-exec.md`](reference/step-exec.md) — single-step execution command
|
|
494
|
-
- [`reference/dataset-format.md`](reference/dataset-format.md) — dataset folder format
|
|
495
|
-
- [`reference/evaluators.md`](reference/evaluators.md) — evaluator configuration
|
|
496
|
-
- [`reference/debugging.md`](reference/debugging.md) — workflow run debugging
|
|
234
|
+
## References
|
|
497
235
|
|
|
498
|
-
|
|
236
|
+
Use generated references for exact flags and defaults:
|
|
499
237
|
|
|
500
238
|
- [`reference/cli/workflow.md`](reference/cli/workflow.md)
|
|
501
239
|
- [`reference/cli/agents.md`](reference/cli/agents.md)
|
|
240
|
+
- [`reference/cli/runs.md`](reference/cli/runs.md)
|
|
502
241
|
- [`reference/cli/auth.md`](reference/cli/auth.md)
|
|
503
|
-
- [`reference/
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
eigenpal
|
|
512
|
-
├── auth
|
|
513
|
-
│ ├── login
|
|
514
|
-
│ ├── list
|
|
515
|
-
│ └── use
|
|
516
|
-
├── status
|
|
517
|
-
├── workflow
|
|
518
|
-
│ ├── list
|
|
519
|
-
│ ├── pull
|
|
520
|
-
│ ├── push
|
|
521
|
-
│ ├── validate
|
|
522
|
-
│ ├── move
|
|
523
|
-
│ ├── clear-local
|
|
524
|
-
│ ├── dataset
|
|
525
|
-
│ ├── evaluators
|
|
526
|
-
│ ├── experiment
|
|
527
|
-
│ ├── versions
|
|
528
|
-
│ ├── step-type
|
|
529
|
-
│ └── evaluator-type
|
|
530
|
-
├── agents
|
|
531
|
-
│ ├── run
|
|
532
|
-
│ ├── rerun
|
|
533
|
-
│ ├── list
|
|
534
|
-
│ ├── validate
|
|
535
|
-
│ ├── clone
|
|
536
|
-
│ ├── install
|
|
537
|
-
│ ├── init
|
|
538
|
-
│ ├── pull
|
|
539
|
-
│ ├── commit
|
|
540
|
-
│ ├── save
|
|
541
|
-
│ ├── push
|
|
542
|
-
│ ├── release
|
|
543
|
-
│ ├── sync
|
|
544
|
-
│ ├── status
|
|
545
|
-
│ ├── deps
|
|
546
|
-
│ ├── clean
|
|
547
|
-
│ ├── doctor
|
|
548
|
-
│ ├── show
|
|
549
|
-
│ ├── versions
|
|
550
|
-
│ ├── secret
|
|
551
|
-
│ ├── env
|
|
552
|
-
│ ├── file
|
|
553
|
-
│ ├── dataset
|
|
554
|
-
│ ├── runs
|
|
555
|
-
│ └── experiment
|
|
556
|
-
│ └── session
|
|
557
|
-
└── skill
|
|
558
|
-
├── install
|
|
559
|
-
├── uninstall
|
|
560
|
-
└── list
|
|
561
|
-
```
|
|
562
|
-
|
|
563
|
-
Agent command roles:
|
|
564
|
-
|
|
565
|
-
- `agents clone` — clone organization source and configure raw Git/IDE auth.
|
|
566
|
-
- `agents pull` — fast-forward source from `origin/main`; not datasets or run artifacts.
|
|
567
|
-
- `agents status` / `agents doctor` / `agents clean` — inspect source health.
|
|
568
|
-
- `agents validate` / `agents deps` — validate one package and inspect workspace dependencies.
|
|
569
|
-
- `agents save` — validate, commit if dirty, and push the current branch.
|
|
570
|
-
- `agents release` — create immutable package release tags.
|
|
571
|
-
- `agents sync` — apply latest released agent manifest/triggers to the server.
|
|
572
|
-
- `run` — start a workflow or agent run from a target ref (`@latest`, version, branch, commit).
|
|
573
|
-
- `runs rerun` — reuse a previous input snapshot; defaults to latest source.
|
|
574
|
-
- `runs list/get/watch/cancel/compare/trace` — inspect and manage executions.
|
|
575
|
-
- `runs artifacts list/fetch` — inventory and download run artifacts by path.
|
|
576
|
-
- `agents dataset` — manage persisted agent examples; runtime data, not Git source.
|
|
577
|
-
- `agents file` — live read/diff inspection only.
|
|
242
|
+
- [`reference/workflow-yaml.md`](reference/workflow-yaml.md)
|
|
243
|
+
- [`reference/step-types.md`](reference/step-types.md)
|
|
244
|
+
- [`reference/dataset-format.md`](reference/dataset-format.md)
|
|
245
|
+
- [`reference/evaluators.md`](reference/evaluators.md)
|
|
246
|
+
- [`reference/debugging.md`](reference/debugging.md)
|
|
247
|
+
|
|
248
|
+
Use `eigenpal --help` for command groups and `eigenpal <command> --help` for live
|
|
249
|
+
command help.
|