@eigenpal/cli 0.4.16 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +1 -0
- package/dist/cli.js +819 -45
- package/dist/skill/SKILL.md +273 -436
- package/dist/skill/reference/cli/agent.md +536 -98
- package/dist/skill/reference/cli/auth.md +5 -5
- package/dist/skill/reference/cli/skill.md +5 -5
- package/dist/skill/reference/cli/workflow.md +97 -76
- package/dist/skill/reference/dataset-format.md +4 -0
- package/dist/skill/reference/debugging.md +14 -0
- package/dist/skill/reference/evaluators.md +4 -0
- package/dist/skill/reference/workflow-yaml.md +8 -0
- package/package.json +1 -1
|
@@ -10,7 +10,7 @@ Manage authentication. Credentials live in ~/.config/eigenpal/credentials.json a
|
|
|
10
10
|
- [Details](#details)
|
|
11
11
|
- [`eigenpal auth login [options]`](#eigenpal-auth-login-options)
|
|
12
12
|
- [`eigenpal auth logout [options] [profile]`](#eigenpal-auth-logout-options-profile)
|
|
13
|
-
- [`eigenpal auth list [options]`](#eigenpal-auth-
|
|
13
|
+
- [`eigenpal auth list|ls [options]`](#eigenpal-auth-listls-options)
|
|
14
14
|
- [`eigenpal auth use [options] [profile]`](#eigenpal-auth-use-options-profile)
|
|
15
15
|
|
|
16
16
|
## Surface
|
|
@@ -19,7 +19,7 @@ Manage authentication. Credentials live in ~/.config/eigenpal/credentials.json a
|
|
|
19
19
|
auth
|
|
20
20
|
├── login
|
|
21
21
|
├── logout [profile]
|
|
22
|
-
├── list
|
|
22
|
+
├── list|ls
|
|
23
23
|
└── use [profile]
|
|
24
24
|
```
|
|
25
25
|
|
|
@@ -28,10 +28,10 @@ auth
|
|
|
28
28
|
### Core
|
|
29
29
|
|
|
30
30
|
| Command | Description |
|
|
31
|
-
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
31
|
+
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
|
|
32
32
|
| `eigenpal auth login [options]` | Authenticate with Eigenpal. Opens a browser to create an API key, stores it under a profile named after the tenant, and makes that profile active. Re-running `login` against the same tenant updates the existing profile in place. |
|
|
33
33
|
| `eigenpal auth logout [options] [profile]` | Remove a profile from the credentials file. Defaults to the active profile if no name is given. After removal the next available profile becomes active. |
|
|
34
|
-
| `eigenpal auth list [options]`
|
|
34
|
+
| `eigenpal auth list | ls [options]` | List configured profiles. The active one is marked ● — switch with `auth use`. |
|
|
35
35
|
| `eigenpal auth use [options] [profile]` | Switch the active profile (persistent across shells). Omit `[profile]` to pick from a list. For one-shot per-shell switching, set `EIGENPAL_PROFILE=<name>` instead. |
|
|
36
36
|
|
|
37
37
|
## Details
|
|
@@ -56,7 +56,7 @@ Remove a profile from the credentials file. Defaults to the active profile if no
|
|
|
56
56
|
| --------- | -------- | -------- | ----------- |
|
|
57
57
|
| `profile` | no | no | |
|
|
58
58
|
|
|
59
|
-
### `eigenpal auth list [options]`
|
|
59
|
+
### `eigenpal auth list|ls [options]`
|
|
60
60
|
|
|
61
61
|
List configured profiles. The active one is marked ● — switch with `auth use`.
|
|
62
62
|
|
|
@@ -10,7 +10,7 @@ Install / uninstall / list the Eigenpal agent skill in your project (Claude Code
|
|
|
10
10
|
- [Details](#details)
|
|
11
11
|
- [`eigenpal skill install [options]`](#eigenpal-skill-install-options)
|
|
12
12
|
- [`eigenpal skill uninstall [options] [toolIds...]`](#eigenpal-skill-uninstall-options-toolids)
|
|
13
|
-
- [`eigenpal skill list [options]`](#eigenpal-skill-
|
|
13
|
+
- [`eigenpal skill list|ls [options]`](#eigenpal-skill-listls-options)
|
|
14
14
|
|
|
15
15
|
## Surface
|
|
16
16
|
|
|
@@ -18,7 +18,7 @@ Install / uninstall / list the Eigenpal agent skill in your project (Claude Code
|
|
|
18
18
|
skill
|
|
19
19
|
├── install
|
|
20
20
|
├── uninstall [toolIds...]
|
|
21
|
-
└── list
|
|
21
|
+
└── list|ls
|
|
22
22
|
```
|
|
23
23
|
|
|
24
24
|
## Commands
|
|
@@ -26,10 +26,10 @@ skill
|
|
|
26
26
|
### Core
|
|
27
27
|
|
|
28
28
|
| Command | Description |
|
|
29
|
-
| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
29
|
+
| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
|
30
30
|
| `eigenpal skill install [options]` | Install the skill into one or more tools. Opens an interactive multiselect picker (toggle on to install, toggle off to uninstall) — pass `--tools` for non-interactive use. |
|
|
31
31
|
| `eigenpal skill uninstall [options] [toolIds...]` | Remove the skill from named tools (e.g. `skill uninstall claude cursor`), pass `--all` to wipe every install, or run with no args in a TTY for an interactive picker showing only installed tools. |
|
|
32
|
-
| `eigenpal skill list [options]`
|
|
32
|
+
| `eigenpal skill list | ls [options]` | Show every tool that has an Eigenpal skill installed in the current directory, with the CLI version that wrote it. |
|
|
33
33
|
|
|
34
34
|
## Details
|
|
35
35
|
|
|
@@ -64,7 +64,7 @@ Remove the skill from named tools (e.g. `skill uninstall claude cursor`), pass `
|
|
|
64
64
|
| `--target <path>` | no | | Power-user override: uninstall a single custom directory (skips tool detection) |
|
|
65
65
|
| `--yes` | no | | Required for non-TTY shells; skips confirmation in TTY |
|
|
66
66
|
|
|
67
|
-
### `eigenpal skill list [options]`
|
|
67
|
+
### `eigenpal skill list|ls [options]`
|
|
68
68
|
|
|
69
69
|
Show every tool that has an Eigenpal skill installed in the current directory, with the CLI version that wrote it.
|
|
70
70
|
|
|
@@ -16,15 +16,16 @@ Manage workflows: push, pull, run, evaluate.
|
|
|
16
16
|
- [Evaluator-type](#evaluator-type)
|
|
17
17
|
- [Step](#step)
|
|
18
18
|
- [Details](#details)
|
|
19
|
-
- [`eigenpal workflow list [options]`](#eigenpal-workflow-
|
|
19
|
+
- [`eigenpal workflow list|ls [options]`](#eigenpal-workflow-listls-options)
|
|
20
20
|
- [`eigenpal workflow pull [options] <workflow-id>`](#eigenpal-workflow-pull-options-workflow-id)
|
|
21
21
|
- [`eigenpal workflow push [options]`](#eigenpal-workflow-push-options)
|
|
22
|
+
- [`eigenpal workflow move [options] <workflow-id>`](#eigenpal-workflow-move-options-workflow-id)
|
|
22
23
|
- [`eigenpal workflow validate [options]`](#eigenpal-workflow-validate-options)
|
|
23
24
|
- [`eigenpal workflow clear-local [options] [examples...]`](#eigenpal-workflow-clear-local-options-examples)
|
|
24
25
|
- [`eigenpal workflow evaluators pull [options] <workflow-id>`](#eigenpal-workflow-evaluators-pull-options-workflow-id)
|
|
25
26
|
- [`eigenpal workflow evaluators push [options] <workflow-id>`](#eigenpal-workflow-evaluators-push-options-workflow-id)
|
|
26
27
|
- [`eigenpal workflow evaluators validate [options] [path]`](#eigenpal-workflow-evaluators-validate-options-path)
|
|
27
|
-
- [`eigenpal workflow dataset list [options] <workflow-id>`](#eigenpal-workflow-dataset-
|
|
28
|
+
- [`eigenpal workflow dataset list|ls [options] <workflow-id>`](#eigenpal-workflow-dataset-listls-options-workflow-id)
|
|
28
29
|
- [`eigenpal workflow dataset pull [options] <workflow-id>`](#eigenpal-workflow-dataset-pull-options-workflow-id)
|
|
29
30
|
- [`eigenpal workflow dataset push [options] <workflow-id>`](#eigenpal-workflow-dataset-push-options-workflow-id)
|
|
30
31
|
- [`eigenpal workflow dataset example create [options] <workflow-id>`](#eigenpal-workflow-dataset-example-create-options-workflow-id)
|
|
@@ -32,24 +33,24 @@ Manage workflows: push, pull, run, evaluate.
|
|
|
32
33
|
- [`eigenpal workflow dataset example delete [options] <workflow-id> <exampleId>`](#eigenpal-workflow-dataset-example-delete-options-workflow-id-exampleid)
|
|
33
34
|
- [`eigenpal workflow dataset example get [options] <workflow-id> <exampleId>`](#eigenpal-workflow-dataset-example-get-options-workflow-id-exampleid)
|
|
34
35
|
- [`eigenpal workflow dataset validate [options] [path]`](#eigenpal-workflow-dataset-validate-options-path)
|
|
35
|
-
- [`eigenpal workflow experiment list [options] <workflow-id>`](#eigenpal-workflow-
|
|
36
|
-
- [`eigenpal workflow experiment run [options] <workflow-id>`](#eigenpal-workflow-
|
|
37
|
-
- [`eigenpal workflow experiment status [options] <workflow-id> <batchId>`](#eigenpal-workflow-
|
|
38
|
-
- [`eigenpal workflow experiment cancel [options] <workflow-id> <batchId>`](#eigenpal-workflow-
|
|
39
|
-
- [`eigenpal workflow experiment results [options] <workflow-id> [batchId]`](#eigenpal-workflow-
|
|
40
|
-
- [`eigenpal workflow experiment compare [options] <batchIdA> <batchIdB>`](#eigenpal-workflow-
|
|
41
|
-
- [`eigenpal workflow experiment watch [options] <workflow-id> <batchId>`](#eigenpal-workflow-
|
|
42
|
-
- [`eigenpal workflow execution run [options] <workflow-id> [examples...]`](#eigenpal-workflow-
|
|
43
|
-
- [`eigenpal workflow execution get [options] <executionId>`](#eigenpal-workflow-
|
|
44
|
-
- [`eigenpal workflow execution list [options] <workflow-id>`](#eigenpal-workflow-
|
|
45
|
-
- [`eigenpal workflow execution watch [options] <executionId>`](#eigenpal-workflow-
|
|
46
|
-
- [`eigenpal workflow execution compare [options] <executionA> <executionB>`](#eigenpal-workflow-
|
|
47
|
-
- [`eigenpal workflow execution cancel [options] <executionId>`](#eigenpal-workflow-
|
|
48
|
-
- [`eigenpal workflow versions list [options] <workflow-id>`](#eigenpal-workflow-versions-
|
|
36
|
+
- [`eigenpal workflow experiment|exp list|ls [options] <workflow-id>`](#eigenpal-workflow-experimentexp-listls-options-workflow-id)
|
|
37
|
+
- [`eigenpal workflow experiment|exp run [options] <workflow-id>`](#eigenpal-workflow-experimentexp-run-options-workflow-id)
|
|
38
|
+
- [`eigenpal workflow experiment|exp status [options] <workflow-id> <batchId>`](#eigenpal-workflow-experimentexp-status-options-workflow-id-batchid)
|
|
39
|
+
- [`eigenpal workflow experiment|exp cancel [options] <workflow-id> <batchId>`](#eigenpal-workflow-experimentexp-cancel-options-workflow-id-batchid)
|
|
40
|
+
- [`eigenpal workflow experiment|exp results [options] <workflow-id> [batchId]`](#eigenpal-workflow-experimentexp-results-options-workflow-id-batchid)
|
|
41
|
+
- [`eigenpal workflow experiment|exp compare|diff [options] <batchIdA> <batchIdB>`](#eigenpal-workflow-experimentexp-comparediff-options-batchida-batchidb)
|
|
42
|
+
- [`eigenpal workflow experiment|exp watch [options] <workflow-id> <batchId>`](#eigenpal-workflow-experimentexp-watch-options-workflow-id-batchid)
|
|
43
|
+
- [`eigenpal workflow execution|exec run [options] <workflow-id> [examples...]`](#eigenpal-workflow-executionexec-run-options-workflow-id-examples)
|
|
44
|
+
- [`eigenpal workflow execution|exec get [options] <executionId>`](#eigenpal-workflow-executionexec-get-options-executionid)
|
|
45
|
+
- [`eigenpal workflow execution|exec list|ls [options] <workflow-id>`](#eigenpal-workflow-executionexec-listls-options-workflow-id)
|
|
46
|
+
- [`eigenpal workflow execution|exec watch [options] <executionId>`](#eigenpal-workflow-executionexec-watch-options-executionid)
|
|
47
|
+
- [`eigenpal workflow execution|exec compare|diff [options] <executionA> <executionB>`](#eigenpal-workflow-executionexec-comparediff-options-executiona-executionb)
|
|
48
|
+
- [`eigenpal workflow execution|exec cancel [options] <executionId>`](#eigenpal-workflow-executionexec-cancel-options-executionid)
|
|
49
|
+
- [`eigenpal workflow versions list|ls [options] <workflow-id>`](#eigenpal-workflow-versions-listls-options-workflow-id)
|
|
49
50
|
- [`eigenpal workflow versions restore [options] <workflow-id> <versionId>`](#eigenpal-workflow-versions-restore-options-workflow-id-versionid)
|
|
50
|
-
- [`eigenpal workflow step-type list [options]`](#eigenpal-workflow-step-type-
|
|
51
|
+
- [`eigenpal workflow step-type list|ls [options]`](#eigenpal-workflow-step-type-listls-options)
|
|
51
52
|
- [`eigenpal workflow step-type get [options] <type>`](#eigenpal-workflow-step-type-get-options-type)
|
|
52
|
-
- [`eigenpal workflow evaluator-type list [options]`](#eigenpal-workflow-evaluator-type-
|
|
53
|
+
- [`eigenpal workflow evaluator-type list|ls [options]`](#eigenpal-workflow-evaluator-type-listls-options)
|
|
53
54
|
- [`eigenpal workflow evaluator-type get [options] <type>`](#eigenpal-workflow-evaluator-type-get-options-type)
|
|
54
55
|
- [`eigenpal workflow step exec [options] <type>`](#eigenpal-workflow-step-exec-options-type)
|
|
55
56
|
|
|
@@ -57,15 +58,16 @@ Manage workflows: push, pull, run, evaluate.
|
|
|
57
58
|
|
|
58
59
|
```
|
|
59
60
|
workflow
|
|
60
|
-
├── list
|
|
61
|
+
├── list|ls
|
|
61
62
|
├── pull <workflow-id>
|
|
62
63
|
├── push
|
|
64
|
+
├── move <workflow-id>
|
|
63
65
|
├── evaluators
|
|
64
66
|
│ ├── pull <workflow-id>
|
|
65
67
|
│ ├── push <workflow-id>
|
|
66
68
|
│ └── validate [path]
|
|
67
69
|
├── dataset
|
|
68
|
-
│ ├── list <workflow-id>
|
|
70
|
+
│ ├── list|ls <workflow-id>
|
|
69
71
|
│ ├── pull <workflow-id>
|
|
70
72
|
│ ├── push <workflow-id>
|
|
71
73
|
│ ├── example
|
|
@@ -74,29 +76,29 @@ workflow
|
|
|
74
76
|
│ │ ├── delete <workflow-id> <exampleId>
|
|
75
77
|
│ │ └── get <workflow-id> <exampleId>
|
|
76
78
|
│ └── validate [path]
|
|
77
|
-
├── experiment
|
|
78
|
-
│ ├── list <workflow-id>
|
|
79
|
+
├── experiment|exp
|
|
80
|
+
│ ├── list|ls <workflow-id>
|
|
79
81
|
│ ├── run <workflow-id>
|
|
80
82
|
│ ├── status <workflow-id> <batchId>
|
|
81
83
|
│ ├── cancel <workflow-id> <batchId>
|
|
82
84
|
│ ├── results <workflow-id> [batchId]
|
|
83
|
-
│ ├── compare <batchIdA> <batchIdB>
|
|
85
|
+
│ ├── compare|diff <batchIdA> <batchIdB>
|
|
84
86
|
│ └── watch <workflow-id> <batchId>
|
|
85
|
-
├── execution
|
|
87
|
+
├── execution|exec
|
|
86
88
|
│ ├── run <workflow-id> [examples...]
|
|
87
89
|
│ ├── get <executionId>
|
|
88
|
-
│ ├── list <workflow-id>
|
|
90
|
+
│ ├── list|ls <workflow-id>
|
|
89
91
|
│ ├── watch <executionId>
|
|
90
|
-
│ ├── compare <executionA> <executionB>
|
|
92
|
+
│ ├── compare|diff <executionA> <executionB>
|
|
91
93
|
│ └── cancel <executionId>
|
|
92
94
|
├── versions
|
|
93
|
-
│ ├── list <workflow-id>
|
|
95
|
+
│ ├── list|ls <workflow-id>
|
|
94
96
|
│ └── restore <workflow-id> <versionId>
|
|
95
97
|
├── step-type
|
|
96
|
-
│ ├── list
|
|
98
|
+
│ ├── list|ls
|
|
97
99
|
│ └── get <type>
|
|
98
100
|
├── evaluator-type
|
|
99
|
-
│ ├── list
|
|
101
|
+
│ ├── list|ls
|
|
100
102
|
│ └── get <type>
|
|
101
103
|
├── validate
|
|
102
104
|
├── clear-local [examples...]
|
|
@@ -109,10 +111,11 @@ workflow
|
|
|
109
111
|
### Core
|
|
110
112
|
|
|
111
113
|
| Command | Description |
|
|
112
|
-
| ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
113
|
-
| `eigenpal workflow list [options]`
|
|
114
|
+
| ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
|
|
115
|
+
| `eigenpal workflow list | ls [options]` | List workflows the caller can read. |
|
|
114
116
|
| `eigenpal workflow pull [options] <workflow-id>` | Download the YAML definition of the workflow at its current version. |
|
|
115
117
|
| `eigenpal workflow push [options]` | Create or update a workflow from a YAML file. |
|
|
118
|
+
| `eigenpal workflow move [options] <workflow-id>` | Move a workflow to a folder path, creating folders as needed |
|
|
116
119
|
| `eigenpal workflow validate [options]` | Local-only validation against the templated project layout: ./workflow.yaml + ./evaluators.yaml + ./dataset/. For targeted validation use `evaluators validate` or `dataset validate`. |
|
|
117
120
|
| `eigenpal workflow clear-local [options] [examples...]` | Delete local execution artifacts under ./dataset/examples/. Keeps the latest run per example by default. |
|
|
118
121
|
|
|
@@ -127,8 +130,8 @@ workflow
|
|
|
127
130
|
### Dataset
|
|
128
131
|
|
|
129
132
|
| Command | Description |
|
|
130
|
-
| ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- |
|
|
131
|
-
| `eigenpal workflow dataset list [options] <workflow-id>`
|
|
133
|
+
| ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- | ------------------------------------ |
|
|
134
|
+
| `eigenpal workflow dataset list | ls [options] <workflow-id>` | List eval examples for the workflow. |
|
|
132
135
|
| `eigenpal workflow dataset pull [options] <workflow-id>` | Download the workflow's dataset as a ZIP archive. |
|
|
133
136
|
| `eigenpal workflow dataset push [options] <workflow-id>` | Replace or extend the workflow's dataset from a ZIP or folder. |
|
|
134
137
|
| `eigenpal workflow dataset example create [options] <workflow-id>` | Create one eval example without re-uploading the dataset. |
|
|
@@ -139,46 +142,46 @@ workflow
|
|
|
139
142
|
|
|
140
143
|
### Experiment
|
|
141
144
|
|
|
142
|
-
| Command
|
|
143
|
-
|
|
|
144
|
-
| `eigenpal workflow experiment list [options] <workflow-id>`
|
|
145
|
-
| `eigenpal workflow experiment run [options] <workflow-id>` | Start a batch eval against the workflow's dataset. |
|
|
146
|
-
| `eigenpal workflow experiment status [options] <workflow-id> <batchId>` | Aggregate progress for a batch by `batchId`. |
|
|
147
|
-
| `eigenpal workflow experiment cancel [options] <workflow-id> <batchId>` | Cancel every execution in a batch. Idempotent. |
|
|
148
|
-
| `eigenpal workflow experiment results [options] <workflow-id> [batchId]` | Download eval results in CSV or JSON. |
|
|
149
|
-
| `eigenpal workflow experiment compare [options] <batchIdA> <batchIdB>`
|
|
150
|
-
| `eigenpal workflow experiment watch [options] <workflow-id> <batchId>` | Poll until terminal, then auto-pull results — replaces `status --watch` + `results --out`. |
|
|
145
|
+
| Command | Description |
|
|
146
|
+
| ----------------------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------ |
|
|
147
|
+
| `eigenpal workflow experiment | exp list | ls [options] <workflow-id>` | List executions for the workflow, newest first. |
|
|
148
|
+
| `eigenpal workflow experiment | exp run [options] <workflow-id>` | Start a batch eval against the workflow's dataset. |
|
|
149
|
+
| `eigenpal workflow experiment | exp status [options] <workflow-id> <batchId>` | Aggregate progress for a batch by `batchId`. |
|
|
150
|
+
| `eigenpal workflow experiment | exp cancel [options] <workflow-id> <batchId>` | Cancel every execution in a batch. Idempotent. |
|
|
151
|
+
| `eigenpal workflow experiment | exp results [options] <workflow-id> [batchId]` | Download eval results in CSV or JSON. |
|
|
152
|
+
| `eigenpal workflow experiment | exp compare | diff [options] <batchIdA> <batchIdB>` | Diff eval scores between two experiment batches. |
|
|
153
|
+
| `eigenpal workflow experiment | exp watch [options] <workflow-id> <batchId>` | Poll until terminal, then auto-pull results — replaces `status --watch` + `results --out`. |
|
|
151
154
|
|
|
152
155
|
### Execution
|
|
153
156
|
|
|
154
|
-
| Command
|
|
155
|
-
|
|
|
156
|
-
| `eigenpal workflow execution run [options] <workflow-id> [examples...]`
|
|
157
|
-
| `eigenpal workflow execution get [options] <executionId>`
|
|
158
|
-
| `eigenpal workflow execution list [options] <workflow-id>`
|
|
159
|
-
| `eigenpal workflow execution watch [options] <executionId>`
|
|
160
|
-
| `eigenpal workflow execution compare [options] <executionA> <executionB>`
|
|
161
|
-
| `eigenpal workflow execution cancel [options] <executionId>`
|
|
157
|
+
| Command | Description |
|
|
158
|
+
| ---------------------------- | ----------------------------------------------- | ---------------------------------------------------------------- | ------------------------------------------- |
|
|
159
|
+
| `eigenpal workflow execution | exec run [options] <workflow-id> [examples...]` | Run a saved workflow against local dataset examples. |
|
|
160
|
+
| `eigenpal workflow execution | exec get [options] <executionId>` | Fetch a single execution payload. Optionally narrow to one step. |
|
|
161
|
+
| `eigenpal workflow execution | exec list | ls [options] <workflow-id>` | List recent executions for a workflow. |
|
|
162
|
+
| `eigenpal workflow execution | exec watch [options] <executionId>` | Stream live execution status until terminal or 30-min detach. |
|
|
163
|
+
| `eigenpal workflow execution | exec compare | diff [options] <executionA> <executionB>` | Diff two executions side-by-side, per step. |
|
|
164
|
+
| `eigenpal workflow execution | exec cancel [options] <executionId>` | Cancel an execution. Idempotent on already-terminal runs. |
|
|
162
165
|
|
|
163
166
|
### Versions
|
|
164
167
|
|
|
165
|
-
| Command | Description
|
|
166
|
-
| ------------------------------------------------------------------------ | ------------------------------------------------ |
|
|
167
|
-
| `eigenpal workflow versions list [options] <workflow-id>`
|
|
168
|
-
| `eigenpal workflow versions restore [options] <workflow-id> <versionId>` | Restore the workflow to a previous version.
|
|
168
|
+
| Command | Description |
|
|
169
|
+
| ------------------------------------------------------------------------ | ------------------------------------------- | ------------------------------------------------ |
|
|
170
|
+
| `eigenpal workflow versions list | ls [options] <workflow-id>` | List historical workflow versions, newest first. |
|
|
171
|
+
| `eigenpal workflow versions restore [options] <workflow-id> <versionId>` | Restore the workflow to a previous version. |
|
|
169
172
|
|
|
170
173
|
### Step-type
|
|
171
174
|
|
|
172
175
|
| Command | Description |
|
|
173
|
-
| -------------------------------------------------- | ------------------------------------------------------------- |
|
|
174
|
-
| `eigenpal workflow step-type list [options]`
|
|
176
|
+
| -------------------------------------------------- | ------------------------------------------------------------- | --------------------------------------------- |
|
|
177
|
+
| `eigenpal workflow step-type list | ls [options]` | List every step type the deployment supports. |
|
|
175
178
|
| `eigenpal workflow step-type get [options] <type>` | Return the full schema and behavioral docs for one step type. |
|
|
176
179
|
|
|
177
180
|
### Evaluator-type
|
|
178
181
|
|
|
179
182
|
| Command | Description |
|
|
180
|
-
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
|
|
181
|
-
| `eigenpal workflow evaluator-type list [options]`
|
|
183
|
+
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
|
|
184
|
+
| `eigenpal workflow evaluator-type list | ls [options]` | List every evaluator type with a one-line description. |
|
|
182
185
|
| `eigenpal workflow evaluator-type get [options] <type>` | Fetch the JSON Schema for one evaluator type. Pipe through `jq` to inspect specific fields. |
|
|
183
186
|
|
|
184
187
|
### Step
|
|
@@ -189,7 +192,7 @@ workflow
|
|
|
189
192
|
|
|
190
193
|
## Details
|
|
191
194
|
|
|
192
|
-
### `eigenpal workflow list [options]`
|
|
195
|
+
### `eigenpal workflow list|ls [options]`
|
|
193
196
|
|
|
194
197
|
List workflows the caller can read.
|
|
195
198
|
|
|
@@ -235,6 +238,24 @@ Create or update a workflow from a YAML file.
|
|
|
235
238
|
| `--base-url <url>` | no | | Server base URL |
|
|
236
239
|
| `--json` | no | | Output the raw server response as JSON |
|
|
237
240
|
|
|
241
|
+
### `eigenpal workflow move [options] <workflow-id>`
|
|
242
|
+
|
|
243
|
+
Move a workflow to a folder path, creating folders as needed
|
|
244
|
+
|
|
245
|
+
### Arguments
|
|
246
|
+
|
|
247
|
+
| Name | Required | Variadic | Description |
|
|
248
|
+
| ------------- | -------- | -------- | ----------- |
|
|
249
|
+
| `workflow-id` | yes | no | |
|
|
250
|
+
|
|
251
|
+
### Options
|
|
252
|
+
|
|
253
|
+
| Flag | Required | Default | Description |
|
|
254
|
+
| ------------------ | -------- | ------- | -------------------------------------- |
|
|
255
|
+
| `--folder <path>` | yes | | Target folder path (`/` for root) |
|
|
256
|
+
| `--base-url <url>` | no | | Server base URL |
|
|
257
|
+
| `--json` | no | | Output the raw server response as JSON |
|
|
258
|
+
|
|
238
259
|
### `eigenpal workflow validate [options]`
|
|
239
260
|
|
|
240
261
|
Local-only validation against the templated project layout: ./workflow.yaml + ./evaluators.yaml + ./dataset/. For targeted validation use `evaluators validate` or `dataset validate`.
|
|
@@ -307,7 +328,7 @@ Validate an evaluators YAML file against the EvalConfig schema. Defaults to ./ev
|
|
|
307
328
|
| ------ | -------- | -------- | ----------- |
|
|
308
329
|
| `path` | no | no | |
|
|
309
330
|
|
|
310
|
-
### `eigenpal workflow dataset list [options] <workflow-id>`
|
|
331
|
+
### `eigenpal workflow dataset list|ls [options] <workflow-id>`
|
|
311
332
|
|
|
312
333
|
List eval examples for the workflow.
|
|
313
334
|
|
|
@@ -458,7 +479,7 @@ Validate a dataset folder against the examples/<name>/{input,expected,meta} conv
|
|
|
458
479
|
| ------ | -------- | -------- | ----------- |
|
|
459
480
|
| `path` | no | no | |
|
|
460
481
|
|
|
461
|
-
### `eigenpal workflow experiment list [options] <workflow-id>`
|
|
482
|
+
### `eigenpal workflow experiment|exp list|ls [options] <workflow-id>`
|
|
462
483
|
|
|
463
484
|
List executions for the workflow, newest first.
|
|
464
485
|
|
|
@@ -478,7 +499,7 @@ List executions for the workflow, newest first.
|
|
|
478
499
|
| `--base-url <url>` | no | | Server base URL |
|
|
479
500
|
| `--json` | no | | Output the raw server response as JSON |
|
|
480
501
|
|
|
481
|
-
### `eigenpal workflow experiment run [options] <workflow-id>`
|
|
502
|
+
### `eigenpal workflow experiment|exp run [options] <workflow-id>`
|
|
482
503
|
|
|
483
504
|
Start a batch eval against the workflow's dataset.
|
|
484
505
|
|
|
@@ -498,7 +519,7 @@ Start a batch eval against the workflow's dataset.
|
|
|
498
519
|
| `--base-url <url>` | no | | Server base URL |
|
|
499
520
|
| `--json` | no | | Output the raw server response as JSON |
|
|
500
521
|
|
|
501
|
-
### `eigenpal workflow experiment status [options] <workflow-id> <batchId>`
|
|
522
|
+
### `eigenpal workflow experiment|exp status [options] <workflow-id> <batchId>`
|
|
502
523
|
|
|
503
524
|
Aggregate progress for a batch by `batchId`.
|
|
504
525
|
|
|
@@ -521,7 +542,7 @@ Aggregate progress for a batch by `batchId`.
|
|
|
521
542
|
| `--base-url <url>` | no | | Server base URL |
|
|
522
543
|
| `--json` | no | | Output the raw server response as JSON |
|
|
523
544
|
|
|
524
|
-
### `eigenpal workflow experiment cancel [options] <workflow-id> <batchId>`
|
|
545
|
+
### `eigenpal workflow experiment|exp cancel [options] <workflow-id> <batchId>`
|
|
525
546
|
|
|
526
547
|
Cancel every execution in a batch. Idempotent.
|
|
527
548
|
|
|
@@ -540,7 +561,7 @@ Cancel every execution in a batch. Idempotent.
|
|
|
540
561
|
| `--base-url <url>` | no | | Server base URL |
|
|
541
562
|
| `--json` | no | | Output the raw server response as JSON |
|
|
542
563
|
|
|
543
|
-
### `eigenpal workflow experiment results [options] <workflow-id> [batchId]`
|
|
564
|
+
### `eigenpal workflow experiment|exp results [options] <workflow-id> [batchId]`
|
|
544
565
|
|
|
545
566
|
Download eval results in CSV or JSON.
|
|
546
567
|
|
|
@@ -559,7 +580,7 @@ Download eval results in CSV or JSON.
|
|
|
559
580
|
| `--out <path>` | no | | Output file. When omitted, the binary streams to stdout. |
|
|
560
581
|
| `--base-url <url>` | no | | Server base URL |
|
|
561
582
|
|
|
562
|
-
### `eigenpal workflow experiment compare [options] <batchIdA> <batchIdB>`
|
|
583
|
+
### `eigenpal workflow experiment|exp compare|diff [options] <batchIdA> <batchIdB>`
|
|
563
584
|
|
|
564
585
|
Diff eval scores between two experiment batches.
|
|
565
586
|
|
|
@@ -579,7 +600,7 @@ Diff eval scores between two experiment batches.
|
|
|
579
600
|
| `--base-url <url>` | no | | Server base URL |
|
|
580
601
|
| `--json` | no | | Output the raw server response as JSON |
|
|
581
602
|
|
|
582
|
-
### `eigenpal workflow experiment watch [options] <workflow-id> <batchId>`
|
|
603
|
+
### `eigenpal workflow experiment|exp watch [options] <workflow-id> <batchId>`
|
|
583
604
|
|
|
584
605
|
Poll until terminal, then auto-pull results — replaces `status --watch` + `results --out`.
|
|
585
606
|
|
|
@@ -601,7 +622,7 @@ Poll until terminal, then auto-pull results — replaces `status --watch` + `res
|
|
|
601
622
|
| `--no-pull` | no | | Skip auto-pulling results on terminal (just watch) |
|
|
602
623
|
| `--base-url <url>` | no | | Server base URL |
|
|
603
624
|
|
|
604
|
-
### `eigenpal workflow execution run [options] <workflow-id> [examples...]`
|
|
625
|
+
### `eigenpal workflow execution|exec run [options] <workflow-id> [examples...]`
|
|
605
626
|
|
|
606
627
|
Run a saved workflow against local dataset examples.
|
|
607
628
|
|
|
@@ -621,7 +642,7 @@ Run a saved workflow against local dataset examples.
|
|
|
621
642
|
| `--base-url <url>` | no | | Server base URL |
|
|
622
643
|
| `--json` | no | | Output the raw server response as JSON |
|
|
623
644
|
|
|
624
|
-
### `eigenpal workflow execution get [options] <executionId>`
|
|
645
|
+
### `eigenpal workflow execution|exec get [options] <executionId>`
|
|
625
646
|
|
|
626
647
|
Fetch a single execution payload. Optionally narrow to one step.
|
|
627
648
|
|
|
@@ -640,7 +661,7 @@ Fetch a single execution payload. Optionally narrow to one step.
|
|
|
640
661
|
| `--base-url <url>` | no | | Server base URL |
|
|
641
662
|
| `--json` | no | | Output the raw server response as JSON |
|
|
642
663
|
|
|
643
|
-
### `eigenpal workflow execution list [options] <workflow-id>`
|
|
664
|
+
### `eigenpal workflow execution|exec list|ls [options] <workflow-id>`
|
|
644
665
|
|
|
645
666
|
List recent executions for a workflow.
|
|
646
667
|
|
|
@@ -660,7 +681,7 @@ List recent executions for a workflow.
|
|
|
660
681
|
| `--base-url <url>` | no | | Server base URL |
|
|
661
682
|
| `--json` | no | | Output the raw server response as JSON |
|
|
662
683
|
|
|
663
|
-
### `eigenpal workflow execution watch [options] <executionId>`
|
|
684
|
+
### `eigenpal workflow execution|exec watch [options] <executionId>`
|
|
664
685
|
|
|
665
686
|
Stream live execution status until terminal or 30-min detach.
|
|
666
687
|
|
|
@@ -677,7 +698,7 @@ Stream live execution status until terminal or 30-min detach.
|
|
|
677
698
|
| `--max-wait <seconds>` | no | `1800` | Detach after N seconds (default 1800 = 30 min) |
|
|
678
699
|
| `--base-url <url>` | no | | Server base URL |
|
|
679
700
|
|
|
680
|
-
### `eigenpal workflow execution compare [options] <executionA> <executionB>`
|
|
701
|
+
### `eigenpal workflow execution|exec compare|diff [options] <executionA> <executionB>`
|
|
681
702
|
|
|
682
703
|
Diff two executions side-by-side, per step.
|
|
683
704
|
|
|
@@ -695,7 +716,7 @@ Diff two executions side-by-side, per step.
|
|
|
695
716
|
| `--step <name>` | no | | Restrict comparison to one step |
|
|
696
717
|
| `--base-url <url>` | no | | Server base URL |
|
|
697
718
|
|
|
698
|
-
### `eigenpal workflow execution cancel [options] <executionId>`
|
|
719
|
+
### `eigenpal workflow execution|exec cancel [options] <executionId>`
|
|
699
720
|
|
|
700
721
|
Cancel an execution. Idempotent on already-terminal runs.
|
|
701
722
|
|
|
@@ -713,7 +734,7 @@ Cancel an execution. Idempotent on already-terminal runs.
|
|
|
713
734
|
| `--base-url <url>` | no | | Server base URL |
|
|
714
735
|
| `--json` | no | | Output the raw server response as JSON |
|
|
715
736
|
|
|
716
|
-
### `eigenpal workflow versions list [options] <workflow-id>`
|
|
737
|
+
### `eigenpal workflow versions list|ls [options] <workflow-id>`
|
|
717
738
|
|
|
718
739
|
List historical workflow versions, newest first.
|
|
719
740
|
|
|
@@ -750,7 +771,7 @@ Restore the workflow to a previous version.
|
|
|
750
771
|
| `--base-url <url>` | no | | Server base URL |
|
|
751
772
|
| `--json` | no | | Output the raw server response as JSON |
|
|
752
773
|
|
|
753
|
-
### `eigenpal workflow step-type list [options]`
|
|
774
|
+
### `eigenpal workflow step-type list|ls [options]`
|
|
754
775
|
|
|
755
776
|
List every step type the deployment supports.
|
|
756
777
|
|
|
@@ -780,7 +801,7 @@ Return the full schema and behavioral docs for one step type.
|
|
|
780
801
|
| -------------- | -------- | ------- | ------------- |
|
|
781
802
|
| `--out <path>` | no | | Write to file |
|
|
782
803
|
|
|
783
|
-
### `eigenpal workflow evaluator-type list [options]`
|
|
804
|
+
### `eigenpal workflow evaluator-type list|ls [options]`
|
|
784
805
|
|
|
785
806
|
List every evaluator type with a one-line description.
|
|
786
807
|
|
|
@@ -11,6 +11,10 @@ on the server, the canonical per-row surface is
|
|
|
11
11
|
`dataset list` for a table view) — the local folder is a pre-push
|
|
12
12
|
staging area, not a synced mirror.
|
|
13
13
|
|
|
14
|
+
Dataset folders are unrelated to workflow organization folders. To place
|
|
15
|
+
the workflow itself under dashboard folders, use `workflow move --folder`;
|
|
16
|
+
dataset push/pull never changes folder placement.
|
|
17
|
+
|
|
14
18
|
## Required layout
|
|
15
19
|
|
|
16
20
|
```
|
|
@@ -134,6 +134,20 @@ The error envelope tells you exactly where:
|
|
|
134
134
|
Follow the hint — every field path is dotted into the YAML so you can
|
|
135
135
|
jump straight to the broken line.
|
|
136
136
|
|
|
137
|
+
### Workflow appears in the wrong folder
|
|
138
|
+
|
|
139
|
+
Folder placement is controlled by server folder state, not by
|
|
140
|
+
`workflow.yaml`. Move by explicit folder path:
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
eigenpal workflow move <workflow-id> --folder billing/invoices
|
|
144
|
+
eigenpal workflow move <workflow-id> --folder /
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
`--folder /` moves the workflow back to root. `workflow push` and
|
|
148
|
+
`workflow pull` do not change folder placement. Missing folders in the
|
|
149
|
+
path are created automatically.
|
|
150
|
+
|
|
137
151
|
## 4. Compare two executions
|
|
138
152
|
|
|
139
153
|
```bash
|
|
@@ -6,6 +6,10 @@ attach to a workflow via `workflow evaluators push` and run in batch via
|
|
|
6
6
|
`EvalConfigYamlSchema` in `@eigenpal/types/src/eval/evaluator-config.ts`;
|
|
7
7
|
this is the agent-facing summary.
|
|
8
8
|
|
|
9
|
+
Evaluator config does not control dashboard folder placement. Workflow
|
|
10
|
+
organization is managed with `workflow move --folder`;
|
|
11
|
+
`workflow evaluators push` leaves the workflow's folder unchanged.
|
|
12
|
+
|
|
9
13
|
## Top-level shape
|
|
10
14
|
|
|
11
15
|
```yaml
|
|
@@ -26,6 +26,14 @@ output: {...} # object — the workflow's return shape
|
|
|
26
26
|
version with the same `name` and a different `version` appends to history;
|
|
27
27
|
re-using the same `version` is rejected.
|
|
28
28
|
|
|
29
|
+
Folder placement is not part of `workflow.yaml`; it lives in the server
|
|
30
|
+
database. Use `workflow move <workflow-id> --folder <path>` to move a
|
|
31
|
+
workflow between dashboard folders. Missing folders in the path are
|
|
32
|
+
created automatically. Use `--folder /` for root. New workflows should stay
|
|
33
|
+
at the top level unless the user specifically asks to place them in a
|
|
34
|
+
folder.
|
|
35
|
+
|
|
36
|
+
|
|
29
37
|
### `triggerMethods`
|
|
30
38
|
|
|
31
39
|
Required. Each entry adds an invocation surface:
|