@eigenpal/cli 0.7.2 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. package/CHANGELOG.md +101 -23
  2. package/README.md +8 -8
  3. package/dist/cli.js +9300 -9059
  4. package/dist/skill/SKILL.md +87 -415
  5. package/dist/skill/reference/cli/agents.md +26 -26
  6. package/dist/skill/reference/cli/rerun.md +1 -1
  7. package/dist/skill/reference/cli/run.md +1 -1
  8. package/dist/skill/reference/cli/runs.md +23 -2
  9. package/dist/skill/reference/cli/surface.md +1 -2
  10. package/dist/skill/reference/cli/workflow.md +10 -10
  11. package/dist/skill/reference/dataset-format.md +74 -241
  12. package/dist/skill/reference/debugging.md +5 -5
  13. package/dist/skill/reference/evaluators.md +10 -10
  14. package/dist/skill/reference/step-exec.md +2 -2
  15. package/dist/skill/reference/step-types.md +7 -7
  16. package/dist/skill/reference/workflow-yaml.md +4 -4
  17. package/dist/templates/blank/README.md +4 -3
  18. package/dist/templates/blank/evaluators.yaml +1 -1
  19. package/package.json +1 -1
  20. package/dist/skill/reference/cli/completion.md +0 -9
  21. package/dist/skill/reference/cli/git.md +0 -9
  22. /package/dist/templates/blank/dataset/examples/sample/{expected/output.json → expected.json} +0 -0
  23. /package/dist/templates/blank/dataset/examples/sample/{input/arguments.json → input.json} +0 -0
  24. /package/dist/templates/pdf-extraction/dataset/examples/sample-invoice/{expected/output.json → expected.json} +0 -0
  25. /package/dist/templates/pdf-extraction/dataset/examples/sample-invoice/{input/arguments.json → input.json} +0 -0
  26. /package/dist/templates/text-classification/dataset/examples/negative/{expected/output.json → expected.json} +0 -0
  27. /package/dist/templates/text-classification/dataset/examples/negative/{input/arguments.json → input.json} +0 -0
  28. /package/dist/templates/text-classification/dataset/examples/positive/{expected/output.json → expected.json} +0 -0
  29. /package/dist/templates/text-classification/dataset/examples/positive/{input/arguments.json → input.json} +0 -0
@@ -54,7 +54,7 @@ The grouped run payload includes:
54
54
  substitution. Lets you see what the LLM actually got, what URL
55
55
  HTTP actually hit, what items the script actually saw. Populated
56
56
  for `ai.*` / `transform.*` / `action.*`. Null for `control.*`
57
- (orchestration steps don't have a meaningful resolved config).
57
+ (orchestration steps do not have a meaningful resolved config).
58
58
  - `output` — what the step returned
59
59
  - `error` — error message + code when `status: 'failed'`
60
60
  - `durationMs`
@@ -108,7 +108,7 @@ no network or filesystem, so most timeouts are infinite-loop bugs.
108
108
 
109
109
  ### `step_type_unknown`
110
110
 
111
- `workflow.yaml` references a `type:` that the deployment doesn't know
111
+ `workflow.yaml` references a `type:` that the deployment does not know
112
112
  about. Common after a typo or a step type that was renamed.
113
113
 
114
114
  ```bash
@@ -219,7 +219,7 @@ broken. If you see:
219
219
  Set EIGENPAL_BASE_URL or pass --base-url <url>.
220
220
  ```
221
221
 
222
- That's a **connection** error — the URL is wrong, the server is down,
222
+ That is a **connection** error — the URL is wrong, the server is down,
223
223
  or your network is blocked. The URL the CLI tried is echoed back so a
224
224
  stale `EIGENPAL_BASE_URL` is immediately obvious.
225
225
 
@@ -230,13 +230,13 @@ If instead you see:
230
230
  Run `eigenpal auth login`, or set EIGENPAL_API_KEY in your env.
231
231
  ```
232
232
 
233
- That's an **auth** error — the URL works but your key is missing /
233
+ That is an **auth** error — the URL works but your key is missing /
234
234
  expired / scoped to the wrong tenant. Run `auth list` to see configured
235
235
  profiles, or `auth use <name>` to switch.
236
236
 
237
237
  ## 8. Override a flaky step in eval mode
238
238
 
239
- If a step calls an external system that's unreliable (an external API, a
239
+ If a step calls an external system that is unreliable (an external API, a
240
240
  private API, a rate-limited connector), short-circuit it for evals via
241
241
  the example's `meta.json`:
242
242
 
@@ -54,7 +54,7 @@ finance) who never see the YAML or the judge prompt. Write it for them:
54
54
 
55
55
  ## Evaluator types
56
56
 
57
- ### `exact-diff` — JSON deep-diff against `expected/output.json` or `expected/error.json`
57
+ ### `exact-diff` — JSON deep-diff against `expected.json`
58
58
 
59
59
  ```yaml
60
60
  - name: invoice-fields
@@ -71,8 +71,8 @@ finance) who never see the YAML or the judge prompt. Write it for them:
71
71
  is compared.
72
72
 
73
73
  `exact-diff` branches on the example's expected shape: success-expected
74
- examples (`expected/output.json`) are diffed against the workflow's actual
75
- output, and failure-expected examples (`expected/error.json`, see
74
+ examples (`expected.json`) are diffed against the workflow's actual output, and
75
+ failure-expected examples (`expected.json` with `$error`, see
76
76
  `reference/dataset-format.md`) are matched against the `control.fail`
77
77
  envelope persisted to `executions.error`. The same evaluator handles both
78
78
  paths; you do not need a separate evaluator for failure cases.
@@ -106,7 +106,7 @@ paths; you do not need a separate evaluator for failure cases.
106
106
  Pick the label that best matches the tone of the response.
107
107
  ```
108
108
 
109
- `promptExtension` is the only required field — it's appended to the harness
109
+ `promptExtension` is the only required field — it is appended to the harness
110
110
  prompt and tells the judge what "good" looks like for this workflow. The
111
111
  harness handles JSON parsing, retries, and feeds the judge `actual`,
112
112
  `expected`, and example metadata. In `discrete` mode the judge MUST return
@@ -140,7 +140,7 @@ verbatim, same text the dashboard editor shows. The `: number` return-type annot
140
140
  enforced at parse time. The dashboard seeds new evaluators with a
141
141
  `WorkflowOutput` type alias derived from the workflow's `output:`
142
142
  declaration, plus `Expected` / `Actual` aliases for compatibility. Plain
143
- `unknown` works too if you don't want a typed signature.
143
+ `unknown` works too if you do not want a typed signature.
144
144
 
145
145
  The signature shape is **locked**:
146
146
 
@@ -151,7 +151,7 @@ The signature shape is **locked**:
151
151
  and scored as 0).
152
152
 
153
153
  `workflow validate` and `workflow evaluators validate` flag YAML where
154
- the function fails to parse, the signature doesn't match, or the body
154
+ the function fails to parse, the signature does not match, or the body
155
155
  contains neither `return` nor `throw` — so a hand-edited config errors
156
156
  before it reaches the worker.
157
157
 
@@ -224,7 +224,7 @@ eigenpal workflow experiment results <workflow-id> <batch-id> \
224
224
  --format csv --out results.csv
225
225
  ```
226
226
 
227
- The signed-URL pattern means large result sets don't go through the CLI
227
+ The signed-URL pattern means large result sets do not go through the CLI
228
228
  process — `--out` writes the file directly from the server's storage.
229
229
 
230
230
  ## Stabilizing flaky judges
@@ -264,9 +264,9 @@ _Generated from `EvalConfigYamlSchema` in `@eigenpal/types/src/eval/evaluator-co
264
264
  | Field | Type | Required | Default | Description |
265
265
  | --- | --- | --- | --- | --- |
266
266
  | `numericTolerance` | number | no | `0.000001` | Maximum absolute difference between numeric values that still counts as equal. Use 1e-6 for floats, 0 for exact integer match. |
267
- | `allowExtraFields` | boolean | no | `true` | When on, extra fields in the actual output don't fail the diff. Off = actual must match expected exactly. |
267
+ | `allowExtraFields` | boolean | no | `true` | When on, extra fields in the actual output do not fail the diff. Off = actual must match expected exactly. |
268
268
  | `passThreshold` | number | no | `1` | Minimum diff score this evaluator must reach for a run to pass. 1.0 = perfect match required. |
269
- | `paths` | array<string> | no | | Dot-paths to scope the diff to a subset of the expected tree. Empty = diff entire expected. Syntax: `header.id`, `lineItems[].total`, `lineItems[0].sku`. Use this when expected has noisy sections you don't want to score. |
269
+ | `paths` | array<string> | no | | Dot-paths to scope the diff to a subset of the expected tree. Empty = diff entire expected. Syntax: `header.id`, `lineItems[].total`, `lineItems[0].sku`. Use this when expected has noisy sections you do not want to score. |
270
270
 
271
271
 
272
272
  ### `llm-judge` — LLM-as-judge scoring
@@ -286,7 +286,7 @@ _Generated from `EvalConfigYamlSchema` in `@eigenpal/types/src/eval/evaluator-co
286
286
  | Field | Type | Required | Default | Description |
287
287
  | --- | --- | --- | --- | --- |
288
288
  | `function` | string | yes | | Full TypeScript source for optional `type` aliases plus `function scoreScript(expected, actual): number { ... }`. Receives `expected` (the example's expected output) and `actual` (the workflow's actual output), returns a number in [0, 1] (the `: number` annotation is required). The `: number` return-type annotation is required and enforced at parse time. Throws are caught and scored as 0. |
289
- | `timeoutMs` | integer | no | `5000` | Maximum wall-clock time the script can run before it's killed and the run is marked failed. |
289
+ | `timeoutMs` | integer | no | `5000` | Maximum wall-clock time the script can run before it is killed and the run is marked failed. |
290
290
  | `memoryLimitMb` | integer | no | `10` | Maximum memory the sandbox may allocate. Increase if the script processes large arrays or strings. |
291
291
  | `passThreshold` | number | no | `1` | Minimum script score this evaluator must reach for a run to pass. |
292
292
  <!-- GENERATED:EVALUATOR_CATALOG END -->
@@ -7,7 +7,7 @@
7
7
  > retries, no tracing, no credit metering, no tenant-default LLM, no
8
8
  > strict-mode response_format) and assumed shell env state
9
9
  > (`WORKER_LLM_*`) that real CLI users — who authenticate via
10
- > `EIGENPAL_API_KEY` — don't have.
10
+ > `EIGENPAL_API_KEY` — do not have.
11
11
  >
12
12
  > Calling `eigenpal workflow step exec <type>` today exits 2 with a
13
13
  > redirect message. The server-side replacement is tracked in EIG-104:
@@ -30,7 +30,7 @@ eigenpal workflow experiment status <workflow-id> <batchId> --watch
30
30
  ```
31
31
 
32
32
  For `--example-id`, the human slug works too (`ex-01-koifer-97zf`),
33
- not just the `evx_…` id — added as part of the same iteration.
33
+ not the `evx_…` id — added as part of the same iteration.
34
34
 
35
35
  To inspect what a step actually produced (input / output / resolved
36
36
  config), use:
@@ -1,7 +1,7 @@
1
1
  # Step types
2
2
 
3
3
  The catalog is generated from `STEP_SCHEMAS` in
4
- `@eigenpal/types/src/workflow/step-configs.ts`. **Don't memorize fields
4
+ `@eigenpal/types/src/workflow/step-configs.ts`. **Do not memorize fields
5
5
  — introspect.** The CLI reads the live schema:
6
6
 
7
7
  ```bash
@@ -106,7 +106,7 @@ Reference user-supplied schema fields the same way:
106
106
 
107
107
  `control.parallel`, `control.parallel_map`, `control.foreach`, `control.if`,
108
108
  and `control.block` contain nested steps. The auto-generated catalog below
109
- can't render that shape, so the YAML form lives here. Same for the scoping
109
+ cannot render that shape, so the YAML form lives here. Same for the scoping
110
110
  rules — important because the runtime treats nested steps differently from
111
111
  top-level ones, and getting the access path wrong silently returns
112
112
  `undefined` (no error).
@@ -114,7 +114,7 @@ top-level ones, and getting the access path wrong silently returns
114
114
  ### `control.parallel` — independent branches
115
115
 
116
116
  Runs each branch concurrently. Use when several pieces of work share input
117
- but don't depend on each other.
117
+ but do not depend on each other.
118
118
 
119
119
  ```yaml
120
120
  - name: enrich
@@ -242,7 +242,7 @@ to preserve intermediate outputs).
242
242
 
243
243
  > ⚠️ Foreach does NOT isolate scopes — the last iteration's inner step
244
244
  > outputs technically persist on the top-level `steps` map and `steps.<inner>.output.x`
245
- > resolves at runtime. That's an only-the-last-row footgun; the autocomplete
245
+ > resolves at runtime. That is an only-the-last-row hazard; the autocomplete
246
246
  > hides these names and you should treat `steps.<foreach>.output.items[]`
247
247
  > as the only correct access.
248
248
 
@@ -304,7 +304,7 @@ not visible — the block runs in an isolated scope.
304
304
  `ai.classify` + `control.fail` together let a workflow reject bad inputs
305
305
  fast with a typed HTTP-style status code that callers (and evals) can
306
306
  match against. The classifier picks one label from a closed set; the
307
- fail step terminates the run when the label is one you don't want to
307
+ fail step terminates the run when the label is one you do not want to
308
308
  process.
309
309
 
310
310
  ```yaml
@@ -339,7 +339,7 @@ process.
339
339
  The synchronous run endpoint surfaces the `statusCode` as the HTTP
340
340
  response status; async runs persist `{ code, message, step }` to
341
341
  `executions.error` so the eval scorer can match against
342
- `expected/error.json` (see `reference/dataset-format.md`). When
342
+ `expected.json` with `$error` (see `reference/dataset-format.md`). When
343
343
  `condition` is omitted, `control.fail` always fails when reached, so
344
344
  compose with `control.if` for legacy gating.
345
345
 
@@ -526,7 +526,7 @@ Merge multiple named inputs into a single output
526
526
 
527
527
  #### `transform.template` — Fill Template
528
528
 
529
- Fill a DOCX template with data. Use list_templates tool to get template IDs and their placeholder schemas.
529
+ Fill a DOCX template with data from previous steps. Select a template in the workflow builder or provide a template ID from your workspace.
530
530
 
531
531
  **Config** (in `step.with`):
532
532
 
@@ -326,7 +326,7 @@ same per-page representation.
326
326
  Description-writing tips (mirrors Reducto's guidance):
327
327
 
328
328
  - Use the document's own terminology — a Slovak contract calls it `Príloha`,
329
- not "annex". List the multilingual variants you've seen.
329
+ not "annex". List the multilingual variants you have seen.
330
330
  - Mention stable visual cues if any — a centered title-page header, a
331
331
  signature line, the start of a specific table.
332
332
  - Be specific. "The collateral schedule table at the start of Príloha 2"
@@ -442,11 +442,11 @@ eigenpal workflow push --file workflow.yaml --workflow-id wf_… --bump patch
442
442
  `--bump` requires `--workflow-id` (it bumps from the current server
443
443
  version). `--set-version` works with or without `--workflow-id`. The
444
444
  YAML's top-level `version:` field is the fallback when neither flag is
445
- passed — server rejects pushes whose version doesn't strictly increase.
445
+ passed — server rejects pushes whose version does not strictly increase.
446
446
 
447
447
  ## Discover step + evaluator types
448
448
 
449
- Don't memorize fields — introspect:
449
+ Do not memorize fields — introspect:
450
450
 
451
451
  ```bash
452
452
  eigenpal workflow step-type list # full step catalog
@@ -456,7 +456,7 @@ eigenpal workflow evaluator-type get <type> # config schema for one ev
456
456
  ```
457
457
 
458
458
  Both catalogs are generated from the canonical Zod schemas in
459
- `@eigenpal/types`, so they're always in sync with what the deployment
459
+ `@eigenpal/types`, so they are always in sync with what the deployment
460
460
  supports.
461
461
 
462
462
  <!-- GENERATED:WORKFLOW_REFERENCE START -->
@@ -12,10 +12,11 @@ eigenpal run workflows.__NAME__ --example sample
12
12
  ```
13
13
 
14
14
  Add more examples under `dataset/examples/<name>/`. Each example needs an
15
- `input/arguments.json`; `expected/output.json` and `meta.json` are optional.
16
- File arguments live in `input/<arg-name>/<filename>`.
15
+ `input.json`; `expected.json` and `meta.json` are optional. File arguments live
16
+ under `input/` and are referenced from `input.json` with
17
+ `{ "$file": "input/<filename>" }`.
17
18
 
18
- When you're ready to push this workflow to the server:
19
+ When you are ready to push this workflow to the server:
19
20
 
20
21
  ```bash
21
22
  eigenpal workflow push --file workflow.yaml
@@ -1,5 +1,5 @@
1
1
  # Evaluators run after every workflow execution that's part of an eval batch.
2
- # Comment them in once you have an `expected/output.json` to compare against.
2
+ # Comment them in once you have an `expected.json` output object to compare against.
3
3
  #
4
4
  # evaluators:
5
5
  # - name: exact-match
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@eigenpal/cli",
3
- "version": "0.7.2",
3
+ "version": "0.9.0",
4
4
  "description": "Eigenpal CLI — author and run document workflows from your terminal. Agent-ready.",
5
5
  "keywords": [
6
6
  "eigenpal",
@@ -1,9 +0,0 @@
1
- # eigenpal completion
2
-
3
- Print a shell completion script to stdout (supported shells: bash, zsh, fish). Pipe into the appropriate completion file for your shell — see the script header for install instructions.
4
-
5
- ### Arguments
6
-
7
- | Name | Required | Variadic | Description |
8
- | ------- | -------- | -------- | ----------- |
9
- | `shell` | yes | no | |
@@ -1,9 +0,0 @@
1
- # eigenpal git
2
-
3
- Passthrough to git with organization remote auth and committer identity.
4
-
5
- ### Arguments
6
-
7
- | Name | Required | Variadic | Description |
8
- | --------- | -------- | -------- | ---------------------- |
9
- | `gitArgs` | no | yes | Git arguments after -- |