@eigenpal/cli 0.7.2 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. package/CHANGELOG.md +101 -23
  2. package/README.md +8 -8
  3. package/dist/cli.js +9300 -9059
  4. package/dist/skill/SKILL.md +87 -415
  5. package/dist/skill/reference/cli/agents.md +26 -26
  6. package/dist/skill/reference/cli/rerun.md +1 -1
  7. package/dist/skill/reference/cli/run.md +1 -1
  8. package/dist/skill/reference/cli/runs.md +23 -2
  9. package/dist/skill/reference/cli/surface.md +1 -2
  10. package/dist/skill/reference/cli/workflow.md +10 -10
  11. package/dist/skill/reference/dataset-format.md +74 -241
  12. package/dist/skill/reference/debugging.md +5 -5
  13. package/dist/skill/reference/evaluators.md +10 -10
  14. package/dist/skill/reference/step-exec.md +2 -2
  15. package/dist/skill/reference/step-types.md +7 -7
  16. package/dist/skill/reference/workflow-yaml.md +4 -4
  17. package/dist/templates/blank/README.md +4 -3
  18. package/dist/templates/blank/evaluators.yaml +1 -1
  19. package/package.json +1 -1
  20. package/dist/skill/reference/cli/completion.md +0 -9
  21. package/dist/skill/reference/cli/git.md +0 -9
  22. /package/dist/templates/blank/dataset/examples/sample/{expected/output.json → expected.json} +0 -0
  23. /package/dist/templates/blank/dataset/examples/sample/{input/arguments.json → input.json} +0 -0
  24. /package/dist/templates/pdf-extraction/dataset/examples/sample-invoice/{expected/output.json → expected.json} +0 -0
  25. /package/dist/templates/pdf-extraction/dataset/examples/sample-invoice/{input/arguments.json → input.json} +0 -0
  26. /package/dist/templates/text-classification/dataset/examples/negative/{expected/output.json → expected.json} +0 -0
  27. /package/dist/templates/text-classification/dataset/examples/negative/{input/arguments.json → input.json} +0 -0
  28. /package/dist/templates/text-classification/dataset/examples/positive/{expected/output.json → expected.json} +0 -0
  29. /package/dist/templates/text-classification/dataset/examples/positive/{input/arguments.json → input.json} +0 -0
package/CHANGELOG.md CHANGED
@@ -1,5 +1,83 @@
1
1
  # @eigenpal/cli
2
2
 
3
+ ## 0.9.0
4
+
5
+ ### Minor Changes
6
+
7
+ - c35d5d4: Clean up the public runs and evaluation API surface: move workflow eval operations onto automations, expose run scores/artifacts through the unified runs API, and update CLI/SDK helpers to use the canonical routes.
8
+
9
+ Breaking: `runs.evalResults` and `GET /api/v1/runs/{id}/eval-results` are replaced by `runs.scores` and `GET /api/v1/runs/{id}/scores`.
10
+
11
+ ### Patch Changes
12
+
13
+ - c35d5d4: Clarify `--max-wait` help text so timeout exit behavior reads naturally.
14
+
15
+ ## 0.8.0
16
+
17
+ ### Minor Changes
18
+
19
+ - d620a00: Expose the clean public automation API around automations, runs, files/artifacts, datasets, evaluators, experiments, evaluation results, and run promotion. This intentionally breaks the pre-v1 SDK/CLI surface that exposed workflow- and agent-specific top-level resources outside the explicit legacy compatibility routes.
20
+
21
+ > [!WARNING]
22
+ > **Breaking change (pre-v1).** The workflow- and agent-specific top-level
23
+ > resources have been removed in favor of a single automation-centric surface.
24
+ > These packages are still 0.x, so this breaking redesign ships as a `minor`
25
+ > bump. Update your integration before upgrading.
26
+
27
+ #### Removed
28
+ - Top-level workflow resources, e.g. `client.workflows.*`, including
29
+ `client.workflows.executions.runAndWait(...)` and the rest of the
30
+ `client.workflows.executions.*` tree.
31
+ - Top-level agent resources, e.g. `client.agents.*` (runs, feedback, traces,
32
+ files, expected artifacts).
33
+ - Top-level source/git resources, e.g. `client.sources.*`.
34
+
35
+ #### New equivalents
36
+ - **Start a run:** use `client.run('workflows.<slug>', input, options)` (and
37
+ `client.run('agents.<slug>', ...)`). In the TypeScript SDK, `run-and-wait`
38
+ is `client.run(target, input, { waitForCompletion })`; in the Python SDK use
39
+ `client.run_and_wait(target, input=...)`.
40
+ - **Browse automations** (workflows + agents): `client.automations.list()`,
41
+ `client.automations.get(id)`, `client.automations.versions(id)`,
42
+ `client.automations.triggers(id)`, plus
43
+ `client.automations.dataset|examples|evaluators|experiments.*`.
44
+ - **Inspect & control runs:** `client.runs.list()`, `client.runs.get(id)`,
45
+ `client.runs.cancel(id)`, `client.runs.rerun(id)`, `client.runs.promote(id)`,
46
+ `client.runs.usage(id)`, `client.runs.steps(id)`, `client.runs.events(id)`,
47
+ and `client.runs.artifacts|eval_results|feedback|trace.*`.
48
+ - **Manage reusable files:** `client.files.upload(...)`, `client.files.get(id)`,
49
+ `client.files.download(id)`, `client.files.delete(id)`.
50
+
51
+ #### Migration
52
+ - `client.workflows.executions.runAndWait('<slug>', input)` →
53
+ `client.run('workflows.<slug>', input, { waitForCompletion })` (TS) /
54
+ `client.run_and_wait('workflows.<slug>', input=...)` (Python).
55
+ - `client.workflows.executions.create('<slug>', input)` →
56
+ `client.run('workflows.<slug>', input)`.
57
+ - `client.agents.runs.create('<slug>', input)` →
58
+ `client.run('agents.<slug>', input)`.
59
+ - `client.workflows.get('<slug>')` / `client.agents.get('<slug>')` →
60
+ `client.automations.get('workflows.<slug>')` /
61
+ `client.automations.get('agents.<slug>')`.
62
+ - Listing/inspecting executions previously under
63
+ `client.workflows.executions.*` / `client.agents.runs.*` →
64
+ `client.runs.*` (e.g. `client.runs.list()`, `client.runs.get(id)`).
65
+ - `eigenpal runs promote` now accepts only `--name` (optional). Use
66
+ `eigenpal runs feedback update` to set expected JSON before promoting.
67
+ Feedback and expected commands work for both workflow and agent runs.
68
+
69
+ #### DX fixes
70
+ - `client.runs.*` read methods are now typed against the generated OpenAPI
71
+ models; use `isRunFinished()` to narrow `run()` / `rerun()` responses before
72
+ accessing `output`.
73
+ - `eigenpal agents dataset push --file` accepts a `.zip` archive (matching
74
+ `pull` output) as well as a directory.
75
+
76
+ ### Patch Changes
77
+
78
+ - e2e88d7: Drop the deprecated `git` passthrough and the `completion` helper from the generated CLI reference docs and bundled skill reference. Both commands remain fully functional and discoverable via `--help`.
79
+ - e67f2f3: Fix staging QA CLI gaps: resolve profiles by tenant display name (e.g. `EIGENPAL_PROFILE="Timur's org"`) and cap workflow experiment example pagination at the API `limit=100` maximum.
80
+
3
81
  ## 0.7.0
4
82
 
5
83
  ### Minor Changes
@@ -26,7 +104,7 @@
26
104
 
27
105
  ### Patch Changes
28
106
 
29
- - 5ea0bb5: `workflow dataset pull` gains `--example-id <id>` (repeatable) to export a single example or a selected subset of a dataset, instead of always pulling the whole dataset. The subset ZIP uses the same archive layout as a full pull, so it re-imports into any dataset via `dataset push --mode append`. Requesting an example id that doesn't exist for the workflow exits non-zero.
107
+ - 5ea0bb5: `workflow dataset pull` gains `--example-id <id>` (repeatable) to export a single example or a selected subset of a dataset, instead of always pulling the whole dataset. The subset ZIP uses the same archive layout as a full pull, so it re-imports into any dataset via `dataset push --mode append`. Requesting an example id that does not exist for the workflow exits non-zero.
30
108
 
31
109
  ## 0.6.10
32
110
 
@@ -57,7 +135,7 @@
57
135
 
58
136
  ### Minor Changes
59
137
 
60
- - 2aeeaf2: Add first-class source `git save` and branch-aware `git release` flows for Git-backed builder workspaces.
138
+ - 2aeeaf2: Add native source `git save` and branch-aware `git release` flows for Git-backed builder workspaces.
61
139
 
62
140
  ### Patch Changes
63
141
 
@@ -79,7 +157,7 @@
79
157
  ### Patch Changes
80
158
 
81
159
  - 99cad97: Skill reference (auto-generated from `WorkflowDefinitionSchema`) now documents the new `values` row for enum inputs, alongside the existing `items` row used by array inputs. Output-only doc refresh — no CLI behavior change.
82
- - d70f697: Document the `control.parallel`, `control.parallel_map`, `control.foreach`, `control.if`, and `control.block` step types in the bundled skill reference. The auto-generated catalog can't render nested step shapes, so the YAML form, output access paths (e.g. `steps.<parallel>.output.<branch>.<inner>.<field>`), and scoping rules now live as hand-written prose. Also documents the multi-step iteration footgun: only the LAST inner step's output is keyed into each `items[i]` for `parallel_map` / `foreach` — end the body with a `transform.script` if you need to preserve intermediate fields.
160
+ - d70f697: Document the `control.parallel`, `control.parallel_map`, `control.foreach`, `control.if`, and `control.block` step types in the bundled skill reference. The auto-generated catalog cannot render nested step shapes, so the YAML form, output access paths (e.g. `steps.<parallel>.output.<branch>.<inner>.<field>`), and scoping rules now live as hand-written prose. Also documents the multi-step iteration hazard: only the LAST inner step's output is keyed into each `items[i]` for `parallel_map` / `foreach` — end the body with a `transform.script` if you need to preserve intermediate fields.
83
161
  - ed2cde6: Fix `eigenpal workflow pull` writing a 0-byte file. The v1 single-workflow endpoint now returns the current YAML at the top level (`yamlContent`) via a new `WorkflowDetail` response shape, and the CLI throws a clear error instead of silently writing empty output when no version has been published. Also fixes adjacent regressions caused by the same picker: `workflow list` now shows the workflow name and version columns (previously always `-`), `workflow push --bump` reads the server's current version correctly, the "did you mean?" suggestion on a slug typo lists candidates again, and `workflow execution list` shows a duration column computed from the run start/end timestamps.
84
162
 
85
163
  ## 0.5.11
@@ -149,7 +227,7 @@
149
227
  - `eigenpal status` exits non-zero when authentication fails, so scripts can `eigenpal status && deploy`.
150
228
  - New `eigenpal completion <bash|zsh|fish>` command — emits a shell-completion script for the live command tree. Install via `eigenpal completion bash > /usr/local/etc/bash_completion.d/eigenpal` (or your shell's equivalent).
151
229
  - New global `--quiet` / `-q` flag — suppresses `success()` / `info()` / `dim()` lines while keeping `error()` and `warn()` (and JSON output) intact.
152
- - New `addJsonFlagMutation()` helper alongside `addJsonFlag()`. Mutating commands that don't have a curated table view now use it so `--verbose` no longer appears in `--help` where it's a no-op. (Wire-up in mutating handlers happens in a follow-up commit on this branch.)
230
+ - New `addJsonFlagMutation()` helper alongside `addJsonFlag()`. Mutating commands that do not have a curated table view now use it so `--verbose` no longer appears in `--help` where it is a no-op. (Wire-up in mutating handlers happens in a follow-up commit on this branch.)
153
231
 
154
232
  - b2c4308: CLI: per-example dataset CRUD.
155
233
 
@@ -164,7 +242,7 @@
164
242
 
165
243
  Skill recipes restored: `SKILL.md` and `reference/dataset-format.md` document the three flows (capture corrected output as new GT, add one example, delete one bad row).
166
244
 
167
- - d41ee8b: CLI: drop the `--wait` flag from `workflow execution run`. The local-folder runner already waits for every example to reach a terminal state, so the flag was a documented no-op kept "for symmetry with `experiment run --wait`." That symmetry isn't worth a dead flag in the user-facing help screen — `experiment run --wait` is meaningful (it polls batch status), `execution run --wait` never was.
245
+ - d41ee8b: CLI: drop the `--wait` flag from `workflow execution run`. The local-folder runner already waits for every example to reach a terminal state, so the flag was a documented no-op kept "for symmetry with `experiment run --wait`." That symmetry is not worth a dead flag in the user-facing help screen — `experiment run --wait` is meaningful (it polls batch status), `execution run --wait` never was.
168
246
 
169
247
  Behavior is unchanged. Scripts that passed `--wait` will now see an unknown-option error from Commander; remove the flag.
170
248
 
@@ -238,7 +316,7 @@
238
316
 
239
317
  - 9dc84b9: CLI: agent-friendly experiment polling, watching, and comparison.
240
318
 
241
- Four small additions so agents (and humans writing monitoring scripts) don't have to roll their own bash pollers, JSON parsers, or aggregators on top of `experiment status`.
319
+ Four small additions so agents (and humans writing monitoring scripts) do not have to roll their own bash pollers, JSON parsers, or aggregators on top of `experiment status`.
242
320
  - **`experiment status --json` now includes a top-level `summary` rollup** — `{ total, terminal, complete, completedCount, failedCount, cancelledCount, rejectedCount, runningCount, pendingCount }`. Polling scripts can read `.summary.complete` / `.summary.failedCount` directly instead of folding `executions[].status` themselves. `executions[]` stays as-is for backwards compat.
243
321
  - **`experiment status --short`** — single-line, awk-friendly stdout summary (e.g. `6/6 done failed=0 cancelled=0 rejected=0`). Mutually exclusive with `--watch`. Drops the failure detail block + closing tip so monitoring loops stay clean.
244
322
  - **New `experiment watch <workflow-id> <batchId>`** — polls until terminal, then auto-pulls eval results to `./results-<batchId>.<format>` in one command. Replaces the old "monitor + status + pull + score" recipe. Flags: `--interval`, `--max-wait`, `--format csv|json`, `--pull-on-complete <path>` to override destination, `--no-pull` to opt out. Exit codes match `experiment status --watch` (0 clean, 1 any failure, 2 deadline).
@@ -321,7 +399,7 @@
321
399
 
322
400
  Existing evaluator YAML must be ported by hand: rename `body` (or
323
401
  `code`) to `function`, and add the `function scoreScript(expected,
324
- actual) { ... }` declaration around your statements. Parameter order is
402
+ actual) {... }` declaration around your statements. Parameter order is
325
403
  load-bearing — swapping `expected` and `actual` would silently invert
326
404
  every score.
327
405
 
@@ -373,7 +451,7 @@ actual) { ... }` declaration around your statements. Parameter order is
373
451
  migrates `code:` to `function:` on the next save. New workflows
374
452
  should use `function:`. The schema rejects:
375
453
  - function names other than `script`
376
- - parameter lists that don't match `Object.keys(inputs)` in order
454
+ - parameter lists that do not match `Object.keys(inputs)` in order
377
455
  - `async` / `import()` / `require()` (sandbox-incompatible)
378
456
  - bodies with no `return` or `throw`
379
457
 
@@ -394,7 +472,7 @@ actual) { ... }` declaration around your statements. Parameter order is
394
472
 
395
473
  **`--json` returns the table's columns, not the server envelope.** Previously `--json` dumped `{ data: [{...full server rows}], total }`. Now it emits an array containing exactly the fields the table renders (e.g. `[{id, name, version, updatedAt}]` for `workflow list`). `total` still goes to stderr in both modes.
396
474
 
397
- **`--verbose`** opts into the firehose: `--json --verbose` emits the full server rows for the rare cases you need fields the table doesn't surface.
475
+ **`--verbose`** opts into full rows: `--json --verbose` emits the full server rows for the rare cases you need fields the table does not surface.
398
476
 
399
477
  `workflow list --json | jq '.[0].id'` — the natural shape for piping.
400
478
 
@@ -408,7 +486,7 @@ actual) { ... }` declaration around your statements. Parameter order is
408
486
  ("Create a new key, copy it, paste below"), and the API key prompt **hides
409
487
  input** as you type or paste — same as `gh auth login`, `stripe login`, etc.
410
488
  - d41ee8b: CLI: two cleanups for the audit follow-ups.
411
- - **`init agent` and bare `agent` now exit 2** instead of 0 when invoked. Both subcommands are reserved-but-unimplemented placeholders, and exiting 0 made it possible for a CI script that misrouted into one of them to silently no-op. POSIX exit 2 ("command exists but isn't usable as invoked") makes the misroute fail loudly. The human-readable "coming soon" stderr message is unchanged.
489
+ - **`init agent` and bare `agent` now exit 2** instead of 0 when invoked. Both subcommands are reserved-but-unimplemented placeholders, and exiting 0 made it possible for a CI script that misrouted into one of them to silently no-op. POSIX exit 2 ("command exists but is not usable as invoked") makes the misroute fail loudly. The human-readable "coming soon" stderr message is unchanged.
412
490
  - **New `--no-color` global flag.** Disables ANSI colors in all output, matching the convention shared by `gh`, `kubectl`, `npm`, and `terraform`. Equivalent to setting `NO_COLOR=1` in the environment, but takes effect for the current invocation only. Implemented as a lazy picocolors wrapper so the flip propagates to every helper in `lib/ui.ts` (and every downstream caller of `pc`) regardless of when in the command lifecycle the flag is parsed.
413
491
 
414
492
  - 809f3d4: `auth list`: the org name is now the primary label (bold, immediately after
@@ -416,7 +494,7 @@ actual) { ... }` declaration around your statements. Parameter order is
416
494
  to `auth use <name>` or `EIGENPAL_PROFILE=<name>`), and the base URL trails
417
495
  on the right. The redundant `(tenantId)` column was dropped — the profile
418
496
  slug already disambiguates. `auth use` (interactive picker) now prints the
419
- non-interactive form on success so you learn the slug you just picked.
497
+ non-interactive form on success so you learn the slug you picked.
420
498
  - 49a8f94: Fixed a stray `undefined` in the `eigenpal auth login` outro. The success message used `dim(...)` (a stderr printer that returns `void`) inside a template literal where `ui.dim(...)` (the inline string formatter) was meant. The dim "Saved profile X — switch with" hint now renders correctly instead of literal `undefined`.
421
499
  - 16c6e83: Updated `@eigenpal/cli` author from `Eigenpal <hello@eigenpal.com>` to `Jedr Blaszyk <jedr@eigenpal.com>`. Surfaces on the npm package page and in `npm view @eigenpal/cli`.
422
500
  - b1d2d77: CLI: default `baseUrl` is now `https://studio.eigenpal.com` instead of `http://localhost:3000`.
@@ -461,16 +539,16 @@ actual) { ... }` declaration around your statements. Parameter order is
461
539
  - c82a215: Polish on `eigenpal auth login` based on DX review:
462
540
  - **Browser fallback** — the prompt's note now repeats the API-keys URL prominently, so when `xdg-open`/`open`/`start` silently fails (headless VMs, WSL without xdg-open, restricted containers) the user has a copyable URL right next to the prompt instead of a dim status line above.
463
541
  - **Outro is informative** — distinguishes "Saved profile X" (first login to this tenant) from "Updated profile X" (re-login overwrote prior creds in place) and shows the exact `eigenpal auth use X` command to switch to it later. No more raw `(profile org_xxxx)` dead-end.
464
- - **Path-rejection error** suggests the corrected origin: instead of just "URL must be the server origin only", it now says e.g. "Try https://studio.eigenpal.com instead of '/api/v1'".
542
+ - **Path-rejection error** suggests the corrected origin: instead of "URL must be the server origin only", it now says e.g. "Try https://studio.eigenpal.com instead of '/api/v1'".
465
543
  - **Non-TTY error copy** is consistent across CI and pipe-redirect paths — both name the missing piece (interactive terminal) and the workaround (`--base-url` / env var).
466
544
 
467
545
  - be693a3: `eigenpal auth login` no longer opens your browser as the very first thing it does. The flow now mirrors mature CLIs (`gh auth login`, `vercel login`):
468
546
  1. **Pick the server** (existing select prompt).
469
- 2. **Explain the flow** — a clear step-by-step note tells you what's about to happen and surfaces the dashboard URL prominently so you can copy it manually if you prefer.
547
+ 2. **Explain the flow** — a clear step-by-step note tells you what is about to happen and surfaces the dashboard URL prominently so you can copy it manually if you prefer.
470
548
  3. **Confirm browser launch** — "Open the dashboard in your browser?" (default Yes). Users on headless VMs, WSL without xdg-open, or who already have the dashboard open can answer No and skip the best-effort browser launch entirely.
471
549
  4. **Paste the API key** — same as before.
472
550
 
473
- No surprise side effects, no "wait, why did a browser pop?" moment. The URL is also in the explanation note so a failed `openBrowser` doesn't strand you.
551
+ No surprise side effects, no "wait, why did a browser pop?" moment. The URL is also in the explanation note so a failed `openBrowser` does not strand you.
474
552
 
475
553
  - c82a215: `package.json` now carries full npm metadata: `license: Apache-2.0`, `keywords`, `repository`, `bugs`, `homepage`, and `author`. The published tarball also bundles `LICENSE` and `CHANGELOG.md` so the npm page renders the license badge instead of "License: none" and surfaces real keywords for search.
476
554
  - ee32a73: Three fixes around the public CLI mirror and login UX:
@@ -481,8 +559,8 @@ actual) { ... }` declaration around your statements. Parameter order is
481
559
  - Dropped `?from=cli` from the dashboard URL — the dashboard ignores the param, and a clean URL is easier to copy/paste/visit manually.
482
560
 
483
561
  - c82a215: Public README cleanup:
484
- - Removed the duplicated `# @eigenpal/cli` heading and redundant install snippet that was being prepended to the public mirror's README. The bundled CLI README is already self-contained, so the synth header was just stacking a second title on top.
485
- - Reordered sections so users hit "what to install + what's available" first: Install → Commands → Use it → Primitives. Primitives is the conceptual reference; it now sits below the hands-on sections instead of above them.
562
+ - Removed the duplicated `# @eigenpal/cli` heading and redundant install snippet that was being prepended to the public mirror's README. The bundled CLI README is already self-contained, so the synth header was stacking a second title on top.
563
+ - Reordered sections so users hit "what to install + what is available" first: Install → Commands → Use it → Primitives. Primitives is the conceptual reference; it now sits below the hands-on sections instead of above them.
486
564
 
487
565
  - ac95906: CLI: `eigenpal skill` now installs into 9 popular AI coding tools instead of 2.
488
566
 
@@ -540,16 +618,16 @@ actual) { ... }` declaration around your statements. Parameter order is
540
618
  - c82a215: Removed all `localhost:3000` references from user-visible help text and source comments. The `auth login --help` examples no longer suggest `--base-url http://localhost:3000` — local development is an internal concern, not something CLI users need to think about. Defaults still target `https://studio.eigenpal.com`; advanced users with their own deployments can still override via `--base-url` or `EIGENPAL_BASE_URL`.
541
619
  - c82a215: `eigenpal --version` now prints `dev` for unpublished local builds instead of the confusing `0.0.0-placeholder`. If you ever see `dev`, the binary on your PATH is a local source build (e.g., `bun cli:install:local`), not the npm-published one — run `which -a eigenpal` to find it.
542
620
 
543
- CI publish also gained a hard check that fails the release if `dist/cli.js --version` doesn't match the `release_tag`, so a placeholder version can never reach users via npm.
621
+ CI publish also gained a hard check that fails the release if `dist/cli.js --version` does not match the `release_tag`, so a placeholder version can never reach users via npm.
544
622
 
545
623
  - 71361fd: `eigenpal --version` (and `-v`) now reports the actual published package version instead of a hardcoded `0.0.1`. For local dev (`eigenpal-dev`) the placeholder value (`0.0.0-placeholder`) makes it obvious the bundle came from a workspace build, not npm.
546
624
  - 71361fd: Dataset folder format made symmetric — `expected/` mirrors `input/` exactly:
547
625
  - `input/arguments.json` (scalar args) ↔ `expected/output.json` (raw data ground truth, **no envelope wrapper**)
548
626
  - `input/<argName>/<file>` (input file binary) ↔ `expected/<docKey>/<file>` (expected file output binary)
549
627
 
550
- `expected/output.json` is now the raw user JSON, mirroring the workflow's `output:` shape 1:1. Drop the `{ data, expectedDocuments }` envelope requirement at the dataset boundary; the server still stores the envelope shape internally but the translation happens at import/export time, just like input file uploads. This is a pre-launch breaking change to the dataset folder layout — re-export your datasets to pick up the cleaner shape.
628
+ `expected/output.json` is now the raw user JSON, mirroring the workflow's `output:` shape 1:1. Drop the `{ data, expectedDocuments }` envelope requirement at the dataset boundary; the server still stores the envelope shape internally but the translation happens at import/export time, matching input file uploads. This is a pre-launch breaking change to the dataset folder layout — re-export your datasets to pick up the cleaner shape.
551
629
 
552
- Server import walks `expected/<docKey>/` folders, uploads binaries via `evalExpectedKey`, builds the `expectedDocuments` map server-side. Server export reverses: writes `expectedOutput.data` as raw `output.json`, emits each `expectedDocuments[docKey]` entry as a binary under `expected/<docKey>/<filename>`. Round-trip clean.
630
+ Server import walks `expected/<docKey>/` folders, uploads binaries via the workflow eval expected storage template, builds the `expectedDocuments` map server-side. Server export reverses: writes `expectedOutput.data` as raw `output.json`, emits each `expectedDocuments[docKey]` entry as a binary under `expected/<docKey>/<filename>`. Round-trip clean.
553
631
 
554
632
  CLI `validate dataset` accepts any JSON object as `output.json` (drops the envelope check), and validates `expected/<docKey>/` folder names match the same `[a-z0-9][a-z0-9-_]*` pattern as input arg folders. `reference/dataset-format.md` documents the new symmetric layout with a worked example.
555
633
 
@@ -592,7 +670,7 @@ actual) { ... }` declaration around your statements. Parameter order is
592
670
  - `workflow get-experiment-status --watch` exited 0 when the first poll raced before the server enqueued executions (`0/0 terminal` was vacuously "done"). Now requires `total > 0` to consider the batch terminal.
593
671
  - `workflow set-dataset` exited 0 on an unrecognized terminal NDJSON phase (truncated stream or future server-side phase). Now exits 1 with a clear `Import response had unexpected terminal phase` message.
594
672
  - Dataset import silently dropped extra files when a folder under `input/<argName>/` had multiple binaries but the workflow input was declared `type: file` (single). Now rejects up front with `code: single_file_input_overpopulated` and a hint to either trim the folder or change the input to the array form.
595
- - Misleading import error: entries under `expected/` that didn't fit the `<docKey>/<file>` pattern got an error message saying "entry under input/ ..." with an `input/` example. Message now branches on the actual parent.
673
+ - Misleading import error: entries under `expected/` that didn't fit the `<docKey>/<file>` pattern got an error message saying "entry under input/..." with an `input/` example. Message now branches on the actual parent.
596
674
  - `validate dataset` could crash on broken symlinks / FIFOs. `statSync` now wrapped in try/catch with a structured "unreadable filesystem entry" issue.
597
675
 
598
676
  **Simplifications** (net −21 LOC after both passes):
@@ -617,11 +695,11 @@ actual) { ... }` declaration around your statements. Parameter order is
617
695
  `NO_COLOR` and non-TTY stdout automatically.
618
696
 
619
697
  - 71361fd: Add a "Common recipes" section to the bundled skill (`SKILL.md`) covering the typical end-to-end command sequences agents need: scaffolding a workflow, validating + pushing workflow YAML, validating + pushing evaluators, building + uploading a dataset (and editing examples server-side), running an experiment + collecting results, and debugging a failing execution.
620
- - 71361fd: Generate the skill reference (step types, evaluators, workflow YAML) from the canonical Zod schemas in `@eigenpal/types` on every build, with a CI gate that blocks PRs whose schema changes don't ship the regenerated docs.
698
+ - 71361fd: Generate the skill reference (step types, evaluators, workflow YAML) from the canonical Zod schemas in `@eigenpal/types` on every build, with a CI gate that blocks PRs whose schema changes do not ship the regenerated docs.
621
699
  - 71361fd: Tighten error-message quality across the dataset/eval/execute pipeline so the CLI surfaces actionable feedback instead of opaque dumps.
622
- - `eigenpal workflow set-dataset` now decodes the import endpoint's NDJSON terminal event: `aborted` renders the structured `issues[]` with an aligned field column and exits non-zero; `done` warns loudly when `expectedSet < created` so missing `expected/output.json` files don't slip through silently. Adds smart hints for `invalid_expected_output` and `argument_name_collision` codes.
700
+ - `eigenpal workflow set-dataset` now decodes the import endpoint's NDJSON terminal event: `aborted` renders the structured `issues[]` with an aligned field column and exits non-zero; `done` warns loudly when `expectedSet < created` so missing `expected/output.json` files do not slip through silently. Adds smart hints for `invalid_expected_output` and `argument_name_collision` codes.
623
701
  - `POST /api/v1/execute` replaced its legacy `{ error, details }` payload with the structured envelope (`{ issues, hint, requestId, docsUrl }`) the CLI's `printApiError` already consumes.
624
- - `exact-diff` evaluator no longer falls back to "compare whole envelope when `.data` is missing." That fallback silently absorbed the import shape bugs we just fixed; it now requires `.data` on both sides (matching the system-wide `WorkflowResult` and `ExpectedOutputSchema` contracts) so any future shape regression fails loudly.
702
+ - `exact-diff` evaluator no longer falls back to "compare whole envelope when `.data` is missing." That fallback silently absorbed the import shape bugs we fixed; it now requires `.data` on both sides (matching the system-wide `WorkflowResult` and `ExpectedOutputSchema` contracts) so any future shape regression fails loudly.
625
703
  - Exports `printIssues` from the CLI's `validate` command so other commands reuse the same field-aligned issue formatter.
626
704
 
627
705
  - 71361fd: Structured validation errors with introspection hints. `workflow set-definition`
package/README.md CHANGED
@@ -57,14 +57,14 @@ eigenpal runs list <id> --type workflow --json | jq # query as JSON
57
57
 
58
58
  ## Environment variables
59
59
 
60
- Most users never need to set these — `eigenpal auth login` writes a profile to `~/.config/eigenpal/credentials.json` and every command derives its config from there. Reach for env vars when you can't run an interactive login (CI), need to switch context for one shell, or want to override a single field without editing the credentials file.
61
-
62
- | Variable | Purpose |
63
- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
64
- | `EIGENPAL_API_KEY` | Bypass the profile entirely. Set in CI to skip `auth login`. When set, `EIGENPAL_BASE_URL` (or the cloud default) is used — the active profile is **not** consulted, so a stale profile can't redirect a CI run to the wrong server. |
65
- | `EIGENPAL_BASE_URL` | Override the server URL for one command or shell. Pairs with `EIGENPAL_API_KEY` to point CI at an on-prem deployment. Without `EIGENPAL_API_KEY`, this overrides whatever the active profile would have used. |
66
- | `EIGENPAL_PROFILE` | Switch the active profile for one shell without touching `~/.config/eigenpal/credentials.json`. Useful for ad-hoc context switches: `EIGENPAL_PROFILE=staging eigenpal status`. Persistent equivalent: `eigenpal auth use <name>`. |
67
- | `EIGENPAL_DIR` | Override the workflow project directory used by `init` / `validate`. Defaults to `./eigenpal`. |
60
+ Most users never need to set these — `eigenpal auth login` writes a profile to `~/.config/eigenpal/credentials.json` and every command derives its config from there. Reach for env vars when you cannot run an interactive login (CI), need to switch context for one shell, or want to override a single field without editing the credentials file.
61
+
62
+ | Variable | Purpose |
63
+ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
64
+ | `EIGENPAL_API_KEY` | Bypass the profile entirely. Set in CI to skip `auth login`. When set, `EIGENPAL_BASE_URL` (or the cloud default) is used — the active profile is **not** consulted, so a stale profile cannot redirect a CI run to the wrong server. |
65
+ | `EIGENPAL_BASE_URL` | Override the server URL for one command or shell. Pairs with `EIGENPAL_API_KEY` to point CI at an on-prem deployment. Without `EIGENPAL_API_KEY`, this overrides whatever the active profile would have used. |
66
+ | `EIGENPAL_PROFILE` | Switch the active profile for one shell without touching `~/.config/eigenpal/credentials.json`. Useful for ad-hoc context switches: `EIGENPAL_PROFILE=staging eigenpal status`. Persistent equivalent: `eigenpal auth use <name>`. |
67
+ | `EIGENPAL_DIR` | Override the workflow project directory used by `init` / `validate`. Defaults to `./eigenpal`. |
68
68
 
69
69
  Resolution precedence: command-line flags > env vars > active profile > defaults.
70
70