@eigenpal/cli 0.10.7 → 0.10.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/dist/cli.js +743 -7438
- package/dist/skill/SKILL.md +4 -0
- package/dist/skill/reference/evaluators.md +4 -4
- package/dist/skill/reference/step-types.md +44 -0
- package/package.json +1 -1
package/dist/skill/SKILL.md
CHANGED
|
@@ -49,6 +49,10 @@ eigenpal workflow step-type get ai.extract --json | jq '.configSchema.properties
|
|
|
49
49
|
|
|
50
50
|
# Validate and push.
|
|
51
51
|
eigenpal workflow validate ./workflow.yaml
|
|
52
|
+
# If the workflow has action.invoke-workflow steps, local validation cannot
|
|
53
|
+
# resolve their targets (sibling workflows live in the server DB). Add --online
|
|
54
|
+
# to check invoke targets, input types, and cycles before pushing.
|
|
55
|
+
eigenpal workflow validate ./workflow.yaml --online
|
|
52
56
|
eigenpal workflow push --file workflow.yaml
|
|
53
57
|
|
|
54
58
|
# Run with ad-hoc input and read the output (use this for API-triggered workflows).
|
|
@@ -249,7 +249,7 @@ _Generated from `EvalConfigYamlSchema` in `@eigenpal/types/src/eval/evaluator-co
|
|
|
249
249
|
| Field | Type | Required | Default | Description |
|
|
250
250
|
| --- | --- | --- | --- | --- |
|
|
251
251
|
| `evaluators` | array<object> | no | `[]` | |
|
|
252
|
-
| `passThreshold` | number | no |
|
|
252
|
+
| `passThreshold` | number | no | | |
|
|
253
253
|
|
|
254
254
|
### Common entry fields (every evaluator type)
|
|
255
255
|
|
|
@@ -265,7 +265,7 @@ _Generated from `EvalConfigYamlSchema` in `@eigenpal/types/src/eval/evaluator-co
|
|
|
265
265
|
| --- | --- | --- | --- | --- |
|
|
266
266
|
| `numericTolerance` | number | no | `0.000001` | Maximum absolute difference between numeric values that still counts as equal. Use 1e-6 for floats, 0 for exact integer match. |
|
|
267
267
|
| `allowExtraFields` | boolean | no | `true` | When on, extra fields in the actual output do not fail the diff. Off = actual must match expected exactly. |
|
|
268
|
-
| `passThreshold` | number | no |
|
|
268
|
+
| `passThreshold` | number | no | | Legacy per-evaluator pass threshold. Pass/fail now uses the single workflow-level pass threshold; this is read only to preserve the run gate of configs authored before the single-threshold model. |
|
|
269
269
|
| `paths` | array<string> | no | | Dot-paths to scope the diff to a subset of the expected tree. Empty = diff entire expected. Syntax: `header.id`, `lineItems[].total`, `lineItems[0].sku`. Use this when expected has noisy sections you do not want to score. |
|
|
270
270
|
|
|
271
271
|
|
|
@@ -276,7 +276,7 @@ _Generated from `EvalConfigYamlSchema` in `@eigenpal/types/src/eval/evaluator-co
|
|
|
276
276
|
| `model` | string | no | | Which LLM grades the output. Falls back to the workspace's default LLM provider when unset. |
|
|
277
277
|
| `mode` | `"continuous"` \| `"discrete"` | no | `"continuous"` | Continuous = the LLM returns a free-form score in [0, 1]. Discrete = the LLM picks one of your labels and the score is looked up from the table. |
|
|
278
278
|
| `labels` | record<string, number> | no | | Allowed labels and the score each maps to. Required in discrete mode. The judge MUST return one of these labels; each score must be in [0, 1]. |
|
|
279
|
-
| `passThreshold` | number | no |
|
|
279
|
+
| `passThreshold` | number | no | | Legacy per-evaluator pass threshold. Pass/fail now uses the single workflow-level pass threshold; this is read only to preserve the run gate of configs authored before the single-threshold model. |
|
|
280
280
|
| `promptExtension` | string | yes | | Required. Evaluation criteria appended to the judge prompt — describe what makes a good answer for this workflow (tone, completeness, edge cases). Cannot be empty: an empty prompt yields the model inventing its own criteria. |
|
|
281
281
|
| `docLabel` | string | no | `"output"` | Name the judge uses to refer to the output in its prompt (e.g. 'invoice', 'extraction'). Default is 'output'. |
|
|
282
282
|
|
|
@@ -288,5 +288,5 @@ _Generated from `EvalConfigYamlSchema` in `@eigenpal/types/src/eval/evaluator-co
|
|
|
288
288
|
| `function` | string | yes | | Full TypeScript source for optional `type` aliases plus `function scoreScript(expected, actual): number { ... }`. Receives `expected` (the example's expected output) and `actual` (the workflow's actual output), returns a number in [0, 1] (the `: number` annotation is required). The `: number` return-type annotation is required and enforced at parse time. Throws are caught and scored as 0. |
|
|
289
289
|
| `timeoutMs` | integer | no | `5000` | Maximum wall-clock time the script can run before it is killed and the run is marked failed. |
|
|
290
290
|
| `memoryLimitMb` | integer | no | `10` | Maximum memory the sandbox may allocate. Increase if the script processes large arrays or strings. |
|
|
291
|
-
| `passThreshold` | number | no |
|
|
291
|
+
| `passThreshold` | number | no | | Legacy per-evaluator pass threshold. Pass/fail now uses the single workflow-level pass threshold; this is read only to preserve the run gate of configs authored before the single-threshold model. |
|
|
292
292
|
<!-- GENERATED:EVALUATOR_CATALOG END -->
|
|
@@ -390,6 +390,10 @@ Extract structured data from text using AI with a JSON schema
|
|
|
390
390
|
| `provider` | string | no | | Provider ID (e.g., "openai-gpt4o") |
|
|
391
391
|
| `model` | string | no | | Model override |
|
|
392
392
|
| `maxInputTokens` | integer | no | | Max input tokens. Truncates input text and logs a warning when exceeded. Omit for no limit. |
|
|
393
|
+
| `grounded` | boolean | no | | Optional. When true, adds a grounding pass: each schema field gets a source span + confidence (high=verbatim, medium=fuzzy, low=ungrounded) under a `_grounding` key, and ungrounded/fuzzy fields are flagged for human review. Grounding always calls OpenAI directly (independent of the extract provider) and requires OPENAI_API_KEY. Values stay the reliable schema-typed ones. |
|
|
394
|
+
| `groundingModel` | string | no | | Provider/model for the grounding pass. Defaults to the workspace default LLM; grounding runs against OpenAI, so pick an OpenAI-compatible model. |
|
|
395
|
+
| `groundingExamples` | array<object> | no | | Optional few-shot examples pinning grounding to verbatim source text per field. |
|
|
396
|
+
| `reviewOn` | `"medium_or_low"` \| `"low_only"` | no | | Which grounding confidences flag a field for review. Default: medium_or_low. |
|
|
393
397
|
|
|
394
398
|
**Output:** `record<string, unknown>`
|
|
395
399
|
|
|
@@ -415,6 +419,27 @@ Split a parsed document into named sections using an LLM. Consumes ai.parse outp
|
|
|
415
419
|
| --- | --- | --- | --- | --- |
|
|
416
420
|
| `splits` | array<object> | yes | | Sections found in the document, in page order. Absent sections are omitted. |
|
|
417
421
|
|
|
422
|
+
#### `ai.segment` — Separate Documents
|
|
423
|
+
|
|
424
|
+
Separate a concatenated batch (one big scan) into typed document instances using an LLM. Consumes ai.parse output and a type taxonomy; discovers an unknown number of documents in any order and emits per-document page ranges + text + type, ready for type-specific ai.extract via control.parallel_map. The inverse of ai.split.
|
|
425
|
+
|
|
426
|
+
**Config** (in `step.with`):
|
|
427
|
+
|
|
428
|
+
| Field | Type | Required | Default | Description |
|
|
429
|
+
| --- | --- | --- | --- | --- |
|
|
430
|
+
| `input` | string | yes | | Template expression resolving to ai.parse output, e.g. "{{ steps.parse.output }}" |
|
|
431
|
+
| `documentTypes` | array<object> | yes | | The document-type taxonomy. The LLM tags each detected document with one of these names, or the reserved "unknown" type when none fit. |
|
|
432
|
+
| `rules` | string | no | | Optional natural-language rules appended to the system prompt. |
|
|
433
|
+
| `provider` | string | no | | Provider ID from eigenpal.config.yaml. Falls back to the tenant default LLM provider when omitted. |
|
|
434
|
+
| `windowTokenBudget` | integer | no | | Override the per-window token ceiling. Defaults to env SPLIT_WINDOW_TOKEN_BUDGET or 20000. |
|
|
435
|
+
|
|
436
|
+
**Output:** `object`
|
|
437
|
+
|
|
438
|
+
| Field | Type | Required | Default | Description |
|
|
439
|
+
| --- | --- | --- | --- | --- |
|
|
440
|
+
| `summary` | object | yes | | Batch-level escalation summary: labelled vs unlabelled counts + docs needing review. |
|
|
441
|
+
| `documents` | array<object> | yes | | Documents discovered in the batch, in page order. The full batch is covered. |
|
|
442
|
+
|
|
418
443
|
#### `ai.classify` — Classify
|
|
419
444
|
|
|
420
445
|
Classify a document or text into one of a fixed label set using an LLM. Output exposes the picked label (constrained to the configured names), a coarse confidence, and a short justification. Pair with control.fail to reject documents that match an undesired label.
|
|
@@ -732,6 +757,25 @@ Branch execution based on a condition expression
|
|
|
732
757
|
| `branch` | `"then"` \| `"else"` | yes | | Which branch was executed |
|
|
733
758
|
| `result` | unknown | yes | | Output from executed branch |
|
|
734
759
|
|
|
760
|
+
#### `control.switch` — Switch
|
|
761
|
+
|
|
762
|
+
Multi-way routing: resolve an expression and run the first case whose value matches (else default). Cleaner than a nested control.if chain for routing an item to one of N pipelines by a discriminator field like a document type.
|
|
763
|
+
|
|
764
|
+
**Config** (at step level):
|
|
765
|
+
|
|
766
|
+
| Field | Type | Required | Default | Description |
|
|
767
|
+
| --- | --- | --- | --- | --- |
|
|
768
|
+
| `on` | string | yes | | Template expression whose resolved value selects the case, e.g. "{{ doc.type }}". |
|
|
769
|
+
| `cases` | array<object> | yes | | Ordered cases; the first whose `when` matches runs. Each case has its own `steps`. |
|
|
770
|
+
|
|
771
|
+
**Output:** `object`
|
|
772
|
+
|
|
773
|
+
| Field | Type | Required | Default | Description |
|
|
774
|
+
| --- | --- | --- | --- | --- |
|
|
775
|
+
| `matched` | string \| number \| boolean \| null | yes | | The `when` value that matched, or null when the default (or no) branch ran. |
|
|
776
|
+
| `branch` | `"case"` \| `"default"` \| `"none"` | yes | | Which branch executed. |
|
|
777
|
+
| `result` | unknown | yes | | Output from the executed branch (its last step). |
|
|
778
|
+
|
|
735
779
|
#### `control.foreach` — For Each
|
|
736
780
|
|
|
737
781
|
Loop over an array and execute steps for each item
|