@eigenpal/cli 0.14.2 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +17 -0
- package/README.md +11 -9
- package/dist/cli.js +67772 -83895
- package/dist/docs-bundle.json +1349 -0
- package/dist/skill/SKILL.md +33 -0
- package/dist/skill/reference/cli/agents.md +111 -106
- package/dist/skill/reference/cli/docs.md +84 -0
- package/dist/skill/reference/cli/models.md +1 -1
- package/dist/skill/reference/cli/rerun.md +1 -1
- package/dist/skill/reference/cli/run.md +1 -1
- package/dist/skill/reference/cli/runs.md +67 -67
- package/dist/skill/reference/cli/surface.md +4 -0
- package/dist/skill/reference/cli/workflow.md +114 -107
- package/dist/skill/reference/dataset-format.md +151 -0
- package/dist/skill/reference/debugging.md +9 -0
- package/dist/skill/reference/evaluators.md +362 -4
- package/dist/skill/reference/step-types.md +4528 -15
- package/dist/skill/reference/workflow-yaml.md +1458 -23
- package/package.json +7 -4
|
@@ -414,6 +414,8 @@ _Generated from `STEP_SCHEMAS` in `@eigenpal/types/src/workflow/step-configs.ts`
|
|
|
414
414
|
|
|
415
415
|
Extract text from documents (PDF, DOCX, images) using native extraction, OCR, or vision models
|
|
416
416
|
|
|
417
|
+
**Behavior and examples:** `eigenpal docs read steps/ai/parse`
|
|
418
|
+
|
|
417
419
|
**Durable retry:** Provider request retries are separate; the workflow engine does not durably retry this step.
|
|
418
420
|
|
|
419
421
|
**Config** (in `step.with`):
|
|
@@ -424,7 +426,9 @@ Extract text from documents (PDF, DOCX, images) using native extraction, OCR, or
|
|
|
424
426
|
| `parseMode` | `"ocr"` \| `"vision"` \| `"native"` \| `"native-or-ocr"` | no | | Base parser for PDF/image inputs. OCR is the default; vision uses the selected LLM. `native` extracts PDF embedded text only and never calls OCR or vision. `native-or-ocr` extracts native text, runs page-quality diagnostics, and requests OCR when pages have detectable anomalies (empty, U+FFFD, lone surrogates, forbidden controls, unassigned/noncharacter, heavy PUA). Page selection, subset egress, and billing are provider-dependent. Valid-looking wrong text and literal "?" are not flagged. Text and Office files always use local parsing. |
|
|
425
427
|
| `ocrModel` | string | no | | OCR provider ID for PDF/image parsing |
|
|
426
428
|
| `llmModel` | string | no | | LLM provider ID for vision-based parsing |
|
|
429
|
+
| `llmReasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
427
430
|
| `figureModel` | string | no | | Vision model used only for the optional figure-description pass |
|
|
431
|
+
| `figureReasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
428
432
|
| `maxConcurrency` | number | no | `3` | Max concurrent VLM batch requests |
|
|
429
433
|
| `pagesPerBatch` | number | no | `5` | Number of page images per VLM request |
|
|
430
434
|
| `pdfRenderScale` | number | no | `1` | Scale factor for rendering PDF pages before VLM parsing. Higher values produce sharper images at larger payload sizes. |
|
|
@@ -450,10 +454,931 @@ Extract text from documents (PDF, DOCX, images) using native extraction, OCR, or
|
|
|
450
454
|
| `processingStrategy` | `"native"` \| `"ocr"` \| `"vision"` \| `"hybrid"` | no | | How this result was produced: `native` (local PDF text only), `ocr`, `vision`, or `hybrid` (native pages merged with OCR on fallback pages). |
|
|
451
455
|
| `structured` | record<string, unknown> | no | | Canonical structured document with ordered blocks, regions, bounding boxes, tables, figures, and chunks when supported by the parser |
|
|
452
456
|
|
|
457
|
+
##### Complete machine-readable schemas
|
|
458
|
+
|
|
459
|
+
Config schema:
|
|
460
|
+
|
|
461
|
+
```json
|
|
462
|
+
{
|
|
463
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
464
|
+
"type": "object",
|
|
465
|
+
"properties": {
|
|
466
|
+
"input": {
|
|
467
|
+
"type": "string",
|
|
468
|
+
"description": "Storage reference or template expression for the document"
|
|
469
|
+
},
|
|
470
|
+
"parseMode": {
|
|
471
|
+
"description": "Base parser for PDF/image inputs. OCR is the default; vision uses the selected LLM. `native` extracts PDF embedded text only and never calls OCR or vision. `native-or-ocr` extracts native text, runs page-quality diagnostics, and requests OCR when pages have detectable anomalies (empty, U+FFFD, lone surrogates, forbidden controls, unassigned/noncharacter, heavy PUA). Page selection, subset egress, and billing are provider-dependent. Valid-looking wrong text and literal \"?\" are not flagged. Text and Office files always use local parsing.",
|
|
472
|
+
"type": "string",
|
|
473
|
+
"enum": [
|
|
474
|
+
"ocr",
|
|
475
|
+
"vision",
|
|
476
|
+
"native",
|
|
477
|
+
"native-or-ocr"
|
|
478
|
+
]
|
|
479
|
+
},
|
|
480
|
+
"ocrModel": {
|
|
481
|
+
"description": "OCR provider ID for PDF/image parsing",
|
|
482
|
+
"type": "string"
|
|
483
|
+
},
|
|
484
|
+
"llmModel": {
|
|
485
|
+
"description": "LLM provider ID for vision-based parsing",
|
|
486
|
+
"type": "string"
|
|
487
|
+
},
|
|
488
|
+
"llmReasoningEffort": {
|
|
489
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
490
|
+
"type": "string",
|
|
491
|
+
"enum": [
|
|
492
|
+
"none",
|
|
493
|
+
"minimal",
|
|
494
|
+
"low",
|
|
495
|
+
"medium",
|
|
496
|
+
"high",
|
|
497
|
+
"xhigh",
|
|
498
|
+
"max"
|
|
499
|
+
]
|
|
500
|
+
},
|
|
501
|
+
"figureModel": {
|
|
502
|
+
"description": "Vision model used only for the optional figure-description pass",
|
|
503
|
+
"type": "string"
|
|
504
|
+
},
|
|
505
|
+
"figureReasoningEffort": {
|
|
506
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
507
|
+
"type": "string",
|
|
508
|
+
"enum": [
|
|
509
|
+
"none",
|
|
510
|
+
"minimal",
|
|
511
|
+
"low",
|
|
512
|
+
"medium",
|
|
513
|
+
"high",
|
|
514
|
+
"xhigh",
|
|
515
|
+
"max"
|
|
516
|
+
]
|
|
517
|
+
},
|
|
518
|
+
"maxConcurrency": {
|
|
519
|
+
"description": "Max concurrent VLM batch requests",
|
|
520
|
+
"default": 3,
|
|
521
|
+
"type": "number",
|
|
522
|
+
"minimum": 1,
|
|
523
|
+
"maximum": 10
|
|
524
|
+
},
|
|
525
|
+
"pagesPerBatch": {
|
|
526
|
+
"description": "Number of page images per VLM request",
|
|
527
|
+
"default": 5,
|
|
528
|
+
"type": "number",
|
|
529
|
+
"minimum": 1,
|
|
530
|
+
"maximum": 20
|
|
531
|
+
},
|
|
532
|
+
"pdfRenderScale": {
|
|
533
|
+
"description": "Scale factor for rendering PDF pages before VLM parsing. Higher values produce sharper images at larger payload sizes.",
|
|
534
|
+
"default": 1,
|
|
535
|
+
"type": "number",
|
|
536
|
+
"minimum": 1,
|
|
537
|
+
"maximum": 4
|
|
538
|
+
},
|
|
539
|
+
"imageQuality": {
|
|
540
|
+
"description": "JPEG quality for rendered PDF page images sent to VLM parsing. Higher values reduce compression artifacts at larger payload sizes.",
|
|
541
|
+
"default": 85,
|
|
542
|
+
"type": "integer",
|
|
543
|
+
"minimum": 1,
|
|
544
|
+
"maximum": 100
|
|
545
|
+
},
|
|
546
|
+
"prompt": {
|
|
547
|
+
"description": "Custom extraction prompt",
|
|
548
|
+
"type": "string"
|
|
549
|
+
},
|
|
550
|
+
"languages": {
|
|
551
|
+
"description": "OCR language hints",
|
|
552
|
+
"type": "array",
|
|
553
|
+
"items": {
|
|
554
|
+
"type": "string"
|
|
555
|
+
}
|
|
556
|
+
},
|
|
557
|
+
"outputFormat": {
|
|
558
|
+
"description": "Format for extracted text. `markdown` (default) keeps structure; `plain` is unstyled text; `djot`/`html` preserve more markup. `layout` is native-PDF spatial text with column gaps preserved as spaces. It requires `parseMode: native` or `parseMode: native-or-ocr` and a PDF — omitted parseMode and `nativeText: true` are not enough. Native mode is layout-only and fail-closed. `native-or-ocr` keeps layout text on accepted native pages and uses OCR markdown on suspect or empty pages, so the document can mix fixed-width layout and markdown. Extractor failure never falls back to another parser. Office, plaintext, and images fail. OCR/vision cannot request `layout`.",
|
|
559
|
+
"default": "markdown",
|
|
560
|
+
"type": "string",
|
|
561
|
+
"enum": [
|
|
562
|
+
"plain",
|
|
563
|
+
"markdown",
|
|
564
|
+
"djot",
|
|
565
|
+
"html",
|
|
566
|
+
"layout"
|
|
567
|
+
]
|
|
568
|
+
},
|
|
569
|
+
"nativeText": {
|
|
570
|
+
"description": "Extract native/embedded text from PDFs without OCR/VLM. Faster and uses no credits. Falls back to OCR/VLM if the PDF has no embedded text.",
|
|
571
|
+
"default": false,
|
|
572
|
+
"type": "boolean"
|
|
573
|
+
},
|
|
574
|
+
"describeFigures": {
|
|
575
|
+
"description": "Opt-in (default off). After text extraction, detect which pages contain figures with an in-worker layout model, then caption those pages with a vision model and append `<figure>description</figure>` to their text — so image-only pages (property photos, signatures, charts) become findable by text-based steps like ai.split. Note: the layout scan runs over all pages, and the caption step and its vision calls are billed. Skipped for plaintext.",
|
|
576
|
+
"type": "boolean"
|
|
577
|
+
},
|
|
578
|
+
"figureInstructions": {
|
|
579
|
+
"description": "Custom instruction for the figure-description pass, e.g. \"Describe each figure; label a handwritten signature as `<figure>signature</figure>` and a stamp as `<figure>stamp</figure>`; for property photos note the room or exterior shown.\" Applied only when describeFigures runs.",
|
|
580
|
+
"type": "string"
|
|
581
|
+
}
|
|
582
|
+
},
|
|
583
|
+
"required": [
|
|
584
|
+
"input"
|
|
585
|
+
],
|
|
586
|
+
"additionalProperties": false
|
|
587
|
+
}
|
|
588
|
+
```
|
|
589
|
+
|
|
590
|
+
|
|
591
|
+
Output schema:
|
|
592
|
+
|
|
593
|
+
```json
|
|
594
|
+
{
|
|
595
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
596
|
+
"type": "object",
|
|
597
|
+
"properties": {
|
|
598
|
+
"document": {
|
|
599
|
+
"type": "object",
|
|
600
|
+
"properties": {
|
|
601
|
+
"filename": {
|
|
602
|
+
"type": "string"
|
|
603
|
+
},
|
|
604
|
+
"mimeType": {
|
|
605
|
+
"type": "string"
|
|
606
|
+
},
|
|
607
|
+
"size": {
|
|
608
|
+
"type": "number",
|
|
609
|
+
"description": "File size in bytes"
|
|
610
|
+
},
|
|
611
|
+
"storageRef": {
|
|
612
|
+
"description": "Storage reference for the original file",
|
|
613
|
+
"type": "string"
|
|
614
|
+
}
|
|
615
|
+
},
|
|
616
|
+
"required": [
|
|
617
|
+
"filename",
|
|
618
|
+
"mimeType",
|
|
619
|
+
"size"
|
|
620
|
+
],
|
|
621
|
+
"additionalProperties": false
|
|
622
|
+
},
|
|
623
|
+
"usage": {
|
|
624
|
+
"type": "object",
|
|
625
|
+
"properties": {
|
|
626
|
+
"pageCount": {
|
|
627
|
+
"type": "number"
|
|
628
|
+
},
|
|
629
|
+
"processingTimeMs": {
|
|
630
|
+
"type": "number"
|
|
631
|
+
},
|
|
632
|
+
"ocrPagesProcessed": {
|
|
633
|
+
"type": "integer",
|
|
634
|
+
"minimum": 0,
|
|
635
|
+
"maximum": 9007199254740991
|
|
636
|
+
}
|
|
637
|
+
},
|
|
638
|
+
"required": [
|
|
639
|
+
"pageCount"
|
|
640
|
+
],
|
|
641
|
+
"additionalProperties": false
|
|
642
|
+
},
|
|
643
|
+
"pages": {
|
|
644
|
+
"type": "array",
|
|
645
|
+
"items": {
|
|
646
|
+
"type": "object",
|
|
647
|
+
"properties": {
|
|
648
|
+
"pageIndex": {
|
|
649
|
+
"type": "number",
|
|
650
|
+
"description": "0-based page index"
|
|
651
|
+
},
|
|
652
|
+
"text": {
|
|
653
|
+
"type": "string",
|
|
654
|
+
"description": "Extracted page text (markdown/HTML/plain, or spatial layout text)"
|
|
655
|
+
},
|
|
656
|
+
"pageName": {
|
|
657
|
+
"description": "Page/sheet name (e.g., Excel sheet name)",
|
|
658
|
+
"type": "string"
|
|
659
|
+
},
|
|
660
|
+
"layoutElements": {
|
|
661
|
+
"description": "Semantic layout elements in reading order",
|
|
662
|
+
"type": "array",
|
|
663
|
+
"items": {
|
|
664
|
+
"type": "object",
|
|
665
|
+
"properties": {
|
|
666
|
+
"type": {
|
|
667
|
+
"type": "string",
|
|
668
|
+
"enum": [
|
|
669
|
+
"paragraph",
|
|
670
|
+
"title",
|
|
671
|
+
"sectionHeading",
|
|
672
|
+
"pageHeader",
|
|
673
|
+
"pageFooter",
|
|
674
|
+
"pageNumber",
|
|
675
|
+
"footnote",
|
|
676
|
+
"table",
|
|
677
|
+
"figure",
|
|
678
|
+
"formulaBlock"
|
|
679
|
+
]
|
|
680
|
+
},
|
|
681
|
+
"content": {
|
|
682
|
+
"type": "string"
|
|
683
|
+
},
|
|
684
|
+
"table": {
|
|
685
|
+
"type": "object",
|
|
686
|
+
"properties": {
|
|
687
|
+
"cells": {
|
|
688
|
+
"type": "array",
|
|
689
|
+
"items": {
|
|
690
|
+
"type": "object",
|
|
691
|
+
"properties": {
|
|
692
|
+
"content": {
|
|
693
|
+
"type": "string"
|
|
694
|
+
},
|
|
695
|
+
"rowIndex": {
|
|
696
|
+
"type": "number"
|
|
697
|
+
},
|
|
698
|
+
"columnIndex": {
|
|
699
|
+
"type": "number"
|
|
700
|
+
},
|
|
701
|
+
"rowSpan": {
|
|
702
|
+
"default": 1,
|
|
703
|
+
"type": "number"
|
|
704
|
+
},
|
|
705
|
+
"columnSpan": {
|
|
706
|
+
"default": 1,
|
|
707
|
+
"type": "number"
|
|
708
|
+
},
|
|
709
|
+
"boundingRegion": {
|
|
710
|
+
"type": "object",
|
|
711
|
+
"properties": {
|
|
712
|
+
"polygon": {
|
|
713
|
+
"minItems": 3,
|
|
714
|
+
"type": "array",
|
|
715
|
+
"items": {
|
|
716
|
+
"type": "object",
|
|
717
|
+
"properties": {
|
|
718
|
+
"x": {
|
|
719
|
+
"type": "number"
|
|
720
|
+
},
|
|
721
|
+
"y": {
|
|
722
|
+
"type": "number"
|
|
723
|
+
}
|
|
724
|
+
},
|
|
725
|
+
"required": [
|
|
726
|
+
"x",
|
|
727
|
+
"y"
|
|
728
|
+
],
|
|
729
|
+
"additionalProperties": false
|
|
730
|
+
}
|
|
731
|
+
},
|
|
732
|
+
"unit": {
|
|
733
|
+
"type": "string",
|
|
734
|
+
"enum": [
|
|
735
|
+
"pixel",
|
|
736
|
+
"inch",
|
|
737
|
+
"point",
|
|
738
|
+
"normalized"
|
|
739
|
+
]
|
|
740
|
+
},
|
|
741
|
+
"pageIndex": {
|
|
742
|
+
"type": "number"
|
|
743
|
+
}
|
|
744
|
+
},
|
|
745
|
+
"required": [
|
|
746
|
+
"polygon",
|
|
747
|
+
"unit",
|
|
748
|
+
"pageIndex"
|
|
749
|
+
],
|
|
750
|
+
"additionalProperties": false
|
|
751
|
+
}
|
|
752
|
+
},
|
|
753
|
+
"required": [
|
|
754
|
+
"content",
|
|
755
|
+
"rowIndex",
|
|
756
|
+
"columnIndex",
|
|
757
|
+
"rowSpan",
|
|
758
|
+
"columnSpan"
|
|
759
|
+
],
|
|
760
|
+
"additionalProperties": false
|
|
761
|
+
}
|
|
762
|
+
},
|
|
763
|
+
"rowCount": {
|
|
764
|
+
"type": "number"
|
|
765
|
+
},
|
|
766
|
+
"columnCount": {
|
|
767
|
+
"type": "number"
|
|
768
|
+
},
|
|
769
|
+
"boundingRegion": {
|
|
770
|
+
"type": "object",
|
|
771
|
+
"properties": {
|
|
772
|
+
"polygon": {
|
|
773
|
+
"minItems": 3,
|
|
774
|
+
"type": "array",
|
|
775
|
+
"items": {
|
|
776
|
+
"type": "object",
|
|
777
|
+
"properties": {
|
|
778
|
+
"x": {
|
|
779
|
+
"type": "number"
|
|
780
|
+
},
|
|
781
|
+
"y": {
|
|
782
|
+
"type": "number"
|
|
783
|
+
}
|
|
784
|
+
},
|
|
785
|
+
"required": [
|
|
786
|
+
"x",
|
|
787
|
+
"y"
|
|
788
|
+
],
|
|
789
|
+
"additionalProperties": false
|
|
790
|
+
}
|
|
791
|
+
},
|
|
792
|
+
"unit": {
|
|
793
|
+
"type": "string",
|
|
794
|
+
"enum": [
|
|
795
|
+
"pixel",
|
|
796
|
+
"inch",
|
|
797
|
+
"point",
|
|
798
|
+
"normalized"
|
|
799
|
+
]
|
|
800
|
+
},
|
|
801
|
+
"pageIndex": {
|
|
802
|
+
"type": "number"
|
|
803
|
+
}
|
|
804
|
+
},
|
|
805
|
+
"required": [
|
|
806
|
+
"polygon",
|
|
807
|
+
"unit",
|
|
808
|
+
"pageIndex"
|
|
809
|
+
],
|
|
810
|
+
"additionalProperties": false
|
|
811
|
+
}
|
|
812
|
+
},
|
|
813
|
+
"required": [
|
|
814
|
+
"cells",
|
|
815
|
+
"rowCount",
|
|
816
|
+
"columnCount"
|
|
817
|
+
],
|
|
818
|
+
"additionalProperties": false
|
|
819
|
+
},
|
|
820
|
+
"caption": {
|
|
821
|
+
"type": "string"
|
|
822
|
+
},
|
|
823
|
+
"figureId": {
|
|
824
|
+
"type": "string"
|
|
825
|
+
},
|
|
826
|
+
"boundingRegion": {
|
|
827
|
+
"type": "object",
|
|
828
|
+
"properties": {
|
|
829
|
+
"polygon": {
|
|
830
|
+
"minItems": 3,
|
|
831
|
+
"type": "array",
|
|
832
|
+
"items": {
|
|
833
|
+
"type": "object",
|
|
834
|
+
"properties": {
|
|
835
|
+
"x": {
|
|
836
|
+
"type": "number"
|
|
837
|
+
},
|
|
838
|
+
"y": {
|
|
839
|
+
"type": "number"
|
|
840
|
+
}
|
|
841
|
+
},
|
|
842
|
+
"required": [
|
|
843
|
+
"x",
|
|
844
|
+
"y"
|
|
845
|
+
],
|
|
846
|
+
"additionalProperties": false
|
|
847
|
+
}
|
|
848
|
+
},
|
|
849
|
+
"unit": {
|
|
850
|
+
"type": "string",
|
|
851
|
+
"enum": [
|
|
852
|
+
"pixel",
|
|
853
|
+
"inch",
|
|
854
|
+
"point",
|
|
855
|
+
"normalized"
|
|
856
|
+
]
|
|
857
|
+
},
|
|
858
|
+
"pageIndex": {
|
|
859
|
+
"type": "number"
|
|
860
|
+
}
|
|
861
|
+
},
|
|
862
|
+
"required": [
|
|
863
|
+
"polygon",
|
|
864
|
+
"unit",
|
|
865
|
+
"pageIndex"
|
|
866
|
+
],
|
|
867
|
+
"additionalProperties": false
|
|
868
|
+
}
|
|
869
|
+
},
|
|
870
|
+
"required": [
|
|
871
|
+
"type",
|
|
872
|
+
"content"
|
|
873
|
+
],
|
|
874
|
+
"additionalProperties": false
|
|
875
|
+
}
|
|
876
|
+
},
|
|
877
|
+
"words": {
|
|
878
|
+
"description": "Word-level positions",
|
|
879
|
+
"type": "array",
|
|
880
|
+
"items": {
|
|
881
|
+
"type": "object",
|
|
882
|
+
"properties": {
|
|
883
|
+
"text": {
|
|
884
|
+
"type": "string"
|
|
885
|
+
},
|
|
886
|
+
"confidence": {
|
|
887
|
+
"type": "number",
|
|
888
|
+
"minimum": 0,
|
|
889
|
+
"maximum": 1
|
|
890
|
+
},
|
|
891
|
+
"boundingRegion": {
|
|
892
|
+
"type": "object",
|
|
893
|
+
"properties": {
|
|
894
|
+
"polygon": {
|
|
895
|
+
"minItems": 3,
|
|
896
|
+
"type": "array",
|
|
897
|
+
"items": {
|
|
898
|
+
"type": "object",
|
|
899
|
+
"properties": {
|
|
900
|
+
"x": {
|
|
901
|
+
"type": "number"
|
|
902
|
+
},
|
|
903
|
+
"y": {
|
|
904
|
+
"type": "number"
|
|
905
|
+
}
|
|
906
|
+
},
|
|
907
|
+
"required": [
|
|
908
|
+
"x",
|
|
909
|
+
"y"
|
|
910
|
+
],
|
|
911
|
+
"additionalProperties": false
|
|
912
|
+
}
|
|
913
|
+
},
|
|
914
|
+
"unit": {
|
|
915
|
+
"type": "string",
|
|
916
|
+
"enum": [
|
|
917
|
+
"pixel",
|
|
918
|
+
"inch",
|
|
919
|
+
"point",
|
|
920
|
+
"normalized"
|
|
921
|
+
]
|
|
922
|
+
},
|
|
923
|
+
"pageIndex": {
|
|
924
|
+
"type": "number"
|
|
925
|
+
}
|
|
926
|
+
},
|
|
927
|
+
"required": [
|
|
928
|
+
"polygon",
|
|
929
|
+
"unit",
|
|
930
|
+
"pageIndex"
|
|
931
|
+
],
|
|
932
|
+
"additionalProperties": false
|
|
933
|
+
}
|
|
934
|
+
},
|
|
935
|
+
"required": [
|
|
936
|
+
"text"
|
|
937
|
+
],
|
|
938
|
+
"additionalProperties": false
|
|
939
|
+
}
|
|
940
|
+
},
|
|
941
|
+
"lines": {
|
|
942
|
+
"description": "Line-level positions",
|
|
943
|
+
"type": "array",
|
|
944
|
+
"items": {
|
|
945
|
+
"type": "object",
|
|
946
|
+
"properties": {
|
|
947
|
+
"text": {
|
|
948
|
+
"type": "string"
|
|
949
|
+
},
|
|
950
|
+
"confidence": {
|
|
951
|
+
"type": "number",
|
|
952
|
+
"minimum": 0,
|
|
953
|
+
"maximum": 1
|
|
954
|
+
},
|
|
955
|
+
"boundingRegion": {
|
|
956
|
+
"type": "object",
|
|
957
|
+
"properties": {
|
|
958
|
+
"polygon": {
|
|
959
|
+
"minItems": 3,
|
|
960
|
+
"type": "array",
|
|
961
|
+
"items": {
|
|
962
|
+
"type": "object",
|
|
963
|
+
"properties": {
|
|
964
|
+
"x": {
|
|
965
|
+
"type": "number"
|
|
966
|
+
},
|
|
967
|
+
"y": {
|
|
968
|
+
"type": "number"
|
|
969
|
+
}
|
|
970
|
+
},
|
|
971
|
+
"required": [
|
|
972
|
+
"x",
|
|
973
|
+
"y"
|
|
974
|
+
],
|
|
975
|
+
"additionalProperties": false
|
|
976
|
+
}
|
|
977
|
+
},
|
|
978
|
+
"unit": {
|
|
979
|
+
"type": "string",
|
|
980
|
+
"enum": [
|
|
981
|
+
"pixel",
|
|
982
|
+
"inch",
|
|
983
|
+
"point",
|
|
984
|
+
"normalized"
|
|
985
|
+
]
|
|
986
|
+
},
|
|
987
|
+
"pageIndex": {
|
|
988
|
+
"type": "number"
|
|
989
|
+
}
|
|
990
|
+
},
|
|
991
|
+
"required": [
|
|
992
|
+
"polygon",
|
|
993
|
+
"unit",
|
|
994
|
+
"pageIndex"
|
|
995
|
+
],
|
|
996
|
+
"additionalProperties": false
|
|
997
|
+
}
|
|
998
|
+
},
|
|
999
|
+
"required": [
|
|
1000
|
+
"text"
|
|
1001
|
+
],
|
|
1002
|
+
"additionalProperties": false
|
|
1003
|
+
}
|
|
1004
|
+
},
|
|
1005
|
+
"tables": {
|
|
1006
|
+
"description": "Extracted tables",
|
|
1007
|
+
"type": "array",
|
|
1008
|
+
"items": {
|
|
1009
|
+
"type": "object",
|
|
1010
|
+
"properties": {
|
|
1011
|
+
"cells": {
|
|
1012
|
+
"type": "array",
|
|
1013
|
+
"items": {
|
|
1014
|
+
"type": "object",
|
|
1015
|
+
"properties": {
|
|
1016
|
+
"content": {
|
|
1017
|
+
"type": "string"
|
|
1018
|
+
},
|
|
1019
|
+
"rowIndex": {
|
|
1020
|
+
"type": "number"
|
|
1021
|
+
},
|
|
1022
|
+
"columnIndex": {
|
|
1023
|
+
"type": "number"
|
|
1024
|
+
},
|
|
1025
|
+
"rowSpan": {
|
|
1026
|
+
"default": 1,
|
|
1027
|
+
"type": "number"
|
|
1028
|
+
},
|
|
1029
|
+
"columnSpan": {
|
|
1030
|
+
"default": 1,
|
|
1031
|
+
"type": "number"
|
|
1032
|
+
},
|
|
1033
|
+
"boundingRegion": {
|
|
1034
|
+
"type": "object",
|
|
1035
|
+
"properties": {
|
|
1036
|
+
"polygon": {
|
|
1037
|
+
"minItems": 3,
|
|
1038
|
+
"type": "array",
|
|
1039
|
+
"items": {
|
|
1040
|
+
"type": "object",
|
|
1041
|
+
"properties": {
|
|
1042
|
+
"x": {
|
|
1043
|
+
"type": "number"
|
|
1044
|
+
},
|
|
1045
|
+
"y": {
|
|
1046
|
+
"type": "number"
|
|
1047
|
+
}
|
|
1048
|
+
},
|
|
1049
|
+
"required": [
|
|
1050
|
+
"x",
|
|
1051
|
+
"y"
|
|
1052
|
+
],
|
|
1053
|
+
"additionalProperties": false
|
|
1054
|
+
}
|
|
1055
|
+
},
|
|
1056
|
+
"unit": {
|
|
1057
|
+
"type": "string",
|
|
1058
|
+
"enum": [
|
|
1059
|
+
"pixel",
|
|
1060
|
+
"inch",
|
|
1061
|
+
"point",
|
|
1062
|
+
"normalized"
|
|
1063
|
+
]
|
|
1064
|
+
},
|
|
1065
|
+
"pageIndex": {
|
|
1066
|
+
"type": "number"
|
|
1067
|
+
}
|
|
1068
|
+
},
|
|
1069
|
+
"required": [
|
|
1070
|
+
"polygon",
|
|
1071
|
+
"unit",
|
|
1072
|
+
"pageIndex"
|
|
1073
|
+
],
|
|
1074
|
+
"additionalProperties": false
|
|
1075
|
+
}
|
|
1076
|
+
},
|
|
1077
|
+
"required": [
|
|
1078
|
+
"content",
|
|
1079
|
+
"rowIndex",
|
|
1080
|
+
"columnIndex",
|
|
1081
|
+
"rowSpan",
|
|
1082
|
+
"columnSpan"
|
|
1083
|
+
],
|
|
1084
|
+
"additionalProperties": false
|
|
1085
|
+
}
|
|
1086
|
+
},
|
|
1087
|
+
"rowCount": {
|
|
1088
|
+
"type": "number"
|
|
1089
|
+
},
|
|
1090
|
+
"columnCount": {
|
|
1091
|
+
"type": "number"
|
|
1092
|
+
},
|
|
1093
|
+
"boundingRegion": {
|
|
1094
|
+
"type": "object",
|
|
1095
|
+
"properties": {
|
|
1096
|
+
"polygon": {
|
|
1097
|
+
"minItems": 3,
|
|
1098
|
+
"type": "array",
|
|
1099
|
+
"items": {
|
|
1100
|
+
"type": "object",
|
|
1101
|
+
"properties": {
|
|
1102
|
+
"x": {
|
|
1103
|
+
"type": "number"
|
|
1104
|
+
},
|
|
1105
|
+
"y": {
|
|
1106
|
+
"type": "number"
|
|
1107
|
+
}
|
|
1108
|
+
},
|
|
1109
|
+
"required": [
|
|
1110
|
+
"x",
|
|
1111
|
+
"y"
|
|
1112
|
+
],
|
|
1113
|
+
"additionalProperties": false
|
|
1114
|
+
}
|
|
1115
|
+
},
|
|
1116
|
+
"unit": {
|
|
1117
|
+
"type": "string",
|
|
1118
|
+
"enum": [
|
|
1119
|
+
"pixel",
|
|
1120
|
+
"inch",
|
|
1121
|
+
"point",
|
|
1122
|
+
"normalized"
|
|
1123
|
+
]
|
|
1124
|
+
},
|
|
1125
|
+
"pageIndex": {
|
|
1126
|
+
"type": "number"
|
|
1127
|
+
}
|
|
1128
|
+
},
|
|
1129
|
+
"required": [
|
|
1130
|
+
"polygon",
|
|
1131
|
+
"unit",
|
|
1132
|
+
"pageIndex"
|
|
1133
|
+
],
|
|
1134
|
+
"additionalProperties": false
|
|
1135
|
+
}
|
|
1136
|
+
},
|
|
1137
|
+
"required": [
|
|
1138
|
+
"cells",
|
|
1139
|
+
"rowCount",
|
|
1140
|
+
"columnCount"
|
|
1141
|
+
],
|
|
1142
|
+
"additionalProperties": false
|
|
1143
|
+
}
|
|
1144
|
+
},
|
|
1145
|
+
"width": {
|
|
1146
|
+
"description": "Page width",
|
|
1147
|
+
"type": "number"
|
|
1148
|
+
},
|
|
1149
|
+
"height": {
|
|
1150
|
+
"description": "Page height",
|
|
1151
|
+
"type": "number"
|
|
1152
|
+
},
|
|
1153
|
+
"unit": {
|
|
1154
|
+
"description": "Unit for width/height and bounding regions",
|
|
1155
|
+
"type": "string",
|
|
1156
|
+
"enum": [
|
|
1157
|
+
"pixel",
|
|
1158
|
+
"inch",
|
|
1159
|
+
"point",
|
|
1160
|
+
"normalized"
|
|
1161
|
+
]
|
|
1162
|
+
},
|
|
1163
|
+
"confidence": {
|
|
1164
|
+
"description": "Overall page confidence",
|
|
1165
|
+
"type": "number",
|
|
1166
|
+
"minimum": 0,
|
|
1167
|
+
"maximum": 1
|
|
1168
|
+
},
|
|
1169
|
+
"nativeTextQuality": {
|
|
1170
|
+
"description": "Per-page diagnosis of native-extracted text. Reports objectively detectable anomalies (empty, U+FFFD, forbidden controls, unassigned/noncharacter code points, heavy private-use). Does not certify semantic correctness — valid-looking wrong mappings and literal \"?\" cannot be distinguished from legitimate text.",
|
|
1171
|
+
"type": "object",
|
|
1172
|
+
"properties": {
|
|
1173
|
+
"status": {
|
|
1174
|
+
"type": "string",
|
|
1175
|
+
"enum": [
|
|
1176
|
+
"clean",
|
|
1177
|
+
"empty",
|
|
1178
|
+
"suspect"
|
|
1179
|
+
]
|
|
1180
|
+
},
|
|
1181
|
+
"reasons": {
|
|
1182
|
+
"type": "array",
|
|
1183
|
+
"items": {
|
|
1184
|
+
"type": "string",
|
|
1185
|
+
"enum": [
|
|
1186
|
+
"empty",
|
|
1187
|
+
"replacement-character",
|
|
1188
|
+
"lone-surrogate",
|
|
1189
|
+
"forbidden-control",
|
|
1190
|
+
"unassigned",
|
|
1191
|
+
"noncharacter",
|
|
1192
|
+
"private-use"
|
|
1193
|
+
]
|
|
1194
|
+
}
|
|
1195
|
+
},
|
|
1196
|
+
"counts": {
|
|
1197
|
+
"type": "object",
|
|
1198
|
+
"properties": {
|
|
1199
|
+
"codePoints": {
|
|
1200
|
+
"type": "integer",
|
|
1201
|
+
"minimum": 0,
|
|
1202
|
+
"maximum": 9007199254740991
|
|
1203
|
+
},
|
|
1204
|
+
"nonWhitespace": {
|
|
1205
|
+
"type": "integer",
|
|
1206
|
+
"minimum": 0,
|
|
1207
|
+
"maximum": 9007199254740991
|
|
1208
|
+
},
|
|
1209
|
+
"replacement": {
|
|
1210
|
+
"type": "integer",
|
|
1211
|
+
"minimum": 0,
|
|
1212
|
+
"maximum": 9007199254740991
|
|
1213
|
+
},
|
|
1214
|
+
"loneSurrogate": {
|
|
1215
|
+
"type": "integer",
|
|
1216
|
+
"minimum": 0,
|
|
1217
|
+
"maximum": 9007199254740991
|
|
1218
|
+
},
|
|
1219
|
+
"forbiddenControl": {
|
|
1220
|
+
"type": "integer",
|
|
1221
|
+
"minimum": 0,
|
|
1222
|
+
"maximum": 9007199254740991
|
|
1223
|
+
},
|
|
1224
|
+
"unassigned": {
|
|
1225
|
+
"type": "integer",
|
|
1226
|
+
"minimum": 0,
|
|
1227
|
+
"maximum": 9007199254740991
|
|
1228
|
+
},
|
|
1229
|
+
"noncharacter": {
|
|
1230
|
+
"type": "integer",
|
|
1231
|
+
"minimum": 0,
|
|
1232
|
+
"maximum": 9007199254740991
|
|
1233
|
+
},
|
|
1234
|
+
"privateUse": {
|
|
1235
|
+
"type": "integer",
|
|
1236
|
+
"minimum": 0,
|
|
1237
|
+
"maximum": 9007199254740991
|
|
1238
|
+
}
|
|
1239
|
+
},
|
|
1240
|
+
"required": [
|
|
1241
|
+
"codePoints",
|
|
1242
|
+
"nonWhitespace",
|
|
1243
|
+
"replacement",
|
|
1244
|
+
"loneSurrogate",
|
|
1245
|
+
"forbiddenControl",
|
|
1246
|
+
"unassigned",
|
|
1247
|
+
"noncharacter",
|
|
1248
|
+
"privateUse"
|
|
1249
|
+
],
|
|
1250
|
+
"additionalProperties": false
|
|
1251
|
+
},
|
|
1252
|
+
"profile": {
|
|
1253
|
+
"type": "object",
|
|
1254
|
+
"properties": {
|
|
1255
|
+
"replacementRatio": {
|
|
1256
|
+
"type": "number",
|
|
1257
|
+
"minimum": 0
|
|
1258
|
+
},
|
|
1259
|
+
"loneSurrogateRatio": {
|
|
1260
|
+
"type": "number",
|
|
1261
|
+
"minimum": 0
|
|
1262
|
+
},
|
|
1263
|
+
"forbiddenControlRatio": {
|
|
1264
|
+
"type": "number",
|
|
1265
|
+
"minimum": 0
|
|
1266
|
+
},
|
|
1267
|
+
"unassignedRatio": {
|
|
1268
|
+
"type": "number",
|
|
1269
|
+
"minimum": 0
|
|
1270
|
+
},
|
|
1271
|
+
"noncharacterRatio": {
|
|
1272
|
+
"type": "number",
|
|
1273
|
+
"minimum": 0
|
|
1274
|
+
},
|
|
1275
|
+
"privateUseRatio": {
|
|
1276
|
+
"type": "number",
|
|
1277
|
+
"minimum": 0
|
|
1278
|
+
}
|
|
1279
|
+
},
|
|
1280
|
+
"required": [
|
|
1281
|
+
"replacementRatio",
|
|
1282
|
+
"loneSurrogateRatio",
|
|
1283
|
+
"forbiddenControlRatio",
|
|
1284
|
+
"unassignedRatio",
|
|
1285
|
+
"noncharacterRatio",
|
|
1286
|
+
"privateUseRatio"
|
|
1287
|
+
],
|
|
1288
|
+
"additionalProperties": false
|
|
1289
|
+
}
|
|
1290
|
+
},
|
|
1291
|
+
"required": [
|
|
1292
|
+
"status",
|
|
1293
|
+
"reasons",
|
|
1294
|
+
"counts",
|
|
1295
|
+
"profile"
|
|
1296
|
+
],
|
|
1297
|
+
"additionalProperties": false
|
|
1298
|
+
},
|
|
1299
|
+
"provenance": {
|
|
1300
|
+
"description": "Which extractor produced the final text/layout on this page",
|
|
1301
|
+
"type": "object",
|
|
1302
|
+
"properties": {
|
|
1303
|
+
"source": {
|
|
1304
|
+
"type": "string",
|
|
1305
|
+
"enum": [
|
|
1306
|
+
"native",
|
|
1307
|
+
"ocr",
|
|
1308
|
+
"vision"
|
|
1309
|
+
]
|
|
1310
|
+
}
|
|
1311
|
+
},
|
|
1312
|
+
"required": [
|
|
1313
|
+
"source"
|
|
1314
|
+
],
|
|
1315
|
+
"additionalProperties": false
|
|
1316
|
+
}
|
|
1317
|
+
},
|
|
1318
|
+
"required": [
|
|
1319
|
+
"pageIndex",
|
|
1320
|
+
"text"
|
|
1321
|
+
],
|
|
1322
|
+
"additionalProperties": false
|
|
1323
|
+
}
|
|
1324
|
+
},
|
|
1325
|
+
"text": {
|
|
1326
|
+
"type": "string",
|
|
1327
|
+
"description": "Combined text from all pages"
|
|
1328
|
+
},
|
|
1329
|
+
"parserType": {
|
|
1330
|
+
"type": "string",
|
|
1331
|
+
"enum": [
|
|
1332
|
+
"plaintext",
|
|
1333
|
+
"office",
|
|
1334
|
+
"llm-vision",
|
|
1335
|
+
"ocr"
|
|
1336
|
+
]
|
|
1337
|
+
},
|
|
1338
|
+
"parserVersion": {
|
|
1339
|
+
"type": "string"
|
|
1340
|
+
},
|
|
1341
|
+
"model": {
|
|
1342
|
+
"description": "Model used (for LLM/OCR parsers)",
|
|
1343
|
+
"type": "string"
|
|
1344
|
+
},
|
|
1345
|
+
"processingStrategy": {
|
|
1346
|
+
"description": "How this result was produced: `native` (local PDF text only), `ocr`, `vision`, or `hybrid` (native pages merged with OCR on fallback pages).",
|
|
1347
|
+
"type": "string",
|
|
1348
|
+
"enum": [
|
|
1349
|
+
"native",
|
|
1350
|
+
"ocr",
|
|
1351
|
+
"vision",
|
|
1352
|
+
"hybrid"
|
|
1353
|
+
]
|
|
1354
|
+
},
|
|
1355
|
+
"structured": {
|
|
1356
|
+
"description": "Canonical structured document with ordered blocks, regions, bounding boxes, tables, figures, and chunks when supported by the parser",
|
|
1357
|
+
"type": "object",
|
|
1358
|
+
"propertyNames": {
|
|
1359
|
+
"type": "string"
|
|
1360
|
+
},
|
|
1361
|
+
"additionalProperties": {}
|
|
1362
|
+
}
|
|
1363
|
+
},
|
|
1364
|
+
"required": [
|
|
1365
|
+
"document",
|
|
1366
|
+
"usage",
|
|
1367
|
+
"pages",
|
|
1368
|
+
"text",
|
|
1369
|
+
"parserType"
|
|
1370
|
+
],
|
|
1371
|
+
"additionalProperties": false
|
|
1372
|
+
}
|
|
1373
|
+
```
|
|
1374
|
+
|
|
1375
|
+
|
|
453
1376
|
#### `ai.extract` — Extract Data
|
|
454
1377
|
|
|
455
1378
|
Extract structured data from text using AI with a JSON schema
|
|
456
1379
|
|
|
1380
|
+
**Behavior and examples:** `eigenpal docs read steps/ai/extract`
|
|
1381
|
+
|
|
457
1382
|
**Durable retry:** Provider request retries are separate; the workflow engine does not durably retry this step.
|
|
458
1383
|
|
|
459
1384
|
**Config** (in `step.with`):
|
|
@@ -465,9 +1390,11 @@ Extract structured data from text using AI with a JSON schema
|
|
|
465
1390
|
| `prompt` | string | no | | Custom prompt template for extraction |
|
|
466
1391
|
| `provider` | string | no | | Provider ID (e.g., "openai-gpt4o") |
|
|
467
1392
|
| `model` | string | no | | Model override |
|
|
1393
|
+
| `reasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
468
1394
|
| `maxInputTokens` | integer | no | | Max input tokens. Truncates input text and logs a warning when exceeded. Omit for no limit. |
|
|
469
1395
|
| `grounded` | boolean | no | | Grounding is ON by default: each schema field gets a source span + confidence (high=verbatim, medium=fuzzy, low=ungrounded) under a reserved `_grounding` output key, and fields whose value cannot be located in the source are flagged for human review. Values stay the reliable schema-typed ones. The pass runs through the workspace LLM (any provider) and chunks long documents automatically. Tri-state: unset (default) = on, degrading gracefully to deterministic text alignment (`_grounding._degraded: true`) if no grounding model is available; `true` = strict, the step fails when the grounding model cannot be resolved; `false` = off, no `_grounding` key at all. |
|
|
470
1396
|
| `groundingModel` | string | no | | Provider/model for the grounding pass. Defaults to the workspace default LLM. Any configured provider works; the pass only fails the step when `grounded: true` is set explicitly and no model resolves. |
|
|
1397
|
+
| `groundingReasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
471
1398
|
| `groundingExamples` | array<object> | no | | Optional few-shot examples pinning grounding to verbatim source text per field. |
|
|
472
1399
|
| `reviewOn` | `"medium_or_low"` \| `"low_only"` | no | | Which grounding confidences set needsReview on a field. Default: low_only (only fields whose value could not be located in the source). Use medium_or_low to also flag approximate and derived matches. |
|
|
473
1400
|
|
|
@@ -475,10 +1402,174 @@ Extract structured data from text using AI with a JSON schema
|
|
|
475
1402
|
|
|
476
1403
|
> Extracted structured data matching the provided schema. Unless grounding is disabled (grounded: false), the output also carries a reserved `_grounding` map keyed by field name: `_grounding.<field> = { confidence: high|medium|low, needsReview, reason?, source_span: { start, end, text, alignment } | null }`, plus reserved `_degraded: true` / `_reason` markers when the grounding LLM pass could not run.
|
|
477
1404
|
|
|
1405
|
+
##### Complete machine-readable schemas
|
|
1406
|
+
|
|
1407
|
+
Config schema:
|
|
1408
|
+
|
|
1409
|
+
```json
|
|
1410
|
+
{
|
|
1411
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
1412
|
+
"type": "object",
|
|
1413
|
+
"properties": {
|
|
1414
|
+
"input": {
|
|
1415
|
+
"type": "string",
|
|
1416
|
+
"description": "Text content or template expression"
|
|
1417
|
+
},
|
|
1418
|
+
"schema": {
|
|
1419
|
+
"type": "object",
|
|
1420
|
+
"properties": {
|
|
1421
|
+
"type": {
|
|
1422
|
+
"type": "string",
|
|
1423
|
+
"const": "object"
|
|
1424
|
+
},
|
|
1425
|
+
"properties": {
|
|
1426
|
+
"type": "object",
|
|
1427
|
+
"propertyNames": {
|
|
1428
|
+
"type": "string"
|
|
1429
|
+
},
|
|
1430
|
+
"additionalProperties": {}
|
|
1431
|
+
},
|
|
1432
|
+
"required": {
|
|
1433
|
+
"type": "array",
|
|
1434
|
+
"items": {
|
|
1435
|
+
"type": "string"
|
|
1436
|
+
}
|
|
1437
|
+
}
|
|
1438
|
+
},
|
|
1439
|
+
"required": [
|
|
1440
|
+
"type",
|
|
1441
|
+
"properties"
|
|
1442
|
+
],
|
|
1443
|
+
"additionalProperties": false,
|
|
1444
|
+
"description": "JSON Schema defining the structure to extract"
|
|
1445
|
+
},
|
|
1446
|
+
"prompt": {
|
|
1447
|
+
"description": "Custom prompt template for extraction",
|
|
1448
|
+
"type": "string"
|
|
1449
|
+
},
|
|
1450
|
+
"provider": {
|
|
1451
|
+
"description": "Provider ID (e.g., \"openai-gpt4o\")",
|
|
1452
|
+
"type": "string"
|
|
1453
|
+
},
|
|
1454
|
+
"model": {
|
|
1455
|
+
"description": "Model override",
|
|
1456
|
+
"type": "string"
|
|
1457
|
+
},
|
|
1458
|
+
"reasoningEffort": {
|
|
1459
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
1460
|
+
"type": "string",
|
|
1461
|
+
"enum": [
|
|
1462
|
+
"none",
|
|
1463
|
+
"minimal",
|
|
1464
|
+
"low",
|
|
1465
|
+
"medium",
|
|
1466
|
+
"high",
|
|
1467
|
+
"xhigh",
|
|
1468
|
+
"max"
|
|
1469
|
+
]
|
|
1470
|
+
},
|
|
1471
|
+
"maxInputTokens": {
|
|
1472
|
+
"description": "Max input tokens. Truncates input text and logs a warning when exceeded. Omit for no limit.",
|
|
1473
|
+
"type": "integer",
|
|
1474
|
+
"minimum": -9007199254740991,
|
|
1475
|
+
"maximum": 9007199254740991
|
|
1476
|
+
},
|
|
1477
|
+
"grounded": {
|
|
1478
|
+
"description": "Grounding is ON by default: each schema field gets a source span + confidence (high=verbatim, medium=fuzzy, low=ungrounded) under a reserved `_grounding` output key, and fields whose value cannot be located in the source are flagged for human review. Values stay the reliable schema-typed ones. The pass runs through the workspace LLM (any provider) and chunks long documents automatically. Tri-state: unset (default) = on, degrading gracefully to deterministic text alignment (`_grounding._degraded: true`) if no grounding model is available; `true` = strict, the step fails when the grounding model cannot be resolved; `false` = off, no `_grounding` key at all.",
|
|
1479
|
+
"type": "boolean"
|
|
1480
|
+
},
|
|
1481
|
+
"groundingModel": {
|
|
1482
|
+
"description": "Provider/model for the grounding pass. Defaults to the workspace default LLM. Any configured provider works; the pass only fails the step when `grounded: true` is set explicitly and no model resolves.",
|
|
1483
|
+
"type": "string"
|
|
1484
|
+
},
|
|
1485
|
+
"groundingReasoningEffort": {
|
|
1486
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
1487
|
+
"type": "string",
|
|
1488
|
+
"enum": [
|
|
1489
|
+
"none",
|
|
1490
|
+
"minimal",
|
|
1491
|
+
"low",
|
|
1492
|
+
"medium",
|
|
1493
|
+
"high",
|
|
1494
|
+
"xhigh",
|
|
1495
|
+
"max"
|
|
1496
|
+
]
|
|
1497
|
+
},
|
|
1498
|
+
"groundingExamples": {
|
|
1499
|
+
"description": "Optional few-shot examples pinning grounding to verbatim source text per field.",
|
|
1500
|
+
"type": "array",
|
|
1501
|
+
"items": {
|
|
1502
|
+
"type": "object",
|
|
1503
|
+
"properties": {
|
|
1504
|
+
"text": {
|
|
1505
|
+
"type": "string"
|
|
1506
|
+
},
|
|
1507
|
+
"extractions": {
|
|
1508
|
+
"type": "array",
|
|
1509
|
+
"items": {
|
|
1510
|
+
"type": "object",
|
|
1511
|
+
"properties": {
|
|
1512
|
+
"field": {
|
|
1513
|
+
"type": "string"
|
|
1514
|
+
},
|
|
1515
|
+
"text": {
|
|
1516
|
+
"type": "string"
|
|
1517
|
+
}
|
|
1518
|
+
},
|
|
1519
|
+
"required": [
|
|
1520
|
+
"field",
|
|
1521
|
+
"text"
|
|
1522
|
+
],
|
|
1523
|
+
"additionalProperties": false
|
|
1524
|
+
}
|
|
1525
|
+
}
|
|
1526
|
+
},
|
|
1527
|
+
"required": [
|
|
1528
|
+
"text",
|
|
1529
|
+
"extractions"
|
|
1530
|
+
],
|
|
1531
|
+
"additionalProperties": false
|
|
1532
|
+
}
|
|
1533
|
+
},
|
|
1534
|
+
"reviewOn": {
|
|
1535
|
+
"description": "Which grounding confidences set needsReview on a field. Default: low_only (only fields whose value could not be located in the source). Use medium_or_low to also flag approximate and derived matches.",
|
|
1536
|
+
"type": "string",
|
|
1537
|
+
"enum": [
|
|
1538
|
+
"medium_or_low",
|
|
1539
|
+
"low_only"
|
|
1540
|
+
]
|
|
1541
|
+
}
|
|
1542
|
+
},
|
|
1543
|
+
"required": [
|
|
1544
|
+
"input",
|
|
1545
|
+
"schema"
|
|
1546
|
+
],
|
|
1547
|
+
"additionalProperties": false
|
|
1548
|
+
}
|
|
1549
|
+
```
|
|
1550
|
+
|
|
1551
|
+
|
|
1552
|
+
Output schema:
|
|
1553
|
+
|
|
1554
|
+
```json
|
|
1555
|
+
{
|
|
1556
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
1557
|
+
"type": "object",
|
|
1558
|
+
"propertyNames": {
|
|
1559
|
+
"type": "string"
|
|
1560
|
+
},
|
|
1561
|
+
"additionalProperties": {},
|
|
1562
|
+
"description": "Extracted structured data matching the provided schema. Unless grounding is disabled (grounded: false), the output also carries a reserved `_grounding` map keyed by field name: `_grounding.<field> = { confidence: high|medium|low, needsReview, reason?, source_span: { start, end, text, alignment } | null }`, plus reserved `_degraded: true` / `_reason` markers when the grounding LLM pass could not run."
|
|
1563
|
+
}
|
|
1564
|
+
```
|
|
1565
|
+
|
|
1566
|
+
|
|
478
1567
|
#### `ai.split` — Split Document
|
|
479
1568
|
|
|
480
1569
|
Split a parsed document into named sections using an LLM. Consumes ai.parse output; emits per-section page ranges and text ready for downstream ai.extract via control.parallel_map.
|
|
481
1570
|
|
|
1571
|
+
**Behavior and examples:** `eigenpal docs read steps/ai/split`
|
|
1572
|
+
|
|
482
1573
|
**Durable retry:** Provider request retries are separate; the workflow engine does not durably retry this step.
|
|
483
1574
|
|
|
484
1575
|
**Config** (in `step.with`):
|
|
@@ -489,6 +1580,7 @@ Split a parsed document into named sections using an LLM. Consumes ai.parse outp
|
|
|
489
1580
|
| `sections` | array<object> | yes | | Named sections to find in the document |
|
|
490
1581
|
| `rules` | string | no | | Optional natural-language rules appended to the system prompt. E.g. "End-of-section markers like *Koniec prílohy 2* close the current section." |
|
|
491
1582
|
| `provider` | string | no | | Provider ID from eigenpal.config.yaml (e.g. "openai-gpt5.4-mini"). Falls back to the tenant default LLM provider when omitted. |
|
|
1583
|
+
| `reasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
492
1584
|
| `windowTokenBudget` | integer | no | | Override the per-window token ceiling for this step. Defaults to env SPLIT_WINDOW_TOKEN_BUDGET or 20000. Smaller windows give sharper anchors on contract-style documents (less competing context for the LLM to mis-anchor on); bump to 50k–100k when sections routinely exceed per-window page count. |
|
|
493
1585
|
|
|
494
1586
|
**Output:** `object`
|
|
@@ -498,10 +1590,458 @@ Split a parsed document into named sections using an LLM. Consumes ai.parse outp
|
|
|
498
1590
|
| `splits` | array<object> | yes | | Sections found in the document, in page order. Absent sections are omitted. |
|
|
499
1591
|
| `sections` | record<string, object> | yes | | The same sections keyed by config name, so a downstream step can reference one directly: `{{ steps.<split>.output.sections.<name>.page_range }}`. Prefer this over filtering `splits`. On a duplicate name the last wins. |
|
|
500
1592
|
|
|
1593
|
+
##### Complete machine-readable schemas
|
|
1594
|
+
|
|
1595
|
+
Config schema:
|
|
1596
|
+
|
|
1597
|
+
```json
|
|
1598
|
+
{
|
|
1599
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
1600
|
+
"type": "object",
|
|
1601
|
+
"properties": {
|
|
1602
|
+
"input": {
|
|
1603
|
+
"type": "string",
|
|
1604
|
+
"description": "Template expression resolving to ai.parse output, e.g. \"{{ steps.parse.output }}\""
|
|
1605
|
+
},
|
|
1606
|
+
"sections": {
|
|
1607
|
+
"minItems": 1,
|
|
1608
|
+
"type": "array",
|
|
1609
|
+
"items": {
|
|
1610
|
+
"type": "object",
|
|
1611
|
+
"properties": {
|
|
1612
|
+
"name": {
|
|
1613
|
+
"type": "string",
|
|
1614
|
+
"minLength": 1,
|
|
1615
|
+
"description": "Stable id, used as the key in output and template references"
|
|
1616
|
+
},
|
|
1617
|
+
"description": {
|
|
1618
|
+
"type": "string",
|
|
1619
|
+
"minLength": 1,
|
|
1620
|
+
"description": "What this section looks like in the document. Use the document's own terminology (e.g. \"Príloha 2 / Anlage 2\") and visual cues. Multilingual variants belong here."
|
|
1621
|
+
},
|
|
1622
|
+
"required": {
|
|
1623
|
+
"description": "If true, log a warn when this section is not found, and lean the LLM toward expecting it. Default: false (sections may be absent).",
|
|
1624
|
+
"type": "boolean"
|
|
1625
|
+
},
|
|
1626
|
+
"endHints": {
|
|
1627
|
+
"description": "Natural-language hints fed to the LLM about what marks the END of this section (e.g. [\"PRÍPAD PORUŠENIA ZMLUVY\", \"*Koniec prílohy*\"]). The LLM uses these as guidance — not a regex match — so casing variants, missing diacritics, and multilingual phrasings all close the section correctly.",
|
|
1628
|
+
"type": "array",
|
|
1629
|
+
"items": {
|
|
1630
|
+
"type": "string"
|
|
1631
|
+
}
|
|
1632
|
+
}
|
|
1633
|
+
},
|
|
1634
|
+
"required": [
|
|
1635
|
+
"name",
|
|
1636
|
+
"description"
|
|
1637
|
+
],
|
|
1638
|
+
"additionalProperties": false
|
|
1639
|
+
},
|
|
1640
|
+
"description": "Named sections to find in the document"
|
|
1641
|
+
},
|
|
1642
|
+
"rules": {
|
|
1643
|
+
"description": "Optional natural-language rules appended to the system prompt. E.g. \"End-of-section markers like *Koniec prílohy 2* close the current section.\"",
|
|
1644
|
+
"type": "string"
|
|
1645
|
+
},
|
|
1646
|
+
"provider": {
|
|
1647
|
+
"description": "Provider ID from eigenpal.config.yaml (e.g. \"openai-gpt5.4-mini\"). Falls back to the tenant default LLM provider when omitted.",
|
|
1648
|
+
"type": "string"
|
|
1649
|
+
},
|
|
1650
|
+
"reasoningEffort": {
|
|
1651
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
1652
|
+
"type": "string",
|
|
1653
|
+
"enum": [
|
|
1654
|
+
"none",
|
|
1655
|
+
"minimal",
|
|
1656
|
+
"low",
|
|
1657
|
+
"medium",
|
|
1658
|
+
"high",
|
|
1659
|
+
"xhigh",
|
|
1660
|
+
"max"
|
|
1661
|
+
]
|
|
1662
|
+
},
|
|
1663
|
+
"windowTokenBudget": {
|
|
1664
|
+
"description": "Override the per-window token ceiling for this step. Defaults to env SPLIT_WINDOW_TOKEN_BUDGET or 20000. Smaller windows give sharper anchors on contract-style documents (less competing context for the LLM to mis-anchor on); bump to 50k–100k when sections routinely exceed per-window page count.",
|
|
1665
|
+
"type": "integer",
|
|
1666
|
+
"exclusiveMinimum": 0,
|
|
1667
|
+
"maximum": 9007199254740991
|
|
1668
|
+
}
|
|
1669
|
+
},
|
|
1670
|
+
"required": [
|
|
1671
|
+
"input",
|
|
1672
|
+
"sections"
|
|
1673
|
+
],
|
|
1674
|
+
"additionalProperties": false
|
|
1675
|
+
}
|
|
1676
|
+
```
|
|
1677
|
+
|
|
1678
|
+
|
|
1679
|
+
Output schema:
|
|
1680
|
+
|
|
1681
|
+
```json
|
|
1682
|
+
{
|
|
1683
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
1684
|
+
"type": "object",
|
|
1685
|
+
"properties": {
|
|
1686
|
+
"splits": {
|
|
1687
|
+
"type": "array",
|
|
1688
|
+
"items": {
|
|
1689
|
+
"type": "object",
|
|
1690
|
+
"properties": {
|
|
1691
|
+
"name": {
|
|
1692
|
+
"type": "string",
|
|
1693
|
+
"description": "Section name from the config"
|
|
1694
|
+
},
|
|
1695
|
+
"page_range": {
|
|
1696
|
+
"type": "array",
|
|
1697
|
+
"items": [
|
|
1698
|
+
{
|
|
1699
|
+
"type": "integer",
|
|
1700
|
+
"minimum": -9007199254740991,
|
|
1701
|
+
"maximum": 9007199254740991
|
|
1702
|
+
},
|
|
1703
|
+
{
|
|
1704
|
+
"type": "integer",
|
|
1705
|
+
"minimum": -9007199254740991,
|
|
1706
|
+
"maximum": 9007199254740991
|
|
1707
|
+
}
|
|
1708
|
+
],
|
|
1709
|
+
"description": "[startIndex, endIndex] inclusive, 0-based page indices"
|
|
1710
|
+
},
|
|
1711
|
+
"confidence": {
|
|
1712
|
+
"type": "string",
|
|
1713
|
+
"enum": [
|
|
1714
|
+
"low",
|
|
1715
|
+
"medium",
|
|
1716
|
+
"high"
|
|
1717
|
+
],
|
|
1718
|
+
"description": "LLM confidence at the anchor page. Coarse enum (low | medium | high) — numeric scores cluster meaninglessly at 0.85-0.95."
|
|
1719
|
+
},
|
|
1720
|
+
"notes": {
|
|
1721
|
+
"type": "string",
|
|
1722
|
+
"description": "LLM's justification for the anchor — useful for debugging"
|
|
1723
|
+
},
|
|
1724
|
+
"evidence": {
|
|
1725
|
+
"description": "Structured start-anchor evidence — parse this instead of regexing over `notes`.",
|
|
1726
|
+
"type": "object",
|
|
1727
|
+
"properties": {
|
|
1728
|
+
"start_heading_text": {
|
|
1729
|
+
"description": "Verbatim heading text the LLM cited as the section start",
|
|
1730
|
+
"anyOf": [
|
|
1731
|
+
{
|
|
1732
|
+
"type": "string"
|
|
1733
|
+
},
|
|
1734
|
+
{
|
|
1735
|
+
"type": "null"
|
|
1736
|
+
}
|
|
1737
|
+
]
|
|
1738
|
+
},
|
|
1739
|
+
"start_page": {
|
|
1740
|
+
"description": "Page where start_heading_text appears",
|
|
1741
|
+
"anyOf": [
|
|
1742
|
+
{
|
|
1743
|
+
"type": "integer",
|
|
1744
|
+
"minimum": -9007199254740991,
|
|
1745
|
+
"maximum": 9007199254740991
|
|
1746
|
+
},
|
|
1747
|
+
{
|
|
1748
|
+
"type": "null"
|
|
1749
|
+
}
|
|
1750
|
+
]
|
|
1751
|
+
}
|
|
1752
|
+
},
|
|
1753
|
+
"additionalProperties": false
|
|
1754
|
+
},
|
|
1755
|
+
"end_evidence": {
|
|
1756
|
+
"description": "Set when the LLM detected an end-of-section cue (matched against the section's `endHints` or an explicit closing marker). Absent when the section was closed by continuity-fill alone.",
|
|
1757
|
+
"type": "object",
|
|
1758
|
+
"properties": {
|
|
1759
|
+
"end_page": {
|
|
1760
|
+
"type": "integer",
|
|
1761
|
+
"minimum": -9007199254740991,
|
|
1762
|
+
"maximum": 9007199254740991,
|
|
1763
|
+
"description": "LAST page of the section (inclusive)"
|
|
1764
|
+
},
|
|
1765
|
+
"confidence": {
|
|
1766
|
+
"type": "string",
|
|
1767
|
+
"enum": [
|
|
1768
|
+
"low",
|
|
1769
|
+
"medium",
|
|
1770
|
+
"high"
|
|
1771
|
+
]
|
|
1772
|
+
},
|
|
1773
|
+
"notes": {
|
|
1774
|
+
"type": "string",
|
|
1775
|
+
"description": "LLM's justification for the end. Pair with start `notes` for reconciliation."
|
|
1776
|
+
}
|
|
1777
|
+
},
|
|
1778
|
+
"required": [
|
|
1779
|
+
"end_page",
|
|
1780
|
+
"confidence",
|
|
1781
|
+
"notes"
|
|
1782
|
+
],
|
|
1783
|
+
"additionalProperties": false
|
|
1784
|
+
},
|
|
1785
|
+
"text": {
|
|
1786
|
+
"type": "string",
|
|
1787
|
+
"description": "Joined per-page content for this section, ready for downstream ai.extract"
|
|
1788
|
+
},
|
|
1789
|
+
"pages": {
|
|
1790
|
+
"type": "array",
|
|
1791
|
+
"items": {
|
|
1792
|
+
"type": "object",
|
|
1793
|
+
"properties": {
|
|
1794
|
+
"pageIndex": {
|
|
1795
|
+
"type": "integer",
|
|
1796
|
+
"minimum": -9007199254740991,
|
|
1797
|
+
"maximum": 9007199254740991
|
|
1798
|
+
},
|
|
1799
|
+
"text": {
|
|
1800
|
+
"type": "string"
|
|
1801
|
+
},
|
|
1802
|
+
"pageName": {
|
|
1803
|
+
"type": "string"
|
|
1804
|
+
},
|
|
1805
|
+
"source": {
|
|
1806
|
+
"type": "string",
|
|
1807
|
+
"enum": [
|
|
1808
|
+
"anchored",
|
|
1809
|
+
"inferred"
|
|
1810
|
+
],
|
|
1811
|
+
"description": "Whether this page is direct LLM evidence (anchored) or continuity-filled (inferred)"
|
|
1812
|
+
}
|
|
1813
|
+
},
|
|
1814
|
+
"required": [
|
|
1815
|
+
"pageIndex",
|
|
1816
|
+
"text",
|
|
1817
|
+
"source"
|
|
1818
|
+
],
|
|
1819
|
+
"additionalProperties": false
|
|
1820
|
+
},
|
|
1821
|
+
"description": "Raw per-page records covered by this split (in page order). Iterate when downstream needs per-page context (e.g. control.parallel_map over section pages)."
|
|
1822
|
+
},
|
|
1823
|
+
"pages_anchored": {
|
|
1824
|
+
"type": "array",
|
|
1825
|
+
"items": {
|
|
1826
|
+
"type": "integer",
|
|
1827
|
+
"minimum": -9007199254740991,
|
|
1828
|
+
"maximum": 9007199254740991
|
|
1829
|
+
},
|
|
1830
|
+
"description": "Pages with direct LLM evidence"
|
|
1831
|
+
},
|
|
1832
|
+
"pages_inferred": {
|
|
1833
|
+
"type": "array",
|
|
1834
|
+
"items": {
|
|
1835
|
+
"type": "integer",
|
|
1836
|
+
"minimum": -9007199254740991,
|
|
1837
|
+
"maximum": 9007199254740991
|
|
1838
|
+
},
|
|
1839
|
+
"description": "Pages assigned by deterministic continuity fill"
|
|
1840
|
+
}
|
|
1841
|
+
},
|
|
1842
|
+
"required": [
|
|
1843
|
+
"name",
|
|
1844
|
+
"page_range",
|
|
1845
|
+
"confidence",
|
|
1846
|
+
"notes",
|
|
1847
|
+
"text",
|
|
1848
|
+
"pages",
|
|
1849
|
+
"pages_anchored",
|
|
1850
|
+
"pages_inferred"
|
|
1851
|
+
],
|
|
1852
|
+
"additionalProperties": false
|
|
1853
|
+
},
|
|
1854
|
+
"description": "Sections found in the document, in page order. Absent sections are omitted."
|
|
1855
|
+
},
|
|
1856
|
+
"sections": {
|
|
1857
|
+
"type": "object",
|
|
1858
|
+
"propertyNames": {
|
|
1859
|
+
"type": "string"
|
|
1860
|
+
},
|
|
1861
|
+
"additionalProperties": {
|
|
1862
|
+
"type": "object",
|
|
1863
|
+
"properties": {
|
|
1864
|
+
"name": {
|
|
1865
|
+
"type": "string",
|
|
1866
|
+
"description": "Section name from the config"
|
|
1867
|
+
},
|
|
1868
|
+
"page_range": {
|
|
1869
|
+
"type": "array",
|
|
1870
|
+
"items": [
|
|
1871
|
+
{
|
|
1872
|
+
"type": "integer",
|
|
1873
|
+
"minimum": -9007199254740991,
|
|
1874
|
+
"maximum": 9007199254740991
|
|
1875
|
+
},
|
|
1876
|
+
{
|
|
1877
|
+
"type": "integer",
|
|
1878
|
+
"minimum": -9007199254740991,
|
|
1879
|
+
"maximum": 9007199254740991
|
|
1880
|
+
}
|
|
1881
|
+
],
|
|
1882
|
+
"description": "[startIndex, endIndex] inclusive, 0-based page indices"
|
|
1883
|
+
},
|
|
1884
|
+
"confidence": {
|
|
1885
|
+
"type": "string",
|
|
1886
|
+
"enum": [
|
|
1887
|
+
"low",
|
|
1888
|
+
"medium",
|
|
1889
|
+
"high"
|
|
1890
|
+
],
|
|
1891
|
+
"description": "LLM confidence at the anchor page. Coarse enum (low | medium | high) — numeric scores cluster meaninglessly at 0.85-0.95."
|
|
1892
|
+
},
|
|
1893
|
+
"notes": {
|
|
1894
|
+
"type": "string",
|
|
1895
|
+
"description": "LLM's justification for the anchor — useful for debugging"
|
|
1896
|
+
},
|
|
1897
|
+
"evidence": {
|
|
1898
|
+
"description": "Structured start-anchor evidence — parse this instead of regexing over `notes`.",
|
|
1899
|
+
"type": "object",
|
|
1900
|
+
"properties": {
|
|
1901
|
+
"start_heading_text": {
|
|
1902
|
+
"description": "Verbatim heading text the LLM cited as the section start",
|
|
1903
|
+
"anyOf": [
|
|
1904
|
+
{
|
|
1905
|
+
"type": "string"
|
|
1906
|
+
},
|
|
1907
|
+
{
|
|
1908
|
+
"type": "null"
|
|
1909
|
+
}
|
|
1910
|
+
]
|
|
1911
|
+
},
|
|
1912
|
+
"start_page": {
|
|
1913
|
+
"description": "Page where start_heading_text appears",
|
|
1914
|
+
"anyOf": [
|
|
1915
|
+
{
|
|
1916
|
+
"type": "integer",
|
|
1917
|
+
"minimum": -9007199254740991,
|
|
1918
|
+
"maximum": 9007199254740991
|
|
1919
|
+
},
|
|
1920
|
+
{
|
|
1921
|
+
"type": "null"
|
|
1922
|
+
}
|
|
1923
|
+
]
|
|
1924
|
+
}
|
|
1925
|
+
},
|
|
1926
|
+
"additionalProperties": false
|
|
1927
|
+
},
|
|
1928
|
+
"end_evidence": {
|
|
1929
|
+
"description": "Set when the LLM detected an end-of-section cue (matched against the section's `endHints` or an explicit closing marker). Absent when the section was closed by continuity-fill alone.",
|
|
1930
|
+
"type": "object",
|
|
1931
|
+
"properties": {
|
|
1932
|
+
"end_page": {
|
|
1933
|
+
"type": "integer",
|
|
1934
|
+
"minimum": -9007199254740991,
|
|
1935
|
+
"maximum": 9007199254740991,
|
|
1936
|
+
"description": "LAST page of the section (inclusive)"
|
|
1937
|
+
},
|
|
1938
|
+
"confidence": {
|
|
1939
|
+
"type": "string",
|
|
1940
|
+
"enum": [
|
|
1941
|
+
"low",
|
|
1942
|
+
"medium",
|
|
1943
|
+
"high"
|
|
1944
|
+
]
|
|
1945
|
+
},
|
|
1946
|
+
"notes": {
|
|
1947
|
+
"type": "string",
|
|
1948
|
+
"description": "LLM's justification for the end. Pair with start `notes` for reconciliation."
|
|
1949
|
+
}
|
|
1950
|
+
},
|
|
1951
|
+
"required": [
|
|
1952
|
+
"end_page",
|
|
1953
|
+
"confidence",
|
|
1954
|
+
"notes"
|
|
1955
|
+
],
|
|
1956
|
+
"additionalProperties": false
|
|
1957
|
+
},
|
|
1958
|
+
"text": {
|
|
1959
|
+
"type": "string",
|
|
1960
|
+
"description": "Joined per-page content for this section, ready for downstream ai.extract"
|
|
1961
|
+
},
|
|
1962
|
+
"pages": {
|
|
1963
|
+
"type": "array",
|
|
1964
|
+
"items": {
|
|
1965
|
+
"type": "object",
|
|
1966
|
+
"properties": {
|
|
1967
|
+
"pageIndex": {
|
|
1968
|
+
"type": "integer",
|
|
1969
|
+
"minimum": -9007199254740991,
|
|
1970
|
+
"maximum": 9007199254740991
|
|
1971
|
+
},
|
|
1972
|
+
"text": {
|
|
1973
|
+
"type": "string"
|
|
1974
|
+
},
|
|
1975
|
+
"pageName": {
|
|
1976
|
+
"type": "string"
|
|
1977
|
+
},
|
|
1978
|
+
"source": {
|
|
1979
|
+
"type": "string",
|
|
1980
|
+
"enum": [
|
|
1981
|
+
"anchored",
|
|
1982
|
+
"inferred"
|
|
1983
|
+
],
|
|
1984
|
+
"description": "Whether this page is direct LLM evidence (anchored) or continuity-filled (inferred)"
|
|
1985
|
+
}
|
|
1986
|
+
},
|
|
1987
|
+
"required": [
|
|
1988
|
+
"pageIndex",
|
|
1989
|
+
"text",
|
|
1990
|
+
"source"
|
|
1991
|
+
],
|
|
1992
|
+
"additionalProperties": false
|
|
1993
|
+
},
|
|
1994
|
+
"description": "Raw per-page records covered by this split (in page order). Iterate when downstream needs per-page context (e.g. control.parallel_map over section pages)."
|
|
1995
|
+
},
|
|
1996
|
+
"pages_anchored": {
|
|
1997
|
+
"type": "array",
|
|
1998
|
+
"items": {
|
|
1999
|
+
"type": "integer",
|
|
2000
|
+
"minimum": -9007199254740991,
|
|
2001
|
+
"maximum": 9007199254740991
|
|
2002
|
+
},
|
|
2003
|
+
"description": "Pages with direct LLM evidence"
|
|
2004
|
+
},
|
|
2005
|
+
"pages_inferred": {
|
|
2006
|
+
"type": "array",
|
|
2007
|
+
"items": {
|
|
2008
|
+
"type": "integer",
|
|
2009
|
+
"minimum": -9007199254740991,
|
|
2010
|
+
"maximum": 9007199254740991
|
|
2011
|
+
},
|
|
2012
|
+
"description": "Pages assigned by deterministic continuity fill"
|
|
2013
|
+
}
|
|
2014
|
+
},
|
|
2015
|
+
"required": [
|
|
2016
|
+
"name",
|
|
2017
|
+
"page_range",
|
|
2018
|
+
"confidence",
|
|
2019
|
+
"notes",
|
|
2020
|
+
"text",
|
|
2021
|
+
"pages",
|
|
2022
|
+
"pages_anchored",
|
|
2023
|
+
"pages_inferred"
|
|
2024
|
+
],
|
|
2025
|
+
"additionalProperties": false
|
|
2026
|
+
},
|
|
2027
|
+
"description": "The same sections keyed by config name, so a downstream step can reference one directly: `{{ steps.<split>.output.sections.<name>.page_range }}`. Prefer this over filtering `splits`. On a duplicate name the last wins."
|
|
2028
|
+
}
|
|
2029
|
+
},
|
|
2030
|
+
"required": [
|
|
2031
|
+
"splits",
|
|
2032
|
+
"sections"
|
|
2033
|
+
],
|
|
2034
|
+
"additionalProperties": false
|
|
2035
|
+
}
|
|
2036
|
+
```
|
|
2037
|
+
|
|
2038
|
+
|
|
501
2039
|
#### `ai.segment` — Separate Documents
|
|
502
2040
|
|
|
503
2041
|
Separate a concatenated batch (one big scan) into typed document instances using an LLM. Consumes ai.parse output and a type taxonomy; discovers an unknown number of documents in any order and emits per-document page ranges + text + type, ready for type-specific ai.extract via control.parallel_map. The inverse of ai.split.
|
|
504
2042
|
|
|
2043
|
+
**Behavior and examples:** `eigenpal docs read steps/ai/segment`
|
|
2044
|
+
|
|
505
2045
|
**Durable retry:** Provider request retries are separate; the workflow engine does not durably retry this step.
|
|
506
2046
|
|
|
507
2047
|
**Config** (in `step.with`):
|
|
@@ -512,6 +2052,7 @@ Separate a concatenated batch (one big scan) into typed document instances using
|
|
|
512
2052
|
| `documentTypes` | array<object> | yes | | The document-type taxonomy. The LLM tags each detected document with one of these names, or the reserved "unknown" type when none fit. |
|
|
513
2053
|
| `rules` | string | no | | Optional natural-language rules appended to the system prompt. |
|
|
514
2054
|
| `provider` | string | no | | Provider ID from eigenpal.config.yaml. Falls back to the tenant default LLM provider when omitted. |
|
|
2055
|
+
| `reasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
515
2056
|
| `windowTokenBudget` | integer | no | | Override the per-window token ceiling. Defaults to env SPLIT_WINDOW_TOKEN_BUDGET or 20000. |
|
|
516
2057
|
|
|
517
2058
|
**Output:** `object`
|
|
@@ -521,10 +2062,349 @@ Separate a concatenated batch (one big scan) into typed document instances using
|
|
|
521
2062
|
| `summary` | object | yes | | Batch-level escalation summary: labelled vs unlabelled counts + docs needing review. |
|
|
522
2063
|
| `documents` | array<object> | yes | | Documents discovered in the batch, in page order. The full batch is covered. |
|
|
523
2064
|
|
|
2065
|
+
##### Complete machine-readable schemas
|
|
2066
|
+
|
|
2067
|
+
Config schema:
|
|
2068
|
+
|
|
2069
|
+
```json
|
|
2070
|
+
{
|
|
2071
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2072
|
+
"type": "object",
|
|
2073
|
+
"properties": {
|
|
2074
|
+
"input": {
|
|
2075
|
+
"type": "string",
|
|
2076
|
+
"description": "Template expression resolving to ai.parse output, e.g. \"{{ steps.parse.output }}\""
|
|
2077
|
+
},
|
|
2078
|
+
"documentTypes": {
|
|
2079
|
+
"minItems": 1,
|
|
2080
|
+
"type": "array",
|
|
2081
|
+
"items": {
|
|
2082
|
+
"type": "object",
|
|
2083
|
+
"properties": {
|
|
2084
|
+
"name": {
|
|
2085
|
+
"type": "string",
|
|
2086
|
+
"minLength": 1,
|
|
2087
|
+
"description": "Type label returned in documents[].type (e.g. \"timesheet\", \"deal-memo\")"
|
|
2088
|
+
},
|
|
2089
|
+
"description": {
|
|
2090
|
+
"description": "What this document type looks like — headings, layout, vocabulary, and the cues that mark its FIRST page. Fed to the LLM for both boundary detection and typing.",
|
|
2091
|
+
"type": "string"
|
|
2092
|
+
}
|
|
2093
|
+
},
|
|
2094
|
+
"required": [
|
|
2095
|
+
"name"
|
|
2096
|
+
],
|
|
2097
|
+
"additionalProperties": false
|
|
2098
|
+
},
|
|
2099
|
+
"description": "The document-type taxonomy. The LLM tags each detected document with one of these names, or the reserved \"unknown\" type when none fit."
|
|
2100
|
+
},
|
|
2101
|
+
"rules": {
|
|
2102
|
+
"description": "Optional natural-language rules appended to the system prompt.",
|
|
2103
|
+
"type": "string"
|
|
2104
|
+
},
|
|
2105
|
+
"provider": {
|
|
2106
|
+
"description": "Provider ID from eigenpal.config.yaml. Falls back to the tenant default LLM provider when omitted.",
|
|
2107
|
+
"type": "string"
|
|
2108
|
+
},
|
|
2109
|
+
"reasoningEffort": {
|
|
2110
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
2111
|
+
"type": "string",
|
|
2112
|
+
"enum": [
|
|
2113
|
+
"none",
|
|
2114
|
+
"minimal",
|
|
2115
|
+
"low",
|
|
2116
|
+
"medium",
|
|
2117
|
+
"high",
|
|
2118
|
+
"xhigh",
|
|
2119
|
+
"max"
|
|
2120
|
+
]
|
|
2121
|
+
},
|
|
2122
|
+
"windowTokenBudget": {
|
|
2123
|
+
"description": "Override the per-window token ceiling. Defaults to env SPLIT_WINDOW_TOKEN_BUDGET or 20000.",
|
|
2124
|
+
"type": "integer",
|
|
2125
|
+
"exclusiveMinimum": 0,
|
|
2126
|
+
"maximum": 9007199254740991
|
|
2127
|
+
}
|
|
2128
|
+
},
|
|
2129
|
+
"required": [
|
|
2130
|
+
"input",
|
|
2131
|
+
"documentTypes"
|
|
2132
|
+
],
|
|
2133
|
+
"additionalProperties": false
|
|
2134
|
+
}
|
|
2135
|
+
```
|
|
2136
|
+
|
|
2137
|
+
|
|
2138
|
+
Output schema:
|
|
2139
|
+
|
|
2140
|
+
```json
|
|
2141
|
+
{
|
|
2142
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2143
|
+
"type": "object",
|
|
2144
|
+
"properties": {
|
|
2145
|
+
"summary": {
|
|
2146
|
+
"type": "object",
|
|
2147
|
+
"properties": {
|
|
2148
|
+
"total": {
|
|
2149
|
+
"type": "integer",
|
|
2150
|
+
"minimum": -9007199254740991,
|
|
2151
|
+
"maximum": 9007199254740991,
|
|
2152
|
+
"description": "Total documents discovered."
|
|
2153
|
+
},
|
|
2154
|
+
"labelled": {
|
|
2155
|
+
"type": "integer",
|
|
2156
|
+
"minimum": -9007199254740991,
|
|
2157
|
+
"maximum": 9007199254740991,
|
|
2158
|
+
"description": "Documents assigned a known taxonomy type."
|
|
2159
|
+
},
|
|
2160
|
+
"unlabelled": {
|
|
2161
|
+
"type": "integer",
|
|
2162
|
+
"minimum": -9007199254740991,
|
|
2163
|
+
"maximum": 9007199254740991,
|
|
2164
|
+
"description": "Documents the segmenter could not label (type \"unknown\") — escalate to a human."
|
|
2165
|
+
},
|
|
2166
|
+
"lowConfidence": {
|
|
2167
|
+
"type": "integer",
|
|
2168
|
+
"minimum": -9007199254740991,
|
|
2169
|
+
"maximum": 9007199254740991,
|
|
2170
|
+
"description": "Documents with low boundary/type confidence."
|
|
2171
|
+
},
|
|
2172
|
+
"byType": {
|
|
2173
|
+
"type": "object",
|
|
2174
|
+
"propertyNames": {
|
|
2175
|
+
"type": "string"
|
|
2176
|
+
},
|
|
2177
|
+
"additionalProperties": {
|
|
2178
|
+
"type": "integer",
|
|
2179
|
+
"minimum": -9007199254740991,
|
|
2180
|
+
"maximum": 9007199254740991
|
|
2181
|
+
},
|
|
2182
|
+
"description": "Per-type document counts."
|
|
2183
|
+
},
|
|
2184
|
+
"needsReview": {
|
|
2185
|
+
"type": "array",
|
|
2186
|
+
"items": {
|
|
2187
|
+
"type": "object",
|
|
2188
|
+
"properties": {
|
|
2189
|
+
"index": {
|
|
2190
|
+
"type": "integer",
|
|
2191
|
+
"minimum": -9007199254740991,
|
|
2192
|
+
"maximum": 9007199254740991
|
|
2193
|
+
},
|
|
2194
|
+
"type": {
|
|
2195
|
+
"type": "string"
|
|
2196
|
+
},
|
|
2197
|
+
"page_range": {
|
|
2198
|
+
"type": "array",
|
|
2199
|
+
"items": [
|
|
2200
|
+
{
|
|
2201
|
+
"type": "integer",
|
|
2202
|
+
"minimum": -9007199254740991,
|
|
2203
|
+
"maximum": 9007199254740991
|
|
2204
|
+
},
|
|
2205
|
+
{
|
|
2206
|
+
"type": "integer",
|
|
2207
|
+
"minimum": -9007199254740991,
|
|
2208
|
+
"maximum": 9007199254740991
|
|
2209
|
+
}
|
|
2210
|
+
]
|
|
2211
|
+
},
|
|
2212
|
+
"reason": {
|
|
2213
|
+
"type": "string",
|
|
2214
|
+
"enum": [
|
|
2215
|
+
"unlabelled",
|
|
2216
|
+
"low_confidence"
|
|
2217
|
+
]
|
|
2218
|
+
},
|
|
2219
|
+
"notes": {
|
|
2220
|
+
"type": "string"
|
|
2221
|
+
}
|
|
2222
|
+
},
|
|
2223
|
+
"required": [
|
|
2224
|
+
"index",
|
|
2225
|
+
"type",
|
|
2226
|
+
"page_range",
|
|
2227
|
+
"reason",
|
|
2228
|
+
"notes"
|
|
2229
|
+
],
|
|
2230
|
+
"additionalProperties": false
|
|
2231
|
+
},
|
|
2232
|
+
"description": "Documents to escalate to a human operator, with the reason for each."
|
|
2233
|
+
}
|
|
2234
|
+
},
|
|
2235
|
+
"required": [
|
|
2236
|
+
"total",
|
|
2237
|
+
"labelled",
|
|
2238
|
+
"unlabelled",
|
|
2239
|
+
"lowConfidence",
|
|
2240
|
+
"byType",
|
|
2241
|
+
"needsReview"
|
|
2242
|
+
],
|
|
2243
|
+
"additionalProperties": false,
|
|
2244
|
+
"description": "Batch-level escalation summary: labelled vs unlabelled counts + docs needing review."
|
|
2245
|
+
},
|
|
2246
|
+
"documents": {
|
|
2247
|
+
"type": "array",
|
|
2248
|
+
"items": {
|
|
2249
|
+
"type": "object",
|
|
2250
|
+
"properties": {
|
|
2251
|
+
"index": {
|
|
2252
|
+
"type": "integer",
|
|
2253
|
+
"minimum": -9007199254740991,
|
|
2254
|
+
"maximum": 9007199254740991,
|
|
2255
|
+
"description": "0-based position of this document in the batch"
|
|
2256
|
+
},
|
|
2257
|
+
"type": {
|
|
2258
|
+
"type": "string",
|
|
2259
|
+
"description": "Chosen type name from the taxonomy, or \"unknown\""
|
|
2260
|
+
},
|
|
2261
|
+
"page_range": {
|
|
2262
|
+
"type": "array",
|
|
2263
|
+
"items": [
|
|
2264
|
+
{
|
|
2265
|
+
"type": "integer",
|
|
2266
|
+
"minimum": -9007199254740991,
|
|
2267
|
+
"maximum": 9007199254740991
|
|
2268
|
+
},
|
|
2269
|
+
{
|
|
2270
|
+
"type": "integer",
|
|
2271
|
+
"minimum": -9007199254740991,
|
|
2272
|
+
"maximum": 9007199254740991
|
|
2273
|
+
}
|
|
2274
|
+
],
|
|
2275
|
+
"description": "[startIndex, endIndex] inclusive, 0-based page indices"
|
|
2276
|
+
},
|
|
2277
|
+
"confidence": {
|
|
2278
|
+
"type": "string",
|
|
2279
|
+
"enum": [
|
|
2280
|
+
"low",
|
|
2281
|
+
"medium",
|
|
2282
|
+
"high"
|
|
2283
|
+
],
|
|
2284
|
+
"description": "LLM confidence at the boundary + type. Coarse enum (low | medium | high)."
|
|
2285
|
+
},
|
|
2286
|
+
"notes": {
|
|
2287
|
+
"type": "string",
|
|
2288
|
+
"description": "LLM justification for the boundary — useful for debugging"
|
|
2289
|
+
},
|
|
2290
|
+
"evidence": {
|
|
2291
|
+
"description": "Structured start evidence — parse this instead of regexing over `notes`.",
|
|
2292
|
+
"type": "object",
|
|
2293
|
+
"properties": {
|
|
2294
|
+
"start_heading_text": {
|
|
2295
|
+
"description": "Verbatim heading/title text the LLM cited as the document start",
|
|
2296
|
+
"anyOf": [
|
|
2297
|
+
{
|
|
2298
|
+
"type": "string"
|
|
2299
|
+
},
|
|
2300
|
+
{
|
|
2301
|
+
"type": "null"
|
|
2302
|
+
}
|
|
2303
|
+
]
|
|
2304
|
+
},
|
|
2305
|
+
"start_page": {
|
|
2306
|
+
"description": "Page where the start appears",
|
|
2307
|
+
"anyOf": [
|
|
2308
|
+
{
|
|
2309
|
+
"type": "integer",
|
|
2310
|
+
"minimum": -9007199254740991,
|
|
2311
|
+
"maximum": 9007199254740991
|
|
2312
|
+
},
|
|
2313
|
+
{
|
|
2314
|
+
"type": "null"
|
|
2315
|
+
}
|
|
2316
|
+
]
|
|
2317
|
+
}
|
|
2318
|
+
},
|
|
2319
|
+
"additionalProperties": false
|
|
2320
|
+
},
|
|
2321
|
+
"text": {
|
|
2322
|
+
"type": "string",
|
|
2323
|
+
"description": "Joined per-page content for this document, ready for downstream ai.extract"
|
|
2324
|
+
},
|
|
2325
|
+
"pages": {
|
|
2326
|
+
"type": "array",
|
|
2327
|
+
"items": {
|
|
2328
|
+
"type": "object",
|
|
2329
|
+
"properties": {
|
|
2330
|
+
"pageIndex": {
|
|
2331
|
+
"type": "integer",
|
|
2332
|
+
"minimum": -9007199254740991,
|
|
2333
|
+
"maximum": 9007199254740991
|
|
2334
|
+
},
|
|
2335
|
+
"text": {
|
|
2336
|
+
"type": "string"
|
|
2337
|
+
},
|
|
2338
|
+
"pageName": {
|
|
2339
|
+
"type": "string"
|
|
2340
|
+
},
|
|
2341
|
+
"source": {
|
|
2342
|
+
"type": "string",
|
|
2343
|
+
"enum": [
|
|
2344
|
+
"anchored",
|
|
2345
|
+
"inferred"
|
|
2346
|
+
]
|
|
2347
|
+
}
|
|
2348
|
+
},
|
|
2349
|
+
"required": [
|
|
2350
|
+
"pageIndex",
|
|
2351
|
+
"text",
|
|
2352
|
+
"source"
|
|
2353
|
+
],
|
|
2354
|
+
"additionalProperties": false
|
|
2355
|
+
},
|
|
2356
|
+
"description": "Raw per-page records covered by this document, in page order."
|
|
2357
|
+
},
|
|
2358
|
+
"pages_anchored": {
|
|
2359
|
+
"type": "array",
|
|
2360
|
+
"items": {
|
|
2361
|
+
"type": "integer",
|
|
2362
|
+
"minimum": -9007199254740991,
|
|
2363
|
+
"maximum": 9007199254740991
|
|
2364
|
+
},
|
|
2365
|
+
"description": "Pages with direct LLM boundary evidence"
|
|
2366
|
+
},
|
|
2367
|
+
"pages_inferred": {
|
|
2368
|
+
"type": "array",
|
|
2369
|
+
"items": {
|
|
2370
|
+
"type": "integer",
|
|
2371
|
+
"minimum": -9007199254740991,
|
|
2372
|
+
"maximum": 9007199254740991
|
|
2373
|
+
},
|
|
2374
|
+
"description": "Pages assigned by deterministic continuity fill"
|
|
2375
|
+
}
|
|
2376
|
+
},
|
|
2377
|
+
"required": [
|
|
2378
|
+
"index",
|
|
2379
|
+
"type",
|
|
2380
|
+
"page_range",
|
|
2381
|
+
"confidence",
|
|
2382
|
+
"notes",
|
|
2383
|
+
"text",
|
|
2384
|
+
"pages",
|
|
2385
|
+
"pages_anchored",
|
|
2386
|
+
"pages_inferred"
|
|
2387
|
+
],
|
|
2388
|
+
"additionalProperties": false
|
|
2389
|
+
},
|
|
2390
|
+
"description": "Documents discovered in the batch, in page order. The full batch is covered."
|
|
2391
|
+
}
|
|
2392
|
+
},
|
|
2393
|
+
"required": [
|
|
2394
|
+
"summary",
|
|
2395
|
+
"documents"
|
|
2396
|
+
],
|
|
2397
|
+
"additionalProperties": false
|
|
2398
|
+
}
|
|
2399
|
+
```
|
|
2400
|
+
|
|
2401
|
+
|
|
524
2402
|
#### `ai.classify` — Classify
|
|
525
2403
|
|
|
526
2404
|
Classify a document or text into one of a fixed label set using an LLM. Output exposes the picked label (constrained to the configured names), a coarse confidence, and a short justification. Pair with control.fail to reject documents that match an undesired label.
|
|
527
2405
|
|
|
2406
|
+
**Behavior and examples:** `eigenpal docs read steps/ai/classify`
|
|
2407
|
+
|
|
528
2408
|
**Durable retry:** Provider request retries are separate; the workflow engine does not durably retry this step.
|
|
529
2409
|
|
|
530
2410
|
**Config** (in `step.with`):
|
|
@@ -536,6 +2416,7 @@ Classify a document or text into one of a fixed label set using an LLM. Output e
|
|
|
536
2416
|
| `prompt` | string | no | | Custom classification instructions appended to the system prompt. Use to clarify edge cases or emphasize evidence the model should weigh. |
|
|
537
2417
|
| `provider` | string | no | | Provider ID from eigenpal.config.yaml (e.g. "openai-gpt4o-mini"). Falls back to the tenant default LLM provider when omitted. |
|
|
538
2418
|
| `model` | string | no | | Model override (advanced) |
|
|
2419
|
+
| `reasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
539
2420
|
| `maxInputTokens` | integer | no | | Max input tokens. Truncates input text when exceeded. Omit for no limit. |
|
|
540
2421
|
|
|
541
2422
|
**Output:** `object`
|
|
@@ -546,10 +2427,124 @@ Classify a document or text into one of a fixed label set using an LLM. Output e
|
|
|
546
2427
|
| `confidence` | `"low"` \| `"medium"` \| `"high"` | yes | | LLM confidence in the classification. Coarse enum — numeric scores cluster meaninglessly at 0.85-0.95. |
|
|
547
2428
|
| `reason` | string | yes | | Short justification for the chosen label — useful for debugging. |
|
|
548
2429
|
|
|
2430
|
+
##### Complete machine-readable schemas
|
|
2431
|
+
|
|
2432
|
+
Config schema:
|
|
2433
|
+
|
|
2434
|
+
```json
|
|
2435
|
+
{
|
|
2436
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2437
|
+
"type": "object",
|
|
2438
|
+
"properties": {
|
|
2439
|
+
"input": {
|
|
2440
|
+
"type": "string",
|
|
2441
|
+
"description": "Template expression for the text to classify. Typically the output of ai.parse, e.g. \"{{ steps.parse.output.text }}\"."
|
|
2442
|
+
},
|
|
2443
|
+
"labels": {
|
|
2444
|
+
"minItems": 2,
|
|
2445
|
+
"type": "array",
|
|
2446
|
+
"items": {
|
|
2447
|
+
"type": "object",
|
|
2448
|
+
"properties": {
|
|
2449
|
+
"name": {
|
|
2450
|
+
"type": "string",
|
|
2451
|
+
"minLength": 1,
|
|
2452
|
+
"description": "The label value returned in output.label. Keep it short and stable (used in downstream conditions)."
|
|
2453
|
+
},
|
|
2454
|
+
"description": {
|
|
2455
|
+
"description": "What documents fall into this label. Fed to the LLM as guidance.",
|
|
2456
|
+
"type": "string"
|
|
2457
|
+
}
|
|
2458
|
+
},
|
|
2459
|
+
"required": [
|
|
2460
|
+
"name"
|
|
2461
|
+
],
|
|
2462
|
+
"additionalProperties": false
|
|
2463
|
+
},
|
|
2464
|
+
"description": "Allowed labels. The LLM is constrained to pick exactly one of these names."
|
|
2465
|
+
},
|
|
2466
|
+
"prompt": {
|
|
2467
|
+
"description": "Custom classification instructions appended to the system prompt. Use to clarify edge cases or emphasize evidence the model should weigh.",
|
|
2468
|
+
"type": "string"
|
|
2469
|
+
},
|
|
2470
|
+
"provider": {
|
|
2471
|
+
"description": "Provider ID from eigenpal.config.yaml (e.g. \"openai-gpt4o-mini\"). Falls back to the tenant default LLM provider when omitted.",
|
|
2472
|
+
"type": "string"
|
|
2473
|
+
},
|
|
2474
|
+
"model": {
|
|
2475
|
+
"description": "Model override (advanced)",
|
|
2476
|
+
"type": "string"
|
|
2477
|
+
},
|
|
2478
|
+
"reasoningEffort": {
|
|
2479
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
2480
|
+
"type": "string",
|
|
2481
|
+
"enum": [
|
|
2482
|
+
"none",
|
|
2483
|
+
"minimal",
|
|
2484
|
+
"low",
|
|
2485
|
+
"medium",
|
|
2486
|
+
"high",
|
|
2487
|
+
"xhigh",
|
|
2488
|
+
"max"
|
|
2489
|
+
]
|
|
2490
|
+
},
|
|
2491
|
+
"maxInputTokens": {
|
|
2492
|
+
"description": "Max input tokens. Truncates input text when exceeded. Omit for no limit.",
|
|
2493
|
+
"type": "integer",
|
|
2494
|
+
"minimum": -9007199254740991,
|
|
2495
|
+
"maximum": 9007199254740991
|
|
2496
|
+
}
|
|
2497
|
+
},
|
|
2498
|
+
"required": [
|
|
2499
|
+
"input",
|
|
2500
|
+
"labels"
|
|
2501
|
+
],
|
|
2502
|
+
"additionalProperties": false
|
|
2503
|
+
}
|
|
2504
|
+
```
|
|
2505
|
+
|
|
2506
|
+
|
|
2507
|
+
Output schema:
|
|
2508
|
+
|
|
2509
|
+
```json
|
|
2510
|
+
{
|
|
2511
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2512
|
+
"type": "object",
|
|
2513
|
+
"properties": {
|
|
2514
|
+
"label": {
|
|
2515
|
+
"type": "string",
|
|
2516
|
+
"description": "The selected label name (one of the configured labels). Compare against literal strings to gate downstream steps."
|
|
2517
|
+
},
|
|
2518
|
+
"confidence": {
|
|
2519
|
+
"type": "string",
|
|
2520
|
+
"enum": [
|
|
2521
|
+
"low",
|
|
2522
|
+
"medium",
|
|
2523
|
+
"high"
|
|
2524
|
+
],
|
|
2525
|
+
"description": "LLM confidence in the classification. Coarse enum — numeric scores cluster meaninglessly at 0.85-0.95."
|
|
2526
|
+
},
|
|
2527
|
+
"reason": {
|
|
2528
|
+
"type": "string",
|
|
2529
|
+
"description": "Short justification for the chosen label — useful for debugging."
|
|
2530
|
+
}
|
|
2531
|
+
},
|
|
2532
|
+
"required": [
|
|
2533
|
+
"label",
|
|
2534
|
+
"confidence",
|
|
2535
|
+
"reason"
|
|
2536
|
+
],
|
|
2537
|
+
"additionalProperties": false
|
|
2538
|
+
}
|
|
2539
|
+
```
|
|
2540
|
+
|
|
2541
|
+
|
|
549
2542
|
#### `ai.classify-pages` — Label Pages
|
|
550
2543
|
|
|
551
2544
|
Assign zero or more labels to each page independently (multi-label) using an LLM. Consumes ai.parse output; emits per-page labels and a byLabel map (label -> page indices) that supports NON-contiguous selections. Feed byLabel.<label> straight into ai.vision `pageIndices` to inspect scattered pages of a type (e.g. every signature or property-photo page).
|
|
552
2545
|
|
|
2546
|
+
**Behavior and examples:** `eigenpal docs read steps/ai/classify-pages`
|
|
2547
|
+
|
|
553
2548
|
**Durable retry:** Provider request retries are separate; the workflow engine does not durably retry this step.
|
|
554
2549
|
|
|
555
2550
|
**Config** (in `step.with`):
|
|
@@ -561,6 +2556,7 @@ Assign zero or more labels to each page independently (multi-label) using an LLM
|
|
|
561
2556
|
| `prompt` | string | no | | Extra classification guidance appended to the system prompt. |
|
|
562
2557
|
| `provider` | string | no | | Provider ID from eigenpal.config.yaml. Falls back to the tenant default. |
|
|
563
2558
|
| `model` | string | no | | Model override (advanced). |
|
|
2559
|
+
| `reasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
564
2560
|
| `windowTokenBudget` | integer | no | | Per-window token ceiling. Pages are packed into windows under this budget; one LLM call per window. Defaults to env SPLIT_WINDOW_TOKEN_BUDGET (20000). |
|
|
565
2561
|
|
|
566
2562
|
**Output:** `object`
|
|
@@ -570,10 +2566,155 @@ Assign zero or more labels to each page independently (multi-label) using an LLM
|
|
|
570
2566
|
| `pages` | array<object> | yes | | Every input page, in order, with the labels the model assigned to it. |
|
|
571
2567
|
| `byLabel` | record<string, array<integer>> | yes | | Label name -> ascending page indices carrying it. Every configured label is present (empty array when no page matched), so dot access is always safe: `{{ steps.<step>.output.byLabel.<label> }}` resolves to a number[] ready for ai.vision `pageIndices`. |
|
|
572
2568
|
|
|
2569
|
+
##### Complete machine-readable schemas
|
|
2570
|
+
|
|
2571
|
+
Config schema:
|
|
2572
|
+
|
|
2573
|
+
```json
|
|
2574
|
+
{
|
|
2575
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2576
|
+
"type": "object",
|
|
2577
|
+
"properties": {
|
|
2578
|
+
"input": {
|
|
2579
|
+
"type": "string",
|
|
2580
|
+
"description": "The ai.parse output to label, e.g. \"{{ steps.parse.output }}\". Each page is labelled independently."
|
|
2581
|
+
},
|
|
2582
|
+
"labels": {
|
|
2583
|
+
"minItems": 1,
|
|
2584
|
+
"type": "array",
|
|
2585
|
+
"items": {
|
|
2586
|
+
"anyOf": [
|
|
2587
|
+
{
|
|
2588
|
+
"type": "string",
|
|
2589
|
+
"minLength": 1
|
|
2590
|
+
},
|
|
2591
|
+
{
|
|
2592
|
+
"type": "object",
|
|
2593
|
+
"properties": {
|
|
2594
|
+
"name": {
|
|
2595
|
+
"type": "string",
|
|
2596
|
+
"minLength": 1,
|
|
2597
|
+
"description": "Label value — the key in output.byLabel and an entry in pages[].labels."
|
|
2598
|
+
},
|
|
2599
|
+
"description": {
|
|
2600
|
+
"description": "What a page must show to earn this label. Fed to the LLM as guidance.",
|
|
2601
|
+
"type": "string"
|
|
2602
|
+
}
|
|
2603
|
+
},
|
|
2604
|
+
"required": [
|
|
2605
|
+
"name"
|
|
2606
|
+
],
|
|
2607
|
+
"additionalProperties": false
|
|
2608
|
+
}
|
|
2609
|
+
]
|
|
2610
|
+
},
|
|
2611
|
+
"description": "Labels a page can carry. A page may match ZERO, ONE, or MANY of them. Each entry is a bare name (e.g. \"photo\") or { name, description }. Names must be unique."
|
|
2612
|
+
},
|
|
2613
|
+
"prompt": {
|
|
2614
|
+
"description": "Extra classification guidance appended to the system prompt.",
|
|
2615
|
+
"type": "string"
|
|
2616
|
+
},
|
|
2617
|
+
"provider": {
|
|
2618
|
+
"description": "Provider ID from eigenpal.config.yaml. Falls back to the tenant default.",
|
|
2619
|
+
"type": "string"
|
|
2620
|
+
},
|
|
2621
|
+
"model": {
|
|
2622
|
+
"description": "Model override (advanced).",
|
|
2623
|
+
"type": "string"
|
|
2624
|
+
},
|
|
2625
|
+
"reasoningEffort": {
|
|
2626
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
2627
|
+
"type": "string",
|
|
2628
|
+
"enum": [
|
|
2629
|
+
"none",
|
|
2630
|
+
"minimal",
|
|
2631
|
+
"low",
|
|
2632
|
+
"medium",
|
|
2633
|
+
"high",
|
|
2634
|
+
"xhigh",
|
|
2635
|
+
"max"
|
|
2636
|
+
]
|
|
2637
|
+
},
|
|
2638
|
+
"windowTokenBudget": {
|
|
2639
|
+
"description": "Per-window token ceiling. Pages are packed into windows under this budget; one LLM call per window. Defaults to env SPLIT_WINDOW_TOKEN_BUDGET (20000).",
|
|
2640
|
+
"type": "integer",
|
|
2641
|
+
"exclusiveMinimum": 0,
|
|
2642
|
+
"maximum": 9007199254740991
|
|
2643
|
+
}
|
|
2644
|
+
},
|
|
2645
|
+
"required": [
|
|
2646
|
+
"input",
|
|
2647
|
+
"labels"
|
|
2648
|
+
],
|
|
2649
|
+
"additionalProperties": false
|
|
2650
|
+
}
|
|
2651
|
+
```
|
|
2652
|
+
|
|
2653
|
+
|
|
2654
|
+
Output schema:
|
|
2655
|
+
|
|
2656
|
+
```json
|
|
2657
|
+
{
|
|
2658
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2659
|
+
"type": "object",
|
|
2660
|
+
"properties": {
|
|
2661
|
+
"pages": {
|
|
2662
|
+
"type": "array",
|
|
2663
|
+
"items": {
|
|
2664
|
+
"type": "object",
|
|
2665
|
+
"properties": {
|
|
2666
|
+
"pageIndex": {
|
|
2667
|
+
"type": "integer",
|
|
2668
|
+
"minimum": -9007199254740991,
|
|
2669
|
+
"maximum": 9007199254740991
|
|
2670
|
+
},
|
|
2671
|
+
"labels": {
|
|
2672
|
+
"type": "array",
|
|
2673
|
+
"items": {
|
|
2674
|
+
"type": "string"
|
|
2675
|
+
},
|
|
2676
|
+
"description": "Labels assigned to this page (possibly empty)."
|
|
2677
|
+
}
|
|
2678
|
+
},
|
|
2679
|
+
"required": [
|
|
2680
|
+
"pageIndex",
|
|
2681
|
+
"labels"
|
|
2682
|
+
],
|
|
2683
|
+
"additionalProperties": false
|
|
2684
|
+
},
|
|
2685
|
+
"description": "Every input page, in order, with the labels the model assigned to it."
|
|
2686
|
+
},
|
|
2687
|
+
"byLabel": {
|
|
2688
|
+
"type": "object",
|
|
2689
|
+
"propertyNames": {
|
|
2690
|
+
"type": "string"
|
|
2691
|
+
},
|
|
2692
|
+
"additionalProperties": {
|
|
2693
|
+
"type": "array",
|
|
2694
|
+
"items": {
|
|
2695
|
+
"type": "integer",
|
|
2696
|
+
"minimum": -9007199254740991,
|
|
2697
|
+
"maximum": 9007199254740991
|
|
2698
|
+
}
|
|
2699
|
+
},
|
|
2700
|
+
"description": "Label name -> ascending page indices carrying it. Every configured label is present (empty array when no page matched), so dot access is always safe: `{{ steps.<step>.output.byLabel.<label> }}` resolves to a number[] ready for ai.vision `pageIndices`."
|
|
2701
|
+
}
|
|
2702
|
+
},
|
|
2703
|
+
"required": [
|
|
2704
|
+
"pages",
|
|
2705
|
+
"byLabel"
|
|
2706
|
+
],
|
|
2707
|
+
"additionalProperties": false
|
|
2708
|
+
}
|
|
2709
|
+
```
|
|
2710
|
+
|
|
2711
|
+
|
|
573
2712
|
#### `ai.vision` — Inspect Pages (Vision)
|
|
574
2713
|
|
|
575
2714
|
Inspect rendered page images with a vision model and return structured JSON matching a schema. The visual counterpart to Extract Data: use it for conclusions that live in the pixels rather than the text (is the document signed? are the photos usable?). Renders PDF, image, or Office/Word inputs; route to specific pages with an ai.split page range to keep it cheap.
|
|
576
2715
|
|
|
2716
|
+
**Behavior and examples:** `eigenpal docs read steps/ai/vision`
|
|
2717
|
+
|
|
577
2718
|
**Durable retry:** Provider request retries are separate; the workflow engine does not durably retry this step.
|
|
578
2719
|
|
|
579
2720
|
**Config** (in `step.with`):
|
|
@@ -588,6 +2729,7 @@ Inspect rendered page images with a vision model and return structured JSON matc
|
|
|
588
2729
|
| `prompt` | string | no | | Optional instruction refining the extraction. The schema drives it when omitted. |
|
|
589
2730
|
| `provider` | string | no | | Provider ID (must support vision). |
|
|
590
2731
|
| `model` | string | no | | Model override (must support vision). |
|
|
2732
|
+
| `reasoningEffort` | `"none"` \| `"minimal"` \| `"low"` \| `"medium"` \| `"high"` \| `"xhigh"` \| `"max"` | no | | Reasoning effort for models that support it. Omit to preserve the current provider default. |
|
|
591
2733
|
| `renderScale` | number | no | | PDF render scale (1.0 = 72 DPI). Raise for small text or weak VLM OCR. Capped at 6 to avoid oversized page rasters. |
|
|
592
2734
|
| `imageQuality` | integer | no | | JPEG quality for rendered pages (default 85). |
|
|
593
2735
|
| `maxPages` | integer | no | | Chunk size: the maximum pages sent to the vision model in a single call. When the inspected range (or the whole document) is larger, it is split into chunks of this size and the per-chunk results are merged by a reduce pass. The reduce follows each field's DESCRIPTION: a boolean described as "present on any page" is OR-ed, one described as "true for every page" is AND-ed, and list fields are concatenated. The merge is most reliable for boolean/scalar claims; for schemas that AGGREGATE long lists across chunks it can still drop or reorder items, so prefer a bounded page range for large list extraction. Default 20, capped at 100. |
|
|
@@ -596,12 +2738,213 @@ Inspect rendered page images with a vision model and return structured JSON matc
|
|
|
596
2738
|
|
|
597
2739
|
> Structured data matching the provided schema. The output also carries a reserved `_vision` key recording the inspected source document ref and the page indices that were rendered, so the UI can re-rasterize exactly those pages client-side. `_vision` is excluded from user-schema validation.
|
|
598
2740
|
|
|
2741
|
+
##### Complete machine-readable schemas
|
|
2742
|
+
|
|
2743
|
+
Config schema:
|
|
2744
|
+
|
|
2745
|
+
```json
|
|
2746
|
+
{
|
|
2747
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2748
|
+
"type": "object",
|
|
2749
|
+
"properties": {
|
|
2750
|
+
"document": {
|
|
2751
|
+
"type": "string",
|
|
2752
|
+
"description": "Template expression resolving to the input file (PDF, image, or Office document)."
|
|
2753
|
+
},
|
|
2754
|
+
"pageFrom": {
|
|
2755
|
+
"description": "First page to inspect (0-based, inclusive). Optional. Accepts a template expression. When omitted, starts from the first page.",
|
|
2756
|
+
"anyOf": [
|
|
2757
|
+
{
|
|
2758
|
+
"type": "integer",
|
|
2759
|
+
"minimum": 0,
|
|
2760
|
+
"maximum": 9007199254740991
|
|
2761
|
+
},
|
|
2762
|
+
{
|
|
2763
|
+
"type": "string"
|
|
2764
|
+
}
|
|
2765
|
+
]
|
|
2766
|
+
},
|
|
2767
|
+
"pageTo": {
|
|
2768
|
+
"description": "Last page to inspect (0-based, inclusive). Optional. Accepts a template expression. When omitted, runs to the last page. When BOTH pageFrom and pageTo are omitted, the whole document is inspected in chunks of maxPages (divide-and-conquer, results merged).",
|
|
2769
|
+
"anyOf": [
|
|
2770
|
+
{
|
|
2771
|
+
"type": "integer",
|
|
2772
|
+
"minimum": 0,
|
|
2773
|
+
"maximum": 9007199254740991
|
|
2774
|
+
},
|
|
2775
|
+
{
|
|
2776
|
+
"type": "string"
|
|
2777
|
+
}
|
|
2778
|
+
]
|
|
2779
|
+
},
|
|
2780
|
+
"pageIndices": {
|
|
2781
|
+
"description": "Explicit list of 0-based page indices to inspect — supports NON-contiguous selections, e.g. [4, 11, 19], or a template resolving to a number[] such as \"{{ steps.label_pages.output.byLabel.photo }}\". Highest priority: when set, pageFrom/pageTo and pages are ignored.",
|
|
2782
|
+
"anyOf": [
|
|
2783
|
+
{
|
|
2784
|
+
"minItems": 1,
|
|
2785
|
+
"type": "array",
|
|
2786
|
+
"items": {
|
|
2787
|
+
"type": "integer",
|
|
2788
|
+
"minimum": 0,
|
|
2789
|
+
"maximum": 9007199254740991
|
|
2790
|
+
}
|
|
2791
|
+
},
|
|
2792
|
+
{
|
|
2793
|
+
"type": "string"
|
|
2794
|
+
}
|
|
2795
|
+
]
|
|
2796
|
+
},
|
|
2797
|
+
"pages": {
|
|
2798
|
+
"description": "DEPRECATED — prefer pageIndices (scattered) or pageFrom/pageTo (contiguous). Which pages to inspect: \"all\", a single [start,end] range, a list of ranges, or a template expression resolving to one of those.",
|
|
2799
|
+
"anyOf": [
|
|
2800
|
+
{
|
|
2801
|
+
"type": "string",
|
|
2802
|
+
"const": "all"
|
|
2803
|
+
},
|
|
2804
|
+
{
|
|
2805
|
+
"type": "array",
|
|
2806
|
+
"items": [
|
|
2807
|
+
{
|
|
2808
|
+
"type": "integer",
|
|
2809
|
+
"minimum": 0,
|
|
2810
|
+
"maximum": 9007199254740991
|
|
2811
|
+
},
|
|
2812
|
+
{
|
|
2813
|
+
"type": "integer",
|
|
2814
|
+
"minimum": 0,
|
|
2815
|
+
"maximum": 9007199254740991
|
|
2816
|
+
}
|
|
2817
|
+
],
|
|
2818
|
+
"description": "Inclusive 0-based [start, end] page range, matching ai.split page_range."
|
|
2819
|
+
},
|
|
2820
|
+
{
|
|
2821
|
+
"type": "array",
|
|
2822
|
+
"items": {
|
|
2823
|
+
"type": "array",
|
|
2824
|
+
"items": [
|
|
2825
|
+
{
|
|
2826
|
+
"type": "integer",
|
|
2827
|
+
"minimum": 0,
|
|
2828
|
+
"maximum": 9007199254740991
|
|
2829
|
+
},
|
|
2830
|
+
{
|
|
2831
|
+
"type": "integer",
|
|
2832
|
+
"minimum": 0,
|
|
2833
|
+
"maximum": 9007199254740991
|
|
2834
|
+
}
|
|
2835
|
+
],
|
|
2836
|
+
"description": "Inclusive 0-based [start, end] page range, matching ai.split page_range."
|
|
2837
|
+
}
|
|
2838
|
+
},
|
|
2839
|
+
{
|
|
2840
|
+
"type": "string"
|
|
2841
|
+
}
|
|
2842
|
+
]
|
|
2843
|
+
},
|
|
2844
|
+
"schema": {
|
|
2845
|
+
"type": "object",
|
|
2846
|
+
"properties": {
|
|
2847
|
+
"type": {
|
|
2848
|
+
"type": "string",
|
|
2849
|
+
"const": "object"
|
|
2850
|
+
},
|
|
2851
|
+
"properties": {
|
|
2852
|
+
"type": "object",
|
|
2853
|
+
"propertyNames": {
|
|
2854
|
+
"type": "string"
|
|
2855
|
+
},
|
|
2856
|
+
"additionalProperties": {}
|
|
2857
|
+
},
|
|
2858
|
+
"required": {
|
|
2859
|
+
"type": "array",
|
|
2860
|
+
"items": {
|
|
2861
|
+
"type": "string"
|
|
2862
|
+
}
|
|
2863
|
+
}
|
|
2864
|
+
},
|
|
2865
|
+
"required": [
|
|
2866
|
+
"type",
|
|
2867
|
+
"properties"
|
|
2868
|
+
],
|
|
2869
|
+
"additionalProperties": false,
|
|
2870
|
+
"description": "JSON Schema defining the structure the vision model should return."
|
|
2871
|
+
},
|
|
2872
|
+
"prompt": {
|
|
2873
|
+
"description": "Optional instruction refining the extraction. The schema drives it when omitted.",
|
|
2874
|
+
"type": "string"
|
|
2875
|
+
},
|
|
2876
|
+
"provider": {
|
|
2877
|
+
"description": "Provider ID (must support vision).",
|
|
2878
|
+
"type": "string"
|
|
2879
|
+
},
|
|
2880
|
+
"model": {
|
|
2881
|
+
"description": "Model override (must support vision).",
|
|
2882
|
+
"type": "string"
|
|
2883
|
+
},
|
|
2884
|
+
"reasoningEffort": {
|
|
2885
|
+
"description": "Reasoning effort for models that support it. Omit to preserve the current provider default.",
|
|
2886
|
+
"type": "string",
|
|
2887
|
+
"enum": [
|
|
2888
|
+
"none",
|
|
2889
|
+
"minimal",
|
|
2890
|
+
"low",
|
|
2891
|
+
"medium",
|
|
2892
|
+
"high",
|
|
2893
|
+
"xhigh",
|
|
2894
|
+
"max"
|
|
2895
|
+
]
|
|
2896
|
+
},
|
|
2897
|
+
"renderScale": {
|
|
2898
|
+
"description": "PDF render scale (1.0 = 72 DPI). Raise for small text or weak VLM OCR. Capped at 6 to avoid oversized page rasters.",
|
|
2899
|
+
"type": "number",
|
|
2900
|
+
"exclusiveMinimum": 0,
|
|
2901
|
+
"maximum": 6
|
|
2902
|
+
},
|
|
2903
|
+
"imageQuality": {
|
|
2904
|
+
"description": "JPEG quality for rendered pages (default 85).",
|
|
2905
|
+
"type": "integer",
|
|
2906
|
+
"minimum": 1,
|
|
2907
|
+
"maximum": 100
|
|
2908
|
+
},
|
|
2909
|
+
"maxPages": {
|
|
2910
|
+
"description": "Chunk size: the maximum pages sent to the vision model in a single call. When the inspected range (or the whole document) is larger, it is split into chunks of this size and the per-chunk results are merged by a reduce pass. The reduce follows each field's DESCRIPTION: a boolean described as \"present on any page\" is OR-ed, one described as \"true for every page\" is AND-ed, and list fields are concatenated. The merge is most reliable for boolean/scalar claims; for schemas that AGGREGATE long lists across chunks it can still drop or reorder items, so prefer a bounded page range for large list extraction. Default 20, capped at 100.",
|
|
2911
|
+
"type": "integer",
|
|
2912
|
+
"exclusiveMinimum": 0,
|
|
2913
|
+
"maximum": 100
|
|
2914
|
+
}
|
|
2915
|
+
},
|
|
2916
|
+
"required": [
|
|
2917
|
+
"document",
|
|
2918
|
+
"schema"
|
|
2919
|
+
],
|
|
2920
|
+
"additionalProperties": false
|
|
2921
|
+
}
|
|
2922
|
+
```
|
|
2923
|
+
|
|
2924
|
+
|
|
2925
|
+
Output schema:
|
|
2926
|
+
|
|
2927
|
+
```json
|
|
2928
|
+
{
|
|
2929
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2930
|
+
"type": "object",
|
|
2931
|
+
"propertyNames": {
|
|
2932
|
+
"type": "string"
|
|
2933
|
+
},
|
|
2934
|
+
"additionalProperties": {},
|
|
2935
|
+
"description": "Structured data matching the provided schema. The output also carries a reserved `_vision` key recording the inspected source document ref and the page indices that were rendered, so the UI can re-rasterize exactly those pages client-side. `_vision` is excluded from user-schema validation."
|
|
2936
|
+
}
|
|
2937
|
+
```
|
|
2938
|
+
|
|
2939
|
+
|
|
599
2940
|
### Transform steps — deterministic data transforms
|
|
600
2941
|
|
|
601
2942
|
#### `transform.set` — Set Value
|
|
602
2943
|
|
|
603
2944
|
Set key-value pairs in the output object
|
|
604
2945
|
|
|
2946
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/set`
|
|
2947
|
+
|
|
605
2948
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
606
2949
|
|
|
607
2950
|
**Config** (in `step.with`):
|
|
@@ -615,10 +2958,61 @@ Set key-value pairs in the output object
|
|
|
615
2958
|
|
|
616
2959
|
> Object with all fields set
|
|
617
2960
|
|
|
2961
|
+
##### Complete machine-readable schemas
|
|
2962
|
+
|
|
2963
|
+
Config schema:
|
|
2964
|
+
|
|
2965
|
+
```json
|
|
2966
|
+
{
|
|
2967
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
2968
|
+
"type": "object",
|
|
2969
|
+
"properties": {
|
|
2970
|
+
"fields": {
|
|
2971
|
+
"type": "object",
|
|
2972
|
+
"propertyNames": {
|
|
2973
|
+
"type": "string"
|
|
2974
|
+
},
|
|
2975
|
+
"additionalProperties": {},
|
|
2976
|
+
"description": "Key-value pairs to set in output"
|
|
2977
|
+
},
|
|
2978
|
+
"input": {
|
|
2979
|
+
"description": "Base object to extend",
|
|
2980
|
+
"type": "object",
|
|
2981
|
+
"propertyNames": {
|
|
2982
|
+
"type": "string"
|
|
2983
|
+
},
|
|
2984
|
+
"additionalProperties": {}
|
|
2985
|
+
}
|
|
2986
|
+
},
|
|
2987
|
+
"required": [
|
|
2988
|
+
"fields"
|
|
2989
|
+
],
|
|
2990
|
+
"additionalProperties": false
|
|
2991
|
+
}
|
|
2992
|
+
```
|
|
2993
|
+
|
|
2994
|
+
|
|
2995
|
+
Output schema:
|
|
2996
|
+
|
|
2997
|
+
```json
|
|
2998
|
+
{
|
|
2999
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3000
|
+
"type": "object",
|
|
3001
|
+
"propertyNames": {
|
|
3002
|
+
"type": "string"
|
|
3003
|
+
},
|
|
3004
|
+
"additionalProperties": {},
|
|
3005
|
+
"description": "Object with all fields set"
|
|
3006
|
+
}
|
|
3007
|
+
```
|
|
3008
|
+
|
|
3009
|
+
|
|
618
3010
|
#### `transform.remove` — Remove Fields
|
|
619
3011
|
|
|
620
3012
|
Remove specified fields from an object
|
|
621
3013
|
|
|
3014
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/remove`
|
|
3015
|
+
|
|
622
3016
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
623
3017
|
|
|
624
3018
|
**Config** (in `step.with`):
|
|
@@ -632,10 +3026,61 @@ Remove specified fields from an object
|
|
|
632
3026
|
|
|
633
3027
|
> Object with fields removed
|
|
634
3028
|
|
|
3029
|
+
##### Complete machine-readable schemas
|
|
3030
|
+
|
|
3031
|
+
Config schema:
|
|
3032
|
+
|
|
3033
|
+
```json
|
|
3034
|
+
{
|
|
3035
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3036
|
+
"type": "object",
|
|
3037
|
+
"properties": {
|
|
3038
|
+
"input": {
|
|
3039
|
+
"description": "Object to remove fields from",
|
|
3040
|
+
"type": "object",
|
|
3041
|
+
"propertyNames": {
|
|
3042
|
+
"type": "string"
|
|
3043
|
+
},
|
|
3044
|
+
"additionalProperties": {}
|
|
3045
|
+
},
|
|
3046
|
+
"fields": {
|
|
3047
|
+
"minItems": 1,
|
|
3048
|
+
"type": "array",
|
|
3049
|
+
"items": {
|
|
3050
|
+
"type": "string"
|
|
3051
|
+
},
|
|
3052
|
+
"description": "Field names to remove"
|
|
3053
|
+
}
|
|
3054
|
+
},
|
|
3055
|
+
"required": [
|
|
3056
|
+
"fields"
|
|
3057
|
+
],
|
|
3058
|
+
"additionalProperties": false
|
|
3059
|
+
}
|
|
3060
|
+
```
|
|
3061
|
+
|
|
3062
|
+
|
|
3063
|
+
Output schema:
|
|
3064
|
+
|
|
3065
|
+
```json
|
|
3066
|
+
{
|
|
3067
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3068
|
+
"type": "object",
|
|
3069
|
+
"propertyNames": {
|
|
3070
|
+
"type": "string"
|
|
3071
|
+
},
|
|
3072
|
+
"additionalProperties": {},
|
|
3073
|
+
"description": "Object with fields removed"
|
|
3074
|
+
}
|
|
3075
|
+
```
|
|
3076
|
+
|
|
3077
|
+
|
|
635
3078
|
#### `transform.combine` — Combine Data
|
|
636
3079
|
|
|
637
3080
|
Merge multiple objects or concatenate arrays
|
|
638
3081
|
|
|
3082
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/combine`
|
|
3083
|
+
|
|
639
3084
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
640
3085
|
|
|
641
3086
|
**Config** (in `step.with`):
|
|
@@ -650,10 +3095,76 @@ Merge multiple objects or concatenate arrays
|
|
|
650
3095
|
|
|
651
3096
|
> Combined result
|
|
652
3097
|
|
|
3098
|
+
##### Complete machine-readable schemas
|
|
3099
|
+
|
|
3100
|
+
Config schema:
|
|
3101
|
+
|
|
3102
|
+
```json
|
|
3103
|
+
{
|
|
3104
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3105
|
+
"type": "object",
|
|
3106
|
+
"properties": {
|
|
3107
|
+
"sources": {
|
|
3108
|
+
"minItems": 1,
|
|
3109
|
+
"type": "array",
|
|
3110
|
+
"items": {
|
|
3111
|
+
"type": "string"
|
|
3112
|
+
},
|
|
3113
|
+
"description": "Template expressions for sources to combine"
|
|
3114
|
+
},
|
|
3115
|
+
"target": {
|
|
3116
|
+
"type": "string",
|
|
3117
|
+
"minLength": 1,
|
|
3118
|
+
"description": "Target path in output"
|
|
3119
|
+
},
|
|
3120
|
+
"mode": {
|
|
3121
|
+
"default": "merge",
|
|
3122
|
+
"type": "string",
|
|
3123
|
+
"enum": [
|
|
3124
|
+
"merge",
|
|
3125
|
+
"concat",
|
|
3126
|
+
"deep"
|
|
3127
|
+
]
|
|
3128
|
+
}
|
|
3129
|
+
},
|
|
3130
|
+
"required": [
|
|
3131
|
+
"sources",
|
|
3132
|
+
"target"
|
|
3133
|
+
],
|
|
3134
|
+
"additionalProperties": false
|
|
3135
|
+
}
|
|
3136
|
+
```
|
|
3137
|
+
|
|
3138
|
+
|
|
3139
|
+
Output schema:
|
|
3140
|
+
|
|
3141
|
+
```json
|
|
3142
|
+
{
|
|
3143
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3144
|
+
"anyOf": [
|
|
3145
|
+
{
|
|
3146
|
+
"type": "object",
|
|
3147
|
+
"propertyNames": {
|
|
3148
|
+
"type": "string"
|
|
3149
|
+
},
|
|
3150
|
+
"additionalProperties": {}
|
|
3151
|
+
},
|
|
3152
|
+
{
|
|
3153
|
+
"type": "array",
|
|
3154
|
+
"items": {}
|
|
3155
|
+
}
|
|
3156
|
+
],
|
|
3157
|
+
"description": "Combined result"
|
|
3158
|
+
}
|
|
3159
|
+
```
|
|
3160
|
+
|
|
3161
|
+
|
|
653
3162
|
#### `transform.split` — Split Data
|
|
654
3163
|
|
|
655
3164
|
Split a string by delimiter or extract keys from an object
|
|
656
3165
|
|
|
3166
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/split`
|
|
3167
|
+
|
|
657
3168
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
658
3169
|
|
|
659
3170
|
**Config** (in `step.with`):
|
|
@@ -671,10 +3182,108 @@ Split a string by delimiter or extract keys from an object
|
|
|
671
3182
|
|
|
672
3183
|
> Split result - array or extracted/remaining object
|
|
673
3184
|
|
|
3185
|
+
##### Complete machine-readable schemas
|
|
3186
|
+
|
|
3187
|
+
Config schema:
|
|
3188
|
+
|
|
3189
|
+
```json
|
|
3190
|
+
{
|
|
3191
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3192
|
+
"type": "object",
|
|
3193
|
+
"properties": {
|
|
3194
|
+
"input": {
|
|
3195
|
+
"description": "String or object to split",
|
|
3196
|
+
"anyOf": [
|
|
3197
|
+
{
|
|
3198
|
+
"type": "string"
|
|
3199
|
+
},
|
|
3200
|
+
{
|
|
3201
|
+
"type": "object",
|
|
3202
|
+
"propertyNames": {
|
|
3203
|
+
"type": "string"
|
|
3204
|
+
},
|
|
3205
|
+
"additionalProperties": {}
|
|
3206
|
+
}
|
|
3207
|
+
]
|
|
3208
|
+
},
|
|
3209
|
+
"source": {
|
|
3210
|
+
"description": "Template expression for source",
|
|
3211
|
+
"type": "string"
|
|
3212
|
+
},
|
|
3213
|
+
"by": {
|
|
3214
|
+
"description": "Delimiter for string splitting",
|
|
3215
|
+
"default": ",",
|
|
3216
|
+
"type": "string"
|
|
3217
|
+
},
|
|
3218
|
+
"delimiter": {
|
|
3219
|
+
"description": "Alias for by",
|
|
3220
|
+
"type": "string"
|
|
3221
|
+
},
|
|
3222
|
+
"limit": {
|
|
3223
|
+
"description": "Max number of splits",
|
|
3224
|
+
"type": "number"
|
|
3225
|
+
},
|
|
3226
|
+
"keys": {
|
|
3227
|
+
"description": "Keys to extract from object",
|
|
3228
|
+
"type": "array",
|
|
3229
|
+
"items": {
|
|
3230
|
+
"type": "string"
|
|
3231
|
+
}
|
|
3232
|
+
}
|
|
3233
|
+
},
|
|
3234
|
+
"additionalProperties": false
|
|
3235
|
+
}
|
|
3236
|
+
```
|
|
3237
|
+
|
|
3238
|
+
|
|
3239
|
+
Output schema:
|
|
3240
|
+
|
|
3241
|
+
```json
|
|
3242
|
+
{
|
|
3243
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3244
|
+
"anyOf": [
|
|
3245
|
+
{
|
|
3246
|
+
"type": "array",
|
|
3247
|
+
"items": {
|
|
3248
|
+
"type": "string"
|
|
3249
|
+
}
|
|
3250
|
+
},
|
|
3251
|
+
{
|
|
3252
|
+
"type": "object",
|
|
3253
|
+
"properties": {
|
|
3254
|
+
"extracted": {
|
|
3255
|
+
"type": "object",
|
|
3256
|
+
"propertyNames": {
|
|
3257
|
+
"type": "string"
|
|
3258
|
+
},
|
|
3259
|
+
"additionalProperties": {}
|
|
3260
|
+
},
|
|
3261
|
+
"remaining": {
|
|
3262
|
+
"type": "object",
|
|
3263
|
+
"propertyNames": {
|
|
3264
|
+
"type": "string"
|
|
3265
|
+
},
|
|
3266
|
+
"additionalProperties": {}
|
|
3267
|
+
}
|
|
3268
|
+
},
|
|
3269
|
+
"required": [
|
|
3270
|
+
"extracted",
|
|
3271
|
+
"remaining"
|
|
3272
|
+
],
|
|
3273
|
+
"additionalProperties": false
|
|
3274
|
+
}
|
|
3275
|
+
],
|
|
3276
|
+
"description": "Split result - array or extracted/remaining object"
|
|
3277
|
+
}
|
|
3278
|
+
```
|
|
3279
|
+
|
|
3280
|
+
|
|
674
3281
|
#### `transform.merge` — Merge Inputs
|
|
675
3282
|
|
|
676
3283
|
Merge multiple named inputs into a single output
|
|
677
3284
|
|
|
3285
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/merge`
|
|
3286
|
+
|
|
678
3287
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
679
3288
|
|
|
680
3289
|
**Config** (in `step.with`):
|
|
@@ -692,10 +3301,65 @@ Merge multiple named inputs into a single output
|
|
|
692
3301
|
| `count` | number | yes | | |
|
|
693
3302
|
| `merged` | record<string, unknown> | no | | |
|
|
694
3303
|
|
|
3304
|
+
##### Complete machine-readable schemas
|
|
3305
|
+
|
|
3306
|
+
Config schema:
|
|
3307
|
+
|
|
3308
|
+
```json
|
|
3309
|
+
{
|
|
3310
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3311
|
+
"type": "object",
|
|
3312
|
+
"properties": {
|
|
3313
|
+
"preservePortNames": {
|
|
3314
|
+
"default": true,
|
|
3315
|
+
"type": "boolean"
|
|
3316
|
+
},
|
|
3317
|
+
"outputKey": {
|
|
3318
|
+
"type": "string"
|
|
3319
|
+
}
|
|
3320
|
+
},
|
|
3321
|
+
"additionalProperties": false
|
|
3322
|
+
}
|
|
3323
|
+
```
|
|
3324
|
+
|
|
3325
|
+
|
|
3326
|
+
Output schema:
|
|
3327
|
+
|
|
3328
|
+
```json
|
|
3329
|
+
{
|
|
3330
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3331
|
+
"type": "object",
|
|
3332
|
+
"properties": {
|
|
3333
|
+
"items": {
|
|
3334
|
+
"type": "array",
|
|
3335
|
+
"items": {}
|
|
3336
|
+
},
|
|
3337
|
+
"count": {
|
|
3338
|
+
"type": "number"
|
|
3339
|
+
},
|
|
3340
|
+
"merged": {
|
|
3341
|
+
"type": "object",
|
|
3342
|
+
"propertyNames": {
|
|
3343
|
+
"type": "string"
|
|
3344
|
+
},
|
|
3345
|
+
"additionalProperties": {}
|
|
3346
|
+
}
|
|
3347
|
+
},
|
|
3348
|
+
"required": [
|
|
3349
|
+
"items",
|
|
3350
|
+
"count"
|
|
3351
|
+
],
|
|
3352
|
+
"additionalProperties": false
|
|
3353
|
+
}
|
|
3354
|
+
```
|
|
3355
|
+
|
|
3356
|
+
|
|
695
3357
|
#### `transform.template` — Fill Template
|
|
696
3358
|
|
|
697
3359
|
Fill a DOCX or XLSX workspace template with data from previous steps. Reference a tmpl_... id, or a local ./templates/file.xlsx path (CLI push uploads it). Git agent templates use the fill-template skill instead and cannot be referenced here. File IDs (file_...) are not accepted. Placeholders are auto-detected. In the Office file use {placeholder} and {table:array.prop}; {{placeholder}} in an XLSX file is rejected ({{ }} is YAML Liquid only).
|
|
698
3360
|
|
|
3361
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/template`
|
|
3362
|
+
|
|
699
3363
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
700
3364
|
|
|
701
3365
|
**Config** (in `step.with`):
|
|
@@ -716,10 +3380,86 @@ Fill a DOCX or XLSX workspace template with data from previous steps. Reference
|
|
|
716
3380
|
| --- | --- | --- | --- | --- |
|
|
717
3381
|
| `fileId` | string | yes | | File ID from files table |
|
|
718
3382
|
|
|
3383
|
+
##### Complete machine-readable schemas
|
|
3384
|
+
|
|
3385
|
+
Config schema:
|
|
3386
|
+
|
|
3387
|
+
```json
|
|
3388
|
+
{
|
|
3389
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3390
|
+
"type": "object",
|
|
3391
|
+
"properties": {
|
|
3392
|
+
"templateId": {
|
|
3393
|
+
"description": "Workspace template ID (tmpl_...) from the Templates table (DOCX or XLSX). Git agent templates cannot be referenced here. File IDs (file_...) are not accepted. Mutually exclusive with `template`.",
|
|
3394
|
+
"type": "string",
|
|
3395
|
+
"pattern": "^tmpl_[A-Za-z0-9_-]{21}$"
|
|
3396
|
+
},
|
|
3397
|
+
"template": {
|
|
3398
|
+
"description": "Local DOCX or XLSX path relative to the workflow YAML (e.g. ./templates/foo.xlsx). CLI push keeps the real path inside the workflow project unless you pass --allow-external-templates, uploads unmatched files as new tmpl_ resources, and does not rewrite the source YAML. Mutually exclusive with templateId.",
|
|
3399
|
+
"type": "string"
|
|
3400
|
+
},
|
|
3401
|
+
"templateRevisionId": {
|
|
3402
|
+
"description": "Immutable template revision ID (tmpr_...). Pin this when executions must keep using the same template bytes after replacements. Requires templateId; local `template` paths are pinned automatically on push.",
|
|
3403
|
+
"type": "string",
|
|
3404
|
+
"pattern": "^tmpr_[A-Za-z0-9_-]{21}$"
|
|
3405
|
+
},
|
|
3406
|
+
"data": {
|
|
3407
|
+
"type": "object",
|
|
3408
|
+
"propertyNames": {
|
|
3409
|
+
"type": "string"
|
|
3410
|
+
},
|
|
3411
|
+
"additionalProperties": {},
|
|
3412
|
+
"description": "Data object to merge into template. Each key must be explicitly defined - cannot pass a whole object as single expression."
|
|
3413
|
+
},
|
|
3414
|
+
"outputFilename": {
|
|
3415
|
+
"description": "Output filename - supports {{field}} syntax; .docx or .xlsx extension is added if omitted",
|
|
3416
|
+
"type": "string"
|
|
3417
|
+
},
|
|
3418
|
+
"highlightNotFound": {
|
|
3419
|
+
"description": "Highlight missing variables with red-colored text in the output document (DOCX only; ignored for XLSX)",
|
|
3420
|
+
"default": true,
|
|
3421
|
+
"type": "boolean"
|
|
3422
|
+
},
|
|
3423
|
+
"notFoundText": {
|
|
3424
|
+
"description": "Text to display for missing variables when highlightNotFound is enabled (DOCX only; ignored for XLSX)",
|
|
3425
|
+
"default": "NOT FOUND",
|
|
3426
|
+
"type": "string"
|
|
3427
|
+
}
|
|
3428
|
+
},
|
|
3429
|
+
"required": [
|
|
3430
|
+
"data"
|
|
3431
|
+
],
|
|
3432
|
+
"additionalProperties": false
|
|
3433
|
+
}
|
|
3434
|
+
```
|
|
3435
|
+
|
|
3436
|
+
|
|
3437
|
+
Output schema:
|
|
3438
|
+
|
|
3439
|
+
```json
|
|
3440
|
+
{
|
|
3441
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3442
|
+
"type": "object",
|
|
3443
|
+
"properties": {
|
|
3444
|
+
"fileId": {
|
|
3445
|
+
"type": "string",
|
|
3446
|
+
"description": "File ID from files table"
|
|
3447
|
+
}
|
|
3448
|
+
},
|
|
3449
|
+
"required": [
|
|
3450
|
+
"fileId"
|
|
3451
|
+
],
|
|
3452
|
+
"additionalProperties": false
|
|
3453
|
+
}
|
|
3454
|
+
```
|
|
3455
|
+
|
|
3456
|
+
|
|
719
3457
|
#### `transform.pdf-embed` — Embed PDF Text
|
|
720
3458
|
|
|
721
3459
|
Embed OCR text layer into scanned PDFs/images to make them searchable
|
|
722
3460
|
|
|
3461
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/pdf-embed`
|
|
3462
|
+
|
|
723
3463
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
724
3464
|
|
|
725
3465
|
**Config** (in `step.with`):
|
|
@@ -740,22 +3480,97 @@ Embed OCR text layer into scanned PDFs/images to make them searchable
|
|
|
740
3480
|
| `wordCount` | number | yes | | Number of words embedded |
|
|
741
3481
|
| `text` | string | yes | | Extracted text from the document |
|
|
742
3482
|
|
|
743
|
-
|
|
3483
|
+
##### Complete machine-readable schemas
|
|
744
3484
|
|
|
745
|
-
|
|
3485
|
+
Config schema:
|
|
746
3486
|
|
|
747
|
-
|
|
3487
|
+
```json
|
|
3488
|
+
{
|
|
3489
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3490
|
+
"type": "object",
|
|
3491
|
+
"properties": {
|
|
3492
|
+
"input": {
|
|
3493
|
+
"type": "string",
|
|
3494
|
+
"description": "File input - template expression e.g. {{input.document}}"
|
|
3495
|
+
},
|
|
3496
|
+
"parseResult": {
|
|
3497
|
+
"type": "string",
|
|
3498
|
+
"description": "Parse result - template expression e.g. {{steps.parse.output}}"
|
|
3499
|
+
},
|
|
3500
|
+
"outputFilename": {
|
|
3501
|
+
"description": "Output filename - supports {{filename}} syntax",
|
|
3502
|
+
"type": "string"
|
|
3503
|
+
},
|
|
3504
|
+
"confidenceThreshold": {
|
|
3505
|
+
"description": "Minimum OCR confidence (0-1) to include a word",
|
|
3506
|
+
"default": 0.7,
|
|
3507
|
+
"type": "number",
|
|
3508
|
+
"minimum": 0,
|
|
3509
|
+
"maximum": 1
|
|
3510
|
+
}
|
|
3511
|
+
},
|
|
3512
|
+
"required": [
|
|
3513
|
+
"input",
|
|
3514
|
+
"parseResult"
|
|
3515
|
+
],
|
|
3516
|
+
"additionalProperties": false
|
|
3517
|
+
}
|
|
3518
|
+
```
|
|
748
3519
|
|
|
749
|
-
**Config** (in `step.with`):
|
|
750
3520
|
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
3521
|
+
Output schema:
|
|
3522
|
+
|
|
3523
|
+
```json
|
|
3524
|
+
{
|
|
3525
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3526
|
+
"type": "object",
|
|
3527
|
+
"properties": {
|
|
3528
|
+
"fileId": {
|
|
3529
|
+
"type": "string",
|
|
3530
|
+
"description": "File ID from files table"
|
|
3531
|
+
},
|
|
3532
|
+
"pageCount": {
|
|
3533
|
+
"type": "number",
|
|
3534
|
+
"description": "Number of pages in the output PDF"
|
|
3535
|
+
},
|
|
3536
|
+
"wordCount": {
|
|
3537
|
+
"type": "number",
|
|
3538
|
+
"description": "Number of words embedded"
|
|
3539
|
+
},
|
|
3540
|
+
"text": {
|
|
3541
|
+
"type": "string",
|
|
3542
|
+
"description": "Extracted text from the document"
|
|
3543
|
+
}
|
|
3544
|
+
},
|
|
3545
|
+
"required": [
|
|
3546
|
+
"fileId",
|
|
3547
|
+
"pageCount",
|
|
3548
|
+
"wordCount",
|
|
3549
|
+
"text"
|
|
3550
|
+
],
|
|
3551
|
+
"additionalProperties": false
|
|
3552
|
+
}
|
|
3553
|
+
```
|
|
3554
|
+
|
|
3555
|
+
|
|
3556
|
+
#### `transform.xlsx-to-json` — Spreadsheet to JSON
|
|
3557
|
+
|
|
3558
|
+
Convert an XLS or XLSX spreadsheet to a JSON array of row objects. Supports headerless files, named or positional columns, displayed text, and a single rectangular range.
|
|
3559
|
+
|
|
3560
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/xlsx-to-json`
|
|
3561
|
+
|
|
3562
|
+
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
3563
|
+
|
|
3564
|
+
**Config** (in `step.with`):
|
|
3565
|
+
|
|
3566
|
+
| Field | Type | Required | Default | Description |
|
|
3567
|
+
| --- | --- | --- | --- | --- |
|
|
3568
|
+
| `input` | string | yes | | File input - template expression e.g. {{input.document}} resolving to a scoped $file artifact at runtime |
|
|
3569
|
+
| `sheet` | integer \| string | no | | Sheet to read: 0-based index or exact sheet name. Omit for the first sheet. |
|
|
3570
|
+
| `outputCsv` | boolean | no | `false` | If true, also write CSV to storage and include fileId. Zero-config uses the historical full-sheet SheetJS CSV. When columns, range, headerRow, valueMode, blankCells, or blankRows are set, CSV matches that projection. |
|
|
3571
|
+
| `includeMetadata` | boolean | no | `false` | If true, include sheet metadata and diagnostics in the step output. Omit to keep output as rows (and fileId when outputCsv is true). Warnings are still logged when this is false. |
|
|
3572
|
+
| `outputFilename` | string | no | | Output CSV filename when outputCsv is true - supports LiquidJS e.g. {{filename}}.csv |
|
|
3573
|
+
| `headerRow` | `false` \| integer | no | | Header row: a positive 1-based Excel row, or false to keep the first effective row as data. Omit to use the first effective row as the header. When range is set and this is omitted, the first range row is the header. |
|
|
759
3574
|
| `columns` | array<object> | no | | Ordered output columns. Each item needs a key and exactly one source: index (0-based absolute column) or header (exact displayed header text). Named header sources require a header row. Omit to keep every column in the effective range. |
|
|
760
3575
|
| `valueMode` | `"raw"` \| `"displayed"` | no | `"raw"` | raw (default) returns typed cached cell values. displayed returns formatted cell text (dates, leading zeros, punctuation, diacritics, embedded newlines). Formulas are never calculated. |
|
|
761
3576
|
| `range` | string | no | | Optional rectangular A1 range without a sheet qualifier, e.g. A1:D20. Disjoint ranges are rejected. |
|
|
@@ -772,10 +3587,294 @@ Convert an XLS or XLSX spreadsheet to a JSON array of row objects. Supports head
|
|
|
772
3587
|
| `sheet` | object | no | | Selected sheet metadata after projection. Present only when includeMetadata is true. |
|
|
773
3588
|
| `diagnostics` | array<object> | no | | Non-fatal warnings collected while reading the sheet. Present only when includeMetadata is true. Warnings are still logged when metadata is omitted. |
|
|
774
3589
|
|
|
3590
|
+
##### Complete machine-readable schemas
|
|
3591
|
+
|
|
3592
|
+
Config schema:
|
|
3593
|
+
|
|
3594
|
+
```json
|
|
3595
|
+
{
|
|
3596
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3597
|
+
"type": "object",
|
|
3598
|
+
"properties": {
|
|
3599
|
+
"input": {
|
|
3600
|
+
"type": "string",
|
|
3601
|
+
"minLength": 1,
|
|
3602
|
+
"description": "File input - template expression e.g. {{input.document}} resolving to a scoped $file artifact at runtime"
|
|
3603
|
+
},
|
|
3604
|
+
"sheet": {
|
|
3605
|
+
"description": "Sheet to read: 0-based index or exact sheet name. Omit for the first sheet.",
|
|
3606
|
+
"anyOf": [
|
|
3607
|
+
{
|
|
3608
|
+
"type": "integer",
|
|
3609
|
+
"minimum": 0,
|
|
3610
|
+
"maximum": 9007199254740991
|
|
3611
|
+
},
|
|
3612
|
+
{
|
|
3613
|
+
"type": "string"
|
|
3614
|
+
}
|
|
3615
|
+
]
|
|
3616
|
+
},
|
|
3617
|
+
"outputCsv": {
|
|
3618
|
+
"description": "If true, also write CSV to storage and include fileId. Zero-config uses the historical full-sheet SheetJS CSV. When columns, range, headerRow, valueMode, blankCells, or blankRows are set, CSV matches that projection.",
|
|
3619
|
+
"default": false,
|
|
3620
|
+
"type": "boolean"
|
|
3621
|
+
},
|
|
3622
|
+
"includeMetadata": {
|
|
3623
|
+
"description": "If true, include sheet metadata and diagnostics in the step output. Omit to keep output as rows (and fileId when outputCsv is true). Warnings are still logged when this is false.",
|
|
3624
|
+
"default": false,
|
|
3625
|
+
"type": "boolean"
|
|
3626
|
+
},
|
|
3627
|
+
"outputFilename": {
|
|
3628
|
+
"description": "Output CSV filename when outputCsv is true - supports LiquidJS e.g. {{filename}}.csv",
|
|
3629
|
+
"type": "string"
|
|
3630
|
+
},
|
|
3631
|
+
"headerRow": {
|
|
3632
|
+
"description": "Header row: a positive 1-based Excel row, or false to keep the first effective row as data. Omit to use the first effective row as the header. When range is set and this is omitted, the first range row is the header.",
|
|
3633
|
+
"anyOf": [
|
|
3634
|
+
{
|
|
3635
|
+
"type": "boolean",
|
|
3636
|
+
"const": false
|
|
3637
|
+
},
|
|
3638
|
+
{
|
|
3639
|
+
"type": "integer",
|
|
3640
|
+
"minimum": 1,
|
|
3641
|
+
"maximum": 9007199254740991
|
|
3642
|
+
}
|
|
3643
|
+
]
|
|
3644
|
+
},
|
|
3645
|
+
"columns": {
|
|
3646
|
+
"description": "Ordered output columns. Each item needs a key and exactly one source: index (0-based absolute column) or header (exact displayed header text). Named header sources require a header row. Omit to keep every column in the effective range.",
|
|
3647
|
+
"minItems": 1,
|
|
3648
|
+
"type": "array",
|
|
3649
|
+
"items": {
|
|
3650
|
+
"type": "object",
|
|
3651
|
+
"properties": {
|
|
3652
|
+
"key": {
|
|
3653
|
+
"type": "string",
|
|
3654
|
+
"minLength": 1,
|
|
3655
|
+
"description": "Output object key for this column"
|
|
3656
|
+
},
|
|
3657
|
+
"index": {
|
|
3658
|
+
"description": "0-based absolute worksheet column. Use instead of header.",
|
|
3659
|
+
"type": "integer",
|
|
3660
|
+
"minimum": 0,
|
|
3661
|
+
"maximum": 9007199254740991
|
|
3662
|
+
},
|
|
3663
|
+
"header": {
|
|
3664
|
+
"description": "Exact displayed header text. Use instead of index. Requires a header row.",
|
|
3665
|
+
"type": "string",
|
|
3666
|
+
"minLength": 1
|
|
3667
|
+
},
|
|
3668
|
+
"occurrence": {
|
|
3669
|
+
"description": "1-based occurrence of header when the same displayed header appears more than once. Only valid with header.",
|
|
3670
|
+
"type": "integer",
|
|
3671
|
+
"minimum": 1,
|
|
3672
|
+
"maximum": 9007199254740991
|
|
3673
|
+
}
|
|
3674
|
+
},
|
|
3675
|
+
"required": [
|
|
3676
|
+
"key"
|
|
3677
|
+
],
|
|
3678
|
+
"additionalProperties": false
|
|
3679
|
+
}
|
|
3680
|
+
},
|
|
3681
|
+
"valueMode": {
|
|
3682
|
+
"description": "raw (default) returns typed cached cell values. displayed returns formatted cell text (dates, leading zeros, punctuation, diacritics, embedded newlines). Formulas are never calculated.",
|
|
3683
|
+
"default": "raw",
|
|
3684
|
+
"type": "string",
|
|
3685
|
+
"enum": [
|
|
3686
|
+
"raw",
|
|
3687
|
+
"displayed"
|
|
3688
|
+
]
|
|
3689
|
+
},
|
|
3690
|
+
"range": {
|
|
3691
|
+
"description": "Optional rectangular A1 range without a sheet qualifier, e.g. A1:D20. Disjoint ranges are rejected.",
|
|
3692
|
+
"type": "string",
|
|
3693
|
+
"minLength": 1
|
|
3694
|
+
},
|
|
3695
|
+
"blankCells": {
|
|
3696
|
+
"description": "How truly empty cells appear in each row object. Default empty-string. 0, false, and a formula that cached an empty string are not empty cells.",
|
|
3697
|
+
"default": "empty-string",
|
|
3698
|
+
"type": "string",
|
|
3699
|
+
"enum": [
|
|
3700
|
+
"empty-string",
|
|
3701
|
+
"null",
|
|
3702
|
+
"omit"
|
|
3703
|
+
]
|
|
3704
|
+
},
|
|
3705
|
+
"blankRows": {
|
|
3706
|
+
"description": "skip (default) drops rows whose projected columns are all truly empty. keep retains them. Detection uses projected columns only.",
|
|
3707
|
+
"default": "skip",
|
|
3708
|
+
"type": "string",
|
|
3709
|
+
"enum": [
|
|
3710
|
+
"skip",
|
|
3711
|
+
"keep"
|
|
3712
|
+
]
|
|
3713
|
+
},
|
|
3714
|
+
"limits": {
|
|
3715
|
+
"type": "object",
|
|
3716
|
+
"properties": {
|
|
3717
|
+
"maxInputBytes": {
|
|
3718
|
+
"description": "Maximum input file size in bytes. Omitted uses the server default (52,428,800, 50 MiB). Cannot exceed it.",
|
|
3719
|
+
"type": "integer",
|
|
3720
|
+
"minimum": 1,
|
|
3721
|
+
"maximum": 52428800
|
|
3722
|
+
},
|
|
3723
|
+
"maxRows": {
|
|
3724
|
+
"description": "Maximum rows in the selected range. Omitted uses the server default (100,000). Cannot exceed it.",
|
|
3725
|
+
"type": "integer",
|
|
3726
|
+
"minimum": 1,
|
|
3727
|
+
"maximum": 100000
|
|
3728
|
+
},
|
|
3729
|
+
"maxColumns": {
|
|
3730
|
+
"description": "Maximum columns in the selected range. Omitted uses the server default (256). Cannot exceed it.",
|
|
3731
|
+
"type": "integer",
|
|
3732
|
+
"minimum": 1,
|
|
3733
|
+
"maximum": 256
|
|
3734
|
+
},
|
|
3735
|
+
"maxCells": {
|
|
3736
|
+
"description": "Maximum cells in the selected range. Omitted uses the server default (5,000,000). Cannot exceed it.",
|
|
3737
|
+
"type": "integer",
|
|
3738
|
+
"minimum": 1,
|
|
3739
|
+
"maximum": 5000000
|
|
3740
|
+
}
|
|
3741
|
+
},
|
|
3742
|
+
"additionalProperties": false,
|
|
3743
|
+
"description": "Optional workload caps that can only lower the server defaults. Omitted fields use the server defaults."
|
|
3744
|
+
}
|
|
3745
|
+
},
|
|
3746
|
+
"required": [
|
|
3747
|
+
"input"
|
|
3748
|
+
],
|
|
3749
|
+
"additionalProperties": false
|
|
3750
|
+
}
|
|
3751
|
+
```
|
|
3752
|
+
|
|
3753
|
+
|
|
3754
|
+
Output schema:
|
|
3755
|
+
|
|
3756
|
+
```json
|
|
3757
|
+
{
|
|
3758
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3759
|
+
"type": "object",
|
|
3760
|
+
"properties": {
|
|
3761
|
+
"rows": {
|
|
3762
|
+
"type": "array",
|
|
3763
|
+
"items": {
|
|
3764
|
+
"type": "object",
|
|
3765
|
+
"propertyNames": {
|
|
3766
|
+
"type": "string"
|
|
3767
|
+
},
|
|
3768
|
+
"additionalProperties": {}
|
|
3769
|
+
},
|
|
3770
|
+
"description": "Array of row objects (first row = headers as keys unless headerRow is false)"
|
|
3771
|
+
},
|
|
3772
|
+
"fileId": {
|
|
3773
|
+
"description": "File ID of stored CSV when outputCsv is true",
|
|
3774
|
+
"type": "string"
|
|
3775
|
+
},
|
|
3776
|
+
"sheet": {
|
|
3777
|
+
"description": "Selected sheet metadata after projection. Present only when includeMetadata is true.",
|
|
3778
|
+
"type": "object",
|
|
3779
|
+
"properties": {
|
|
3780
|
+
"name": {
|
|
3781
|
+
"type": "string",
|
|
3782
|
+
"description": "Selected sheet name"
|
|
3783
|
+
},
|
|
3784
|
+
"index": {
|
|
3785
|
+
"type": "integer",
|
|
3786
|
+
"minimum": 0,
|
|
3787
|
+
"maximum": 9007199254740991,
|
|
3788
|
+
"description": "0-based sheet index in the workbook"
|
|
3789
|
+
},
|
|
3790
|
+
"range": {
|
|
3791
|
+
"type": "string",
|
|
3792
|
+
"description": "Effective rectangular A1 range that was read"
|
|
3793
|
+
},
|
|
3794
|
+
"rowCount": {
|
|
3795
|
+
"type": "integer",
|
|
3796
|
+
"minimum": 0,
|
|
3797
|
+
"maximum": 9007199254740991,
|
|
3798
|
+
"description": "Number of data rows returned in rows"
|
|
3799
|
+
},
|
|
3800
|
+
"columns": {
|
|
3801
|
+
"type": "array",
|
|
3802
|
+
"items": {
|
|
3803
|
+
"type": "string"
|
|
3804
|
+
},
|
|
3805
|
+
"description": "Output keys in order"
|
|
3806
|
+
}
|
|
3807
|
+
},
|
|
3808
|
+
"required": [
|
|
3809
|
+
"name",
|
|
3810
|
+
"index",
|
|
3811
|
+
"range",
|
|
3812
|
+
"rowCount",
|
|
3813
|
+
"columns"
|
|
3814
|
+
],
|
|
3815
|
+
"additionalProperties": false
|
|
3816
|
+
},
|
|
3817
|
+
"diagnostics": {
|
|
3818
|
+
"description": "Non-fatal warnings collected while reading the sheet. Present only when includeMetadata is true. Warnings are still logged when metadata is omitted.",
|
|
3819
|
+
"type": "array",
|
|
3820
|
+
"items": {
|
|
3821
|
+
"type": "object",
|
|
3822
|
+
"properties": {
|
|
3823
|
+
"severity": {
|
|
3824
|
+
"type": "string",
|
|
3825
|
+
"const": "warning",
|
|
3826
|
+
"description": "Diagnostic severity"
|
|
3827
|
+
},
|
|
3828
|
+
"code": {
|
|
3829
|
+
"type": "string",
|
|
3830
|
+
"minLength": 1,
|
|
3831
|
+
"description": "Stable diagnostic code"
|
|
3832
|
+
},
|
|
3833
|
+
"message": {
|
|
3834
|
+
"type": "string",
|
|
3835
|
+
"minLength": 1,
|
|
3836
|
+
"description": "Human-readable diagnostic message"
|
|
3837
|
+
},
|
|
3838
|
+
"cell": {
|
|
3839
|
+
"description": "A1 address of the related cell when applicable",
|
|
3840
|
+
"type": "string"
|
|
3841
|
+
},
|
|
3842
|
+
"row": {
|
|
3843
|
+
"description": "1-based Excel row when applicable",
|
|
3844
|
+
"type": "integer",
|
|
3845
|
+
"minimum": -9007199254740991,
|
|
3846
|
+
"maximum": 9007199254740991
|
|
3847
|
+
},
|
|
3848
|
+
"column": {
|
|
3849
|
+
"description": "0-based worksheet column when applicable",
|
|
3850
|
+
"type": "integer",
|
|
3851
|
+
"minimum": -9007199254740991,
|
|
3852
|
+
"maximum": 9007199254740991
|
|
3853
|
+
}
|
|
3854
|
+
},
|
|
3855
|
+
"required": [
|
|
3856
|
+
"severity",
|
|
3857
|
+
"code",
|
|
3858
|
+
"message"
|
|
3859
|
+
],
|
|
3860
|
+
"additionalProperties": false
|
|
3861
|
+
}
|
|
3862
|
+
}
|
|
3863
|
+
},
|
|
3864
|
+
"required": [
|
|
3865
|
+
"rows"
|
|
3866
|
+
],
|
|
3867
|
+
"additionalProperties": false
|
|
3868
|
+
}
|
|
3869
|
+
```
|
|
3870
|
+
|
|
3871
|
+
|
|
775
3872
|
#### `transform.json-to-xlsx` — JSON to XLSX
|
|
776
3873
|
|
|
777
3874
|
Convert JSON rows into an XLSX spreadsheet. Pair with transform.xlsx-to-json. Formula-looking strings stay text; nested cell values are rejected.
|
|
778
3875
|
|
|
3876
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/json-to-xlsx`
|
|
3877
|
+
|
|
779
3878
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
780
3879
|
|
|
781
3880
|
**Config** (in `step.with`):
|
|
@@ -797,9 +3896,257 @@ Convert JSON rows into an XLSX spreadsheet. Pair with transform.xlsx-to-json. Fo
|
|
|
797
3896
|
| `sheetCount` | integer | yes | | Number of sheets in the workbook |
|
|
798
3897
|
| `sheets` | array<object> | yes | | Per-sheet name and row count |
|
|
799
3898
|
|
|
3899
|
+
##### Complete machine-readable schemas
|
|
3900
|
+
|
|
3901
|
+
Config schema:
|
|
3902
|
+
|
|
3903
|
+
```json
|
|
3904
|
+
{
|
|
3905
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3906
|
+
"type": "object",
|
|
3907
|
+
"properties": {
|
|
3908
|
+
"filename": {
|
|
3909
|
+
"description": "Output filename — supports LiquidJS; .xlsx is added if omitted",
|
|
3910
|
+
"type": "string"
|
|
3911
|
+
},
|
|
3912
|
+
"columns": {
|
|
3913
|
+
"description": "Ordered columns for a single-sheet workbook. Use sheets for multiple sheets.",
|
|
3914
|
+
"minItems": 1,
|
|
3915
|
+
"type": "array",
|
|
3916
|
+
"items": {
|
|
3917
|
+
"type": "object",
|
|
3918
|
+
"properties": {
|
|
3919
|
+
"key": {
|
|
3920
|
+
"type": "string",
|
|
3921
|
+
"minLength": 1,
|
|
3922
|
+
"description": "Row object key to read for this column"
|
|
3923
|
+
},
|
|
3924
|
+
"header": {
|
|
3925
|
+
"description": "Header cell text. Defaults to key.",
|
|
3926
|
+
"type": "string"
|
|
3927
|
+
},
|
|
3928
|
+
"type": {
|
|
3929
|
+
"type": "string",
|
|
3930
|
+
"enum": [
|
|
3931
|
+
"string",
|
|
3932
|
+
"number",
|
|
3933
|
+
"boolean",
|
|
3934
|
+
"date"
|
|
3935
|
+
],
|
|
3936
|
+
"description": "Cell type. Omit to infer from the JSON value. YYYY-MM-DD strings stay text unless type is date."
|
|
3937
|
+
}
|
|
3938
|
+
},
|
|
3939
|
+
"required": [
|
|
3940
|
+
"key"
|
|
3941
|
+
],
|
|
3942
|
+
"additionalProperties": false
|
|
3943
|
+
}
|
|
3944
|
+
},
|
|
3945
|
+
"rows": {
|
|
3946
|
+
"anyOf": [
|
|
3947
|
+
{
|
|
3948
|
+
"type": "string",
|
|
3949
|
+
"minLength": 1,
|
|
3950
|
+
"description": "Template expression resolving to an array of row objects, e.g. {{ steps.extract.output.items }}"
|
|
3951
|
+
},
|
|
3952
|
+
{
|
|
3953
|
+
"type": "array",
|
|
3954
|
+
"items": {
|
|
3955
|
+
"type": "object",
|
|
3956
|
+
"propertyNames": {
|
|
3957
|
+
"type": "string"
|
|
3958
|
+
},
|
|
3959
|
+
"additionalProperties": {}
|
|
3960
|
+
},
|
|
3961
|
+
"description": "Array of row objects"
|
|
3962
|
+
}
|
|
3963
|
+
],
|
|
3964
|
+
"description": "Array of row objects, or a template expression that resolves to one"
|
|
3965
|
+
},
|
|
3966
|
+
"sheets": {
|
|
3967
|
+
"description": "Multiple sheets. Do not combine with top-level columns/rows.",
|
|
3968
|
+
"minItems": 1,
|
|
3969
|
+
"type": "array",
|
|
3970
|
+
"items": {
|
|
3971
|
+
"type": "object",
|
|
3972
|
+
"properties": {
|
|
3973
|
+
"name": {
|
|
3974
|
+
"type": "string",
|
|
3975
|
+
"description": "Sheet name. Defaults to Sheet1, Sheet2, …. Excel limit: 31 characters. Cannot contain : \\ / ? * [ ], cannot start or end with a single quote, and History is reserved."
|
|
3976
|
+
},
|
|
3977
|
+
"columns": {
|
|
3978
|
+
"minItems": 1,
|
|
3979
|
+
"type": "array",
|
|
3980
|
+
"items": {
|
|
3981
|
+
"type": "object",
|
|
3982
|
+
"properties": {
|
|
3983
|
+
"key": {
|
|
3984
|
+
"type": "string",
|
|
3985
|
+
"minLength": 1,
|
|
3986
|
+
"description": "Row object key to read for this column"
|
|
3987
|
+
},
|
|
3988
|
+
"header": {
|
|
3989
|
+
"description": "Header cell text. Defaults to key.",
|
|
3990
|
+
"type": "string"
|
|
3991
|
+
},
|
|
3992
|
+
"type": {
|
|
3993
|
+
"type": "string",
|
|
3994
|
+
"enum": [
|
|
3995
|
+
"string",
|
|
3996
|
+
"number",
|
|
3997
|
+
"boolean",
|
|
3998
|
+
"date"
|
|
3999
|
+
],
|
|
4000
|
+
"description": "Cell type. Omit to infer from the JSON value. YYYY-MM-DD strings stay text unless type is date."
|
|
4001
|
+
}
|
|
4002
|
+
},
|
|
4003
|
+
"required": [
|
|
4004
|
+
"key"
|
|
4005
|
+
],
|
|
4006
|
+
"additionalProperties": false
|
|
4007
|
+
},
|
|
4008
|
+
"description": "Ordered columns for this sheet"
|
|
4009
|
+
},
|
|
4010
|
+
"rows": {
|
|
4011
|
+
"anyOf": [
|
|
4012
|
+
{
|
|
4013
|
+
"type": "string",
|
|
4014
|
+
"minLength": 1,
|
|
4015
|
+
"description": "Template expression resolving to an array of row objects, e.g. {{ steps.extract.output.items }}"
|
|
4016
|
+
},
|
|
4017
|
+
{
|
|
4018
|
+
"type": "array",
|
|
4019
|
+
"items": {
|
|
4020
|
+
"type": "object",
|
|
4021
|
+
"propertyNames": {
|
|
4022
|
+
"type": "string"
|
|
4023
|
+
},
|
|
4024
|
+
"additionalProperties": {}
|
|
4025
|
+
},
|
|
4026
|
+
"description": "Array of row objects"
|
|
4027
|
+
}
|
|
4028
|
+
],
|
|
4029
|
+
"description": "Array of row objects, or a template expression that resolves to one"
|
|
4030
|
+
}
|
|
4031
|
+
},
|
|
4032
|
+
"required": [
|
|
4033
|
+
"columns",
|
|
4034
|
+
"rows"
|
|
4035
|
+
],
|
|
4036
|
+
"additionalProperties": false
|
|
4037
|
+
}
|
|
4038
|
+
},
|
|
4039
|
+
"limits": {
|
|
4040
|
+
"type": "object",
|
|
4041
|
+
"properties": {
|
|
4042
|
+
"maxSheets": {
|
|
4043
|
+
"description": "Maximum worksheets. Omitted uses the server default (50). Cannot exceed it.",
|
|
4044
|
+
"type": "integer",
|
|
4045
|
+
"minimum": 1,
|
|
4046
|
+
"maximum": 50
|
|
4047
|
+
},
|
|
4048
|
+
"maxRowsPerSheet": {
|
|
4049
|
+
"description": "Maximum data rows per sheet. Omitted uses the server default (100,000). Cannot exceed it.",
|
|
4050
|
+
"type": "integer",
|
|
4051
|
+
"minimum": 1,
|
|
4052
|
+
"maximum": 100000
|
|
4053
|
+
},
|
|
4054
|
+
"maxTotalRows": {
|
|
4055
|
+
"description": "Maximum data rows summed across all sheets. Omitted uses the server default (250,000). Cannot exceed it.",
|
|
4056
|
+
"type": "integer",
|
|
4057
|
+
"minimum": 1,
|
|
4058
|
+
"maximum": 250000
|
|
4059
|
+
},
|
|
4060
|
+
"maxColumns": {
|
|
4061
|
+
"description": "Maximum columns per sheet. Omitted uses the server default (256). Cannot exceed it.",
|
|
4062
|
+
"type": "integer",
|
|
4063
|
+
"minimum": 1,
|
|
4064
|
+
"maximum": 256
|
|
4065
|
+
},
|
|
4066
|
+
"maxTotalCells": {
|
|
4067
|
+
"description": "Maximum populated cells across the workbook. Omitted uses the server default (5,000,000). Cannot exceed it.",
|
|
4068
|
+
"type": "integer",
|
|
4069
|
+
"minimum": 1,
|
|
4070
|
+
"maximum": 5000000
|
|
4071
|
+
},
|
|
4072
|
+
"maxOutputBytes": {
|
|
4073
|
+
"description": "Maximum serialized .xlsx size in bytes. Omitted uses the server default (52,428,800, 50 MiB). Cannot exceed it.",
|
|
4074
|
+
"type": "integer",
|
|
4075
|
+
"minimum": 1,
|
|
4076
|
+
"maximum": 52428800
|
|
4077
|
+
}
|
|
4078
|
+
},
|
|
4079
|
+
"additionalProperties": false,
|
|
4080
|
+
"description": "Optional workload caps that can only lower the server defaults. Omitted fields use the server defaults."
|
|
4081
|
+
}
|
|
4082
|
+
},
|
|
4083
|
+
"additionalProperties": false
|
|
4084
|
+
}
|
|
4085
|
+
```
|
|
4086
|
+
|
|
4087
|
+
|
|
4088
|
+
Output schema:
|
|
4089
|
+
|
|
4090
|
+
```json
|
|
4091
|
+
{
|
|
4092
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4093
|
+
"type": "object",
|
|
4094
|
+
"properties": {
|
|
4095
|
+
"fileId": {
|
|
4096
|
+
"type": "string",
|
|
4097
|
+
"description": "File ID from the files table"
|
|
4098
|
+
},
|
|
4099
|
+
"filename": {
|
|
4100
|
+
"type": "string",
|
|
4101
|
+
"description": "Sanitized output filename including .xlsx"
|
|
4102
|
+
},
|
|
4103
|
+
"sheetCount": {
|
|
4104
|
+
"type": "integer",
|
|
4105
|
+
"minimum": 1,
|
|
4106
|
+
"maximum": 9007199254740991,
|
|
4107
|
+
"description": "Number of sheets in the workbook"
|
|
4108
|
+
},
|
|
4109
|
+
"sheets": {
|
|
4110
|
+
"type": "array",
|
|
4111
|
+
"items": {
|
|
4112
|
+
"type": "object",
|
|
4113
|
+
"properties": {
|
|
4114
|
+
"name": {
|
|
4115
|
+
"type": "string",
|
|
4116
|
+
"description": "Written sheet name"
|
|
4117
|
+
},
|
|
4118
|
+
"rowCount": {
|
|
4119
|
+
"type": "integer",
|
|
4120
|
+
"minimum": 0,
|
|
4121
|
+
"maximum": 9007199254740991,
|
|
4122
|
+
"description": "Number of data rows written (excludes header)"
|
|
4123
|
+
}
|
|
4124
|
+
},
|
|
4125
|
+
"required": [
|
|
4126
|
+
"name",
|
|
4127
|
+
"rowCount"
|
|
4128
|
+
],
|
|
4129
|
+
"additionalProperties": false
|
|
4130
|
+
},
|
|
4131
|
+
"description": "Per-sheet name and row count"
|
|
4132
|
+
}
|
|
4133
|
+
},
|
|
4134
|
+
"required": [
|
|
4135
|
+
"fileId",
|
|
4136
|
+
"filename",
|
|
4137
|
+
"sheetCount",
|
|
4138
|
+
"sheets"
|
|
4139
|
+
],
|
|
4140
|
+
"additionalProperties": false
|
|
4141
|
+
}
|
|
4142
|
+
```
|
|
4143
|
+
|
|
4144
|
+
|
|
800
4145
|
#### `transform.script` — Script
|
|
801
4146
|
|
|
802
|
-
Execute a TypeScript function in a QuickJS sandbox. Input keys become
|
|
4147
|
+
Execute a TypeScript function in a QuickJS sandbox. Input keys become function parameters in any order, and the required `: R` return-type annotation IS this step's output schema: `inputs: { items, taxRate }` accepts `function script(taxRate: …, items: …): R { … }`.
|
|
4148
|
+
|
|
4149
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/script`
|
|
803
4150
|
|
|
804
4151
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
805
4152
|
|
|
@@ -807,8 +4154,8 @@ Execute a TypeScript function in a QuickJS sandbox. Input keys become the functi
|
|
|
807
4154
|
|
|
808
4155
|
| Field | Type | Required | Default | Description |
|
|
809
4156
|
| --- | --- | --- | --- | --- |
|
|
810
|
-
| `inputs` | record<string, string> | no | | Named inputs mapped from template expressions. Keys become
|
|
811
|
-
| `function` | string | yes | | TypeScript function declaration. Must be `function script(args): R { … }` where the
|
|
4157
|
+
| `inputs` | record<string, string> | no | | Named inputs mapped from template expressions. Keys become function parameters; parameter order is independent of YAML/JSON key order: `inputs: { items, taxRate }` accepts `function script(taxRate: …, items: …): R { … }`. |
|
|
4158
|
+
| `function` | string | yes | | TypeScript function declaration. Must be `function script(args): R { … }` where parameter names exactly match the configured input keys in any order and `R` is a return type annotation. The annotation IS this step's output schema. |
|
|
812
4159
|
| `timeout` | number | no | `5000` | Max execution time in milliseconds (default: 5000) |
|
|
813
4160
|
| `memoryLimit` | number | no | `10485760` | Max memory in bytes (default: 10MB) |
|
|
814
4161
|
|
|
@@ -816,10 +4163,68 @@ Execute a TypeScript function in a QuickJS sandbox. Input keys become the functi
|
|
|
816
4163
|
|
|
817
4164
|
> Value returned from script. Validated at runtime against the JSON Schema derived from the function's return type annotation.
|
|
818
4165
|
|
|
4166
|
+
##### Complete machine-readable schemas
|
|
4167
|
+
|
|
4168
|
+
Config schema:
|
|
4169
|
+
|
|
4170
|
+
```json
|
|
4171
|
+
{
|
|
4172
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4173
|
+
"type": "object",
|
|
4174
|
+
"properties": {
|
|
4175
|
+
"inputs": {
|
|
4176
|
+
"description": "Named inputs mapped from template expressions. Keys become function parameters; parameter order is independent of YAML/JSON key order: `inputs: { items, taxRate }` accepts `function script(taxRate: …, items: …): R { … }`.",
|
|
4177
|
+
"type": "object",
|
|
4178
|
+
"propertyNames": {
|
|
4179
|
+
"type": "string"
|
|
4180
|
+
},
|
|
4181
|
+
"additionalProperties": {
|
|
4182
|
+
"type": "string"
|
|
4183
|
+
}
|
|
4184
|
+
},
|
|
4185
|
+
"function": {
|
|
4186
|
+
"type": "string",
|
|
4187
|
+
"minLength": 1,
|
|
4188
|
+
"maxLength": 50000,
|
|
4189
|
+
"description": "TypeScript function declaration. Must be `function script(args): R { … }` where parameter names exactly match the configured input keys in any order and `R` is a return type annotation. The annotation IS this step's output schema."
|
|
4190
|
+
},
|
|
4191
|
+
"timeout": {
|
|
4192
|
+
"description": "Max execution time in milliseconds (default: 5000)",
|
|
4193
|
+
"default": 5000,
|
|
4194
|
+
"type": "number",
|
|
4195
|
+
"exclusiveMinimum": 0
|
|
4196
|
+
},
|
|
4197
|
+
"memoryLimit": {
|
|
4198
|
+
"description": "Max memory in bytes (default: 10MB)",
|
|
4199
|
+
"default": 10485760,
|
|
4200
|
+
"type": "number",
|
|
4201
|
+
"exclusiveMinimum": 0
|
|
4202
|
+
}
|
|
4203
|
+
},
|
|
4204
|
+
"required": [
|
|
4205
|
+
"function"
|
|
4206
|
+
],
|
|
4207
|
+
"additionalProperties": false
|
|
4208
|
+
}
|
|
4209
|
+
```
|
|
4210
|
+
|
|
4211
|
+
|
|
4212
|
+
Output schema:
|
|
4213
|
+
|
|
4214
|
+
```json
|
|
4215
|
+
{
|
|
4216
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4217
|
+
"description": "Value returned from script. Validated at runtime against the JSON Schema derived from the function's return type annotation."
|
|
4218
|
+
}
|
|
4219
|
+
```
|
|
4220
|
+
|
|
4221
|
+
|
|
819
4222
|
#### `transform.text-chunker` — Text Chunker
|
|
820
4223
|
|
|
821
4224
|
Split long text into chunks with regex-anchored boundaries, overlap, and header preservation. Accepts raw text or a parsed-document object; chunks carry source page indexes when pages are provided.
|
|
822
4225
|
|
|
4226
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/text-chunker`
|
|
4227
|
+
|
|
823
4228
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
824
4229
|
|
|
825
4230
|
**Config** (in `step.with`):
|
|
@@ -841,10 +4246,217 @@ Split long text into chunks with regex-anchored boundaries, overlap, and header
|
|
|
841
4246
|
| `chunks` | array<object> | yes | | |
|
|
842
4247
|
| `summary` | object | yes | | |
|
|
843
4248
|
|
|
4249
|
+
##### Complete machine-readable schemas
|
|
4250
|
+
|
|
4251
|
+
Config schema:
|
|
4252
|
+
|
|
4253
|
+
```json
|
|
4254
|
+
{
|
|
4255
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4256
|
+
"type": "object",
|
|
4257
|
+
"properties": {
|
|
4258
|
+
"input": {
|
|
4259
|
+
"anyOf": [
|
|
4260
|
+
{
|
|
4261
|
+
"type": "string"
|
|
4262
|
+
},
|
|
4263
|
+
{
|
|
4264
|
+
"type": "object",
|
|
4265
|
+
"properties": {
|
|
4266
|
+
"pages": {
|
|
4267
|
+
"minItems": 1,
|
|
4268
|
+
"type": "array",
|
|
4269
|
+
"items": {
|
|
4270
|
+
"type": "object",
|
|
4271
|
+
"properties": {
|
|
4272
|
+
"pageIndex": {
|
|
4273
|
+
"type": "integer",
|
|
4274
|
+
"minimum": 0,
|
|
4275
|
+
"maximum": 9007199254740991
|
|
4276
|
+
},
|
|
4277
|
+
"text": {
|
|
4278
|
+
"type": "string"
|
|
4279
|
+
}
|
|
4280
|
+
},
|
|
4281
|
+
"required": [
|
|
4282
|
+
"pageIndex",
|
|
4283
|
+
"text"
|
|
4284
|
+
],
|
|
4285
|
+
"additionalProperties": false
|
|
4286
|
+
}
|
|
4287
|
+
}
|
|
4288
|
+
},
|
|
4289
|
+
"required": [
|
|
4290
|
+
"pages"
|
|
4291
|
+
],
|
|
4292
|
+
"additionalProperties": {}
|
|
4293
|
+
}
|
|
4294
|
+
],
|
|
4295
|
+
"description": "Either raw text or a parsed-document object `{ pages: [{ pageIndex, text }] }` (e.g. `{{ steps.parse.output }}`). Pages preserve per-chunk page provenance."
|
|
4296
|
+
},
|
|
4297
|
+
"maxChars": {
|
|
4298
|
+
"type": "integer",
|
|
4299
|
+
"minimum": 100,
|
|
4300
|
+
"maximum": 100000,
|
|
4301
|
+
"description": "Target chunk size in characters. Hard ceiling per chunk is 1.5×."
|
|
4302
|
+
},
|
|
4303
|
+
"overlap": {
|
|
4304
|
+
"default": 0,
|
|
4305
|
+
"description": "Characters duplicated at chunk boundaries (default 0). Must be < maxChars / 2.",
|
|
4306
|
+
"type": "integer",
|
|
4307
|
+
"minimum": 0,
|
|
4308
|
+
"maximum": 9007199254740991
|
|
4309
|
+
},
|
|
4310
|
+
"splitOn": {
|
|
4311
|
+
"description": "Ordered list of regexes; the first that matches near the chunk boundary wins. Falls back to char-cut when none match. Tip: list narrowest first (e.g. /\\d+\\.\\d+\\s+/ before /\\n\\n+/).",
|
|
4312
|
+
"type": "array",
|
|
4313
|
+
"items": {
|
|
4314
|
+
"type": "string"
|
|
4315
|
+
}
|
|
4316
|
+
},
|
|
4317
|
+
"maxChunks": {
|
|
4318
|
+
"default": 64,
|
|
4319
|
+
"description": "Safety cap; later chunks are dropped and `summary.truncated` flips to true.",
|
|
4320
|
+
"type": "integer",
|
|
4321
|
+
"minimum": 1,
|
|
4322
|
+
"maximum": 1000
|
|
4323
|
+
},
|
|
4324
|
+
"preserveHeader": {
|
|
4325
|
+
"default": 0,
|
|
4326
|
+
"description": "Prepend the first N characters of the input to every chunk (good for \"always include the contract title\").",
|
|
4327
|
+
"type": "integer",
|
|
4328
|
+
"minimum": 0,
|
|
4329
|
+
"maximum": 9007199254740991
|
|
4330
|
+
},
|
|
4331
|
+
"minChunkChars": {
|
|
4332
|
+
"default": 0,
|
|
4333
|
+
"description": "Trailing chunks shorter than this are merged into the previous chunk.",
|
|
4334
|
+
"type": "integer",
|
|
4335
|
+
"minimum": 0,
|
|
4336
|
+
"maximum": 9007199254740991
|
|
4337
|
+
}
|
|
4338
|
+
},
|
|
4339
|
+
"required": [
|
|
4340
|
+
"input",
|
|
4341
|
+
"maxChars",
|
|
4342
|
+
"overlap",
|
|
4343
|
+
"maxChunks",
|
|
4344
|
+
"preserveHeader",
|
|
4345
|
+
"minChunkChars"
|
|
4346
|
+
],
|
|
4347
|
+
"additionalProperties": false
|
|
4348
|
+
}
|
|
4349
|
+
```
|
|
4350
|
+
|
|
4351
|
+
|
|
4352
|
+
Output schema:
|
|
4353
|
+
|
|
4354
|
+
```json
|
|
4355
|
+
{
|
|
4356
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4357
|
+
"type": "object",
|
|
4358
|
+
"properties": {
|
|
4359
|
+
"chunks": {
|
|
4360
|
+
"type": "array",
|
|
4361
|
+
"items": {
|
|
4362
|
+
"type": "object",
|
|
4363
|
+
"properties": {
|
|
4364
|
+
"text": {
|
|
4365
|
+
"type": "string"
|
|
4366
|
+
},
|
|
4367
|
+
"index": {
|
|
4368
|
+
"type": "integer",
|
|
4369
|
+
"minimum": 0,
|
|
4370
|
+
"maximum": 9007199254740991
|
|
4371
|
+
},
|
|
4372
|
+
"total": {
|
|
4373
|
+
"type": "integer",
|
|
4374
|
+
"minimum": 0,
|
|
4375
|
+
"maximum": 9007199254740991
|
|
4376
|
+
},
|
|
4377
|
+
"startOffset": {
|
|
4378
|
+
"type": "integer",
|
|
4379
|
+
"minimum": 0,
|
|
4380
|
+
"maximum": 9007199254740991
|
|
4381
|
+
},
|
|
4382
|
+
"endOffset": {
|
|
4383
|
+
"type": "integer",
|
|
4384
|
+
"minimum": 0,
|
|
4385
|
+
"maximum": 9007199254740991
|
|
4386
|
+
},
|
|
4387
|
+
"pages": {
|
|
4388
|
+
"type": "array",
|
|
4389
|
+
"items": {
|
|
4390
|
+
"type": "integer",
|
|
4391
|
+
"minimum": 0,
|
|
4392
|
+
"maximum": 9007199254740991
|
|
4393
|
+
},
|
|
4394
|
+
"description": "Page indexes that contributed text to this chunk (empty for raw-string input)."
|
|
4395
|
+
}
|
|
4396
|
+
},
|
|
4397
|
+
"required": [
|
|
4398
|
+
"text",
|
|
4399
|
+
"index",
|
|
4400
|
+
"total",
|
|
4401
|
+
"startOffset",
|
|
4402
|
+
"endOffset",
|
|
4403
|
+
"pages"
|
|
4404
|
+
],
|
|
4405
|
+
"additionalProperties": false
|
|
4406
|
+
}
|
|
4407
|
+
},
|
|
4408
|
+
"summary": {
|
|
4409
|
+
"type": "object",
|
|
4410
|
+
"properties": {
|
|
4411
|
+
"inputChars": {
|
|
4412
|
+
"type": "integer",
|
|
4413
|
+
"minimum": 0,
|
|
4414
|
+
"maximum": 9007199254740991
|
|
4415
|
+
},
|
|
4416
|
+
"totalChunks": {
|
|
4417
|
+
"type": "integer",
|
|
4418
|
+
"minimum": 0,
|
|
4419
|
+
"maximum": 9007199254740991
|
|
4420
|
+
},
|
|
4421
|
+
"avgChunkChars": {
|
|
4422
|
+
"type": "integer",
|
|
4423
|
+
"minimum": 0,
|
|
4424
|
+
"maximum": 9007199254740991
|
|
4425
|
+
},
|
|
4426
|
+
"maxChunkChars": {
|
|
4427
|
+
"type": "integer",
|
|
4428
|
+
"minimum": 0,
|
|
4429
|
+
"maximum": 9007199254740991
|
|
4430
|
+
},
|
|
4431
|
+
"truncated": {
|
|
4432
|
+
"type": "boolean"
|
|
4433
|
+
}
|
|
4434
|
+
},
|
|
4435
|
+
"required": [
|
|
4436
|
+
"inputChars",
|
|
4437
|
+
"totalChunks",
|
|
4438
|
+
"avgChunkChars",
|
|
4439
|
+
"maxChunkChars",
|
|
4440
|
+
"truncated"
|
|
4441
|
+
],
|
|
4442
|
+
"additionalProperties": false
|
|
4443
|
+
}
|
|
4444
|
+
},
|
|
4445
|
+
"required": [
|
|
4446
|
+
"chunks",
|
|
4447
|
+
"summary"
|
|
4448
|
+
],
|
|
4449
|
+
"additionalProperties": false
|
|
4450
|
+
}
|
|
4451
|
+
```
|
|
4452
|
+
|
|
4453
|
+
|
|
844
4454
|
#### `transform.regex-extract` — Regex Extract
|
|
845
4455
|
|
|
846
4456
|
Pull named fields from text via regex patterns (deterministic counterpart to ai.extract). Accepts raw text or a parsed-document object; matches carry `_evidence.pageIndex` when pages are provided.
|
|
847
4457
|
|
|
4458
|
+
**Behavior and examples:** `eigenpal docs read steps/transform/regex-extract`
|
|
4459
|
+
|
|
848
4460
|
**Durable retry:** Transforms, including those that write files, are not durably retried.
|
|
849
4461
|
|
|
850
4462
|
**Config** (in `step.with`):
|
|
@@ -860,12 +4472,194 @@ Pull named fields from text via regex patterns (deterministic counterpart to ai.
|
|
|
860
4472
|
|
|
861
4473
|
> Field name → extracted value (or default), plus `_evidence: { [field]: { pageIndex, matchOffset, raw } }` and `_unmatched: string[]`.
|
|
862
4474
|
|
|
4475
|
+
##### Complete machine-readable schemas
|
|
4476
|
+
|
|
4477
|
+
Config schema:
|
|
4478
|
+
|
|
4479
|
+
```json
|
|
4480
|
+
{
|
|
4481
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4482
|
+
"type": "object",
|
|
4483
|
+
"properties": {
|
|
4484
|
+
"input": {
|
|
4485
|
+
"anyOf": [
|
|
4486
|
+
{
|
|
4487
|
+
"type": "string"
|
|
4488
|
+
},
|
|
4489
|
+
{
|
|
4490
|
+
"type": "object",
|
|
4491
|
+
"properties": {
|
|
4492
|
+
"pages": {
|
|
4493
|
+
"minItems": 1,
|
|
4494
|
+
"type": "array",
|
|
4495
|
+
"items": {
|
|
4496
|
+
"type": "object",
|
|
4497
|
+
"properties": {
|
|
4498
|
+
"pageIndex": {
|
|
4499
|
+
"type": "integer",
|
|
4500
|
+
"minimum": 0,
|
|
4501
|
+
"maximum": 9007199254740991
|
|
4502
|
+
},
|
|
4503
|
+
"text": {
|
|
4504
|
+
"type": "string"
|
|
4505
|
+
}
|
|
4506
|
+
},
|
|
4507
|
+
"required": [
|
|
4508
|
+
"pageIndex",
|
|
4509
|
+
"text"
|
|
4510
|
+
],
|
|
4511
|
+
"additionalProperties": false
|
|
4512
|
+
}
|
|
4513
|
+
}
|
|
4514
|
+
},
|
|
4515
|
+
"required": [
|
|
4516
|
+
"pages"
|
|
4517
|
+
],
|
|
4518
|
+
"additionalProperties": {}
|
|
4519
|
+
}
|
|
4520
|
+
],
|
|
4521
|
+
"description": "Either raw text or a parsed-document object `{ pages: [{ pageIndex, text }] }`. Pages enable per-match `_evidence.pageIndex`."
|
|
4522
|
+
},
|
|
4523
|
+
"fields": {
|
|
4524
|
+
"type": "object",
|
|
4525
|
+
"propertyNames": {
|
|
4526
|
+
"type": "string"
|
|
4527
|
+
},
|
|
4528
|
+
"additionalProperties": {
|
|
4529
|
+
"type": "object",
|
|
4530
|
+
"properties": {
|
|
4531
|
+
"pattern": {
|
|
4532
|
+
"type": "string",
|
|
4533
|
+
"minLength": 1,
|
|
4534
|
+
"description": "Regex source. Must contain at least one capturing group."
|
|
4535
|
+
},
|
|
4536
|
+
"flags": {
|
|
4537
|
+
"description": "Per-field regex flags (overrides the step-level `flags`). Subset of \"gimsuy\".",
|
|
4538
|
+
"type": "string",
|
|
4539
|
+
"pattern": "^(?!.*(.).*\\1)[gimsuy]*$"
|
|
4540
|
+
},
|
|
4541
|
+
"max": {
|
|
4542
|
+
"default": 1,
|
|
4543
|
+
"description": "Max matches to keep. Default 1. Use \"all\" plus `select` to combine multiple hits.",
|
|
4544
|
+
"anyOf": [
|
|
4545
|
+
{
|
|
4546
|
+
"type": "number",
|
|
4547
|
+
"const": 1
|
|
4548
|
+
},
|
|
4549
|
+
{
|
|
4550
|
+
"type": "string",
|
|
4551
|
+
"const": "all"
|
|
4552
|
+
},
|
|
4553
|
+
{
|
|
4554
|
+
"type": "integer",
|
|
4555
|
+
"exclusiveMinimum": 0,
|
|
4556
|
+
"maximum": 9007199254740991
|
|
4557
|
+
}
|
|
4558
|
+
]
|
|
4559
|
+
},
|
|
4560
|
+
"select": {
|
|
4561
|
+
"description": "When max ≠ 1: which match to surface. \"first\" (default), \"last\", \"min\", \"max\", or \"all\" — `all` returns the full array of matches AND a parallel array of evidence entries.",
|
|
4562
|
+
"type": "string",
|
|
4563
|
+
"enum": [
|
|
4564
|
+
"first",
|
|
4565
|
+
"last",
|
|
4566
|
+
"min",
|
|
4567
|
+
"max",
|
|
4568
|
+
"all"
|
|
4569
|
+
]
|
|
4570
|
+
},
|
|
4571
|
+
"normalize": {
|
|
4572
|
+
"type": "string",
|
|
4573
|
+
"enum": [
|
|
4574
|
+
"strip-spaces",
|
|
4575
|
+
"strip-punct",
|
|
4576
|
+
"collapse-whitespace",
|
|
4577
|
+
"lower",
|
|
4578
|
+
"upper",
|
|
4579
|
+
"trim"
|
|
4580
|
+
]
|
|
4581
|
+
},
|
|
4582
|
+
"format": {
|
|
4583
|
+
"description": "Post-processing. iso-date accepts DD.MM.YYYY (1 group) or DD/MM/YYYY split across 3 groups → \"YYYY-MM-DD\".",
|
|
4584
|
+
"type": "string",
|
|
4585
|
+
"enum": [
|
|
4586
|
+
"iso-date",
|
|
4587
|
+
"iso-number",
|
|
4588
|
+
"lower",
|
|
4589
|
+
"upper",
|
|
4590
|
+
"trim"
|
|
4591
|
+
]
|
|
4592
|
+
},
|
|
4593
|
+
"exclude": {
|
|
4594
|
+
"description": "Post-normalize values to skip (known wrong matches).",
|
|
4595
|
+
"type": "array",
|
|
4596
|
+
"items": {
|
|
4597
|
+
"type": "string"
|
|
4598
|
+
}
|
|
4599
|
+
},
|
|
4600
|
+
"default": {
|
|
4601
|
+
"description": "Value when nothing matches.",
|
|
4602
|
+
"type": "string"
|
|
4603
|
+
},
|
|
4604
|
+
"contextChars": {
|
|
4605
|
+
"description": "When > 0, evidence entries gain `contextBefore` and `contextAfter` snippets of N characters surrounding the match. Useful for UI/debug display.",
|
|
4606
|
+
"type": "integer",
|
|
4607
|
+
"minimum": 0,
|
|
4608
|
+
"maximum": 2000
|
|
4609
|
+
}
|
|
4610
|
+
},
|
|
4611
|
+
"required": [
|
|
4612
|
+
"pattern",
|
|
4613
|
+
"max"
|
|
4614
|
+
],
|
|
4615
|
+
"additionalProperties": false
|
|
4616
|
+
},
|
|
4617
|
+
"description": "Named field → pattern mapping."
|
|
4618
|
+
},
|
|
4619
|
+
"flags": {
|
|
4620
|
+
"description": "Default regex flags applied when a field omits its own `flags`. Subset of \"gimsuy\".",
|
|
4621
|
+
"type": "string",
|
|
4622
|
+
"pattern": "^(?!.*(.).*\\1)[gimsuy]*$"
|
|
4623
|
+
},
|
|
4624
|
+
"searchWindow": {
|
|
4625
|
+
"description": "Only search the first N characters of input (perf). Omit for full search.",
|
|
4626
|
+
"type": "integer",
|
|
4627
|
+
"minimum": 100,
|
|
4628
|
+
"maximum": 9007199254740991
|
|
4629
|
+
}
|
|
4630
|
+
},
|
|
4631
|
+
"required": [
|
|
4632
|
+
"input",
|
|
4633
|
+
"fields"
|
|
4634
|
+
],
|
|
4635
|
+
"additionalProperties": false
|
|
4636
|
+
}
|
|
4637
|
+
```
|
|
4638
|
+
|
|
4639
|
+
|
|
4640
|
+
Output schema:
|
|
4641
|
+
|
|
4642
|
+
```json
|
|
4643
|
+
{
|
|
4644
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4645
|
+
"type": "object",
|
|
4646
|
+
"propertyNames": {
|
|
4647
|
+
"type": "string"
|
|
4648
|
+
},
|
|
4649
|
+
"additionalProperties": {},
|
|
4650
|
+
"description": "Field name → extracted value (or default), plus `_evidence: { [field]: { pageIndex, matchOffset, raw } }` and `_unmatched: string[]`."
|
|
4651
|
+
}
|
|
4652
|
+
```
|
|
4653
|
+
|
|
4654
|
+
|
|
863
4655
|
### Action steps — external side effects
|
|
864
4656
|
|
|
865
4657
|
#### `action.http` — HTTP Request
|
|
866
4658
|
|
|
867
4659
|
Make an HTTP request to an external API
|
|
868
4660
|
|
|
4661
|
+
**Behavior and examples:** `eigenpal docs read steps/action/http`
|
|
4662
|
+
|
|
869
4663
|
**Durable retry:** HTTP `GET` and `HEAD` can durably retry transient timeout, rate limits, selected retryable server failures failures. Other methods are not replayed.
|
|
870
4664
|
|
|
871
4665
|
**Config** (in `step.with`):
|
|
@@ -889,10 +4683,114 @@ Make an HTTP request to an external API
|
|
|
889
4683
|
| `body` | unknown | yes | | Response body (parsed JSON or string) |
|
|
890
4684
|
| `responseCharset` | string | yes | | Charset used to decode the response body (e.g. utf-8, windows-1250) |
|
|
891
4685
|
|
|
4686
|
+
##### Complete machine-readable schemas
|
|
4687
|
+
|
|
4688
|
+
Config schema:
|
|
4689
|
+
|
|
4690
|
+
```json
|
|
4691
|
+
{
|
|
4692
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4693
|
+
"type": "object",
|
|
4694
|
+
"properties": {
|
|
4695
|
+
"url": {
|
|
4696
|
+
"type": "string",
|
|
4697
|
+
"minLength": 1,
|
|
4698
|
+
"description": "Request URL (supports template expressions)"
|
|
4699
|
+
},
|
|
4700
|
+
"method": {
|
|
4701
|
+
"default": "GET",
|
|
4702
|
+
"type": "string",
|
|
4703
|
+
"enum": [
|
|
4704
|
+
"GET",
|
|
4705
|
+
"HEAD",
|
|
4706
|
+
"POST",
|
|
4707
|
+
"PUT",
|
|
4708
|
+
"PATCH",
|
|
4709
|
+
"DELETE"
|
|
4710
|
+
]
|
|
4711
|
+
},
|
|
4712
|
+
"headers": {
|
|
4713
|
+
"description": "HTTP headers",
|
|
4714
|
+
"type": "object",
|
|
4715
|
+
"propertyNames": {
|
|
4716
|
+
"type": "string"
|
|
4717
|
+
},
|
|
4718
|
+
"additionalProperties": {
|
|
4719
|
+
"type": "string"
|
|
4720
|
+
}
|
|
4721
|
+
},
|
|
4722
|
+
"body": {
|
|
4723
|
+
"description": "Request body (JSON or string)"
|
|
4724
|
+
},
|
|
4725
|
+
"timeout": {
|
|
4726
|
+
"description": "Timeout in milliseconds",
|
|
4727
|
+
"default": 30000,
|
|
4728
|
+
"type": "number",
|
|
4729
|
+
"exclusiveMinimum": 0
|
|
4730
|
+
},
|
|
4731
|
+
"insecureSkipTlsVerify": {
|
|
4732
|
+
"description": "If true, skip TLS certificate verification (use only for read-only public endpoints with bad/expired certs)",
|
|
4733
|
+
"default": false,
|
|
4734
|
+
"type": "boolean"
|
|
4735
|
+
}
|
|
4736
|
+
},
|
|
4737
|
+
"required": [
|
|
4738
|
+
"url"
|
|
4739
|
+
],
|
|
4740
|
+
"additionalProperties": false
|
|
4741
|
+
}
|
|
4742
|
+
```
|
|
4743
|
+
|
|
4744
|
+
|
|
4745
|
+
Output schema:
|
|
4746
|
+
|
|
4747
|
+
```json
|
|
4748
|
+
{
|
|
4749
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4750
|
+
"type": "object",
|
|
4751
|
+
"properties": {
|
|
4752
|
+
"status": {
|
|
4753
|
+
"type": "number",
|
|
4754
|
+
"description": "HTTP status code"
|
|
4755
|
+
},
|
|
4756
|
+
"statusText": {
|
|
4757
|
+
"type": "string"
|
|
4758
|
+
},
|
|
4759
|
+
"headers": {
|
|
4760
|
+
"type": "object",
|
|
4761
|
+
"propertyNames": {
|
|
4762
|
+
"type": "string"
|
|
4763
|
+
},
|
|
4764
|
+
"additionalProperties": {
|
|
4765
|
+
"type": "string"
|
|
4766
|
+
}
|
|
4767
|
+
},
|
|
4768
|
+
"body": {
|
|
4769
|
+
"description": "Response body (parsed JSON or string)"
|
|
4770
|
+
},
|
|
4771
|
+
"responseCharset": {
|
|
4772
|
+
"type": "string",
|
|
4773
|
+
"description": "Charset used to decode the response body (e.g. utf-8, windows-1250)"
|
|
4774
|
+
}
|
|
4775
|
+
},
|
|
4776
|
+
"required": [
|
|
4777
|
+
"status",
|
|
4778
|
+
"statusText",
|
|
4779
|
+
"headers",
|
|
4780
|
+
"body",
|
|
4781
|
+
"responseCharset"
|
|
4782
|
+
],
|
|
4783
|
+
"additionalProperties": false
|
|
4784
|
+
}
|
|
4785
|
+
```
|
|
4786
|
+
|
|
4787
|
+
|
|
892
4788
|
#### `action.invoke-workflow` — Invoke Workflow
|
|
893
4789
|
|
|
894
4790
|
Execute another workflow and return its output
|
|
895
4791
|
|
|
4792
|
+
**Behavior and examples:** `eigenpal docs read steps/action/invoke-workflow`
|
|
4793
|
+
|
|
896
4794
|
**Durable retry:** Invoked workflows are not replayed as durable leaf attempts.
|
|
897
4795
|
|
|
898
4796
|
**Config** (in `step.with`):
|
|
@@ -911,10 +4809,80 @@ Execute another workflow and return its output
|
|
|
911
4809
|
|
|
912
4810
|
> When wait is true, the invoked workflow's declared output fields are flattened to the top level alongside a `files` array — reference them as {{ steps.<invoke>.output.<field> }} (there is no `.data` or `.result` wrapper). When wait is false, execution metadata. CLI authors can run `eigenpal workflow schema <workflow-id>` to inspect resolved fields; Studio builder agents can call `get_workflow_output_schema`.
|
|
913
4811
|
|
|
4812
|
+
##### Complete machine-readable schemas
|
|
4813
|
+
|
|
4814
|
+
Config schema:
|
|
4815
|
+
|
|
4816
|
+
```json
|
|
4817
|
+
{
|
|
4818
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4819
|
+
"type": "object",
|
|
4820
|
+
"properties": {
|
|
4821
|
+
"workflow": {
|
|
4822
|
+
"description": "Workflow to invoke — definition name or wf_ id (tenant-scoped)",
|
|
4823
|
+
"type": "string",
|
|
4824
|
+
"minLength": 1
|
|
4825
|
+
},
|
|
4826
|
+
"workflowId": {
|
|
4827
|
+
"description": "Legacy alias for workflow when the value is a wf_ id",
|
|
4828
|
+
"type": "string",
|
|
4829
|
+
"minLength": 1
|
|
4830
|
+
},
|
|
4831
|
+
"execution": {
|
|
4832
|
+
"description": "inline: run target steps in this execution (default). child: spawn a separate run with lineage.",
|
|
4833
|
+
"type": "string",
|
|
4834
|
+
"enum": [
|
|
4835
|
+
"inline",
|
|
4836
|
+
"child"
|
|
4837
|
+
]
|
|
4838
|
+
},
|
|
4839
|
+
"input": {
|
|
4840
|
+
"description": "Input record keyed by the invoked workflow's declared inputs",
|
|
4841
|
+
"type": "object",
|
|
4842
|
+
"propertyNames": {
|
|
4843
|
+
"type": "string"
|
|
4844
|
+
},
|
|
4845
|
+
"additionalProperties": {}
|
|
4846
|
+
},
|
|
4847
|
+
"wait": {
|
|
4848
|
+
"description": "Child mode only. Wait for the invoked workflow to complete and return its output (default: true). Set false for fire-and-forget.",
|
|
4849
|
+
"type": "boolean"
|
|
4850
|
+
},
|
|
4851
|
+
"timeout": {
|
|
4852
|
+
"description": "Child mode only. Max wait time in ms when waiting (default: 300000)",
|
|
4853
|
+
"type": "number"
|
|
4854
|
+
},
|
|
4855
|
+
"pollInterval": {
|
|
4856
|
+
"description": "Child mode only. How often to poll status in ms when waiting (default: 1000)",
|
|
4857
|
+
"type": "number"
|
|
4858
|
+
}
|
|
4859
|
+
},
|
|
4860
|
+
"additionalProperties": false
|
|
4861
|
+
}
|
|
4862
|
+
```
|
|
4863
|
+
|
|
4864
|
+
|
|
4865
|
+
Output schema:
|
|
4866
|
+
|
|
4867
|
+
```json
|
|
4868
|
+
{
|
|
4869
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4870
|
+
"type": "object",
|
|
4871
|
+
"propertyNames": {
|
|
4872
|
+
"type": "string"
|
|
4873
|
+
},
|
|
4874
|
+
"additionalProperties": {},
|
|
4875
|
+
"description": "When wait is true, the invoked workflow's declared output fields are flattened to the top level alongside a `files` array — reference them as {{ steps.<invoke>.output.<field> }} (there is no `.data` or `.result` wrapper). When wait is false, execution metadata. CLI authors can run `eigenpal workflow schema <workflow-id>` to inspect resolved fields; Studio builder agents can call `get_workflow_output_schema`."
|
|
4876
|
+
}
|
|
4877
|
+
```
|
|
4878
|
+
|
|
4879
|
+
|
|
914
4880
|
#### `action.website-reader` — Website Reader
|
|
915
4881
|
|
|
916
4882
|
Fetch a webpage and convert content to markdown
|
|
917
4883
|
|
|
4884
|
+
**Behavior and examples:** `eigenpal docs read steps/action/website-reader`
|
|
4885
|
+
|
|
918
4886
|
**Durable retry:** Supported for transient timeout, rate limits, selected retryable server failures failures.
|
|
919
4887
|
|
|
920
4888
|
**Config** (in `step.with`):
|
|
@@ -937,12 +4905,134 @@ Fetch a webpage and convert content to markdown
|
|
|
937
4905
|
| `length` | number | yes | | Content length in characters |
|
|
938
4906
|
| `url` | string | yes | | Final URL after redirects |
|
|
939
4907
|
|
|
4908
|
+
##### Complete machine-readable schemas
|
|
4909
|
+
|
|
4910
|
+
Config schema:
|
|
4911
|
+
|
|
4912
|
+
```json
|
|
4913
|
+
{
|
|
4914
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4915
|
+
"type": "object",
|
|
4916
|
+
"properties": {
|
|
4917
|
+
"url": {
|
|
4918
|
+
"type": "string",
|
|
4919
|
+
"minLength": 1,
|
|
4920
|
+
"description": "Website URL to fetch (supports template expressions)"
|
|
4921
|
+
},
|
|
4922
|
+
"timeout": {
|
|
4923
|
+
"description": "Timeout in milliseconds",
|
|
4924
|
+
"default": 30000,
|
|
4925
|
+
"type": "number",
|
|
4926
|
+
"exclusiveMinimum": 0
|
|
4927
|
+
},
|
|
4928
|
+
"encoding": {
|
|
4929
|
+
"description": "Response encoding. Auto detects from Content-Type header and HTML meta tags.",
|
|
4930
|
+
"default": "auto",
|
|
4931
|
+
"type": "string",
|
|
4932
|
+
"enum": [
|
|
4933
|
+
"auto",
|
|
4934
|
+
"utf-8",
|
|
4935
|
+
"latin1",
|
|
4936
|
+
"windows-1250",
|
|
4937
|
+
"windows-1252",
|
|
4938
|
+
"iso-8859-2"
|
|
4939
|
+
]
|
|
4940
|
+
}
|
|
4941
|
+
},
|
|
4942
|
+
"required": [
|
|
4943
|
+
"url"
|
|
4944
|
+
],
|
|
4945
|
+
"additionalProperties": false
|
|
4946
|
+
}
|
|
4947
|
+
```
|
|
4948
|
+
|
|
4949
|
+
|
|
4950
|
+
Output schema:
|
|
4951
|
+
|
|
4952
|
+
```json
|
|
4953
|
+
{
|
|
4954
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
4955
|
+
"type": "object",
|
|
4956
|
+
"properties": {
|
|
4957
|
+
"markdown": {
|
|
4958
|
+
"type": "string",
|
|
4959
|
+
"description": "Page content converted to markdown"
|
|
4960
|
+
},
|
|
4961
|
+
"title": {
|
|
4962
|
+
"anyOf": [
|
|
4963
|
+
{
|
|
4964
|
+
"type": "string"
|
|
4965
|
+
},
|
|
4966
|
+
{
|
|
4967
|
+
"type": "null"
|
|
4968
|
+
}
|
|
4969
|
+
],
|
|
4970
|
+
"description": "Page title"
|
|
4971
|
+
},
|
|
4972
|
+
"excerpt": {
|
|
4973
|
+
"anyOf": [
|
|
4974
|
+
{
|
|
4975
|
+
"type": "string"
|
|
4976
|
+
},
|
|
4977
|
+
{
|
|
4978
|
+
"type": "null"
|
|
4979
|
+
}
|
|
4980
|
+
],
|
|
4981
|
+
"description": "Page excerpt/description"
|
|
4982
|
+
},
|
|
4983
|
+
"byline": {
|
|
4984
|
+
"anyOf": [
|
|
4985
|
+
{
|
|
4986
|
+
"type": "string"
|
|
4987
|
+
},
|
|
4988
|
+
{
|
|
4989
|
+
"type": "null"
|
|
4990
|
+
}
|
|
4991
|
+
],
|
|
4992
|
+
"description": "Author information"
|
|
4993
|
+
},
|
|
4994
|
+
"siteName": {
|
|
4995
|
+
"anyOf": [
|
|
4996
|
+
{
|
|
4997
|
+
"type": "string"
|
|
4998
|
+
},
|
|
4999
|
+
{
|
|
5000
|
+
"type": "null"
|
|
5001
|
+
}
|
|
5002
|
+
],
|
|
5003
|
+
"description": "Site name"
|
|
5004
|
+
},
|
|
5005
|
+
"length": {
|
|
5006
|
+
"type": "number",
|
|
5007
|
+
"description": "Content length in characters"
|
|
5008
|
+
},
|
|
5009
|
+
"url": {
|
|
5010
|
+
"type": "string",
|
|
5011
|
+
"description": "Final URL after redirects"
|
|
5012
|
+
}
|
|
5013
|
+
},
|
|
5014
|
+
"required": [
|
|
5015
|
+
"markdown",
|
|
5016
|
+
"title",
|
|
5017
|
+
"excerpt",
|
|
5018
|
+
"byline",
|
|
5019
|
+
"siteName",
|
|
5020
|
+
"length",
|
|
5021
|
+
"url"
|
|
5022
|
+
],
|
|
5023
|
+
"additionalProperties": false
|
|
5024
|
+
}
|
|
5025
|
+
```
|
|
5026
|
+
|
|
5027
|
+
|
|
940
5028
|
### Control steps — flow control
|
|
941
5029
|
|
|
942
5030
|
#### `control.if` — Condition
|
|
943
5031
|
|
|
944
5032
|
Branch execution based on a condition expression
|
|
945
5033
|
|
|
5034
|
+
**Behavior and examples:** `eigenpal docs read steps/control/if`
|
|
5035
|
+
|
|
946
5036
|
**Durable retry:** The control container itself is not retried, but eligible leaves inside its sequential scope may retry durably.
|
|
947
5037
|
|
|
948
5038
|
**Config** (at step level):
|
|
@@ -959,10 +5049,68 @@ Branch execution based on a condition expression
|
|
|
959
5049
|
| `branch` | `"then"` \| `"else"` | yes | | Which branch was executed |
|
|
960
5050
|
| `result` | unknown | yes | | Output from executed branch |
|
|
961
5051
|
|
|
5052
|
+
##### Complete machine-readable schemas
|
|
5053
|
+
|
|
5054
|
+
Config schema:
|
|
5055
|
+
|
|
5056
|
+
```json
|
|
5057
|
+
{
|
|
5058
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5059
|
+
"type": "object",
|
|
5060
|
+
"properties": {
|
|
5061
|
+
"condition": {
|
|
5062
|
+
"type": "string",
|
|
5063
|
+
"minLength": 1,
|
|
5064
|
+
"description": "LiquidJS expression that evaluates to boolean"
|
|
5065
|
+
}
|
|
5066
|
+
},
|
|
5067
|
+
"required": [
|
|
5068
|
+
"condition"
|
|
5069
|
+
],
|
|
5070
|
+
"additionalProperties": false
|
|
5071
|
+
}
|
|
5072
|
+
```
|
|
5073
|
+
|
|
5074
|
+
|
|
5075
|
+
Output schema:
|
|
5076
|
+
|
|
5077
|
+
```json
|
|
5078
|
+
{
|
|
5079
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5080
|
+
"type": "object",
|
|
5081
|
+
"properties": {
|
|
5082
|
+
"condition": {
|
|
5083
|
+
"type": "boolean",
|
|
5084
|
+
"description": "Evaluated condition result"
|
|
5085
|
+
},
|
|
5086
|
+
"branch": {
|
|
5087
|
+
"type": "string",
|
|
5088
|
+
"enum": [
|
|
5089
|
+
"then",
|
|
5090
|
+
"else"
|
|
5091
|
+
],
|
|
5092
|
+
"description": "Which branch was executed"
|
|
5093
|
+
},
|
|
5094
|
+
"result": {
|
|
5095
|
+
"description": "Output from executed branch"
|
|
5096
|
+
}
|
|
5097
|
+
},
|
|
5098
|
+
"required": [
|
|
5099
|
+
"condition",
|
|
5100
|
+
"branch",
|
|
5101
|
+
"result"
|
|
5102
|
+
],
|
|
5103
|
+
"additionalProperties": false
|
|
5104
|
+
}
|
|
5105
|
+
```
|
|
5106
|
+
|
|
5107
|
+
|
|
962
5108
|
#### `control.switch` — Switch
|
|
963
5109
|
|
|
964
5110
|
Multi-way routing: resolve an expression and run the first case whose value matches (else default). Cleaner than a nested control.if chain for routing an item to one of N pipelines by a discriminator field like a document type.
|
|
965
5111
|
|
|
5112
|
+
**Behavior and examples:** `eigenpal docs read steps/control/switch`
|
|
5113
|
+
|
|
966
5114
|
**Durable retry:** The control container itself is not retried, but eligible leaves inside its sequential scope may retry durably.
|
|
967
5115
|
|
|
968
5116
|
**Config** (at step level):
|
|
@@ -980,10 +5128,115 @@ Multi-way routing: resolve an expression and run the first case whose value matc
|
|
|
980
5128
|
| `branch` | `"case"` \| `"default"` \| `"none"` | yes | | Which branch executed. |
|
|
981
5129
|
| `result` | unknown | yes | | Output from the executed branch (its last step). |
|
|
982
5130
|
|
|
5131
|
+
##### Complete machine-readable schemas
|
|
5132
|
+
|
|
5133
|
+
Config schema:
|
|
5134
|
+
|
|
5135
|
+
```json
|
|
5136
|
+
{
|
|
5137
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5138
|
+
"type": "object",
|
|
5139
|
+
"properties": {
|
|
5140
|
+
"on": {
|
|
5141
|
+
"type": "string",
|
|
5142
|
+
"minLength": 1,
|
|
5143
|
+
"description": "Template expression whose resolved value selects the case, e.g. \"{{ doc.type }}\"."
|
|
5144
|
+
},
|
|
5145
|
+
"cases": {
|
|
5146
|
+
"minItems": 1,
|
|
5147
|
+
"type": "array",
|
|
5148
|
+
"items": {
|
|
5149
|
+
"type": "object",
|
|
5150
|
+
"properties": {
|
|
5151
|
+
"when": {
|
|
5152
|
+
"anyOf": [
|
|
5153
|
+
{
|
|
5154
|
+
"type": "string"
|
|
5155
|
+
},
|
|
5156
|
+
{
|
|
5157
|
+
"type": "number"
|
|
5158
|
+
},
|
|
5159
|
+
{
|
|
5160
|
+
"type": "boolean"
|
|
5161
|
+
}
|
|
5162
|
+
],
|
|
5163
|
+
"description": "Value matched (string-compared) against the resolved `on` value."
|
|
5164
|
+
}
|
|
5165
|
+
},
|
|
5166
|
+
"required": [
|
|
5167
|
+
"when"
|
|
5168
|
+
],
|
|
5169
|
+
"additionalProperties": false
|
|
5170
|
+
},
|
|
5171
|
+
"description": "Ordered cases; the first whose `when` matches runs. Each case has its own `steps`."
|
|
5172
|
+
}
|
|
5173
|
+
},
|
|
5174
|
+
"required": [
|
|
5175
|
+
"on",
|
|
5176
|
+
"cases"
|
|
5177
|
+
],
|
|
5178
|
+
"additionalProperties": false
|
|
5179
|
+
}
|
|
5180
|
+
```
|
|
5181
|
+
|
|
5182
|
+
|
|
5183
|
+
Output schema:
|
|
5184
|
+
|
|
5185
|
+
```json
|
|
5186
|
+
{
|
|
5187
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5188
|
+
"type": "object",
|
|
5189
|
+
"properties": {
|
|
5190
|
+
"matched": {
|
|
5191
|
+
"anyOf": [
|
|
5192
|
+
{
|
|
5193
|
+
"anyOf": [
|
|
5194
|
+
{
|
|
5195
|
+
"type": "string"
|
|
5196
|
+
},
|
|
5197
|
+
{
|
|
5198
|
+
"type": "number"
|
|
5199
|
+
},
|
|
5200
|
+
{
|
|
5201
|
+
"type": "boolean"
|
|
5202
|
+
}
|
|
5203
|
+
]
|
|
5204
|
+
},
|
|
5205
|
+
{
|
|
5206
|
+
"type": "null"
|
|
5207
|
+
}
|
|
5208
|
+
],
|
|
5209
|
+
"description": "The `when` value that matched, or null when the default (or no) branch ran."
|
|
5210
|
+
},
|
|
5211
|
+
"branch": {
|
|
5212
|
+
"type": "string",
|
|
5213
|
+
"enum": [
|
|
5214
|
+
"case",
|
|
5215
|
+
"default",
|
|
5216
|
+
"none"
|
|
5217
|
+
],
|
|
5218
|
+
"description": "Which branch executed."
|
|
5219
|
+
},
|
|
5220
|
+
"result": {
|
|
5221
|
+
"description": "Output from the executed branch (its last step)."
|
|
5222
|
+
}
|
|
5223
|
+
},
|
|
5224
|
+
"required": [
|
|
5225
|
+
"matched",
|
|
5226
|
+
"branch",
|
|
5227
|
+
"result"
|
|
5228
|
+
],
|
|
5229
|
+
"additionalProperties": false
|
|
5230
|
+
}
|
|
5231
|
+
```
|
|
5232
|
+
|
|
5233
|
+
|
|
983
5234
|
#### `control.foreach` — For Each
|
|
984
5235
|
|
|
985
5236
|
Loop over an array and execute steps for each item
|
|
986
5237
|
|
|
5238
|
+
**Behavior and examples:** `eigenpal docs read steps/control/foreach`
|
|
5239
|
+
|
|
987
5240
|
**Durable retry:** The control container itself is not retried, but eligible leaves inside its sequential scope may retry durably.
|
|
988
5241
|
|
|
989
5242
|
**Config** (at step level):
|
|
@@ -1002,10 +5255,76 @@ Loop over an array and execute steps for each item
|
|
|
1002
5255
|
| `count` | number | yes | | Number of completed iterations |
|
|
1003
5256
|
| `totalIterations` | number | yes | | Total iterations |
|
|
1004
5257
|
|
|
5258
|
+
##### Complete machine-readable schemas
|
|
5259
|
+
|
|
5260
|
+
Config schema:
|
|
5261
|
+
|
|
5262
|
+
```json
|
|
5263
|
+
{
|
|
5264
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5265
|
+
"type": "object",
|
|
5266
|
+
"properties": {
|
|
5267
|
+
"items": {
|
|
5268
|
+
"type": "string",
|
|
5269
|
+
"minLength": 1,
|
|
5270
|
+
"description": "Expression resolving to array"
|
|
5271
|
+
},
|
|
5272
|
+
"as": {
|
|
5273
|
+
"type": "string",
|
|
5274
|
+
"minLength": 1,
|
|
5275
|
+
"description": "Variable name for current item"
|
|
5276
|
+
},
|
|
5277
|
+
"indexAs": {
|
|
5278
|
+
"description": "Variable name for current index",
|
|
5279
|
+
"type": "string"
|
|
5280
|
+
}
|
|
5281
|
+
},
|
|
5282
|
+
"required": [
|
|
5283
|
+
"items",
|
|
5284
|
+
"as"
|
|
5285
|
+
],
|
|
5286
|
+
"additionalProperties": false
|
|
5287
|
+
}
|
|
5288
|
+
```
|
|
5289
|
+
|
|
5290
|
+
|
|
5291
|
+
Output schema:
|
|
5292
|
+
|
|
5293
|
+
```json
|
|
5294
|
+
{
|
|
5295
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5296
|
+
"type": "object",
|
|
5297
|
+
"properties": {
|
|
5298
|
+
"items": {
|
|
5299
|
+
"type": "array",
|
|
5300
|
+
"items": {},
|
|
5301
|
+
"description": "Results from each iteration"
|
|
5302
|
+
},
|
|
5303
|
+
"count": {
|
|
5304
|
+
"type": "number",
|
|
5305
|
+
"description": "Number of completed iterations"
|
|
5306
|
+
},
|
|
5307
|
+
"totalIterations": {
|
|
5308
|
+
"type": "number",
|
|
5309
|
+
"description": "Total iterations"
|
|
5310
|
+
}
|
|
5311
|
+
},
|
|
5312
|
+
"required": [
|
|
5313
|
+
"items",
|
|
5314
|
+
"count",
|
|
5315
|
+
"totalIterations"
|
|
5316
|
+
],
|
|
5317
|
+
"additionalProperties": false
|
|
5318
|
+
}
|
|
5319
|
+
```
|
|
5320
|
+
|
|
5321
|
+
|
|
1005
5322
|
#### `control.parallel_map` — Parallel Map
|
|
1006
5323
|
|
|
1007
5324
|
Iterate over an array with concurrent execution up to a limit
|
|
1008
5325
|
|
|
5326
|
+
**Behavior and examples:** `eigenpal docs read steps/control/parallel_map`
|
|
5327
|
+
|
|
1009
5328
|
**Durable retry:** The control container and leaves inside its concurrent branches do not retry durably.
|
|
1010
5329
|
|
|
1011
5330
|
**Config** (at step level):
|
|
@@ -1025,10 +5344,83 @@ Iterate over an array with concurrent execution up to a limit
|
|
|
1025
5344
|
| `count` | number | yes | | Number of completed iterations |
|
|
1026
5345
|
| `totalIterations` | number | yes | | Total iterations |
|
|
1027
5346
|
|
|
5347
|
+
##### Complete machine-readable schemas
|
|
5348
|
+
|
|
5349
|
+
Config schema:
|
|
5350
|
+
|
|
5351
|
+
```json
|
|
5352
|
+
{
|
|
5353
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5354
|
+
"type": "object",
|
|
5355
|
+
"properties": {
|
|
5356
|
+
"items": {
|
|
5357
|
+
"type": "string",
|
|
5358
|
+
"minLength": 1,
|
|
5359
|
+
"description": "Expression resolving to array"
|
|
5360
|
+
},
|
|
5361
|
+
"as": {
|
|
5362
|
+
"type": "string",
|
|
5363
|
+
"minLength": 1,
|
|
5364
|
+
"description": "Variable name for current item"
|
|
5365
|
+
},
|
|
5366
|
+
"indexAs": {
|
|
5367
|
+
"description": "Variable name for current index",
|
|
5368
|
+
"type": "string"
|
|
5369
|
+
},
|
|
5370
|
+
"concurrency": {
|
|
5371
|
+
"description": "Maximum concurrent executions (1-50, default 5)",
|
|
5372
|
+
"default": 5,
|
|
5373
|
+
"type": "integer",
|
|
5374
|
+
"minimum": 1,
|
|
5375
|
+
"maximum": 50
|
|
5376
|
+
}
|
|
5377
|
+
},
|
|
5378
|
+
"required": [
|
|
5379
|
+
"items",
|
|
5380
|
+
"as"
|
|
5381
|
+
],
|
|
5382
|
+
"additionalProperties": false
|
|
5383
|
+
}
|
|
5384
|
+
```
|
|
5385
|
+
|
|
5386
|
+
|
|
5387
|
+
Output schema:
|
|
5388
|
+
|
|
5389
|
+
```json
|
|
5390
|
+
{
|
|
5391
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5392
|
+
"type": "object",
|
|
5393
|
+
"properties": {
|
|
5394
|
+
"items": {
|
|
5395
|
+
"type": "array",
|
|
5396
|
+
"items": {},
|
|
5397
|
+
"description": "Results from each iteration (maintains original order)"
|
|
5398
|
+
},
|
|
5399
|
+
"count": {
|
|
5400
|
+
"type": "number",
|
|
5401
|
+
"description": "Number of completed iterations"
|
|
5402
|
+
},
|
|
5403
|
+
"totalIterations": {
|
|
5404
|
+
"type": "number",
|
|
5405
|
+
"description": "Total iterations"
|
|
5406
|
+
}
|
|
5407
|
+
},
|
|
5408
|
+
"required": [
|
|
5409
|
+
"items",
|
|
5410
|
+
"count",
|
|
5411
|
+
"totalIterations"
|
|
5412
|
+
],
|
|
5413
|
+
"additionalProperties": false
|
|
5414
|
+
}
|
|
5415
|
+
```
|
|
5416
|
+
|
|
5417
|
+
|
|
1028
5418
|
#### `control.parallel` — Parallel
|
|
1029
5419
|
|
|
1030
5420
|
Execute multiple branches concurrently
|
|
1031
5421
|
|
|
5422
|
+
**Behavior and examples:** `eigenpal docs read steps/control/parallel`
|
|
5423
|
+
|
|
1032
5424
|
**Durable retry:** The control container and leaves inside its concurrent branches do not retry durably.
|
|
1033
5425
|
|
|
1034
5426
|
**Config** (at step level):
|
|
@@ -1039,10 +5431,41 @@ _No fields._
|
|
|
1039
5431
|
|
|
1040
5432
|
> Results keyed by branch name
|
|
1041
5433
|
|
|
5434
|
+
##### Complete machine-readable schemas
|
|
5435
|
+
|
|
5436
|
+
Config schema:
|
|
5437
|
+
|
|
5438
|
+
```json
|
|
5439
|
+
{
|
|
5440
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5441
|
+
"type": "object",
|
|
5442
|
+
"properties": {},
|
|
5443
|
+
"additionalProperties": false
|
|
5444
|
+
}
|
|
5445
|
+
```
|
|
5446
|
+
|
|
5447
|
+
|
|
5448
|
+
Output schema:
|
|
5449
|
+
|
|
5450
|
+
```json
|
|
5451
|
+
{
|
|
5452
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5453
|
+
"type": "object",
|
|
5454
|
+
"propertyNames": {
|
|
5455
|
+
"type": "string"
|
|
5456
|
+
},
|
|
5457
|
+
"additionalProperties": {},
|
|
5458
|
+
"description": "Results keyed by branch name"
|
|
5459
|
+
}
|
|
5460
|
+
```
|
|
5461
|
+
|
|
5462
|
+
|
|
1042
5463
|
#### `control.wait` — Wait
|
|
1043
5464
|
|
|
1044
5465
|
Pause workflow execution for a specified duration
|
|
1045
5466
|
|
|
5467
|
+
**Behavior and examples:** `eigenpal docs read steps/control/wait`
|
|
5468
|
+
|
|
1046
5469
|
**Durable retry:** This control step is not retried durably.
|
|
1047
5470
|
|
|
1048
5471
|
**Config** (at step level):
|
|
@@ -1057,10 +5480,55 @@ Pause workflow execution for a specified duration
|
|
|
1057
5480
|
| --- | --- | --- | --- | --- |
|
|
1058
5481
|
| `waited` | number | yes | | Actual milliseconds waited |
|
|
1059
5482
|
|
|
5483
|
+
##### Complete machine-readable schemas
|
|
5484
|
+
|
|
5485
|
+
Config schema:
|
|
5486
|
+
|
|
5487
|
+
```json
|
|
5488
|
+
{
|
|
5489
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5490
|
+
"type": "object",
|
|
5491
|
+
"properties": {
|
|
5492
|
+
"duration": {
|
|
5493
|
+
"type": "number",
|
|
5494
|
+
"exclusiveMinimum": 0,
|
|
5495
|
+
"description": "Duration in milliseconds"
|
|
5496
|
+
}
|
|
5497
|
+
},
|
|
5498
|
+
"required": [
|
|
5499
|
+
"duration"
|
|
5500
|
+
],
|
|
5501
|
+
"additionalProperties": false
|
|
5502
|
+
}
|
|
5503
|
+
```
|
|
5504
|
+
|
|
5505
|
+
|
|
5506
|
+
Output schema:
|
|
5507
|
+
|
|
5508
|
+
```json
|
|
5509
|
+
{
|
|
5510
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5511
|
+
"type": "object",
|
|
5512
|
+
"properties": {
|
|
5513
|
+
"waited": {
|
|
5514
|
+
"type": "number",
|
|
5515
|
+
"description": "Actual milliseconds waited"
|
|
5516
|
+
}
|
|
5517
|
+
},
|
|
5518
|
+
"required": [
|
|
5519
|
+
"waited"
|
|
5520
|
+
],
|
|
5521
|
+
"additionalProperties": false
|
|
5522
|
+
}
|
|
5523
|
+
```
|
|
5524
|
+
|
|
5525
|
+
|
|
1060
5526
|
#### `control.fail` — Fail
|
|
1061
5527
|
|
|
1062
5528
|
Terminate the workflow with a typed status code + message. With an optional condition, only fails when the condition is truthy; otherwise always fails when reached. Pair with ai.classify or any prior step to fail fast on bad inputs.
|
|
1063
5529
|
|
|
5530
|
+
**Behavior and examples:** `eigenpal docs read steps/control/fail`
|
|
5531
|
+
|
|
1064
5532
|
**Durable retry:** This control step is not retried durably.
|
|
1065
5533
|
|
|
1066
5534
|
**Config** (at step level):
|
|
@@ -1074,4 +5542,49 @@ Terminate the workflow with a typed status code + message. With an optional cond
|
|
|
1074
5542
|
**Output:** `unknown`
|
|
1075
5543
|
|
|
1076
5544
|
> control.fail never produces output — it terminates the workflow when triggered.
|
|
5545
|
+
|
|
5546
|
+
##### Complete machine-readable schemas
|
|
5547
|
+
|
|
5548
|
+
Config schema:
|
|
5549
|
+
|
|
5550
|
+
```json
|
|
5551
|
+
{
|
|
5552
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5553
|
+
"type": "object",
|
|
5554
|
+
"properties": {
|
|
5555
|
+
"condition": {
|
|
5556
|
+
"description": "Optional LiquidJS expression. When set, the step only fails if this evaluates truthy; when omitted, it always fails when reached (compose with control.if for legacy gating).",
|
|
5557
|
+
"type": "string"
|
|
5558
|
+
},
|
|
5559
|
+
"statusCode": {
|
|
5560
|
+
"default": 422,
|
|
5561
|
+
"description": "HTTP-style status code returned to the caller (sync runs) and persisted on the execution. Default 422 (Unprocessable Entity).",
|
|
5562
|
+
"type": "integer",
|
|
5563
|
+
"minimum": 400,
|
|
5564
|
+
"maximum": 599
|
|
5565
|
+
},
|
|
5566
|
+
"message": {
|
|
5567
|
+
"type": "string",
|
|
5568
|
+
"minLength": 1,
|
|
5569
|
+
"description": "Human-readable failure message. Supports template expressions, e.g. \"Document classified as {{ steps.classify.output.label }}\"."
|
|
5570
|
+
}
|
|
5571
|
+
},
|
|
5572
|
+
"required": [
|
|
5573
|
+
"statusCode",
|
|
5574
|
+
"message"
|
|
5575
|
+
],
|
|
5576
|
+
"additionalProperties": false
|
|
5577
|
+
}
|
|
5578
|
+
```
|
|
5579
|
+
|
|
5580
|
+
|
|
5581
|
+
Output schema:
|
|
5582
|
+
|
|
5583
|
+
```json
|
|
5584
|
+
{
|
|
5585
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
5586
|
+
"not": {},
|
|
5587
|
+
"description": "control.fail never produces output — it terminates the workflow when triggered."
|
|
5588
|
+
}
|
|
5589
|
+
```
|
|
1077
5590
|
<!-- GENERATED:STEP_CATALOG END -->
|