@vaudience/aigentflow-flow-validator 0.15.3 → 0.19.0

package/PARITY.md

@@ -4,7 +4,62 @@ This document maps every rule in this JavaScript validator back to the AIgentFlo
 Go reference implementation, records the intentional divergences, and defines the
 discipline for keeping the two in sync.
 
-**Tracks AIgentFlow flow schema: `v2.802.0`** (`SPEC_VERSION` in [`src/spec/aigentflow-spec.json`](./src/spec/aigentflow-spec.json)), plus `server_owned_query_key` (package 0.15.1, below) from the AIgentFlow release after v2.790.0, and `credential_endpoint_unpaired` (package 0.15.2, below) from AIgentFlow v2.793.0 and the release after it, and `display_name_too_long` (package 0.15.3, below) from AIgentFlow v2.802.0.
+**Tracks AIgentFlow flow schema: `v2.820.0`** (`SPEC_VERSION` in [`src/spec/aigentflow-spec.json`](./src/spec/aigentflow-spec.json)), which adds `orchestrator.region` / `orchestrator.enforce_region` (package 0.19.0, below). Earlier: the AigentChat executor was removed (package 0.18.0). Earlier additions: the `examples:` block (package 0.17.0), `server_owned_query_key` (package 0.15.1) from the AIgentFlow release after v2.790.0, `credential_endpoint_unpaired` (package 0.15.2) from AIgentFlow v2.793.0 and the release after it, and `display_name_too_long` (package 0.15.3) from AIgentFlow v2.802.0.
+
+> package 0.19.0 — **`orchestrator.region` and `orchestrator.enforce_region`**, in step
+> with the reference (AIgentFlow v2.820.0; `specVersion` 2.820.0). Both are new keys of
+> the `orchestrator:` block (a `string` and a `bool`), added to the known-key table.
+> One rule ported: `enforce_region: true` with an absent, null, empty or
+> whitespace-only `region` is an **error**, `orchestrator_enforce_region_needs_region`
+> (field `orchestrator.enforce_region`), because the provider would refuse every turn.
+> The reference raises it from its parser's orchestrator validation as a plain
+> flow-validation error with no machine-readable code; this package's code and field
+> are its own naming, the same as for `orchestrator_mode_invalid`. Only a YAML boolean
+> `true` arms the rule. The runtime effect (the region reaching the provider request) is
+> engine behaviour and out of scope. New conformance fixtures:
+> `valid-orchestrator-region.yaml`, `invalid-orchestrator-enforce-region-no-region.yaml`.
+>
+> package 0.18.0 — **the AigentChat executor is removed**, in step with the reference
+> (AIgentFlow v2.819.0; `specVersion` 2.819.0). The reference no longer registers the
+> `nexus://` protocol or its `aigentchat://` alias, so the spec drops: `nexus` from
+> `executorSchemes` (42 protocols); the `nexus` and `aigentchat` keys of
+> `executorConfigEnvScopes.scopes`; the `nexus` row of `credentialEndpointPairing.families`
+> together with `protocolAliases` and the row-level `storedKeyShape` shape (no row uses
+> them now, so the validator code for them is gone too). A `nexus://` or `aigentchat://`
+> step is now an ordinary unknown scheme: `unknown_executor_scheme` (warning), no
+> credential/endpoint pairing, and a `${AIGENTFLOW_NEXUS_*}` reference under an
+> `executor_config.nexus` key is judged like any key with an empty scope
+> (`executor_config_env_scope`). `executor_config.*.service_channel` and
+> `enable_streaming` stay known keys; the reference keeps them declared but inert.
+> Fixtures: `warn-credential-endpoint-nexus.yaml` is deleted, and the nexus steps and
+> scope entry are removed from `valid-credential-endpoint-own-key-own-url.yaml` and
+> `valid-executor-config-env-scope.yaml`.
+
+> package 0.17.0 — **the `examples:` block**, in step with the reference. The spec is
+> regenerated (`specVersion` 2.811.0): `knownKeys` gains `ExampleDefinition`,
+> `Expectation`, `Checkpoint`, `FieldMatcher`, `FileRef`, `ExampleReference` and
+> `ExampleVariant`, `Flow.examples`, and the `trusted_output` / `allow_untrusted` /
+> `untrusted` keys the previous spec lacked; and a new top-level `examples` section
+> carries every limit, vocabulary, pattern, the credential-shaped-literal set and each
+> code's severity. Rules and divergences are under [The examples block](#the-examples-block).
+>
+> | Rule (reference)                                           | Codes                                           | Severity       | Module                                                |
+> | ---------------------------------------------------------- | ----------------------------------------------- | -------------- | ----------------------------------------------------- |
+> | the `examples:` walker (every rule, one pass over a block) | 58 `example_*` / `examples_*` codes, spec table | error, warning | `validators/examples.ts`, spec `examples`, `goUrl.ts` |
+> | the reference's blocked-host rule                          | `example_ref_blocked_host`                      | error          | **not ported** (divergence #19)                       |
+
+> package 0.16.0 — **the untrusted-input marker keys** (AIgentFlow v2.811.0).
+> The spec's `knownKeys` gain `untrusted` on `QueryDefinition` and `InputSchemaField`, and
+> `trusted_output` / `allow_untrusted` on `StepDefinition` and `LoopStepDefinition`;
+> `specVersion` becomes `2.811.0`. No new rule: a misspelling of any of them is
+> `unknown_yaml_key`, as for every other key. What the keys DO — a marked input is sealed
+> in model-facing steps and placed only fenced in the user message, a tainted value in a
+> sensitive data-executor parameter refuses the step unless `allow_untrusted` names it,
+> `trusted_output` declassifies a step's output — is engine behaviour at run time and out
+> of scope for static validation. Fixtures: `valid-untrusted-inputs.yaml`,
+> `invalid-untrusted-misspelt.yaml`; focused test `v0160-untrusted-inputs.test.ts`.
+> ||||||| /tmp/b.txt
+> **Tracks AIgentFlow flow schema: `v2.802.0`** (`SPEC_VERSION` in [`src/spec/aigentflow-spec.json`](./src/spec/aigentflow-spec.json)), plus `server_owned_query_key` (package 0.15.1, below) from the AIgentFlow release after v2.790.0, and `credential_endpoint_unpaired` (package 0.15.2, below) from AIgentFlow v2.793.0 and the release after it, and `display_name_too_long` (package 0.15.3, below) from AIgentFlow v2.802.0.
 
 > package 0.15.3 — **the optional top-level `display_name`** (AIgentFlow #187). The spec
 > gains `display_name: scalar` on `Flow` and `specVersion` becomes `2.802.0`.
@@ -46,7 +101,7 @@ discipline for keeping the two in sync.
 >   is exactly `${NAME}` for one of the row's server variables on a row whose plugin
 >   expands references, and its endpoint (same order) is a literal: not templated, not a
 >   reference, and not of the same origin as the row's default. Implicit-credential
->   families are not judged; `nexus://` and `aigentchat://` get the ai:// shape.
+>   families are not judged.
 > - **The default-origin comparison needs Go's URL semantics**, so `validators/goUrl.ts`
 >   ports `net/url.Parse` (what it refuses, and the scheme and host it extracts),
 >   `strings.TrimSpace` (Go trims U+0085 and keeps U+FEFF) and `strings.ToLower` (simple
@@ -856,7 +911,7 @@ max_retries` and `QualityGateDefinition.max_retries` are unchanged. Removing
 >    separate target functions; widening `hasCycle` too would invent a divergence
 >    and emit a spurious `potential_infinite_loop` on any rendezvous or error
 >    redirect back to an earlier step.
-> 4. **`executorSchemes` is AIgentFlow's registered set exactly** (43 protocols,
+> 4. **`executorSchemes` is AIgentFlow's registered set exactly** (43 protocols at v2.608.0, 42 since the AigentChat executor was removed,
 >    enumerated from `NewExecutorSchemaRegistry`). `web://` was missing, so a
 >    correct step warned; nine schemes AIgentFlow does not register (`openai`,
 >    `anthropic`, `perplexity`, `vertexai`, `ollama`, `vllm`, `aigentchat`,
@@ -953,6 +1008,7 @@ Comparison contract: **error `code` + `valid` verdict**, not message wording. Th
 | `executor_config` `${NAME}` references outside the key's scope (`executor_config_env_scope`)                            | `ValidateExecutorConfigEnvScopes` (parser.go)                                                           | `validators/executorConfigEnv.ts`                         | `v0150-save-door.test.ts`, `conformance.test.ts`           |
 | A step or loop sub-step `query:` declaring a server-owned key (`server_owned_query_key`)                                | `validateNoServerOwnedStepQueryKeys` (validation.go), `IsServerOwnedParamKey`                           | `validators/serverOwnedQueryKeys.ts`                      | `v0151-save-door.test.ts`, `conformance.test.ts`           |
 | An endpoint a server-supplied credential will never be sent to (`credential_endpoint_unpaired`, warning)                | `validateCredentialEndpointPairing`, `validateFamilyCredentialEndpointPairing` (validation.go)          | `validators/credentialEndpoint.ts`, `validators/goUrl.ts` | `v0152-credential-endpoint.test.ts`, `conformance.test.ts` |
+| The `examples:` block: 58 codes, one walker (ids, inputs, references, expectations, checkpoints, variants)              | the examples walker (`ValidateExamples`)                                                                | `validators/examples.ts`, `validators/goUrl.ts`           | `examples.test.ts`, `conformance.test.ts`                  |
 | Orchestrator `.exons` frontmatter: spec, provider, `requirements.resources`, `tools.allow` withholds                    | `validateOrchestrator` (parser.go), `validateOrchestratorToolAllowWithholds`                            | `validators/exons.ts`                                     | `v0150-save-door.test.ts`, `conformance.test.ts`           |
 | Credential bindings (`stored/...`, inject_as, exclusivity)                                                              | `validateStepCredentialBindings`                                                                        | `validators/credentialBindings.ts`                        | `validate.test.ts`                                         |
 | `input_schema` definition + ordering lint                                                                               | `ValidateInputSchemaDefinition`, `LintInputSchemaFieldOrdering`                                         | `validators/inputSchema.ts`                               | `validate.test.ts`                                         |
@@ -1155,10 +1211,10 @@ number their shared divergences alike._
     base URL), the service configuration an executor fills, and whether a client attaches
     the deployment's service credential. Neither side reads them statically.
     - **Default endpoints of rows the rule never reads are not carried** (`getmd://`, which
-      is never judged, and `nexus://`, whose shape consults no default). Verdict-neutral.
+      is never judged). Verdict-neutral.
     - **A `credentials:` mapping with a non-string key (looser, shared).** The reference
-      decodes it into `map[any]any`, which its nexus own-key check does not accept, so the
-      endpoint warns there; both ports stringify mapping keys and see a key of the step's own.
+      decoded it into `map[any]any` for the since-removed `nexus://` own-key check; with that
+      family gone the shared looseness no longer has a rule to act on.
     - **JS only, looser: an endpoint shaped like a YAML timestamp is not read as a string.**
       yaml.v3 decodes a PLAIN `2024-01-01` into `time.Time`, which the reference's
       `.(string)` rejects; the `yaml` package gives a string and the source's quoting is not
@@ -1168,6 +1224,135 @@ number their shared divergences alike._
     - **Not ported, verdict-neutral:** Go validates a bracketed IPv6 host with
       `netip.ParseAddr`; a bracketed host is never a default's host either way.
 
+19. **`example_ref_blocked_host` is reference-only (0.16.0).** The reference refuses a file
+    reference whose host is private or local (`localhost`, loopback, link-local and cloud
+    metadata addresses, RFC 1918 and carrier-grade ranges, the unspecified address, and any
+    name ending `.internal`, `.local` or `.localhost`). It reuses the server's own address
+    tables, so a second literal table here would be another copy to keep in step; it is also
+    advice, because the control is the run-time dialer. `https://localhost/a.json` therefore
+    passes here (looser). The code is still listed in the spec's `examples.codes` table (it is
+    the reference's code), and no fixture in this repository raises it.
+
+20. **The example's inline input is judged by a NARROWED check (0.16.0; looser).** The
+    reference judges an example's `input` (and a variant's patched input) with its full
+    run-time input validator. This port has no value validator, so it judges only: an unknown
+    field against `input_schema` (or against the declared `query` keys when there is no
+    schema), a required field that is missing (honouring `visible_when`, and skipping a
+    required `secret` field, which the person running the evaluation supplies), a value of the
+    wrong KIND (`string`, `multiline`, `secret`, `enum` and `date` are strings; `number` is any
+    YAML number, `.nan` and the infinities included, as the reference admits them; `bool` is a
+    boolean; `array_of_strings` is a list of strings; `file` is a mapping whose keys are only
+    `url`, `sha256`, `media_type` and `note`, else `example_file_input_needs_ref`) and an enum
+    value outside its list. Not judged: `min_length` / `max_length`, `min` / `max`,
+    `min_items` / `max_items`, `pattern`, the `YYYY-MM-DD` shape and calendar of a `date`, the
+    256-key cap and the string-length cap. A document the reference accepts is never refused
+    here; one it refuses only for those constraints is accepted here. The message wording of
+    the four judged failures is the reference's, and the same `(code, field)` is reported.
+    A date or timestamp a loader resolved to a date object (`validateFlowObject`) is read as the
+    string a JSON body would carry; the `yaml` package this validator parses with leaves such a
+    scalar a string, so for parsed text this is the identity.
+
+21. **A field matcher's `matches` pattern is judged by length only (0.16.0; looser).** The
+    reference compiles the pattern with its regular-expression engine (RE2), and RE2 and a
+    JavaScript `RegExp` disagree about what is valid (`(?P<n>x)` is valid RE2 and a JS syntax
+    error), so a compile check here could refuse what the reference accepts. This port judges
+    the length (the spec's `limits.maxPatternLength`, counted in UTF-8 bytes as Go counts it)
+    and the "exactly one operator" rule, and nothing about the pattern's syntax: `(` passes.
+
+22. **Counts and wording that differ in the margin (0.16.0).**
+    - The byte counts behind `example_input_too_large` and `examples_block_too_large` are
+      measured over the JSON of the parsed document (for the block, with empty values
+      omitted, as the reference's typed structs omit them), not over the reference's typed
+      marshalling, which also escapes `<`, `>` and `&`. The count here is never larger than the
+      reference's, so the verdict can differ only within a few bytes of the limit, and only
+      looser; the number printed in the message can differ by those bytes.
+    - `example_expectation_contradiction` compares with simple case folding per code point,
+      as `strings.EqualFold` does; a code point whose fold orbit has three members is not
+      followed.
+    - A float in a message (`weight`, `min_score`) is printed as Go prints a `float64` with
+      `%v` (shortest form, an exponent below `1e-4` and from `1e6`).
+    - The recursion bound of 32 on a nested value is the reference's own and is not in the
+      spec.
+
+---
+
+## The examples block
+
+An `examples:` list records concrete cases inside the flow document: an input, what a good
+result looks like, why the case matters. The engine never reads it; the validator is the only
+reader here, so every rule is a judgement of the DOCUMENT at save time, pure and offline (no
+reference is fetched and no host is resolved). One walker, `validators/examples.ts`, reports
+every finding of a block in one pass; each finding's `field` is an index path
+(`examples[2].expected.fields.total`, `examples[0].variants[1].id`), never an id-based one,
+because an id can be invalid or duplicate while it is being reported.
+
+**What is judged**, in the order the walker reads it:
+
+- **The set.** At most 50 examples (`examples_too_many`); the block's JSON at most 256 KiB
+  (`examples_block_too_large`); a warning when every example is held out
+  (`examples_all_held_out`) and when the flow is public (`examples_published_with_public_flow`).
+- **Identity and prose.** `id` required, matching the spec's `idPattern`, unique
+  (`example_id_missing` / `_invalid` / `_duplicate`); `title` non-blank, at most 80 code points;
+  `guidance` at most 2000 code points, and a warning when it is absent
+  (`example_guidance_missing`); `notes` at most 2000; `tags` at most 8, each matching
+  `tagPattern`, none repeated (`example_tag_invalid`); `weight` above 0 and at most 10;
+  `min_score` from 0 to 1; `side_effects` `refuse` or `allow`; `hold_out` refused on a public
+  flow (`example_holdout_in_public_flow`).
+- **Input.** Exactly one of `input` / `input_ref` (`example_input_conflict`,
+  `example_input_missing`; `input: {}` is a present, empty input); an inline input at most 32 KiB
+  and judged as divergence #20 describes (`example_input_invalid`, `example_file_input_needs_ref`,
+  `example_input_unvalidated` when the flow declares neither `input_schema` nor `query`).
+  A value for a `secret` field is refused (`example_secret_value`), because examples are saved
+  inside the flow.
+- **File references** (`input_ref`, a file-typed input, `expected.reference.ref`): `url` at most
+  2048 bytes and parseable as Go parses a URL; a reserved scheme (`artifact`, `trove`, `aiv`,
+  `s3`, `gs`) is refused by name (`example_ref_scheme_reserved`), any other non-`https` scheme
+  too (`example_ref_scheme`); a login, a fragment or an empty host is `example_ref_invalid`;
+  `sha256` is 64 lower-case hex digits, `media_type` is `type/subtype`, `note` at most 300;
+  warnings for a reference with no `sha256` (`example_ref_unpinned`) and for a signed-link query
+  key (`example_ref_signed_url`).
+- **Expectations** (`expected`, a checkpoint, a variant's replacing `expected`): `status` one of
+  `completed`, `failed`, `paused_for_human`; `fields` and `exact` are mutually exclusive and
+  absent when the case is meant to end `failed` or `paused_for_human`
+  (`example_expected_conflict`); every `fields` / `exact` key is an output of the flow (a
+  checkpoint: a field of the step's `output_schema`), or a warning when no list is declared;
+  each field matcher has exactly one operator and `approx` pairs with a non-negative
+  `tolerance` (`example_field_matcher_invalid`); `rubric` non-blank and at most 4000;
+  `reference` has exactly one of `text` / `ref`, text at most 32 KiB, and a warning for a
+  reference that is not text; `must_not` at most 12 statements of at most 300, none blank;
+  `must_not_contain` at most 20 literals of 1 to 200; a warning when a field's `equals` is also
+  forbidden by `must_not_contain`; an expectation must check something
+  (`example_expected_empty`; `status: completed` alone checks nothing), and a missing
+  `expected` is `example_expected_missing`. A reference the judge cannot read, alone, is the
+  warning `examples_unreadable_by_judge`.
+- **Checkpoints.** The key must be a step of the flow, and not a `for_each`, `loop` or
+  `next.parallel` step (`example_checkpoint_step_unknown`, `_composite`); a checkpoint cannot
+  name a `status` (`example_checkpoint_status`).
+- **Variants.** At most 10; `id` matching `idPattern` and unique within the example;
+  `origin` required with no default, `author` or `synthetic`; a variant must change the input
+  or the expectation, and sets `input_patch` or `input_ref`, not both. The patch is an RFC 7386
+  merge patch applied to the parent's inline input (a mapping merges, `null` deletes, anything
+  else replaces; the input is never mutated) and the patched input is judged like the example's
+  own (`example_variant_input_invalid`); a patch giving a secret a value is refused
+  (`example_secret_value`), one deleting it with `null` is fine; a patch on a parent whose
+  input is `input_ref` cannot be checked (`example_variant_unchecked`).
+- **Credential-shaped literals.** Every string leaf of an example, a checkpoint and a variant
+  (title, guidance, notes, input, rubric, reference text, must_not, matcher operands, `exact`,
+  patches) is matched against the spec's eight patterns (`example_secret_like_value`). The
+  patterns are data (`examples.secretPatterns`, each with its flags); ordinary hyphenated slugs
+  and prose must not match, and a test holds that.
+- **Unknown keys.** A typo at example, expectation, matcher, file-reference or variant depth is
+  `unknown_yaml_key`, from the regenerated `knownKeys`: nothing is hand-listed.
+
+Severities are the spec's `examples.codes` table, held to the walker by a test (each code's
+fixture asserts the walker's severity equals the table's).
+
+**Measured.** The conformance corpus is shared byte for byte with the Go port; every fixture
+that carries an expected `(code, field, message)` agrees here, but the one divergence #21 names.
+Mutants of the rules (a secret pattern dropped, the id check disabled, the duplicate check,
+the weight bound, the origin vocabulary, the unknown-checkpoint code, the https-only scheme
+rule, the empty-expectation rule, the merge patch's null-delete) are each killed by a test.
+
 ---
 
 ## Unknown keys and value kinds

package/README.md

@@ -116,7 +116,7 @@ Low-level: parse YAML into `{ flow?, parseErrors, parseWarnings }` without valid
 
 ### `SPEC_VERSION: string`
 
-The AIgentFlow flow-schema version this build tracks (e.g. `"2.802.0"`).
+The AIgentFlow flow-schema version this build tracks (e.g. `"2.820.0"`).
 
 ### Options
 
@@ -174,7 +174,7 @@ interface ValidationIssue {
 - **Orchestrator `.exons` frontmatter** — the document must open with a closed `---` frontmatter spec (`orchestrator_exons_parse_failed`) that sets `execution.provider` (`orchestrator_exons_no_provider`) and declares no `requirements.resources` (`exons_resources_unhonoured`); a tool that `orchestrator.tools` and the definition's `tools.allow` disagree about is never offered (`orchestrator_tool_withheld`, warning). The document body is not parsed: that needs the `.exons` engine (see PARITY.md).
 - **`executor_config` environment references** — a `${NAME}` value may name only the variables of the provider or protocol it is written under (`executor_config_env_scope`).
 - **Server-owned query keys** — a step's or loop sub-step's `query:` may not declare `aiv_api_key`, `aiv_base_url` or `aiv_delegation`, which only the server sets (`server_owned_query_key`).
-- **Credential endpoints** — an endpoint a server-supplied credential will never be sent to (warning, `credential_endpoint_unpaired`): an `ai://` `<provider>_base_url` or a literal `executor_config.<provider>.base_url` with no key of the flow's own, a family secret that is a server `${AIGENTFLOW_…}` variable beside a literal non-default endpoint, a `nexus://` endpoint with no key of its own.
+- **Credential endpoints** — an endpoint a server-supplied credential will never be sent to (warning, `credential_endpoint_unpaired`): an `ai://` `<provider>_base_url` or a literal `executor_config.<provider>.base_url` with no key of the flow's own, a family secret that is a server `${AIGENTFLOW_…}` variable beside a literal non-default endpoint.
 - **Credential bindings** — `stored/{provider}/{name}` format, `inject_as`, `credential`/`credentials` mutual exclusion.
 - **Expression functions** — exactly one of `package`/`function`.
 - **`input_schema`** — version, field-name pattern, type enum, per-type constraints, `pattern` compilation, `visible_when` predicate + reference resolution, duplicate-name detection, file-ordering lint.