@particle-academy/fancy-conformance 0.5.0 → 0.9.0

package/suites/flow/graph-runs/manifest.json

@@ -0,0 +1,27 @@
+{
+  "$schema": "../../../schema/suite-manifest.schema.json",
+  "suite": "flow/graph-runs",
+  "title": "Whole-graph execution: the same WorkflowSchema in, the same outputs out",
+  "since": "0.6.0",
+  "caseFormat": "table",
+  "cases": "cases.json",
+  "contract": {
+    "function": "runGraph(schema: WorkflowSchema, initialInputs: object) -> { ok: true, outputs: object } | { ok: false, error: string }",
+    "summary": "Import a WorkflowSchema v1 document leniently against the built-in kind registry (including the structural kinds), run it through the engine with the built-in offline executors, and return the run's outputs. This is the end-to-end parity contract for the fancy-flow engine: Kahn topological order, the three port-activation conventions, branch routing, dead-edge handling at merge points, cycle detection, and the closed failure of an unregistered kind.",
+    "reference": "php",
+    "referenceNote": "The goldens were produced by running particle-academy/fancy-flow-php, where they lived as tests/Parity/fixtures/ before being promoted here. Two were CORRECTED rather than transliterated during the promotion; each says so in its own notes.",
+    "implementations": [
+      { "language": "php", "package": "particle-academy/fancy-flow-php", "symbol": "FancyFlow\\Engine\\FlowRunner" },
+      { "language": "python", "package": "fancy-flow", "symbol": "fancy_flow.engine.FlowRunner" },
+      { "language": "rust", "package": "fancy-flow", "symbol": "fancy_flow::engine::FlowRunner" },
+      { "language": "node", "package": "@particle-academy/fancy-flow", "symbol": "runFlow" }
+    ]
+  },
+  "notes": [
+    "These 23 cases spent their whole life as a COPY. They were fancy-flow-php's own test fixtures; fancy-flow-py duplicated the directory byte for byte and held the two together with a provenance test, and its fixtures/SOURCE.md said plainly that the real fix was to promote them here. A fourth runtime is what made a third copy indefensible.",
+    "The reference implementation asserting these rows is the PHP twin. `@particle-academy/fancy-flow` — the runtime whose run-flow.ts is the behavioural source of truth for all of them — did NOT run them at any point while they were private fixtures, so the goldens describing its behaviour had zero implementations checking them on its own side. Wiring the Node runner is the remaining half of this promotion.",
+    "The run is deliberately fully specified rather than left to the runner: LENIENT import, a LOCAL kind registry with the structural kinds registered, and the built-in offline executors. A runner that populates the SHARED kind registry instead gets different declared output ports for `for_each` and disagrees on a case nobody changed.",
+    "Two goldens were corrected on promotion. `0014` recorded PHP's ENCODING of an empty header map (`[]`) rather than its value; the shared golden is `{}`, which PHP still satisfies because its loader decodes to assoc arrays. `0021` and `0022` asserted a SUBSTRING (`errorContains`) and now assert the exact message — the substring is what hid a live divergence in which Python emitted an ASCII hyphen where PHP and TypeScript emit an em dash.",
+    "There is NO discrimination probe for this suite, and that is a real gap rather than an oversight. Every other suite here ships deliberately-wrong implementations that must fail an exact set of ids; the implementation under test here is an entire workflow engine, which this package does not carry and should not grow. Until a probe exists, these rows guard DRIFT between four runtimes but do not prove they discriminate — read the green tick as the weaker claim it is."
+  ]
+}

package/suites/flow/subflow-registry/cases.json

@@ -0,0 +1,126 @@
+{
+  "$schema": "../../../schema/cases.schema.json",
+  "suite": "flow/subflow-registry",
+  "cases": [
+    {
+      "id": "0001-host-kind-survives-nesting",
+      "title": "A kind the HOST registered is still resolvable inside the child.",
+      "since": "0.8.0",
+      "tags": ["inheritance", "regression"],
+      "input": {
+        "parent": { "@particle-academy/output": "builtin", "host_kind": "host" },
+        "injected": null,
+        "graphConfig": null
+      },
+      "expected": { "@particle-academy/output": "builtin", "host_kind": "host" }
+    },
+    {
+      "id": "0002-bare-builtins-are-not-enough",
+      "title": "A parent carrying several host kinds passes all of them down, not just the builtins.",
+      "since": "0.8.0",
+      "tags": ["inheritance", "regression"],
+      "input": {
+        "parent": {
+          "@particle-academy/output": "builtin",
+          "@particle-academy/transform": "builtin",
+          "agent": "host",
+          "invoice_lookup": "host",
+          "tenant_notify": "host"
+        },
+        "injected": null,
+        "graphConfig": null
+      },
+      "expected": {
+        "@particle-academy/output": "builtin",
+        "@particle-academy/transform": "builtin",
+        "agent": "host",
+        "invoice_lookup": "host",
+        "tenant_notify": "host"
+      }
+    },
+    {
+      "id": "0003-host-override-of-a-builtin-reaches-the-child",
+      "title": "A host that REPLACED a builtin keeps its own version inside the child.",
+      "since": "0.8.0",
+      "tags": ["inheritance", "override", "regression"],
+      "input": {
+        "parent": { "@particle-academy/llm_call": "host", "@particle-academy/output": "builtin" },
+        "injected": null,
+        "graphConfig": null
+      },
+      "expected": { "@particle-academy/llm_call": "host", "@particle-academy/output": "builtin" }
+    },
+    {
+      "id": "0004-graph-config-adds-to-the-inherited-set",
+      "title": "A registry the graph declares on the node is layered ON TOP of the inherited one.",
+      "since": "0.8.0",
+      "tags": ["config"],
+      "input": {
+        "parent": { "@particle-academy/output": "builtin", "host_kind": "host" },
+        "injected": null,
+        "graphConfig": { "child_only": "child-specific" }
+      },
+      "expected": {
+        "@particle-academy/output": "builtin",
+        "host_kind": "host",
+        "child_only": "child-specific"
+      }
+    },
+    {
+      "id": "0005-graph-config-wins-on-a-conflict",
+      "title": "An executor the graph names for its child beats the inherited one of the same kind.",
+      "since": "0.8.0",
+      "tags": ["config", "precedence"],
+      "input": {
+        "parent": { "@particle-academy/output": "builtin", "host_kind": "host" },
+        "injected": null,
+        "graphConfig": { "host_kind": "child-specific" }
+      },
+      "expected": { "@particle-academy/output": "builtin", "host_kind": "child-specific" }
+    },
+    {
+      "id": "0006-an-injected-registry-replaces-the-inherited-one",
+      "title": "A registry handed to the executor at construction is used INSTEAD of the parent's.",
+      "since": "0.8.0",
+      "tags": ["injection", "precedence"],
+      "input": {
+        "parent": { "@particle-academy/output": "builtin", "host_kind": "host" },
+        "injected": { "@particle-academy/output": "builtin", "sandboxed_kind": "host" },
+        "graphConfig": null
+      },
+      "expected": { "@particle-academy/output": "builtin", "sandboxed_kind": "host" }
+    },
+    {
+      "id": "0007-graph-config-still-layers-over-an-injected-registry",
+      "title": "Injection replaces the inherited registry; the graph's own still layers on top of that.",
+      "since": "0.8.0",
+      "tags": ["injection", "config", "precedence"],
+      "input": {
+        "parent": { "host_kind": "host" },
+        "injected": { "@particle-academy/output": "builtin" },
+        "graphConfig": { "child_only": "child-specific" }
+      },
+      "expected": { "@particle-academy/output": "builtin", "child_only": "child-specific" }
+    },
+    {
+      "id": "0008-an-empty-graph-config-changes-nothing",
+      "title": "An empty config registry is not the same as an absent one, and neither empties the child.",
+      "since": "0.8.0",
+      "tags": ["config", "edge"],
+      "input": {
+        "parent": { "@particle-academy/output": "builtin", "host_kind": "host" },
+        "injected": null,
+        "graphConfig": {}
+      },
+      "expected": { "@particle-academy/output": "builtin", "host_kind": "host" }
+    },
+    {
+      "id": "0009-an-empty-parent-yields-an-empty-child",
+      "title": "Inheritance copies what is there; it never invents builtins the parent did not have.",
+      "since": "0.8.0",
+      "tags": ["edge"],
+      "input": { "parent": {}, "injected": null, "graphConfig": null },
+      "expected": {}
+    }
+  ]
+}

package/suites/flow/subflow-registry/manifest.json

@@ -0,0 +1,28 @@
+{
+  "$schema": "../../../schema/suite-manifest.schema.json",
+  "suite": "flow/subflow-registry",
+  "title": "What executors a subflow's child runs against",
+  "since": "0.8.0",
+  "caseFormat": "table",
+  "cases": "cases.json",
+  "contract": {
+    "function": "childRegistry(parent: object, injected: object|null, graphConfig: object|null) -> object",
+    "summary": "Compose the executor registry a `subflow` node runs its CHILD graph against. `parent` is the registry the parent run is executing with, keyed by kind id, whose values stand in for executors. `injected` is a registry handed to the subflow executor at construction, or null. `graphConfig` is the registry the graph itself declares on the subflow node's `config.executors`, or null. The result is the registry the child must resolve against, keyed by kind id.",
+    "reference": "node",
+    "referenceNote": "Designed here rather than lifted from an implementation, because at the time this suite was written all four runtimes were WRONG in the same way and none could serve as the reference. `node` is recorded as the reference because @particle-academy/fancy-flow's run-flow.ts is the behavioural source of truth for the engine generally.",
+    "implementations": [
+      { "language": "node", "package": "@particle-academy/fancy-flow", "symbol": "subflowExecutor" },
+      { "language": "php", "package": "particle-academy/fancy-flow-php", "symbol": "FancyFlow\\Nodes\\Structural\\SubflowExecutor" },
+      { "language": "python", "package": "fancy-flow", "symbol": "fancy_flow.nodes.structural.SubflowExecutor" },
+      { "language": "rust", "package": "fancy-flow", "symbol": "fancy_flow::nodes::structural::SubflowExecutor" }
+    ]
+  },
+  "notes": [
+    "This table exists because ALL FOUR runtimes had the same defect at once, and each of the three that were fixed carried its own hand-written test for it. Three copies of an assertion agree right up until someone changes one of them, and nothing anywhere reports the divergence — which is the same shape as the bug the table describes.",
+    "Reported against the PHP twin as fancy-flow-php#7 by a consumer running a real graph: a child containing a host-registered kind failed with `No executor registered for kind=<host-kind>`, while the identical graph run at top level succeeded. Same workflow, two behaviours depending on nesting depth.",
+    "The TypeScript engine had it with a WORSE default. Its child ran against `config.executors ?? {}` — an EMPTY registry unless the graph happened to carry one — so nesting a graph did not merely lose the host's kinds, it lost everything. PHP and Python fell back to the bare builtins. None of the three warned, because an unregistered kind fails closed with no outputs, which is the correct default and exactly what makes this silent.",
+    "The costly case is not a MISSING kind but a REPLACED one. A host that overrides `llm_call` to add tenancy, budgeting or token accounting gets its own version in the parent and the package's version in the child: the same graph billing two different ways depending on where it was invoked from, with nothing failing. Case 0003 is that case.",
+    "The contract is deliberately a function over REGISTRIES rather than a graph run. `flow/graph-runs` cannot ship a discrimination probe because the implementation under test there is an entire workflow engine, which this package does not carry and should not grow. Registry composition is a pure function over maps, so this suite CAN be probed — and is, in tests/discrimination.test.ts, by the exact mutant that shipped in production: a child that sees only the builtins.",
+    "Values in these maps are marker strings, not executors. `builtin`, `host` and `child-specific` stand for whichever callable a runtime would bind; what the table pins is WHICH one survives into the child, not what it does."
+  ]
+}

package/suites/flow/workflow-props/cases.json

@@ -0,0 +1,226 @@
+{
+  "$schema": "../../../schema/cases.schema.json",
+  "suite": "flow/workflow-props",
+  "cases": [
+    {
+      "id": "0001-supplied-value-passes-through",
+      "title": "A declared, supplied value arrives in the resolved map unchanged.",
+      "since": "0.9.0",
+      "tags": ["happy-path"],
+      "input": {
+        "declared": [{ "name": "topic", "type": "string" }],
+        "passed": { "topic": "otters" }
+      },
+      "expected": { "ok": true, "props": { "topic": "otters" } }
+    },
+    {
+      "id": "0002-default-fills-an-omitted-value",
+      "title": "An omitted input with a default resolves to the default.",
+      "since": "0.9.0",
+      "tags": ["defaults"],
+      "input": {
+        "declared": [{ "name": "limit", "type": "number", "default": 10 }],
+        "passed": {}
+      },
+      "expected": { "ok": true, "props": { "limit": 10 } }
+    },
+    {
+      "id": "0003-supplied-beats-default",
+      "title": "An explicitly supplied value wins over the declared default.",
+      "since": "0.9.0",
+      "tags": ["defaults"],
+      "input": {
+        "declared": [{ "name": "limit", "type": "number", "default": 10 }],
+        "passed": { "limit": 25 }
+      },
+      "expected": { "ok": true, "props": { "limit": 25 } }
+    },
+    {
+      "id": "0004-explicit-zero-is-not-replaced-by-a-default",
+      "title": "A supplied 0 survives a non-zero default.",
+      "since": "0.9.0",
+      "tags": ["defaults", "falsy", "trap"],
+      "input": {
+        "declared": [{ "name": "limit", "type": "number", "default": 10 }],
+        "passed": { "limit": 0 }
+      },
+      "expected": { "ok": true, "props": { "limit": 0 } }
+    },
+    {
+      "id": "0005-explicit-false-is-not-replaced-by-a-default",
+      "title": "A supplied false survives a true default.",
+      "since": "0.9.0",
+      "tags": ["defaults", "falsy", "trap"],
+      "input": {
+        "declared": [{ "name": "dryRun", "type": "boolean", "default": true }],
+        "passed": { "dryRun": false }
+      },
+      "expected": { "ok": true, "props": { "dryRun": false } }
+    },
+    {
+      "id": "0006-explicit-empty-string-is-not-replaced-by-a-default",
+      "title": "A supplied empty string survives a non-empty default.",
+      "since": "0.9.0",
+      "tags": ["defaults", "falsy", "trap"],
+      "input": {
+        "declared": [{ "name": "note", "type": "string", "default": "unset" }],
+        "passed": { "note": "" }
+      },
+      "expected": { "ok": true, "props": { "note": "" } }
+    },
+    {
+      "id": "0007-absent-optional-is-absent-not-null",
+      "title": "An optional input with no default and no value is missing from the map entirely.",
+      "since": "0.9.0",
+      "tags": ["absence"],
+      "input": {
+        "declared": [
+          { "name": "topic", "type": "string" },
+          { "name": "note", "type": "string" }
+        ],
+        "passed": { "topic": "otters" }
+      },
+      "expected": { "ok": true, "props": { "topic": "otters" } }
+    },
+    {
+      "id": "0008-untyped-declaration-accepts-anything",
+      "title": "An input declaring no type accepts a nested object.",
+      "since": "0.9.0",
+      "tags": ["types"],
+      "input": {
+        "declared": [{ "name": "payload" }],
+        "passed": { "payload": { "nested": [1, 2] } }
+      },
+      "expected": { "ok": true, "props": { "payload": { "nested": [1, 2] } } }
+    },
+    {
+      "id": "0009-required-is-satisfied-by-its-default",
+      "title": "A required input carrying a default does not need the caller to supply it.",
+      "since": "0.9.0",
+      "tags": ["required", "defaults"],
+      "input": {
+        "declared": [{ "name": "limit", "type": "number", "required": true, "default": 5 }],
+        "passed": {}
+      },
+      "expected": { "ok": true, "props": { "limit": 5 } }
+    },
+    {
+      "id": "0010-array-satisfies-array-not-object",
+      "title": "A list satisfies a declared array.",
+      "since": "0.9.0",
+      "tags": ["types", "trap"],
+      "input": {
+        "declared": [{ "name": "tags", "type": "array" }],
+        "passed": { "tags": ["a", "b"] }
+      },
+      "expected": { "ok": true, "props": { "tags": ["a", "b"] } }
+    },
+    {
+      "id": "0011-object-satisfies-object",
+      "title": "A map satisfies a declared object.",
+      "since": "0.9.0",
+      "tags": ["types"],
+      "input": {
+        "declared": [{ "name": "meta", "type": "object" }],
+        "passed": { "meta": { "k": "v" } }
+      },
+      "expected": { "ok": true, "props": { "meta": { "k": "v" } } }
+    },
+    {
+      "id": "0012-no-declaration-and-no-props",
+      "title": "A workflow that declares nothing, called with nothing, resolves to an empty map.",
+      "since": "0.9.0",
+      "tags": ["happy-path", "absence"],
+      "input": { "declared": null, "passed": null },
+      "expected": { "ok": true, "props": {} }
+    },
+    {
+      "id": "0101-an-unknown-key-fails",
+      "title": "A misspelled input name FAILS rather than sitting unread.",
+      "since": "0.9.0",
+      "tags": ["validation", "regression", "silent-failure"],
+      "input": {
+        "declared": [{ "name": "topic", "type": "string" }],
+        "passed": { "topik": "otters" }
+      },
+      "expected": { "ok": false, "code": "unknown_input" }
+    },
+    {
+      "id": "0102-props-passed-to-a-workflow-declaring-none-fails",
+      "title": "Passing anything to a workflow that declares no inputs FAILS.",
+      "since": "0.9.0",
+      "tags": ["validation", "silent-failure"],
+      "input": {
+        "declared": null,
+        "passed": { "topic": "otters" }
+      },
+      "expected": { "ok": false, "code": "unknown_input" }
+    },
+    {
+      "id": "0103-unknown-is-reported-before-missing-required",
+      "title": "A caller who misspells a required input is told about the word they typed.",
+      "since": "0.9.0",
+      "tags": ["validation", "ordering"],
+      "input": {
+        "declared": [{ "name": "topic", "type": "string", "required": true }],
+        "passed": { "topik": "otters" }
+      },
+      "expected": { "ok": false, "code": "unknown_input" }
+    },
+    {
+      "id": "0104-missing-required-fails",
+      "title": "A required input with no default and no value FAILS.",
+      "since": "0.9.0",
+      "tags": ["validation", "required"],
+      "input": {
+        "declared": [{ "name": "topic", "type": "string", "required": true }],
+        "passed": {}
+      },
+      "expected": { "ok": false, "code": "missing_required" }
+    },
+    {
+      "id": "0105-wrong-type-fails",
+      "title": "A string supplied where a number is declared FAILS.",
+      "since": "0.9.0",
+      "tags": ["validation", "types"],
+      "input": {
+        "declared": [{ "name": "limit", "type": "number" }],
+        "passed": { "limit": "ten" }
+      },
+      "expected": { "ok": false, "code": "type_mismatch" }
+    },
+    {
+      "id": "0106-object-does-not-satisfy-array",
+      "title": "A map supplied where an array is declared FAILS.",
+      "since": "0.9.0",
+      "tags": ["validation", "types", "trap"],
+      "input": {
+        "declared": [{ "name": "tags", "type": "array" }],
+        "passed": { "tags": { "0": "a" } }
+      },
+      "expected": { "ok": false, "code": "type_mismatch" }
+    },
+    {
+      "id": "0107-array-does-not-satisfy-object",
+      "title": "A list supplied where an object is declared FAILS.",
+      "since": "0.9.0",
+      "tags": ["validation", "types", "trap"],
+      "input": {
+        "declared": [{ "name": "meta", "type": "object" }],
+        "passed": { "meta": ["a"] }
+      },
+      "expected": { "ok": false, "code": "type_mismatch" }
+    },
+    {
+      "id": "0108-null-does-not-satisfy-a-declared-type",
+      "title": "An explicit null supplied where a string is declared FAILS.",
+      "since": "0.9.0",
+      "tags": ["validation", "types"],
+      "input": {
+        "declared": [{ "name": "topic", "type": "string" }],
+        "passed": { "topic": null }
+      },
+      "expected": { "ok": false, "code": "type_mismatch" }
+    }
+  ]
+}

package/suites/flow/workflow-props/manifest.json

@@ -0,0 +1,28 @@
+{
+  "$schema": "../../../schema/suite-manifest.schema.json",
+  "suite": "flow/workflow-props",
+  "title": "Resolving a caller's props against what a workflow declares",
+  "since": "0.9.0",
+  "caseFormat": "table",
+  "cases": "cases.json",
+  "contract": {
+    "function": "resolveWorkflowProps(declared: array|null, passed: object|null) -> {ok: true, props: object} | {ok: false, code: string}",
+    "summary": "Check the flat, by-name object a caller passed against the `inputs` a workflow declares, and fill in declared defaults. `declared` is the list of input declarations ({name, type?, required?, default?}) or null when the workflow declares none. `passed` is what the caller supplied, or null. On success the result carries the RESOLVED map -- supplied values plus defaults for anything omitted, and nothing else. On failure it carries a stable `code`; the human-readable message is deliberately NOT part of the contract.",
+    "reference": "node",
+    "referenceNote": "@particle-academy/fancy-flow's src/runtime/workflow-props.ts is the behavioural source of truth, and the rules were designed there with this table written alongside rather than after.",
+    "implementations": [
+      { "language": "node", "package": "@particle-academy/fancy-flow", "symbol": "resolveWorkflowProps" },
+      { "language": "php", "package": "particle-academy/fancy-flow-php", "symbol": "FancyFlow\\Runtime\\WorkflowProps::resolve" },
+      { "language": "python", "package": "fancy-flow", "symbol": "fancy_flow.runtime.workflow_props.resolve_workflow_props" }
+    ]
+  },
+  "notes": [
+    "The CODE is asserted, never the message. Each runtime words its errors idiomatically and an English sentence is not a behaviour -- pinning the prose would hold three implementations to a translation, and would go red on a wording improvement that changed nothing.",
+    "The reason this feature exists is case 0101: passing a MISSPELLED key. Before workflow props, `initialInputs` was keyed by node id and an unrecognised key was not an error -- the value simply sat unread, the node saw nothing, and the run reported success with output that was quietly wrong. Every implementation must fail that case, because the silent version is what shipped.",
+    "Cases 0004-0006 are the falsy-value trap and they are the ones a re-implementation gets wrong. `0`, `false` and `\"\"` are values a caller MEANT to pass. A default applied with `||`, or with `??` on the wrong side, silently replaces them -- and a declared limit of 0 quietly becoming 10 is not an error anybody observes.",
+    "Absent is ABSENT. An input that is not supplied, has no default and is not required must not appear in the resolved map at all -- not as null, not as an empty string. PHP has one absent value and JS has two; letting that difference leak would make `{{ $props.x }}` disagree across runtimes for no useful reason.",
+    "`type` is optional and an omitted type accepts ANYTHING (case 0008). 'I am not asserting a shape' must not degrade into 'nothing is allowed', which is the failure mode of a validator written defensively.",
+    "Arrays are checked before objects (case 0010). `typeof []` is `\"object\"` in JavaScript, and a declaration saying `array` that accepts `{}` is a check that passes while meaning nothing.",
+    "Unknown keys are checked BEFORE required ones (case 0103). A caller who typed `topik` instead of `topic` should be told about `topik` -- the word they wrote -- rather than that `topic` is missing, which describes a key they believe they supplied."
+  ]
+}