@particle-academy/fancy-conformance 0.11.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.
|
|
1
|
+
0.12.0
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@particle-academy/fancy-conformance",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.12.0",
|
|
4
4
|
"description": "Shared cross-language conformance fixtures for the Fancy suite. One contract, N implementations, and a single table that every implementation asserts in its own CI \u2014 so 'parity' is a test result rather than a claim. Ships the fixture data itself, so a Rust, Go or Python runner can consume it without a JavaScript toolchain.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -0,0 +1,231 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "../../../schema/case-table.schema.json",
|
|
3
|
+
"suite": "flow/executor-resolution",
|
|
4
|
+
"cases": [
|
|
5
|
+
{
|
|
6
|
+
"id": "0101-node-id-wins-over-kind",
|
|
7
|
+
"title": "A binding on the node's own id beats one on its kind.",
|
|
8
|
+
"since": "0.12.0",
|
|
9
|
+
"tags": ["order"],
|
|
10
|
+
"input": {
|
|
11
|
+
"kinds": [{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] }],
|
|
12
|
+
"bindings": [
|
|
13
|
+
{ "key": "llm_call", "executor": "by-kind" },
|
|
14
|
+
{ "key": "n1", "executor": "by-id" }
|
|
15
|
+
],
|
|
16
|
+
"node": { "id": "n1", "type": "llm_call" }
|
|
17
|
+
},
|
|
18
|
+
"expected": "by-id",
|
|
19
|
+
"notes": "The per-node override is the whole reason id is consulted first — a graph pins ONE node to a stub without unbinding the kind for every other node using it."
|
|
20
|
+
},
|
|
21
|
+
{
|
|
22
|
+
"id": "0102-kind-wins-over-fallback",
|
|
23
|
+
"title": "A binding on the kind beats the `*` fallback.",
|
|
24
|
+
"since": "0.12.0",
|
|
25
|
+
"tags": ["order"],
|
|
26
|
+
"input": {
|
|
27
|
+
"kinds": [{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] }],
|
|
28
|
+
"bindings": [
|
|
29
|
+
{ "key": "*", "executor": "catch-all" },
|
|
30
|
+
{ "key": "llm_call", "executor": "by-kind" }
|
|
31
|
+
],
|
|
32
|
+
"node": { "id": "n1", "type": "llm_call" }
|
|
33
|
+
},
|
|
34
|
+
"expected": "by-kind"
|
|
35
|
+
},
|
|
36
|
+
{
|
|
37
|
+
"id": "0103-bare-binding-namespaced-node",
|
|
38
|
+
"title": "A node naming the canonical id resolves a binding made under the bare alias.",
|
|
39
|
+
"since": "0.12.0",
|
|
40
|
+
"tags": ["alias"],
|
|
41
|
+
"input": {
|
|
42
|
+
"kinds": [{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] }],
|
|
43
|
+
"bindings": [{ "key": "llm_call", "executor": "bare" }],
|
|
44
|
+
"node": { "id": "n1", "type": "@particle-academy/llm_call" }
|
|
45
|
+
},
|
|
46
|
+
"expected": "bare",
|
|
47
|
+
"notes": "This is the direction a consumer hit in the wild: resolveKindId() hands back the NAMESPACED id, they keyed their registry by it, and the bare binding they already had stopped matching. A rename must not become a breaking change wearing a rename's costume."
|
|
48
|
+
},
|
|
49
|
+
{
|
|
50
|
+
"id": "0104-namespaced-binding-bare-node",
|
|
51
|
+
"title": "A node naming the bare alias resolves a binding made under the canonical id.",
|
|
52
|
+
"since": "0.12.0",
|
|
53
|
+
"tags": ["alias"],
|
|
54
|
+
"input": {
|
|
55
|
+
"kinds": [{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] }],
|
|
56
|
+
"bindings": [{ "key": "@particle-academy/llm_call", "executor": "namespaced" }],
|
|
57
|
+
"node": { "id": "n1", "type": "llm_call" }
|
|
58
|
+
},
|
|
59
|
+
"expected": "namespaced",
|
|
60
|
+
"notes": "The mirror of 0103. Asserting only one direction would pass against an implementation that resolves aliases one way, which is half a feature and the harder half to notice is missing."
|
|
61
|
+
},
|
|
62
|
+
{
|
|
63
|
+
"id": "0105-unknown-kind-resolves-nothing",
|
|
64
|
+
"title": "A node whose kind nothing is bound to, with no fallback, resolves null.",
|
|
65
|
+
"since": "0.12.0",
|
|
66
|
+
"tags": ["closed"],
|
|
67
|
+
"input": {
|
|
68
|
+
"kinds": [{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] }],
|
|
69
|
+
"bindings": [{ "key": "llm_call", "executor": "by-kind" }],
|
|
70
|
+
"node": { "id": "n1", "type": "http_request" }
|
|
71
|
+
},
|
|
72
|
+
"expected": null,
|
|
73
|
+
"notes": "Failing CLOSED is the correct default and is also what makes every miss in this suite silent — an unresolved node simply produces no outputs. Pinning it stops a well-meaning fall-through to an arbitrary binding from being added later."
|
|
74
|
+
},
|
|
75
|
+
{
|
|
76
|
+
"id": "0106-fallback-is-last",
|
|
77
|
+
"title": "The `*` fallback runs only after every id and kind candidate has missed.",
|
|
78
|
+
"since": "0.12.0",
|
|
79
|
+
"tags": ["closed", "order"],
|
|
80
|
+
"input": {
|
|
81
|
+
"kinds": [{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] }],
|
|
82
|
+
"bindings": [{ "key": "*", "executor": "catch-all" }],
|
|
83
|
+
"node": { "id": "n1", "type": "http_request" }
|
|
84
|
+
},
|
|
85
|
+
"expected": "catch-all"
|
|
86
|
+
},
|
|
87
|
+
{
|
|
88
|
+
"id": "0107-fallback-has-no-aliases",
|
|
89
|
+
"title": "The fallback is a sentinel: a kind literally named `*` is not expanded through the alias machinery.",
|
|
90
|
+
"since": "0.12.0",
|
|
91
|
+
"tags": ["closed"],
|
|
92
|
+
"input": {
|
|
93
|
+
"kinds": [{ "name": "*", "aliases": ["everything"] }],
|
|
94
|
+
"bindings": [{ "key": "everything", "executor": "aliased-star" }],
|
|
95
|
+
"node": { "id": "n1", "type": "http_request" }
|
|
96
|
+
},
|
|
97
|
+
"expected": null,
|
|
98
|
+
"notes": "Pathological on purpose. A runtime that ran the `*` sentinel through alias expansion would bind every unmatched node to whatever `everything` points at, and the graph would still complete — the failure mode this whole suite is about."
|
|
99
|
+
},
|
|
100
|
+
{
|
|
101
|
+
"id": "0108-unregistered-kind-binds-literally",
|
|
102
|
+
"title": "A kind the registry has never heard of still resolves its own literal binding.",
|
|
103
|
+
"since": "0.12.0",
|
|
104
|
+
"tags": ["alias"],
|
|
105
|
+
"input": {
|
|
106
|
+
"kinds": [],
|
|
107
|
+
"bindings": [{ "key": "my_custom_node", "executor": "custom" }],
|
|
108
|
+
"node": { "id": "n1", "type": "my_custom_node" }
|
|
109
|
+
},
|
|
110
|
+
"expected": "custom",
|
|
111
|
+
"notes": "Alias awareness must not become a REQUIREMENT to be registered. A host binding an ad-hoc kind it invented gets exactly that kind and no expansion, because there is no alias list to expand from and inventing one would claim knowledge nothing has."
|
|
112
|
+
},
|
|
113
|
+
{
|
|
114
|
+
"id": "0201-data-kind-used-when-type-is-absent",
|
|
115
|
+
"title": "With no type, the kind carried in data.kind resolves the executor.",
|
|
116
|
+
"since": "0.12.0",
|
|
117
|
+
"tags": ["data-kind"],
|
|
118
|
+
"input": {
|
|
119
|
+
"kinds": [{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] }],
|
|
120
|
+
"bindings": [{ "key": "llm_call", "executor": "by-kind" }],
|
|
121
|
+
"node": { "id": "n1", "type": null, "dataKind": "llm_call" }
|
|
122
|
+
},
|
|
123
|
+
"expected": "by-kind",
|
|
124
|
+
"skip": {
|
|
125
|
+
"php": "FlowNode is flattened — type IS the kind and there is no data slot, so a node without a type carries no kind at all.",
|
|
126
|
+
"python": "FlowNode is flattened — type IS the kind and there is no data slot, so a node without a type carries no kind at all."
|
|
127
|
+
},
|
|
128
|
+
"notes": "The legitimate use of data.kind, and the reason it is consulted at all."
|
|
129
|
+
},
|
|
130
|
+
{
|
|
131
|
+
"id": "0202-real-type-beats-unrelated-data-kind",
|
|
132
|
+
"title": "A type naming a registered kind wins over a data.kind naming a DIFFERENT registered kind.",
|
|
133
|
+
"since": "0.12.0",
|
|
134
|
+
"tags": ["data-kind", "regression"],
|
|
135
|
+
"input": {
|
|
136
|
+
"kinds": [
|
|
137
|
+
{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] },
|
|
138
|
+
{ "name": "@particle-academy/output", "aliases": ["output"] }
|
|
139
|
+
],
|
|
140
|
+
"bindings": [{ "key": "@particle-academy/output", "executor": "output-exec" }],
|
|
141
|
+
"node": { "id": "n1", "type": "llm_call", "dataKind": "output" }
|
|
142
|
+
},
|
|
143
|
+
"expected": null,
|
|
144
|
+
"skip": {
|
|
145
|
+
"php": "FlowNode is flattened — there is no data.kind for a second opinion to live in, so this precedence question cannot arise.",
|
|
146
|
+
"python": "FlowNode is flattened — there is no data.kind for a second opinion to live in, so this precedence question cannot arise."
|
|
147
|
+
},
|
|
148
|
+
"notes": "fancy-flow 0.51.1 and earlier answered output-exec here: the alias step tried data.kind's ids before node.type's, so a node declaring itself an llm_call ran the OUTPUT executor. Null is correct — the node IS an llm_call and nothing is bound to that kind, so it must fail closed rather than run something else's code."
|
|
149
|
+
},
|
|
150
|
+
{
|
|
151
|
+
"id": "0203-real-type-beats-data-kind-even-when-both-bound",
|
|
152
|
+
"title": "The same precedence holds when BOTH kinds have an executor bound.",
|
|
153
|
+
"since": "0.12.0",
|
|
154
|
+
"tags": ["data-kind", "regression"],
|
|
155
|
+
"input": {
|
|
156
|
+
"kinds": [
|
|
157
|
+
{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] },
|
|
158
|
+
{ "name": "@particle-academy/output", "aliases": ["output"] }
|
|
159
|
+
],
|
|
160
|
+
"bindings": [
|
|
161
|
+
{ "key": "@particle-academy/llm_call", "executor": "llm-exec" },
|
|
162
|
+
{ "key": "@particle-academy/output", "executor": "output-exec" }
|
|
163
|
+
],
|
|
164
|
+
"node": { "id": "n1", "type": "llm_call", "dataKind": "output" }
|
|
165
|
+
},
|
|
166
|
+
"expected": "llm-exec",
|
|
167
|
+
"skip": {
|
|
168
|
+
"php": "FlowNode is flattened — no data.kind exists to take precedence over type.",
|
|
169
|
+
"python": "FlowNode is flattened — no data.kind exists to take precedence over type."
|
|
170
|
+
},
|
|
171
|
+
"notes": "The bug at its worst and the row worth reading twice. The CORRECT executor was registered, under the node's own declared kind, and 0.51.1 ran the other one anyway. 0202 could be argued away as an obscure unbound-kind edge; this cannot."
|
|
172
|
+
},
|
|
173
|
+
{
|
|
174
|
+
"id": "0204-category-label-in-data-kind-is-ignored",
|
|
175
|
+
"title": "A data.kind holding a category label rather than a kind id does not disable type.",
|
|
176
|
+
"since": "0.12.0",
|
|
177
|
+
"tags": ["data-kind", "regression"],
|
|
178
|
+
"input": {
|
|
179
|
+
"kinds": [{ "name": "@particle-academy/manual_trigger", "aliases": ["manual_trigger"] }],
|
|
180
|
+
"bindings": [{ "key": "@particle-academy/manual_trigger", "executor": "trigger-exec" }],
|
|
181
|
+
"node": { "id": "n1", "type": "manual_trigger", "dataKind": "trigger" }
|
|
182
|
+
},
|
|
183
|
+
"expected": "trigger-exec",
|
|
184
|
+
"skip": {
|
|
185
|
+
"php": "FlowNode is flattened — no data.kind.",
|
|
186
|
+
"python": "FlowNode is flattened — no data.kind."
|
|
187
|
+
},
|
|
188
|
+
"notes": "A category label is not a kind id — easy to write, says nothing false, and names nothing in the registry. This was the shape a consumer actually reported: the namespaced id (the one resolveKindId() hands you) was the spelling that failed, while the bare name worked. Fixed in 0.51.1; kept here so it stays fixed."
|
|
189
|
+
},
|
|
190
|
+
{
|
|
191
|
+
"id": "0205-renderer-type-defers-to-data-kind",
|
|
192
|
+
"title": "A type that names NO registered kind is a renderer type, and data.kind decides.",
|
|
193
|
+
"since": "0.12.0",
|
|
194
|
+
"tags": ["data-kind"],
|
|
195
|
+
"input": {
|
|
196
|
+
"kinds": [{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] }],
|
|
197
|
+
"bindings": [{ "key": "llm_call", "executor": "by-kind" }],
|
|
198
|
+
"node": { "id": "n1", "type": "fancyNode", "dataKind": "llm_call" }
|
|
199
|
+
},
|
|
200
|
+
"expected": "by-kind",
|
|
201
|
+
"skip": {
|
|
202
|
+
"php": "FlowNode is flattened — no data.kind.",
|
|
203
|
+
"python": "FlowNode is flattened — no data.kind."
|
|
204
|
+
},
|
|
205
|
+
"notes": "The row that stops the 0202-0204 fix from over-reaching. Registering ONE custom xyflow renderer and carrying the kind in data is ordinary xyflow practice; type here is a React component name, not a claim about behaviour. This is why the rule is that a type NAMING A REGISTERED KIND is authoritative, rather than the simpler and wrong rule that type always wins."
|
|
206
|
+
},
|
|
207
|
+
{
|
|
208
|
+
"id": "0206-node-id-still-wins-over-both",
|
|
209
|
+
"title": "A per-node binding still beats type and data.kind alike.",
|
|
210
|
+
"since": "0.12.0",
|
|
211
|
+
"tags": ["data-kind", "order"],
|
|
212
|
+
"input": {
|
|
213
|
+
"kinds": [
|
|
214
|
+
{ "name": "@particle-academy/llm_call", "aliases": ["llm_call"] },
|
|
215
|
+
{ "name": "@particle-academy/output", "aliases": ["output"] }
|
|
216
|
+
],
|
|
217
|
+
"bindings": [
|
|
218
|
+
{ "key": "@particle-academy/llm_call", "executor": "llm-exec" },
|
|
219
|
+
{ "key": "n1", "executor": "pinned" }
|
|
220
|
+
],
|
|
221
|
+
"node": { "id": "n1", "type": "llm_call", "dataKind": "output" }
|
|
222
|
+
},
|
|
223
|
+
"expected": "pinned",
|
|
224
|
+
"skip": {
|
|
225
|
+
"php": "FlowNode is flattened — no data.kind.",
|
|
226
|
+
"python": "FlowNode is flattened — no data.kind."
|
|
227
|
+
},
|
|
228
|
+
"notes": "Guards the fix's blast radius: reordering the kind step must not disturb the step above it."
|
|
229
|
+
}
|
|
230
|
+
]
|
|
231
|
+
}
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "../../../schema/suite-manifest.schema.json",
|
|
3
|
+
"suite": "flow/executor-resolution",
|
|
4
|
+
"title": "Which executor a node runs — id, kind, alias, fallback",
|
|
5
|
+
"since": "0.12.0",
|
|
6
|
+
"caseFormat": "table",
|
|
7
|
+
"cases": "cases.json",
|
|
8
|
+
"contract": {
|
|
9
|
+
"function": "resolveExecutor(kinds, bindings, node) -> string | null",
|
|
10
|
+
"summary": "Given a kind registry (each kind with its aliases), a set of executor bindings (each an opaque LABEL bound under some key), and a node, return the label of the executor that runs — or null when nothing resolves. The answer is a label rather than a key so the table is neutral about WHEN a runtime expands aliases: PHP and Python expand at bind time and would report the key they stored under, TypeScript expands at lookup time and would report the key it matched. Both run the same executor, which is the thing a consumer can actually observe.",
|
|
11
|
+
"reference": "node",
|
|
12
|
+
"referenceNote": "The order — node id, then kind, then `*` — is fancy-flow's, and the alias step exists because kinds are namespaced (`@particle-academy/llm_call`) while hosts routinely bind the bare name. Rows 0201+ pin a TYPESCRIPT-ONLY question and are skipped elsewhere; see the notes.",
|
|
13
|
+
"implementations": [
|
|
14
|
+
{ "language": "node", "package": "@particle-academy/fancy-flow", "symbol": "pickExecutor" },
|
|
15
|
+
{ "language": "php", "package": "particle-academy/fancy-flow-php", "symbol": "FancyFlow\\ExecutorRegistry::resolveFor" },
|
|
16
|
+
{ "language": "python", "package": "fancy-flow", "symbol": "fancy_flow.executors.ExecutorRegistry.resolve_for" }
|
|
17
|
+
]
|
|
18
|
+
},
|
|
19
|
+
"notes": [
|
|
20
|
+
"THE 0200 ROWS ARE SKIPPED FOR PHP AND PYTHON ON A STRUCTURAL GROUND, NOT A CONVENIENCE ONE. TypeScript's `FlowNode` is an xyflow node — `{ id, type, data }` — so a graph can carry its kind in `data.kind` while `type` holds the RENDERER type, which is ordinary xyflow practice. PHP's and Python's `FlowNode` are FLATTENED: `type` IS the kind and there is no `data` slot for a second opinion to live in. The precedence question therefore cannot arise there, and inventing a `data.kind` field in two runtimes so they could answer rows about it would be writing code to satisfy a table — the exact inversion this package exists to prevent.",
|
|
21
|
+
"That asymmetry is worth stating rather than hiding, because it is WHY only one runtime had the bug these rows were written for. In fancy-flow <= 0.51.1 the alias step tried `data.kind`'s aliases BEFORE `node.type`'s, so a node with `type: \"llm_call\"` and `data.kind: \"output\"` ran the OUTPUT executor — even when an `llm_call` executor was registered (row 0203). Nothing reported it: running the wrong executor and running the right one look identical from the outside, and the graph still completes.",
|
|
22
|
+
"The rule the 0200 rows pin: WHEN `node.type` NAMES A REGISTERED KIND IT IS AUTHORITATIVE, and `data.kind` does not contribute at all. Otherwise `data.kind` is consulted. That keeps the xyflow pattern working (row 0205: `type: \"fancyNode\"` is a renderer, not a kind, so `data.kind` decides) while making a real kind in `type` mean what it says.",
|
|
23
|
+
"The 0100 rows are NOT skipped anywhere and are the substance of the cross-runtime claim: id beats kind, kind beats `*`, an alias resolves in both directions (bare binding + namespaced node, and the reverse), and an unresolvable node returns null rather than falling through to an arbitrary binding.",
|
|
24
|
+
"Bindings are a LIST, not a map, because insertion order is observable when two keys could both match and a JSON object's key order is not something every language's decoder preserves.",
|
|
25
|
+
"`*` is a sentinel, not a kind: it has no aliases and is only consulted after every id-and-kind candidate has missed (row 0106). A runtime that expanded `*` through the alias machinery would bind every kind to the fallback."
|
|
26
|
+
]
|
|
27
|
+
}
|