@jarenjs/linq 0.49.2 → 0.66.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (79) hide show
  1. package/ARCHITECTURE.md +227 -0
  2. package/README.md +650 -17
  3. package/docs/APP-PEN.md +1143 -0
  4. package/docs/CONTRACT-PEN.md +1221 -0
  5. package/docs/DB-CLIENT.md +882 -0
  6. package/docs/FLOW-PEN.md +1033 -0
  7. package/docs/FORMS-PEN.md +940 -0
  8. package/docs/JSLT-PEN.md +955 -0
  9. package/docs/LINQ-FORMAT.md +778 -383
  10. package/docs/MIGRATION-PEN.md +781 -0
  11. package/docs/MODEL-PEN.md +1092 -0
  12. package/docs/QUERY-PEN.md +1724 -0
  13. package/docs/SCHEMA-PEN.md +1218 -0
  14. package/package.json +57 -4
  15. package/src/app/action.js +251 -0
  16. package/src/app/capture.js +63 -0
  17. package/src/app/define.js +255 -0
  18. package/src/app/index.js +20 -0
  19. package/src/app/patch.js +277 -0
  20. package/src/app/sub.js +106 -0
  21. package/src/async.js +377 -75
  22. package/src/capture-root.js +82 -0
  23. package/src/concurrency.js +48 -11
  24. package/src/contract/define.js +282 -0
  25. package/src/contract/http.js +247 -0
  26. package/src/contract/index.js +23 -0
  27. package/src/contract/operation.js +338 -0
  28. package/src/db/handle.js +89 -0
  29. package/src/db/include.js +351 -0
  30. package/src/db/index.js +24 -0
  31. package/src/db/ledger.js +195 -0
  32. package/src/db/live.js +43 -0
  33. package/src/db/membership.js +37 -0
  34. package/src/db/open.js +130 -0
  35. package/src/document.js +143 -13
  36. package/src/effect.js +65 -0
  37. package/src/errors.js +78 -6
  38. package/src/expression.js +463 -36
  39. package/src/federate.js +531 -0
  40. package/src/flow/capture.js +33 -0
  41. package/src/flow/dag.js +316 -0
  42. package/src/flow/fsm.js +323 -0
  43. package/src/flow/index.js +22 -0
  44. package/src/forms/index.js +43 -0
  45. package/src/forms/rules.js +170 -0
  46. package/src/forms/submit.js +177 -0
  47. package/src/index.js +5 -2
  48. package/src/jslt/body.js +226 -0
  49. package/src/jslt/index.js +18 -0
  50. package/src/jslt/rules.js +202 -0
  51. package/src/json-boundary.js +90 -0
  52. package/src/migration/define.js +318 -0
  53. package/src/migration/index.js +15 -0
  54. package/src/migration/steps.js +244 -0
  55. package/src/model/collection.js +273 -0
  56. package/src/model/define.js +125 -0
  57. package/src/model/entity.js +307 -0
  58. package/src/model/index.js +47 -0
  59. package/src/model/relation.js +85 -0
  60. package/src/provider.js +137 -20
  61. package/src/schema/brand.js +31 -0
  62. package/src/schema/builders.js +526 -0
  63. package/src/schema/check.js +29 -0
  64. package/src/schema/emit.js +394 -0
  65. package/src/schema/factories.js +239 -0
  66. package/src/schema/index.js +37 -0
  67. package/src/schema-of.js +24 -0
  68. package/src/sequence.js +233 -103
  69. package/src/sources.js +10 -3
  70. package/types/app.d.ts +293 -0
  71. package/types/contract.d.ts +468 -0
  72. package/types/db.d.ts +359 -0
  73. package/types/flow.d.ts +285 -0
  74. package/types/forms.d.ts +253 -0
  75. package/types/index.d.ts +296 -26
  76. package/types/jslt.d.ts +193 -0
  77. package/types/migration.d.ts +201 -0
  78. package/types/model.d.ts +526 -0
  79. package/types/schema.d.ts +494 -0
@@ -0,0 +1,1724 @@
1
+ # The Jaren query pen
2
+
3
+ > the chain, `.` — query documents (`jaren-query`) and the provider
4
+ > seam. **Read it when** you are querying data, or implementing a
5
+ > provider that answers a query document
6
+
7
+ Version 0.1. The key words MUST, MUST NOT, SHOULD and MAY are to be
8
+ interpreted as described in RFC 2119. This document is a **guide** — read
9
+ it in order and you can write the format — whose normative section is
10
+ [§4 The mapping table](#4-the-mapping-table); the rules every pen keeps,
11
+ the shared refusal table, the index of the other pens and every pen's
12
+ mapping table collected in one place are the normative reference,
13
+ [LINQ-FORMAT.md](LINQ-FORMAT.md).
14
+
15
+ ## 1. Scope
16
+
17
+ You have data — an array in memory, a store's rows, a stream — and a
18
+ question to ask it, and you would like to write that question the way you
19
+ write code: filter, order, project, join, group. What you actually need
20
+ to hand the engine is a JSON document. So you either write the document,
21
+ in a grammar your editor knows nothing about, or you write JavaScript
22
+ and lose the ability to send it anywhere. This is the third option: a
23
+ method chain that RECORDS what you wrote and hands you the document.
24
+
25
+ `@jarenjs/linq` is a fluent front-end to the Jaren JSON Query language
26
+ ([QUERY-FORMAT.md](../../json/docs/QUERY-FORMAT.md)): a C#-familiar
27
+ method chain whose expressions are CAPTURED as plain query documents
28
+ and executed deferred — over any iterable in memory, or by any
29
+ **provider** exposing `execute(document, options)` (§8). The builder
30
+ emits the query language and nothing else; there is no second grammar,
31
+ no private protocol, and no `Function.prototype.toString` anywhere.
32
+
33
+ **Scope.** This document is normative for the CHAIN — the `.` entry, the
34
+ query documents it emits, and the provider seam. The package's other
35
+ subpaths are **pens**: the same idea applied to the suite's other
36
+ formats, each writing exactly the published document its engine already
37
+ takes. They have one document each, indexed by the binder,
38
+ [LINQ-FORMAT.md](LINQ-FORMAT.md), which covers the rules every pen keeps
39
+ (§1) and the `JL01xx` refusal table this document's §9 mirrors;
40
+ `@jarenjs/linq/db` — the store's typed front door and the package's one
41
+ runtime edge — is [DB-CLIENT.md](DB-CLIENT.md).
42
+
43
+ What this package is NOT: it is not a storage engine — the store, its
44
+ tables, its planner, its unit of work and its migrations are
45
+ `@jarenjs/db`'s, and the client subpath is that store's front door
46
+ rather than a second engine; it does not evaluate JavaScript callbacks
47
+ per element (callbacks run ONCE, at build time, against recording
48
+ proxies); it infers nothing from a JSON literal (a document stays a
49
+ document — `from(json)` is `unknown` until the caller says otherwise,
50
+ and the pens are the only inference route); and it promises nothing the
51
+ query grammar cannot express — §4 records every such gap as
52
+ `unsupported`, by name.
53
+
54
+ **Why this one is the longest.** The other ten documents target 600–1000
55
+ lines; this one is half again as long, and it stays one document. Its
56
+ §1–§12 are cited by section number from more than seventy places, so the
57
+ numbering is fixed and the sections cannot be split or moved. It also
58
+ covers three surfaces no pen has — the chain, the asynchronous surface
59
+ and the provider contract — each of which a different reader arrives for.
60
+ A reader who wants only one of the three should use the section list: §1
61
+ to §7 are the chain, §8 and §12 the provider seam, §10 and §11 the
62
+ asynchronous surface, and §13 to §17 the same seven sections every pen
63
+ document carries.
64
+
65
+ **The running example.** §13's eight fences are one question asked eight
66
+ ways over one small blog's data — the users, the posts they wrote and the
67
+ orders placed against them — and §15 reads the types back off the same
68
+ chains. Two of the eight need a PROVIDER rather than an array, and they
69
+ are that same blog seen as a store's entity sets.
70
+
71
+ **How to read this document.** The ten pen documents this one is indexed
72
+ beside share a fixed seven-section shape, and a reader who has learned
73
+ one of them arrives here expecting it. This document keeps its own twelve
74
+ sections instead — 72 citations across the repository point at them by
75
+ number, and moving one would break them silently — so the same seven
76
+ questions are answered where they already were, and the five that had no
77
+ home were appended rather than inserted:
78
+
79
+ | What you came for | Read |
80
+ |---|---|
81
+ | what it writes, and the one-import example | §1 Scope, §2 The surface |
82
+ | the mapping table: every operator and what it emits | §4 |
83
+ | worked examples, executed by the docs gate | §13 |
84
+ | refusals: the spelling that trips each code, and the one that works | §14 (§9 is the normative code table) |
85
+ | the types | §15 |
86
+ | what it cannot spell | §16 |
87
+ | cost | §17 |
88
+
89
+ The eight sections the table does not name have no counterpart in a pen
90
+ document at all, because they are the chain's own subject: §3 expression
91
+ capture, §5 deferred execution and re-enumeration, §6 terminal
92
+ semantics, §7 parameters, §8 the provider contract, and §§10–12 the
93
+ asynchronous surface, its concurrency boundary and its source adapters.
94
+ A reader following the suite from the binder,
95
+ [LINQ-FORMAT.md](LINQ-FORMAT.md), can skip to the row they need; a
96
+ reader learning the chain should read §3, §5 and §6 in order first,
97
+ because everything else assumes them.
98
+
99
+ ## 2. The surface
100
+
101
+ ```js
102
+ import { from } from '@jarenjs/linq';
103
+
104
+ const adults = from(users)
105
+ .where((u) => u.age.gt(21))
106
+ .orderBy((u) => u.name)
107
+ .select((u) => ({ id: u.id, name: u.name }));
108
+
109
+ adults.toArray(); // executes in memory
110
+ adults.toDocument(); // the SAME query, as one JSON document:
111
+ // { "$for": { "it": ["$[*]"] },
112
+ // "$where": { "$gt": ["$it.age", 21] },
113
+ // "$orderby": { "$key": "$it.name" },
114
+ // "$return": { "id": "$it.id", "name": "$it.name" } }
115
+ ```
116
+
117
+ (The source is bound through an array constructor, `["$[*]"]`, so that
118
+ an array-valued row stays one item — §5 says why; a provider's own root
119
+ is bound bare.)
120
+
121
+ - `from(source, options?)` — `source` is any iterable (arrays,
122
+ strings, generators, Sets…) or a provider (§8); anything else is
123
+ `JL0001` at `from()` time, never at enumeration time. A provider's
124
+ items are bound through ITS root (`root` — `'$.Post[*]'` for a store's
125
+ entity set; the emitted `$for` iterates that root, bare); a provider
126
+ that serves several roots and none of its own (a store with entities,
127
+ `roots`) is `JL0007` at `from()` time, naming the roots to chain over;
128
+ a provider carrying a relation table (`relations` — a store's entity
129
+ set does) lets a relation member NAVIGATE (§3): `p.author.email`
130
+ lowers to the correlated phrase the engine and the store run, and the
131
+ document never carries the relation's name.
132
+ `options.compileTypeTest` enables the schema operators behind
133
+ `ofType`/`cast` (§4); absent, those two are `JL0003` with the fix in
134
+ the message.
135
+ - `fromDocument(source, document, options?)` — attach a hand-written
136
+ or stored query document; its result is the item sequence and every
137
+ operator chains over it. Exactly the `{ "$query": "0.1", "$expr": … }`
138
+ envelope is unwrapped; any other envelope — an unknown version, a
139
+ stray member, a missing half — is compiled first so the ENGINE's
140
+ verdict (`JQ0006`, `JQ0001`, `JQ0003`) is what surfaces, never a
141
+ silent run under this version.
142
+ - Every operator returns a NEW immutable sequence (§5); terminal
143
+ operations execute (§6).
144
+
145
+ ## 3. Expression capture
146
+
147
+ A predicate or projection callback receives a **recording proxy** per
148
+ binding (and the parameters proxy last, §7). Member access records a
149
+ path segment; a method call records an operator; the callback's return
150
+ value becomes the expression:
151
+
152
+ - `u.a.b` records the path `$it.a.b`; `u.list.at(0)` records
153
+ `$it.list[0]` (negative integers count from the end);
154
+ `u.list.all()` records `$it.list[*]`; a key that is not an
155
+ identifier — or one that collides with a method name — goes through
156
+ `u.get('odd key')`.
157
+ - Method names SHADOW member access: `u.eq` is the operator, never
158
+ the member. `u.get('eq')` reaches the member.
159
+ - Returned object literals become constructors: plain-keyed objects
160
+ are Rule 1 map constructors, arrays are Rule 3 array constructors,
161
+ and a data object with `$`-prefixed keys embeds through `$map`.
162
+ Literal strings embed with the `$$` escape when they start with
163
+ `$`; plain data trees embed as `$const`.
164
+ - A proxy belongs to exactly ONE capture. Storing one and replaying
165
+ it into a later operator is `JL0002` — the emitted document would
166
+ silently reference the wrong binding, so the build fails instead.
167
+ Captures NEST: a chain built and run inside a callback is ordinary
168
+ (`select((u) => ({ id: u.id, n: from(rows).count() }))`); an
169
+ enclosing capture's proxy used inside the nested one is `JL0002` by
170
+ name — the inner document rebinds `$it`, so a correlated subquery
171
+ cannot be spelled this way. (`===` between proxies is untrappable and
172
+ therefore undetectable; do not compare proxies.)
173
+ - **JavaScript's own operators are not trappable, and they do not
174
+ fail loudly.** A proxy is an object, so `&&`, `||`, `!`, `?:`, `in`,
175
+ `typeof`, `Object.keys` and `===` evaluate against the PROXY and
176
+ yield a silently wrong document: `u.age.gt(21) && u.name.eq('x')`
177
+ captures only the right operand, `!u.deleted` is the constant
178
+ `false`, `u.deleted ? 'a' : 'b'` is always `'a'`. Use the expression
179
+ surface — `.and()`, `.or()`, `.not()` — for logic. Arithmetic and
180
+ comparison operators (`u.age > 21`, `u.age + 1`, `${u.name}`) throw a
181
+ plain `TypeError` (a proxy cannot be converted to a primitive): loud,
182
+ but not coded.
183
+ - The item binding is always named `it` in the emitted document
184
+ (nested phrases shadow it legally), so captured expressions read
185
+ `$it.…` at every depth and the document stays hand-readable.
186
+ - **A relation name hops.** When the items are the rows of an entity
187
+ whose provider carries a relation table (§8: `relations`, a store's
188
+ entity set), a member access naming a declared relation records a
189
+ HOP rather than a path segment — `p.author` is the related row,
190
+ `u.posts` the array of related rows — and is lowered, at capture, to
191
+ the correlated phrase §4's "relation navigation" rows spell; the
192
+ emitted document carries the phrase, never the member's name. The
193
+ hop's target is the target entity's row, with ITS relation table, so
194
+ hops chain (`p.author.posts`); a trailing path continues on the target
195
+ (`p.author.email`); `all()` on a to-many hop fans the related rows,
196
+ and the aggregates and `exists()`/`isEmpty()` range over them. A
197
+ relation name reached through `get()` hops too (the escape for a
198
+ relation that collides with a method name). The rows stop being rows
199
+ at a projection — after `select`, `selectMany`, `groupBy`, `join`,
200
+ `groupJoin`, `aggregate` (and a `mapAsync`) a relation name is an
201
+ ordinary member again — and a `fromDocument` chain never hops: there
202
+ the document decides what the items are. A hop that cannot lower is
203
+ `JL0105` at build time (a composite key; a many-to-many entry that
204
+ does not name its join row's columns, so there is no join root to
205
+ lower through — MODEL-FORMAT §10.7); a member read off
206
+ the to-many ARRAY before `all()` is `JL0005` with the fix named, where
207
+ the same read off a stored array would answer nothing.
208
+
209
+ ## 4. The mapping table
210
+
211
+ Status vocabulary: **native** (emits the named construct), **emulated**
212
+ (emits a composition with identical semantics), **unsupported** (throws
213
+ a coded error naming the reason — an honest row beats a silently wrong
214
+ emission). The *typing* column is the intended TypeScript signature
215
+ shipped by the typed surface order; an operator whose signature cannot
216
+ be written is an operator whose runtime shape is wrong, so the column
217
+ is part of THIS design.
218
+
219
+ | C# / LINQ | Emission | Status | Typing (element `T`) |
220
+ |---|---|---|---|
221
+ | `Where` | FLWOR `$where` | native | `(e: Expr<T>) => Expr<boolean>` → `Seq<T>` |
222
+ | `Select` | `$return` constructor | native | `(e: Expr<T>) => Expr<R>` → `Seq<R>` |
223
+ | `SelectMany` | `$return` of a `$for` phrase over the projection — the projected value is iterated ONE level (an array member's elements, a constructed array's members; a scalar is itself), and the FLWOR `$return` concatenates per tuple. Emitted as `{ "$for": { "it": <projection> }, "$return": "$it" }` (the nested phrase rebinds `it` legally) | native | `(e: Expr<T>) => Expr<R[]>` → `Seq<R>` |
224
+ | `OrderBy` / `OrderByDescending` | `$orderby` key spec (`$dir`; `$empty`/`$collation` via `options`) | native | `(e: Expr<T>) => Expr<K>` → `Seq<T>` |
225
+ | `ThenBy` / `ThenByDescending` | appended `$orderby` spec; must directly follow `orderBy*` (`JL0005`) | native | as `OrderBy` |
226
+ | `GroupBy` | `$groupby`; downstream items are `{ key, items }` | native | `(e: Expr<T>) => Expr<K>` → `Seq<{key: K, items: T[]}>` |
227
+ | `Join` | nested `$for` + `$where` `$eq` — the engine rewrites this shape to a HASH JOIN (compile-time, QUERY-FORMAT §6), which is why it is fast **when both keys are plain member paths** (`o => o.pid`, `i => i.id`); a key with an operator in it (`o => o.name.lower()`, `o => o.p.add(0)`) is not a probe key and the join runs as a nested loop. Both sides MUST derive from the same source, or from two providers sharing one `scope` (§8 — two entity sets of one store are two roots of ONE multi-entity input, and the store answers the equijoin in one statement); anything else is `JL0005`: a query document reads one input. On the async surface the join exists only over a provider, pushed whole (§10). The inner side's declared parameters ride along (§7) | native | `(inner: Seq<U>, ok, ik, (o: Expr<T>, i: Expr<U>) => Expr<R>)` → `Seq<R>` |
228
+ | `GroupJoin` | the matching group bound as an ARRAY value — `$let: { g: [ <correlated inner phrase> ] }` — so the result selector can index it (`g.at(0)`), fan it (`g.all()`), place it in a member (`{ matches: g }`) and aggregate over its members (`(u, g) => ({ n: g.count() })` counts the matches, `g.exists()` is whether there are any); same-source rule and parameter merge as `Join` | emulated | `(inner: Seq<U>, ok, ik, (o: Expr<T>, g: ArrayExpr<U> & AggregatableExpr) => Expr<R>)` → `Seq<R>` |
229
+ | `Skip` / `Take` | `$subsequence` | native | `(n: number)` → `Seq<T>` |
230
+ | `Distinct` | `$distinct` (deep structural equality — the grouping relation) | native | `()` → `Seq<T>` |
231
+ | `Reverse` | `$reverse` | native | `()` → `Seq<T>` |
232
+ | `Count` / `Sum` / `Average` / `Min` / `Max` | §8.8 aggregates (`Average` → `$avg`) | native | `count(): number`; `sum(): number`; `average/min/max(): number` (throw `JL2001` on empty; `min`/`max` follow the operand family) |
233
+ | `Any()` | `$exists` | native | `(): boolean` |
234
+ | `Any(pred)` / `All(pred)` | `$some` / `$every` quantifier phrase | native | `(pred): boolean` (`all` vacuously true on empty) |
235
+ | `Aggregate(seed, fn)` | `$fold` — the accumulator clause; the result is a sequence of exactly ONE accumulated value (`.first()` reads it) | native | `(seed: A, (acc: Expr<A>, e: Expr<T>) => Expr<A>)` → `Seq<A>` |
236
+ | `Aggregate(fn)` (unseeded) | — JSON cannot spell "the implicit first element" as a lambda seed | unsupported (`JL0006`) | — |
237
+ | `First` / `FirstOrDefault` | `[ $subsequence [expr, 0, 1] ]` window | native | `(): T` (`JL2001` on empty) / `(d?): T \| D` |
238
+ | `Single` / `SingleOrDefault` | `[ $subsequence [expr, 0, 2] ]` window | native | `(): T` (`JL2001`/`JL2002`) / `(d?): T \| D` (`JL2002` on 2+) |
239
+ | `Last` / `LastOrDefault` | `[ $subsequence [$reverse expr, 0, 1] ]` | native | as `First` |
240
+ | `ElementAt` / `ElementAtOrDefault` | `[ $subsequence [expr, i, 1] ]` | native | `(i): T` (`JL2003` out of range) / `(i, d?)` |
241
+ | `Concat` | `$seq` (a constant array's elements join the stream) | native | `(other: Seq<T> \| T[])` → `Seq<T>` |
242
+ | `DefaultIfEmpty` | `$default` | native | `(fallback?: T)` → `Seq<T>` |
243
+ | `OfType<S>` | `$valid` filter with a JSON Schema literal | native | `(schema)` → `Seq<S>`; needs `compileTypeTest` (`JL0003`) |
244
+ | `Cast<S>` | `$assert` per item | native | `(schema)` → `Seq<S>`; needs `compileTypeTest` (`JL0003`) |
245
+ | `Zip` | — no positional co-iteration in the grammar | unsupported (`JL0006`) | — |
246
+ | expression methods | `eq ne lt le gt ge` → `$eq…$ge`; `and or not`; `add sub mul div idiv mod neg`; `startsWith endsWith contains matches upper lower length concat substring replace` → §8.7; `count sum avg min max` → §8.8 (aggregates as expressions, e.g. over a group); `exists isEmpty`; `at all get` | native | on `Expr<…>`, per the typed-surface order |
247
+ | date family (§8.13) | the whole family, one method per operator. Components `year month day hours minutes seconds offset week weekYear quarter weekday`; instants `epoch datetime`; predicates `isDate isTime isDatetime isDuration`; arithmetic `startOf(unit) endOf(unit) dateAdd(duration \| amount, unit?) dateSub(…) dateDiff(to, unit) dateFormat(pattern)`. `dateAdd`/`dateSub`/`dateFormat` carry the prefix because `add`, `sub` and `format` are taken or ambiguous on this surface — the same reason §8.14 spells `geoArea`. There is no `now()`: §8.13 has no clock, and a fluent surface does not get to add one | native | on `DateTimeExpr` (the `DateTime` brand) and on `UnknownExpr` |
248
+ | series family (§8.16) | `overlaps(other)` → `$overlaps`; `timeBucket(every, origin?, context?)` → `$time-bucket`; `resample(spec)`, `rolling(spec)` and `asof(right, spec?)` → the three sequence operators. A **spec is a literal** and is embedded verbatim — it is read once when the query compiles, so a spec built from the row is `JL0005`, and every rule about what it may *say* stays in the compiler (`JQ0003`). Note that a member literally named `at` is read with `get('at')`: `at(index)` is path navigation on this surface | native | on `ArrayExpr`/fanned paths for the three sequence operators, on `Expr<…>` for the two scalar ones |
249
+ | spatial family (§8.14) | `bbox geoArea geoLength centroid` → `$bbox $area $length $centroid`; `distance within bboxIntersects` → `$distance $within $bbox-intersects`; `geohash(precision?)` → `$geohash` (optional arity, like `substring`); `geoParse geoText geohashBounds geohashNeighbours` → the conversion family; `geoSimplify(tolerance)` → `$geo-simplify`. A plain JSON polygon embeds as a literal (`p.at.within(poly)`); `.params({ region })` makes it an external instead | native | on `Expr<…>`, per the typed-surface order |
250
+ | vector family (§8.15) | `similarity(other)` → `$similarity`. The other operand is an array of numbers: a captured one embeds as a literal, `.params({ query })` binds it at call time. There is no `knn` method — k-nearest is `orderByDescending(...).take(k)`, which is the composition the emitted document already is | native | on `Expr<…>`, per the typed-surface order |
251
+ | relation navigation — to-one hop (`p.author`, `p.author.email`) | over a provider with a relation table (§3, §8): `{ "$for": { "r1": "$.User[*]" }, "$where": { "$eq": ["$r1.<targetKey>", "$it.<via>"] }, "$return": "$r1.email" }` — the target's key against the row's foreign key (`kind: "oneToOne"`, the key on the declaring entity). Zero or one item: an object member's one value (absent when there is none), an operand elsewhere (empty compares false; `exists()`/`isEmpty()` say which), and under `$orderby` a key that may be empty (`$empty` applies). The binding is `r1`, `r2`, … per capture | native by desugaring — the document is the phrase; a store runs it as a named residual (`explain()`, MODEL-FORMAT §10.6) | `Expr<Post>['author']` is `ObjectExpr<User>` — emit's optional relation member, nothing new |
252
+ | relation navigation — to-many hop (`u.posts`, `u.posts.all()`) | `{ "$for": { "r1": "$.Post[*]" }, "$where": { "$eq": ["$r1.<via>", "$it.<targetKey>"] }, "$return": "$r1" }` — the target's foreign key against the row's key (`kind: "oneToMany"`, the key on the target). As a VALUE the phrase is packed, `[ <phrase> ]`, the array of related rows a member holds (`{ posts: u.posts }`; `u.posts.at(0)` indexes it); fanned, `u.posts.all()` is the bare phrase, a sequence: `.all().count()` → `{ "$count": <phrase> }`, `.all().exists()` → `{ "$exists": <phrase> }`, `.all().title` returns `"$r1.title"` per row (`[u.posts.all().title]` packs the titles). `count()`/`exists()` on the value range over the rows too, as a group-join's group's do | native by desugaring, as above | `ArrayExpr<Post>`; `all()` is `FannedExpr<Post>` |
253
+ | relation navigation — chained, and from every row binding | hops nest: `p.author.posts.all().count()` is `{ "$count": { "$for": { "r1": "$.User[*]" }, "$where": …, "$return": { "$for": { "r2": "$.Post[*]" }, "$where": { "$eq": ["$r2.authorId", "$r1.id"] }, "$return": "$r2" } } }` — the inner phrase correlates with the outer binding; a hop off a fanned to-many (`u.posts.all().author`) is a sequence, one target per row; a join's `it2` hops from the inner row; a group-join's fanned group (`g.all().author`) binds each row first (`{ "$for": { "r1": "$g[*]" }, "$return": <hop over $r1> }`); the group itself is an array, not a row | native by desugaring, as above | as the target's `Expr<…>` |
254
+ | relation navigation — many-to-many (`u.labels`) | — the join table is not a queryable root in this version, so no phrase exists to lower to; `load({ include: { labels: true } })` reads the memberships | unsupported (`JL0105`, naming the join table) | — |
255
+
256
+ Two spatial names are deliberately not the obvious ones, and the reason
257
+ is the same one that made §8.14's `$length` and §8.7's `$string-length`
258
+ two operators: **`length` on this surface is already `$string-length`**,
259
+ and §8.14's `$length` is a geodesic line measurement. One method name
260
+ cannot carry both, and renaming the shipped string method for symmetry
261
+ would break a published surface for a cosmetic gain — so the spatial one
262
+ is **`geoLength`**, and **`geoArea`** joins it, because a bare `area()`
263
+ on an arbitrary expression reads as arithmetic to a C# eye. The prefix
264
+ names the family the way `geoParse`/`geoText` already do.
265
+
266
+ Every method name shadows a data member of the same name — that is what
267
+ the null prototype on the method table is for, and what `get(name)`
268
+ escapes. A position stored as `at` is the case that bites: `p.at` is the
269
+ index method, so it reads `p.get('at').within(region)`. A stored score
270
+ named `similarity` is the same bite with a worse error — `r.similarity`
271
+ is the *method*, so calling it as a member yields a `TypeError` about a
272
+ function rather than a coded build error, because the surface never sees
273
+ a member access at all. `r.get('similarity')` reads the data.
274
+
275
+ **k-nearest is a chain, not a method.** `similarity()` is one operator
276
+ and the ordering and the window are stages that already exist, so the
277
+ top k reads as what it is:
278
+
279
+ ```js
280
+ from(memories)
281
+ .params({ query })
282
+ .orderByDescending((m, p) => m.embedding.similarity(p.query), { empty: 'least' })
283
+ .thenBy((m) => m.id)
284
+ .take(10)
285
+ .select((m) => m.text)
286
+ ```
287
+
288
+ `{ empty: 'least' }` under a descending sort puts the rows whose key is
289
+ empty — no vector, or one of the wrong width — **last**, and `thenBy` on
290
+ the identity breaks ties, so the chain answers the same rows in the same
291
+ order every time it runs. `.params({ query })` rather than a captured
292
+ array is what makes the emitted document one query for every question,
293
+ which is the shape a provider can push down.
294
+
295
+ ## 5. Deferred execution and re-enumeration
296
+
297
+ Every operator returns a new immutable `Sequence`; NOTHING runs until a
298
+ terminal operation. A sequence may be enumerated repeatedly and **each
299
+ enumeration re-reads the source** — the C# contract, and the one that
300
+ surprises people:
301
+
302
+ ```js
303
+ const rows = [1, 2, 3];
304
+ const q = from(rows).where((n) => n.gt(1));
305
+ q.toArray(); // [2, 3]
306
+ rows.push(4);
307
+ q.toArray(); // [2, 3, 4] — the source was read AGAIN
308
+ ```
309
+
310
+ The compiled query is shared through a bounded cache keyed by the
311
+ document's exact JSON text — COLLISION-FREE and ORDER-SENSITIVE — so
312
+ re-enumeration is cheap without pretending the results are frozen.
313
+ A 32-bit fingerprint would not do here: it collides after tens of
314
+ thousands of documents, and a collision means one query runs another
315
+ query's compiled program — wrong rows, cache hit reported, nothing said.
316
+ Nor would an order-insensitive identity: a constructor's member order is
317
+ part of a document's meaning, and `{ id, name }` and `{ name, id }` must
318
+ each answer in their own order however the cache is warmed.
319
+ `for…of` a sequence iterates `toArray()`'s result (one enumeration per
320
+ loop).
321
+
322
+ **`toDocument()` is a deep snapshot**, on both surfaces. A sequence is
323
+ immutable, so the document it hands out is an independent tree: writing
324
+ into a returned document (or into `explain()`'s) cannot change what a
325
+ later enumeration answers.
326
+
327
+ **An item is an item.** The engine's `$for` unpacks an item that is an
328
+ array into its members, one level (QUERY-FORMAT §6.2, D4) — right for a
329
+ path like `$.tags`, wrong for a chain, where an array-valued ROW (a CSV
330
+ record, a pair) is one item that `where`, `select` and `count` never
331
+ split. So the emitter binds every source a phrase iterates through an
332
+ array constructor — `{ "$for": { "it": ["$[*]"] } }` — whose one array
333
+ item is unpacked exactly once, into the rows as they are; a reseated
334
+ phrase and a join side are packed the same way. `from([[1, 2], [3]])
335
+ .where(() => true).count()` is `2` on both surfaces, and the streaming
336
+ async surface agrees by construction. The one source left bare is a
337
+ PROVIDER's own root (`$.Post[*]`): a stored document is an object, so D4
338
+ never applies there, and the bare root is the shape the provider's
339
+ planner pushes.
340
+
341
+ **Constants come back frozen and shared on the sync surface.** A
342
+ literal object or array in a projection, a `defaultIfEmpty` fallback or
343
+ a `concat` array is engine data: every row that yields it yields the
344
+ SAME frozen value (`rows[0] === rows[1]`, and writing into it throws).
345
+ The async surface hands out a fresh copy per enumeration instead —
346
+ same values, no shared identity — because a streamed row is yours.
347
+
348
+ **A `null` a callback returns is a VALUE, not an absent clause.**
349
+ `where(() => null)` filters everything out (null is not true),
350
+ `select(() => null)` projects nulls, `groupBy(() => null)` is one
351
+ null-keyed group, and a null seed still folds. The emitted document
352
+ carries the clause with its null in place.
353
+
354
+ **A captured constant crosses a real JSON boundary.** The query data
355
+ model is JSON, so a `Date`, `Map`, `Set`, `RegExp` or class instance is
356
+ refused (`JL0005`) rather than embedded — `Object.keys` reports nothing
357
+ for them, so they would embed as `{}` and the query would compare against
358
+ an empty object. `NaN` and `±Infinity` are refused for the same reason
359
+ (JSON has neither, and lenient serialization folds them into `null`), and
360
+ so is `-0`, which shares its JSON text with `0` while dividing to the
361
+ opposite infinity. Convert first — a `Date` to its ISO string or epoch
362
+ number. The boundary is the same for a `concat` array and for a
363
+ `params()` binding (`JL0004`): a parameter becomes an external and, on a
364
+ provider, a bound SQL parameter, so a `Date` there would compare against
365
+ nothing and answer `[]` with no error anywhere.
366
+
367
+ ## 6. Terminal semantics
368
+
369
+ The real C# semantics, because getting these wrong is how a
370
+ "LINQ-like" library becomes lodash with different names:
371
+
372
+ - `first()` on empty throws `JL2001`; `firstOrDefault(d)` returns `d`
373
+ (or `undefined` when omitted).
374
+ - `single()` on empty throws `JL2001`; on two-or-more throws `JL2002`;
375
+ `singleOrDefault(d)` throws on two-or-more and returns `d` on empty.
376
+ - `last()`/`lastOrDefault(d)` mirror `first` over the reversed window.
377
+ - `elementAt(i)` out of range throws `JL2003`;
378
+ `elementAtOrDefault(i, d)` returns `d`.
379
+ - `average()`, `min()` and `max()` over an empty sequence throw
380
+ `JL2001` (C# `InvalidOperationException`); `sum()` of nothing is `0`;
381
+ `count()` of nothing is `0`.
382
+ - `any()` is existence; `all(pred)` is vacuously true over the empty
383
+ sequence.
384
+
385
+ Element terminals emit their window inside an ARRAY constructor
386
+ (`[ … ]`), so the engine's result mapping (`undefined | item | items`)
387
+ can never confuse "one array-valued item" with "several items" — the
388
+ window array is always the single result and its elements are read
389
+ positionally.
390
+
391
+ ## 7. Parameters
392
+
393
+ `.params({ tenantId })` declares AND binds externals; a callback reads
394
+ them through its last argument:
395
+
396
+ ```js
397
+ from(rows)
398
+ .params({ tenantId: 'a7' })
399
+ .where((r, p) => r.tenant.eq(p.tenantId))
400
+ .toDocument();
401
+ // { "$for": { "it": ["$[*]"] },
402
+ // "$where": { "$eq": ["$it.tenant", "$tenantId"] },
403
+ // "$return": "$it" }
404
+ ```
405
+
406
+ The emitted document carries `$tenantId` as an external parameter
407
+ (QUERY-FORMAT §9) — the seam that later becomes a bound SQL parameter.
408
+ Undeclared use is `JL0004` at BUILD time with the fix in the message
409
+ (the engine would say JQ0005 at compile time; earlier and clearer
410
+ wins). The names `it`, `it2`, `acc` and `g` are RESERVED — they are the
411
+ emitted document's own binding names — and so are `r1`, `r2`, … (`r`
412
+ followed by a positive integer): the bindings a relation hop allocates,
413
+ numbered per capture (§3, §4 "relation navigation"). Declaring any of
414
+ them is `JL0004`. A binding must be query data (§5): a `Date`, `Map`,
415
+ `NaN` or `-0` is `JL0004` with the conversion named.
416
+
417
+ The inner side of a `join`, `groupJoin` or `concat` contributes its
418
+ document WHOLE, so its declared parameters ride along into the new
419
+ sequence (`explain().externals` lists the union, `explain().bindings`
420
+ the values bound so far — what a provider receives as `externals`, and
421
+ what a host handed the chain, such as a live registration, forwards
422
+ without a second spelling); a name both sides
423
+ bind to different values is `JL0004` — one document carries one binding
424
+ per name. Rebinding a name later (`.params({ k: 3 })`) re-runs the
425
+ whole document under the new value, on the async surface too: a
426
+ `params()` after a `mapAsync` rebinds the pushed prefix as well as the
427
+ residual.
428
+
429
+ ## 8. The provider contract
430
+
431
+ A **provider** is any object exposing:
432
+
433
+ ```
434
+ execute(queryDocument, options) -> undefined | item | items[]
435
+ ```
436
+
437
+ - `queryDocument` arrives WHOLE — a terminal hands over the full
438
+ emitted document (including the terminal's own wrapper, §6);
439
+ nothing is enumerated locally, ever.
440
+ - `options.externals` is the `{ name: value }` record of bound
441
+ parameters (§7).
442
+ - The return value uses the ENGINE's result mapping
443
+ (`undefined` = empty, a single item as itself, several items as an
444
+ array) — the in-memory runner is the reference semantics every
445
+ provider MUST match, and it implements this same interface.
446
+ - **`execute` is SYNCHRONOUS on this surface.** A `Sequence` terminal
447
+ is a value — `toArray(): T[]`, `count(): number` — so a promise cannot
448
+ be returned under that type. A provider that answers one is refused
449
+ with `JL2004` at the seam, because the alternative is not a slow
450
+ answer but a wrong one: the promise came back typed as the value,
451
+ `count()` handed a `Promise` to arithmetic, and `first()` indexed the
452
+ promise and returned `undefined`. An asynchronous provider (a
453
+ wasm/OPFS driver, a store's asynchronous entity set) is `fromAsync`'s
454
+ source (§12): the same document arrives whole, and `execute` MAY
455
+ answer a promise there.
456
+
457
+ A provider MAY carry three more members, read at `from()`/`fromAsync()`
458
+ time:
459
+
460
+ - `root` — the path expression its items are bound through
461
+ (`'$.Post[*]'` for a store's entity set); absent means the whole
462
+ input, `'$[*]'`. The emitted document iterates the root BARE (§5): a
463
+ stored document is an object, so an item is never an array there.
464
+ - `roots` — the entity roots a STORE-LEVEL provider serves when it has
465
+ no root of its own (`['User', 'Post']`). Such a provider is refused by
466
+ `from()`/`fromAsync()` with `JL0007`, naming them: `$[*]` over the
467
+ entity map would answer every entity's rows mixed, or count the sets.
468
+ `fromDocument` keeps its own rule — there the document IS the root.
469
+ - `scope` — an identity two providers share when their documents may be
470
+ joined. One store's entity sets carry one `scope`, so
471
+ `from(posts).join(from(users), (p) => p.authorId, (u) => u.id, (p) => p)`
472
+ emits `{ "$for": { "it": "$.Post[*]", "it2": "$.User[*]" }, "$where":
473
+ { "$eq": ["$it.authorId", "$it2.id"] }, "$return": "$it" }` — the shape
474
+ the store's translator answers in ONE statement when the result is a
475
+ bare binding (MODEL-FORMAT §10.2), and the declared residual over both
476
+ fetched roots when it is a projection (§10.6). `concat` stays
477
+ same-source even within a scope: one input per document. A scope MAY
478
+ carry `relations` — the relation tables of every root of the scope,
479
+ keyed by root name (a store's does) — which is where a chained hop
480
+ finds its target's table; without it the first hop lowers and the
481
+ target's members are plain paths.
482
+ - `relations` — the relation table of the rows the provider serves
483
+ (MODEL-FORMAT §10.1; a store's entity set carries its entity's): a
484
+ plain record, one entry per declared relation member, `{ to, kind,
485
+ via?, fkEntity?, fkTargets?, joinTable?, targetKey }`. With it, a
486
+ relation name on a callback's row hops (§3) and is lowered to the
487
+ phrase §4's "relation navigation" rows spell; `kind` decides the
488
+ equality's sides (`oneToOne`: the key on the declaring entity;
489
+ `oneToMany`: on the target), `to` the root the hop binds (`$.<to>[*]`),
490
+ `via` and `targetKey` its two columns. A `manyToMany` entry is
491
+ `JL0105`. Absent, a relation name is an ordinary member.
492
+
493
+ An element terminal hands over the one-item WINDOW `[<phrase>]` (§6). A
494
+ provider that plans documents reads through that window — the store
495
+ plans the phrase inside as if it were bare and answers its rows as the
496
+ one array the constructor yields (`[]` for none, `[row]` for one) — so
497
+ `toArray()` and `first()` push exactly as `count()` does.
498
+
499
+ `@jarenjs/db` implements this contract without either package
500
+ importing the other: its collections and its entity sets are providers
501
+ (the sets carry `root`, `scope` and `relations`; the store carries
502
+ `roots` and `relations`), and a test double proves the document arrives
503
+ whole. A lowered hop is what a store receives as any other document: it
504
+ runs the correlated phrase in its residual over the fetched roots and
505
+ `explain()` names the §10.6 reason — no lowered shape pushes natively in
506
+ this version, and the store's `strict` refuses them all (`JD0010`).
507
+
508
+ ### 8.1 Compilation registries
509
+
510
+ `from(source, options)` and `fromDocument(source, doc, options)` take the
511
+ engine's own compile options, so a document that is expressible is also
512
+ executable in memory:
513
+
514
+ | option | what it enables |
515
+ |---|---|
516
+ | `compileTypeTest` | `ofType`/`cast` (the schema operators) |
517
+ | `collations` | `orderBy(…, { collation })` — a `nl` sort is `JQ0010` without it |
518
+ | `functions` | `$call` in a hand-written or saved document |
519
+ | `pathFunctions` | custom RFC 9535 path function extensions |
520
+ | `limits` | step, depth and sequence bounds — the reason a SAVED document can be run at all. `resultItems` does not bind a chain: a terminal reads ONE packed window (§6), so the bound that guards a chain's size is `sequenceItems` on its phrases; on the async surface only `steps`/`depth` and a barrier phase's `sequenceItems` apply, because streaming stages evaluate one item at a time |
521
+ | `registry` | an explicit cache-partition key, when the hooks above are rebuilt per call |
522
+
523
+ Compiled documents are cached per registry COMBINATION, not per document
524
+ alone: the same document compiles to different code with and without a
525
+ collation registry, so sharing one partition would answer a caller who
526
+ passed no collations with the compiled-with version.
527
+
528
+ ## 9. Error codes
529
+
530
+ Build errors (`LinqBuildError`; `docPath` where a document position
531
+ exists):
532
+
533
+ | Code | Condition |
534
+ |---|---|
535
+ | `JL0001` | `from()` received neither an iterable nor a provider |
536
+ | `JL0002` | an expression proxy escaped its capture callback |
537
+ | `JL0003` | `ofType`/`cast` need an injected `compileTypeTest` |
538
+ | `JL0004` | an undeclared or reserved parameter name was used |
539
+ | `JL0005` | an operator was used invalidly at build time |
540
+ | `JL0006` | an unsupported operator was invoked |
541
+ | `JL0007` | a provider serves several entity roots (`roots`) and has no root of its own — chain over `store.entity(name)` |
542
+
543
+ Pen build errors (`LinqBuildError`, raised by `@jarenjs/linq/schema`,
544
+ `/model`, `/jslt` and the pens that follow them; LINQ-FORMAT.md §1.3 is
545
+ the normative home, this table mirrors it):
546
+
547
+ | Code | Condition |
548
+ |---|---|
549
+ | `JL0101` | a pen received a value it cannot spell: not JSON, not what the keyword takes, or a name → value map whose prototype a `__proto__:` literal replaced |
550
+ | `JL0102` | a pen was asked for a construct the format cannot carry |
551
+ | `JL0103` | a `$defs` name collision, a dangling ref, or an unnamed recursion |
552
+ | `JL0104` | a pen-owned keyword through `meta()`, or an external a captured rule did not declare |
553
+ | `JL0105` | a relation hop on the chain cannot lower: a many-to-many member (its join table is not a queryable root), a composite or undeclared key, or a malformed relation entry (§3, §4 "relation navigation") |
554
+ | `JL0106` | a migration step names a table the target model does not declare, or a draft it cannot match |
555
+ | `JL0107` | the client (`@jarenjs/linq/db`, [DB-CLIENT.md](DB-CLIENT.md)) named a member that is not the relation kind the operation needs: `include()` over a member that is not a declared relation, `link()`/`unlink()` over a relation that is not many-to-many |
556
+
557
+ Runtime errors (`LinqRuntimeError`):
558
+
559
+ | Code | Condition |
560
+ |---|---|
561
+ | `JL2001` | `first`/`single` found no element |
562
+ | `JL2002` | `single` found more than one element |
563
+ | `JL2003` | `elementAt` is out of range |
564
+ | `JL2004` | an asynchronous provider cannot back the synchronous surface |
565
+ | `JL2005` | a push queue was fed after it ended |
566
+ | `JL2006` | a provider answered an element terminal with something other than one array |
567
+ | `JL2007` | a ledger settlement named a ref that settles no started record — `createDbLedger`'s fence ([DB-CLIENT.md §2.6](DB-CLIENT.md#26-the-ledger)) |
568
+ | `JL2008` | a federated fetch reached its row or byte budget (§12.1) |
569
+
570
+ Engine errors (`JQ…`) from a hand-written `fromDocument` document pass
571
+ through unwrapped — they already carry their own code and `docPath` —
572
+ with one exception: `JQ0008` (a schema operator with no type-test
573
+ compiler) is reported as `JL0003`, because the fix is the same
574
+ `compileTypeTest` hook whether `ofType`/`cast` or the document spelled
575
+ the operator.
576
+
577
+ ## 10. The asynchronous surface: streaming and barriers
578
+
579
+ `fromAsync(source, options?)` gives the same operator surface over
580
+ async sources — joins only over a provider, see the table — emitting the SAME query
581
+ documents: the same chain through `from` and `fromAsync` MUST emit
582
+ byte-identical documents (the one-operator-set proof), with terminals
583
+ returning promises. The rule: **the pipeline is synchronous, the
584
+ boundaries are async.** A compiled query never awaits; what is
585
+ asynchronous is where rows come from and where element-wise host work
586
+ happens (§11).
587
+
588
+ Per operator, whether it STREAMS (per-item evaluation, flat memory) or
589
+ is a BARRIER (materialises the stream so far and runs the maximal run
590
+ of document stages through the engine over the buffer — inherent,
591
+ because the engine itself materialises for `$orderby`/`$groupby`):
592
+
593
+ | Operator | Async behaviour |
594
+ |---|---|
595
+ | `where`, `select`, `selectMany`, `ofType`, `cast` | stream (per-item compiled evaluators — the engine, one item at a time) |
596
+ | `skip`, `take` | stream; `take` CLOSES the source when satisfied |
597
+ | `distinct` | stream, with a running key set (the grouping relation: `NaN` groups with `NaN`) |
598
+ | `defaultIfEmpty` | stream (an emptiness flag) |
599
+ | `concat` | stream for a CONSTANT array; another sequence is refused (`JL0005`) — an async source is single-pass and cannot be re-iterated for a second chain |
600
+ | | on the SYNC surface, `concat` also requires the same source: a query document reads one input, so the other sequence contributes its EXPRESSION, and a foreign sequence would have that expression evaluated against THIS source — reading the wrong rows twice instead of concatenating two inputs |
601
+ | `orderBy`/`thenBy`, `groupBy`, `aggregate`, `reverse` | BARRIER, named by `explain()` with the reason (a `thenBy` is part of the `$orderby` barrier it extends) |
602
+ | `join`, `groupJoin` | over a PROVIDER origin, before any `mapAsync`: pushed WHOLE inside the one document, with an async sequence over the same provider (or one sharing its `scope`) as the inner side — the store answers a two-root equijoin in one statement; over an iterable, a cursor or a push queue `JL0005`: a join's inner side re-reads the source, and a single-pass source cannot be read twice (join on the sync surface, or collect the stream first) |
603
+ | `count`, `any`, `all`, `first`, `single`, `elementAt` | stream with early exit where semantics allow |
604
+ | `sum`, `average`, `min`, `max`, `last` | consume the stream; the aggregate itself runs through the ENGINE over the collected items, so its semantics (type errors included) are identical to the sync surface |
605
+
606
+ **Early termination MUST close the source**: `first()`, `any()`,
607
+ `take(n)`, and an exception mid-chain all call `.return()` on the
608
+ iterator — a generator left suspended holds a file handle or a read
609
+ transaction open. `explain()` reports `{ barriers: [{ operator,
610
+ reason }], streaming, barrier, hops, document }` — `hops` the relation
611
+ hops the callbacks navigated, as on the sync surface (§4); `streaming`
612
+ (`'row'` or `'buffered'`) and `barrier` what THIS surface does with the
613
+ item stream, one item at a time or a buffer at its first local barrier
614
+ — or, when a `mapAsync` sits in the chain, `{ split: { pushed,
615
+ residual } }` instead of `document` (`toDocument()` refuses with
616
+ `JL0005`: a host callback has no document form). Over a provider the
617
+ pushed document's own class — a set residual, an external the store
618
+ cannot bind — is the provider's `explain(document, { externals:
619
+ bindings })` to report, and the cursor a `for await` pulls from
620
+ carries the same `streaming`/`barrier` (§12). No silent caps, no silent
621
+ buffering: if a chain materialises, the report says which operator
622
+ forced it.
623
+
624
+ Re-enumeration follows the sync contract: each enumeration calls the
625
+ source's iterator method again. A one-shot generator object simply
626
+ exhausts — the same way it does under `from`. Streamed constants
627
+ (`concat`, `defaultIfEmpty`) are handed out as a fresh copy per
628
+ enumeration (§5).
629
+
630
+ ## 11. The concurrency boundary
631
+
632
+ ```js
633
+ await fromAsync(rows)
634
+ .mapAsync(async (row, signal) => fetchScore(row.id, signal),
635
+ { concurrency: 8, mode: 'parallel', ordered: true })
636
+ .where((r) => r.score.gt(0.5))
637
+ .toArray();
638
+ ```
639
+
640
+ `mapAsync` is the ONE explicit boundary for element-wise asynchronous
641
+ host work. There is no parallel universe of `selectAwait`-shaped
642
+ operators; a per-element async *predicate* is `mapAsync` then `where`.
643
+
644
+ - `concurrency` is REQUIRED and MUST be a positive integer (`JL0005`)
645
+ — the unbounded default is how libraries like this take down a
646
+ downstream service.
647
+ - `mode` reuses the `createTaskEffect` vocabulary (`@jarenjs/app` §9),
648
+ deliberately, so a reader who knows one knows the other:
649
+ `parallel` (a sliding window of N), `concat` (strictly sequential),
650
+ `switch` (a newer item supersedes and ABORTS the in-flight task),
651
+ `exhaust` (items arriving while busy are dropped). The source is
652
+ pulled eagerly under `switch`/`exhaust` — that race IS the mode.
653
+ - `ordered: true` (default) preserves source order and buffers at most
654
+ `concurrency` results — the stated cost; `ordered: false` yields on
655
+ completion.
656
+ - An `AbortSignal` is threaded to every callback and aborted on early
657
+ termination and on failure. A rejected callback FAILS CLOSED: the
658
+ first rejection aborts every in-flight sibling and closes the source,
659
+ including while its next pull is pending. Ordered mode still delivers
660
+ preceding values and reports callback failures in source order. The
661
+ producer is closed once; a failed close accompanies the callback's
662
+ original failure in an `AggregateError`.
663
+ - `mapAsync` is NOT translatable to a provider. A provider-backed
664
+ chain that reaches it SPLITS: everything before is pushed to the
665
+ provider whole, everything after runs locally, and `explain()`
666
+ reports `{ split: { pushed, residual } }` — the same residual
667
+ honesty the SQL pushdown owes (D8), applied to the async boundary.
668
+
669
+ ## 12. The cursor contract and the source adapters
670
+
671
+ `fromAsync` accepts, in order of preference:
672
+
673
+ - a **provider** (§8) — asked for BEFORE the shapes below, so an
674
+ `execute` duck that also happens to be iterable is a provider. The
675
+ whole chain up to the first `mapAsync` — the terminal's wrapper
676
+ included — is ONE document `execute` receives, once, with the bound
677
+ externals, and `execute` MAY answer a promise here (D8: the contract
678
+ mirrors §8's; the awaiting is this surface's). The residual after a
679
+ `mapAsync` streams locally over the pushed rows, and `explain()`
680
+ reports `{ split: { pushed, residual } }` exactly as for a
681
+ synchronous prefix, with `barriers` naming only the residual's own.
682
+ The same chain through `from(store.sync.entity('X'))` and
683
+ `fromAsync(store.entity('X'))` MUST emit byte-identical documents —
684
+ the one-operator-set proof, extended to roots;
685
+ - any **`AsyncIterable`** (async generators, `ReadableStream` — every
686
+ target exposes `Symbol.asyncIterator` on it, josl's
687
+ `iterateCsvStream` output);
688
+ - any sync iterable (wrapped);
689
+ - a **cursor**: `{ next(): Promise<{done, value}>, return?() }` — the
690
+ shape the store's own row cursor implements (`QueryCursor`), adopted
691
+ as-is. A provider that offers `cursor(document, options)` — the
692
+ store's collections and entity sets do — is handed the pushed
693
+ document there when the chain is ITERATED, so a `for await` pulls
694
+ one row at a time from an open statement and a `break` releases it;
695
+ `toArray()` and the other terminals still push one whole window;
696
+ - a **push queue** (`createPushQueue({ highWaterMark = 1024 })`) for
697
+ feed/end-style readers with no pull protocol of their own (josl's
698
+ push parsers deliberately have no backpressure protocol; the queue
699
+ is where one appears): `feed(value)` returns `false` once the queue
700
+ exceeds the mark — a pause HINT, never a hard stop — and
701
+ `end(error?)` closes (or fails) the stream. Anything else is
702
+ `JL0001` at `fromAsync()` time.
703
+
704
+ What this surface does NOT do, by design: it does not make the query
705
+ engine async (`packages/json` is untouched and strictly synchronous),
706
+ it does not add a second operator table, and it does not add
707
+ `selectAwait`/`whereAwait` variants.
708
+
709
+ ### 12.1 The federation boundary (`federate`)
710
+
711
+ A query document reads ONE input, and a join whose sides come from two
712
+ unrelated sources is `JL0005` (§8). That refusal stands: nothing in a
713
+ chain relaxes it. What `federate()` adds is the one place to opt OUT of
714
+ it explicitly, by naming the sources and the bounds together:
715
+
716
+ ```js
717
+ const fed = federate({
718
+ sources: { orders: shop.entity('Order'), events: analytics },
719
+ maxRows: 50_000,
720
+ maxBytes: 32 * 1024 * 1024,
721
+ });
722
+
723
+ const rows = await fromAsync(fed.source('orders'))
724
+ .where((o) => o.placedAt.gt(cutoff))
725
+ .join(fromAsync(fed.source('events')), (o) => o.id, (e) => e.orderId,
726
+ (o, e) => ({ id: o.id, at: e.at }))
727
+ .toArray();
728
+ ```
729
+
730
+ `fed.source(name)` is an ordinary provider source (§8) with the root
731
+ `$.<name>[*]`, and all of a federation's sources share one scope — which
732
+ is exactly what admits the join. The federation is what executes it:
733
+
734
+ 1. each side's own document — the filters and the projection the chain
735
+ already packed per side — runs against ITS source, over that
736
+ source's own root;
737
+ 2. the smaller side (by declared `estimatedRows`, else the first named)
738
+ is streamed into a hash table keyed by the join key, counting rows
739
+ and serialized bytes against the budget as it fills;
740
+ 3. the other side is streamed and PROBED: a row whose key no build row
741
+ carries cannot pair, so it is dropped before it costs anything;
742
+ 4. the caller's own document runs in the engine over the two reduced
743
+ sets — the **resident join**, which is what decides.
744
+
745
+ Step 4 is the contract. This boundary spells no join semantics of its
746
+ own: the engine's `$eq` decides which rows pair, its ordering orders
747
+ them, its projection shapes them. The hash table bounds the FETCH and
748
+ nothing else, so a reduction that cannot key a value (a compound join
749
+ key) keeps the row rather than guessing at it.
750
+
751
+ `fed.source(name).explain(document)` answers the plan without running
752
+ any of it: the `strategy`, the `budget`, the `build` and `probe` sides
753
+ with their estimates, their own documents and whether each streams, and
754
+ the `resident` document the engine answers.
755
+
756
+ **A budget is a refusal, not a spill.** A side that reaches `maxRows` or
757
+ `maxBytes` stops at the row that WOULD have broken it and raises
758
+ `JL2008`; every cursor the call opened is closed exactly once, whether
759
+ it answered, refused, failed or was aborted. A `signal` on the call is
760
+ read at the ROW boundary — where a cursor can be let go without
761
+ abandoning a pull the source is still inside — and a declared
762
+ `estimatedRows` decides only which side BUILDS, never how much is held:
763
+ a source that under-reports is refused by the budget all the same. A source offering a
764
+ cursor (§12) is pulled row by row, so the bound is enforced before the
765
+ memory is spent; one offering only `execute` answers whole, and
766
+ `explain()` says `buffered` rather than pretending otherwise.
767
+
768
+ Refused, by name: more than two sides in one federated document, a
769
+ binding over a root the federation does not carry, and a join with no
770
+ equality between one member of each side — without one the fetch is the
771
+ cross product of two sources, which is what the budget exists to
772
+ refuse. Non-goals: no spill, no distributed transaction, no
773
+ cross-source write.
774
+
775
+ ## 13. Worked examples
776
+
777
+ Every `js` fence below is EXECUTED. `test/linq/pen-docs.test.js` writes
778
+ it as a module beside the workspace's `node_modules` — so `@jarenjs/linq`
779
+ resolves exactly as it does for a consumer — imports it, and asserts that
780
+ the single export's `toDocument()` equals the `json` fence beside it. A
781
+ fence that drifts from the emitter fails the suite; nothing here is a
782
+ sketch.
783
+
784
+ The eight are chosen to teach the CAPTURE MODEL rather than to cover the
785
+ operator table (§4 is the table). Read them in order: the first shows
786
+ what a chain is, and each one after it adds one thing the emitted
787
+ document does that the source does not obviously say.
788
+
789
+ They are also one question, asked eight ways, over one small blog's data
790
+ — the users, the posts they wrote and the orders placed against them.
791
+ Nothing is shared between the fences at run time (each is a whole module,
792
+ and that is what the gate runs), but the shapes are the same throughout,
793
+ so a member you meet in §13.1 means the same thing in §13.8, and the two
794
+ fences that need a PROVIDER rather than an array — the join and the hop —
795
+ are that same blog seen as a store's entity sets.
796
+
797
+ ### 13.1 The chain, whole
798
+
799
+ The opening example of the README and of §2, executed. `where` becomes
800
+ the FLWOR `$where`, `orderBy` an `$orderby` key spec
801
+ (`orderByDescending`, `thenBy` and `thenByDescending` extend the same
802
+ clause), and `select` the `$return` constructor — one phrase, in the
803
+ order a reader writes it.
804
+
805
+ ```js
806
+ import { from } from '@jarenjs/linq';
807
+
808
+ const users = [
809
+ { id: 1, name: 'Ada', age: 36 },
810
+ { id: 2, name: 'Bo', age: 19 },
811
+ ];
812
+
813
+ export const adults = from(users)
814
+ .where((u) => u.age.gt(21))
815
+ .orderBy((u) => u.name)
816
+ .select((u) => ({ id: u.id, name: u.name }));
817
+ ```
818
+
819
+ ```json
820
+ {
821
+ "$for": { "it": ["$[*]"] },
822
+ "$where": { "$gt": ["$it.age", 21] },
823
+ "$orderby": { "$key": "$it.name" },
824
+ "$return": { "id": "$it.id", "name": "$it.name" }
825
+ }
826
+ ```
827
+
828
+ `adults.toArray()` answers `[{ "id": 1, "name": "Ada" }]`. The source is
829
+ bound through an array constructor, `["$[*]"]`, and §5 says why: the
830
+ engine unpacks an item that is an array one level, which is right for a
831
+ path and wrong for a row.
832
+
833
+ ### 13.2 A join is a nested `$for` and an equality
834
+
835
+ Both sides read ONE input, so a join's other side derives from the same
836
+ source — or, as here, from a second provider sharing its `scope`: two
837
+ entity sets of one store are two roots of one multi-entity document, and
838
+ the store answers the equijoin in a single statement (§8).
839
+
840
+ ```js
841
+ import { from } from '@jarenjs/linq';
842
+
843
+ // two entity sets of ONE store: two roots of one multi-entity input,
844
+ // which is what a shared `scope` declares (§8)
845
+ const scope = {};
846
+ const postSet = { execute: () => [], root: '$.Post[*]', scope };
847
+ const userSet = { execute: () => [], root: '$.User[*]', scope };
848
+
849
+ export const bylines = from(postSet).join(
850
+ from(userSet),
851
+ (p) => p.authorId,
852
+ (u) => u.id,
853
+ (p, u) => ({ title: p.title, author: u.name }),
854
+ );
855
+ ```
856
+
857
+ ```json
858
+ {
859
+ "$for": { "it": "$.Post[*]", "it2": "$.User[*]" },
860
+ "$where": { "$eq": ["$it.authorId", "$it2.id"] },
861
+ "$return": { "title": "$it.title", "author": "$it2.name" }
862
+ }
863
+ ```
864
+
865
+ There is no `$join` operator in the emitted document and there is no need
866
+ for one: the engine recognises this shape at COMPILE time and runs a hash
867
+ join (QUERY-FORMAT §6). It recognises it **only when both key
868
+ expressions are plain member paths** — `(p) => p.authorId` and
869
+ `(u) => u.id` are; `(p) => p.title.lower()` is not, and that join runs as
870
+ a nested loop with the same answer and a different cost. A provider's
871
+ roots stay bare (`"$.Post[*]"`, not `["$.Post[*]"]`): a stored document
872
+ is an object, so the unpacking rule §5 guards against cannot arise.
873
+
874
+ ### 13.3 A group is a phrase, and its items are an array
875
+
876
+ `groupBy` emits `$groupby` and reseats: the downstream items are
877
+ `{ key, items }` objects, and every operator after it reads THOSE. The
878
+ `key` carries a `$default` to `null` because a group whose key expression
879
+ yielded nothing still has rows.
880
+
881
+ ```js
882
+ import { from } from '@jarenjs/linq';
883
+
884
+ const orders = [
885
+ { id: 1, city: 'Delft', total: 12 },
886
+ { id: 2, city: 'Delft', total: 30 },
887
+ { id: 3, city: 'Gouda', total: 7 },
888
+ ];
889
+
890
+ export const perCity = from(orders)
891
+ .groupBy((o) => o.city)
892
+ .select((g) => ({ city: g.key, orders: g.items.count(), total: g.items.all().total.sum() }));
893
+ ```
894
+
895
+ ```json
896
+ {
897
+ "$for": {
898
+ "it": [
899
+ {
900
+ "$for": { "it": ["$[*]"] },
901
+ "$groupby": { "g": "$it.city" },
902
+ "$return": { "key": { "$default": ["$g", null] }, "items": ["$it"] }
903
+ }
904
+ ]
905
+ },
906
+ "$return": {
907
+ "city": "$it.key",
908
+ "orders": { "$count": "$it.items[*]" },
909
+ "total": { "$sum": "$it.items[*].total" }
910
+ }
911
+ }
912
+ ```
913
+
914
+ `perCity.toArray()` answers `[{ city: 'Delft', orders: 2, total: 42 },
915
+ { city: 'Gouda', orders: 1, total: 7 }]`.
916
+
917
+ **A group aggregates as its ROWS.** `g.items.count()` is the number of
918
+ rows in the group: the chain knows `items` holds a group and emits
919
+ `{ "$count": "$it.items[*]" }` — the fan — rather than `$count` over the
920
+ one array value, which would answer `1` for every group. This is the
921
+ same rule a group-JOIN's group has always kept (`g.count()` there is the
922
+ number of matches, §4), and the two group shapes now spell it the same
923
+ way.
924
+
925
+ `g.items` itself is still the array, and everything an array can do it
926
+ still does: a member takes it whole (`{ rows: g.items }` emits
927
+ `"$it.items"`), `at(0)` indexes it, and `all()` fans it explicitly —
928
+ which is what `g.items.all().total.sum()` above needs, because summing a
929
+ MEMBER of each row means fanning the rows first and then reading the
930
+ member (`"$it.items[*].total"`). Only the aggregates changed, and only
931
+ for the member the emitter writes the group into.
932
+
933
+ An array a CALLER stored is a different thing and keeps the older rule:
934
+ `u.tags.count()` is `1`. At capture time an array member and a scalar
935
+ member are the same path — the chain has no type to tell them apart, and
936
+ inventing one would be a guess — so `u.tags.all().count()` is how the
937
+ elements are counted, and `u.tags.exists()` is what the un-fanned form
938
+ was really answering.
939
+
940
+ ### 13.4 A relation name hops, and the document never carries it
941
+
942
+ When the items are the rows of an entity whose provider carries a
943
+ relation table, a member access naming a declared relation records a HOP
944
+ and is lowered, at capture, into the correlated phrase §4's "relation
945
+ navigation" rows spell. What the reader writes is `p.author.name`; what
946
+ the store receives has no member called `author` anywhere in it.
947
+
948
+ ```js
949
+ import { from } from '@jarenjs/linq';
950
+
951
+ // a store's entity set: its rows carry the entity's relation table
952
+ // (MODEL-FORMAT §10.1), which is what makes a relation name hop
953
+ const postSet = {
954
+ execute: () => [],
955
+ root: '$.Post[*]',
956
+ scope: {
957
+ relations: {
958
+ Post: { author: { to: 'User', kind: 'oneToOne', via: 'authorId', targetKey: 'id' } },
959
+ User: { posts: { to: 'Post', kind: 'oneToMany', via: 'authorId', targetKey: 'id' } },
960
+ },
961
+ },
962
+ relations: { author: { to: 'User', kind: 'oneToOne', via: 'authorId', targetKey: 'id' } },
963
+ };
964
+
965
+ export const bylines = from(postSet)
966
+ .select((p) => ({ title: p.title, author: p.author.name, siblings: p.author.posts.count() }));
967
+ ```
968
+
969
+ ```json
970
+ {
971
+ "$for": { "it": "$.Post[*]" },
972
+ "$return": {
973
+ "title": "$it.title",
974
+ "author": {
975
+ "$for": { "r1": "$.User[*]" },
976
+ "$where": { "$eq": ["$r1.id", "$it.authorId"] },
977
+ "$return": "$r1.name"
978
+ },
979
+ "siblings": {
980
+ "$count": {
981
+ "$for": { "r2": "$.User[*]" },
982
+ "$where": { "$eq": ["$r2.id", "$it.authorId"] },
983
+ "$return": {
984
+ "$for": { "r3": "$.Post[*]" },
985
+ "$where": { "$eq": ["$r3.authorId", "$r2.id"] },
986
+ "$return": "$r3"
987
+ }
988
+ }
989
+ }
990
+ }
991
+ }
992
+ ```
993
+
994
+ Three things this document shows that the source does not:
995
+
996
+ - **The hop bindings are numbered per CAPTURE, not per hop site.** Both
997
+ members are captured by one `select` callback, so the first hop takes
998
+ `r1` and the chained one takes `r2` and `r3`. A second callback — a
999
+ `where` before this `select` — would start again at `r1` in its own
1000
+ phrase.
1001
+ - **`kind` decides which side of the equality carries the key.** The
1002
+ to-one hop compares the TARGET's key with the row's foreign key
1003
+ (`$r1.id` against `$it.authorId`); the to-many hop inside it compares
1004
+ the target's foreign key with the row's key (`$r3.authorId` against
1005
+ `$r2.id`).
1006
+ - **A chained hop re-binds its source.** `p.author.posts` is not one
1007
+ phrase with two roots; it is a phrase inside a phrase, the inner one
1008
+ correlated with the outer's binding. The scope's `relations` is what
1009
+ lets the second link find `User`'s table — without it the first hop
1010
+ lowers and `posts` would be an ordinary member of the target.
1011
+
1012
+ `p.author` and `p.author.posts` are declarations of intent, not
1013
+ instructions: `explain().hops` lists what the callbacks navigated, and
1014
+ no lowered shape pushes natively in this version — a store runs the
1015
+ phrase as a named residual and says so (§8).
1016
+
1017
+ ### 13.5 A parameter is a seam, not a value
1018
+
1019
+ `.params()` DECLARES and BINDS in one call. The declaration is what the
1020
+ document carries — `$tenantId`, an external (QUERY-FORMAT §9) — and the
1021
+ binding is what the runner is handed beside it. The same document serves
1022
+ every tenant, which is what makes it cacheable, loggable and pushable to
1023
+ a provider as a prepared statement.
1024
+
1025
+ ```js
1026
+ import { from } from '@jarenjs/linq';
1027
+
1028
+ const orders = [{ id: 1, tenant: 'a7', total: 12 }];
1029
+
1030
+ export const ours = from(orders)
1031
+ .params({ tenantId: 'a7' })
1032
+ .where((r, p) => r.tenant.eq(p.tenantId));
1033
+ ```
1034
+
1035
+ ```json
1036
+ {
1037
+ "$for": { "it": ["$[*]"] },
1038
+ "$where": { "$eq": ["$it.tenant", "$tenantId"] },
1039
+ "$return": "$it"
1040
+ }
1041
+ ```
1042
+
1043
+ `ours.explain().externals` is `['tenantId']` and `explain().bindings` is
1044
+ `{ tenantId: 'a7' }` — the two halves the seam keeps apart. Reading an
1045
+ undeclared name is `JL0004` at build time rather than `JQ0005` at compile
1046
+ time (§14), and `.params({ tenantId: 'b3' })` on the result re-runs the
1047
+ same document under the new value.
1048
+
1049
+ ### 13.6 `selectMany` unpacks exactly one level
1050
+
1051
+ The projected value is iterated once — an array member's elements, a
1052
+ constructed array's members, a scalar as itself — and the FLWOR `$return`
1053
+ concatenates per tuple. That is a nested `$for` whose binding legally
1054
+ shadows the outer `it`.
1055
+
1056
+ ```js
1057
+ import { from } from '@jarenjs/linq';
1058
+
1059
+ const posts = [{ id: 1, tags: ['linq', 'json'] }, { id: 2, tags: [] }];
1060
+
1061
+ export const tags = from(posts).selectMany((p) => p.tags);
1062
+ ```
1063
+
1064
+ ```json
1065
+ {
1066
+ "$for": { "it": ["$[*]"] },
1067
+ "$return": {
1068
+ "$for": { "it": "$it.tags" },
1069
+ "$return": "$it"
1070
+ }
1071
+ }
1072
+ ```
1073
+
1074
+ `tags.toArray()` answers `["linq", "json"]`: the second post contributes
1075
+ nothing, and an array of arrays would come back as an array of arrays —
1076
+ one level, never a deep flatten.
1077
+
1078
+ ### 13.7 `ofType` is a `$valid` filter, and it needs a compiler
1079
+
1080
+ `ofType` emits a `$valid` over a JSON Schema literal and `cast` an
1081
+ `$assert` per item. Both are SCHEMA operators, and the query engine
1082
+ compiles a schema operator only when a type-test compiler is injected —
1083
+ so the option travels with the source, not with the operator.
1084
+
1085
+ ```js
1086
+ import { from } from '@jarenjs/linq';
1087
+ import { createTypeTestCompiler } from '@jarenjs/validate/query';
1088
+
1089
+ const users = [{ id: 1, email: 'ada@example.com' }, { id: 2 }];
1090
+
1091
+ export const reachable = from(users, { compileTypeTest: createTypeTestCompiler() })
1092
+ .ofType({ type: 'object', required: ['email'] });
1093
+ ```
1094
+
1095
+ ```json
1096
+ {
1097
+ "$for": { "it": ["$[*]"] },
1098
+ "$where": { "$valid": ["$it", { "type": "object", "required": ["email"] }] },
1099
+ "$return": "$it"
1100
+ }
1101
+ ```
1102
+
1103
+ The document is the same with or without the hook — emission never needs
1104
+ it. What needs it is running: `from(users).ofType(…).toArray()` is
1105
+ `JL0003` with the fix in the message (§14), and a schema-pen builder may
1106
+ stand in for the literal (`s.object({ email: s.string() })`), whose
1107
+ document is taken.
1108
+
1109
+ ### 13.8 A hand-written document is a source of items
1110
+
1111
+ `fromDocument` attaches a stored or hand-written query document; its
1112
+ result is the item sequence, and every operator chains over it. Only the
1113
+ envelope this version knows is unwrapped — anything else is handed to the
1114
+ engine so ITS verdict is what surfaces (§14).
1115
+
1116
+ ```js
1117
+ import { fromDocument } from '@jarenjs/linq';
1118
+
1119
+ const users = [{ id: 1, name: 'Ada', age: 36 }];
1120
+ const saved = {
1121
+ $query: '0.1',
1122
+ $expr: { $for: { it: ['$[*]'] }, $where: { $gt: ['$it.age', 21] }, $return: '$it' },
1123
+ };
1124
+
1125
+ export const names = fromDocument(users, saved).select((u) => u.name);
1126
+ ```
1127
+
1128
+ ```json
1129
+ {
1130
+ "$for": {
1131
+ "it": [
1132
+ {
1133
+ "$for": { "it": ["$[*]"] },
1134
+ "$where": { "$gt": ["$it.age", 21] },
1135
+ "$return": "$it"
1136
+ }
1137
+ ]
1138
+ },
1139
+ "$return": "$it.name"
1140
+ }
1141
+ ```
1142
+
1143
+ `names.toArray()` answers `["Ada"]`. The saved document became the source
1144
+ of a new phrase rather than being merged into one — which is what keeps a
1145
+ document a reader did not write from being reinterpreted. A
1146
+ `fromDocument` chain never hops (§3): there the document decides what the
1147
+ items are, so a relation table has nothing to attach to.
1148
+
1149
+ ## 14. Refusals
1150
+
1151
+ §9 is the normative code table: every `JL` code this package can raise,
1152
+ held equal to the runtime's `LINQ_CODES` by
1153
+ `test/errors/code-tables.test.js`. This section is the other half a
1154
+ reader needs — the SPELLING that trips each one and the spelling that
1155
+ works.
1156
+
1157
+ The chain raises fourteen of the twenty: `JL0001`–`JL0007` at build
1158
+ time, `JL2001`–`JL2006` while a terminal runs, and `JL0105`, which sits
1159
+ in the `JL01xx` block because a relation hop is a pen-shaped refusal but
1160
+ is raised by the chain's own expression capture. The other six —
1161
+ `JL0101`–`JL0104`, `JL0106` and `JL0107` — are the PENS' and the
1162
+ CLIENT's. Their per-code conditions are the binder's,
1163
+ [LINQ-FORMAT.md](LINQ-FORMAT.md) §1.3, and the spelling that trips each
1164
+ one is in §4 of the document of the pen that raises it:
1165
+ [SCHEMA-PEN.md](SCHEMA-PEN.md#4-refusals),
1166
+ [MODEL-PEN.md](MODEL-PEN.md#4-refusals),
1167
+ [JSLT-PEN.md](JSLT-PEN.md#4-refusals),
1168
+ [MIGRATION-PEN.md](MIGRATION-PEN.md#4-refusals),
1169
+ [CONTRACT-PEN.md](CONTRACT-PEN.md#4-refusals),
1170
+ [FLOW-PEN.md](FLOW-PEN.md#4-refusals),
1171
+ [APP-PEN.md](APP-PEN.md#4-refusals),
1172
+ [FORMS-PEN.md](FORMS-PEN.md#4-refusals), and
1173
+ [DB-CLIENT.md](DB-CLIENT.md#4-refusals) for the client.
1174
+
1175
+ `test/linq/pen-docs.test.js` holds the list below equal, in both
1176
+ directions, to the codes thrown by the chain's own modules —
1177
+ `packages/linq/src/*.js` less `json-boundary.js` and `capture-root.js`,
1178
+ which are the pens' shared doors and raise only `JL01xx` (the gate
1179
+ proves that too, so the exclusion cannot hide a chain refusal).
1180
+
1181
+ | Code | What the chain raises it for |
1182
+ |---|---|
1183
+ | `JL0001` | `from()`/`fromAsync()` received a source that is neither a supported shape nor a provider |
1184
+ | `JL0002` | an expression proxy was used outside the capture it belongs to |
1185
+ | `JL0003` | `ofType`/`cast` compiled a schema operator with no type-test compiler injected |
1186
+ | `JL0004` | a parameter was read undeclared, declared under a reserved or invalid name, bound to a non-JSON value, or bound to two values by one join |
1187
+ | `JL0005` | an operator was used invalidly at build time: a value the document cannot carry, a stage in the wrong place, a bad argument, an async-surface rule |
1188
+ | `JL0006` | an operator §4 records as `unsupported` was invoked |
1189
+ | `JL0007` | a provider serves several entity roots and has none of its own |
1190
+ | `JL0105` | a relation hop cannot lower to a phrase |
1191
+ | `JL2001` | `first`/`single`/`last`, or `average`/`min`/`max`, over an empty sequence |
1192
+ | `JL2002` | `single`/`singleOrDefault` over two or more elements |
1193
+ | `JL2003` | `elementAt` out of range |
1194
+ | `JL2004` | a provider's `execute()` answered a promise on the synchronous surface |
1195
+ | `JL2005` | a push queue was fed after `end()` |
1196
+ | `JL2006` | a provider answered an element terminal with something other than one array |
1197
+ | `JL2008` | a federated fetch reached one side's row or byte budget (§12.1) |
1198
+
1199
+ `JL2007` is the client door's, not the chain's: `createDbLedger`'s stale
1200
+ settlement ([DB-CLIENT.md §2.6](DB-CLIENT.md#26-the-ledger)); it is
1201
+ listed with the runtime errors above and raised by no chain module.
1202
+
1203
+ Every message below is the one the chain raised when the spelling beside
1204
+ it was run, with the code prefix (`JL0005: `) removed. Where a refusal
1205
+ carries a `docPath`, it is appended to the message text as well
1206
+ (`… at /0/$where/$valid`).
1207
+
1208
+ ### 14.1 `JL0001` — the source
1209
+
1210
+ Dispatch happens ONCE, at `from()`/`fromAsync()` time, never at
1211
+ enumeration time: an `execute` duck is a provider and is never
1212
+ enumerated locally, any iterable gets the in-memory reference semantics,
1213
+ and anything else is refused before a single row is read.
1214
+
1215
+ | The spelling that trips it | The message | The spelling that works |
1216
+ |---|---|---|
1217
+ | `from(42)` | `from() needs an iterable or a provider exposing execute(document, options)` | an array, a string, a `Set`, a generator, or a provider |
1218
+ | `fromAsync(42)` | `fromAsync() needs an async iterable, an iterable, a cursor ({ next, return? }) or a push queue` | one of the five shapes §12 lists |
1219
+ | `fromAsync('abc')` | the same message | a string is a CHUNK on the async surface, not a character stream — feed it through `createPushQueue()`. `from('abc')` iterates characters, and the twins differ here by design (§12) |
1220
+
1221
+ ### 14.2 `JL0002` — the proxy left its capture
1222
+
1223
+ A recording proxy belongs to exactly ONE capture. The two conditions read
1224
+ alike and mean different things, so they carry different messages.
1225
+
1226
+ | The spelling that trips it | The message | The spelling that works |
1227
+ |---|---|---|
1228
+ | `let saved; from(rows).where((u) => { saved = u; return u.id.gt(0); }); from(rows).where(() => saved.id.gt(0))` | `an expression proxy escaped its capture callback; expressions cannot be stored and replayed across operators` | capture in the callback that uses it — the document would otherwise reference a binding this phrase does not have |
1229
+ | `from(rows).select((u) => ({ n: from(rows).where((v) => v.id.eq(u.id)).count() }))` | `an expression proxy of an enclosing capture was used inside a nested capture — a correlated subquery cannot be spelled this way (the inner document rebinds the item); compute the inner query first and use its result` | compute the inner query first, or — over a provider with a relation table — write the hop (§13.4), which is what a correlated phrase is |
1230
+
1231
+ Captures NEST legally: a chain built and run inside a callback
1232
+ (`select((u) => ({ n: from(other).count() }))`) is ordinary, because it
1233
+ touches none of the enclosing proxies. `===` between proxies is
1234
+ untrappable and therefore undetectable; do not compare proxies.
1235
+
1236
+ ### 14.3 `JL0003` — the schema operators need a compiler
1237
+
1238
+ `ofType` and `cast` emit `$valid` and `$assert`, and the query engine
1239
+ compiles those only against an injected type-test compiler. The refusal
1240
+ arrives when the document COMPILES, not when it is emitted (§13.7), and
1241
+ it carries the `docPath` of the operator that needed it.
1242
+
1243
+ | The spelling that trips it | The message | The spelling that works |
1244
+ |---|---|---|
1245
+ | `from(rows).ofType({ type: 'object' }).toArray()` | `ofType/cast compile schema operators, which need a type-test compiler — pass options.compileTypeTest to from()/fromDocument() (e.g. createTypeTestCompiler() from @jarenjs/validate/query)` — `docPath` `/0/$where/$valid` | `from(rows, { compileTypeTest: createTypeTestCompiler() })` |
1246
+ | `from(rows).cast({ type: 'object' }).toArray()` | the same message — `docPath` `/0/$return/$assert` | the same option |
1247
+
1248
+ The fix is one option on the source, and it is the same fix whether the
1249
+ operator came from `ofType`/`cast` or from a hand-written document: the
1250
+ engine's own `JQ0008` is re-reported under this code for that reason
1251
+ (§9).
1252
+
1253
+ ### 14.4 `JL0004` — the parameters
1254
+
1255
+ `.params({ … })` declares AND binds. Everything that can go wrong with a
1256
+ name or a value is one code, because the reader's next action is the same
1257
+ in every case: fix the `params()` call.
1258
+
1259
+ | The spelling that trips it | The message | The spelling that works |
1260
+ |---|---|---|
1261
+ | `from(rows).where((r, p) => r.tenant.eq(p.tenantId))` | `parameter 'tenantId' is not declared — declare it first: .params({ tenantId: value })` | declare it, as the message spells |
1262
+ | `from(rows).params(42)` | `params takes an object of name → value bindings` | an object literal |
1263
+ | `from(rows).params({ 'a-b': 1 })` | `'a-b' is not a valid parameter name` | an identifier: letters, digits and `_`, not starting with a digit |
1264
+ | `from(rows).params({ it: 1 })` | `'it' is reserved (the emitted document's own binding names: it, it2, acc, g, and r1, r2, … for relation hops)` | any other name |
1265
+ | `from(rows).params({ r1: 1 })` | the same message | `r`-plus-digits is reserved for the bindings a hop allocates (§13.4) |
1266
+ | `from(rows).params({ d: new Date() })` | `parameter 'd' is bound to a Date instance, which is not query data — convert it first (a Date to its ISO string or epoch number, a Map to an object, NaN or -0 to a number)` | `d: date.toISOString()` |
1267
+ | `from(rows).params({ z: -0 })` | `parameter 'z' is bound to -0, which is not query data — …` (the same tail) | `0`, or negate at query time — and note the message names `-0`, not the `0` its JSON text would suggest |
1268
+ | `a.params({ k: 1 }).join(b.params({ k: 2 }), …)` | `parameter 'k' is bound to different values by the two sides of join — one document carries one binding per name; bind it once, or rename one side` | bind it once on the outer side, or rename one |
1269
+
1270
+ A binding is not a captured constant: it becomes an external, and later a
1271
+ bound SQL parameter. A `Date` there would compare against nothing and
1272
+ answer `[]` with no error anywhere — which is why the check is at
1273
+ `params()` time and not at the boundary.
1274
+
1275
+ ### 14.5 `JL0005` — the build-time catch-all
1276
+
1277
+ The widest code the chain has, and deliberately one code: every condition
1278
+ under it is a defect in the chain as WRITTEN, found before anything runs.
1279
+ They group into four families.
1280
+
1281
+ **A value the document cannot carry.** The query data model is JSON
1282
+ (§5).
1283
+
1284
+ | The spelling that trips it | The message | The spelling that works |
1285
+ |---|---|---|
1286
+ | `select(() => NaN)` | `a captured expression cannot embed NaN — the query data model is JSON, which has no NaN or Infinity, and lenient serialization would fold it into null` | a finite number |
1287
+ | `select(() => Infinity)` | the same message, naming `Infinity` | a finite number, or a bound |
1288
+ | `select(() => -0)` | `a captured expression cannot embed -0 — it shares its JSON text with 0 while dividing to the opposite infinity, so a document holding it cannot be keyed, stored or compared faithfully; use 0, or negate at query time` | `0` |
1289
+ | `where((u) => u.at.eq(new Date()))` | `a captured expression cannot embed a Date instance — it carries no own enumerable members, so it would embed as {}. Convert it to query data first (a Date to its ISO string or epoch number, a Map to an object), or bind it through params().` | the ISO string, or `.params({ when })` |
1290
+ | `select(() => new Map())` | the same message, naming `Map` | a plain object |
1291
+ | `select(() => MyArray.from([1]))` | `a captured expression cannot embed an Array subclass instance — its behaviour is not expressible as query data` | a plain array |
1292
+ | `select(() => undefined)` | `a captured expression cannot embed an undefined value` | `null`, which IS a value (§5) — a callback that forgot its `return` is the usual cause |
1293
+
1294
+ `-0` is the one nobody guesses, and it is worth the sentence. It is
1295
+ JSON-representable by TEXT and not by value: `JSON.stringify(-0)` is
1296
+ `"0"`, so a document holding it round-trips to a different number, while
1297
+ `1 / -0` is `-Infinity` and `1 / 0` is `+Infinity`. A key built from it
1298
+ would not match itself, a stored document would not compare equal to the
1299
+ one that was written, and a cached compilation keyed by the document's
1300
+ text would serve the `0` query for the `-0` one. There is no spelling
1301
+ that preserves it, so there is no spelling that is allowed to.
1302
+
1303
+ **A stage in the wrong place, or an argument that is not one.**
1304
+
1305
+ | The spelling that trips it | The message | The spelling that works |
1306
+ |---|---|---|
1307
+ | `from(rows).thenBy((u) => u.id)` | `thenBy/thenByDescending must directly follow orderBy/orderByDescending` | an `orderBy` first — `thenBy` extends that clause, it does not open one |
1308
+ | `select((u) => u.age.add(1).all())` | `all() fans out a PATH ('$it.tags[*]'); it cannot follow an operator result` | `all()` on the path, then the operator |
1309
+ | `select((u) => u.posts.title)` (a to-many hop) | `.title is read off a to-many relation, which holds an array of related rows — fan them first (.all().title), index one (.at(0)), or aggregate the array` | `u.posts.all().title`, `u.posts.at(0).title` |
1310
+ | `from(rows).skip(-1)` | `skip takes a non-negative integer, got -1` | a non-negative integer |
1311
+ | `from(rows).take(1.5)` | `take takes a non-negative integer, got 1.5` | an integer |
1312
+ | `from(rows).where(42)` | `this operator takes a callback function` | a callback |
1313
+ | `from(rows).join([], …)` | `join takes another sequence as its inner side` | `from(sameSource)` |
1314
+ | `from(a).join(from(b), …)` | `join's other side must derive from the same source, or from two providers sharing one scope (one store's entity sets) — a query document reads one input; load both collections under one root, or join two entity sets of one store` | one source, or one store's two entity sets (§13.2) |
1315
+ | `from(rows).concat(42)` | `concat takes a sequence or a constant array` | a sequence over the same source, or an array |
1316
+ | `select((r) => r.v.all().rolling(spec))` where `spec` reads the row | `rolling() takes a plain literal spec object; it is read once when the query compiles, so it cannot be an expression or carry a captured value` | a literal spec |
1317
+
1318
+ **A provider or a document that is not shaped as the contract says.**
1319
+
1320
+ | The spelling that trips it | The message | The spelling that works |
1321
+ |---|---|---|
1322
+ | `from({ execute, root: 7 })` | `a provider's root is a path expression string ('$.Post[*]'), got number` | a path expression, or no `root` at all |
1323
+ | `createPushQueue({ highWaterMark: 0 })` | `highWaterMark must be a positive integer` | a positive integer (1024 by default) |
1324
+
1325
+ A malformed VERSION envelope is not this code: `fromDocument(rows,
1326
+ { $query: '0.2', $expr })` is compiled by the engine first, so the
1327
+ engine's own verdict is what surfaces — `JQ0006: unknown query format
1328
+ version "0.2"` — and a future document is never silently run as a 0.1
1329
+ one. Only a spelling the engine accepts and this version does not
1330
+ reaches `JL0005` (`a version envelope is exactly { $query: '0.1',
1331
+ $expr: … } (QUERY-FORMAT §4.1)`).
1332
+
1333
+ **The async surface's own rules** (§§10–12).
1334
+
1335
+ | The spelling that trips it | The message | The spelling that works |
1336
+ |---|---|---|
1337
+ | `fromAsync(rows).mapAsync(fn, {})` | `mapAsync requires { concurrency: <positive integer> } — an unbounded default is a denial of service waiting for a slow downstream` | `{ concurrency: 8 }` |
1338
+ | `mapAsync(fn, { concurrency: 2, mode: 'x' })` | `mapAsync mode must be one of parallel\|concat\|switch\|exhaust, got 'x'` | one of the four |
1339
+ | `mapAsync(42, { concurrency: 1 })` | `mapAsync takes an async callback` | a callback |
1340
+ | `fromAsync(rows).concat(fromAsync(rows))` | `concat on an async sequence takes a constant array — an async source cannot be re-iterated for a second sequence` | a constant array, or `concat` on the sync surface |
1341
+ | `fromAsync(rows).join(fromAsync(rows), …)` | `join on the async surface is pushed whole to a provider — it needs a provider source and comes before any mapAsync; over an iterable, a cursor or a push queue there is no join, because a single-pass source cannot be read twice (QUERY-PEN.md §10)` | join over a provider, join on the sync surface, or collect the stream first |
1342
+ | `fromAsync(rows).mapAsync(fn, { concurrency: 1 }).toDocument()` | `toDocument() cannot represent mapAsync (a host callback); explain() reports the split` | `explain()`, which reports `{ split: { pushed, residual } }` |
1343
+
1344
+ ### 14.6 `JL0006` — an operator §4 records as `unsupported`
1345
+
1346
+ Two operators, both refused by NAME rather than emulated wrongly. §16
1347
+ carries the reasons.
1348
+
1349
+ | The spelling that trips it | The message | The spelling that works |
1350
+ |---|---|---|
1351
+ | `from(rows).zip(other)` | `zip is unsupported: the query grammar has no positional co-iteration (see QUERY-PEN.md §4)` | there is none — index both sides and join on the index, in host code |
1352
+ | `from(rows).aggregate((acc, it) => …)` | `aggregate(fn) is unsupported: JSON cannot spell the implicit first element as a lambda seed — pass a seed, aggregate(seed, fn) (see QUERY-PEN.md §4)` | `aggregate(0, (acc, it) => acc.add(it.n))` |
1353
+
1354
+ ### 14.7 `JL0007` — a provider with several roots and none of its own
1355
+
1356
+ A store is a provider that serves many entity roots. `$[*]` over its
1357
+ entity map would answer every entity's rows mixed together, or count the
1358
+ SETS rather than the rows — an answer that looks like an answer. So the
1359
+ chain refuses at `from()` time and names the roots to chain over.
1360
+
1361
+ | The spelling that trips it | The message | The spelling that works |
1362
+ |---|---|---|
1363
+ | `from(store)` where the store serves `User` and `Post` | `this provider serves entity roots User, Post and has no root of its own — chain over one of them: from(store.entity(name)) (QUERY-PEN.md §8)` | `from(store.sync.entity('User'))` |
1364
+ | `fromAsync(store)` | the same message | `fromAsync(store.entity('User'))` |
1365
+
1366
+ `fromDocument` keeps its own rule and is not refused here: there the
1367
+ document IS the root, so there is nothing to choose.
1368
+
1369
+ ### 14.8 `JL0105` — a hop that cannot lower
1370
+
1371
+ A relation hop is lowered at CAPTURE into the correlated phrase §4
1372
+ spells. Four conditions have no phrase to lower to, and each names what
1373
+ is missing.
1374
+
1375
+ | The spelling that trips it | The message | The spelling that works |
1376
+ |---|---|---|
1377
+ | `u.labels` where `labels` is many-to-many | `'labels' is a many-to-many relation: the join table 'UserLabel' is not a queryable root in this version, so the hop has no phrase to lower to — read the memberships with load({ include: { labels: true } })` | `load({ include: { labels: true } })` through the client ([DB-CLIENT.md](DB-CLIENT.md) §2) |
1378
+ | a relation entry that is not a relation record | `the relation table names 'labels' but its entry is not a relation record ({ to, kind, via, fkEntity, fkTargets, targetKey } — MODEL-FORMAT §10.1)` | a provider whose `relations` is the store's own table |
1379
+ | a relation whose `kind` is neither of the two | `'labels' has relation kind 'oneToNone', which is not one this surface lowers (oneToOne, oneToMany)` | `oneToOne` or `oneToMany` |
1380
+ | a relation over a composite or undeclared key | `'labels' cannot lower: its foreign key or the key it references is composite or undeclared, and the hop's equality would need a tuple the vocabulary does not spell` | a single-column key, or `load({ include })` |
1381
+
1382
+ The first is the one a reader meets: a many-to-many member is exactly
1383
+ the relation that has no direction to correlate in. The hop would need
1384
+ to bind the JOIN TABLE as a root and correlate twice, and a join table
1385
+ is not a queryable root in this version — so there is no phrase, and an
1386
+ honest refusal that names the join table beats a document that quietly
1387
+ reads the wrong rows. `load({ include })` reads the memberships through
1388
+ the client instead, which is the operation the store already has.
1389
+
1390
+ ### 14.9 The runtime codes
1391
+
1392
+ `JL2001`–`JL2006` are raised while a terminal RUNS. The first three are
1393
+ the C# semantics, exactly (§6); the last three are the seam between a
1394
+ terminal and the provider behind it.
1395
+
1396
+ | The spelling that trips it | The message | The spelling that works |
1397
+ |---|---|---|
1398
+ | `from([]).first()` | `first() found no element` | `firstOrDefault()`, which answers `undefined` |
1399
+ | `from([]).single()` | `single() found no element` | `singleOrDefault(d)` |
1400
+ | `from([]).last()` | `last() found no element` | `lastOrDefault(d)` |
1401
+ | `from([]).average()` | `average() of an empty sequence` | guard with `any()`; `sum()` of nothing is `0` and `count()` of nothing is `0` |
1402
+ | `from([]).min()` | `min() of an empty sequence` | as above |
1403
+ | `from([]).max()` | `max() of an empty sequence` | as above |
1404
+ | `from([1, 2]).single()` | `single() found more than one element` | `first()`, or a narrower `where` |
1405
+ | `from([1, 2]).singleOrDefault(0)` | `singleOrDefault() found more than one element` | the default covers EMPTY, never ambiguity |
1406
+ | `from([1]).elementAt(5)` | `elementAt(5) is out of range` | `elementAtOrDefault(5, d)` |
1407
+ | `from(asyncProvider).toArray()` | `this provider's execute() answered a promise, and a Sequence terminal is a value — an asynchronous provider cannot back the synchronous surface. Emit the document with toDocument() and await the provider directly, or use a synchronous provider.` | `fromAsync(provider)` (§12), or `toDocument()` and await |
1408
+ | `q.end(); q.feed(1)` on a push queue | `feed() after end(): the push queue is closed and takes no more values` | feed before `end()`; `end(error)` fails the stream |
1409
+ | a provider answering `toArray()` with `undefined` | `the provider answered toArray() with undefined — an element terminal emits an array constructor, so a conforming execute() answers exactly one array (QUERY-PEN.md §8)` | answer the one array the window constructor yields (`[]` for none) |
1410
+
1411
+ `JL2004` is worth the sentence its message spends. The old seam let the
1412
+ promise through under the value's type: `count()` handed back a `Promise`
1413
+ typed `number`, and `first()` indexed the promise and returned
1414
+ `undefined`. A wrong answer with no error anywhere is worse than a slow
1415
+ one, so the synchronous surface refuses an asynchronous provider by name.
1416
+
1417
+ `JL2006` is the same argument one layer out: an element terminal emits
1418
+ `[window]` precisely so a single array-valued item cannot be confused
1419
+ with several items (§6), so a provider that answers anything but one
1420
+ array is named rather than indexed into a `TypeError`. `count()` and the
1421
+ other scalar terminals are not windowed and are not checked — a provider
1422
+ answering `3` there is answering correctly.
1423
+
1424
+ ## 15. The types
1425
+
1426
+ The declarations are HAND-AUTHORED, in `packages/linq/types/index.d.ts`
1427
+ — chosen over emitting them from JSDoc, so the implementation stays
1428
+ plain JavaScript and this file is the public type contract. The line it
1429
+ holds, stated in the README and at the top of the file:
1430
+
1431
+ > the common path is precisely typed; the exotic path is honestly
1432
+ > `unknown`; nothing is ever a WRONG type.
1433
+
1434
+ Every claim below has two pins. `test/consumer/types.ts` compiles it as
1435
+ a consumer would (`strict`, `skipLibCheck: false`, NodeNext — the chain's
1436
+ block runs from its `@jarenjs/linq` import to the end of the file), and
1437
+ `test/linq/types.test.js` is its runtime twin: the same spelling, asserted
1438
+ to emit and to answer what the type says it does. A claim with only one
1439
+ of the two is half a claim.
1440
+
1441
+ ### 15.1 The recording proxy is a type, not a shape
1442
+
1443
+ §3 describes what a proxy RECORDS; this is what it is declared as. The
1444
+ callback's first argument is `Expr<T>` — a conditional that picks the
1445
+ expression family from the element type, in an order that matters
1446
+ because the `DateTime` brand is a string subtype and must match first:
1447
+
1448
+ | The element is | The proxy is | It carries |
1449
+ |---|---|---|
1450
+ | a `DateTime`-branded string | `DateTimeExpr` | the whole §8.13 date family |
1451
+ | a `string` | `StringExpr` | comparison, the string operators, the spatial family (a geohash is a string) |
1452
+ | a `number` | `NumberExpr` | comparison and arithmetic |
1453
+ | a `boolean` | `BoolExpr` | `and`, `or`, `not` |
1454
+ | an array | `ArrayExpr<E>` | `all()`, `at()`, `count()`, `similarity()`, the §8.16 sequence operators |
1455
+ | an object | `ObjectExpr<T>` | exactly its members, recursively typed |
1456
+ | anything else | `UnknownExpr` | everything, precisely nothing |
1457
+
1458
+ `UnknownExpr` is the honest top and the whole reason the line above can
1459
+ be kept: where inference ends — a dynamic `get(name)`, a member read
1460
+ after an operator, an element the source never declared — the surface
1461
+ widens rather than guesses. `from(users).select((u) => u.get('odd key'))
1462
+ .first()` is `unknown`, and a caller who knows better narrows it
1463
+ themselves.
1464
+
1465
+ A member's type goes through `MemberExpr<V>`, which has one job: an
1466
+ `unknown` (or `any`) member answers `UnknownExpr` rather than the first
1467
+ arm `Expr<>` would otherwise pick for it. An OPTIONAL member is its
1468
+ non-nullable expression — `u.address.city` on `address?: { city: string }`
1469
+ is a `StringExpr`, and the projection's element type is `string` — because
1470
+ absence is a query-time fact (§4: an empty operand compares false,
1471
+ `exists()`/`isEmpty()` say which), not a type-level one.
1472
+
1473
+ Method names shadow member access on the proxy (§3), and the types say
1474
+ so: `u.count` is the aggregate, and the escape `u.get('count')` is
1475
+ declared to answer `UnknownExpr` because a dynamic key cannot be looked
1476
+ up in `T`.
1477
+
1478
+ ### 15.2 The sequence carries two type parameters
1479
+
1480
+ `Sequence<T, P>` and `AsyncSequence<T, P>`: `T` is the element, `P` the
1481
+ parameters declared so far.
1482
+
1483
+ - `select` re-types through `Unwrap<R>` — an expression by its `__value`
1484
+ phantom, an object or array literal recursively, a literal as itself —
1485
+ so `select((u) => ({ id: u.id, name: u.name }))` is
1486
+ `Sequence<{ id: number, name: string }>` with nothing written down.
1487
+ - `selectMany` unwraps and then takes the ELEMENT, one level, exactly as
1488
+ the runtime does (§13.6).
1489
+ - `groupBy` reseats to `Sequence<{ key: K | null, items: T[] }>` — the
1490
+ `| null` is the `$default` the emitted document carries.
1491
+ - `params<Q>(bindings: Q)` answers `Sequence<T, P & Q>`, and every
1492
+ callback's last argument is `ParamsExpr<P>`: exactly the declared
1493
+ names, each typed from its bound value. Reading an undeclared name is
1494
+ a compile error before it is `JL0004` (§14.4).
1495
+ - `mapAsync<R>` crosses to `AsyncSequence<Awaited<R>, P>`: the element
1496
+ becomes the callback's RESOLVED type, and every terminal becomes a
1497
+ promise.
1498
+ - `min()`/`max()` follow the operand family — a sequence of strings
1499
+ answers a string, everything else a number.
1500
+
1501
+ ### 15.3 A document is a document
1502
+
1503
+ `fromDocument<T = unknown>(source, document, options?)` infers NOTHING
1504
+ from the document it is handed: a query document is data, not a type,
1505
+ and there is no honest way to read an element type out of it. It answers
1506
+ `Sequence<unknown>` until the caller states otherwise
1507
+ (`fromDocument<User>(rows, saved)`), and so does a parsed JSON literal
1508
+ handed to `from()` — `from(JSON.parse(text)).toArray()` is `unknown[]`.
1509
+ The pens are the inference route: `ofType`/`cast` given a schema-pen
1510
+ builder re-type the sequence from the builder's own `Infer<>`
1511
+ (`ofType<S>(schema: SchemaBuilder<S, …>): Sequence<S, P>`), and a
1512
+ hand-written schema literal is caller-asserted with `unknown` as the
1513
+ default, because a JSON Schema is not a TypeScript type.
1514
+
1515
+ The same rule runs through the provider seam: `Provider<T>` carries an
1516
+ `__item` phantom, so a typed entity set infers its rows without a cast
1517
+ and an untyped provider is `unknown`.
1518
+
1519
+ ### 15.4 The exports that are not vocabulary
1520
+
1521
+ Five exports are surface a caller meets without ever calling:
1522
+
1523
+ | Export | Why a caller meets it |
1524
+ |---|---|
1525
+ | `Sequence` | to ANNOTATE (`function page(q: Sequence<User>)`). Its constructor is `private`: a sequence is built by `from`/`fromDocument`, never with `new` |
1526
+ | `AsyncSequence` | the same, for the asynchronous surface (`fromAsync`) |
1527
+ | `LinqBuildError` | `instanceof` on the build-time refusals — `code`, `reason` and `docPath` are declared readonly |
1528
+ | `LinqRuntimeError` | `instanceof` on the terminal-time refusals, same three members |
1529
+ | `LINQ_CODES` | the runtime code table §9 is held equal to; a `Readonly<Record<string, string>>` a host can render |
1530
+
1531
+ `DateTime` is a type-only export and costs nothing at run time: it is
1532
+ `string & { __jarenTag: 'date-time' }`, a marker that turns on the date
1533
+ family for a member without making every string a date.
1534
+
1535
+ **A refusal encoded in the types takes a `never` PARAMETER, not just a
1536
+ `never` return.** `zip(unsupported: never): never` makes both
1537
+ `from(rows).zip()` and `from(rows).zip(other)` compile errors. The
1538
+ return type alone does not: `zip(...args: never[])` refuses an argument
1539
+ and accepts none, so the one spelling a caller would actually write
1540
+ type-checked and failed at run time instead. A JavaScript caller still
1541
+ gets the coded refusal (`JL0006`, §14.6) — the encoding closes the
1542
+ TypeScript half, and `test/consumer/types.ts` pins both spellings with
1543
+ `@ts-expect-error`.
1544
+
1545
+ ## 16. What it cannot spell
1546
+
1547
+ §4's table records three constructs as `unsupported` — a status that
1548
+ means "throws a coded error naming the reason", never "emits something
1549
+ close". This section gathers them with their reasons, and adds the
1550
+ boundaries the package draws on purpose, so a reader can check each one
1551
+ rather than discover it.
1552
+
1553
+ ### 16.1 The three refused operators
1554
+
1555
+ | Construct | Why there is no emission | What it raises |
1556
+ |---|---|---|
1557
+ | `aggregate(fn)`, unseeded | C#'s unseeded overload means "the first element is the seed". A query document is data: there is no clause that says "start from whichever item comes first", and inventing one would make the document mean something the grammar does not define | `JL0006`, naming the seeded form |
1558
+ | `zip()` | positional co-iteration — pair the *n*th of one input with the *n*th of another — has no operator in the grammar, and a FLWOR phrase reads ONE input. Emulating it would mean materialising both sides in the host, which is exactly the "runs somewhere other than the document says" the chain exists to avoid | `JL0006` |
1559
+ | a many-to-many hop (`u.labels`) | the phrase would have to bind the JOIN TABLE as a root and correlate twice, and a join table is not a queryable root in this version | `JL0105`, naming the join table and pointing at `load({ include })` |
1560
+
1561
+ All three are checked in both directions: `test/linq/pen-docs.test.js`
1562
+ holds §14's code list equal to what the chain's modules throw, and the
1563
+ spellings above are the ones §14 shows raising them.
1564
+
1565
+ ### 16.2 The operators that are not on the surface at all
1566
+
1567
+ The C# operator set is larger than the query grammar, and the chain does
1568
+ not carry a method for an operator it cannot lower. `union`,
1569
+ `intersect`, `except`, `skipWhile`, `takeWhile`, `chunk`, `append`,
1570
+ `prepend`, `sequenceEqual`, `toDictionary` and `toLookup` are not
1571
+ declared and not defined — reaching for one is a plain `TypeError`, not a
1572
+ coded refusal, because there is no method to refuse from.
1573
+
1574
+ Two of them are compositions a reader can write today, and they are
1575
+ worth naming because the absence otherwise reads as a gap:
1576
+
1577
+ - **`Union`** is `.concat(other).distinct()` — `$seq` followed by
1578
+ `$distinct`, whose equality is the grammar's deep structural one.
1579
+ - **`Append`** is `.concat([value])`: a constant array's elements join
1580
+ the stream. There is no general `Prepend`, because `concat` appends;
1581
+ starting from the single-element source and concatenating the rest
1582
+ (`from([first]).concat(rest)`) works only when `rest` is a constant
1583
+ array.
1584
+
1585
+ The rest have no composition on this surface. `skipWhile`/`takeWhile`
1586
+ need a predicate-terminated window and `$subsequence` takes positions;
1587
+ `chunk` needs a windowing operator; `sequenceEqual`, `toDictionary` and
1588
+ `toLookup` are host-side shapes rather than query results — read the
1589
+ sequence and build them.
1590
+
1591
+ ### 16.3 The boundaries this package draws on purpose
1592
+
1593
+ Each of these is a design commitment, checkable in the source:
1594
+
1595
+ - **No `Function.prototype.toString`, anywhere.** A callback is executed
1596
+ ONCE against recording proxies; nothing parses its text. `grep -rn
1597
+ 'toString()' packages/linq/src/` finds none — the only `toString` in
1598
+ the package is a radix conversion escaping a control character in a
1599
+ path segment.
1600
+ - **No per-element callback evaluation.** A predicate runs at BUILD
1601
+ time, produces an expression, and the engine evaluates that expression
1602
+ per row. This is why a JavaScript operator inside a callback is a trap
1603
+ rather than a slow path (§3): `&&`, `||`, `!`, `?:`, `in`, `typeof`,
1604
+ `Object.keys` and `===` evaluate against the proxy and yield a
1605
+ silently wrong document, while `>` and `+` throw a plain `TypeError`.
1606
+ Use `.and()`, `.or()`, `.not()` and the comparison methods.
1607
+ - **No second grammar.** The chain emits the published query language
1608
+ and nothing else; `toDocument()` is compilable by a bare
1609
+ `compileJsonQuery` with no linq involvement, which is what makes a
1610
+ query loggable, storable, diffable and authorable by a constrained
1611
+ decoder.
1612
+ - **No inference from a document.** `fromDocument` and a parsed JSON
1613
+ literal answer `unknown` (§15.3). The pens are the inference route.
1614
+ - **No clock.** There is no `now()`: §8.13 has no clock operator, and a
1615
+ fluent surface does not get to add one. Bind the instant with
1616
+ `.params({ now })`.
1617
+ - **No `knn` method.** k-nearest is `orderByDescending(… similarity …)`
1618
+ then `take(k)` — the composition the emitted document already is (§4).
1619
+ - **No async query engine.** `packages/json` is strictly synchronous.
1620
+ `fromAsync` makes the SOURCE and the host boundary asynchronous and
1621
+ emits byte-identical documents (§10); there are no `selectAwait` or
1622
+ `whereAwait` variants, because a per-element async predicate is
1623
+ `mapAsync` then `where` (§11).
1624
+ - **No correlated subquery through a captured proxy.** An enclosing
1625
+ capture's proxy used inside a nested one is `JL0002` (§14.2) — the
1626
+ inner document rebinds `$it`. Over a provider with a relation table
1627
+ the correlated phrase has a spelling: the hop (§13.4).
1628
+ - **No non-JSON constant.** A `Date`, `Map`, `Set`, `RegExp`, class
1629
+ instance, `NaN`, `±Infinity` or `-0` in a captured expression is
1630
+ `JL0005`, and in a `params()` binding `JL0004` (§14.4, §14.5). The
1631
+ query data model is JSON, and a value that cannot survive the
1632
+ round-trip cannot be compared faithfully.
1633
+ - **No document form for a host callback.** A chain carrying `mapAsync`
1634
+ has no `toDocument()`; `explain()` reports `{ split: { pushed,
1635
+ residual } }` instead (§11). The split is stated rather than hidden,
1636
+ which is the same honesty a SQL pushdown owes its residual.
1637
+
1638
+ ### 16.4 When not to reach for the chain
1639
+
1640
+ The chain earns its place when a query has to TRAVEL — to a store, into a
1641
+ saved document, across a version. Where it does not, the honest answers
1642
+ are shorter:
1643
+
1644
+ - **The data is in memory and the query stays there.** `rows.filter()`
1645
+ and `rows.map()` are the language's own, need no import, and any
1646
+ JavaScript reader can follow them. A chain over an array buys one
1647
+ thing: a document you could have sent somewhere. If you are not going
1648
+ to send it, you are paying for a capture you never read.
1649
+ - **The query is one statement of SQL you already know.** A store takes
1650
+ raw statements. A reporting query with three joins and a window
1651
+ function is a statement; expressing it as a chain either does not
1652
+ translate (§4 records every such gap) or translates into something
1653
+ nobody can review against the original.
1654
+ - **The document already exists.** A saved `$query` is run with
1655
+ `fromDocument` (§13.8) or handed to the engine directly. Re-authoring
1656
+ it through the chain to "keep it typed" makes two spellings of one
1657
+ query, and the one that runs in production is whichever the deploy
1658
+ picked.
1659
+ - **The predicate needs JavaScript.** A callback runs ONCE, at build
1660
+ time, against a proxy — so `if`, `&&`, a loop, a call into a library
1661
+ and a closure over a mutable variable all either throw or record
1662
+ something you did not mean (§3). A predicate that genuinely needs the
1663
+ language is `mapAsync`'s host boundary (§11), and a chain that is
1664
+ mostly host boundary is a program with a `where` at the front.
1665
+ - **You want a type, not a query.** `ofType` and `cast` narrow a
1666
+ sequence's element type; neither validates unless a compiler was
1667
+ handed in (§13.7, §14.3). A chain reached for as a type assertion is a
1668
+ cast with extra steps — `from(rows)` already answers
1669
+ `Sequence<unknown>` and `as` is the language's own spelling.
1670
+
1671
+ ## 17. Cost
1672
+
1673
+ A consumer importing `from` from `@jarenjs/linq` and calling one
1674
+ terminal bundles **<!--fact:bundle.chain-->174,264<!--/fact--> bytes** (esbuild, ESM, minified, tree-shaken,
1675
+ `platform: 'neutral'`). The figure is measured by
1676
+ `scripts/check-tree-shaking.js`'s chain probe and compared with this
1677
+ section on every `npm run test:tree-shaking`: it is derived, never typed,
1678
+ and a stale one is red here rather than wrong in a document somebody
1679
+ reads.
1680
+
1681
+ Of that, **<!--fact:bundle.chain.own-->40,280<!--/fact--> bytes** are the chain's own modules — `sequence.js`,
1682
+ `async.js`, `expression.js`, `document.js`, `provider.js`,
1683
+ `concurrency.js`, `errors.js` and `schema-of.js`. The remaining ~134 kB
1684
+ is the query ENGINE and the core it stands on: a chain's document has to
1685
+ run somewhere, and the in-memory runner is the reference semantics every
1686
+ provider is measured against (§8). A consumer that only ever hands
1687
+ `toDocument()` to a provider still pays it today, because the terminal
1688
+ that emits the document is the same terminal that would run it.
1689
+
1690
+ The probe asserts four exclusions, and they are the cost claims worth
1691
+ making:
1692
+
1693
+ - **no schema-pen module** — `ofType`/`cast` reach a builder through a
1694
+ registry symbol looked up by key (`schema-of.js`), so the chain
1695
+ imports nothing from `src/schema/`;
1696
+ - **no client module** — `src/db/` is the package's one runtime edge and
1697
+ is not on this path;
1698
+ - **not one byte of `@jarenjs/db`, `@jarenjs/validate` or
1699
+ `@jarenjs/formats`** — the client's optional peers. A consumer of the
1700
+ chain alone installs nothing new;
1701
+ - **no pen bytes at all**, in either direction: the pens carry no chain
1702
+ module either, which is what keeps a <!--fact:bundle.jslt.kb-->20<!--/fact--> kB JSLT
1703
+ pen <!--fact:bundle.jslt.kb-->20<!--/fact--> kB.
1704
+
1705
+ `docs/CONSUMING.md` states the rounded price of all ten subpaths in one
1706
+ table, each figure held equal to the same measurements. Two of its rows
1707
+ are the ones to read together: the chain at <!--fact:bundle.chain.kb-->174<!--/fact--> kB and
1708
+ `./db` at <!--fact:bundle.db.kb-->592<!--/fact--> kB.
1709
+ The client costs what the store costs, by construction, and the chain
1710
+ costs what running a query costs.
1711
+
1712
+ **Taking a pen as well costs less than the two figures suggest**, and
1713
+ the reason is worth knowing: a bundler counts a shared module once, and
1714
+ the chain and every pen share the expression capture (`expression.js`)
1715
+ and the coded errors under it (`errors.js`, and `@jarenjs/core`'s error
1716
+ and object helpers). A consumer importing the chain AND the schema pen
1717
+ bundles **<!--fact:bundle.chain.withSchemaPen-->195,342<!--/fact--> bytes** — **<!--fact:bundle.chain.shared-->12,078<!--/fact--> bytes** less than the sum of the
1718
+ figure above and [SCHEMA-PEN.md](SCHEMA-PEN.md#7-cost) §7's, which is
1719
+ what those shared modules weigh. The probe measures that pair too, so
1720
+ the saving is derived like everything else here. What the chain does NOT
1721
+ share with a pen is the pens' own two shared doors, `capture-root.js`
1722
+ and `json-boundary.js`: no chain callback reaches either, and neither is
1723
+ in the figure above. Every pen document's §7 carries its own
1724
+ subpath's figure; nothing here restates one.