@jarenjs/json 0.43.3 → 0.46.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/ARCHITECTURE.md CHANGED
@@ -163,7 +163,7 @@ Grouping, `$distinct` and the `$orderby` machinery need a total, deterministic e
163
163
 
164
164
  ### The operator registry
165
165
 
166
- All 98 §8 operators live in one table in `operators.js` — the query-language analogue of `path.js`'s `FUNCTIONS` table. A test derives the count from the registry and asserts it against QUERY-FORMAT §8 *and* against every committed document that states a number — this file, the README twice, `site.md` and `@jarenjs/linq`'s architecture — so an operator cannot be added without moving all of them in the same change:
166
+ All 99 §8 operators live in one table in `operators.js` — the query-language analogue of `path.js`'s `FUNCTIONS` table. A test derives the count from the registry and asserts it against QUERY-FORMAT §8 *and* against every committed document that states a number — this file, the README twice, `site.md` and `@jarenjs/linq`'s architecture — so an operator cannot be added without moving all of them in the same change:
167
167
 
168
168
  ```javascript
169
169
  '$substring': {
package/README.md CHANGED
@@ -386,9 +386,9 @@ queryJson({
386
386
  }, [1, 2, 3, 4, 5]); // [2, 3, 4] — a 3-point moving average
387
387
  ```
388
388
 
389
- The operator library (98 operators: comparisons, IEEE-double arithmetic, logic, strings with I-Regexp `$match`/`$search`/`$replace`, aggregates, sequence tools like `$distinct`/`$subsequence`/`$range`, type predicates and casts, `$coalesce`, the RFC 3339 date family, and the spatial family) is cataloged in [QUERY-FORMAT.md §8](./docs/QUERY-FORMAT.md#8-operators).
389
+ The operator library (99 operators: comparisons, IEEE-double arithmetic, logic, strings with I-Regexp `$match`/`$search`/`$replace`, aggregates, sequence tools like `$distinct`/`$subsequence`/`$range`, type predicates and casts, `$coalesce`, the RFC 3339 date family, the spatial family and `$similarity`) is cataloged in [QUERY-FORMAT.md §8](./docs/QUERY-FORMAT.md#8-operators).
390
390
 
391
- **Extending the vocabulary (host opt-in).** The 98 are closed, but a host can add more the way `@jarenjs/validate` gains formats from `@jarenjs/formats`: `createJsltRegistry().use(mathPack).use(financePack)` composes packs of pure `@jarenjs/core` functions into a compiler, so `{ "$sqrt": "$.variance" }` and `{ "$npv": ["$.rate", "$.cashflows[*]"] }` work in a stylesheet, a bare query, and `@jarenjs/linq` — while a document compiled *without* the registry still rejects them. Aggregators fold a `seq` operand to an array before the pure call. See [JSLT-FORMAT.md §13](./docs/JSLT-FORMAT.md#13-registered-operators-host-opt-in-non-normative).
391
+ **Extending the vocabulary (host opt-in).** The 99 are closed, but a host can add more the way `@jarenjs/validate` gains formats from `@jarenjs/formats`: `createJsltRegistry().use(mathPack).use(financePack)` composes packs of pure `@jarenjs/core` functions into a compiler, so `{ "$sqrt": "$.variance" }` and `{ "$npv": ["$.rate", "$.cashflows[*]"] }` work in a stylesheet, a bare query, and `@jarenjs/linq` — while a document compiled *without* the registry still rejects them. Aggregators fold a `seq` operand to an array before the pure call. See [JSLT-FORMAT.md §13](./docs/JSLT-FORMAT.md#13-registered-operators-host-opt-in-non-normative).
392
392
 
393
393
  **Dates are RFC 3339 strings** ([§8.13](./docs/QUERY-FORMAT.md#813-dates-and-times)): `$is-date`/`$is-time`/`$is-datetime`/`$is-duration` test the lexical forms, `$year`…`$seconds` and `$offset` read components *lexically, in the value's own offset* (so "group by month" means what you expect), `$week`/`$week-year`/`$quarter`/`$weekday` add the derived calendar fields, and `$epoch`/`$datetime` convert to and from epoch milliseconds — the one place a value is shifted to UTC, and therefore the way to compare instants across offsets. There is deliberately no `current-dateTime`: a compiled query is cached by document identity and saved as a rule, so it must answer the same for the same input forever.
394
394
 
@@ -406,7 +406,7 @@ queryJson({
406
406
 
407
407
  Bucketing by week is the shape components alone cannot express, because a week boundary is arithmetic rather than a field. Patterns stay locale-independent — `MMMM` and `EEEE` are rejected rather than silently rendered in English — because localized text belongs to the presentation layer, not to a query.
408
408
 
409
- **Geography is GeoJSON** ([§8.14](./docs/QUERY-FORMAT.md#814-spatial)), for the same reason dates are RFC 3339 strings: it is what the document already holds. Operands are a bare `[longitude, latitude]` position, a geometry, a `Feature` or a `FeatureCollection`, and wrappers unwrap for you. `$distance`, `$area` and `$length` answer in metres on the WGS 84 sphere — never planar, because a Euclidean answer over raw degrees is wrong by two thirds over a kilometre at Dutch latitudes. `$within` tests containment, `$bbox`/`$centroid` measure, and `$bbox-intersects` is named for exactly what it tests, since an `$intersects` that compared only boxes would be a lie the first time two L-shapes shared one.
409
+ **Geography is GeoJSON** ([§8.14](./docs/QUERY-FORMAT.md#814-spatial)), for the same reason dates are RFC 3339 strings: it is what the document already holds. Thirteen of the 98 operators are spatial — eight measurements and predicates, five conversions — and JSLT and JTLT inherit every one. Operands are a bare `[longitude, latitude]` position, a geometry, a `Feature` or a `FeatureCollection`, and wrappers unwrap for you. `$distance`, `$area` and `$length` answer in metres on the WGS 84 sphere — never planar, because a Euclidean answer over raw degrees is wrong by two thirds over a kilometre at Dutch latitudes. `$within` tests containment, `$bbox`/`$centroid` measure, and `$bbox-intersects` is named for exactly what it tests, since an `$intersects` that compared only boxes would be a lie the first time two L-shapes shared one.
410
410
 
411
411
  ```js
412
412
  queryJson({
@@ -442,6 +442,45 @@ The one trap: `coordinates` and `features` take the **array constructor** (the b
442
442
 
443
443
  A geohash is a string, so **bucketing and tiling** need no operator: `$groupby` over `$substring` groups by cell and an index over the hash is a spatial index. **Proximity is different**, and a prefix test is not it — two points ten metres apart can differ in the *first* character of their cell, so a single prefix misses a neighbour at every cell boundary. `$geohash-neighbours` gives the nine-cell probe a correct proximity query needs; narrow it with `$distance` when an exact radius matters. The whole recipe, end to end, is in [HOWTO](../../docs/HOWTO.md#getting-geographic-data-in-and-out).
444
444
 
445
+ ### Similarity, and why there is no `$knn`
446
+
447
+ An embedding is an array of numbers, which a JSON document can already hold, so
448
+ the language needs exactly one operator to compare two of them
449
+ ([§8.15](./docs/QUERY-FORMAT.md#815-vectors)):
450
+
451
+ ```javascript
452
+ queryJson({ $similarity: [[3, 4], [4, 3]] }, {}); // 0.96
453
+ ```
454
+
455
+ `$similarity` is cosine, in [-1, 1], higher-is-better. One metric, because a
456
+ stored vector is normally normalized and cosine, the dot product and Euclidean
457
+ distance then rank the same candidates in the same order — a second metric
458
+ would buy a different number for the same answer. An operand that is not an
459
+ array of numbers is `JQ2001`; two vectors that cannot be compared — different
460
+ widths, empty, or carrying a component a computation made non-finite — answer
461
+ **empty**, so a document never carries a score nobody computed. Nothing is
462
+ padded or truncated to make a comparison possible.
463
+
464
+ "The k most similar" needs no keyword either: it is an ordering and a window,
465
+ and both already exist.
466
+
467
+ ```javascript
468
+ queryJson({ $subsequence: [{
469
+ $for: { m: '$.memories[*]' },
470
+ $orderby: [
471
+ { $key: { $similarity: ['$m.embedding', '$query'] }, $dir: 'desc', $empty: 'least' },
472
+ '$m.id',
473
+ ],
474
+ $return: '$m.text',
475
+ }, 0, 10] }, data, { query: vector });
476
+ ```
477
+
478
+ `$empty: 'least'` under a descending sort puts the rows with no vector **last**
479
+ — present in the input, never in the top k, never scored — and the second key
480
+ breaks ties by identity, so the same document answers with the same rows every
481
+ time it runs. `@jarenjs/linq` spells the whole chain fluently as
482
+ `.orderByDescending(m => m.embedding.similarity(q), { empty: 'least' }).take(10)`.
483
+
445
484
  ### External parameters
446
485
 
447
486
  A variable no phrase binds is an **external**: use is the declaration. The compiled query exposes the collected names and takes bindings at call time:
@@ -524,7 +563,7 @@ The validate step is not redundant: provider structured-output implementations s
524
563
 
525
564
  For those strict subsets each grammar also ships an **LLM profile** — [`jaren-query.llm-profile.schema.json`](./schemas/jaren-query.llm-profile.schema.json) and [`jaren-jslt.llm-profile.schema.json`](./schemas/jaren-jslt.llm-profile.schema.json) — a mechanically derived, pure *relaxation* of the canonical artifact: `patternProperties`, `propertyNames` and asserted `format`s are removed (each restated in the nearest `description`, which the model still reads) and every `oneOf` becomes `anyOf`. That last swap costs no precision: the canonical grammar discriminates its branches by member name — a map constructor has no `$`-prefixed key, every phrase pins its own (`$const`, `$map`, `$call`, `$return`, …) or draws its single key from a disjoint operator enum — so at most one branch can ever match a document, and exactly-one and at-least-one accept the same language. It is also *required* once the name constraints are gone: `oneOf` would then reject `{"$eq": [1, 2]}` for matching two now-overlapping branches. Every canonical-valid document validates under the profile; the reverse is deliberately not guaranteed, which is exactly why the local validate-then-compile step above stays mandatory. Hand the profile to the provider's constrained decoder, validate locally against the canonical schema. (`@jarenjs/ai`'s `createStructuredOutput` packages this handshake.)
526
565
 
527
- A relaxation, though, is not a *shrink*: restating every removed constraint in a `description` means the JSLT profile is 19,158 characters against the canonical 18,736, and on a small model an oversized `response_format` is the difference between a document and an empty reply. So the JSLT grammar also ships an **authoring profile** — [`jaren-jslt.authoring.schema.json`](./schemas/jaren-jslt.authoring.schema.json), **3,491 characters** — the same mechanical derivation with one more step: the `queryDocument` `$ref` that pulls the entire expression language into the document grammar is left *open*. What survives is the document shape (envelope, rules, match specs, modes); what leaves is the ~16 kB of phrase shapes. It is a pure widening of the canonical language, so nothing the engine can run becomes inexpressible, and it is deliberately weaker — a body of pure nonsense decodes under it. That is the trade, and it is only safe because the two authorities after the decoder are unchanged: validate against the canonical schema, then compile. The vocabulary the cut removes belongs in the prompt instead, where `@jarenjs/ai`'s `operatorCrib` puts it (operator names grouped by arity, ~1.1 kB, read off the query artifact). `@jarenjs/ai`'s [`createStylesheetAuthor`](../ai/README.md#when-the-grammar-is-too-big-to-decode-measured) packages the whole handshake.
566
+ A relaxation, though, is not a *shrink*: restating every removed constraint in a `description` means the JSLT profile is 19,158 characters against the canonical 18,817, and on a small model an oversized `response_format` is the difference between a document and an empty reply. So the JSLT grammar also ships an **authoring profile** — [`jaren-jslt.authoring.schema.json`](./schemas/jaren-jslt.authoring.schema.json), **3,491 characters** — the same mechanical derivation with one more step: the `queryDocument` `$ref` that pulls the entire expression language into the document grammar is left *open*. What survives is the document shape (envelope, rules, match specs, modes); what leaves is the ~16 kB of phrase shapes. It is a pure widening of the canonical language, so nothing the engine can run becomes inexpressible, and it is deliberately weaker — a body of pure nonsense decodes under it. That is the trade, and it is only safe because the two authorities after the decoder are unchanged: validate against the canonical schema, then compile. The vocabulary the cut removes belongs in the prompt instead, where `@jarenjs/ai`'s `operatorCrib` puts it (operator names grouped by arity, ~1.1 kB, read off the query artifact). `@jarenjs/ai`'s [`createStylesheetAuthor`](../ai/README.md#when-the-grammar-is-too-big-to-decode-measured) packages the whole handshake.
528
567
 
529
568
  Query documents are plain JSON, so they travel through the rest of an LLM toolchain as-is: function-call arguments, retrieval filters, audit logs, replay.
530
569
 
@@ -573,6 +573,14 @@ export declare const OPERATORS: Readonly<{
573
573
  result: typeof resultEmptyPropagates;
574
574
  compile: (gets: any, args: any) => (f: any) => any;
575
575
  };
576
+ $similarity: {
577
+ params: Readonly<{
578
+ kinds: readonly string[];
579
+ min: 2;
580
+ }>;
581
+ result: typeof RESULT_OPT;
582
+ compile: (gets: any, args: any) => (f: any) => number | typeof EMPTY;
583
+ };
576
584
  '$date-add': {
577
585
  params: Readonly<{
578
586
  kinds: readonly string[];
@@ -1385,6 +1385,13 @@ value. This matters wherever such an expression sits in a position that expects
1385
1385
  one item: `[{"$bbox": expr}]` builds an empty array rather than a one-item one,
1386
1386
  and `{"m": {"$bbox": expr}}` omits the member entirely.
1387
1387
 
1388
+ **Traversal is not validation.** `$within`, `$area` and `$length` walk the
1389
+ rings they are given and treat a ring as closed whether or not its last
1390
+ position repeats its first — the reading every implementation makes — so a
1391
+ ring `isValidGeoJson` (or the meta-schema) refuses still measures. A document
1392
+ that needs the judgment composes it: `$valid` against the GeoJSON meta-schema,
1393
+ or the `geojson` format, before the measurement, not inside it.
1394
+
1388
1395
  **Boxes do not cross the antimeridian.** RFC 7946 §3.1.9 tells producers to cut
1389
1396
  geometries at ±180° rather than let them span it, and this format follows that
1390
1397
  rather than re-joining what a producer split: `$bbox` of an uncut geometry whose
@@ -1428,6 +1435,96 @@ the answer.
1428
1435
  selects the cities inside a region and orders them by how far they are from a
1429
1436
  point — a spatial filter and a spatial sort, in the language's own clauses.
1430
1437
 
1438
+ ### 8.15 Vectors
1439
+
1440
+ A vector is an **array of numbers** — what a JSON document already holds when
1441
+ something has been embedded. There is no vector type to construct, no width to
1442
+ declare, and nothing to register: `[0.1, 0.2, 0.3]` is one.
1443
+
1444
+ | Operator | Definition |
1445
+ |---|---|
1446
+ | `$similarity` | `[a, b]` → the cosine similarity of two vectors, a number in [-1, 1]; two vectors that cannot be compared → empty |
1447
+
1448
+ Cosine is the only metric, and that is a decision rather than a first
1449
+ instalment. For vectors of unit length — the form an embedding is normally
1450
+ stored in — cosine, the dot product and Euclidean distance all rank the same
1451
+ candidates in the same order, so a second metric would buy a different *number*
1452
+ for the same *answer* while doubling every recipe below. **Higher is better**:
1453
+ `1` is the same direction, `0` unrelated, `-1` opposite, which is why a
1454
+ similarity sort is always descending.
1455
+
1456
+ **The operand split is §8.14's.** A value that is not an array of numbers has
1457
+ no similarity to anything — a string, an object, an array with a non-number in
1458
+ it — and is `JQ2001`, the same type-level refusal a non-geographic operand
1459
+ gets. Two operands that *are* arrays of numbers but cannot be compared —
1460
+ different widths, an empty array, or a component a computation made non-finite
1461
+ — answer the **empty sequence**, the same data-level answer a measurement with
1462
+ no bounded position gives. Nothing is padded, truncated or zero-filled to make
1463
+ a comparison possible: two vectors of different widths are not a near miss,
1464
+ they are unrelated.
1465
+
1466
+ One degenerate pair is a **score, not a refusal**: a vector of all zeros
1467
+ points nowhere, and it answers `0` against everything, including itself. That
1468
+ is the one number here not derived from a direction, and it is the truthful
1469
+ answer to "how aligned are these" when one of them has no alignment — an
1470
+ all-zero embedding sorts below every real one and above the rows that have no
1471
+ vector at all, which is exactly where it belongs.
1472
+
1473
+ Because of that rule `$similarity` is **optional-valued**: it can answer empty
1474
+ for two operands that are present, so `[{"$similarity": [a, b]}]` builds an
1475
+ empty array rather than a one-item one and `{"score": {"$similarity": [a, b]}}`
1476
+ omits the member entirely. That is the answer to want — a document that carries
1477
+ no score is honest, and one carrying `0` for a comparison that never happened
1478
+ is not, because `0` is a real similarity.
1479
+
1480
+ **Nearest-neighbours is a composition, not an operator.** "The k most similar"
1481
+ is an ordering and a window, and this format already spells both: `$orderby` on
1482
+ a `$similarity` key, descending, then `$subsequence`.
1483
+
1484
+ ```json
1485
+ { "$for": { "m": "$.memories[*]" },
1486
+ "$orderby": [ { "$key": { "$similarity": ["$m.embedding", "$query"] },
1487
+ "$dir": "desc", "$empty": "least" } ],
1488
+ "$return": "$m.text" }
1489
+ ```
1490
+
1491
+ ```json
1492
+ { "$subsequence": [ { "$for": { "m": "$.memories[*]" },
1493
+ "$orderby": [ { "$key": { "$similarity": ["$m.embedding", "$query"] },
1494
+ "$dir": "desc", "$empty": "least" },
1495
+ "$m.id" ],
1496
+ "$return": "$m" },
1497
+ 0, 10 ] }
1498
+ ```
1499
+
1500
+ The first orders every memory by how close it is to an external `$query`
1501
+ vector; the second wraps that ordering in a window and takes the first ten of
1502
+ it, which is the whole of "k nearest". The window goes **outside** the phrase,
1503
+ because `$subsequence` inside `$return` would trim each tuple's own result
1504
+ rather than the stream of tuples.
1505
+
1506
+ Two details in the second document are load-bearing. `"$empty": "least"` places
1507
+ an empty key **last** under `"desc"` (§6.6), which is where a row with no
1508
+ vector, or with one of the wrong width, belongs: present in the input, never in
1509
+ the top k, and never scored. And the second key, `"$m.id"`, breaks ties by row
1510
+ identity — the sort is stable, but stability is only about the *input* order,
1511
+ and two candidates at an identical similarity are exactly what a corpus of
1512
+ near-duplicates produces. An explicit tie-break is what makes the same document
1513
+ answer with the same rows every time it runs.
1514
+
1515
+ A filter is the same key in a `$where`, and it composes with the ordering:
1516
+
1517
+ ```json
1518
+ { "$for": { "m": "$.memories[*]" },
1519
+ "$where": { "$gt": [{ "$similarity": ["$m.embedding", "$query"] }, 0.8] },
1520
+ "$return": "$m.text" }
1521
+ ```
1522
+
1523
+ — everything closer than a threshold, in the language's own clauses. Note that
1524
+ `$gt` over an empty key answers `false`, so an unvectored row drops out of a
1525
+ threshold filter for free.
1526
+
1527
+
1431
1528
  ## 9. Variables, scoping, and external parameters
1432
1529
 
1433
1530
  1. Variables are introduced by `$for`, `$let`, `$at`, `$count`, `$groupby`
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@jarenjs/json",
3
3
  "private": false,
4
- "version": "0.43.3",
4
+ "version": "0.46.4",
5
5
  "type": "module",
6
6
  "main": "./src/index.js",
7
7
  "types": "./dist/types/index.d.ts",
@@ -90,6 +90,6 @@
90
90
  "prepack": "npm run build:types"
91
91
  },
92
92
  "dependencies": {
93
- "@jarenjs/core": "^0.43.3"
93
+ "@jarenjs/core": "^0.46.4"
94
94
  }
95
95
  }
@@ -621,7 +621,11 @@
621
621
  "$bbox",
622
622
  "$area",
623
623
  "$length",
624
- "$centroid"
624
+ "$centroid",
625
+ "$geo-parse",
626
+ "$geo-text",
627
+ "$geohash-bounds",
628
+ "$geohash-neighbours"
625
629
  ]
626
630
  },
627
631
  "additionalProperties": {
@@ -661,6 +665,8 @@
661
665
  "$distance",
662
666
  "$within",
663
667
  "$bbox-intersects",
668
+ "$geo-simplify",
669
+ "$similarity",
664
670
  "$default"
665
671
  ]
666
672
  },
@@ -621,7 +621,11 @@
621
621
  "$bbox",
622
622
  "$area",
623
623
  "$length",
624
- "$centroid"
624
+ "$centroid",
625
+ "$geo-parse",
626
+ "$geo-text",
627
+ "$geohash-bounds",
628
+ "$geohash-neighbours"
625
629
  ]
626
630
  },
627
631
  "additionalProperties": {
@@ -661,6 +665,8 @@
661
665
  "$distance",
662
666
  "$within",
663
667
  "$bbox-intersects",
668
+ "$geo-simplify",
669
+ "$similarity",
664
670
  "$default"
665
671
  ]
666
672
  },
@@ -530,7 +530,11 @@
530
530
  "$bbox",
531
531
  "$area",
532
532
  "$length",
533
- "$centroid"
533
+ "$centroid",
534
+ "$geo-parse",
535
+ "$geo-text",
536
+ "$geohash-bounds",
537
+ "$geohash-neighbours"
534
538
  ]
535
539
  },
536
540
  "additionalProperties": {
@@ -570,6 +574,8 @@
570
574
  "$distance",
571
575
  "$within",
572
576
  "$bbox-intersects",
577
+ "$geo-simplify",
578
+ "$similarity",
573
579
  "$default"
574
580
  ]
575
581
  },
@@ -289,7 +289,8 @@
289
289
  "$year", "$month", "$day", "$hours", "$minutes", "$seconds",
290
290
  "$offset", "$epoch", "$datetime",
291
291
  "$week", "$week-year", "$quarter", "$weekday",
292
- "$bbox", "$area", "$length", "$centroid"
292
+ "$bbox", "$area", "$length", "$centroid",
293
+ "$geo-parse", "$geo-text", "$geohash-bounds", "$geohash-neighbours"
293
294
  ]
294
295
  },
295
296
  "additionalProperties": { "$ref": "#/$defs/expression" }
@@ -306,7 +307,8 @@
306
307
  "$contains", "$starts-with", "$ends-with", "$match", "$search",
307
308
  "$index-of", "$range", "$get",
308
309
  "$start-of", "$end-of", "$date-format",
309
- "$distance", "$within", "$bbox-intersects", "$default"
310
+ "$distance", "$within", "$bbox-intersects", "$geo-simplify",
311
+ "$similarity", "$default"
310
312
  ]
311
313
  },
312
314
  "additionalProperties": {
@@ -417,6 +417,22 @@ const OPERATOR_ALIASES = {
417
417
  $project: "the renderer — the language cannot make a projected coordinate at all, so a measurement can never land on one; measurement here is geodesic",
418
418
  "$geo-project": "the renderer, as for '$project'",
419
419
  $srid: null, $transform: null,
420
+ // vectors: the spellings a writer reaches for from a ranking library.
421
+ // The metric names all point at the one operator because the packed
422
+ // form is normalized and they then rank identically; `$knn` and its
423
+ // synonyms get the composition, because ordering and windowing are
424
+ // clauses this language already has and a second spelling of them
425
+ // would be the drift, not the convenience
426
+ $cosine: "$similarity", "$cosine-similarity": "$similarity",
427
+ "$cos-sim": "$similarity", "$dot-product": "$similarity",
428
+ $dot: "$similarity", "$inner-product": "$similarity",
429
+ "$vector-distance": "$similarity (higher is closer; there is no distance metric)",
430
+ "$l2-distance": "$similarity (higher is closer; there is no distance metric)",
431
+ $knn: "$orderby on a $similarity key with $dir 'desc', then $subsequence for the k",
432
+ $nearest: "$orderby on a $similarity key with $dir 'desc', then $subsequence for the k",
433
+ "$nearest-neighbours": "$orderby on a $similarity key, then $subsequence",
434
+ "$nearest-neighbors": "$orderby on a $similarity key, then $subsequence",
435
+ "$top-k": "$orderby then $subsequence", $embed: null, $normalize: null,
420
436
  };
421
437
 
422
438
  // JQ0002 for an unknown $-key, with a "did you mean" suggestion. A
@@ -56,6 +56,7 @@ import {
56
56
  fixedUnitMs,
57
57
  compileDateFormat,
58
58
  } from '@jarenjs/core/dates';
59
+ import { isVector, cosineSimilarity } from '@jarenjs/core/vector';
59
60
  import {
60
61
  isPosition,
61
62
  bboxOf,
@@ -840,6 +841,41 @@ function geoUnaryEntry(measure, check = geoArg) {
840
841
 
841
842
  //#endregion
842
843
 
844
+ //#region vector operators (section 8.15)
845
+ // One operator, one metric. A vector here is what a JSON document can
846
+ // hold — an array of numbers — and the arithmetic lives once, in
847
+ // `@jarenjs/core/vector`, where the ledger and the store read it too.
848
+ //
849
+ // The kernel and the language answer a malformed comparison
850
+ // differently, on purpose. A kernel scores 0 so that one bad vector
851
+ // among ten thousand loses a ranking sweep instead of killing it; the
852
+ // language answers the empty sequence, the same thing §8.14 answers for
853
+ // a measurement it cannot make, so a document never carries a score
854
+ // that was never computed. The gate between the two is `isVector`, the
855
+ // one shape guard the suite shares.
856
+
857
+ // A vector operand, split the way §8.14 splits geography: a value that
858
+ // is not an array of numbers is the wrong TYPE and raises, while an
859
+ // array of numbers that cannot be compared — empty, or carrying a
860
+ // component a computation made non-finite — is a DATA-level answer of
861
+ // empty, decided by the caller against the other operand's width.
862
+ function vectorArg(v, docPath) {
863
+ if (!Array.isArray(v)) {
864
+ throw runtimeError('JQ2001',
865
+ `expected a vector (an array of numbers), got ${describeItem(v)}`, docPath);
866
+ }
867
+ for (let i = 0; i < v.length; i++) {
868
+ if (typeof v[i] !== 'number') {
869
+ throw runtimeError('JQ2001',
870
+ `expected a vector (an array of numbers), got ${describeItem(v[i])} at index ${i}`,
871
+ docPath);
872
+ }
873
+ }
874
+ return v;
875
+ }
876
+
877
+ //#endregion
878
+
843
879
  // the RFC 3339 type tests, shaped like the section 8.10 $is-* family:
844
880
  // one item of the right lexical form, never an error
845
881
  function dateTestEntry(test) {
@@ -1748,6 +1784,39 @@ export const OPERATORS = Object.freeze({
1748
1784
 
1749
1785
  //#endregion
1750
1786
 
1787
+ //#region section 8.15 - vectors
1788
+
1789
+ '$similarity': { // cosine similarity of two vectors, higher-is-better
1790
+ params: ARGS_2,
1791
+ // OPTIONAL: two present operands can still have no comparison — a
1792
+ // width mismatch, an empty vector, a component a computation made
1793
+ // non-finite. Declaring exactly-one would let the internal empty
1794
+ // marker escape into an array or object constructor.
1795
+ result: RESULT_OPT,
1796
+ compile: (gets, args) => {
1797
+ const aGet = gets[0];
1798
+ const aPath = args[0].docPath;
1799
+ const bGet = gets[1];
1800
+ const bPath = args[1].docPath;
1801
+ return (f) => {
1802
+ const a = aGet(f);
1803
+ const b = bGet(f);
1804
+ if (a === EMPTY || b === EMPTY)
1805
+ return EMPTY;
1806
+ const va = vectorArg(a, aPath);
1807
+ const vb = vectorArg(b, bPath);
1808
+ // the width is the second operand's obligation, so the same
1809
+ // guard answers both "is this comparable at all" and "is it
1810
+ // comparable to THAT" — nothing here pads or truncates
1811
+ if (!isVector(va) || !isVector(vb, va.length))
1812
+ return EMPTY;
1813
+ return cosineSimilarity(va, vb);
1814
+ };
1815
+ },
1816
+ },
1817
+
1818
+ //#endregion
1819
+
1751
1820
  //#region section 8.13 - dates and times, continued
1752
1821
 
1753
1822
  '$date-add': dateShiftEntry(1),