@lde/search 0.5.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +71 -60
- package/dist/adapter.d.ts +2 -2
- package/dist/adapter.d.ts.map +1 -1
- package/dist/adapter.js +2 -2
- package/dist/project.d.ts +3 -3
- package/dist/project.d.ts.map +1 -1
- package/dist/project.js +50 -35
- package/dist/schema.d.ts +84 -37
- package/dist/schema.d.ts.map +1 -1
- package/dist/schema.js +102 -22
- package/dist/testing.js +3 -3
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
The core of the LDE search family: packages that together act as a **generator
|
|
4
4
|
for search engines**. You write one declarative `SearchSchema`, and everything
|
|
5
5
|
a running search engine needs is derived from it: the document projection, the
|
|
6
|
-
engine collection
|
|
6
|
+
engine collection definition, the query semantics, and the API surface. All these
|
|
7
7
|
are kept in sync automatically rather than handwritten per deployment.
|
|
8
8
|
|
|
9
9
|
The core itself is **engine-, API- and domain-agnostic**: it bakes in no search
|
|
@@ -21,26 +21,26 @@ The library never names your domain: the same core drives a `Dataset`,
|
|
|
21
21
|
|
|
22
22
|
It provides four things:
|
|
23
23
|
|
|
24
|
-
- **unified field model**
|
|
24
|
+
- **unified field model** – `SearchField` / `SearchType` / `SearchSchema`:
|
|
25
25
|
one declaration per field that drives all four consumers below, so they
|
|
26
26
|
cannot drift;
|
|
27
|
-
- **neutral query IR**
|
|
27
|
+
- **neutral query IR** – `SearchQuery` / `Filter` / `Sort` + filter
|
|
28
28
|
semantics: every API surface compiles into it, every engine adapter compiles
|
|
29
29
|
out of it, so the two cannot drift;
|
|
30
|
-
- **engine port**
|
|
30
|
+
- **engine port** – `SearchEngine` and the logical result types
|
|
31
31
|
(`SearchResult` / `SearchHit` / `ResultDocument` / `Reference` / …);
|
|
32
|
-
- **streaming projection**
|
|
32
|
+
- **streaming projection** – `projectGraph`, RDF `CONSTRUCT` quads → flat
|
|
33
33
|
search documents.
|
|
34
34
|
|
|
35
35
|
```
|
|
36
36
|
SearchSchema ─┬─► projection (projectGraph → flat documents) [here]
|
|
37
|
-
├─► engine adapter (collection
|
|
37
|
+
├─► engine adapter (collection definition + query compiler) e.g. @lde/search-typesense
|
|
38
38
|
├─► query semantics (SearchQuery, filter/sort/facet) [here]
|
|
39
39
|
└─► API surface (GraphQL / REST) e.g. @lde/search-api-graphql
|
|
40
40
|
```
|
|
41
41
|
|
|
42
42
|
At runtime, everything those consumers do is a **pure transformation between
|
|
43
|
-
data shapes**, each one parameterised by the schema
|
|
43
|
+
data shapes**, each one parameterised by the schema – three chains, meeting at
|
|
44
44
|
the engine:
|
|
45
45
|
|
|
46
46
|
```
|
|
@@ -50,20 +50,20 @@ results: engine response ──parse──► SearchResult ──shape──
|
|
|
50
50
|
```
|
|
51
51
|
|
|
52
52
|
Validation happens before the first arrow (SHACL over the RDF) and inside the
|
|
53
|
-
last (the engine enforces its collection
|
|
54
|
-
a typed, deterministic function
|
|
53
|
+
last (the engine enforces its collection definition); between them every stage is
|
|
54
|
+
a typed, deterministic function – easy to test, and swappable per deployment.
|
|
55
55
|
|
|
56
56
|
## Entry points
|
|
57
57
|
|
|
58
58
|
Exports are stratified by audience:
|
|
59
59
|
|
|
60
|
-
- **`@lde/search`**
|
|
60
|
+
- **`@lde/search`** – the authoring surface: `defineSearchType`,
|
|
61
61
|
`searchSchema`, `projectGraph` (+ the IR readers `derive` functions use),
|
|
62
62
|
validation, and every model/query/result type.
|
|
63
|
-
- **`@lde/search/adapter`**
|
|
63
|
+
- **`@lde/search/adapter`** – plumbing for engine adapters and API surfaces:
|
|
64
64
|
`physicalFields`, the field selectors, `assertValidQuery`, the filter
|
|
65
65
|
operators and storage codecs.
|
|
66
|
-
- **`@lde/search/testing`**
|
|
66
|
+
- **`@lde/search/testing`** – `describeSearchEngineContract`, the executable
|
|
67
67
|
port contract every engine adapter runs against a live instance of itself
|
|
68
68
|
(vitest; optional peer).
|
|
69
69
|
|
|
@@ -72,11 +72,11 @@ Exports are stratified by audience:
|
|
|
72
72
|
The model has three levels, with analogues in SHACL ([one possible source](#why-a-declarative-model))
|
|
73
73
|
and GraphQL (one of the surfaces):
|
|
74
74
|
|
|
75
|
-
| Term | What it is
|
|
76
|
-
| -------------- |
|
|
77
|
-
| `SearchField` | One queryable field: a `kind`, the IR `path` it projects from, and the capability flags it opts into
|
|
78
|
-
| `SearchType` | One root type’s complete declaration: its logical API `name`, its `
|
|
79
|
-
| `SearchSchema` | The whole search declaration: every `SearchType`, keyed by `
|
|
75
|
+
| Term | What it is | SHACL | GraphQL |
|
|
76
|
+
| -------------- | ---------------------------------------------------------------------------------------------------------------- | -------------- | ----------- |
|
|
77
|
+
| `SearchField` | One queryable field: a `kind`, the IR `path` it projects from, and the capability flags it opts into | property shape | field |
|
|
78
|
+
| `SearchType` | One root type’s complete declaration: its logical API `name`, its `class` IRI and its fields (incl. derived) | NodeShape | object type |
|
|
79
|
+
| `SearchSchema` | The whole search declaration: every `SearchType`, keyed by `class` IRI – build one with `searchSchema(...types)` | shapes graph | schema |
|
|
80
80
|
|
|
81
81
|
`projectGraph` and the GraphQL surface consume a `SearchSchema` (projecting
|
|
82
82
|
every type in one pass); the engine port executes one `SearchType` at a time.
|
|
@@ -85,14 +85,14 @@ every type in one pass); the engine port executes one `SearchType` at a time.
|
|
|
85
85
|
|
|
86
86
|
Two conventions hold across the whole family:
|
|
87
87
|
|
|
88
|
-
- **Parameter order**
|
|
88
|
+
- **Parameter order** – a function takes the value it operates on first and
|
|
89
89
|
the declaration right after it: `projectDocument(node, type)`,
|
|
90
90
|
`buildSearchParams(query, type)`,
|
|
91
91
|
`createTypesenseSearchEngine(client, schema, options)`,
|
|
92
92
|
`engine.search(type, query)`.
|
|
93
|
-
- **Factory verbs**
|
|
93
|
+
- **Factory verbs** – the verb tells you what kind of thing comes back.
|
|
94
94
|
`define*` captures a declaration as a literal (`defineSearchType`);
|
|
95
|
-
`build*` is a pure data-to-data constructor (`
|
|
95
|
+
`build*` is a pure data-to-data constructor (`buildCollectionDefinition`,
|
|
96
96
|
`buildSearchParams`, `buildGraphQLSchema`); `create*` makes a stateful
|
|
97
97
|
instance (`createTypesenseSearchEngine`). A bare noun (`searchSchema`)
|
|
98
98
|
constructs the trivial container it names.
|
|
@@ -101,10 +101,10 @@ Two conventions hold across the whole family:
|
|
|
101
101
|
|
|
102
102
|
The mapping is data, not code. Each field declares its `kind`, the IR `path` to
|
|
103
103
|
read (or a `derive` function for a **derived** field, computed from the framed
|
|
104
|
-
node in declaration order
|
|
104
|
+
node in declaration order – so it may read fields declared before it), and the
|
|
105
105
|
capabilities it opts into. The physical field names a declaration fans out to
|
|
106
106
|
(per-locale search/sort keys) come from
|
|
107
|
-
`physicalFields`, the single convention projection, the collection
|
|
107
|
+
`physicalFields`, the single convention projection, the collection definition and the
|
|
108
108
|
query compiler all share.
|
|
109
109
|
|
|
110
110
|
```ts
|
|
@@ -117,9 +117,9 @@ import {
|
|
|
117
117
|
|
|
118
118
|
const DATASET = defineSearchType({
|
|
119
119
|
name: 'Dataset', // logical API name: names the GraphQL type, a REST path, …
|
|
120
|
-
|
|
120
|
+
class: 'http://www.w3.org/ns/dcat#Dataset',
|
|
121
121
|
fields: [
|
|
122
|
-
// →
|
|
122
|
+
// → title_<lang> (display, every present language), title_search_nl/_en, title_sort_nl/_en
|
|
123
123
|
{
|
|
124
124
|
name: 'title',
|
|
125
125
|
path: 'http://purl.org/dc/terms/title',
|
|
@@ -163,33 +163,35 @@ for await (const { searchType, document } of projectGraph(
|
|
|
163
163
|
|
|
164
164
|
`defineSearchType` captures the declaration as a literal (what
|
|
165
165
|
`as const satisfies SearchType` would do manually, with nothing to remember),
|
|
166
|
-
so typed facet/output keys can be derived from it
|
|
166
|
+
so typed facet/output keys can be derived from it – see
|
|
167
167
|
[Typed results](#typed-results) and `@lde/search-api-graphql`.
|
|
168
168
|
|
|
169
169
|
**Kinds** (`FieldKind`): `text`, `keyword`, `integer`, `number`, `boolean`,
|
|
170
170
|
`date`, `reference`. The Typesense/engine vocabulary and the GraphQL types are
|
|
171
|
-
_derived_ from the kind by the adapter and the surface
|
|
171
|
+
_derived_ from the kind by the adapter and the surface – never declared here.
|
|
172
172
|
|
|
173
173
|
`SearchField` is a **discriminated union by `kind`** (`TextField`,
|
|
174
174
|
`KeywordField`, `ReferenceField`, `NumericField`, `BooleanField`): each kind
|
|
175
|
-
declares exactly the properties it can honour
|
|
176
|
-
references, `facetRanges` on numerics
|
|
175
|
+
declares exactly the properties it can honour – `locales` on text, `ref` on
|
|
176
|
+
references, `facetRanges` on numerics – so an illegal declaration fails to
|
|
177
177
|
compile. Text is **always multilingual in shape**: `locales` lists the
|
|
178
|
-
language tags to
|
|
179
|
-
RDF `und`) buckets untagged literals
|
|
180
|
-
declares `locales: ['und']`, mixed data `['nl', 'und']`.
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
178
|
+
language tags to **index** (search/sort), and the reserved **`und`** locale
|
|
179
|
+
(JSON-LD `@none`, RDF `und`) buckets untagged literals – a monolingual or
|
|
180
|
+
untagged corpus declares `locales: ['und']`, mixed data `['nl', 'und']`.
|
|
181
|
+
Display, by contrast, keeps every language present, not only the listed ones
|
|
182
|
+
(see [Locales](#locales)). Declaring a real language is recommended (it drives
|
|
183
|
+
per-locale stemming); `und` is folded but unstemmed unless `defaultLocale` opts
|
|
184
|
+
in, is never demoted in search weighting, and adding a language later is
|
|
185
|
+
additive – the API output shape never changes. Use `keyword` for exact-match
|
|
186
|
+
tokens, never for prose.
|
|
185
187
|
|
|
186
188
|
**Declarations are also validated at runtime** (for declarations built
|
|
187
|
-
outside TypeScript
|
|
189
|
+
outside TypeScript – a SHACL generator, plain JS): `searchSchema()`
|
|
188
190
|
rejects a structurally invalid declaration (duplicate field names, an `output`
|
|
189
191
|
reference without `ref`, `text` without locales, `locales` on a non-text kind,
|
|
190
192
|
`facetRanges` on a non-numeric kind, `searchable`/`transform` on a kind whose
|
|
191
193
|
projection cannot honour it, `filterable`/`facetable` on `text`, two types
|
|
192
|
-
sharing a `
|
|
194
|
+
sharing a `class` IRI or `name`) – the declaration-time counterpart of the
|
|
193
195
|
port’s always-on query validation, so a bad schema fails at startup rather
|
|
194
196
|
than per document at index time. `validateSearchType` /
|
|
195
197
|
`assertValidSearchType` are exported for validating a single declaration
|
|
@@ -228,7 +230,7 @@ Each yielded value is a `{ searchType, document }` pair: the whole schema
|
|
|
228
230
|
projects into **one** mixed stream, and every document is tagged with the
|
|
229
231
|
`SearchType` it was framed from. That tag is what lets the write side fan the
|
|
230
232
|
stream out to per-type collections without re-deriving the type from the
|
|
231
|
-
document
|
|
233
|
+
document – see `@lde/search-pipeline`’s multi-collection writer.
|
|
232
234
|
|
|
233
235
|
`projectGraph` **consumes the quads once** – a single scan builds the subject
|
|
234
236
|
index every type frames off – and so accepts any `Iterable<Quad>`, not just a
|
|
@@ -247,30 +249,39 @@ projectGraph(
|
|
|
247
249
|
|
|
248
250
|
## Locales
|
|
249
251
|
|
|
250
|
-
`locales`
|
|
251
|
-
untagged literals)
|
|
252
|
-
`
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
- `output` → `
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
252
|
+
`locales` declares the languages a `text` field wants **indexed** (`und` =
|
|
253
|
+
untagged literals). It drives the two per-locale, in-memory fanouts –
|
|
254
|
+
`searchable` and `sortable` – but **not** display, which preserves every
|
|
255
|
+
language present:
|
|
256
|
+
|
|
257
|
+
- `output` → `title_<lang>` for **every** language the data carries (`title_nl`,
|
|
258
|
+
`title_en`, `title_fr`, `title_und`, …), accents preserved. Display is stored
|
|
259
|
+
`index: false` (on disk, off RAM), so extra languages cost nothing – a value
|
|
260
|
+
in a language outside `locales`, or an untagged one, still renders rather than
|
|
261
|
+
collapsing to a bare IRI. One un-indexed regex field (`${name}_<lang>`, see
|
|
262
|
+
`displayFieldPattern`) captures them all;
|
|
263
|
+
- `searchable` → `title_search_nl`/`title_search_en` (folded; one field per
|
|
264
|
+
locale lets a query `query_by` them and rank the user’s language higher, and
|
|
265
|
+
lets a language that needs a dedicated tokenizer set its own stemming `locale`
|
|
266
|
+
in the engine schema);
|
|
260
267
|
- `sortable` → `title_sort_nl`/`title_sort_en` (folded, so a locale-switching UI
|
|
261
268
|
sorts on the active language).
|
|
262
269
|
|
|
263
|
-
A field with `searchable` but no `output` is **search-only**
|
|
270
|
+
A field with `searchable` but no `output` is **search-only** – folded and stemmed
|
|
264
271
|
for retrieval but never rendered (e.g. a creator searched here but shown via a
|
|
265
|
-
separate label). **Only listed locales are indexed
|
|
266
|
-
|
|
267
|
-
**omitted, never empty**, when a document
|
|
268
|
-
|
|
272
|
+
separate label). **Only listed locales are indexed** (searched and sorted); a
|
|
273
|
+
literal whose language tag is not in `locales` is still **displayed** but not
|
|
274
|
+
matched or sorted on. Display fields are **omitted, never empty**, when a document
|
|
275
|
+
lacks that language, and the per-locale search/sort fields likewise, so declare
|
|
276
|
+
them optional in the engine schema and sort with `missing_values: last`. A
|
|
277
|
+
deployment that wants to bound the displayed languages narrows them upstream
|
|
278
|
+
(e.g. selecting a language subset in its CONSTRUCT query), since preservation is
|
|
279
|
+
the default.
|
|
269
280
|
|
|
270
281
|
Folding the search fields is what lets diacritic-insensitive matching and
|
|
271
282
|
stemming coexist. A search engine on its **default** locale typically folds case
|
|
272
283
|
and diacritics for you; enabling a language’s **stemming** switches it to ICU
|
|
273
|
-
tokenization, which **preserves** diacritics
|
|
284
|
+
tokenization, which **preserves** diacritics – at which point `fold()` (from
|
|
274
285
|
[`@lde/text-normalization`](../text-normalization)) is what keeps matching
|
|
275
286
|
diacritic-insensitive. Stemming is rules-based and can mangle proper nouns (the
|
|
276
287
|
Dutch stemmer reduces the city `Bergen` to `berg`), so enable it on free-text
|
|
@@ -281,21 +292,21 @@ fields and keep proper-noun facets on a separate, unstemmed field.
|
|
|
281
292
|
The search fields are stored already case- and diacritic-folded, so **the query
|
|
282
293
|
must be folded the same way** with the same `fold()` before it reaches the engine,
|
|
283
294
|
or index and query normalize differently and matches silently miss. This contract
|
|
284
|
-
holds for **any** consumer, including an API built on this package
|
|
295
|
+
holds for **any** consumer, including an API built on this package – which is why
|
|
285
296
|
engine adapters and surfaces compile through the shared `SearchQuery` IR and the
|
|
286
297
|
`physicalFields` convention rather than re-deriving field names.
|
|
287
298
|
|
|
288
299
|
Queries are **always validated**: the port contract requires every engine
|
|
289
|
-
adapter to reject a structurally invalid `SearchQuery` (`assertValidQuery`)
|
|
300
|
+
adapter to reject a structurally invalid `SearchQuery` (`assertValidQuery`) –
|
|
290
301
|
unknown or non-`filterable` fields in `where`, an operator not matching the
|
|
291
|
-
field’s kind, non-`facetable` facet requests
|
|
302
|
+
field’s kind, non-`facetable` facet requests – no matter which surface or
|
|
292
303
|
policy produced it. A typed surface like GraphQL makes most of these
|
|
293
304
|
unrepresentable; the port enforces them for everyone else (deployment
|
|
294
305
|
`queryDefaults`, in-process callers, weaker-typed surfaces).
|
|
295
306
|
|
|
296
307
|
## Typed results
|
|
297
308
|
|
|
298
|
-
An engine is **bound to the whole `SearchSchema` at construction**
|
|
309
|
+
An engine is **bound to the whole `SearchSchema` at construction** – like
|
|
299
310
|
every other schema consumer (`projectGraph(quads, schema)`,
|
|
300
311
|
`buildGraphQLSchema(schema)`): the adapter factory takes the deployment’s
|
|
301
312
|
declaration together with each type’s physical location, so a query can never
|
|
@@ -304,7 +315,7 @@ cross-type search, facet batching) have one home. A search names its type per
|
|
|
304
315
|
call. Because `searchSchema()` captures the declared types as a literal
|
|
305
316
|
tuple, `search()` accepts **only the deployment’s own types** (a foreign type
|
|
306
317
|
is a compile error) and returns facet/document keys typed by the type passed
|
|
307
|
-
|
|
318
|
+
– no caller-side generics:
|
|
308
319
|
|
|
309
320
|
```ts
|
|
310
321
|
const engine = createTypesenseSearchEngine(client, schema, {
|
|
@@ -337,5 +348,5 @@ signatures, and `engine.schema` exposes the bound declaration for routing.
|
|
|
337
348
|
The vocabulary mirrors SHACL on purpose: `path` is `sh:path`, `array` is
|
|
338
349
|
`sh:maxCount`, `required` is `sh:minCount`, `locales` is `sh:languageIn`, `ref`
|
|
339
350
|
is `sh:class`/`sh:node`. So the same core that runs a hand-written `SearchSchema`
|
|
340
|
-
today will run a **SHACL-generated** one tomorrow
|
|
351
|
+
today will run a **SHACL-generated** one tomorrow – the model, the ports and the
|
|
341
352
|
IR stay; only schema-authoring gets automated.
|
package/dist/adapter.d.ts
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* The adapter-author surface of `@lde/search` (import from
|
|
3
3
|
* `@lde/search/adapter`): the plumbing an engine adapter or API surface needs
|
|
4
|
-
* to compile queries and reconstruct results
|
|
4
|
+
* to compile queries and reconstruct results – field selectors, the physical
|
|
5
5
|
* field-name convention, query validation and the storage codecs. A
|
|
6
6
|
* deployment author declaring a schema and running searches needs none of
|
|
7
7
|
* this; the main entry point carries that (small) authoring surface.
|
|
8
8
|
*/
|
|
9
|
-
export { assertTypeInSchema, physicalFields, searchableFields, facetableFields, filterableFields, sortableFields, outputFields, referenceFields, fieldNamed, labelFieldOf, isRangeFacet, isoToUnixSeconds, unixSecondsToIso, } from './schema.js';
|
|
9
|
+
export { assertTypeInSchema, physicalFields, displayFieldName, displayFieldPattern, displayLangOf, searchableFields, facetableFields, filterableFields, sortableFields, outputFields, referenceFields, fieldNamed, labelFieldOf, isRangeFacet, isoToUnixSeconds, unixSecondsToIso, } from './schema.js';
|
|
10
10
|
export type { PhysicalFields } from './schema.js';
|
|
11
11
|
export { filterOperatorFor, filterOperator, validateQuery, assertValidQuery, pageForOffset, } from './query.js';
|
|
12
12
|
export type { FilterOperator, QueryIssue } from './query.js';
|
package/dist/adapter.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"adapter.d.ts","sourceRoot":"","sources":["../src/adapter.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AACH,OAAO,EACL,kBAAkB,EAClB,cAAc,EACd,gBAAgB,EAChB,eAAe,EACf,gBAAgB,EAChB,cAAc,EACd,YAAY,EACZ,eAAe,EACf,UAAU,EACV,YAAY,EACZ,YAAY,EACZ,gBAAgB,EAChB,gBAAgB,GACjB,MAAM,aAAa,CAAC;AACrB,YAAY,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAClD,OAAO,EACL,iBAAiB,EACjB,cAAc,EACd,aAAa,EACb,gBAAgB,EAChB,aAAa,GACd,MAAM,YAAY,CAAC;AACpB,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC"}
|
|
1
|
+
{"version":3,"file":"adapter.d.ts","sourceRoot":"","sources":["../src/adapter.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AACH,OAAO,EACL,kBAAkB,EAClB,cAAc,EACd,gBAAgB,EAChB,mBAAmB,EACnB,aAAa,EACb,gBAAgB,EAChB,eAAe,EACf,gBAAgB,EAChB,cAAc,EACd,YAAY,EACZ,eAAe,EACf,UAAU,EACV,YAAY,EACZ,YAAY,EACZ,gBAAgB,EAChB,gBAAgB,GACjB,MAAM,aAAa,CAAC;AACrB,YAAY,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAClD,OAAO,EACL,iBAAiB,EACjB,cAAc,EACd,aAAa,EACb,gBAAgB,EAChB,aAAa,GACd,MAAM,YAAY,CAAC;AACpB,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC"}
|
package/dist/adapter.js
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* The adapter-author surface of `@lde/search` (import from
|
|
3
3
|
* `@lde/search/adapter`): the plumbing an engine adapter or API surface needs
|
|
4
|
-
* to compile queries and reconstruct results
|
|
4
|
+
* to compile queries and reconstruct results – field selectors, the physical
|
|
5
5
|
* field-name convention, query validation and the storage codecs. A
|
|
6
6
|
* deployment author declaring a schema and running searches needs none of
|
|
7
7
|
* this; the main entry point carries that (small) authoring surface.
|
|
8
8
|
*/
|
|
9
|
-
export { assertTypeInSchema, physicalFields, searchableFields, facetableFields, filterableFields, sortableFields, outputFields, referenceFields, fieldNamed, labelFieldOf, isRangeFacet, isoToUnixSeconds, unixSecondsToIso, } from './schema.js';
|
|
9
|
+
export { assertTypeInSchema, physicalFields, displayFieldName, displayFieldPattern, displayLangOf, searchableFields, facetableFields, filterableFields, sortableFields, outputFields, referenceFields, fieldNamed, labelFieldOf, isRangeFacet, isoToUnixSeconds, unixSecondsToIso, } from './schema.js';
|
|
10
10
|
export { filterOperatorFor, filterOperator, validateQuery, assertValidQuery, pageForOffset, } from './query.js';
|
package/dist/project.d.ts
CHANGED
|
@@ -7,7 +7,7 @@ export type SearchDocument = {
|
|
|
7
7
|
} & Record<string, unknown>;
|
|
8
8
|
/**
|
|
9
9
|
* A projected document tagged with the {@link SearchType} it was projected
|
|
10
|
-
* from
|
|
10
|
+
* from – one element of {@link projectGraph}’s mixed, whole-schema stream. The
|
|
11
11
|
* tag is what lets a multi-collection writer route each document to the
|
|
12
12
|
* collection for its type without re-deriving the type from the document.
|
|
13
13
|
*/
|
|
@@ -21,12 +21,12 @@ export interface TypedSearchDocument {
|
|
|
21
21
|
* its value from the node and the document as populated so far (so a derived
|
|
22
22
|
* field may read fields declared before it). The physical field names a field
|
|
23
23
|
* fans out to come from {@link physicalFields}, the single source shared with
|
|
24
|
-
* the engine collection
|
|
24
|
+
* the engine collection definition and the query compiler.
|
|
25
25
|
*/
|
|
26
26
|
export declare function projectDocument(node: FramedNode, searchType: SearchType): SearchDocument;
|
|
27
27
|
/**
|
|
28
28
|
* Frame `quads` for every root type in the schema and project each node with its
|
|
29
|
-
* type’s declaration
|
|
29
|
+
* type’s declaration – the multi-shape pipeline. Yields each document tagged
|
|
30
30
|
* with its {@link SearchType} ({@link TypedSearchDocument}), so a downstream
|
|
31
31
|
* multi-collection writer can route it to that type’s collection; a
|
|
32
32
|
* single-collection consumer just reads `.document`. Streams one document at a
|
package/dist/project.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"project.d.ts","sourceRoot":"","sources":["../src/project.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,cAAc,CAAC;AAEzC,OAAO,EAGL,KAAK,UAAU,EAChB,MAAM,oBAAoB,CAAC;AAC5B,OAAO,
|
|
1
|
+
{"version":3,"file":"project.d.ts","sourceRoot":"","sources":["../src/project.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,cAAc,CAAC;AAEzC,OAAO,EAGL,KAAK,UAAU,EAChB,MAAM,oBAAoB,CAAC;AAC5B,OAAO,EAOL,KAAK,YAAY,EACjB,KAAK,UAAU,EAEhB,MAAM,aAAa,CAAC;AAErB,+DAA+D;AAC/D,MAAM,MAAM,cAAc,GAAG;IAAE,EAAE,EAAE,MAAM,CAAA;CAAE,GAAG,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;AAEtE;;;;;GAKG;AACH,MAAM,WAAW,mBAAmB;IAClC,QAAQ,CAAC,UAAU,EAAE,UAAU,CAAC;IAChC,QAAQ,CAAC,QAAQ,EAAE,cAAc,CAAC;CACnC;AAED;;;;;;;GAOG;AACH,wBAAgB,eAAe,CAC7B,IAAI,EAAE,UAAU,EAChB,UAAU,EAAE,UAAU,GACrB,cAAc,CAYhB;AAED;;;;;;;;;;;;;GAaG;AACH,wBAAuB,YAAY,CACjC,KAAK,EAAE,QAAQ,CAAC,IAAI,CAAC,EACrB,MAAM,EAAE,YAAY,GACnB,aAAa,CAAC,mBAAmB,CAAC,CAWpC;AAqJD,wBAAgB,UAAU,CAAC,IAAI,EAAE,UAAU,EAAE,IAAI,EAAE,MAAM,GAAG,MAAM,EAAE,CAInE;AAED,wBAAgB,cAAc,CAC5B,IAAI,EAAE,UAAU,EAChB,IAAI,EAAE,MAAM,GACX,MAAM,GAAG,SAAS,CAEpB;AAED,wBAAgB,MAAM,CAAC,IAAI,EAAE,UAAU,EAAE,IAAI,EAAE,MAAM,GAAG,MAAM,EAAE,CAI/D"}
|
package/dist/project.js
CHANGED
|
@@ -1,18 +1,18 @@
|
|
|
1
1
|
import { fold } from '@lde/text-normalization';
|
|
2
2
|
import { buildSubjectIndex, frameSubjects, } from './frame-by-type.js';
|
|
3
|
-
import { isoToUnixSeconds, physicalFields, } from './schema.js';
|
|
3
|
+
import { displayFieldName, isoToUnixSeconds, physicalFields, } from './schema.js';
|
|
4
4
|
/**
|
|
5
5
|
* Project one framed JSON-LD node into a flat search document: apply each field
|
|
6
6
|
* of the type in declaration order. A field with a `derive` function computes
|
|
7
7
|
* its value from the node and the document as populated so far (so a derived
|
|
8
8
|
* field may read fields declared before it). The physical field names a field
|
|
9
9
|
* fans out to come from {@link physicalFields}, the single source shared with
|
|
10
|
-
* the engine collection
|
|
10
|
+
* the engine collection definition and the query compiler.
|
|
11
11
|
*/
|
|
12
12
|
export function projectDocument(node, searchType) {
|
|
13
13
|
const id = node['@id'];
|
|
14
14
|
if (typeof id !== 'string') {
|
|
15
|
-
throw new Error(`Cannot project a ${searchType.
|
|
15
|
+
throw new Error(`Cannot project a ${searchType.class} node without an @id: every search document needs a stable key, and an empty one would collide with other keyless nodes.`);
|
|
16
16
|
}
|
|
17
17
|
const document = { id };
|
|
18
18
|
for (const field of searchType.fields) {
|
|
@@ -22,7 +22,7 @@ export function projectDocument(node, searchType) {
|
|
|
22
22
|
}
|
|
23
23
|
/**
|
|
24
24
|
* Frame `quads` for every root type in the schema and project each node with its
|
|
25
|
-
* type’s declaration
|
|
25
|
+
* type’s declaration – the multi-shape pipeline. Yields each document tagged
|
|
26
26
|
* with its {@link SearchType} ({@link TypedSearchDocument}), so a downstream
|
|
27
27
|
* multi-collection writer can route it to that type’s collection; a
|
|
28
28
|
* single-collection consumer just reads `.document`. Streams one document at a
|
|
@@ -36,9 +36,9 @@ export function projectDocument(node, searchType) {
|
|
|
36
36
|
*/
|
|
37
37
|
export async function* projectGraph(quads, schema) {
|
|
38
38
|
const types = [...schema.values()];
|
|
39
|
-
const index = buildSubjectIndex(quads, types.map((searchType) => searchType.
|
|
39
|
+
const index = buildSubjectIndex(quads, types.map((searchType) => searchType.class));
|
|
40
40
|
for (const searchType of types) {
|
|
41
|
-
for await (const node of frameSubjects(index, searchType.
|
|
41
|
+
for await (const node of frameSubjects(index, searchType.class)) {
|
|
42
42
|
yield { searchType, document: projectDocument(node, searchType) };
|
|
43
43
|
}
|
|
44
44
|
}
|
|
@@ -82,35 +82,46 @@ function applyField(document, node, field) {
|
|
|
82
82
|
}
|
|
83
83
|
}
|
|
84
84
|
/**
|
|
85
|
-
* Project a text field
|
|
86
|
-
*
|
|
87
|
-
* `
|
|
88
|
-
*
|
|
89
|
-
*
|
|
85
|
+
* Project a text field. **Display** (when `output`) preserves *every* language
|
|
86
|
+
* present – one label per language (accents preserved, untagged under `und`),
|
|
87
|
+
* stored `index: false` so extra languages cost nothing – so a value in an
|
|
88
|
+
* undeclared language still renders rather than collapsing to a bare IRI.
|
|
89
|
+
* **Search** (folded, when `searchable`) and **sort** (folded primary, when
|
|
90
|
+
* `sortable`) stay on the declared `locales`, which drive the indexed, stemmed,
|
|
91
|
+
* weighted fanout; a value in an undeclared language is not indexed. Absent
|
|
92
|
+
* languages emit nothing.
|
|
90
93
|
*/
|
|
91
94
|
function applyText(document, values, field) {
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
return;
|
|
102
|
-
}
|
|
103
|
-
const [primary] = localeValues;
|
|
104
|
-
if (field.output) {
|
|
105
|
-
setString(document, names.display[index], primary);
|
|
106
|
-
}
|
|
107
|
-
if (field.searchable) {
|
|
108
|
-
setString(document, names.search[index], foldedSearchValue(localeValues));
|
|
109
|
-
}
|
|
110
|
-
if (field.sortable) {
|
|
111
|
-
setString(document, names.sort[index], fold(primary));
|
|
95
|
+
if (field.output) {
|
|
96
|
+
// First value of each present language wins; a language absent from
|
|
97
|
+
// `locales` still lands as a display field (kept off the search index).
|
|
98
|
+
const seenLangs = new Set();
|
|
99
|
+
for (const { lang, value } of values) {
|
|
100
|
+
if (!seenLangs.has(lang)) {
|
|
101
|
+
seenLangs.add(lang);
|
|
102
|
+
setString(document, displayFieldName(field, lang), value);
|
|
103
|
+
}
|
|
112
104
|
}
|
|
113
|
-
}
|
|
105
|
+
}
|
|
106
|
+
// Empty `locales` is rejected at declaration time (`validateSearchType`);
|
|
107
|
+
// here it simply indexes nothing.
|
|
108
|
+
if (field.searchable !== undefined || field.sortable === true) {
|
|
109
|
+
const names = physicalFields(field);
|
|
110
|
+
field.locales.forEach((locale, index) => {
|
|
111
|
+
const localeValues = values
|
|
112
|
+
.filter((value) => value.lang === locale)
|
|
113
|
+
.map((value) => value.value);
|
|
114
|
+
if (localeValues.length === 0) {
|
|
115
|
+
return;
|
|
116
|
+
}
|
|
117
|
+
if (field.searchable) {
|
|
118
|
+
setString(document, names.search[index], foldedSearchValue(localeValues));
|
|
119
|
+
}
|
|
120
|
+
if (field.sortable) {
|
|
121
|
+
setString(document, names.sort[index], fold(localeValues[0]));
|
|
122
|
+
}
|
|
123
|
+
});
|
|
124
|
+
}
|
|
114
125
|
}
|
|
115
126
|
/** The projection’s definition of a folded free-text search value. */
|
|
116
127
|
function foldedSearchValue(values) {
|
|
@@ -118,7 +129,7 @@ function foldedSearchValue(values) {
|
|
|
118
129
|
}
|
|
119
130
|
/**
|
|
120
131
|
* Project a faceted multi-value field: dedupe (after the optional transform),
|
|
121
|
-
* write the value field, and
|
|
132
|
+
* write the value field, and – when `searchable` – a folded `${name}_search`
|
|
122
133
|
* array. `keyword` reads literals; `reference` reads IRIs (the caller passes the
|
|
123
134
|
* already-read raw values).
|
|
124
135
|
*/
|
|
@@ -160,10 +171,14 @@ function toLangValue(value) {
|
|
|
160
171
|
return undefined;
|
|
161
172
|
}
|
|
162
173
|
// Untagged literals (JSON-LD @none) land in the reserved `und` locale.
|
|
163
|
-
|
|
174
|
+
// Normalise the tag to its BCP-47 shape: `_` is the reserved separator in the
|
|
175
|
+
// physical/display field naming, so a non-conformant `pt_BR` tag becomes
|
|
176
|
+
// `pt-BR`, which round-trips through display and matches a declared locale
|
|
177
|
+
// instead of being silently dropped.
|
|
178
|
+
const rawLang = isObject(value) && typeof value['@language'] === 'string'
|
|
164
179
|
? value['@language']
|
|
165
180
|
: 'und';
|
|
166
|
-
return { value: literal, lang };
|
|
181
|
+
return { value: literal, lang: rawLang.replace(/_/g, '-') };
|
|
167
182
|
}
|
|
168
183
|
function literalString(value) {
|
|
169
184
|
if (typeof value === 'string') {
|
package/dist/schema.d.ts
CHANGED
|
@@ -3,7 +3,7 @@ import type { SearchDocument } from './project.js';
|
|
|
3
3
|
/**
|
|
4
4
|
* The engine-neutral kind of a queryable field. It drives every downstream
|
|
5
5
|
* behavior: which physical fields the projection emits, the engine
|
|
6
|
-
* collection-
|
|
6
|
+
* collection-definition type, the `where`/facet/sort semantics, and the GraphQL
|
|
7
7
|
* output/input type. The Typesense-vocabulary types (`string`, `int32`, …) are
|
|
8
8
|
* *derived* from this by the engine adapter, never declared here.
|
|
9
9
|
*/
|
|
@@ -13,14 +13,14 @@ export type FieldKind = 'text' | 'keyword' | 'integer' | 'number' | 'boolean' |
|
|
|
13
13
|
export type FilterOperator = 'in' | 'range' | 'is';
|
|
14
14
|
/**
|
|
15
15
|
* The `where` operator a field of this kind accepts (per the ADR filter-semantics
|
|
16
|
-
* table), or `undefined` for `text`
|
|
16
|
+
* table), or `undefined` for `text` – which feeds the free-text `query` rather
|
|
17
17
|
* than `where`. The ONE source for the surface’s `where` input type, the
|
|
18
18
|
* adapter’s filter compiler and declaration validation, so they cannot drift.
|
|
19
19
|
*/
|
|
20
20
|
export declare function filterOperatorFor(kind: FieldKind): FilterOperator | undefined;
|
|
21
21
|
/**
|
|
22
|
-
* One queryable field
|
|
23
|
-
* consumers (projection, engine collection
|
|
22
|
+
* One queryable field – the single declarative source that drives all four
|
|
23
|
+
* consumers (projection, engine collection definition, query semantics, and the
|
|
24
24
|
* GraphQL surface).
|
|
25
25
|
*
|
|
26
26
|
* A **discriminated union by `kind`**: each kind declares exactly the
|
|
@@ -32,13 +32,13 @@ export declare function filterOperatorFor(kind: FieldKind): FilterOperator | und
|
|
|
32
32
|
* Capability flags (`searchable`/`filterable`/`facetable`/`sortable`/`output`)
|
|
33
33
|
* are independent opt-ins: a field exposes exactly the roles it declares. A
|
|
34
34
|
* field with a {@link SearchFieldBase.derive `derive`} function instead of a
|
|
35
|
-
* `path` is a **derived field**
|
|
36
|
-
* projected from the IR
|
|
35
|
+
* `path` is a **derived field** – computed from the framed node rather than
|
|
36
|
+
* projected from the IR – yet it still carries full query/schema/output
|
|
37
37
|
* behavior (e.g. `status`, the compatibility booleans).
|
|
38
38
|
*
|
|
39
39
|
* The physical field names a declaration fans out to (per-locale search/sort
|
|
40
40
|
* keys) follow one convention, owned by
|
|
41
|
-
* {@link physicalFields} so projection, collection-
|
|
41
|
+
* {@link physicalFields} so projection, collection-definition and query compiler
|
|
42
42
|
* cannot disagree.
|
|
43
43
|
*
|
|
44
44
|
* SHACL is one possible *source*, not a dependency: a generator can emit a
|
|
@@ -72,7 +72,7 @@ export interface SearchFieldBase {
|
|
|
72
72
|
/** Publicly selectable in `orderBy`; localized text also emits a folded sort key. */
|
|
73
73
|
readonly sortable?: boolean;
|
|
74
74
|
/**
|
|
75
|
-
* Compute this field’s value instead of projecting it from a `path`
|
|
75
|
+
* Compute this field’s value instead of projecting it from a `path` – a
|
|
76
76
|
* status token, a compatibility boolean, a count over the framed node.
|
|
77
77
|
* Mutually exclusive with `path`. Runs in declaration order during
|
|
78
78
|
* projection, receiving the framed node and the document as populated so
|
|
@@ -95,7 +95,7 @@ export interface Searchable {
|
|
|
95
95
|
* locale into display/search/sort companions. `locales` lists the language
|
|
96
96
|
* tags to emit; the reserved locale **`und`** (JSON-LD `@none`, RDF `und`)
|
|
97
97
|
* buckets untagged literals, so a monolingual or untagged corpus declares
|
|
98
|
-
* `locales: ['und']` and mixed data `['nl', 'und']`
|
|
98
|
+
* `locales: ['und']` and mixed data `['nl', 'und']` – one mechanism, and
|
|
99
99
|
* adding a language later is additive (the API output shape never changes).
|
|
100
100
|
* Declaring a real language is RECOMMENDED where the data has one: it drives
|
|
101
101
|
* the engine’s per-locale stemming; `und` is folded but unstemmed (unless the
|
|
@@ -139,7 +139,7 @@ export interface ReferenceField extends SearchFieldBase, Searchable {
|
|
|
139
139
|
* optional for a facet- or filter-only reference. */
|
|
140
140
|
readonly ref?: {
|
|
141
141
|
/** Logical API type name of the referenced entity (PascalCase, e.g.
|
|
142
|
-
* `Organization`)
|
|
142
|
+
* `Organization`) – names the reference’s type in the API surfaces, the
|
|
143
143
|
* way {@link SearchType.name} names a root type; fields sharing it share
|
|
144
144
|
* one emitted type. A name, not a key: it need not correspond to any
|
|
145
145
|
* indexed root type (and until cross-collection references exist, it must
|
|
@@ -156,7 +156,7 @@ export interface ReferenceField extends SearchFieldBase, Searchable {
|
|
|
156
156
|
/**
|
|
157
157
|
* Range-facet bins for a numeric (`integer`/`number`/`date`) facetable field.
|
|
158
158
|
* When set, the field facets into these fixed half-open `[min, max)` ranges (a
|
|
159
|
-
* histogram) rather than one bucket per distinct value
|
|
159
|
+
* histogram) rather than one bucket per distinct value – the per-bucket counts
|
|
160
160
|
* a UI slider needs. Bins are query-time only (no index impact) and
|
|
161
161
|
* engine-neutral: the Typesense adapter emits a `facet_by` range, an
|
|
162
162
|
* OpenSearch adapter a `range` aggregation. See {@link FacetRange}.
|
|
@@ -167,7 +167,7 @@ export interface RangeFacetable {
|
|
|
167
167
|
/**
|
|
168
168
|
* A numeric value: range-filtered, range- or value-faceted, sortable.
|
|
169
169
|
* `integer` is a whole number, `number` a float, `date` a point in time
|
|
170
|
-
* (ISO 8601 at the edges, Unix seconds in the index)
|
|
170
|
+
* (ISO 8601 at the edges, Unix seconds in the index) – identical capabilities,
|
|
171
171
|
* so one interface serves all three kinds; `field.kind` still narrows.
|
|
172
172
|
*/
|
|
173
173
|
export interface NumericField extends SearchFieldBase, RangeFacetable {
|
|
@@ -193,25 +193,28 @@ export interface FacetRange {
|
|
|
193
193
|
}
|
|
194
194
|
/**
|
|
195
195
|
* One root type’s complete search declaration: its logical API `name`, the
|
|
196
|
-
* `
|
|
197
|
-
* (including {@link SearchField.derive derived} ones). A SHACL
|
|
198
|
-
* emit one per NodeShape (`name`←`sh:name`/local name,
|
|
199
|
-
* `fields`←its property shapes), but that is a source,
|
|
196
|
+
* `class` IRI its documents are instances of (the RDF class), and the queryable
|
|
197
|
+
* `fields` (including {@link SearchField.derive derived} ones). A SHACL
|
|
198
|
+
* generator can emit one per NodeShape (`name`←`sh:name`/local name,
|
|
199
|
+
* `class`←`sh:targetClass`, `fields`←its property shapes), but that is a source,
|
|
200
|
+
* not a requirement.
|
|
200
201
|
*/
|
|
201
202
|
export interface SearchType {
|
|
202
|
-
/** Logical API name (PascalCase, e.g. `Dataset`)
|
|
203
|
+
/** Logical API name (PascalCase, e.g. `Dataset`) – names the type in the API
|
|
203
204
|
* surfaces (GraphQL type names, a REST path), the way each field’s
|
|
204
205
|
* {@link SearchField.name} names that field. Deliberately declared rather
|
|
205
|
-
* than derived from the `
|
|
206
|
+
* than derived from the `class` IRI, so re-modelling the vocabulary cannot
|
|
206
207
|
* silently rename the public contract. */
|
|
207
208
|
readonly name: string;
|
|
208
|
-
|
|
209
|
+
/** The RDF class IRI its documents are instances of (`sh:targetClass`); the
|
|
210
|
+
* key a {@link SearchSchema} maps this type under. */
|
|
211
|
+
readonly class: string;
|
|
209
212
|
readonly fields: readonly SearchField[];
|
|
210
213
|
}
|
|
211
214
|
/**
|
|
212
215
|
* Declare a {@link SearchType}, capturing it as a literal: the `const` type
|
|
213
216
|
* parameter preserves the field names and capability flags that the type-level
|
|
214
|
-
* helpers (`FacetFieldsOf`, `OutputFieldsOf`) read off the type
|
|
217
|
+
* helpers (`FacetFieldsOf`, `OutputFieldsOf`) read off the type –
|
|
215
218
|
* with none of the widening a plain `: SearchType` annotation causes and
|
|
216
219
|
* without having to remember `as const satisfies SearchType`. Identity at
|
|
217
220
|
* runtime.
|
|
@@ -219,13 +222,13 @@ export interface SearchType {
|
|
|
219
222
|
export declare function defineSearchType<const Type extends SearchType>(searchType: Type): Type;
|
|
220
223
|
/**
|
|
221
224
|
* The complete search declaration of a deployment: every root {@link SearchType},
|
|
222
|
-
* keyed by its `
|
|
225
|
+
* keyed by its `class` IRI. Build one with {@link searchSchema}, which captures
|
|
223
226
|
* the declared types as a literal tuple (`Types`), so schema-bound consumers
|
|
224
227
|
* (the engine port) can type their per-type behaviour off it. A plain
|
|
225
228
|
* `: SearchSchema` annotation widens gracefully to `SearchType`.
|
|
226
229
|
*/
|
|
227
230
|
/** Brand for {@link SearchSchema}: type-only, no runtime existence. Makes the
|
|
228
|
-
* schema NOMINAL
|
|
231
|
+
* schema NOMINAL – a hand-built `Map` is not assignable, so `searchSchema()`
|
|
229
232
|
* (which validates) is the only way to obtain one and downstream consumers
|
|
230
233
|
* need no defensive re-validation. */
|
|
231
234
|
export declare const validSearchSchema: unique symbol;
|
|
@@ -233,11 +236,11 @@ export interface SearchSchema<Types extends readonly SearchType[] = readonly Sea
|
|
|
233
236
|
readonly [validSearchSchema]: true;
|
|
234
237
|
}
|
|
235
238
|
/**
|
|
236
|
-
* Build a {@link SearchSchema} from root-type declarations, keyed by `
|
|
239
|
+
* Build a {@link SearchSchema} from root-type declarations, keyed by `class`.
|
|
237
240
|
*
|
|
238
|
-
* Every declaration is validated ({@link assertValidSearchType})
|
|
239
|
-
* declaration-time counterpart of the port’s `assertValidQuery`
|
|
240
|
-
* schema-wide invariants are enforced: no two types may share a `
|
|
241
|
+
* Every declaration is validated ({@link assertValidSearchType}) – the
|
|
242
|
+
* declaration-time counterpart of the port’s `assertValidQuery` – and the
|
|
243
|
+
* schema-wide invariants are enforced: no two types may share a `class` IRI
|
|
241
244
|
* (they would silently overwrite each other in the map) or a `name` (names
|
|
242
245
|
* key the API surfaces). Throws on the first invalid declaration, so a bad
|
|
243
246
|
* schema fails at startup, not per document at index time or per query.
|
|
@@ -260,21 +263,25 @@ export declare function labelFieldOf(searchType: SearchType): TextField | undefi
|
|
|
260
263
|
*/
|
|
261
264
|
export interface SearchTypeIssue {
|
|
262
265
|
readonly field: string;
|
|
263
|
-
readonly reason: 'duplicate-field-name' | 'missing-ref' | 'ref-not-allowed' | 'text-requires-locales' | 'locales-not-allowed' | 'facet-ranges-not-allowed' | 'searchable-not-allowed' | 'transform-not-allowed' | 'derive-with-path' | 'text-not-filterable' | 'text-not-facetable';
|
|
266
|
+
readonly reason: 'duplicate-field-name' | 'invalid-field-name' | 'invalid-locale' | 'missing-ref' | 'ref-not-allowed' | 'text-requires-locales' | 'locales-not-allowed' | 'facet-ranges-not-allowed' | 'searchable-not-allowed' | 'transform-not-allowed' | 'derive-with-path' | 'text-not-filterable' | 'text-not-facetable';
|
|
264
267
|
}
|
|
265
268
|
/**
|
|
266
|
-
* Structurally validate one {@link SearchType} declaration
|
|
269
|
+
* Structurally validate one {@link SearchType} declaration – the
|
|
267
270
|
* declaration-time counterpart of `validateQuery`. Rules:
|
|
268
271
|
*
|
|
269
272
|
* - field names are unique (a duplicate would silently shadow in every
|
|
270
|
-
* consumer, each picking a different winner)
|
|
273
|
+
* consumer, each picking a different winner) and a metacharacter-free
|
|
274
|
+
* identifier (the name is interpolated into physical field names and the
|
|
275
|
+
* display RE2 pattern);
|
|
276
|
+
* - a `text` field’s declared locales are BCP-47-shaped (no `_`, which is the
|
|
277
|
+
* reserved name↔locale separator);
|
|
271
278
|
* - a `reference` field that is `output` declares `ref` (the API surfaces
|
|
272
279
|
* need the reference type name); `ref` on any other kind is meaningless;
|
|
273
280
|
* - a `text` field declares at least one locale (`und` = untagged; projection and
|
|
274
|
-
* result reconstruction have no representation for unlocalized text
|
|
281
|
+
* result reconstruction have no representation for unlocalized text – use
|
|
275
282
|
* `keyword` for untagged strings); `locales` on any other kind is
|
|
276
283
|
* meaningless;
|
|
277
|
-
* - a kind without a `where` operator (`text`
|
|
284
|
+
* - a kind without a `where` operator (`text` – it feeds the free-text query)
|
|
278
285
|
* is neither `filterable` nor `facetable`;
|
|
279
286
|
* - `facetRanges` only on the `range`-operator kinds (`integer`/`number`/`date`);
|
|
280
287
|
* - `searchable` only on `text`/`keyword`/`reference` (projection emits no
|
|
@@ -289,7 +296,7 @@ export interface SearchTypeIssue {
|
|
|
289
296
|
*/
|
|
290
297
|
export declare function validateSearchType(searchType: SearchType): readonly SearchTypeIssue[];
|
|
291
298
|
/**
|
|
292
|
-
* Throw when `searchType` is not a member of `schema`
|
|
299
|
+
* Throw when `searchType` is not a member of `schema` – the port membership
|
|
293
300
|
* guard every engine adapter applies before searching, so a query can never
|
|
294
301
|
* meet an index the deployment did not declare. Identity-based: the exact
|
|
295
302
|
* declaration object must be in the schema, not a lookalike.
|
|
@@ -301,12 +308,10 @@ export declare function assertValidSearchType(searchType: SearchType): void;
|
|
|
301
308
|
/**
|
|
302
309
|
* The physical engine fields one {@link SearchField} fans out into, grouped by
|
|
303
310
|
* the role each plays. The single source of truth for the naming convention, so
|
|
304
|
-
* the projection (writes them), the collection
|
|
311
|
+
* the projection (writes them), the collection definition (declares them) and the
|
|
305
312
|
* query compiler (reads them) cannot disagree.
|
|
306
313
|
*/
|
|
307
314
|
export interface PhysicalFields {
|
|
308
|
-
/** Per-locale output labels `${name}_${locale}` (localized text, `output`). */
|
|
309
|
-
readonly display: readonly string[];
|
|
310
315
|
/** Folded match fields: `${name}_search_${locale}` per locale (localized) or a
|
|
311
316
|
* single `${name}_search` (non-localized), when `searchable`. */
|
|
312
317
|
readonly search: readonly string[];
|
|
@@ -315,7 +320,44 @@ export interface PhysicalFields {
|
|
|
315
320
|
readonly sort: readonly string[];
|
|
316
321
|
}
|
|
317
322
|
/**
|
|
318
|
-
*
|
|
323
|
+
* The display fields of a localized `text` field are pattern-based, not
|
|
324
|
+
* enumerated per declared locale: projection stores `${name}_${lang}` for
|
|
325
|
+
* **every** language present in the data (not only those in `locales`), so a
|
|
326
|
+
* label in an undeclared language or an untagged one still renders rather than
|
|
327
|
+
* collapsing to a bare IRI. `locales` governs only the indexed search/sort
|
|
328
|
+
* fanout ({@link PhysicalFields}); display costs nothing per language (stored
|
|
329
|
+
* `index: false`), so it preserves them all. A deployment that wants fewer
|
|
330
|
+
* display languages restricts them upstream (e.g. in its CONSTRUCT).
|
|
331
|
+
*
|
|
332
|
+
* A language subtag never contains `_`, but the `search_`/`sort_` infixes do,
|
|
333
|
+
* so `${name}_${lang}` with `lang` matching `[^_]+` is unambiguously a display
|
|
334
|
+
* field: `label_nl`, `label_fr`, `label_zh-hant`, `label_und` are display;
|
|
335
|
+
* `label_search_nl` and `label_sort_nl` are not. This trio – the name a value
|
|
336
|
+
* is written under ({@link displayFieldName}), the collection pattern that
|
|
337
|
+
* accepts them all ({@link displayFieldPattern}), and the reader that recovers a
|
|
338
|
+
* key’s language ({@link displayLangOf}) – is the single home of that
|
|
339
|
+
* convention, so projection, collection-definition and result reconstruction
|
|
340
|
+
* cannot disagree.
|
|
341
|
+
*/
|
|
342
|
+
export declare function displayFieldName(field: TextField, lang: string): string;
|
|
343
|
+
/**
|
|
344
|
+
* The RE2 pattern a collection declares to store every present language’s
|
|
345
|
+
* display value un-indexed, or `undefined` when the field is not `output` (no
|
|
346
|
+
* display at all). Matches `${name}_${lang}` for any underscore-free `lang`, so
|
|
347
|
+
* it never collides with the field’s `${name}_search_${locale}` /
|
|
348
|
+
* `${name}_sort_${locale}` companions.
|
|
349
|
+
*/
|
|
350
|
+
export declare function displayFieldPattern(field: TextField): string | undefined;
|
|
351
|
+
/**
|
|
352
|
+
* The language a stored document key carries for `field`’s display, or
|
|
353
|
+
* `undefined` when the key is not one of `field`’s display fields – the inverse
|
|
354
|
+
* of {@link displayFieldName}. A key qualifies when it is `${name}_` followed by
|
|
355
|
+
* an underscore-free remainder, so the `${name}_search_…`/`${name}_sort_…`
|
|
356
|
+
* companions (and unrelated fields) are rejected.
|
|
357
|
+
*/
|
|
358
|
+
export declare function displayLangOf(field: TextField, key: string): string | undefined;
|
|
359
|
+
/**
|
|
360
|
+
* Full-text searchable fields, highest `query_by` weight first – the order the
|
|
319
361
|
* engine adapter weights `query_by` in. A field is searchable iff it carries a
|
|
320
362
|
* `searchable` weight.
|
|
321
363
|
*/
|
|
@@ -356,6 +398,11 @@ export declare function isRangeFacet(field: SearchField): field is NumericField
|
|
|
356
398
|
export declare function isoToUnixSeconds(iso: string): number | undefined;
|
|
357
399
|
/** The inverse of {@link isoToUnixSeconds}: stored Unix seconds → ISO 8601. */
|
|
358
400
|
export declare function unixSecondsToIso(seconds: number): string;
|
|
359
|
-
/**
|
|
401
|
+
/**
|
|
402
|
+
* Derive the **indexed** physical engine field names a declaration produces:
|
|
403
|
+
* the per-locale `search`/`sort` fanout. A localized `text` field’s display
|
|
404
|
+
* fields are pattern-based and not enumerated here – see {@link displayFieldName}
|
|
405
|
+
* and its siblings.
|
|
406
|
+
*/
|
|
360
407
|
export declare function physicalFields(field: SearchField): PhysicalFields;
|
|
361
408
|
//# sourceMappingURL=schema.d.ts.map
|
package/dist/schema.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"schema.d.ts","sourceRoot":"","sources":["../src/schema.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC;AACrD,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,cAAc,CAAC;AAEnD;;;;;;GAMG;AACH,MAAM,MAAM,SAAS,GACjB,MAAM,GACN,SAAS,GACT,SAAS,GACT,QAAQ,GACR,SAAS,GACT,MAAM,GACN,WAAW,CAAC;AAEhB;oEACoE;AACpE,MAAM,MAAM,cAAc,GAAG,IAAI,GAAG,OAAO,GAAG,IAAI,CAAC;AAcnD;;;;;GAKG;AACH,wBAAgB,iBAAiB,CAAC,IAAI,EAAE,SAAS,GAAG,cAAc,GAAG,SAAS,CAE7E;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA6BG;AACH,MAAM,MAAM,WAAW,GACnB,SAAS,GACT,YAAY,GACZ,cAAc,GACd,YAAY,GACZ,YAAY,CAAC;AAEjB,qEAAqE;AACrE,MAAM,WAAW,eAAe;IAC9B;wCACoC;IACpC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB;+EAC2E;IAC3E,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;IACvB,oBAAoB;IACpB,QAAQ,CAAC,KAAK,CAAC,EAAE,OAAO,CAAC;IACzB;;yCAEqC;IACrC,QAAQ,CAAC,QAAQ,CAAC,EAAE,OAAO,CAAC;IAC5B,gEAAgE;IAChE,QAAQ,CAAC,MAAM,CAAC,EAAE,OAAO,CAAC;IAC1B,yBAAyB;IACzB,QAAQ,CAAC,UAAU,CAAC,EAAE,OAAO,CAAC;IAC9B,iCAAiC;IACjC,QAAQ,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC;IAC7B,qFAAqF;IACrF,QAAQ,CAAC,QAAQ,CAAC,EAAE,OAAO,CAAC;IAC5B;;;;;;;;;OASG;IACH,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,EAAE,UAAU,EAAE,QAAQ,EAAE,cAAc,KAAK,OAAO,CAAC;CAC3E;AAED;kEACkE;AAClE,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,UAAU,CAAC,EAAE;QAAE,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;CACnD;AAED;;;;;;;;;;;;GAYG;AACH,MAAM,WAAW,SAAU,SAAQ,eAAe,EAAE,UAAU;IAC5D,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,+EAA+E;IAC/E,QAAQ,CAAC,OAAO,EAAE,SAAS,MAAM,EAAE,CAAC;IACpC,QAAQ,CAAC,UAAU,CAAC,EAAE,KAAK,CAAC;IAC5B,QAAQ,CAAC,SAAS,CAAC,EAAE,KAAK,CAAC;IAC3B,QAAQ,CAAC,WAAW,CAAC,EAAE,KAAK,CAAC;CAC9B;AAED;gCACgC;AAChC,MAAM,WAAW,YAAa,SAAQ,eAAe,EAAE,UAAU;IAC/D,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IACzB,wEAAwE;IACxE,QAAQ,CAAC,SAAS,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,KAAK,MAAM,CAAC;IAC/C,QAAQ,CAAC,WAAW,CAAC,EAAE,KAAK,CAAC;CAC9B;AAED,gFAAgF;AAChF,MAAM,WAAW,cAAe,SAAQ,eAAe,EAAE,UAAU;IACjE,QAAQ,CAAC,IAAI,EAAE,WAAW,CAAC;IAC3B,uCAAuC;IACvC,QAAQ,CAAC,SAAS,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,KAAK,MAAM,CAAC;IAC/C,QAAQ,CAAC,WAAW,CAAC,EAAE,KAAK,CAAC;IAC7B;;;;;;;OAOG;IACH,QAAQ,CAAC,WAAW,CAAC,EAAE,MAAM,CAAC;IAC9B;;0DAEsD;IACtD,QAAQ,CAAC,GAAG,CAAC,EAAE;QACb;;;;;qCAK6B;QAC7B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;QAC1B;;;;6EAIqE;QACrE,QAAQ,CAAC,QAAQ,EAAE,WAAW,GAAG,QAAQ,GAAG,QAAQ,CAAC;KACtD,CAAC;CACH;AAED;;;;;;;GAOG;AACH,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,WAAW,CAAC,EAAE,SAAS,UAAU,EAAE,CAAC;CAC9C;AAED;;;;;GAKG;AACH,MAAM,WAAW,YAAa,SAAQ,eAAe,EAAE,cAAc;IACnE,QAAQ,CAAC,IAAI,EAAE,SAAS,GAAG,QAAQ,GAAG,MAAM,CAAC;IAC7C,QAAQ,CAAC,UAAU,CAAC,EAAE,KAAK,CAAC;CAC7B;AAED,0DAA0D;AAC1D,MAAM,WAAW,YAAa,SAAQ,eAAe;IACnD,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IACzB,QAAQ,CAAC,UAAU,CAAC,EAAE,KAAK,CAAC;IAC5B,QAAQ,CAAC,WAAW,CAAC,EAAE,KAAK,CAAC;CAC9B;AAED;;;;;GAKG;AACH,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;IACrB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;CACvB;AAED
|
|
1
|
+
{"version":3,"file":"schema.d.ts","sourceRoot":"","sources":["../src/schema.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC;AACrD,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,cAAc,CAAC;AAEnD;;;;;;GAMG;AACH,MAAM,MAAM,SAAS,GACjB,MAAM,GACN,SAAS,GACT,SAAS,GACT,QAAQ,GACR,SAAS,GACT,MAAM,GACN,WAAW,CAAC;AAEhB;oEACoE;AACpE,MAAM,MAAM,cAAc,GAAG,IAAI,GAAG,OAAO,GAAG,IAAI,CAAC;AAcnD;;;;;GAKG;AACH,wBAAgB,iBAAiB,CAAC,IAAI,EAAE,SAAS,GAAG,cAAc,GAAG,SAAS,CAE7E;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA6BG;AACH,MAAM,MAAM,WAAW,GACnB,SAAS,GACT,YAAY,GACZ,cAAc,GACd,YAAY,GACZ,YAAY,CAAC;AAEjB,qEAAqE;AACrE,MAAM,WAAW,eAAe;IAC9B;wCACoC;IACpC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB;+EAC2E;IAC3E,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;IACvB,oBAAoB;IACpB,QAAQ,CAAC,KAAK,CAAC,EAAE,OAAO,CAAC;IACzB;;yCAEqC;IACrC,QAAQ,CAAC,QAAQ,CAAC,EAAE,OAAO,CAAC;IAC5B,gEAAgE;IAChE,QAAQ,CAAC,MAAM,CAAC,EAAE,OAAO,CAAC;IAC1B,yBAAyB;IACzB,QAAQ,CAAC,UAAU,CAAC,EAAE,OAAO,CAAC;IAC9B,iCAAiC;IACjC,QAAQ,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC;IAC7B,qFAAqF;IACrF,QAAQ,CAAC,QAAQ,CAAC,EAAE,OAAO,CAAC;IAC5B;;;;;;;;;OASG;IACH,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,EAAE,UAAU,EAAE,QAAQ,EAAE,cAAc,KAAK,OAAO,CAAC;CAC3E;AAED;kEACkE;AAClE,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,UAAU,CAAC,EAAE;QAAE,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;CACnD;AAED;;;;;;;;;;;;GAYG;AACH,MAAM,WAAW,SAAU,SAAQ,eAAe,EAAE,UAAU;IAC5D,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,+EAA+E;IAC/E,QAAQ,CAAC,OAAO,EAAE,SAAS,MAAM,EAAE,CAAC;IACpC,QAAQ,CAAC,UAAU,CAAC,EAAE,KAAK,CAAC;IAC5B,QAAQ,CAAC,SAAS,CAAC,EAAE,KAAK,CAAC;IAC3B,QAAQ,CAAC,WAAW,CAAC,EAAE,KAAK,CAAC;CAC9B;AAED;gCACgC;AAChC,MAAM,WAAW,YAAa,SAAQ,eAAe,EAAE,UAAU;IAC/D,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IACzB,wEAAwE;IACxE,QAAQ,CAAC,SAAS,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,KAAK,MAAM,CAAC;IAC/C,QAAQ,CAAC,WAAW,CAAC,EAAE,KAAK,CAAC;CAC9B;AAED,gFAAgF;AAChF,MAAM,WAAW,cAAe,SAAQ,eAAe,EAAE,UAAU;IACjE,QAAQ,CAAC,IAAI,EAAE,WAAW,CAAC;IAC3B,uCAAuC;IACvC,QAAQ,CAAC,SAAS,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,KAAK,MAAM,CAAC;IAC/C,QAAQ,CAAC,WAAW,CAAC,EAAE,KAAK,CAAC;IAC7B;;;;;;;OAOG;IACH,QAAQ,CAAC,WAAW,CAAC,EAAE,MAAM,CAAC;IAC9B;;0DAEsD;IACtD,QAAQ,CAAC,GAAG,CAAC,EAAE;QACb;;;;;qCAK6B;QAC7B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;QAC1B;;;;6EAIqE;QACrE,QAAQ,CAAC,QAAQ,EAAE,WAAW,GAAG,QAAQ,GAAG,QAAQ,CAAC;KACtD,CAAC;CACH;AAED;;;;;;;GAOG;AACH,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,WAAW,CAAC,EAAE,SAAS,UAAU,EAAE,CAAC;CAC9C;AAED;;;;;GAKG;AACH,MAAM,WAAW,YAAa,SAAQ,eAAe,EAAE,cAAc;IACnE,QAAQ,CAAC,IAAI,EAAE,SAAS,GAAG,QAAQ,GAAG,MAAM,CAAC;IAC7C,QAAQ,CAAC,UAAU,CAAC,EAAE,KAAK,CAAC;CAC7B;AAED,0DAA0D;AAC1D,MAAM,WAAW,YAAa,SAAQ,eAAe;IACnD,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IACzB,QAAQ,CAAC,UAAU,CAAC,EAAE,KAAK,CAAC;IAC5B,QAAQ,CAAC,WAAW,CAAC,EAAE,KAAK,CAAC;CAC9B;AAED;;;;;GAKG;AACH,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;IACrB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;CACvB;AAED;;;;;;;GAOG;AACH,MAAM,WAAW,UAAU;IACzB;;;;+CAI2C;IAC3C,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB;2DACuD;IACvD,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,MAAM,EAAE,SAAS,WAAW,EAAE,CAAC;CACzC;AAED;;;;;;;GAOG;AACH,wBAAgB,gBAAgB,CAAC,KAAK,CAAC,IAAI,SAAS,UAAU,EAC5D,UAAU,EAAE,IAAI,GACf,IAAI,CAEN;AAED;;;;;;GAMG;AACH;;;uCAGuC;AACvC,MAAM,CAAC,OAAO,CAAC,MAAM,iBAAiB,EAAE,OAAO,MAAM,CAAC;AAEtD,MAAM,WAAW,YAAY,CAC3B,KAAK,SAAS,SAAS,UAAU,EAAE,GAAG,SAAS,UAAU,EAAE,CAC3D,SAAQ,WAAW,CAAC,MAAM,EAAE,KAAK,CAAC,MAAM,CAAC,CAAC;IAC1C,QAAQ,CAAC,CAAC,iBAAiB,CAAC,EAAE,IAAI,CAAC;CACpC;AAED;;;;;;;;;GASG;AACH,wBAAgB,YAAY,CAAC,KAAK,CAAC,KAAK,SAAS,SAAS,UAAU,EAAE,EACpE,GAAG,KAAK,EAAE,KAAK,GACd,YAAY,CAAC,KAAK,CAAC,CAuBrB;AAED;;;;;;;GAOG;AACH,wBAAgB,YAAY,CAAC,UAAU,EAAE,UAAU,GAAG,SAAS,GAAG,SAAS,CAQ1E;AAsCD;;;;;GAKG;AACH,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,MAAM,EACX,sBAAsB,GACtB,oBAAoB,GACpB,gBAAgB,GAChB,aAAa,GACb,iBAAiB,GACjB,uBAAuB,GACvB,qBAAqB,GACrB,0BAA0B,GAC1B,wBAAwB,GACxB,uBAAuB,GACvB,kBAAkB,GAClB,qBAAqB,GACrB,oBAAoB,CAAC;CAC1B;AA0BD;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA4BG;AACH,wBAAgB,kBAAkB,CAChC,UAAU,EAAE,UAAU,GACrB,SAAS,eAAe,EAAE,CA+E5B;AAWD;;;;;GAKG;AACH,wBAAgB,kBAAkB,CAChC,MAAM,EAAE,YAAY,EACpB,UAAU,EAAE,UAAU,GACrB,IAAI,CAUN;AAED;+EAC+E;AAC/E,wBAAgB,qBAAqB,CAAC,UAAU,EAAE,UAAU,GAAG,IAAI,CAQlE;AAED;;;;;GAKG;AACH,MAAM,WAAW,cAAc;IAC7B;sEACkE;IAClE,QAAQ,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC;IACnC;4EACwE;IACxE,QAAQ,CAAC,IAAI,EAAE,SAAS,MAAM,EAAE,CAAC;CAClC;AAED;;;;;;;;;;;;;;;;;;;GAmBG;AACH,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,SAAS,EAAE,IAAI,EAAE,MAAM,GAAG,MAAM,CAEvE;AAED;;;;;;GAMG;AACH,wBAAgB,mBAAmB,CAAC,KAAK,EAAE,SAAS,GAAG,MAAM,GAAG,SAAS,CAExE;AAED;;;;;;GAMG;AACH,wBAAgB,aAAa,CAC3B,KAAK,EAAE,SAAS,EAChB,GAAG,EAAE,MAAM,GACV,MAAM,GAAG,SAAS,CAOpB;AAED;;;;GAIG;AACH,wBAAgB,gBAAgB,CAC9B,UAAU,EAAE,UAAU,GACrB,SAAS,CAAC,WAAW,GAAG;IACzB,QAAQ,CAAC,UAAU,EAAE;QAAE,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;CAClD,CAAC,EAAE,CAOH;AAED,8DAA8D;AAC9D,wBAAgB,eAAe,CAC7B,UAAU,EAAE,UAAU,GACrB,SAAS,WAAW,EAAE,CAExB;AAED,sDAAsD;AACtD,wBAAgB,gBAAgB,CAC9B,UAAU,EAAE,UAAU,GACrB,SAAS,WAAW,EAAE,CAExB;AAED,qEAAqE;AACrE,wBAAgB,cAAc,CAAC,UAAU,EAAE,UAAU,GAAG,SAAS,WAAW,EAAE,CAE7E;AAED,uEAAuE;AACvE,wBAAgB,YAAY,CAAC,UAAU,EAAE,UAAU,GAAG,SAAS,WAAW,EAAE,CAE3E;AAED,qFAAqF;AACrF,wBAAgB,eAAe,CAC7B,UAAU,EAAE,UAAU,GACrB,SAAS,cAAc,EAAE,CAE3B;AAED,2CAA2C;AAC3C,wBAAgB,UAAU,CACxB,UAAU,EAAE,UAAU,EACtB,IAAI,EAAE,MAAM,GACX,WAAW,GAAG,SAAS,CAEzB;AAED;;;;;;GAMG;AACH,wBAAgB,YAAY,CAC1B,KAAK,EAAE,WAAW,GACjB,KAAK,IAAI,YAAY,GAAG;IAAE,QAAQ,CAAC,WAAW,EAAE,SAAS,UAAU,EAAE,CAAA;CAAE,CAEzE;AAED;;;;;;GAMG;AACH,wBAAgB,gBAAgB,CAAC,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAGhE;AAED,+EAA+E;AAC/E,wBAAgB,gBAAgB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,CAExD;AAED;;;;;GAKG;AACH,wBAAgB,cAAc,CAAC,KAAK,EAAE,WAAW,GAAG,cAAc,CAgBjE"}
|
package/dist/schema.js
CHANGED
|
@@ -9,7 +9,7 @@ const OPERATOR_BY_KIND = {
|
|
|
9
9
|
};
|
|
10
10
|
/**
|
|
11
11
|
* The `where` operator a field of this kind accepts (per the ADR filter-semantics
|
|
12
|
-
* table), or `undefined` for `text`
|
|
12
|
+
* table), or `undefined` for `text` – which feeds the free-text `query` rather
|
|
13
13
|
* than `where`. The ONE source for the surface’s `where` input type, the
|
|
14
14
|
* adapter’s filter compiler and declaration validation, so they cannot drift.
|
|
15
15
|
*/
|
|
@@ -19,7 +19,7 @@ export function filterOperatorFor(kind) {
|
|
|
19
19
|
/**
|
|
20
20
|
* Declare a {@link SearchType}, capturing it as a literal: the `const` type
|
|
21
21
|
* parameter preserves the field names and capability flags that the type-level
|
|
22
|
-
* helpers (`FacetFieldsOf`, `OutputFieldsOf`) read off the type
|
|
22
|
+
* helpers (`FacetFieldsOf`, `OutputFieldsOf`) read off the type –
|
|
23
23
|
* with none of the widening a plain `: SearchType` annotation causes and
|
|
24
24
|
* without having to remember `as const satisfies SearchType`. Identity at
|
|
25
25
|
* runtime.
|
|
@@ -28,11 +28,11 @@ export function defineSearchType(searchType) {
|
|
|
28
28
|
return searchType;
|
|
29
29
|
}
|
|
30
30
|
/**
|
|
31
|
-
* Build a {@link SearchSchema} from root-type declarations, keyed by `
|
|
31
|
+
* Build a {@link SearchSchema} from root-type declarations, keyed by `class`.
|
|
32
32
|
*
|
|
33
|
-
* Every declaration is validated ({@link assertValidSearchType})
|
|
34
|
-
* declaration-time counterpart of the port’s `assertValidQuery`
|
|
35
|
-
* schema-wide invariants are enforced: no two types may share a `
|
|
33
|
+
* Every declaration is validated ({@link assertValidSearchType}) – the
|
|
34
|
+
* declaration-time counterpart of the port’s `assertValidQuery` – and the
|
|
35
|
+
* schema-wide invariants are enforced: no two types may share a `class` IRI
|
|
36
36
|
* (they would silently overwrite each other in the map) or a `name` (names
|
|
37
37
|
* key the API surfaces). Throws on the first invalid declaration, so a bad
|
|
38
38
|
* schema fails at startup, not per document at index time or per query.
|
|
@@ -42,18 +42,18 @@ export function searchSchema(...types) {
|
|
|
42
42
|
const names = new Set();
|
|
43
43
|
for (const searchType of types) {
|
|
44
44
|
assertValidSearchType(searchType);
|
|
45
|
-
if (typeIris.has(searchType.
|
|
46
|
-
throw new Error(`Duplicate search type IRI “${searchType.
|
|
45
|
+
if (typeIris.has(searchType.class)) {
|
|
46
|
+
throw new Error(`Duplicate search type IRI “${searchType.class}”; each SearchType must declare a distinct class.`);
|
|
47
47
|
}
|
|
48
48
|
if (names.has(searchType.name)) {
|
|
49
49
|
throw new Error(`Duplicate search type name “${searchType.name}”; each SearchType must declare a distinct name.`);
|
|
50
50
|
}
|
|
51
|
-
typeIris.add(searchType.
|
|
51
|
+
typeIris.add(searchType.class);
|
|
52
52
|
names.add(searchType.name);
|
|
53
53
|
}
|
|
54
54
|
assertResolvableLabelSources(types);
|
|
55
55
|
// The one blessed cast: only this validated constructor mints the brand.
|
|
56
|
-
return new Map(types.map((searchType) => [searchType.
|
|
56
|
+
return new Map(types.map((searchType) => [searchType.class, searchType]));
|
|
57
57
|
}
|
|
58
58
|
/**
|
|
59
59
|
* The text field a label source serves labels from – the ‘label’ convention
|
|
@@ -104,18 +104,38 @@ const SEARCHABLE_KINDS = ['text', 'keyword', 'reference'];
|
|
|
104
104
|
/** Kinds whose projection applies the {@link KeywordField.transform}. */
|
|
105
105
|
const TRANSFORMABLE_KINDS = ['keyword', 'reference'];
|
|
106
106
|
/**
|
|
107
|
-
*
|
|
107
|
+
* A safe logical field name: a GraphQL-style identifier. The name is
|
|
108
|
+
* interpolated raw into physical field names AND, for a display text field,
|
|
109
|
+
* into the RE2 collection pattern `${name}_[^_]+` ({@link displayFieldPattern}),
|
|
110
|
+
* so it must contain no regex metacharacter – this charset (letters, digits,
|
|
111
|
+
* `_`) guarantees that, and is exactly what a GraphQL field name allows anyway.
|
|
112
|
+
*/
|
|
113
|
+
const FIELD_NAME_PATTERN = /^[A-Za-z_][A-Za-z0-9_]*$/;
|
|
114
|
+
/**
|
|
115
|
+
* A safe declared locale: BCP-47-shaped (letters/digits, `-` between subtags),
|
|
116
|
+
* never containing `_`. The `_` is the reserved separator between a text field’s
|
|
117
|
+
* name and its language subtag, so a locale carrying one would collide with the
|
|
118
|
+
* `${name}_search_${locale}` / display naming; incoming data tags are normalised
|
|
119
|
+
* to this shape at projection time.
|
|
120
|
+
*/
|
|
121
|
+
const LOCALE_PATTERN = /^[A-Za-z0-9]+(-[A-Za-z0-9]+)*$/;
|
|
122
|
+
/**
|
|
123
|
+
* Structurally validate one {@link SearchType} declaration – the
|
|
108
124
|
* declaration-time counterpart of `validateQuery`. Rules:
|
|
109
125
|
*
|
|
110
126
|
* - field names are unique (a duplicate would silently shadow in every
|
|
111
|
-
* consumer, each picking a different winner)
|
|
127
|
+
* consumer, each picking a different winner) and a metacharacter-free
|
|
128
|
+
* identifier (the name is interpolated into physical field names and the
|
|
129
|
+
* display RE2 pattern);
|
|
130
|
+
* - a `text` field’s declared locales are BCP-47-shaped (no `_`, which is the
|
|
131
|
+
* reserved name↔locale separator);
|
|
112
132
|
* - a `reference` field that is `output` declares `ref` (the API surfaces
|
|
113
133
|
* need the reference type name); `ref` on any other kind is meaningless;
|
|
114
134
|
* - a `text` field declares at least one locale (`und` = untagged; projection and
|
|
115
|
-
* result reconstruction have no representation for unlocalized text
|
|
135
|
+
* result reconstruction have no representation for unlocalized text – use
|
|
116
136
|
* `keyword` for untagged strings); `locales` on any other kind is
|
|
117
137
|
* meaningless;
|
|
118
|
-
* - a kind without a `where` operator (`text`
|
|
138
|
+
* - a kind without a `where` operator (`text` – it feeds the free-text query)
|
|
119
139
|
* is neither `filterable` nor `facetable`;
|
|
120
140
|
* - `facetRanges` only on the `range`-operator kinds (`integer`/`number`/`date`);
|
|
121
141
|
* - `searchable` only on `text`/`keyword`/`reference` (projection emits no
|
|
@@ -141,6 +161,11 @@ export function validateSearchType(searchType) {
|
|
|
141
161
|
issue('duplicate-field-name');
|
|
142
162
|
}
|
|
143
163
|
seen.add(field.name);
|
|
164
|
+
// The name is interpolated into physical field names and the display RE2
|
|
165
|
+
// pattern, so it must be a metacharacter-free identifier.
|
|
166
|
+
if (!FIELD_NAME_PATTERN.test(field.name)) {
|
|
167
|
+
issue('invalid-field-name');
|
|
168
|
+
}
|
|
144
169
|
if (field.kind === 'reference') {
|
|
145
170
|
if (field.output === true && field.ref === undefined) {
|
|
146
171
|
issue('missing-ref');
|
|
@@ -153,6 +178,12 @@ export function validateSearchType(searchType) {
|
|
|
153
178
|
if ((field.locales ?? []).length === 0) {
|
|
154
179
|
issue('text-requires-locales');
|
|
155
180
|
}
|
|
181
|
+
// A locale carrying `_` would collide with the name/locale separator in
|
|
182
|
+
// the physical and display field naming, so declared locales are
|
|
183
|
+
// BCP-47-shaped (data tags are normalised to match at projection time).
|
|
184
|
+
if ((field.locales ?? []).some((locale) => !LOCALE_PATTERN.test(locale))) {
|
|
185
|
+
issue('invalid-locale');
|
|
186
|
+
}
|
|
156
187
|
}
|
|
157
188
|
else if (field.locales !== undefined) {
|
|
158
189
|
issue('locales-not-allowed');
|
|
@@ -187,13 +218,13 @@ export function validateSearchType(searchType) {
|
|
|
187
218
|
return issues;
|
|
188
219
|
}
|
|
189
220
|
/**
|
|
190
|
-
* Throw when `searchType` is not a member of `schema`
|
|
221
|
+
* Throw when `searchType` is not a member of `schema` – the port membership
|
|
191
222
|
* guard every engine adapter applies before searching, so a query can never
|
|
192
223
|
* meet an index the deployment did not declare. Identity-based: the exact
|
|
193
224
|
* declaration object must be in the schema, not a lookalike.
|
|
194
225
|
*/
|
|
195
226
|
export function assertTypeInSchema(schema, searchType) {
|
|
196
|
-
if (schema.get(searchType.
|
|
227
|
+
if (schema.get(searchType.class) !== searchType) {
|
|
197
228
|
throw new Error(`Search type “${searchType.name}” is not in this engine’s schema; it serves ${[
|
|
198
229
|
...schema.values(),
|
|
199
230
|
]
|
|
@@ -213,7 +244,55 @@ export function assertValidSearchType(searchType) {
|
|
|
213
244
|
}
|
|
214
245
|
}
|
|
215
246
|
/**
|
|
216
|
-
*
|
|
247
|
+
* The display fields of a localized `text` field are pattern-based, not
|
|
248
|
+
* enumerated per declared locale: projection stores `${name}_${lang}` for
|
|
249
|
+
* **every** language present in the data (not only those in `locales`), so a
|
|
250
|
+
* label in an undeclared language or an untagged one still renders rather than
|
|
251
|
+
* collapsing to a bare IRI. `locales` governs only the indexed search/sort
|
|
252
|
+
* fanout ({@link PhysicalFields}); display costs nothing per language (stored
|
|
253
|
+
* `index: false`), so it preserves them all. A deployment that wants fewer
|
|
254
|
+
* display languages restricts them upstream (e.g. in its CONSTRUCT).
|
|
255
|
+
*
|
|
256
|
+
* A language subtag never contains `_`, but the `search_`/`sort_` infixes do,
|
|
257
|
+
* so `${name}_${lang}` with `lang` matching `[^_]+` is unambiguously a display
|
|
258
|
+
* field: `label_nl`, `label_fr`, `label_zh-hant`, `label_und` are display;
|
|
259
|
+
* `label_search_nl` and `label_sort_nl` are not. This trio – the name a value
|
|
260
|
+
* is written under ({@link displayFieldName}), the collection pattern that
|
|
261
|
+
* accepts them all ({@link displayFieldPattern}), and the reader that recovers a
|
|
262
|
+
* key’s language ({@link displayLangOf}) – is the single home of that
|
|
263
|
+
* convention, so projection, collection-definition and result reconstruction
|
|
264
|
+
* cannot disagree.
|
|
265
|
+
*/
|
|
266
|
+
export function displayFieldName(field, lang) {
|
|
267
|
+
return `${field.name}_${lang}`;
|
|
268
|
+
}
|
|
269
|
+
/**
|
|
270
|
+
* The RE2 pattern a collection declares to store every present language’s
|
|
271
|
+
* display value un-indexed, or `undefined` when the field is not `output` (no
|
|
272
|
+
* display at all). Matches `${name}_${lang}` for any underscore-free `lang`, so
|
|
273
|
+
* it never collides with the field’s `${name}_search_${locale}` /
|
|
274
|
+
* `${name}_sort_${locale}` companions.
|
|
275
|
+
*/
|
|
276
|
+
export function displayFieldPattern(field) {
|
|
277
|
+
return field.output ? `${field.name}_[^_]+` : undefined;
|
|
278
|
+
}
|
|
279
|
+
/**
|
|
280
|
+
* The language a stored document key carries for `field`’s display, or
|
|
281
|
+
* `undefined` when the key is not one of `field`’s display fields – the inverse
|
|
282
|
+
* of {@link displayFieldName}. A key qualifies when it is `${name}_` followed by
|
|
283
|
+
* an underscore-free remainder, so the `${name}_search_…`/`${name}_sort_…`
|
|
284
|
+
* companions (and unrelated fields) are rejected.
|
|
285
|
+
*/
|
|
286
|
+
export function displayLangOf(field, key) {
|
|
287
|
+
const prefix = `${field.name}_`;
|
|
288
|
+
if (!key.startsWith(prefix)) {
|
|
289
|
+
return undefined;
|
|
290
|
+
}
|
|
291
|
+
const lang = key.slice(prefix.length);
|
|
292
|
+
return lang.length > 0 && !lang.includes('_') ? lang : undefined;
|
|
293
|
+
}
|
|
294
|
+
/**
|
|
295
|
+
* Full-text searchable fields, highest `query_by` weight first – the order the
|
|
217
296
|
* engine adapter weights `query_by` in. A field is searchable iff it carries a
|
|
218
297
|
* `searchable` weight.
|
|
219
298
|
*/
|
|
@@ -271,14 +350,16 @@ export function isoToUnixSeconds(iso) {
|
|
|
271
350
|
export function unixSecondsToIso(seconds) {
|
|
272
351
|
return new Date(seconds * 1000).toISOString();
|
|
273
352
|
}
|
|
274
|
-
/**
|
|
353
|
+
/**
|
|
354
|
+
* Derive the **indexed** physical engine field names a declaration produces:
|
|
355
|
+
* the per-locale `search`/`sort` fanout. A localized `text` field’s display
|
|
356
|
+
* fields are pattern-based and not enumerated here – see {@link displayFieldName}
|
|
357
|
+
* and its siblings.
|
|
358
|
+
*/
|
|
275
359
|
export function physicalFields(field) {
|
|
276
360
|
if (field.kind === 'text') {
|
|
277
361
|
const locales = field.locales;
|
|
278
362
|
return {
|
|
279
|
-
display: field.output
|
|
280
|
-
? locales.map((locale) => `${field.name}_${locale}`)
|
|
281
|
-
: [],
|
|
282
363
|
search: field.searchable
|
|
283
364
|
? locales.map((locale) => `${field.name}_search_${locale}`)
|
|
284
365
|
: [],
|
|
@@ -288,7 +369,6 @@ export function physicalFields(field) {
|
|
|
288
369
|
};
|
|
289
370
|
}
|
|
290
371
|
return {
|
|
291
|
-
display: [],
|
|
292
372
|
search: field.searchable !== undefined ? [`${field.name}_search`] : [],
|
|
293
373
|
sort: [],
|
|
294
374
|
};
|
package/dist/testing.js
CHANGED
|
@@ -32,13 +32,13 @@ export function describeSearchEngineContract(name, engine) {
|
|
|
32
32
|
expect(types().length).toBeGreaterThan(0);
|
|
33
33
|
for (const searchType of types()) {
|
|
34
34
|
expect(typeof searchType.name).toBe('string');
|
|
35
|
-
expect(typeof searchType.
|
|
35
|
+
expect(typeof searchType.class).toBe('string');
|
|
36
36
|
}
|
|
37
37
|
});
|
|
38
38
|
it('rejects a search type outside its schema', async () => {
|
|
39
39
|
const foreign = {
|
|
40
40
|
name: 'NotInSchema',
|
|
41
|
-
|
|
41
|
+
class: 'urn:test:not-in-schema',
|
|
42
42
|
fields: [],
|
|
43
43
|
};
|
|
44
44
|
await expect(engine().search(foreign, browse(foreign))).rejects.toThrow(/not in this engine/);
|
|
@@ -78,7 +78,7 @@ export function describeSearchEngineContract(name, engine) {
|
|
|
78
78
|
it('rejects a searchFacets batch for a type outside its schema', async () => {
|
|
79
79
|
const foreign = {
|
|
80
80
|
name: 'NotInSchema',
|
|
81
|
-
|
|
81
|
+
class: 'urn:test:not-in-schema',
|
|
82
82
|
fields: [],
|
|
83
83
|
};
|
|
84
84
|
await expect(engine().searchFacets(foreign, [browse(foreign)])).rejects.toThrow(/not in this engine/);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@lde/search",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.7.0",
|
|
4
4
|
"description": "Engine- and domain-agnostic core for RDF-backed search: a unified declarative field model (SearchField/SearchType/SearchSchema), a neutral query IR, the SearchEngine port with logical result types, and a streaming CONSTRUCT-to-document projection. Bakes in no engine, protocol, or domain.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"url": "git+https://github.com/ldelements/lde.git",
|