@lde/search 0.1.1 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +249 -119
- package/dist/adapter.d.ts +13 -0
- package/dist/adapter.d.ts.map +1 -0
- package/dist/adapter.js +10 -0
- package/dist/engine.d.ts +124 -0
- package/dist/engine.d.ts.map +1 -0
- package/dist/engine.js +1 -0
- package/dist/index.d.ts +5 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +12 -0
- package/dist/project.d.ts +12 -84
- package/dist/project.d.ts.map +1 -1
- package/dist/project.js +91 -53
- package/dist/query.d.ts +87 -0
- package/dist/query.d.ts.map +1 -0
- package/dist/query.js +89 -0
- package/dist/schema.d.ts +343 -0
- package/dist/schema.d.ts.map +1 -0
- package/dist/schema.js +250 -0
- package/dist/testing.d.ts +19 -0
- package/dist/testing.d.ts.map +1 -0
- package/dist/testing.js +106 -0
- package/package.json +24 -4
package/README.md
CHANGED
|
@@ -1,170 +1,300 @@
|
|
|
1
1
|
# @lde/search
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
3
|
+
The core of the LDE search family: packages that together act as a **generator
|
|
4
|
+
for search engines**. You write one declarative `SearchSchema`, and everything
|
|
5
|
+
a running search engine needs is derived from it: the document projection, the
|
|
6
|
+
engine collection schema, the query semantics, and the API surface. All these
|
|
7
|
+
are kept in sync automatically rather than handwritten per deployment.
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
|
|
9
|
+
The core itself is **engine-, API- and domain-agnostic**: it bakes in no search
|
|
10
|
+
engine, no API protocol, and no domain vocabulary. The engine- and API-specific
|
|
11
|
+
halves are adapters that plug into the ports defined here:
|
|
11
12
|
|
|
12
|
-
|
|
13
|
-
|
|
13
|
+
- **engine adapters** implement the `SearchEngine` port:
|
|
14
|
+
[`@lde/search-typesense`](../search-typesense), with OpenSearch to follow;
|
|
15
|
+
- **API surfaces** drive it, parsing client input into `search(SearchQuery)`:
|
|
16
|
+
[`@lde/search-api-graphql`](../search-api-graphql), with a REST
|
|
17
|
+
surface to follow.
|
|
14
18
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
};
|
|
19
|
+
The library never names your domain: the same core drives a `Dataset`,
|
|
20
|
+
`Person`, or `CreativeWork` search.
|
|
18
21
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
+
It provides four things:
|
|
23
|
+
|
|
24
|
+
- **unified field model** — `SearchField` / `SearchType` / `SearchSchema`:
|
|
25
|
+
one declaration per field that drives all four consumers below, so they
|
|
26
|
+
cannot drift;
|
|
27
|
+
- **neutral query IR** — `SearchQuery` / `Filter` / `Sort` + filter
|
|
28
|
+
semantics: every API surface compiles into it, every engine adapter compiles
|
|
29
|
+
out of it, so the two cannot drift;
|
|
30
|
+
- **engine port** — `SearchEngine` and the logical result types
|
|
31
|
+
(`SearchResult` / `SearchHit` / `ResultDocument` / `Reference` / …);
|
|
32
|
+
- **streaming projection** — `projectGraph`, RDF `CONSTRUCT` quads → flat
|
|
33
|
+
search documents.
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
SearchSchema ─┬─► projection (projectGraph → flat documents) [here]
|
|
37
|
+
├─► engine adapter (collection schema + query compiler) e.g. @lde/search-typesense
|
|
38
|
+
├─► query semantics (SearchQuery, filter/sort/facet) [here]
|
|
39
|
+
└─► API surface (GraphQL / REST) e.g. @lde/search-api-graphql
|
|
22
40
|
```
|
|
23
41
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
triples are collapsed first, because some SPARQL engines (e.g. QLever) do not
|
|
28
|
-
deduplicate `CONSTRUCT` output. The IR carries no `@context`, so a `derivation`
|
|
29
|
-
reading it sees full predicate IRIs with language tags preserved.
|
|
42
|
+
At runtime, everything those consumers do is a **pure transformation between
|
|
43
|
+
data shapes**, each one parameterised by the schema — three chains, meeting at
|
|
44
|
+
the engine:
|
|
30
45
|
|
|
31
|
-
|
|
46
|
+
```
|
|
47
|
+
indexing: RDF quads ──frame──► FramedNode ──project──► SearchDocument ──import──► engine
|
|
48
|
+
querying: client input ──parse──► SearchQuery ──compile──► engine query
|
|
49
|
+
results: engine response ──parse──► SearchResult ──shape──► API output
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
Validation happens before the first arrow (SHACL over the RDF) and inside the
|
|
53
|
+
last (the engine enforces its collection schema); between them every stage is
|
|
54
|
+
a typed, deterministic function — easy to test, and swappable per deployment.
|
|
55
|
+
|
|
56
|
+
## Entry points
|
|
57
|
+
|
|
58
|
+
Exports are stratified by audience:
|
|
59
|
+
|
|
60
|
+
- **`@lde/search`** — the authoring surface: `defineSearchType`,
|
|
61
|
+
`searchSchema`, `projectGraph` (+ the IR readers `derive` functions use),
|
|
62
|
+
validation, and every model/query/result type.
|
|
63
|
+
- **`@lde/search/adapter`** — plumbing for engine adapters and API surfaces:
|
|
64
|
+
`physicalFields`, the field selectors, `assertValidQuery`, the filter
|
|
65
|
+
operators and storage codecs.
|
|
66
|
+
- **`@lde/search/testing`** — `describeSearchEngineContract`, the executable
|
|
67
|
+
port contract every engine adapter runs against a live instance of itself
|
|
68
|
+
(vitest; optional peer).
|
|
69
|
+
|
|
70
|
+
## Terminology
|
|
71
|
+
|
|
72
|
+
The model has three levels, with analogues in SHACL ([one possible source](#why-a-declarative-model))
|
|
73
|
+
and GraphQL (one of the surfaces):
|
|
74
|
+
|
|
75
|
+
| Term | What it is | SHACL | GraphQL |
|
|
76
|
+
| -------------- | --------------------------------------------------------------------------------------------------------------- | -------------- | ----------- |
|
|
77
|
+
| `SearchField` | One queryable field: a `kind`, the IR `path` it projects from, and the capability flags it opts into | property shape | field |
|
|
78
|
+
| `SearchType` | One root type’s complete declaration: its logical API `name`, its `type` IRI and its fields (incl. derived) | NodeShape | object type |
|
|
79
|
+
| `SearchSchema` | The whole search declaration: every `SearchType`, keyed by `type` IRI — build one with `searchSchema(...types)` | shapes graph | schema |
|
|
80
|
+
|
|
81
|
+
`projectGraph` and the GraphQL surface consume a `SearchSchema` (projecting
|
|
82
|
+
every type in one pass); the engine port executes one `SearchType` at a time.
|
|
83
|
+
|
|
84
|
+
### API conventions
|
|
32
85
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
86
|
+
Two conventions hold across the whole family:
|
|
87
|
+
|
|
88
|
+
- **Parameter order** — a function takes the value it operates on first and
|
|
89
|
+
the declaration right after it: `projectDocument(node, type)`,
|
|
90
|
+
`buildSearchParams(query, type)`,
|
|
91
|
+
`createTypesenseSearchEngine(client, schema, options)`,
|
|
92
|
+
`engine.search(type, query)`.
|
|
93
|
+
- **Factory verbs** — the verb tells you what kind of thing comes back.
|
|
94
|
+
`define*` captures a declaration as a literal (`defineSearchType`);
|
|
95
|
+
`build*` is a pure data-to-data constructor (`buildCollectionSchema`,
|
|
96
|
+
`buildSearchParams`, `buildGraphQLSchema`); `create*` makes a stateful
|
|
97
|
+
instance (`createTypesenseSearchEngine`). A bare noun (`searchSchema`)
|
|
98
|
+
constructs the trivial container it names.
|
|
99
|
+
|
|
100
|
+
## Field model
|
|
101
|
+
|
|
102
|
+
The mapping is data, not code. Each field declares its `kind`, the IR `path` to
|
|
103
|
+
read (or a `derive` function for a **derived** field, computed from the framed
|
|
104
|
+
node in declaration order — so it may read fields declared before it), and the
|
|
105
|
+
capabilities it opts into. The physical field names a declaration fans out to
|
|
106
|
+
(per-locale search/sort keys) come from
|
|
107
|
+
`physicalFields`, the single convention projection, the collection schema and the
|
|
108
|
+
query compiler all share.
|
|
38
109
|
|
|
39
110
|
```ts
|
|
40
|
-
import {
|
|
111
|
+
import {
|
|
112
|
+
defineSearchType,
|
|
113
|
+
projectGraph,
|
|
114
|
+
irisOf,
|
|
115
|
+
searchSchema,
|
|
116
|
+
} from '@lde/search';
|
|
41
117
|
|
|
42
|
-
const
|
|
118
|
+
const DATASET = defineSearchType({
|
|
119
|
+
name: 'Dataset', // logical API name: names the GraphQL type, a REST path, …
|
|
43
120
|
type: 'http://www.w3.org/ns/dcat#Dataset',
|
|
44
121
|
fields: [
|
|
45
|
-
// → title_nl, title_en, title_search_nl,
|
|
122
|
+
// → title_nl, title_en, title_search_nl/_en, title_sort_nl/_en
|
|
46
123
|
{
|
|
47
124
|
name: 'title',
|
|
48
125
|
path: 'http://purl.org/dc/terms/title',
|
|
49
|
-
kind:
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
sort: true,
|
|
55
|
-
},
|
|
126
|
+
kind: 'text',
|
|
127
|
+
locales: ['nl', 'en'],
|
|
128
|
+
output: true,
|
|
129
|
+
searchable: { weight: 5 },
|
|
130
|
+
sortable: true,
|
|
56
131
|
},
|
|
57
|
-
// → publisher (IRI facet)
|
|
132
|
+
// → publisher (IRI facet, resolved to a labelled reference at the surface)
|
|
58
133
|
{
|
|
59
134
|
name: 'publisher',
|
|
60
135
|
path: 'http://purl.org/dc/terms/publisher',
|
|
61
|
-
kind:
|
|
136
|
+
kind: 'reference',
|
|
137
|
+
facetable: true,
|
|
138
|
+
output: true,
|
|
139
|
+
ref: { typeName: 'Organization', strategy: 'labelOnly' },
|
|
62
140
|
},
|
|
63
141
|
// → size (int)
|
|
64
|
-
{ name: 'size', path: 'urn:dr:size', kind:
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
142
|
+
{ name: 'size', path: 'urn:dr:size', kind: 'integer', sortable: true },
|
|
143
|
+
// derived field (no path): computed from the framed node
|
|
144
|
+
{
|
|
145
|
+
name: 'classCount',
|
|
146
|
+
kind: 'integer',
|
|
147
|
+
sortable: true,
|
|
148
|
+
derive: (node) => irisOf(node, 'urn:dr:class').length,
|
|
69
149
|
},
|
|
70
150
|
],
|
|
71
|
-
};
|
|
151
|
+
});
|
|
72
152
|
|
|
73
|
-
for await (const document of projectGraph(quads,
|
|
74
|
-
//
|
|
153
|
+
for await (const document of projectGraph(quads, searchSchema(DATASET))) {
|
|
154
|
+
// one flat search document per matching subject, streamed
|
|
75
155
|
}
|
|
76
156
|
```
|
|
77
157
|
|
|
78
|
-
|
|
158
|
+
`defineSearchType` captures the declaration as a literal (what
|
|
159
|
+
`as const satisfies SearchType` would do manually, with nothing to remember),
|
|
160
|
+
so typed facet/output keys can be derived from it — see
|
|
161
|
+
[Typed results](#typed-results) and `@lde/search-api-graphql`.
|
|
162
|
+
|
|
163
|
+
**Kinds** (`FieldKind`): `text`, `keyword`, `integer`, `number`, `boolean`,
|
|
164
|
+
`date`, `reference`. The Typesense/engine vocabulary and the GraphQL types are
|
|
165
|
+
_derived_ from the kind by the adapter and the surface — never declared here.
|
|
166
|
+
|
|
167
|
+
`SearchField` is a **discriminated union by `kind`** (`TextField`,
|
|
168
|
+
`KeywordField`, `ReferenceField`, `NumericField`, `BooleanField`): each kind
|
|
169
|
+
declares exactly the properties it can honour — `locales` on text, `ref` on
|
|
170
|
+
references, `facetRanges` on numerics — so an illegal declaration fails to
|
|
171
|
+
compile. Text is **always multilingual in shape**: `locales` lists the
|
|
172
|
+
language tags to fan out, and the reserved **`und`** locale (JSON-LD `@none`,
|
|
173
|
+
RDF `und`) buckets untagged literals — a monolingual or untagged corpus
|
|
174
|
+
declares `locales: ['und']`, mixed data `['nl', 'und']`. Declaring a real
|
|
175
|
+
language is recommended (it drives per-locale stemming); `und` is folded but
|
|
176
|
+
unstemmed unless `defaultLocale` opts in, is never demoted in search
|
|
177
|
+
weighting, and adding a language later is additive — the API output shape
|
|
178
|
+
never changes. Use `keyword` for exact-match tokens, never for prose.
|
|
179
|
+
|
|
180
|
+
**Declarations are also validated at runtime** (for declarations built
|
|
181
|
+
outside TypeScript — a SHACL generator, plain JS): `searchSchema()`
|
|
182
|
+
rejects a structurally invalid declaration (duplicate field names, an `output`
|
|
183
|
+
reference without `ref`, `text` without locales, `locales` on a non-text kind,
|
|
184
|
+
`facetRanges` on a non-numeric kind, `searchable`/`transform` on a kind whose
|
|
185
|
+
projection cannot honour it, `filterable`/`facetable` on `text`, two types
|
|
186
|
+
sharing a `type` IRI or `name`) — the declaration-time counterpart of the
|
|
187
|
+
port’s always-on query validation, so a bad schema fails at startup rather
|
|
188
|
+
than per document at index time. `validateSearchType` /
|
|
189
|
+
`assertValidSearchType` are exported for validating a single declaration
|
|
190
|
+
directly.
|
|
191
|
+
|
|
192
|
+
| kind | `where` | facet | sort | output |
|
|
193
|
+
| -------------------- | -------------------- | ----- | ---------------- | ------------------------------- |
|
|
194
|
+
| `text` | – (feeds free text) | – | yes (per-locale) | best-first language list |
|
|
195
|
+
| `keyword` | `in` (membership) | yes | – | string / `string[]` |
|
|
196
|
+
| `reference` | `in` (membership) | yes | – | labelled reference (id + label) |
|
|
197
|
+
| `integer` / `number` | `range { min, max }` | yes | yes | number |
|
|
198
|
+
| `date` | `range` (inclusive) | yes | yes | ISO 8601 string (surface) |
|
|
199
|
+
| `boolean` | `is` | yes | – | boolean (absent = false) |
|
|
79
200
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
201
|
+
A `reference` carries `labelOnly` today (id + display label); the `idOnly` and
|
|
202
|
+
`inline` strategies are forward declarations. References are object-shaped from
|
|
203
|
+
day one so that `inline` can later **add** fields to a reference type without
|
|
204
|
+
breaking clients.
|
|
205
|
+
|
|
206
|
+
## Projection
|
|
207
|
+
|
|
208
|
+
`projectGraph` is fully streaming: subjects are grouped and framed one at a time
|
|
209
|
+
and documents are yielded as produced, so beyond a subject index memory stays
|
|
210
|
+
flat at scale (framing the whole graph at once is roughly O(N²)). Duplicate
|
|
211
|
+
triples are collapsed first, because some SPARQL engines (e.g. QLever) do not
|
|
212
|
+
deduplicate `CONSTRUCT` output. The IR carries no `@context`, so a `derive` function
|
|
213
|
+
reading it sees full predicate IRIs with language tags preserved.
|
|
85
214
|
|
|
86
215
|
## Locales
|
|
87
216
|
|
|
88
|
-
`locales` is the **single** list of
|
|
89
|
-
|
|
217
|
+
`locales` is the **single** list of locales a `text` field projects (`und` =
|
|
218
|
+
untagged literals);
|
|
219
|
+
`output`, `searchable` and `sortable` are independent opt-ins that each fan out
|
|
90
220
|
over it (so a field emits exactly what it opts into):
|
|
91
221
|
|
|
92
|
-
- `
|
|
93
|
-
- `
|
|
94
|
-
lets a query `query_by` them and rank the user’s language higher
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
- `
|
|
222
|
+
- `output` → `title_nl`/`title_en` (accents preserved);
|
|
223
|
+
- `searchable` → `title_search_nl`/`title_search_en` (folded; one field per locale
|
|
224
|
+
lets a query `query_by` them and rank the user’s language higher, and lets a
|
|
225
|
+
language that needs a dedicated tokenizer set its own stemming `locale` in the
|
|
226
|
+
engine schema);
|
|
227
|
+
- `sortable` → `title_sort_nl`/`title_sort_en` (folded, so a locale-switching UI
|
|
98
228
|
sorts on the active language).
|
|
99
229
|
|
|
100
|
-
A field with `
|
|
101
|
-
for retrieval but never rendered (e.g. a
|
|
102
|
-
|
|
230
|
+
A field with `searchable` but no `output` is **search-only** — folded and stemmed
|
|
231
|
+
for retrieval but never rendered (e.g. a creator searched here but shown via a
|
|
232
|
+
separate label). **Only listed locales are indexed**; a literal whose language tag is not in
|
|
233
|
+
`locales` is not projected (declare `und` to catch untagged literals). Per-locale fields are
|
|
234
|
+
**omitted, never empty**, when a document lacks that language, so declare them
|
|
235
|
+
optional in the engine schema and sort with `missing_values: last`.
|
|
103
236
|
|
|
104
237
|
Folding the search fields is what lets diacritic-insensitive matching and
|
|
105
238
|
stemming coexist. A search engine on its **default** locale typically folds case
|
|
106
|
-
and diacritics for you
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
is a per-field engine-schema choice (the consumer’s), and being rules-based it
|
|
113
|
-
can mangle proper nouns and place names — e.g. the Dutch stemmer reduces the city
|
|
114
|
-
`Bergen` to `berg`, colliding it with “mountain”.
|
|
115
|
-
|
|
116
|
-
Recommended split: enable stemming on the **free-text** search fields
|
|
117
|
-
(`*_search_${locale}`, descriptions, keywords) where morphological recall helps
|
|
118
|
-
(`verhaal` ↔ `verhalen`), and keep **place names and other proper-noun facets on
|
|
119
|
-
a separate, unstemmed field** (facets are exact-match anyway). That captures the
|
|
120
|
-
recall without the `Bergen`/`berg` collision in the facet. A `stem_dictionary`
|
|
121
|
-
can pin specific names if you need stemmed free-text without given collisions.
|
|
122
|
-
|
|
123
|
-
**Only listed locales are indexed.** A literal whose language tag is not in
|
|
124
|
-
`locales` is not projected at all — no display, no search, no sort field — so it
|
|
125
|
-
is invisible to the index. To index a language, add it to `locales`.
|
|
126
|
-
|
|
127
|
-
Per-locale fields are **omitted, never empty**, when a document lacks that
|
|
128
|
-
language, so declare them `optional: true` in the engine schema. At query time,
|
|
129
|
-
sort with `missing_values: last` to push documents lacking the active locale to
|
|
130
|
-
the end, and `query_by` all the per-locale search fields (weighting the user’s
|
|
131
|
-
locale higher) to keep cross-language recall.
|
|
132
|
-
|
|
133
|
-
A literal with no `@language` tag matches no locale, so it is not projected. Tag
|
|
134
|
-
your source literals (or pre-process them) for the languages you index.
|
|
239
|
+
and diacritics for you; enabling a language’s **stemming** switches it to ICU
|
|
240
|
+
tokenization, which **preserves** diacritics — at which point `fold()` (from
|
|
241
|
+
[`@lde/text-normalization`](../text-normalization)) is what keeps matching
|
|
242
|
+
diacritic-insensitive. Stemming is rules-based and can mangle proper nouns (the
|
|
243
|
+
Dutch stemmer reduces the city `Bergen` to `berg`), so enable it on free-text
|
|
244
|
+
fields and keep proper-noun facets on a separate, unstemmed field.
|
|
135
245
|
|
|
136
246
|
## Querying
|
|
137
247
|
|
|
138
248
|
The search fields are stored already case- and diacritic-folded, so **the query
|
|
139
|
-
must be folded the same way** with the same `fold()`
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
249
|
+
must be folded the same way** with the same `fold()` before it reaches the engine,
|
|
250
|
+
or index and query normalize differently and matches silently miss. This contract
|
|
251
|
+
holds for **any** consumer, including an API built on this package — which is why
|
|
252
|
+
engine adapters and surfaces compile through the shared `SearchQuery` IR and the
|
|
253
|
+
`physicalFields` convention rather than re-deriving field names.
|
|
254
|
+
|
|
255
|
+
Queries are **always validated**: the port contract requires every engine
|
|
256
|
+
adapter to reject a structurally invalid `SearchQuery` (`assertValidQuery`) —
|
|
257
|
+
unknown or non-`filterable` fields in `where`, an operator not matching the
|
|
258
|
+
field’s kind, non-`facetable` facet requests — no matter which surface or
|
|
259
|
+
policy produced it. A typed surface like GraphQL makes most of these
|
|
260
|
+
unrepresentable; the port enforces them for everyone else (deployment
|
|
261
|
+
`queryDefaults`, in-process callers, weaker-typed surfaces).
|
|
262
|
+
|
|
263
|
+
## Typed results
|
|
264
|
+
|
|
265
|
+
An engine is **bound to the whole `SearchSchema` at construction** — like
|
|
266
|
+
every other schema consumer (`projectGraph(quads, schema)`,
|
|
267
|
+
`buildGraphQLSchema(schema)`): the adapter factory takes the deployment’s
|
|
268
|
+
declaration together with each type’s physical location, so a query can never
|
|
269
|
+
meet the wrong index, and deployment-level concerns (the label cache,
|
|
270
|
+
cross-type search, facet batching) have one home. A search names its type per
|
|
271
|
+
call. Because `searchSchema()` captures the declared types as a literal
|
|
272
|
+
tuple, `search()` accepts **only the deployment’s own types** (a foreign type
|
|
273
|
+
is a compile error) and returns facet/document keys typed by the type passed
|
|
274
|
+
— no caller-side generics:
|
|
146
275
|
|
|
147
276
|
```ts
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
});
|
|
277
|
+
const engine = createTypesenseSearchEngine(client, schema, {
|
|
278
|
+
collections: { Dataset: 'datasets', Person: 'people' }, // omitting one = type error
|
|
279
|
+
});
|
|
280
|
+
|
|
281
|
+
const result = await engine.search(DATASET, query);
|
|
282
|
+
result.facets.publisher; // typed: only DATASET’s facetable fields are keys
|
|
283
|
+
result.facets.publsher; // compile error (typo)
|
|
284
|
+
result.hits[0].document.title; // typed: only DATASET’s output fields are keys
|
|
285
|
+
await engine.search(OTHER_TYPE, query); // compile error: not in this schema
|
|
158
286
|
```
|
|
159
287
|
|
|
160
|
-
This
|
|
161
|
-
|
|
162
|
-
|
|
288
|
+
This only works when the types were declared with `defineSearchType` (or
|
|
289
|
+
captured `as const satisfies SearchType`) and composed with `searchSchema()`;
|
|
290
|
+
a plain `: SearchSchema` annotation widens gracefully to string keys.
|
|
291
|
+
`FacetFieldsOf`/`OutputFieldsOf` are exported for annotating your own
|
|
292
|
+
signatures, and `engine.schema` exposes the bound declaration for routing.
|
|
163
293
|
|
|
164
|
-
## Why a
|
|
294
|
+
## Why a declarative model
|
|
165
295
|
|
|
166
|
-
The
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
today will run a **SHACL-generated**
|
|
170
|
-
only
|
|
296
|
+
The vocabulary mirrors SHACL on purpose: `path` is `sh:path`, `array` is
|
|
297
|
+
`sh:maxCount`, `required` is `sh:minCount`, `locales` is `sh:languageIn`, `ref`
|
|
298
|
+
is `sh:class`/`sh:node`. So the same core that runs a hand-written `SearchSchema`
|
|
299
|
+
today will run a **SHACL-generated** one tomorrow — the model, the ports and the
|
|
300
|
+
IR stay; only schema-authoring gets automated.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The adapter-author surface of `@lde/search` (import from
|
|
3
|
+
* `@lde/search/adapter`): the plumbing an engine adapter or API surface needs
|
|
4
|
+
* to compile queries and reconstruct results — field selectors, the physical
|
|
5
|
+
* field-name convention, query validation and the storage codecs. A
|
|
6
|
+
* deployment author declaring a schema and running searches needs none of
|
|
7
|
+
* this; the main entry point carries that (small) authoring surface.
|
|
8
|
+
*/
|
|
9
|
+
export { assertTypeInSchema, physicalFields, searchableFields, facetableFields, filterableFields, sortableFields, outputFields, referenceFields, fieldNamed, isRangeFacet, isoToUnixSeconds, unixSecondsToIso, } from './schema.js';
|
|
10
|
+
export type { PhysicalFields } from './schema.js';
|
|
11
|
+
export { filterOperatorFor, filterOperator, validateQuery, assertValidQuery, pageForOffset, } from './query.js';
|
|
12
|
+
export type { FilterOperator, QueryIssue } from './query.js';
|
|
13
|
+
//# sourceMappingURL=adapter.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"adapter.d.ts","sourceRoot":"","sources":["../src/adapter.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AACH,OAAO,EACL,kBAAkB,EAClB,cAAc,EACd,gBAAgB,EAChB,eAAe,EACf,gBAAgB,EAChB,cAAc,EACd,YAAY,EACZ,eAAe,EACf,UAAU,EACV,YAAY,EACZ,gBAAgB,EAChB,gBAAgB,GACjB,MAAM,aAAa,CAAC;AACrB,YAAY,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAClD,OAAO,EACL,iBAAiB,EACjB,cAAc,EACd,aAAa,EACb,gBAAgB,EAChB,aAAa,GACd,MAAM,YAAY,CAAC;AACpB,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC"}
|
package/dist/adapter.js
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The adapter-author surface of `@lde/search` (import from
|
|
3
|
+
* `@lde/search/adapter`): the plumbing an engine adapter or API surface needs
|
|
4
|
+
* to compile queries and reconstruct results — field selectors, the physical
|
|
5
|
+
* field-name convention, query validation and the storage codecs. A
|
|
6
|
+
* deployment author declaring a schema and running searches needs none of
|
|
7
|
+
* this; the main entry point carries that (small) authoring surface.
|
|
8
|
+
*/
|
|
9
|
+
export { assertTypeInSchema, physicalFields, searchableFields, facetableFields, filterableFields, sortableFields, outputFields, referenceFields, fieldNamed, isRangeFacet, isoToUnixSeconds, unixSecondsToIso, } from './schema.js';
|
|
10
|
+
export { filterOperatorFor, filterOperator, validateQuery, assertValidQuery, pageForOffset, } from './query.js';
|
package/dist/engine.d.ts
ADDED
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
import type { SearchQuery } from './query.js';
|
|
2
|
+
import type { SearchSchema, SearchType } from './schema.js';
|
|
3
|
+
/**
|
|
4
|
+
* The engine port: the boundary a concrete engine adapter (e.g. the engine
|
|
5
|
+
* `@lde/search-typesense`’s `createTypesenseSearchEngine` returns) implements.
|
|
6
|
+
* An engine is **bound to the whole {@link SearchSchema} at construction** —
|
|
7
|
+
* the adapter factory takes the deployment’s declaration together with the
|
|
8
|
+
* physical location of every type (its collections map), mirroring the other
|
|
9
|
+
* schema consumers (`projectGraph(quads, schema)`,
|
|
10
|
+
* `buildGraphQLSchema(schema)`): everything is a function of the schema. A
|
|
11
|
+
* query can never meet the wrong index, deployment-level concerns (label
|
|
12
|
+
* cache, cross-type search, facet batching) have one home, and a search names
|
|
13
|
+
* its type per call.
|
|
14
|
+
* The adapter owns every engine specific (companion-field expansion,
|
|
15
|
+
* full-text field selection and weights, filter compilation, sorting, result
|
|
16
|
+
* folding, faceting) and returns only logical documents, so a deployment can
|
|
17
|
+
* swap engines without any consumer noticing. Nothing engine-specific and
|
|
18
|
+
* nothing RDF-specific leaks past this port.
|
|
19
|
+
*
|
|
20
|
+
* Port contract: an adapter ALWAYS rejects a `searchType` that is not in its
|
|
21
|
+
* bound schema, and ALWAYS validates the incoming query against it
|
|
22
|
+
* (`assertValidQuery`) — unknown or non-filterable fields, mismatched
|
|
23
|
+
* operators, unknown facets — rather than passing garbage to its engine.
|
|
24
|
+
* Validation is not the caller’s job: it must hold for every surface and for
|
|
25
|
+
* injected deployment policy.
|
|
26
|
+
*
|
|
27
|
+
* `Types` is the literal tuple {@link searchSchema} captured: with a
|
|
28
|
+
* `defineSearchType` declaration, `search()` only accepts the deployment’s
|
|
29
|
+
* own types (a foreign type is a compile error) and returns facet/document
|
|
30
|
+
* keys typed by the type passed. A widened `: SearchSchema` degrades
|
|
31
|
+
* gracefully to string keys ({@link FacetFieldsOf}).
|
|
32
|
+
*/
|
|
33
|
+
export interface SearchEngine<Types extends readonly SearchType[] = readonly SearchType[]> {
|
|
34
|
+
/** The declaration this engine serves — exposed so a surface can route and
|
|
35
|
+
* a caller can enumerate the searchable types. */
|
|
36
|
+
readonly schema: SearchSchema<Types>;
|
|
37
|
+
search<T extends Types[number]>(searchType: T, query: SearchQuery): Promise<SearchResult<FacetFieldsOf<T>, OutputFieldsOf<T>>>;
|
|
38
|
+
}
|
|
39
|
+
/** What an engine returns: logical hits, a total, and the requested facets. */
|
|
40
|
+
export interface SearchResult<FacetField extends string = string, OutputField extends string = string> {
|
|
41
|
+
readonly hits: readonly SearchHit<OutputField>[];
|
|
42
|
+
readonly total: number;
|
|
43
|
+
readonly facets: FacetMap<FacetField>;
|
|
44
|
+
}
|
|
45
|
+
/**
|
|
46
|
+
* Facet buckets keyed by facet field name. `Partial` because a result carries
|
|
47
|
+
* buckets only for the fields the query asked for, not every facetable field.
|
|
48
|
+
*/
|
|
49
|
+
export type FacetMap<FacetField extends string = string> = Readonly<Partial<Record<FacetField, readonly FacetBucket[]>>>;
|
|
50
|
+
/**
|
|
51
|
+
* The facet-field-name union of a search type — the keys a {@link SearchResult}’s
|
|
52
|
+
* `facets` can hold. Narrow for a declaration captured as a literal (via
|
|
53
|
+
* `defineSearchType` or `as const satisfies SearchType`); a plain
|
|
54
|
+
* `: SearchType` annotation degrades to `string` (its field names are not
|
|
55
|
+
* statically known).
|
|
56
|
+
*/
|
|
57
|
+
export type FacetFieldsOf<Type extends SearchType> = SearchType extends Type ? string : Extract<Type['fields'][number], {
|
|
58
|
+
readonly facetable: true;
|
|
59
|
+
}>['name'];
|
|
60
|
+
/**
|
|
61
|
+
* The output-field-name union of a search type — the keys a {@link ResultDocument}
|
|
62
|
+
* can hold. Like {@link FacetFieldsOf}, degrades to `string` for a widened
|
|
63
|
+
* declaration.
|
|
64
|
+
*/
|
|
65
|
+
export type OutputFieldsOf<Type extends SearchType> = SearchType extends Type ? string : Extract<Type['fields'][number], {
|
|
66
|
+
readonly output: true;
|
|
67
|
+
}>['name'];
|
|
68
|
+
/**
|
|
69
|
+
* One result row. `id` (the stable document key, an IRI) is kept *out* of
|
|
70
|
+
* {@link ResultDocument}: it is always present and is the hit’s identity, a
|
|
71
|
+
* different contract from the optional, typed logical field values — and it maps
|
|
72
|
+
* straight onto the GraphQL output’s guaranteed `id: String!`. The document
|
|
73
|
+
* holds only the selectable fields.
|
|
74
|
+
*/
|
|
75
|
+
export interface SearchHit<OutputField extends string = string> {
|
|
76
|
+
readonly id: string;
|
|
77
|
+
readonly document: ResultDocument<OutputField>;
|
|
78
|
+
}
|
|
79
|
+
/**
|
|
80
|
+
* The logical result document at the query seam — engine- and RDF-neutral.
|
|
81
|
+
* Distinct from the flat, fanned-out projection `SearchDocument` that lives
|
|
82
|
+
* index-side: this carries logical fields with language maps and references,
|
|
83
|
+
* ready for a surface to shape. Keyed by output field name; `Partial` because a
|
|
84
|
+
* document omits absent optional fields. `OutputField` defaults to `string`; a
|
|
85
|
+
* deployment narrows it via {@link OutputFieldsOf} for typo-safe field access.
|
|
86
|
+
*/
|
|
87
|
+
export type ResultDocument<OutputField extends string = string> = Readonly<Partial<Record<OutputField, SearchValue>>>;
|
|
88
|
+
/** A logical field value. */
|
|
89
|
+
export type SearchValue = string | number | boolean | readonly string[] | LocalizedValue | Reference | readonly Reference[];
|
|
90
|
+
/**
|
|
91
|
+
* A JSON-LD-style language map (`@container: @language`, `@set` arrays); the key
|
|
92
|
+
* `und` carries untagged (`@none`) values. The surface flattens it to a
|
|
93
|
+
* best-first `Accept-Language`-ordered list.
|
|
94
|
+
*/
|
|
95
|
+
export type LocalizedValue = Readonly<Record<string, readonly string[]>>;
|
|
96
|
+
/**
|
|
97
|
+
* The generic internal carrier for a referenced entity. The GraphQL surface maps
|
|
98
|
+
* it to a named per-shape type (e.g. `Organization`, `Term`) with `label`
|
|
99
|
+
* exposed as `name`.
|
|
100
|
+
*/
|
|
101
|
+
export interface Reference {
|
|
102
|
+
readonly id: string;
|
|
103
|
+
readonly label?: LocalizedValue;
|
|
104
|
+
}
|
|
105
|
+
/**
|
|
106
|
+
* One facet bucket: a value and how many documents carry it. `label` is the
|
|
107
|
+
* engine-resolved canonical **data** label, present only for reference facets
|
|
108
|
+
* (IRI-keyed); it is absent for facets whose value is a token or free string
|
|
109
|
+
* whose display the consumer owns (its own i18n, or the value itself).
|
|
110
|
+
*/
|
|
111
|
+
export interface FacetBucket {
|
|
112
|
+
readonly value: string;
|
|
113
|
+
readonly count: number;
|
|
114
|
+
readonly label?: LocalizedValue;
|
|
115
|
+
/**
|
|
116
|
+
* For a range-facet bucket: its half-open bounds (`min` inclusive, `max`
|
|
117
|
+
* exclusive), echoing the declared {@link FacetRange} so the bucket is
|
|
118
|
+
* self-describing and a consumer never hardcodes the bin formula. Both absent
|
|
119
|
+
* for a value facet; either absent for an open-ended bin.
|
|
120
|
+
*/
|
|
121
|
+
readonly min?: number;
|
|
122
|
+
readonly max?: number;
|
|
123
|
+
}
|
|
124
|
+
//# sourceMappingURL=engine.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../src/engine.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAC9C,OAAO,KAAK,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAE5D;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA6BG;AACH,MAAM,WAAW,YAAY,CAC3B,KAAK,SAAS,SAAS,UAAU,EAAE,GAAG,SAAS,UAAU,EAAE;IAE3D;uDACmD;IACnD,QAAQ,CAAC,MAAM,EAAE,YAAY,CAAC,KAAK,CAAC,CAAC;IACrC,MAAM,CAAC,CAAC,SAAS,KAAK,CAAC,MAAM,CAAC,EAC5B,UAAU,EAAE,CAAC,EACb,KAAK,EAAE,WAAW,GACjB,OAAO,CAAC,YAAY,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;CAC/D;AAED,+EAA+E;AAC/E,MAAM,WAAW,YAAY,CAC3B,UAAU,SAAS,MAAM,GAAG,MAAM,EAClC,WAAW,SAAS,MAAM,GAAG,MAAM;IAEnC,QAAQ,CAAC,IAAI,EAAE,SAAS,SAAS,CAAC,WAAW,CAAC,EAAE,CAAC;IACjD,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,MAAM,EAAE,QAAQ,CAAC,UAAU,CAAC,CAAC;CACvC;AAED;;;GAGG;AACH,MAAM,MAAM,QAAQ,CAAC,UAAU,SAAS,MAAM,GAAG,MAAM,IAAI,QAAQ,CACjE,OAAO,CAAC,MAAM,CAAC,UAAU,EAAE,SAAS,WAAW,EAAE,CAAC,CAAC,CACpD,CAAC;AAEF;;;;;;GAMG;AACH,MAAM,MAAM,aAAa,CAAC,IAAI,SAAS,UAAU,IAAI,UAAU,SAAS,IAAI,GACxE,MAAM,GACN,OAAO,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,EAAE;IAAE,QAAQ,CAAC,SAAS,EAAE,IAAI,CAAA;CAAE,CAAC,CAAC,MAAM,CAAC,CAAC;AAE1E;;;;GAIG;AACH,MAAM,MAAM,cAAc,CAAC,IAAI,SAAS,UAAU,IAAI,UAAU,SAAS,IAAI,GACzE,MAAM,GACN,OAAO,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,EAAE;IAAE,QAAQ,CAAC,MAAM,EAAE,IAAI,CAAA;CAAE,CAAC,CAAC,MAAM,CAAC,CAAC;AAEvE;;;;;;GAMG;AACH,MAAM,WAAW,SAAS,CAAC,WAAW,SAAS,MAAM,GAAG,MAAM;IAC5D,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,QAAQ,EAAE,cAAc,CAAC,WAAW,CAAC,CAAC;CAChD;AAED;;;;;;;GAOG;AACH,MAAM,MAAM,cAAc,CAAC,WAAW,SAAS,MAAM,GAAG,MAAM,IAAI,QAAQ,CACxE,OAAO,CAAC,MAAM,CAAC,WAAW,EAAE,WAAW,CAAC,CAAC,CAC1C,CAAC;AAEF,6BAA6B;AAC7B,MAAM,MAAM,WAAW,GACnB,MAAM,GACN,MAAM,GACN,OAAO,GACP,SAAS,MAAM,EAAE,GACjB,cAAc,GACd,SAAS,GACT,SAAS,SAAS,EAAE,CAAC;AAEzB;;;;GAIG;AACH,MAAM,MAAM,cAAc,GAAG,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC,CAAC,CAAC;AAEzE;;;;GAIG;AACH,MAAM,WAAW,SAAS;IACxB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,KAAK,CAAC,EAAE,cAAc,CAAC;CACjC;AAED;;;;;GAKG;AACH,MAAM,WAAW,WAAW;IAC1B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,KAAK,CAAC,EAAE,cAAc,CAAC;IAChC;;;;;OAKG;IACH,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;CACvB"}
|
package/dist/engine.js
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export {};
|
package/dist/index.d.ts
CHANGED
|
@@ -1,4 +1,8 @@
|
|
|
1
1
|
export { projectGraph, irisOf, literalsOf, firstLiteralOf } from './project.js';
|
|
2
|
-
export type { SearchDocument
|
|
2
|
+
export type { SearchDocument } from './project.js';
|
|
3
|
+
export { defineSearchType, searchSchema, validateSearchType, assertValidSearchType, } from './schema.js';
|
|
4
|
+
export type { FieldKind, SearchField, SearchFieldBase, TextField, KeywordField, ReferenceField, NumericField, BooleanField, SearchType, SearchTypeIssue, SearchSchema, FacetRange, } from './schema.js';
|
|
5
|
+
export type { SearchQuery, Filter, Sort } from './query.js';
|
|
6
|
+
export type { SearchEngine, SearchResult, SearchHit, ResultDocument, SearchValue, LocalizedValue, Reference, FacetBucket, FacetMap, FacetFieldsOf, OutputFieldsOf, } from './engine.js';
|
|
3
7
|
export type { FramedNode } from './frame-by-type.js';
|
|
4
8
|
//# sourceMappingURL=index.d.ts.map
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AASA,OAAO,EAAE,YAAY,EAAE,MAAM,EAAE,UAAU,EAAE,cAAc,EAAE,MAAM,cAAc,CAAC;AAChF,YAAY,EAAE,cAAc,EAAE,MAAM,cAAc,CAAC;AAKnD,OAAO,EACL,gBAAgB,EAChB,YAAY,EACZ,kBAAkB,EAClB,qBAAqB,GACtB,MAAM,aAAa,CAAC;AACrB,YAAY,EACV,SAAS,EACT,WAAW,EACX,eAAe,EACf,SAAS,EACT,YAAY,EACZ,cAAc,EACd,YAAY,EACZ,YAAY,EACZ,UAAU,EACV,eAAe,EACf,YAAY,EACZ,UAAU,GACX,MAAM,aAAa,CAAC;AAIrB,YAAY,EAAE,WAAW,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,YAAY,CAAC;AAI5D,YAAY,EACV,YAAY,EACZ,YAAY,EACZ,SAAS,EACT,cAAc,EACd,WAAW,EACX,cAAc,EACd,SAAS,EACT,WAAW,EACX,QAAQ,EACR,aAAa,EACb,cAAc,GACf,MAAM,aAAa,CAAC;AAErB,YAAY,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC"}
|
package/dist/index.js
CHANGED
|
@@ -1 +1,13 @@
|
|
|
1
|
+
// The AUTHORING surface: what a deployment declares a schema with, projects
|
|
2
|
+
// documents through, and holds engines and results as. The adapter/surface
|
|
3
|
+
// plumbing (field selectors, physical fanout, query validation, codecs) lives
|
|
4
|
+
// under `@lde/search/adapter`; the engine-port conformance suite under
|
|
5
|
+
// `@lde/search/testing`.
|
|
6
|
+
// Projection: RDF CONSTRUCT quads → flat search documents, driven by the
|
|
7
|
+
// unified SearchField/SearchType model. The IR readers (irisOf, …) are here
|
|
8
|
+
// because `derive` functions are written against them.
|
|
1
9
|
export { projectGraph, irisOf, literalsOf, firstLiteralOf } from './project.js';
|
|
10
|
+
// Unified field model: one declaration drives projection, engine collection
|
|
11
|
+
// schema, query semantics and the GraphQL surface — a discriminated union by
|
|
12
|
+
// `kind`, validated again at runtime when the schema is built.
|
|
13
|
+
export { defineSearchType, searchSchema, validateSearchType, assertValidSearchType, } from './schema.js';
|