@lde/search-typesense 0.15.5 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -191
- package/dist/query-compiler.d.ts +3 -3
- package/dist/query-compiler.d.ts.map +1 -1
- package/dist/query-compiler.js +13 -3
- package/dist/search.d.ts.map +1 -1
- package/dist/search.js +61 -12
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -3,150 +3,14 @@
|
|
|
3
3
|
[Typesense](https://typesense.org/) engine adapter for the engine- and
|
|
4
4
|
domain-agnostic [`@lde/search`](../search) core. **Engine-specific (Typesense) but
|
|
5
5
|
domain-agnostic** – you supply a `SearchType`; this package never names your
|
|
6
|
-
domain.
|
|
7
|
-
a collection definition from the field model, compiles the neutral `SearchQuery` into
|
|
8
|
-
Typesense search params, runs it, reconstructs the engine-neutral `SearchResult`,
|
|
9
|
-
and manages the search index lifecycle as transactional `@lde/pipeline`
|
|
10
|
-
writers (Blue/green Rebuild and In-place Rebuild).
|
|
6
|
+
domain.
|
|
11
7
|
|
|
12
|
-
##
|
|
8
|
+
## Installation
|
|
13
9
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
convention – snake_case, named after the plural of what the collection holds, as
|
|
17
|
-
its [collection guide](https://typesense.org/docs/guide/organizing-collections.html)
|
|
18
|
-
writes them (`people`, `companies`, `blog_articles`):
|
|
19
|
-
|
|
20
|
-
| `SearchType.name` | collection |
|
|
21
|
-
| ----------------- | ---------------- |
|
|
22
|
-
| `Dataset` | `datasets` |
|
|
23
|
-
| `CreativeWork` | `creative_works` |
|
|
24
|
-
| `Person` | `people` |
|
|
25
|
-
| `TVSeries` | `tv_series` |
|
|
26
|
-
| `DCATDataset` | `dcat_datasets` |
|
|
27
|
-
|
|
28
|
-
The writers and the engine all fall back to this one convention, so documents
|
|
29
|
-
cannot be written to `creative_works` and queried from `creativeWorks` – and a
|
|
30
|
-
reference’s `labelSource` resolves to its label type’s collection the same way.
|
|
31
|
-
Every naming input is therefore optional: `buildCollectionDefinition(type)`,
|
|
32
|
-
`new BlueGreenRebuild(client, type)`, `createTypesenseSearchEngine(client, schema)`.
|
|
33
|
-
|
|
34
|
-
Naming lives here rather than in `@lde/search` because it is engine-specific:
|
|
35
|
-
Elasticsearch/OpenSearch index names are kebab-case and
|
|
36
|
-
[constrained](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html)
|
|
37
|
-
(lowercase only, no `\ / * ? " < > | space , #`, no leading `-` `_` `+`, ≤ 255
|
|
38
|
-
bytes), so a future `@lde/search-elasticsearch` formats the same neutral tokens
|
|
39
|
-
(`@lde/search`’s `physicalNameTokens`) its own way. The core stays engine-neutral.
|
|
40
|
-
|
|
41
|
-
Pass an explicit name to **override** the convention – an env prefix, a
|
|
42
|
-
multi-tenant name, a collection that already exists, a migration:
|
|
43
|
-
|
|
44
|
-
```ts
|
|
45
|
-
const writer = new BlueGreenRebuild(client, CREATIVE_WORK, {
|
|
46
|
-
name: 'staging_creative_works',
|
|
47
|
-
});
|
|
48
|
-
const engine = createTypesenseSearchEngine(client, schema, {
|
|
49
|
-
collections: { CreativeWork: 'staging_creative_works' },
|
|
50
|
-
});
|
|
51
|
-
writer.collectionName; // 'staging_creative_works' – read-only, for logs/health checks
|
|
52
|
-
engine.collectionNameFor(CREATIVE_WORK); // the same name, resolved once at construction
|
|
10
|
+
```sh
|
|
11
|
+
npm install @lde/search-typesense
|
|
53
12
|
```
|
|
54
13
|
|
|
55
|
-
`collections` overrides only the types it names; the rest stay derived.
|
|
56
|
-
|
|
57
|
-
Everything the convention cannot do, it refuses to guess at, when the writer or
|
|
58
|
-
engine is constructed rather than on the first rebuild or search:
|
|
59
|
-
|
|
60
|
-
- a name it cannot spell (`Café` would quietly become `cafs`) or turn into a
|
|
61
|
-
legal collection name;
|
|
62
|
-
- two types whose names **derive to the same collection** – English collapses
|
|
63
|
-
`Medium` and `Media` onto `media`, `Person` and `People` onto `people` – which
|
|
64
|
-
would land both types’ documents in one collection, each search returning the
|
|
65
|
-
other’s. Name either one explicitly to resolve it. Types that share a
|
|
66
|
-
collection _deliberately_ (say, several label sources in one `labels`
|
|
67
|
-
collection) are fine: name both, and it is your call, not an accident.
|
|
68
|
-
|
|
69
|
-
## Collection schema and engine
|
|
70
|
-
|
|
71
|
-
`buildCollectionDefinition(searchType, { name?, defaultSortingField, … })` derives a
|
|
72
|
-
Typesense collection from the unified `SearchField` model – the Typesense field
|
|
73
|
-
type comes from each field’s `kind`, and the physical fanout (per-locale
|
|
74
|
-
search/sort keys, plus one regex display field per output text field) matches
|
|
75
|
-
what the projection writes, via `@lde/search`’s `physicalFields` and its display
|
|
76
|
-
helpers, so the index and the documents cannot drift.
|
|
77
|
-
|
|
78
|
-
**Memory lever.** Typesense keeps the index in RAM (with a raw copy of each
|
|
79
|
-
document on disk), so RAM tracks the _indexed_ surface – roughly 2–3× the size
|
|
80
|
-
of the fields you search, facet or sort on – not the full document.
|
|
81
|
-
`buildCollectionDefinition` keeps that surface minimal: the `output` display
|
|
82
|
-
labels land in one `index: false` regex field (`${name}_<lang>`, one value per
|
|
83
|
-
present language), stored on disk and fetched only for a hit, so they cost no RAM
|
|
84
|
-
and preserve every language; only the folded `*_search_${locale}`,
|
|
85
|
-
facet/reference and `*_sort_${locale}` companions are indexed. Keeping
|
|
86
|
-
retrieval-only fields un-indexed is the lever for holding a large index’s RAM
|
|
87
|
-
down.
|
|
88
|
-
|
|
89
|
-
`createTypesenseSearchEngine(client, schema, { collections? })` is the
|
|
90
|
-
`SearchEngine` implementation. Each search:
|
|
91
|
-
|
|
92
|
-
- validates the query against the search type (the port contract – a
|
|
93
|
-
structurally invalid query is rejected, never sent);
|
|
94
|
-
- compiles it into Typesense search params (`buildSearchParams`);
|
|
95
|
-
- runs the search;
|
|
96
|
-
- resolves reference (and reference-facet) labels **per reference field**
|
|
97
|
-
from the collection of the `SearchType` its `labelSource` names – all
|
|
98
|
-
sources bundled into a single lookup. A reference without a `labelSource`
|
|
99
|
-
stays id-only. With `labelCacheTtlMs` set, each label-source collection is
|
|
100
|
-
instead loaded once into an in-memory cache;
|
|
101
|
-
- reconstructs the logical `SearchResult` (`parseSearchResponse`) – language
|
|
102
|
-
maps, labelled references, labelled facet buckets.
|
|
103
|
-
|
|
104
|
-
A label source is just another `SearchType` in the schema (with an `output`,
|
|
105
|
-
`searchable` text field called `label`), so its collection is named by the same
|
|
106
|
-
convention as any other type’s – a typed entity collection and a ‘labels
|
|
107
|
-
collection’ are the same kind of thing.
|
|
108
|
-
|
|
109
|
-
`searchFacets` – the port’s batch entry point – answers a whole batch of
|
|
110
|
-
facet-only queries (e.g. a faceted listing’s skip-own-filter variants) as a
|
|
111
|
-
**single `multi_search` round-trip**, with one bundled label lookup shared by
|
|
112
|
-
every facet result in the batch. A failed entry is reported in place as a
|
|
113
|
-
per-query outcome, so its siblings’ facets survive.
|
|
114
|
-
|
|
115
|
-
The pure halves `buildSearchParams` and `parseSearchResponse` are exported for
|
|
116
|
-
direct use and testing.
|
|
117
|
-
|
|
118
|
-
## Indexing
|
|
119
|
-
|
|
120
|
-
Indexing runs through two transactional writers, one per update mode – the
|
|
121
|
-
[NDE Stack](https://docs.nde.nl/stack/patterns) patterns of the same names:
|
|
122
|
-
|
|
123
|
-
- [**Blue/green Rebuild**](#bluegreen-rebuild): build a fresh index, then swap to
|
|
124
|
-
it atomically;
|
|
125
|
-
- [**In-place Rebuild**](#in-place-rebuild): update the live index directly by
|
|
126
|
-
upserting changed sources and sweeping the rest.
|
|
127
|
-
|
|
128
|
-
Both implement `@lde/pipeline`’s `Writer` – each run is
|
|
129
|
-
`openRun(context)` → `write` per dataset → `commit()` or `abort(error)` – so
|
|
130
|
-
an `@lde/pipeline` `Pipeline` drives them without branching on the mode. Both
|
|
131
|
-
derive the collection definition from your `SearchType` (via
|
|
132
|
-
`buildCollectionDefinition`), and their options accept everything
|
|
133
|
-
`buildCollectionDefinition` does (`defaultLocale`, `defaultSortingField`,
|
|
134
|
-
`synonymSets`) plus the tuning knobs (`batchSize`, `lockTtlMs`).
|
|
135
|
-
|
|
136
|
-
### Blue/green Rebuild
|
|
137
|
-
|
|
138
|
-
`BlueGreenRebuild` rebuilds the index from zero and goes live atomically:
|
|
139
|
-
`openRun` creates a fresh versioned collection (`${name}_<timestamp>`),
|
|
140
|
-
`write` streams documents into it in batches (each stamped with its `source`
|
|
141
|
-
dataset IRI), and `commit` atomically repoints the `name` alias and drops the
|
|
142
|
-
collection it superseded. Until commit, the live alias never points at a
|
|
143
|
-
partial build; `abort` drops the half-built collection. Deletion is implicit –
|
|
144
|
-
whatever a run does not write does not exist in the new collection. A dataset
|
|
145
|
-
that fails (or is reset before a dump re-run) is rolled back out of the
|
|
146
|
-
not-yet-live collection by `source`, so the swap never ships a half-processed
|
|
147
|
-
dataset. Right-sized for small collections (e.g. one document per dataset
|
|
148
|
-
description).
|
|
149
|
-
|
|
150
14
|
```ts
|
|
151
15
|
import { Client } from 'typesense';
|
|
152
16
|
import { BlueGreenRebuild } from '@lde/search-typesense';
|
|
@@ -164,55 +28,6 @@ await run.write(dataset, documents);
|
|
|
164
28
|
await run.commit();
|
|
165
29
|
```
|
|
166
30
|
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
`InPlaceRebuild` maintains one long-lived collection with per-source
|
|
170
|
-
atomicity – no swap, no staging – for large, mostly-static corpora (e.g.
|
|
171
|
-
millions of objects across many datasets, where a daily run touches only the
|
|
172
|
-
changed ones). Every document is stamped with its `source` (the dataset IRI)
|
|
173
|
-
and `last_seen` (the run id); deletion is a sweep, never special-cased:
|
|
174
|
-
|
|
175
|
-
- a successful dataset flush deletes the source’s documents the run did not
|
|
176
|
-
rewrite (`source = dataset && last_seen != runId`); a failed dataset is not
|
|
177
|
-
swept – its output is incomplete – and the next successful run reconciles;
|
|
178
|
-
- `commit` deletes every document whose source left the run’s selection (the
|
|
179
|
-
registry-membership sweep over `RunContext.selectedSources()`, which
|
|
180
|
-
includes datasets skipped as unchanged);
|
|
181
|
-
- `abort` only releases the lock: upserts are idempotent, so whatever landed
|
|
182
|
-
stays until the next run reconciles.
|
|
183
|
-
|
|
184
|
-
Document ids must be unique per (source, entity) – the caller keys them.
|
|
185
|
-
|
|
186
|
-
Both writers take a `Client` the caller owns (and reuses for queries), so this
|
|
187
|
-
package adds no connection or document type of its own – any object with an `id`
|
|
188
|
-
is a valid document, including the `SearchDocument`s `@lde/search` produces.
|
|
189
|
-
|
|
190
|
-
## Concurrency
|
|
191
|
-
|
|
192
|
-
Rebuilds are **single-flight per index**: `openRun` takes a lock (a marker
|
|
193
|
-
document in a `rebuild_locks` collection, created on demand) via Typesense’s
|
|
194
|
-
atomic create, so concurrent runs across pods never rebuild the same index at
|
|
195
|
-
once – a run opened while another holds the lock throws
|
|
196
|
-
`RebuildAlreadyRunning` (catch it to treat a concurrent rebuild as a graceful
|
|
197
|
-
skip). This keeps blue/green safe under replication: without it, two
|
|
198
|
-
same-millisecond rebuilds would collide on the versioned collection name and
|
|
199
|
-
one would delete the other’s in-flight build.
|
|
200
|
-
|
|
201
|
-
Limitations to design around:
|
|
31
|
+
## Documentation
|
|
202
32
|
|
|
203
|
-
|
|
204
|
-
consensus store. Under a TTL-reclaim race two rebuilds can briefly run at
|
|
205
|
-
once; this is safe because blue/green is idempotent (worst case: redundant
|
|
206
|
-
work and a transient orphaned collection) and in-place upserts are
|
|
207
|
-
idempotent.
|
|
208
|
-
- **Single-flight, not coalescing.** A run refused with `RebuildAlreadyRunning`
|
|
209
|
-
is _not_ queued. If you must capture state that changed mid-build,
|
|
210
|
-
re-trigger after the running rebuild finishes.
|
|
211
|
-
- **Lock TTL.** A rebuild running longer than `lockTtlMs` (default 10 minutes)
|
|
212
|
-
can be reclaimed by another caller and run concurrently; size the TTL above
|
|
213
|
-
your longest rebuild.
|
|
214
|
-
- **Membership-sweep cap.** The in-place membership sweep enumerates distinct
|
|
215
|
-
sources via a single facet, capped at `maxSweepableSources` (default
|
|
216
|
-
10 000); beyond that the commit throws rather than sweeping blind. Raise the
|
|
217
|
-
option (up to the engine’s `max_facet_values` limit) before an index nears
|
|
218
|
-
the cap so it stays a tunable guard rather than a hard wall.
|
|
33
|
+
See the [full documentation](https://ldelements.org/reference/search-typesense).
|
package/dist/query-compiler.d.ts
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
import type { SearchParams } from 'typesense/lib/Typesense/Documents.js';
|
|
2
2
|
import { type Filter, type SearchQuery, type SearchType } from '@lde/search';
|
|
3
3
|
/**
|
|
4
|
-
* Options for {@link buildSearchParams}
|
|
4
|
+
* Options for {@link buildSearchParams} – the query half of the engine
|
|
5
5
|
* adapter. {@link TypesenseSearchEngineOptions} extends this, so each knob is
|
|
6
6
|
* declared once and the engine forwards its options wholesale.
|
|
7
7
|
*/
|
|
8
8
|
export interface BuildSearchParamsOptions {
|
|
9
9
|
/**
|
|
10
10
|
* Cap on the number of buckets returned per facet (`max_facet_values`). Left
|
|
11
|
-
* unset, Typesense defaults to 10
|
|
11
|
+
* unset, Typesense defaults to 10 – too few for high-cardinality facets
|
|
12
12
|
* (publisher, keyword), so a deployment with such facets must raise it. Range
|
|
13
13
|
* facets return one bucket per declared range regardless, but a value > the
|
|
14
14
|
* range count is still safe.
|
|
@@ -27,7 +27,7 @@ export interface BuildSearchParamsOptions {
|
|
|
27
27
|
}
|
|
28
28
|
/**
|
|
29
29
|
* Compile the engine-neutral {@link SearchQuery} into Typesense search
|
|
30
|
-
* parameters
|
|
30
|
+
* parameters – the query half of the engine adapter. Pure (no client, no env),
|
|
31
31
|
* so the mapping is asserted directly in unit tests. Field names come from
|
|
32
32
|
* {@link physicalFields}, the same convention the projection and the collection
|
|
33
33
|
* schema use, so a query can never reference a field the index does not carry.
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"query-compiler.d.ts","sourceRoot":"","sources":["../src/query-compiler.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,sCAAsC,CAAC;AAEzE,OAAO,EAEL,KAAK,MAAM,EAEX,KAAK,WAAW,EAChB,KAAK,UAAU,EAEhB,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"query-compiler.d.ts","sourceRoot":"","sources":["../src/query-compiler.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,sCAAsC,CAAC;AAEzE,OAAO,EAEL,KAAK,MAAM,EAEX,KAAK,WAAW,EAChB,KAAK,UAAU,EAEhB,MAAM,aAAa,CAAC;AAarB;;;;GAIG;AACH,MAAM,WAAW,wBAAwB;IACvC;;;;;;OAMG;IACH,QAAQ,CAAC,cAAc,CAAC,EAAE,MAAM,CAAC;IACjC;;;;;;;;OAQG;IACH,QAAQ,CAAC,eAAe,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,IAAI,CAAC;CACrD;AAED;;;;;;GAMG;AACH,wBAAgB,iBAAiB,CAC/B,KAAK,EAAE,WAAW,EAClB,UAAU,EAAE,UAAU,EACtB,OAAO,GAAE,wBAA6B,GACrC,YAAY,CAAC,MAAM,CAAC,CAkCtB;AAqMD;;;;GAIG;AACH,wBAAgB,iBAAiB,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,CAEvD"}
|
package/dist/query-compiler.js
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
import { fold } from '@lde/text-normalization';
|
|
2
|
-
import { fieldNamed, filterOperator, filterOperatorFor, isoToUnixSeconds, isRangeFacet, pageForOffset, physicalFields, searchableFields, } from '@lde/search/adapter';
|
|
2
|
+
import { fieldNamed, filterOperator, filterOperatorFor, ID_FIELD, isoToUnixSeconds, isRangeFacet, pageForOffset, physicalFields, searchableFields, } from '@lde/search/adapter';
|
|
3
3
|
/**
|
|
4
4
|
* Compile the engine-neutral {@link SearchQuery} into Typesense search
|
|
5
|
-
* parameters
|
|
5
|
+
* parameters – the query half of the engine adapter. Pure (no client, no env),
|
|
6
6
|
* so the mapping is asserted directly in unit tests. Field names come from
|
|
7
7
|
* {@link physicalFields}, the same convention the projection and the collection
|
|
8
8
|
* schema use, so a query can never reference a field the index does not carry.
|
|
@@ -110,12 +110,22 @@ function compileFilterBy(where, searchType, onIgnoredFilter) {
|
|
|
110
110
|
.join(' && ');
|
|
111
111
|
}
|
|
112
112
|
function compileFilter(filter, searchType) {
|
|
113
|
+
// `id` is the Typesense document key, not a declared field. Exact `:=`
|
|
114
|
+
// membership, like a non-facet field ({@link compileMembership}), so an IRI
|
|
115
|
+
// cannot partial-match on a shared path segment. (`fetchLabels` resolves
|
|
116
|
+
// labels with the looser `id:[…]`; these are deliberately not the same
|
|
117
|
+
// clause.)
|
|
118
|
+
if (filter.field === ID_FIELD) {
|
|
119
|
+
return 'in' in filter && filter.in.length > 0
|
|
120
|
+
? `${ID_FIELD}:=[${filter.in.map(escapeFilterValue).join(',')}]`
|
|
121
|
+
: undefined;
|
|
122
|
+
}
|
|
113
123
|
const field = fieldNamed(searchType, filter.field);
|
|
114
124
|
if (field === undefined) {
|
|
115
125
|
return undefined;
|
|
116
126
|
}
|
|
117
127
|
// A clause whose operator does not match the field's kind (e.g. `range` on a
|
|
118
|
-
// keyword) would reach the engine as garbage syntax
|
|
128
|
+
// keyword) would reach the engine as garbage syntax – skip it instead.
|
|
119
129
|
if (filterOperatorFor(field.kind) !== filterOperator(filter)) {
|
|
120
130
|
return undefined;
|
|
121
131
|
}
|
package/dist/search.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"search.d.ts","sourceRoot":"","sources":["../src/search.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,WAAW,CAAC;
|
|
1
|
+
{"version":3,"file":"search.d.ts","sourceRoot":"","sources":["../src/search.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,WAAW,CAAC;AAExC,OAAO,EAGL,KAAK,cAAc,EAKnB,KAAK,UAAU,EAGf,KAAK,YAAY,EAEjB,KAAK,YAAY,EACjB,KAAK,YAAY,EACjB,KAAK,UAAU,EAEf,KAAK,SAAS,EACf,MAAM,aAAa,CAAC;AAarB,OAAO,EAGL,KAAK,wBAAwB,EAC9B,MAAM,qBAAqB,CAAC;AAG7B;;;;sDAIsD;AACtD,MAAM,WAAW,4BAA4B,CAC3C,QAAQ,SAAS,MAAM,GAAG,MAAM,CAChC,SAAQ,wBAAwB;IAChC;;;;;;;;OAQG;IACH,QAAQ,CAAC,WAAW,CAAC,EAAE,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC;IACnE;;;OAGG;IACH,QAAQ,CAAC,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,OAAO,KAAK,IAAI,CAAC;IACjD;;;;;;;OAOG;IACH,QAAQ,CAAC,eAAe,CAAC,EAAE,MAAM,CAAC;CACnC;AAED;;;GAGG;AACH,MAAM,WAAW,qBAAqB,CACpC,KAAK,SAAS,SAAS,UAAU,EAAE,GAAG,SAAS,UAAU,EAAE,CAC3D,SAAQ,YAAY,CAAC,KAAK,CAAC;IAC3B;;;;;;;OAOG;IACH,iBAAiB,CAAC,UAAU,EAAE,UAAU,CAAC,KAAK,CAAC,GAAG,MAAM,CAAC;CAC1D;AAED;;;;;GAKG;AACH,MAAM,WAAW,WAAW;IAC1B,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,UAAU,EAAE,SAAS,CAAC;IAC/B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;CAC1B;AAED;;;;;;;;;;;;;;;;;;;GAmBG;AACH,wBAAgB,2BAA2B,CACzC,KAAK,CAAC,KAAK,SAAS,SAAS,UAAU,EAAE,EAEzC,MAAM,EAAE,MAAM,EACd,MAAM,EAAE,YAAY,CAAC,KAAK,CAAC,EAC3B,OAAO,GAAE,4BAA4B,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,MAAM,CAAC,CAAM,GAChE,qBAAqB,CAAC,KAAK,CAAC,CA+T9B;AAyCD,8EAA8E;AAC9E,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAC;IAC7B,QAAQ,CAAC,IAAI,EAAE,SAAS,MAAM,EAAE,CAAC;CAClC;AA0HD;;;;;;;;;;;;;;GAcG;AACH,wBAAsB,WAAW,CAC/B,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,aAAa,CAAC,EACnC,MAAM,EAAE,SAAS,gBAAgB,EAAE,EACnC,OAAO,CAAC,EAAE,CAAC,KAAK,EAAE,KAAK,KAAK,IAAI,GAC/B,OAAO,CAAC,GAAG,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC,CAgDtC;AA6BD,oEAAoE;AACpE,MAAM,WAAW,uBAAuB;IACtC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS;QAAE,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAA;KAAE,EAAE,CAAC;IAC1E,QAAQ,CAAC,YAAY,CAAC,EAAE,SAAS;QAC/B,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;QAC5B,QAAQ,CAAC,MAAM,EAAE,SAAS;YACxB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;YACvB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;SACxB,EAAE,CAAC;KACL,EAAE,CAAC;CACL;AAED;;;;;;GAMG;AACH,wBAAgB,mBAAmB,CACjC,QAAQ,EAAE,uBAAuB,EACjC,UAAU,EAAE,UAAU,EACtB,MAAM,EAAE,WAAW,CAAC,MAAM,EAAE,cAAc,CAAC,GAC1C,YAAY,CAsCd"}
|
package/dist/search.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { assertTypeInSchema, assertValidQuery, displayLangOf, fieldNamed, isRangeFacet, labelFieldOf, outputFields, physicalFields, referenceFields, } from '@lde/search/adapter';
|
|
1
|
+
import { assertTypeInSchema, assertValidQuery, displayLangOf, fieldNamed, isRangeFacet, isUnsatisfiable, labelFieldOf, outputFields, physicalFields, referenceFields, } from '@lde/search/adapter';
|
|
2
2
|
import { buildSearchParams, escapeFilterValue, } from './query-compiler.js';
|
|
3
3
|
import { deriveCollectionName } from './collection-name.js';
|
|
4
4
|
/**
|
|
@@ -176,12 +176,21 @@ export function createTypesenseSearchEngine(client, schema, options = {}) {
|
|
|
176
176
|
// rejected up front, for EVERY caller.
|
|
177
177
|
assertTypeInSchema(schema, searchType);
|
|
178
178
|
assertValidQuery(query, searchType);
|
|
179
|
+
// Asks for no document (an empty `id` membership): answer it without a
|
|
180
|
+
// round-trip rather than dispatching a query whose only filter compiles
|
|
181
|
+
// away, which would hand back the whole collection.
|
|
182
|
+
if (isUnsatisfiable(query)) {
|
|
183
|
+
return { total: 0, hits: [], facets: {} };
|
|
184
|
+
}
|
|
179
185
|
const params = buildSearchParams(query, searchType, options);
|
|
180
186
|
const cachedLabelsPromise = startCachedLabels(searchType);
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
187
|
+
// ONE search, but through `multi_search` (POST) like the facet batch and
|
|
188
|
+
// the label lookup: `documents().search()` is a GET, so a long
|
|
189
|
+
// `filter_by` – a batch `id` lookup, or any membership over many IRIs –
|
|
190
|
+
// would overflow Typesense's 4000-char query-string limit. POST puts it
|
|
191
|
+
// in the body, so the filter is bounded by the caller's request, not by
|
|
192
|
+
// the URL.
|
|
193
|
+
const response = await performSingleSearch(client, collections.get(searchType.class), params);
|
|
185
194
|
const labels = await resolveLabels(cachedLabelsPromise, () => labelLookupGroups([response], labelSources.get(searchType.class), outputReferenceSources.get(searchType.class) ?? []));
|
|
186
195
|
return parseSearchResponse(response, searchType, labels);
|
|
187
196
|
},
|
|
@@ -195,12 +204,23 @@ export function createTypesenseSearchEngine(client, schema, options = {}) {
|
|
|
195
204
|
}
|
|
196
205
|
const collection = collections.get(searchType.class);
|
|
197
206
|
const cachedLabelsPromise = startCachedLabels(searchType);
|
|
207
|
+
// A query asking for no document faces the same inversion as in
|
|
208
|
+
// `search()`: its only filter would compile away and it would count
|
|
209
|
+
// facets over the whole collection. Such a query is answered with empty
|
|
210
|
+
// facets and kept OUT of the batch, so `dispatched` maps each sent search
|
|
211
|
+
// back to its caller position.
|
|
212
|
+
const dispatched = queries
|
|
213
|
+
.map((query, index) => ({ query, index }))
|
|
214
|
+
.filter(({ query }) => !isUnsatisfiable(query));
|
|
215
|
+
if (dispatched.length === 0) {
|
|
216
|
+
return queries.map(() => ({ facets: {} }));
|
|
217
|
+
}
|
|
198
218
|
// The whole batch travels as ONE multi_search round-trip. Each query
|
|
199
219
|
// compiles as facet-only regardless of what it carries: no hits
|
|
200
220
|
// (per_page 0) and no ordering – nothing is transferred or sorted that
|
|
201
221
|
// this method cannot return.
|
|
202
222
|
const { results } = (await client.multiSearch.perform({
|
|
203
|
-
searches:
|
|
223
|
+
searches: dispatched.map(({ query }) => ({
|
|
204
224
|
collection,
|
|
205
225
|
...buildSearchParams({ ...query, orderBy: [], limit: 0, offset: 0 }, searchType, options),
|
|
206
226
|
})),
|
|
@@ -211,12 +231,22 @@ export function createTypesenseSearchEngine(client, schema, options = {}) {
|
|
|
211
231
|
// multi_search reports a failed entry inline instead of rejecting the
|
|
212
232
|
// call; pass that through as a per-query outcome – naming the facets,
|
|
213
233
|
// not the position, since the batch order is the caller's internal –
|
|
214
|
-
// so one failed query never discards its siblings' facets.
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
234
|
+
// so one failed query never discards its siblings' facets. Every caller
|
|
235
|
+
// position is answered: a dispatched query by its entry, an
|
|
236
|
+
// unsatisfiable one by empty facets.
|
|
237
|
+
const outcomes = queries.map(() => ({ facets: {} }));
|
|
238
|
+
dispatched.forEach(({ index }, entry) => {
|
|
239
|
+
const result = results[entry];
|
|
240
|
+
outcomes[index] =
|
|
241
|
+
result === undefined || 'error' in result
|
|
242
|
+
? {
|
|
243
|
+
error: new Error(`Typesense facet search for “${queries[index].facets.join('”, “')}” failed${result?.code !== undefined ? ` (${result.code})` : ''}: ${result?.error ?? 'no result entry'}`),
|
|
244
|
+
}
|
|
245
|
+
: {
|
|
246
|
+
facets: parseSearchResponse(result, searchType, labels).facets,
|
|
247
|
+
};
|
|
248
|
+
});
|
|
249
|
+
return outcomes;
|
|
220
250
|
},
|
|
221
251
|
};
|
|
222
252
|
// The runtime object is string-keyed; the literal-schema typing narrows it.
|
|
@@ -329,6 +359,25 @@ function labelLookupGroups(responses, sources, outputSources) {
|
|
|
329
359
|
iris: [...iris],
|
|
330
360
|
}));
|
|
331
361
|
}
|
|
362
|
+
/**
|
|
363
|
+
* Dispatch one search as a single-entry `multi_search` (POST). `multi_search`
|
|
364
|
+
* reports a failed entry inline instead of rejecting the call, so the entry is
|
|
365
|
+
* translated back into a throw: `search()` rejects on engine failure, and that
|
|
366
|
+
* contract must not weaken just because the transport moved off GET.
|
|
367
|
+
*/
|
|
368
|
+
async function performSingleSearch(client, collection, params) {
|
|
369
|
+
const { results } = (await client.multiSearch.perform({
|
|
370
|
+
searches: [{ collection, ...params }],
|
|
371
|
+
}));
|
|
372
|
+
const [result] = results;
|
|
373
|
+
if (result === undefined) {
|
|
374
|
+
throw new Error(`Typesense search of “${collection}” returned no result entry.`);
|
|
375
|
+
}
|
|
376
|
+
if ('error' in result) {
|
|
377
|
+
throw new Error(`Typesense search of “${collection}” failed${result.code !== undefined ? ` (${result.code})` : ''}: ${result.error}`);
|
|
378
|
+
}
|
|
379
|
+
return result;
|
|
380
|
+
}
|
|
332
381
|
/**
|
|
333
382
|
* Resolve labels from each group’s label-source collection. Labels are
|
|
334
383
|
* reconstructed from the source type’s `label` declaration ({@link
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@lde/search-typesense",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.16.0",
|
|
4
4
|
"description": "Typesense implementation of the @lde/search SearchEngine port: collection-schema builder, query compiler, label-resolving result reconstruction, and blue/green index lifecycle. Engine-specific (Typesense) but domain-agnostic.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"url": "git+https://github.com/ldelements/lde.git",
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
"!**/*.tsbuildinfo"
|
|
26
26
|
],
|
|
27
27
|
"dependencies": {
|
|
28
|
-
"@lde/search": "^0.
|
|
28
|
+
"@lde/search": "^0.13.0",
|
|
29
29
|
"@lde/text-normalization": "^0.1.2",
|
|
30
30
|
"tslib": "^2.3.0",
|
|
31
31
|
"typesense": "^3.0.6"
|