iit-mcp-hudu 0.6.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +45 -4
- package/CHANGELOG.md +88 -0
- package/README.md +107 -20
- package/dist/config.js +233 -2
- package/dist/doctor.js +33 -0
- package/dist/main.js +7 -1
- package/dist/search-budget.js +121 -27
- package/dist/tools/reads.js +21 -4
- package/dist/tools/register.js +23 -9
- package/dist/tools/results.js +13 -0
- package/dist/tools/schemas.js +20 -0
- package/dist/tools/search-policy.js +13 -8
- package/dist/tools/search-prose.js +153 -34
- package/dist/tools/search.js +30 -15
- package/dist/tools/server.js +68 -29
- package/dist/tools/tool-prose.js +1 -1
- package/dist/tools/typesense-probe.js +73 -0
- package/dist/tools.js +1 -1
- package/package.json +2 -2
package/.env.example
CHANGED
|
@@ -106,16 +106,57 @@ LOG_LEVEL=info
|
|
|
106
106
|
# hudu_read; the dedicated per-resource resolve tools are unaffected.
|
|
107
107
|
# HUDU_SEARCH_RESOURCES=articles,assets,companies,users,groups,websites
|
|
108
108
|
|
|
109
|
-
# Which body
|
|
110
|
-
#
|
|
111
|
-
#
|
|
112
|
-
#
|
|
109
|
+
# Which body tier hudu_search uses: auto (answer from Hudu search now, build the body index in the
|
|
110
|
+
# background), index (a call may wait for a complete body-index answer), typesense (search Hudu's
|
|
111
|
+
# whole-corpus Typesense index -- needs the pair below) or vendor (never use a body tier). Unset means
|
|
112
|
+
# the transport default: auto under http, index under stdio, and typesense under BOTH when the
|
|
113
|
+
# Typesense pair is configured (it has no cold start, so the transport asymmetry does not apply).
|
|
114
|
+
# Naming typesense without the pair is a startup error. Under http the tier is policy, not a call
|
|
115
|
+
# parameter, so the served schema offers neither tier nor refresh.
|
|
113
116
|
# HUDU_SEARCH_TIER_POLICY=auto
|
|
114
117
|
|
|
115
118
|
# Body-index TTL in milliseconds (the SDK's indexTtlMs; default 600000, ten minutes). Beyond it, an
|
|
116
119
|
# answer is marked stale and the next search rebuilds in the background.
|
|
117
120
|
# HUDU_SEARCH_INDEX_TTL=600000
|
|
118
121
|
|
|
122
|
+
# --- Typesense body tier (optional; both URL and key, or neither) -------------------------------
|
|
123
|
+
#
|
|
124
|
+
# Hudu's own Typesense index already holds the whole corpus (article bodies and asset custom-field
|
|
125
|
+
# text), so this tier makes hudu_search body-complete without the in-memory index and its heap cap.
|
|
126
|
+
# With the pair set, `typesense` becomes the default tier for `hudu_search`: the call reads the whole
|
|
127
|
+
# corpus from Hudu's index instead of building a local body index. HUDU_SEARCH_TIER_POLICY=index or
|
|
128
|
+
# vendor still overrides it; naming typesense without the pair fails boot.
|
|
129
|
+
#
|
|
130
|
+
# HUDU_TYPESENSE_URL is the ORIGIN only -- no path, no query. Three ways to reach it:
|
|
131
|
+
# LAN / VPN : https://typesense.hudu.intellectit.com.au (source IP must be allow-listed)
|
|
132
|
+
# stdio anywhere : http://127.0.0.1:8108 with `ssh -L 8108:127.0.0.1:8108 <hudu-host>` running
|
|
133
|
+
# fixed egress IP: a public hostname, with that CIDR allow-listed on the Hudu host
|
|
134
|
+
# HUDU_TYPESENSE_API_KEY is a SEARCH-ONLY key (`documents:search` on the two collections below).
|
|
135
|
+
# Provisioning it is an ops step on the Hudu host; nothing in this server creates or rotates a key.
|
|
136
|
+
#
|
|
137
|
+
# Setting one without the other is a startup error. Leaving BOTH unset keeps today's behaviour
|
|
138
|
+
# exactly: the in-memory body index, `HUDU_SEARCH_TIER_POLICY` as documented above, nothing else.
|
|
139
|
+
# The key is never logged and never returned by a tool: only booleans and result codes leave the
|
|
140
|
+
# process. `doctor` stays zero-wire and says nothing about this tier.
|
|
141
|
+
# HUDU_TYPESENSE_URL=https://typesense.hudu.intellectit.com.au
|
|
142
|
+
# HUDU_TYPESENSE_API_KEY=
|
|
143
|
+
|
|
144
|
+
# Collection names, defaults Hudu's own. Keep them distinct; they name the index a search reads.
|
|
145
|
+
# HUDU_TYPESENSE_ARTICLES_COLLECTION=articles
|
|
146
|
+
# HUDU_TYPESENSE_ASSETS_COLLECTION=assets
|
|
147
|
+
|
|
148
|
+
# Per-search deadline in milliseconds (default 4000, maximum 15000). A search has a vendor fallback,
|
|
149
|
+
# so a longer wait is never the right trade; a value above the maximum fails boot.
|
|
150
|
+
# HUDU_TYPESENSE_TIMEOUT_MS=4000
|
|
151
|
+
|
|
152
|
+
# The credential re-read flag (default true). The SDK re-reads every Typesense hit through the Hudu
|
|
153
|
+
# API under the CALLER's own credential before returning it; this repo only sets the flag. Set true
|
|
154
|
+
# unless you are certain about the deployment: Hudu's index does not carry folder permissions and
|
|
155
|
+
# keeps discarded articles, so with this false a hit can be a record the caller's Hudu permission
|
|
156
|
+
# would hide, or one that was deleted.
|
|
157
|
+
# SINGLE-TENANT, NON-RESTRICTED, READ-ONLY DEPLOYMENTS ONLY.
|
|
158
|
+
# HUDU_TYPESENSE_VERIFY_HITS=true
|
|
159
|
+
|
|
119
160
|
# Image tag docker-compose.yml builds and runs as. scripts/deploy.sh sets it to the git short SHA
|
|
120
161
|
# of the deployed commit, so the running container is always tied to a commit and the previous
|
|
121
162
|
# tag stays resolvable for rollback. Do not deploy by hand with it unset, and never tag :latest.
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,93 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.0
|
|
4
|
+
|
|
5
|
+
**Upgrading from 0.7.0:** no configuration change for any existing deployment. Six new optional
|
|
6
|
+
variables turn on a Typesense body tier; with both `HUDU_TYPESENSE_URL` and `HUDU_TYPESENSE_API_KEY`
|
|
7
|
+
unset, the default tier, prose, the served tool schema, the doctor report, boot output and search
|
|
8
|
+
results are byte-identical to the env-wiring head (#59, 41ad81e) — not to 0.7.0, whose `hudu_read`
|
|
9
|
+
description differs; the frozen-surface test proves identity to that head only. node-hudu is pinned at
|
|
10
|
+
0.21.0 (exact), which is the release
|
|
11
|
+
that types `search.typesense` — the boundary cast the tier needed is gone.
|
|
12
|
+
|
|
13
|
+
- Add: `HUDU_TYPESENSE_*` configure the optional Typesense body tier and the SDK client is built with
|
|
14
|
+
the resulting typed `search.typesense` block. The pair is set together or not at all: one without
|
|
15
|
+
the other fails boot naming both, the URL must be an http(s) origin (no path, query, fragment or
|
|
16
|
+
userinfo), the two collection names must differ, and the deadline is bounded at 15000 ms. The key
|
|
17
|
+
is never logged and never returned by a tool. `HUDU_TYPESENSE_VERIFY_HITS` accepts any case and
|
|
18
|
+
treats a blank as unset.
|
|
19
|
+
- Add: optional **Typesense body tier**. It searches Hudu's own Typesense index, so article bodies
|
|
20
|
+
and asset custom-field text are covered for the WHOLE corpus instead of the in-memory index's
|
|
21
|
+
heap-capped slice. `HUDU_TYPESENSE_VERIFY_HITS` (default `true`) is the credential re-read flag:
|
|
22
|
+
the SDK re-reads every hit through the Hudu API under the caller's own credential, because Hudu's
|
|
23
|
+
index carries neither folder permissions nor deletions. This repo sets the flag (a test asserts it
|
|
24
|
+
reaches the client) and has not measured the re-read. `false` is documented as single-tenant,
|
|
25
|
+
non-restricted, read-only deployments only.
|
|
26
|
+
- Add: `HUDU_SEARCH_TIER_POLICY` accepts `typesense`, and when the pair is configured the default
|
|
27
|
+
tier is `typesense` under **both** transports — the cold-start asymmetry `auto` exists for does
|
|
28
|
+
not apply to an index that is already built. Naming `typesense` without the pair is a startup
|
|
29
|
+
error, never a silent fallback, and so is naming it on an SDK build whose catalogue lacks the
|
|
30
|
+
probe operation. The corpus gate is renamed `bodySearchable` and now covers every body tier: when
|
|
31
|
+
`HUDU_SEARCH_RESOURCES` excludes articles or assets, the policy stays `vendor` and a request for
|
|
32
|
+
`index`, `auto` or `typesense` is refused, naming both body tiers.
|
|
33
|
+
- Add: `hudu_get_api_info` reports `typesense: { configured, reachable, collections, probeMs }`
|
|
34
|
+
when the tier is configured (absent otherwise), from the SDK's one-request probe that a
|
|
35
|
+
search-only key may run; `error {code, message}` appears only when the probe could not answer, and
|
|
36
|
+
its code says which question failed — `NOT_IMPLEMENTED` (this SDK version has no probe),
|
|
37
|
+
`POLICY_DENIED` (the client's policy does not permit it), `PROBE_FAILED` (the call threw without a
|
|
38
|
+
string code, so the failure cannot be attributed) or the endpoint's own code. The tool
|
|
39
|
+
description names each field, and a test pins the exact emitted key set, because the output schema
|
|
40
|
+
keeps that field open (`z.unknown()`) to stay inside the `tools/list` byte budget. `doctor` stays zero-wire: it gains a config-shape row
|
|
41
|
+
(`configured`, `keyPresent`, `urlShapeOk`) only, and the row is absent when the tier is off.
|
|
42
|
+
- Docs: `hudu_search` help names `meta.bodySource` (`index` | `typesense` | `none`) and all five
|
|
43
|
+
Typesense degraded reasons, grouped by what the operator must do — `typesense-unavailable`
|
|
44
|
+
(refused or timed out: check URL, tunnel, egress), `typesense-query-error` (the collection name or
|
|
45
|
+
query is wrong: Typesense answered 400/404/422, so the fix is `HUDU_TYPESENSE_*_COLLECTION` and
|
|
46
|
+
NOT an outage), `typesense-unauthorized` (key rejected), `typesense-hits-stale` (records still in
|
|
47
|
+
Hudu's index that this credential can no longer read: retry later or widen), and
|
|
48
|
+
`typesense-hits-unhydratable` (candidates unreadable for structural reasons — no company_id,
|
|
49
|
+
deleted layout, no usable id — so the data is the cause, not staleness). The staleness entry also
|
|
50
|
+
carries a stated LIMITATION: which answer Hudu returns for a permission-hidden record is
|
|
51
|
+
unverified upstream (that needs a live tenant). What is settled is the split — a 404 is a silent
|
|
52
|
+
stale drop (the record is genuinely gone), while a 403 is a surfaced read failure, so the answer
|
|
53
|
+
reports `complete: false` and names the resource in `meta.errors` and a hidden record is not folded
|
|
54
|
+
into staleness. Reported as a limitation, with no sixth degraded reason invented for it. The five strings live in
|
|
55
|
+
ONE exported literal (`TYPESENSE_DEGRADED_REASONS`, in the SDK's union order) that both the help
|
|
56
|
+
builder and the test read, so the prose cannot drift from the assertion; this server holds no
|
|
57
|
+
schema enum of its own for them. `degraded.reason` is passed through unchanged — a walk test feeds
|
|
58
|
+
each of the five, and a reason the server has never seen, and asserts the response and the text
|
|
59
|
+
carry them verbatim, because a rename here would break the pairing silently. The compact result
|
|
60
|
+
header states the body source. The tier sentence
|
|
61
|
+
names the corpus resolution where it states the policy: on a deployment that excludes articles or
|
|
62
|
+
assets, a configured Typesense tier is corpus-wide too, so the corpus gate resolves it to `vendor`
|
|
63
|
+
and the answer carries no body source. All of it is present only where the tier is configured, so
|
|
64
|
+
the unset surface stays byte-identical. README and `.env.example` document the six variables, the
|
|
65
|
+
reach modes (LAN, SSH tunnel, allow-listed egress) and the verification trade.
|
|
66
|
+
- Add: the five reason strings live in ONE exported literal (`TYPESENSE_DEGRADED_REASONS`), and
|
|
67
|
+
`test/tools/typesense-tier.test.ts` compares that literal against the SDK's union at test time, in
|
|
68
|
+
order. The server does not compare at runtime: it prints the reason it receives verbatim. Counts
|
|
69
|
+
stay disclosures, never reasons.
|
|
70
|
+
- Tests: config XOR both ways, defaults and every refusal branch, key never echoed; the block
|
|
71
|
+
reaching every SDK client and staying absent when unset; the default-tier matrix
|
|
72
|
+
(`configured`/`unset` x `stdio`/`http` x full/excluded corpus); the schema enum and the prose (only
|
|
73
|
+
when configured); `bodySource` header rendering; the api_info probe block and its emitted key set;
|
|
74
|
+
the doctor row without key material; the capability gate failing on a mock SDK without the probe;
|
|
75
|
+
the runtime enum comparison against the SDK's; and a documentation drift test derived from the
|
|
76
|
+
config schema rather than a hand-written list.
|
|
77
|
+
|
|
78
|
+
## 0.7.0
|
|
79
|
+
|
|
80
|
+
**Upgrading from 0.6.1:** no configuration change. The articles in the body index are now reserved their slots, so an account with many more assets than articles indexes every article instead of the freshest few: the 512 MiB write-enabled profile derives a 1,820-document cap against a larger article corpus and therefore indexes no assets at all (they stay searchable through Hudu's own title search). node-hudu is now 0.20.0, and `meta.index.docs.<resource>.capped` reports what the caps dropped. Serve over `http` with a 1 GiB container to index a whole midsize article corpus.
|
|
81
|
+
|
|
82
|
+
- Fix: articles can no longer be crowded out of the body index by assets. node-hudu 0.20.0 adds `search.maxDocsByResource`, a hard reservation inside `maxDocs`; the server now names `articles` with the WHOLE document cap, so articles keep their slots (up to the cap) and assets fill whatever room is left, in `updated_at` order. MEASURED on the production tenant (19,991 assets + 3,204 articles = 23,195 documents): without a reservation the 21,845-document cap retained 19,991 assets and 1,854 of the 3,204 articles, so 1,350 articles had no index entry at all while asset rows (titles and custom fields, no long text) held 91% of the cap; 0.6.1's expectation that dividing the pool by two would keep every article was therefore false. With the cap reserved, that tenant's 21,845 documents hold all 3,204 articles and the remaining 18,641 asset slots (arithmetic from the SDK's reservation semantics, verified with a synthetic probe: 10 articles + 500 assets at a 100-document cap give 10 + 90, and 300 articles + 5 assets give 100 + 0). The asset share is derived and can be zero: assets keep `maxDocs - min(articles, maxDocs)` slots, so the 512 MiB write-enabled profile (a 1,820-document cap against 3,204 articles) indexes no assets and leans on Hudu's own title search for them. A quarter-reservation was rejected because a reservation is also a hard ceiling on the named resource (300 articles + 5 assets at a 100-document cap retained 30 documents with a quarter, 100 with the whole cap).
|
|
83
|
+
- Chore: node-hudu 0.19.0 -> 0.20.0 (`server.json` `versionRange` 0.20.0). The SDK's other change also reaches us: `search.maxDocsByResource` is the only new option, and `meta.index.docs.<resource>.capped` (documents dropped per resource by the caps) now travels through `sanitizeIndexMeta` unchanged.
|
|
84
|
+
|
|
85
|
+
## 0.6.1
|
|
86
|
+
|
|
87
|
+
**Upgrading from 0.6.0:** no configuration change. Each scoped client's body index is deeper: a read-only http deployment gives one credential a third of the pool instead of an eighth. A write-enabled deployment on a small heap gets a shallower index than 0.6.0 (a ninth of the pool), because the eviction margin and the per-mode caches are both counted.
|
|
88
|
+
|
|
89
|
+
- Fix: a read-only http deployment now gives each scoped client a THIRD of the pooled index text budget instead of an eighth (the pool is divided by two clients per cache plus one eviction of margin), so the body index is deep enough to search the articles it holds. MEASURED on the production tenant (2026-10-26) with the former division by eight at 1 GiB: a scope retained 4,666 assets and 1,478 of 3,204 articles; 1,036 article bodies were indexed, 317 had been evicted by the text budget and 3 truncated (the counts do not decompose the 1,478 — some retained articles carry no body text). The cause is the SDK's document cap, which keeps the freshest `updated_at` across ALL resources, so newer assets displaced older articles. A read-only deployment at a ~816 MB V8 ceiling now gets 17 MiB of text and 17,408 documents per scope instead of 6 MiB and 6,144 (INFERRED to keep all 3,204 articles and about 2,800 of their bodies, from the measured article density at 6,144; the SDK's cap is global, so this is an expectation, not a guarantee); a 512 MiB container gets 5.33 MiB and 5,461 instead of 2 MiB and 2,048. The document cap lost its 1,500-document floor and now follows the share (one document per KiB): the smallest pool with writes (an 8 MiB pool, 932,067 bytes per client) gets 910 documents instead of a 1,500-document cap above its own share. The pool is also divided by the number of per-mode caches: a read-only deployment creates one (`read`), a write-enabled deployment three (`read`, `write`, `delete`), because each mode entry keeps its own cache of scoped clients, and each cache's clients are counted with one extra eviction transient (the scope the LRU just dropped while an in-flight request finishes). So a write-enabled deployment gets 5.67 MiB and 5,802 per scope at ~816 MB, and a write-enabled 512 MiB container gets LESS than before (1.78 MiB and 1,820 instead of 2 MiB and 2,048) because the eviction margin is now counted, and in every profile the share is sized so the cache capacity (clients x caches) plus one eviction per cache stays within the pool; concurrent evictions beyond that margin can exceed it until their requests finish, because the SDK cannot revoke a live index. This also corrects a pre-existing oversubscription, since with eight shares three caches could hold 144 MiB of text against a 51 MiB pool. The trade is breadth for depth: at most two (origin, credential) pairs stay warm per mode cache, and a credential occupies one client per cache (read, plus write and delete when writes are enabled), so a deployment serving more credentials also loses each evicted scope's read-through TTL cache and re-fetches more from Hudu, and rebuilds an evicted index on demand (searches answer from the vendor tier meanwhile, unless `HUDU_SEARCH_TIER_POLICY=index` makes the call wait on that rebuild, with progress and cancellation), and a deployment needing both breadth and depth needs a larger heap. `MAX_SCOPED_CLIENTS` still equals `SCOPED_INDEX_BUDGET_DIVISOR` (now 2); the per-mode cache count (3: read, write, delete) is the second divisor, and `SCOPED_INDEX_EVICTION_TRANSIENT` (1) is added to the per-cache count before multiplying — it is a margin, not a third multiplicand.
|
|
90
|
+
|
|
3
91
|
## 0.6.0
|
|
4
92
|
|
|
5
93
|
**Upgrading from 0.5.1:** under `MCP_TRANSPORT=http` the served `hudu_search` schema no longer offers `tier` or `refresh`, and the tier is server-side policy: `auto` by default, so a call answers from the vendor tier while the index builds. Set `HUDU_SEARCH_TIER_POLICY=index` for an http deployment that should wait. The body-index text budget and document cap are now derived from the process heap (a sixteenth of the V8 ceiling) instead of the SDK's 64 MiB / 50,000 defaults; no configuration change is needed. Under `stdio` the default tier stays `index` and both fields are still served.
|
package/README.md
CHANGED
|
@@ -31,7 +31,7 @@ four old names. Rows are compact summaries by default (id, name, key scalar fiel
|
|
|
31
31
|
25-row assets page under 20,000 characters instead of ~88,000; pass `expand: true` for the full
|
|
32
32
|
records (`hudu_read` `articles.list` rows are always full). Article rows carry no body unless expanded. Paging is unchanged (`asset_layouts` still takes no `page_size`).
|
|
33
33
|
|
|
34
|
-
These tools run on node-hudu 0.
|
|
34
|
+
These tools run on node-hudu 0.20.0; the `SDK behaviour (0.12.0)` section below covers the behaviours that shape them: redirect refusal, the closed input contract, the error envelope, and the client opt-ins. Numeric ids must be positive integers on every tool; anything else is refused before a request is made.
|
|
35
35
|
|
|
36
36
|
`hudu_fetch_many` (node-hudu 0.12.0) fetches up to 20 single records across mixed resources in
|
|
37
37
|
one call: each item is exactly `{ resource, id, fields? }` — one of the 22 bare-id resources, a
|
|
@@ -158,8 +158,10 @@ gate on who can use it. It is only ever presented to the default
|
|
|
158
158
|
`HUDU_BASE_URL`: a request that names some other origin and brings no key of its own gets no
|
|
159
159
|
credential and fails closed, so a caller cannot aim the server's key at a host it controls. (A
|
|
160
160
|
blank value, as a copied `.env.example` leaves it, is treated as unset.) Each (origin, credential)
|
|
161
|
-
pair gets its own scoped client (its own search index) cached by the pair's hash and capped
|
|
162
|
-
|
|
161
|
+
pair gets its own scoped client (its own search index) cached by the pair's hash and capped — at most
|
|
162
|
+
two pairs stay warm in EACH mode cache, so a credential occupies one client per cache (read, plus
|
|
163
|
+
write and delete when writes are enabled) and a further scope rebuilds its index on demand — so one
|
|
164
|
+
caller's indexed article bodies can never answer another's search.
|
|
163
165
|
|
|
164
166
|
`HUDU_ALLOWED_BASE_HOSTS` (`http` only; required unless `HUDU_ALLOW_ANY_BASE_HOST=true`) restricts which origins a caller may name in
|
|
165
167
|
`X-Hudu-Base-Url`: a comma-separated list of exact hostnames or `*.example.com` subdomain suffixes
|
|
@@ -275,15 +277,56 @@ which is exactly that knob's documented purpose. `meta.index` reports the index
|
|
|
275
277
|
`stdio` the one long-lived index stays warm, both fields stay in the schema, and the default is
|
|
276
278
|
`"index"`; a caller there can still pass `tier:"index"` for a complete first answer.
|
|
277
279
|
|
|
278
|
-
`HUDU_SEARCH_TIER_POLICY=auto|index|vendor` sets that policy explicitly (default `auto`
|
|
279
|
-
`http`, `index` under `stdio
|
|
280
|
-
|
|
281
|
-
|
|
280
|
+
`HUDU_SEARCH_TIER_POLICY=auto|index|vendor|typesense` sets that policy explicitly (default `auto`
|
|
281
|
+
under `http`, `index` under `stdio`, and `typesense` under **both** when the Typesense pair is
|
|
282
|
+
configured — it has no cold start, so the transport asymmetry does not apply): `vendor` forces the
|
|
283
|
+
vendor tier and never uses a body tier, `index` lets a call wait for a complete body-index answer,
|
|
284
|
+
`typesense` searches the whole corpus through Hudu's own index. Naming `typesense` without the pair
|
|
285
|
+
fails boot. When `HUDU_SEARCH_RESOURCES` excludes articles or assets the policy is `vendor` and no
|
|
286
|
+
body tier runs, whatever the variable says: every body tier is corpus-wide.
|
|
287
|
+
|
|
288
|
+
A search answer also reports **which** body source produced it: `meta.bodySource` is `index`,
|
|
289
|
+
`typesense` or `none`, and the compact header names it. `meta.degraded.reason` gains
|
|
290
|
+
`typesense-unavailable` (the configured endpoint refused or timed out; the answer fell back to the
|
|
291
|
+
local index or the vendor tier) and `typesense-unauthorized` (the key was rejected — the key is the
|
|
292
|
+
thing to fix), so a body-blind answer always says why.
|
|
282
293
|
`HUDU_SEARCH_INDEX_TTL` (milliseconds, default `600000`) is the body index's freshness window. What
|
|
283
294
|
the SDK does past it — answer from the stale index while rebuilding in the background — is
|
|
284
295
|
UNVERIFIED here: this repo sets the knob (a test asserts it reaches the client) but has not measured
|
|
285
296
|
the stale-rebuild behaviour.
|
|
286
297
|
|
|
298
|
+
`HUDU_TYPESENSE_*` configure the optional Typesense body tier — the same index Hudu's own UI
|
|
299
|
+
searches, so body coverage does not depend on this process's heap:
|
|
300
|
+
|
|
301
|
+
| Variable | Default | Meaning |
|
|
302
|
+
|---|---|---|
|
|
303
|
+
| `HUDU_TYPESENSE_URL` | — | Origin only (`https://typesense.example.com`, or `http://127.0.0.1:8108` through an SSH tunnel). No path, query or fragment. |
|
|
304
|
+
| `HUDU_TYPESENSE_API_KEY` | — | Search-only key (`documents:search` on the two collections). Never logged, never returned by a tool. |
|
|
305
|
+
| `HUDU_TYPESENSE_ARTICLES_COLLECTION` | `articles` | Articles collection name. |
|
|
306
|
+
| `HUDU_TYPESENSE_ASSETS_COLLECTION` | `assets` | Assets collection name; must differ from the articles one. |
|
|
307
|
+
| `HUDU_TYPESENSE_TIMEOUT_MS` | `4000` | Per-search deadline; `15000` is the maximum and a larger value fails boot. |
|
|
308
|
+
| `HUDU_TYPESENSE_VERIFY_HITS` | `true` | Re-read every hit through the Hudu API under the caller's credential (see below). |
|
|
309
|
+
|
|
310
|
+
`HUDU_TYPESENSE_URL` and `HUDU_TYPESENSE_API_KEY` are set together or not at all: one without the
|
|
311
|
+
other fails boot, naming both. With both set the server builds its SDK client with this block and
|
|
312
|
+
the boot log gains one line, `typesense: enabled (collections articles, assets)` — the collections
|
|
313
|
+
only, never the URL or the key. With both unset nothing changes, boot output included: no Typesense
|
|
314
|
+
request is ever made.
|
|
315
|
+
|
|
316
|
+
Setting the pair makes the `typesense` tier the default for `hudu_search`: the call reads the whole
|
|
317
|
+
corpus directly from Hudu's index, with no local build. `HUDU_SEARCH_TIER_POLICY` still chooses the
|
|
318
|
+
tier explicitly — `index` or `vendor` overrides that default, and naming `typesense` without the pair
|
|
319
|
+
fails boot. The served schema advertises the `typesense` value only on a deployment that configured
|
|
320
|
+
it.
|
|
321
|
+
|
|
322
|
+
`HUDU_TYPESENSE_VERIFY_HITS` (default `true`) is the credential re-read flag. What the SDK does when
|
|
323
|
+
it is on — re-read every candidate id through the Hudu API under the credential that made the call
|
|
324
|
+
before returning a hit — is the SDK's behaviour: this repo sets the flag (a test asserts it reaches
|
|
325
|
+
the client) and has not measured the re-read itself. Hudu's index carries neither folder permissions
|
|
326
|
+
nor deletions (it keeps discarded articles), so the re-read is what stops a hit being a record the
|
|
327
|
+
caller may not see. `false` skips it: faster, but a hit can be discarded or permission-hidden. Use
|
|
328
|
+
`false` only on a single-tenant, non-restricted, read-only deployment.
|
|
329
|
+
|
|
287
330
|
A request cancelled while a call waits on a body-index build answers instead of throwing.
|
|
288
331
|
`meta.degraded.reason` stays `body-not-indexed` (the SDK's closed union), `meta.cancelled` records
|
|
289
332
|
`{ during: "index-build", waited }` as a deliberate SERVER extension (the SDK's meta has no
|
|
@@ -298,7 +341,7 @@ surface headroom); the meta vocabulary is enforced by the tests that publish it
|
|
|
298
341
|
|
|
299
342
|
The body index is bounded by TWO caps derived from the same heap estimate: a text budget and a
|
|
300
343
|
document cap. The text budget is a SIXTEENTH of the process's V8 heap ceiling, clamped to 8-64 MiB;
|
|
301
|
-
the document cap is one document per KiB of that budget (`maxDocs`),
|
|
344
|
+
the document cap is one document per KiB of that budget (`maxDocs`), capped at 50,000. Both
|
|
302
345
|
travel the SDK's `search` options, so neither is an env knob. The divisor moved from 10 to 16 in
|
|
303
346
|
round 11 because the tenth left no usable production margin (the arithmetic below).
|
|
304
347
|
|
|
@@ -308,28 +351,70 @@ of heap and died with `Reached heap limit` mid-search. It measured the ~90 MB ba
|
|
|
308
351
|
4 bytes of index structures per byte of text ACTUALLY HELD, not per byte of budget: that index held
|
|
309
352
|
~41 MB of text out of its 64 MiB budget, and the peak reconciles as ~90 + 4 x 41 = ~254 MB. Read
|
|
310
353
|
against the budget instead of the held text the same ratio would predict 90 + 4 x 64 = ~346 MB, which
|
|
311
|
-
the trace never saw. A 512 MiB container (roughly a 259 MB heap ceiling) now gets a 16 MiB pool
|
|
312
|
-
|
|
354
|
+
the trace never saw. A 512 MiB container (roughly a 259 MB heap ceiling) now gets a 16 MiB pool,
|
|
355
|
+
divided THREE ways for one read-only cache (5.33 MiB per scoped client) or nine ways when writes are
|
|
356
|
+
enabled, because read, write and delete each keep their own cache and each counts one eviction of
|
|
357
|
+
margin (1.78 MiB per client). The division is an exact byte quotient, so no pool space is lost to
|
|
358
|
+
rounding. The share is SIZED so the cache capacity plus one eviction of margin per cache fits the
|
|
359
|
+
pool; it is not an enforced peak. A scope evicted by the LRU stays resident while an in-flight
|
|
360
|
+
request that resolved it finishes, and because the SDK cannot revoke a live index, concurrent
|
|
361
|
+
evictions of distinct credentials can exceed the pool until those requests settle. A standalone walk of a synthetic 25,000-asset / 3,300-article tenant through
|
|
313
362
|
the real SDK client under a ~262 MB V8 ceiling (an uncommitted one-off probe; `--max-semi-space-size=1`
|
|
314
363
|
brings V8's reported limit to ~262 MB, while `--max-old-space-size=259` alone reports ~451 MB in
|
|
315
364
|
Node 24) peaked at ~136 MiB with the ROUND-10 tenth settings; the same walk with the OLD
|
|
316
365
|
64 MiB / 50,000 settings peaked at ~147 MiB. Both survived. That probe's baseline is lower than the
|
|
317
|
-
server's ~90 MB. A 1 GiB container's ~816 MB V8 ceiling — an ASSUMED ~80% container-to-heap ratio; the MEASURED ratio for the 512 MiB container is ~50.6% (259 MB) — gets a 51 MiB pool and
|
|
366
|
+
server's ~90 MB. A 1 GiB container's ~816 MB V8 ceiling — an ASSUMED ~80% container-to-heap ratio; the MEASURED ratio for the 512 MiB container is ~50.6% (259 MB) — gets a 51 MiB pool and 17 MiB per scoped client (5.67 MiB with writes enabled); a 1024 MB heap ceiling is
|
|
367
|
+
what reaches the 64 MiB cap.
|
|
318
368
|
|
|
319
369
|
The document cap exists because the text budget bounds index TEXT only: a document whose text was
|
|
320
370
|
evicted keeps its record and postings, the SDK's own default is 50,000 documents, and a walk can
|
|
321
371
|
reach 250 pages x 100 rows x 2 body resources. Pairing the caps bounds that overhead — a 512 MiB
|
|
322
|
-
container indexes up to
|
|
323
|
-
|
|
372
|
+
container indexes up to 5,461 documents per scoped client read-only, 1,820 with writes (a stdio
|
|
373
|
+
parent carries the whole pool instead: 16,384 documents; writes enabled builds one parent per mode,
|
|
374
|
+
so up to three of them carry it), a 1 GiB container's ~816 MB ceiling (assumed ~80%) 17,408 per
|
|
375
|
+
client read-only, 5,802 with writes (50,000 in a stdio parent), and only a 1024 MB ceiling 21,845
|
|
376
|
+
read-only, 7,281 with writes. The per-document cost is an ESTIMATE (~1 KB of record
|
|
324
377
|
plus postings per document, on top of the text ratio), so the pair bounds growth without proving a
|
|
325
378
|
peak. HONEST SCOPE: the SDK upserts every walked document and calls its cap only AFTER the walk, so
|
|
326
379
|
`maxDocs` bounds the RETAINED index, not the build-time peak; the peak during a walk is bounded by
|
|
327
380
|
the heap ceiling and the text budget's eviction, not by `maxDocs`. A capped index reports index state
|
|
328
|
-
`partial`, exactly as it already does when the text budget bites.
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
381
|
+
`partial`, exactly as it already does when the text budget bites.
|
|
382
|
+
|
|
383
|
+
ARTICLES ARE RESERVED THE WHOLE CAP. Without a reservation the SDK keeps the freshest `updated_at`
|
|
384
|
+
across ALL resources, so on an account with more assets than articles the assets take every slot and
|
|
385
|
+
the older articles have no index entry at all. `maxDocsByResource` (node-hudu 0.20.0) is a hard
|
|
386
|
+
reservation inside `maxDocs`: the server names `articles` with the whole cap, which makes it
|
|
387
|
+
articles-first — each article keeps its slot up to the cap, and assets fill whatever room is left,
|
|
388
|
+
in the SDK's `updated_at` order. That is the resource split the index exists for: article rows carry
|
|
389
|
+
the long text, asset rows do not.
|
|
390
|
+
A smaller reservation would also be a hard ceiling on articles, so an article-heavy account would
|
|
391
|
+
drop articles while its cap sat half empty (measured against the SDK: 300 articles and 5 assets at a
|
|
392
|
+
100-document cap retain 30 documents with a quarter reserved and 100 with the whole cap; an
|
|
393
|
+
asset-heavy corpus fills the cap either way).
|
|
394
|
+
|
|
395
|
+
THE ASSET SHARE IS DERIVED, AND CAN BE ZERO: assets keep `maxDocs - min(articles, maxDocs)` slots.
|
|
396
|
+
On the production corpus that is 18,641 of 21,845; where the article corpus meets or exceeds the cap
|
|
397
|
+
the assets get none — the 512 MiB write-enabled profile derives a 1,820-document cap against 3,204
|
|
398
|
+
articles, so `docs.assets.indexed` is 0 there and asset hits come from Hudu's own title/custom-field
|
|
399
|
+
search, which every call runs. `meta.index.docs.<resource>.capped` (node-hudu 0.20.0) names the
|
|
400
|
+
documents the caps dropped, per resource, so the loss is visible in the answer. The reservation
|
|
401
|
+
tracks the cap,
|
|
402
|
+
so it deepens with the heap like the rest of the budget: 17,408 article slots at a 1 GiB container's
|
|
403
|
+
assumed ~816 MB heap ceiling, 21,845 at a 1024 MB one, 50,000 in a stdio parent. This is what the
|
|
404
|
+
Unreleased CHANGELOG entry measures.
|
|
405
|
+
|
|
406
|
+
MEASURED on the production tenant (19,991 assets + 3,204 articles = 23,195 documents). At the former
|
|
407
|
+
division by eight (6,144 documents): 4,666 assets and 1,478 of the 3,204 articles retained, 1,036
|
|
408
|
+
article bodies indexed, 317 evicted by the text budget and 3 truncated (the counts do not decompose
|
|
409
|
+
the 1,478 — some retained articles carry no body text). Dividing the pool by two instead of eight
|
|
410
|
+
was expected (INFERRED at the time) to keep all 3,204 articles at 17 MiB / 17,408 documents; the
|
|
411
|
+
deployment measured that expectation FALSE — 15,588 assets and 1,820 articles — because the asset
|
|
412
|
+
count fills whatever cap the pool offers. At the 21,845-document ceiling the split was 19,991 assets
|
|
413
|
+
and 1,854 articles, i.e. 1,350 articles with no body index, while the asset rows (titles and custom
|
|
414
|
+
fields, no long text) held 91% of the cap. Reserving the cap for articles is the fix: the same
|
|
415
|
+
21,845-document cap then keeps all 3,204 articles and gives assets the remaining 18,641 slots (the
|
|
416
|
+
SDK's reservation arithmetic, reproduced with a synthetic corpus in `test/index-reservation.test.ts`);
|
|
417
|
+
`node-hudu` #122 has the upstream detail.
|
|
333
418
|
|
|
334
419
|
BEFORE THE DIVISOR CHANGE. The round-10 end-to-end run measured the real server (`dist/main.js`) under
|
|
335
420
|
a 259 MiB V8 ceiling: peak heapUsed 187.96 MB with a 24.6 MB baseline, so the INDEX STRUCTURES cost
|
|
@@ -430,7 +515,9 @@ Every client the server builds — the parent, the fail-closed placeholders and
|
|
|
430
515
|
blindly.
|
|
431
516
|
- `cache: 'recommended'` — fixed on: the
|
|
432
517
|
per-entity read-through TTL preset (1 h structural / 5 min mutable / 1 min base). Caches are
|
|
433
|
-
per client, so per scope
|
|
518
|
+
per client, so per scope — and a scope's client survives only while it stays in its mode cache,
|
|
519
|
+
which holds two pairs: a deployment serving more than two credentials therefore re-fetches more
|
|
520
|
+
from Hudu after an eviction, the same breadth-for-depth trade the body index makes. The write and delete clients have their own caches, so after every `hudu_write` /
|
|
434
521
|
`hudu_delete` attempt that is not a dry run (including a failed or timed-out one, which may have
|
|
435
522
|
applied) the server clears the read client's cache for the same scope: a record read (get, getSection, fetch_many, article context) after a write in
|
|
436
523
|
the same server process is fresh. A write made by another process or deployment can still be served
|
|
@@ -469,7 +556,7 @@ pnpm run dev
|
|
|
469
556
|
|
|
470
557
|
CI (`.github/workflows/ci.yml`) runs `typecheck`, `lint` and `test` on every push and pull request.
|
|
471
558
|
|
|
472
|
-
`node-hudu` is pinned to `0.
|
|
559
|
+
`node-hudu` is pinned to `0.20.0`. The catalog hash test detects metadata drift; upgrades require
|
|
473
560
|
review of the client authority, operation enums and redaction regressions. Docker uses `pnpm install --frozen-lockfile`.
|
|
474
561
|
|
|
475
562
|
## Deployment
|
package/dist/config.js
CHANGED
|
@@ -43,9 +43,100 @@ export const SECRET_READ_POLICIES = ['deny', 'allow'];
|
|
|
43
43
|
* `vendor` never builds an index at all. Unset means the transport default: `index` under `stdio`
|
|
44
44
|
* (one long-lived credential, one warmed index) and `auto` under `http` (every caller's scope
|
|
45
45
|
* starts cold, and a cold walk can outlast the consumer's call timeout).
|
|
46
|
+
*
|
|
47
|
+
* `typesense` answers the body search from Hudu's own Typesense index. It has no cold start, so it
|
|
48
|
+
* is the default under BOTH transports when `HUDU_TYPESENSE_URL` / `HUDU_TYPESENSE_API_KEY` are set
|
|
49
|
+
* — the transport asymmetry exists only for the in-memory walk. Naming it without that pair fails
|
|
50
|
+
* boot: a tier that cannot run must not be reachable by configuration typo.
|
|
51
|
+
*/
|
|
52
|
+
export const SEARCH_TIER_POLICIES = ['auto', 'index', 'vendor', 'typesense'];
|
|
53
|
+
/**
|
|
54
|
+
* The registry key of the SDK operation the Typesense tier is built on (node-hudu#124).
|
|
55
|
+
*
|
|
56
|
+
* Exported because two very different questions read it: config asks whether this BUILD has it (a
|
|
57
|
+
* startup refusal), and the probe asks whether the live client may call it (a report field).
|
|
58
|
+
*/
|
|
59
|
+
export const BODY_PROBE_OPERATION = 'operations.probeBodySearch';
|
|
60
|
+
/** The first node-hudu release that builds the probe operation. Named in the boot refusal. */
|
|
61
|
+
export const MIN_NODE_HUDU_FOR_TYPESENSE = '0.21.0';
|
|
62
|
+
/** A blank value is a copied `.env.example` placeholder, never a value: normalise it to unset. */
|
|
63
|
+
function blankToUndefined(raw) {
|
|
64
|
+
return typeof raw === 'string' && raw.trim() === '' ? undefined : raw;
|
|
65
|
+
}
|
|
66
|
+
/**
|
|
67
|
+
* The Typesense body-search tier's fixed defaults. Of the knobs the design names, only the
|
|
68
|
+
* collection names, the deadline and the verification gate are environment-tunable.
|
|
69
|
+
*
|
|
70
|
+
* Two knobs are deliberately NOT here and NOT set by this server: the candidate count and the
|
|
71
|
+
* circuit-breaker window. They are the SDK's own calibrated values (DESIGN §3 WP-1) — 40 candidates
|
|
72
|
+
* and a 30 s cooldown at node-hudu 0.21.0 — and this server passes neither key, so the library's
|
|
73
|
+
* calibration applies and CANNOT be shadowed by a stale copy here. The intent is not left untested:
|
|
74
|
+
* `test/typesense-config.test.ts` reads the RESOLVED SDK config and asserts those two values, so a
|
|
75
|
+
* recalibration in a future node-hudu release fails a test and shows up in review instead of
|
|
76
|
+
* silently changing behaviour.
|
|
77
|
+
*/
|
|
78
|
+
export const TYPESENSE_DEFAULTS = {
|
|
79
|
+
/** Hudu's own collection names, so an operator who provisions a search-only key sets nothing. */
|
|
80
|
+
articlesCollection: 'articles',
|
|
81
|
+
assetsCollection: 'assets',
|
|
82
|
+
/** Whole-call deadline for one Typesense search request. */
|
|
83
|
+
timeoutMs: 4000,
|
|
84
|
+
/** Refused above this: one search must never become the slowest thing in a tool call. */
|
|
85
|
+
maxTimeoutMs: 15000,
|
|
86
|
+
};
|
|
87
|
+
/**
|
|
88
|
+
* Whether `url` is an http(s) ORIGIN: no path, query or fragment.
|
|
89
|
+
*
|
|
90
|
+
* The boot check in `loadConfig`'s refinement applies these rules with a per-cause message; this
|
|
91
|
+
* predicate is the one the zero-wire doctor reports on, so the two cannot disagree about "shape ok".
|
|
92
|
+
*/
|
|
93
|
+
export function isTypesenseOrigin(url) {
|
|
94
|
+
try {
|
|
95
|
+
const parsed = new URL(url);
|
|
96
|
+
return ((parsed.protocol === 'http:' || parsed.protocol === 'https:') &&
|
|
97
|
+
(parsed.pathname === '' || parsed.pathname === '/') &&
|
|
98
|
+
parsed.search === '' &&
|
|
99
|
+
parsed.hash === '');
|
|
100
|
+
}
|
|
101
|
+
catch {
|
|
102
|
+
return false;
|
|
103
|
+
}
|
|
104
|
+
}
|
|
105
|
+
/** Build the SDK block from an already-validated config. Never carries a value into a log sink. */
|
|
106
|
+
export function typesenseSearchBlock(config) {
|
|
107
|
+
const url = config.HUDU_TYPESENSE_URL;
|
|
108
|
+
const apiKey = config.HUDU_TYPESENSE_API_KEY;
|
|
109
|
+
if (url === undefined || apiKey === undefined)
|
|
110
|
+
return undefined;
|
|
111
|
+
return {
|
|
112
|
+
url,
|
|
113
|
+
apiKey,
|
|
114
|
+
collections: {
|
|
115
|
+
articles: config.HUDU_TYPESENSE_ARTICLES_COLLECTION ?? TYPESENSE_DEFAULTS.articlesCollection,
|
|
116
|
+
assets: config.HUDU_TYPESENSE_ASSETS_COLLECTION ?? TYPESENSE_DEFAULTS.assetsCollection,
|
|
117
|
+
},
|
|
118
|
+
timeoutMs: config.HUDU_TYPESENSE_TIMEOUT_MS ?? TYPESENSE_DEFAULTS.timeoutMs,
|
|
119
|
+
// No `candidateLimit` and no `cooldownMs`: both are the SDK's calibration, and omission is what
|
|
120
|
+
// keeps this server from pinning a copy that silently overrides a future recalibration.
|
|
121
|
+
verifyHits: config.HUDU_TYPESENSE_VERIFY_HITS ?? true,
|
|
122
|
+
};
|
|
123
|
+
}
|
|
124
|
+
/**
|
|
125
|
+
* The one boot-time line describing the tier — `undefined` when it is off.
|
|
126
|
+
*
|
|
127
|
+
* Shape only: never the URL or the key, only the collection names the operator can already read in
|
|
128
|
+
* the environment. A disabled feature is INVISIBLE in boot output, because the byte-identical
|
|
129
|
+
* promise for unset deployments covers the log as well as the tool surface. Exported so a test can
|
|
130
|
+
* assert the line without booting a server.
|
|
46
131
|
*/
|
|
47
|
-
export
|
|
48
|
-
|
|
132
|
+
export function describeTypesense(config) {
|
|
133
|
+
const block = typesenseSearchBlock(config);
|
|
134
|
+
if (block === undefined)
|
|
135
|
+
return undefined;
|
|
136
|
+
return `typesense: enabled (collections ${block.collections.articles}, ${block.collections.assets})`;
|
|
137
|
+
}
|
|
138
|
+
/**
|
|
139
|
+
* Parse `HUDU_WRITE_ALLOW` once, here, so every consumer reads the same list. */
|
|
49
140
|
function parseAllowList(raw) {
|
|
50
141
|
return (raw ?? '')
|
|
51
142
|
.split(',')
|
|
@@ -171,6 +262,55 @@ const schema = z
|
|
|
171
262
|
* background; a shorter value trades vendor-tier reads for fresher bodies.
|
|
172
263
|
*/
|
|
173
264
|
HUDU_SEARCH_INDEX_TTL: z.coerce.number().int().positive().optional(),
|
|
265
|
+
/**
|
|
266
|
+
* The Typesense body-search endpoint: the origin only, e.g.
|
|
267
|
+
* `https://typesense.hudu.intellectit.com.au`, or `http://127.0.0.1:8108` through an SSH tunnel
|
|
268
|
+
* (DESIGN §7.3). No path, query or fragment — a Typesense base URL is an origin and a path there
|
|
269
|
+
* would silently point every search at nothing.
|
|
270
|
+
*
|
|
271
|
+
* Set together with {@link HUDU_TYPESENSE_API_KEY} or not at all: `loadConfig` refuses a
|
|
272
|
+
* half-configured pair, because a URL with no key is an unreachable tier and a key with no URL
|
|
273
|
+
* has nowhere to go. Both unset means the tier is off and the server behaves exactly as before.
|
|
274
|
+
*/
|
|
275
|
+
HUDU_TYPESENSE_URL: z.preprocess(blankToUndefined, z.string().url('HUDU_TYPESENSE_URL must be an absolute URL, e.g. https://typesense.example.com').optional()),
|
|
276
|
+
/**
|
|
277
|
+
* The Typesense search-only key (`documents:search` on the two body collections). Never logged,
|
|
278
|
+
* never echoed in an error, never part of `doctor` output — a boolean and a probe result code
|
|
279
|
+
* are the only facts that leave the process.
|
|
280
|
+
*/
|
|
281
|
+
HUDU_TYPESENSE_API_KEY: z.preprocess(blankToUndefined, z.string().min(1, 'HUDU_TYPESENSE_API_KEY must not be empty').optional()),
|
|
282
|
+
/** The articles collection name. Defaults to Hudu's own name; a rename is config, not code. */
|
|
283
|
+
HUDU_TYPESENSE_ARTICLES_COLLECTION: z.preprocess(blankToUndefined, z.string().min(1).optional()),
|
|
284
|
+
/** The assets collection name. Defaults to Hudu's own name. */
|
|
285
|
+
HUDU_TYPESENSE_ASSETS_COLLECTION: z.preprocess(blankToUndefined, z.string().min(1).optional()),
|
|
286
|
+
/**
|
|
287
|
+
* Whole-call deadline for one Typesense search, in milliseconds (default 4000).
|
|
288
|
+
*
|
|
289
|
+
* Bounded rather than free: a search has a vendor fallback, so waiting longer than a few seconds
|
|
290
|
+
* is always the wrong trade. Above {@link TYPESENSE_DEFAULTS.maxTimeoutMs} the container fails
|
|
291
|
+
* boot instead of accepting a value the tier was never designed for.
|
|
292
|
+
*/
|
|
293
|
+
HUDU_TYPESENSE_TIMEOUT_MS: z.preprocess(blankToUndefined, z.coerce
|
|
294
|
+
.number()
|
|
295
|
+
.int()
|
|
296
|
+
.positive()
|
|
297
|
+
.max(TYPESENSE_DEFAULTS.maxTimeoutMs, `HUDU_TYPESENSE_TIMEOUT_MS must be at most ${TYPESENSE_DEFAULTS.maxTimeoutMs}; the tier has a vendor fallback and a search must not outlast the call`)
|
|
298
|
+
.optional()),
|
|
299
|
+
/**
|
|
300
|
+
* Whether every Typesense hit is re-read through the Hudu API under the caller's own credential
|
|
301
|
+
* before it is returned (default `true`).
|
|
302
|
+
*
|
|
303
|
+
* Hudu's index does not carry folder permissions and keeps discarded articles, so a raw hit is
|
|
304
|
+
* an unverified claim about a record this credential may not be allowed to see. `false` is
|
|
305
|
+
* faster and is documented as acceptable only for single-tenant, non-restricted deployments.
|
|
306
|
+
*/
|
|
307
|
+
HUDU_TYPESENSE_VERIFY_HITS: z.preprocess(
|
|
308
|
+
// Same treatment as every other new knob: a blank placeholder is unset, and the value is
|
|
309
|
+
// matched case-insensitively so `TRUE` (a shell convention) is not a boot failure.
|
|
310
|
+
(raw) => {
|
|
311
|
+
const value = blankToUndefined(raw);
|
|
312
|
+
return typeof value === 'string' ? value.trim().toLowerCase() : value;
|
|
313
|
+
}, z.enum(['true', 'false']).optional().transform((value) => (value === undefined ? undefined : value === 'true'))),
|
|
174
314
|
/**
|
|
175
315
|
* The single switch for mutation. `deny` (the default) registers no `hudu_write` / `hudu_delete`
|
|
176
316
|
* tool at all; `allow_list` and `all` register them and gate every call. Read effects are never
|
|
@@ -284,6 +424,89 @@ const schema = z
|
|
|
284
424
|
}
|
|
285
425
|
}
|
|
286
426
|
}
|
|
427
|
+
// The Typesense pair is all-or-nothing. A URL with no key searches nowhere; a key with no URL
|
|
428
|
+
// has nowhere to go, and a key typed into the wrong deployment is a secret in the wrong place.
|
|
429
|
+
// Name BOTH variables in one message so the operator fixes the pair, not one guess at a time.
|
|
430
|
+
const typesenseUrl = c.HUDU_TYPESENSE_URL;
|
|
431
|
+
const typesenseKey = c.HUDU_TYPESENSE_API_KEY;
|
|
432
|
+
if ((typesenseUrl === undefined) !== (typesenseKey === undefined)) {
|
|
433
|
+
ctx.addIssue({
|
|
434
|
+
code: 'custom',
|
|
435
|
+
path: [typesenseUrl === undefined ? 'HUDU_TYPESENSE_URL' : 'HUDU_TYPESENSE_API_KEY'],
|
|
436
|
+
message: 'HUDU_TYPESENSE_URL and HUDU_TYPESENSE_API_KEY must be set together or not at all: ' +
|
|
437
|
+
(typesenseUrl === undefined
|
|
438
|
+
? 'a key was set with no URL for it to reach (set HUDU_TYPESENSE_URL, or unset the key)'
|
|
439
|
+
: 'a URL was set with no search key (set HUDU_TYPESENSE_API_KEY, or unset the URL to leave the Typesense tier off)'),
|
|
440
|
+
});
|
|
441
|
+
}
|
|
442
|
+
// The capability gate belongs HERE, with the rest of the boot refusals: a configured tier on an
|
|
443
|
+
// SDK build that cannot probe it would otherwise serve a default tier the SDK ignores while the
|
|
444
|
+
// prose claims Hudu's index answered — two contradictory stories in one process. Configuration
|
|
445
|
+
// that cannot run is a startup error, never a silent fallback.
|
|
446
|
+
if (typesenseUrl !== undefined && c.HUDU_TYPESENSE_API_KEY !== undefined && getCapability(BODY_PROBE_OPERATION) === undefined) {
|
|
447
|
+
ctx.addIssue({
|
|
448
|
+
code: 'custom',
|
|
449
|
+
path: ['HUDU_TYPESENSE_URL'],
|
|
450
|
+
message: `HUDU_TYPESENSE_URL/HUDU_TYPESENSE_API_KEY are set, but this build of node-hudu has no ${BODY_PROBE_OPERATION}: the Typesense tier needs node-hudu >= ${MIN_NODE_HUDU_FOR_TYPESENSE}. Unset both variables, or upgrade node-hudu.`,
|
|
451
|
+
});
|
|
452
|
+
}
|
|
453
|
+
// An origin, not a URL. A scheme other than http/https is not a Typesense endpoint, and any
|
|
454
|
+
// path, query or fragment would be dropped or misapplied by a client that joins paths onto it —
|
|
455
|
+
// a silent wrong endpoint rather than a refused boot.
|
|
456
|
+
if (typesenseUrl !== undefined) {
|
|
457
|
+
let parsed;
|
|
458
|
+
try {
|
|
459
|
+
parsed = new URL(typesenseUrl);
|
|
460
|
+
}
|
|
461
|
+
catch {
|
|
462
|
+
parsed = undefined;
|
|
463
|
+
}
|
|
464
|
+
if (parsed === undefined || (parsed.protocol !== 'http:' && parsed.protocol !== 'https:')) {
|
|
465
|
+
ctx.addIssue({
|
|
466
|
+
code: 'custom',
|
|
467
|
+
path: ['HUDU_TYPESENSE_URL'],
|
|
468
|
+
message: 'HUDU_TYPESENSE_URL must be an http:// or https:// origin, e.g. https://typesense.example.com',
|
|
469
|
+
});
|
|
470
|
+
}
|
|
471
|
+
else if (parsed.username !== '' || parsed.password !== '') {
|
|
472
|
+
// Credentials in the URL would be sent to the vendor on every probe and would land in a log
|
|
473
|
+
// the moment anything printed the endpoint. Refused, and the message never repeats the value
|
|
474
|
+
// (it is a credential, and this is the one place it could leak by accident).
|
|
475
|
+
ctx.addIssue({
|
|
476
|
+
code: 'custom',
|
|
477
|
+
path: ['HUDU_TYPESENSE_URL'],
|
|
478
|
+
message: 'HUDU_TYPESENSE_URL must not carry credentials: put the key in HUDU_TYPESENSE_API_KEY, and never in the URL',
|
|
479
|
+
});
|
|
480
|
+
}
|
|
481
|
+
else if ((parsed.pathname !== '' && parsed.pathname !== '/') || parsed.search !== '' || parsed.hash !== '') {
|
|
482
|
+
ctx.addIssue({
|
|
483
|
+
code: 'custom',
|
|
484
|
+
path: ['HUDU_TYPESENSE_URL'],
|
|
485
|
+
message: 'HUDU_TYPESENSE_URL must be an origin only: no path, query or fragment (e.g. https://typesense.example.com)',
|
|
486
|
+
});
|
|
487
|
+
}
|
|
488
|
+
}
|
|
489
|
+
// Two resources in one collection would make every hit ambiguous: an article id and an asset id
|
|
490
|
+
// are both bare numbers, and the caller's `resources` scope could not be honoured.
|
|
491
|
+
const articlesCollection = c.HUDU_TYPESENSE_ARTICLES_COLLECTION ?? TYPESENSE_DEFAULTS.articlesCollection;
|
|
492
|
+
const assetsCollection = c.HUDU_TYPESENSE_ASSETS_COLLECTION ?? TYPESENSE_DEFAULTS.assetsCollection;
|
|
493
|
+
if (articlesCollection === assetsCollection) {
|
|
494
|
+
ctx.addIssue({
|
|
495
|
+
code: 'custom',
|
|
496
|
+
path: [c.HUDU_TYPESENSE_ASSETS_COLLECTION === undefined ? 'HUDU_TYPESENSE_ARTICLES_COLLECTION' : 'HUDU_TYPESENSE_ASSETS_COLLECTION'],
|
|
497
|
+
message: `articles and assets must search different collections (both resolve to "${articlesCollection}"); set HUDU_TYPESENSE_ARTICLES_COLLECTION or HUDU_TYPESENSE_ASSETS_COLLECTION`,
|
|
498
|
+
});
|
|
499
|
+
}
|
|
500
|
+
// A tier that cannot run must not be reachable by configuration: naming `typesense` without the
|
|
501
|
+
// endpoint and key would otherwise degrade every search at the first call instead of failing
|
|
502
|
+
// the container once, at boot.
|
|
503
|
+
if (c.HUDU_SEARCH_TIER_POLICY === 'typesense' && typesenseUrl === undefined) {
|
|
504
|
+
ctx.addIssue({
|
|
505
|
+
code: 'custom',
|
|
506
|
+
path: ['HUDU_SEARCH_TIER_POLICY'],
|
|
507
|
+
message: 'HUDU_SEARCH_TIER_POLICY=typesense requires HUDU_TYPESENSE_URL and HUDU_TYPESENSE_API_KEY; set both, or choose auto|index|vendor',
|
|
508
|
+
});
|
|
509
|
+
}
|
|
287
510
|
// `HUDU_WRITE_ALLOW` is only meaningful under `allow_list`, where a bad list must fail the
|
|
288
511
|
// container rather than a call six months later. Every problem is reported at once, on the one
|
|
289
512
|
// error path `loadConfig` already owns.
|
|
@@ -325,6 +548,14 @@ const schema = z
|
|
|
325
548
|
}
|
|
326
549
|
}
|
|
327
550
|
});
|
|
551
|
+
/**
|
|
552
|
+
* Every `HUDU_TYPESENSE_*` knob the schema accepts, derived from the schema rather than retyped.
|
|
553
|
+
*
|
|
554
|
+
* The documentation-drift test reads THIS list: a knob added to the schema and forgotten in
|
|
555
|
+
* `.env.example` / the README then fails the test, which a hand-written list in the test could not
|
|
556
|
+
* catch — the test would have been asserting its own copy.
|
|
557
|
+
*/
|
|
558
|
+
export const TYPESENSE_ENV_NAMES = Object.keys(schema.shape).filter((name) => name.startsWith('HUDU_TYPESENSE_'));
|
|
328
559
|
/**
|
|
329
560
|
* Apply the same check to `HUDU_API_KEY` here as to the per-request key under `http`.
|
|
330
561
|
*
|