@hiai-gg/docsmint 0.7.5 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +57 -10
- package/dist/backend/index.js +477 -106
- package/dist/client.d.ts +2 -2
- package/dist/client.js +6 -6
- package/dist/lifecycle-runtime.js +40 -4
- package/dist/mcp-cli.js +6 -6
- package/dist/mcp-server.js +6 -6
- package/package.json +1 -1
- package/packages/cli/src/index.ts +1 -1
- package/server.json +2 -2
- package/skills/docsmint-document-manager/SKILL.md +2 -2
package/README.md
CHANGED
|
@@ -6,9 +6,9 @@ applications, and AI agents.**
|
|
|
6
6
|
DocsMint stores documents in a structured JSON editor model first. Markdown is
|
|
7
7
|
the convenient second format for editing, importing, and exporting content.
|
|
8
8
|
Automatic chunking, 1024-dimensional embeddings, multilingual hybrid search,
|
|
9
|
-
and
|
|
10
|
-
agents through the web application, REST API,
|
|
11
|
-
server.
|
|
9
|
+
GraphRAG, and cross-encoder rerank make the same knowledge base useful to
|
|
10
|
+
people, applications, and agents through the web application, REST API,
|
|
11
|
+
TypeScript SDK, CLI, and MCP server.
|
|
12
12
|
|
|
13
13
|
[](LICENSE)
|
|
14
14
|
[](https://github.com/hiai-gg/docsmint/releases)
|
|
@@ -36,7 +36,8 @@ server.
|
|
|
36
36
|
|
|
37
37
|
- **Write naturally** in a rich visual editor or raw Markdown.
|
|
38
38
|
- **Find meaning, not only keywords** with exact, lexical, fuzzy, vector,
|
|
39
|
-
multilingual expansion, and graph retrieval fused through RRF
|
|
39
|
+
multilingual expansion, and graph retrieval fused through RRF, then
|
|
40
|
+
reranked with a cross-encoder (Voyage rerank-2.5 in the reference profile).
|
|
40
41
|
- **Keep retrieval current** with automatic, incremental chunking and
|
|
41
42
|
re-embedding after document or metadata changes.
|
|
42
43
|
- **Connect agents directly** through REST, a typed SDK, CLI, or MCP.
|
|
@@ -45,8 +46,50 @@ server.
|
|
|
45
46
|
- **Own the full stack**: application data, vectors, graph, queue, and files run
|
|
46
47
|
on infrastructure you control.
|
|
47
48
|
|
|
49
|
+
## What's new in 0.8.1?
|
|
50
|
+
|
|
51
|
+
- **GraphRAG expansion is connection-safe.** AGE reads pin `ag_catalog` on the
|
|
52
|
+
same pooled connection as `cypher()`, so scoped search keeps graph neighbors
|
|
53
|
+
even when the connection did not already have AGE first in `search_path`.
|
|
54
|
+
|
|
55
|
+
## What's new in 0.7.9?
|
|
56
|
+
|
|
57
|
+
- **GraphRAG on category/share search keeps inherited-folder neighbors.** The
|
|
58
|
+
allowed document list is the ACL; category is not applied a second time.
|
|
59
|
+
|
|
60
|
+
## What's new in 0.7.8?
|
|
61
|
+
|
|
62
|
+
- **GraphRAG still contributes after rerank.** Neighbors already in the fused
|
|
63
|
+
prefix keep the graph channel, so scoped search does not lose graph evidence.
|
|
64
|
+
- **Public API identity is DocsMint.** Health reports `service: "docsmint"`;
|
|
65
|
+
startup and SDK errors no longer say `hiai-docs`.
|
|
66
|
+
|
|
67
|
+
## What's new in 0.7.7?
|
|
68
|
+
|
|
69
|
+
- **Cross-encoder rerank is on by default.** After RRF, search reranks the
|
|
70
|
+
original query against the fused prefix with Voyage `rerank-2.5` (Cohere
|
|
71
|
+
and NVIDIA fallbacks). Provider failures keep RRF order. Set
|
|
72
|
+
`SEARCH_RERANK_ENABLED=false` to turn it off.
|
|
73
|
+
- **Live retrieval improved first-hit ranking.** On a 24-document labeled
|
|
74
|
+
corpus, RRF+rerank moved MRR 0.969 → 1.000 and nDCG@10 0.958 → 0.986.
|
|
75
|
+
Recall@10 stayed 1.0. The paraphrase query “operator API key header”
|
|
76
|
+
moved the API-key document from rank 2 to rank 1 (MRR 0.50 → 1.00).
|
|
77
|
+
- **Three-model fallbacks** for embeddings, extraction, expansion, and
|
|
78
|
+
rerank so a single provider outage does not drop the channel.
|
|
79
|
+
|
|
80
|
+
## What's new in 0.7.6?
|
|
81
|
+
|
|
82
|
+
- **Better Auth 1.6 rollback remains possible** after the issuer migration by
|
|
83
|
+
deriving the legacy account issuer in a database trigger.
|
|
84
|
+
|
|
48
85
|
## What's new in 0.7.5?
|
|
49
86
|
|
|
87
|
+
- **Runtime and dependency graph** to Bun 1.4, TypeScript 6, Better Auth 1.7,
|
|
88
|
+
and current compatible editor/queue packages, with fail-closed issuer
|
|
89
|
+
identity.
|
|
90
|
+
|
|
91
|
+
## What's new in 0.7.4?
|
|
92
|
+
|
|
50
93
|
- **MCP Registry description fits the 100-character limit**, so official
|
|
51
94
|
catalog publication can complete after 0.7.3 reached npm.
|
|
52
95
|
|
|
@@ -137,7 +180,7 @@ docker pull vgalibov/docsmint:web-latest
|
|
|
137
180
|
docker pull vgalibov/docsmint:caddy-latest
|
|
138
181
|
```
|
|
139
182
|
|
|
140
|
-
Use versioned tags such as `api-v0.
|
|
183
|
+
Use versioned tags such as `api-v0.8.1` for reproducible deploys. The
|
|
141
184
|
quickstart still builds the Compose stack from this repository so PostgreSQL,
|
|
142
185
|
Redis, and SeaweedFS start together with the application.
|
|
143
186
|
|
|
@@ -279,19 +322,19 @@ resources, skill, and API routes.
|
|
|
279
322
|
### Prompts (2)
|
|
280
323
|
|
|
281
324
|
- `organize_workspace`: Plan safe document organization using DocsMint categories and folders.
|
|
282
|
-
- `research_workspace`: Research a question with hybrid search and
|
|
325
|
+
- `research_workspace`: Research a question with hybrid search, GraphRAG, and rerank citing document IDs.
|
|
283
326
|
|
|
284
327
|
### Resources (3)
|
|
285
328
|
|
|
286
329
|
- `docsmint://guide/editor`: Editor usage guide.
|
|
287
|
-
- `docsmint://guide/search`: Search and
|
|
330
|
+
- `docsmint://guide/search`: Search, GraphRAG, and rerank guide.
|
|
288
331
|
- `docsmint://workspace/catalog`: Live scoped workspace catalog.
|
|
289
332
|
|
|
290
333
|
### Skills (1)
|
|
291
334
|
|
|
292
335
|
- [`docsmint-document-manager`](skills/docsmint-document-manager/SKILL.md):
|
|
293
336
|
create, organize, edit, and research DocsMint documents through the 17 MCP
|
|
294
|
-
tools, including hybrid search, GraphRAG, and index refresh.
|
|
337
|
+
tools, including hybrid search, GraphRAG, rerank, and index refresh.
|
|
295
338
|
|
|
296
339
|
The same Skill ships in the npm package under `skills/docsmint-document-manager/SKILL.md`.
|
|
297
340
|
|
|
@@ -382,8 +425,12 @@ previous valid generation remains searchable if a provider call fails.
|
|
|
382
425
|
Search combines exact title matches, multilingual lexical search, typo-tolerant
|
|
383
426
|
fuzzy matching, semantic vectors, adaptive query expansion, and Apache AGE
|
|
384
427
|
graph neighbors. Reciprocal rank fusion combines the channels without allowing
|
|
385
|
-
one weak provider result to dominate.
|
|
386
|
-
|
|
428
|
+
one weak provider result to dominate. A cross-encoder then reranks the fused
|
|
429
|
+
prefix against the original query (Voyage rerank-2.5 by default). On a labeled
|
|
430
|
+
24-document corpus that moved MRR 0.969 → 1.000 and nDCG@10 0.958 → 0.986
|
|
431
|
+
versus RRF-only. Rerank, expansion, embeddings, and AGE failures keep the
|
|
432
|
+
remaining channels. Authorization is applied before retrieval and again before
|
|
433
|
+
results are returned.
|
|
387
434
|
|
|
388
435
|
GraphRAG is part of the normal search path in the reference configuration. It
|
|
389
436
|
extracts entities after embeddings are ready and finds related documents beyond
|
package/dist/backend/index.js
CHANGED
|
@@ -79815,10 +79815,26 @@ var init_kysely_adapter = __esm(() => {
|
|
|
79815
79815
|
init_dist3();
|
|
79816
79816
|
});
|
|
79817
79817
|
|
|
79818
|
+
// ../../backend/src/lib/openrouter-public-matrix.ts
|
|
79819
|
+
var OPENROUTER_PUBLIC_BASE_URL = "https://openrouter.ai/api/v1", OPENROUTER_PUBLIC_EXPANSION_MODELS, OPENROUTER_PUBLIC_RERANK_MODELS;
|
|
79820
|
+
var init_openrouter_public_matrix = __esm(() => {
|
|
79821
|
+
OPENROUTER_PUBLIC_EXPANSION_MODELS = {
|
|
79822
|
+
primary: "poolside/laguna-xs-2.1",
|
|
79823
|
+
fallback: "openai/gpt-5.6-luna",
|
|
79824
|
+
fallback_2: "google/gemini-3.5-flash-lite"
|
|
79825
|
+
};
|
|
79826
|
+
OPENROUTER_PUBLIC_RERANK_MODELS = {
|
|
79827
|
+
primary: "voyageai/rerank-2.5",
|
|
79828
|
+
fallback: "cohere/rerank-v3.5",
|
|
79829
|
+
fallback_2: "nvidia/llama-nemotron-rerank-vl-1b-v2:free"
|
|
79830
|
+
};
|
|
79831
|
+
});
|
|
79832
|
+
|
|
79818
79833
|
// ../../backend/src/lib/config-schema.ts
|
|
79819
79834
|
var adminApiKeyPlaceholder, envSchema;
|
|
79820
79835
|
var init_config_schema = __esm(() => {
|
|
79821
79836
|
init_zod();
|
|
79837
|
+
init_openrouter_public_matrix();
|
|
79822
79838
|
adminApiKeyPlaceholder = /^(?:change[-_ ]?me|changeme|your[-_ ]?api[-_ ]?key)(?:\b|[-_ ])/i;
|
|
79823
79839
|
envSchema = exports_external2.object({
|
|
79824
79840
|
DATABASE_URL: exports_external2.string().default("postgresql://hiai_app:changeme@localhost:5437/hiai_docs"),
|
|
@@ -79875,6 +79891,9 @@ var init_config_schema = __esm(() => {
|
|
|
79875
79891
|
EMBEDDING_FALLBACK_BASE_URL: exports_external2.string().optional(),
|
|
79876
79892
|
EMBEDDING_FALLBACK_API_KEY: exports_external2.string().optional(),
|
|
79877
79893
|
EMBEDDING_FALLBACK_MODEL: exports_external2.string().optional(),
|
|
79894
|
+
EMBEDDING_FALLBACK_2_BASE_URL: exports_external2.string().optional(),
|
|
79895
|
+
EMBEDDING_FALLBACK_2_API_KEY: exports_external2.string().optional(),
|
|
79896
|
+
EMBEDDING_FALLBACK_2_MODEL: exports_external2.string().optional(),
|
|
79878
79897
|
API_PORT: exports_external2.coerce.number().default(50700),
|
|
79879
79898
|
NODE_ENV: exports_external2.enum(["development", "production", "test"]).default("development"),
|
|
79880
79899
|
LOG_LEVEL: exports_external2.enum(["trace", "debug", "info", "warn", "error", "fatal"]).default("info"),
|
|
@@ -79901,14 +79920,34 @@ var init_config_schema = __esm(() => {
|
|
|
79901
79920
|
GRAPH_EXTRACT_FALLBACK_BASE_URL: exports_external2.string().optional(),
|
|
79902
79921
|
GRAPH_EXTRACT_FALLBACK_API_KEY: exports_external2.string().optional(),
|
|
79903
79922
|
GRAPH_EXTRACT_FALLBACK_MODEL: exports_external2.string().optional(),
|
|
79923
|
+
GRAPH_EXTRACT_FALLBACK_2_BASE_URL: exports_external2.string().optional(),
|
|
79924
|
+
GRAPH_EXTRACT_FALLBACK_2_API_KEY: exports_external2.string().optional(),
|
|
79925
|
+
GRAPH_EXTRACT_FALLBACK_2_MODEL: exports_external2.string().optional(),
|
|
79904
79926
|
GRAPH_EXTRACT_MIN_CONFIDENCE: exports_external2.coerce.number().min(0).max(1).default(0.5),
|
|
79905
79927
|
SEARCH_EXPANSION_ENABLED: exports_external2.string().optional().default("true").transform((v2) => v2 === "true"),
|
|
79906
|
-
SEARCH_EXPANSION_BASE_URL: exports_external2.string().default(
|
|
79928
|
+
SEARCH_EXPANSION_BASE_URL: exports_external2.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
79907
79929
|
SEARCH_EXPANSION_API_KEY: exports_external2.string().optional(),
|
|
79908
|
-
SEARCH_EXPANSION_MODEL: exports_external2.string().default(
|
|
79909
|
-
SEARCH_EXPANSION_FALLBACK_BASE_URL: exports_external2.string().default(
|
|
79930
|
+
SEARCH_EXPANSION_MODEL: exports_external2.string().default(OPENROUTER_PUBLIC_EXPANSION_MODELS.primary),
|
|
79931
|
+
SEARCH_EXPANSION_FALLBACK_BASE_URL: exports_external2.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
79910
79932
|
SEARCH_EXPANSION_FALLBACK_API_KEY: exports_external2.string().optional(),
|
|
79911
|
-
SEARCH_EXPANSION_FALLBACK_MODEL: exports_external2.string().default(
|
|
79933
|
+
SEARCH_EXPANSION_FALLBACK_MODEL: exports_external2.string().default(OPENROUTER_PUBLIC_EXPANSION_MODELS.fallback),
|
|
79934
|
+
SEARCH_EXPANSION_FALLBACK_2_BASE_URL: exports_external2.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
79935
|
+
SEARCH_EXPANSION_FALLBACK_2_API_KEY: exports_external2.string().optional(),
|
|
79936
|
+
SEARCH_EXPANSION_FALLBACK_2_MODEL: exports_external2.string().default(OPENROUTER_PUBLIC_EXPANSION_MODELS.fallback_2),
|
|
79937
|
+
SEARCH_RERANK_ENABLED: exports_external2.string().optional().default("true").transform((v2) => v2 === "true"),
|
|
79938
|
+
SEARCH_RERANK_BASE_URL: exports_external2.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
79939
|
+
SEARCH_RERANK_API_KEY: exports_external2.string().optional(),
|
|
79940
|
+
SEARCH_RERANK_MODEL: exports_external2.string().default(OPENROUTER_PUBLIC_RERANK_MODELS.primary),
|
|
79941
|
+
SEARCH_RERANK_FALLBACK_BASE_URL: exports_external2.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
79942
|
+
SEARCH_RERANK_FALLBACK_API_KEY: exports_external2.string().optional(),
|
|
79943
|
+
SEARCH_RERANK_FALLBACK_MODEL: exports_external2.string().default(OPENROUTER_PUBLIC_RERANK_MODELS.fallback),
|
|
79944
|
+
SEARCH_RERANK_FALLBACK_2_BASE_URL: exports_external2.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
79945
|
+
SEARCH_RERANK_FALLBACK_2_API_KEY: exports_external2.string().optional(),
|
|
79946
|
+
SEARCH_RERANK_FALLBACK_2_MODEL: exports_external2.string().default(OPENROUTER_PUBLIC_RERANK_MODELS.fallback_2),
|
|
79947
|
+
SEARCH_RERANK_TIMEOUT_MS: exports_external2.coerce.number().int().min(250).max(30000).default(3000),
|
|
79948
|
+
SEARCH_RERANK_TOP_N: exports_external2.coerce.number().int().min(1).max(100).default(20),
|
|
79949
|
+
SEARCH_RERANK_MAX_CHARS: exports_external2.coerce.number().int().min(120).max(8000).default(1500),
|
|
79950
|
+
SEARCH_RERANK_GRAPH_POSITION: exports_external2.enum(["before", "after"]).default("after"),
|
|
79912
79951
|
SEARCH_EXPANSION_TIMEOUT_MS: exports_external2.coerce.number().int().min(1000).max(300000).default(6000),
|
|
79913
79952
|
SEARCH_VECTOR_PROVIDER_TIMEOUT_MS: exports_external2.coerce.number().int().min(250).max(30000).default(2500),
|
|
79914
79953
|
SEARCH_EXPANSION_CACHE_TTL_SECONDS: exports_external2.coerce.number().int().min(0).max(2592000).default(86400),
|
|
@@ -195670,6 +195709,10 @@ var METRIC_NAMES = {
|
|
|
195670
195709
|
SEARCH_EXPANSION_LOW_VECTOR_SIMILARITY_TOTAL: "search_expansion_reason_low_vector_similarity_total",
|
|
195671
195710
|
SEARCH_EXPANSION_LANGUAGE_MISMATCH_TOTAL: "search_expansion_reason_language_mismatch_total",
|
|
195672
195711
|
SEARCH_EXPANSION_EMPTY_CANDIDATES_TOTAL: "search_expansion_reason_empty_candidates_total",
|
|
195712
|
+
SEARCH_RERANK_DURATION_MS: "search_rerank_duration_ms",
|
|
195713
|
+
SEARCH_RERANK_SUCCESS_TOTAL: "search_rerank_success_total",
|
|
195714
|
+
SEARCH_RERANK_FALLBACK_TOTAL: "search_rerank_fallback_total",
|
|
195715
|
+
SEARCH_RERANK_CANDIDATES_TOTAL: "search_rerank_candidates_total",
|
|
195673
195716
|
SEARCH_EXACT_DURATION_MS: "search_exact_duration_ms",
|
|
195674
195717
|
SEARCH_EXACT_ERRORS_TOTAL: "search_exact_errors_total",
|
|
195675
195718
|
SEARCH_EXACT_CANDIDATES_TOTAL: "search_exact_candidates_total",
|
|
@@ -195841,7 +195884,7 @@ function recordSearchExpansionMetrics(input) {
|
|
|
195841
195884
|
break;
|
|
195842
195885
|
}
|
|
195843
195886
|
}
|
|
195844
|
-
if (input.used !== false && input.model && input.model === input.fallbackModel) {
|
|
195887
|
+
if (input.used !== false && input.model && (input.model === input.fallbackModel || input.model === input.fallback2Model)) {
|
|
195845
195888
|
incrementCounter(METRIC_NAMES.SEARCH_EXPANSION_FALLBACK_TOTAL);
|
|
195846
195889
|
} else if (input.used !== false && input.model) {
|
|
195847
195890
|
incrementCounter(METRIC_NAMES.SEARCH_EXPANSION_PRIMARY_TOTAL);
|
|
@@ -196176,31 +196219,52 @@ async function getEmbedding(text3) {
|
|
|
196176
196219
|
recordDuration(METRIC_NAMES.EMBEDDING_DURATION_MS, Date.now() - start);
|
|
196177
196220
|
}
|
|
196178
196221
|
}
|
|
196179
|
-
|
|
196180
|
-
const
|
|
196181
|
-
|
|
196182
|
-
providers.push({
|
|
196222
|
+
function configuredEmbeddingProviders() {
|
|
196223
|
+
const slots = [
|
|
196224
|
+
{
|
|
196183
196225
|
baseUrl: config3.EMBEDDING_BASE_URL,
|
|
196184
196226
|
apiKey: config3.EMBEDDING_API_KEY,
|
|
196185
196227
|
model: config3.EMBEDDING_MODEL,
|
|
196186
196228
|
provider: "primary"
|
|
196187
|
-
}
|
|
196188
|
-
|
|
196189
|
-
if (config3.EMBEDDING_FALLBACK_BASE_URL && config3.EMBEDDING_FALLBACK_MODEL) {
|
|
196190
|
-
providers.push({
|
|
196229
|
+
},
|
|
196230
|
+
{
|
|
196191
196231
|
baseUrl: config3.EMBEDDING_FALLBACK_BASE_URL,
|
|
196192
196232
|
apiKey: config3.EMBEDDING_FALLBACK_API_KEY,
|
|
196193
196233
|
model: config3.EMBEDDING_FALLBACK_MODEL,
|
|
196194
196234
|
provider: "fallback"
|
|
196195
|
-
}
|
|
196196
|
-
|
|
196235
|
+
},
|
|
196236
|
+
{
|
|
196237
|
+
baseUrl: config3.EMBEDDING_FALLBACK_2_BASE_URL,
|
|
196238
|
+
apiKey: config3.EMBEDDING_FALLBACK_2_API_KEY,
|
|
196239
|
+
model: config3.EMBEDDING_FALLBACK_2_MODEL,
|
|
196240
|
+
provider: "fallback_2"
|
|
196241
|
+
}
|
|
196242
|
+
];
|
|
196243
|
+
return slots.flatMap((slot) => slot.baseUrl && slot.model ? [
|
|
196244
|
+
{
|
|
196245
|
+
baseUrl: slot.baseUrl,
|
|
196246
|
+
apiKey: slot.apiKey,
|
|
196247
|
+
model: slot.model,
|
|
196248
|
+
provider: slot.provider
|
|
196249
|
+
}
|
|
196250
|
+
] : []);
|
|
196251
|
+
}
|
|
196252
|
+
function assignProviderError(provider, message2, errors6) {
|
|
196253
|
+
if (provider === "primary")
|
|
196254
|
+
errors6.primaryError = message2;
|
|
196255
|
+
else if (provider === "fallback")
|
|
196256
|
+
errors6.fallbackError = message2;
|
|
196257
|
+
else
|
|
196258
|
+
errors6.fallback2Error = message2;
|
|
196259
|
+
}
|
|
196260
|
+
async function getEmbeddingInner(text3) {
|
|
196261
|
+
const providers = configuredEmbeddingProviders();
|
|
196197
196262
|
if (providers.length === 0) {
|
|
196198
196263
|
logger3.warn("No embedding provider configured (primary or fallback); embedding result is unavailable");
|
|
196199
196264
|
incrementCounter(METRIC_NAMES.EMBEDDING_ZERO);
|
|
196200
196265
|
return { ok: false, code: "not_configured" };
|
|
196201
196266
|
}
|
|
196202
|
-
|
|
196203
|
-
let fallbackError;
|
|
196267
|
+
const errors6 = {};
|
|
196204
196268
|
let finalCode = "provider_error";
|
|
196205
196269
|
for (const provider of providers) {
|
|
196206
196270
|
try {
|
|
@@ -196208,11 +196272,7 @@ async function getEmbeddingInner(text3) {
|
|
|
196208
196272
|
const validation = validateEmbeddingVector(vector5);
|
|
196209
196273
|
if (!validation.ok) {
|
|
196210
196274
|
finalCode = validation.code;
|
|
196211
|
-
|
|
196212
|
-
if (provider.provider === "primary")
|
|
196213
|
-
primaryError = message2;
|
|
196214
|
-
else
|
|
196215
|
-
fallbackError = message2;
|
|
196275
|
+
assignProviderError(provider.provider, `provider returned ${validation.code}`, errors6);
|
|
196216
196276
|
logger3.warn({
|
|
196217
196277
|
model: provider.model,
|
|
196218
196278
|
provider: provider.provider,
|
|
@@ -196235,11 +196295,7 @@ async function getEmbeddingInner(text3) {
|
|
|
196235
196295
|
};
|
|
196236
196296
|
} catch (err) {
|
|
196237
196297
|
finalCode = failureCodeFromError(err);
|
|
196238
|
-
|
|
196239
|
-
if (provider.provider === "primary")
|
|
196240
|
-
primaryError = message2;
|
|
196241
|
-
else
|
|
196242
|
-
fallbackError = message2;
|
|
196298
|
+
assignProviderError(provider.provider, safeErrorMessage(err), errors6);
|
|
196243
196299
|
logger3.warn({ err, model: provider.model, provider: provider.provider }, "Embedding provider failed, trying next configured provider");
|
|
196244
196300
|
}
|
|
196245
196301
|
}
|
|
@@ -196247,8 +196303,7 @@ async function getEmbeddingInner(text3) {
|
|
|
196247
196303
|
return {
|
|
196248
196304
|
ok: false,
|
|
196249
196305
|
code: finalCode,
|
|
196250
|
-
...
|
|
196251
|
-
...fallbackError ? { fallbackError } : {}
|
|
196306
|
+
...errors6
|
|
196252
196307
|
};
|
|
196253
196308
|
}
|
|
196254
196309
|
function safeErrorMessage(error53) {
|
|
@@ -231076,8 +231131,9 @@ async function expandFromQueryPlan(plan, limit = 20) {
|
|
|
231076
231131
|
const boundedLimit = Math.max(1, Math.min(Math.floor(limit), 100));
|
|
231077
231132
|
try {
|
|
231078
231133
|
const cypher = buildQuerySeedCypher(terms, boundedLimit);
|
|
231079
|
-
const
|
|
231080
|
-
|
|
231134
|
+
const rows = await runCypher(sql6, cypher, {
|
|
231135
|
+
columns: "neighbor_id agtype, generation_id agtype, relation agtype, hops agtype"
|
|
231136
|
+
});
|
|
231081
231137
|
const seen = new Set;
|
|
231082
231138
|
const out = [];
|
|
231083
231139
|
for (const row of rows) {
|
|
@@ -231112,8 +231168,9 @@ async function expandResults(docIds, maxHops = DEFAULT_MAX_HOPS) {
|
|
|
231112
231168
|
const sql6 = await getGraphDbRequired();
|
|
231113
231169
|
try {
|
|
231114
231170
|
const cypher = buildTraversalCypher(seeds, clampedHops);
|
|
231115
|
-
const
|
|
231116
|
-
|
|
231171
|
+
const rows = await runCypher(sql6, cypher, {
|
|
231172
|
+
columns: "seed_id agtype, seed_generation_id agtype, neighbor_id agtype, generation_id agtype, relation agtype, hops agtype"
|
|
231173
|
+
});
|
|
231117
231174
|
for (const row of rows) {
|
|
231118
231175
|
const seedId = stripQuotes(String(row.seed_id ?? ""));
|
|
231119
231176
|
const neighborId = stripQuotes(String(row.neighbor_id ?? ""));
|
|
@@ -231185,6 +231242,13 @@ function buildCypherSql(cypher, columns2) {
|
|
|
231185
231242
|
const quoted = `$${tag2}$ ${cypher} $${tag2}$`;
|
|
231186
231243
|
return `SELECT * FROM cypher('docs_graph', ${quoted}) AS (${columns2})`;
|
|
231187
231244
|
}
|
|
231245
|
+
async function runCypher(sql6, cypher, options) {
|
|
231246
|
+
const queryString = buildCypherSql(cypher, options.columns);
|
|
231247
|
+
return sql6.begin(async (tx) => {
|
|
231248
|
+
await tx.unsafe("SET LOCAL search_path = ag_catalog, public");
|
|
231249
|
+
return await tx.unsafe(queryString);
|
|
231250
|
+
});
|
|
231251
|
+
}
|
|
231188
231252
|
function stripQuotes(value) {
|
|
231189
231253
|
const trimmed = value.trim();
|
|
231190
231254
|
if (trimmed.length >= 2 && trimmed.startsWith('"') && trimmed.endsWith('"')) {
|
|
@@ -231571,7 +231635,7 @@ function createHealthRoutes(dependencies) {
|
|
|
231571
231635
|
set2.status = 503;
|
|
231572
231636
|
return {
|
|
231573
231637
|
status: readiness.status === "unhealthy" ? "unhealthy" : "ok",
|
|
231574
|
-
service: "
|
|
231638
|
+
service: "docsmint",
|
|
231575
231639
|
timestamp: new Date().toISOString(),
|
|
231576
231640
|
database: databaseAvailable ? "ok" : "error",
|
|
231577
231641
|
redis: redisAvailable ? "ok" : "error",
|
|
@@ -231876,9 +231940,10 @@ async function retrieveGraphCandidates(ctx, request, adapters = {}) {
|
|
|
231876
231940
|
const requestedSeeds = dedupe2(request.documentSeeds).slice(0, config3.SEARCH_GRAPH_SEED_LIMIT);
|
|
231877
231941
|
const visibilityScope = request.visibilityScope ?? _buildGraphVisibilityScope(ctx);
|
|
231878
231942
|
const tenantTransaction = adapters.withTenant ?? withTenant;
|
|
231879
|
-
const
|
|
231943
|
+
const categoryId = visibilityScope.kind === "share" ? undefined : request.categoryId;
|
|
231944
|
+
const authorizedSeeds = requestedSeeds.length === 0 ? new Set : await resolveVisibleIds(ctx, requestedSeeds, adapters.visibleDocumentIds, visibilityScope, categoryId, tenantTransaction);
|
|
231880
231945
|
const seeds = requestedSeeds.filter((id3) => authorizedSeeds.has(id3));
|
|
231881
|
-
const activeSeedGenerations = await resolveActiveGenerations(ctx, seeds, adapters.visibleDocumentGenerations, visibilityScope,
|
|
231946
|
+
const activeSeedGenerations = await resolveActiveGenerations(ctx, seeds, adapters.visibleDocumentGenerations, visibilityScope, categoryId, Boolean(adapters.visibleDocumentIds), tenantTransaction);
|
|
231882
231947
|
const expand2 = adapters.expandResults ?? expandResults;
|
|
231883
231948
|
const expandQuery = adapters.expandFromQueryPlan ?? expandFromQueryPlan;
|
|
231884
231949
|
let related = [];
|
|
@@ -231902,8 +231967,8 @@ async function retrieveGraphCandidates(ctx, request, adapters = {}) {
|
|
|
231902
231967
|
});
|
|
231903
231968
|
});
|
|
231904
231969
|
const allIds = dedupe2(relatedWithCurrentSeed.map((candidate) => candidate.docId));
|
|
231905
|
-
const visible = await resolveVisibleIds(ctx, allIds, adapters.visibleDocumentIds, visibilityScope,
|
|
231906
|
-
const activeGenerations = await resolveActiveGenerations(ctx, allIds, adapters.visibleDocumentGenerations, visibilityScope,
|
|
231970
|
+
const visible = await resolveVisibleIds(ctx, allIds, adapters.visibleDocumentIds, visibilityScope, categoryId, tenantTransaction);
|
|
231971
|
+
const activeGenerations = await resolveActiveGenerations(ctx, allIds, adapters.visibleDocumentGenerations, visibilityScope, categoryId, Boolean(adapters.visibleDocumentIds), tenantTransaction);
|
|
231907
231972
|
const unique = new Map;
|
|
231908
231973
|
for (const candidate of relatedWithCurrentSeed) {
|
|
231909
231974
|
if (!visible.has(candidate.docId))
|
|
@@ -232168,8 +232233,30 @@ function resolveChatProviderKey(baseUrl, explicitKey, sharedOpenRouterKey) {
|
|
|
232168
232233
|
return "";
|
|
232169
232234
|
return sharedOpenRouterKey?.trim() ?? "";
|
|
232170
232235
|
}
|
|
232236
|
+
function uniqueChatProviders(providers) {
|
|
232237
|
+
const seen = new Set;
|
|
232238
|
+
const unique = [];
|
|
232239
|
+
for (const provider of providers) {
|
|
232240
|
+
if (!provider?.baseUrl || !provider.model)
|
|
232241
|
+
continue;
|
|
232242
|
+
const identity = [
|
|
232243
|
+
provider.baseUrl,
|
|
232244
|
+
provider.model,
|
|
232245
|
+
provider.apiKey ?? ""
|
|
232246
|
+
].join("|");
|
|
232247
|
+
if (seen.has(identity))
|
|
232248
|
+
continue;
|
|
232249
|
+
seen.add(identity);
|
|
232250
|
+
unique.push(provider);
|
|
232251
|
+
}
|
|
232252
|
+
return unique;
|
|
232253
|
+
}
|
|
232171
232254
|
async function requestStructuredChat(options) {
|
|
232172
|
-
const providers = [
|
|
232255
|
+
const providers = uniqueChatProviders([
|
|
232256
|
+
options.primary,
|
|
232257
|
+
options.fallback,
|
|
232258
|
+
...options.fallbacks ?? []
|
|
232259
|
+
]);
|
|
232173
232260
|
for (const provider of providers) {
|
|
232174
232261
|
try {
|
|
232175
232262
|
const raw = await requestChatContent(provider, options);
|
|
@@ -232241,13 +232328,18 @@ async function expandQuery(plan, scope) {
|
|
|
232241
232328
|
const tenantScope = typeof scope === "string" ? scope.trim() : (scope.tenantScope ?? scope.tenantId ?? scope.ownerId ?? "").trim();
|
|
232242
232329
|
if (!tenantScope || !plan.normalized.trim())
|
|
232243
232330
|
return null;
|
|
232244
|
-
const
|
|
232245
|
-
|
|
232246
|
-
|
|
232247
|
-
|
|
232248
|
-
|
|
232249
|
-
|
|
232250
|
-
|
|
232331
|
+
const chain3 = uniqueChatProviders([
|
|
232332
|
+
providerConfig(config3.SEARCH_EXPANSION_BASE_URL, config3.SEARCH_EXPANSION_API_KEY, config3.SEARCH_EXPANSION_MODEL),
|
|
232333
|
+
providerConfig(config3.SEARCH_EXPANSION_FALLBACK_BASE_URL, config3.SEARCH_EXPANSION_FALLBACK_API_KEY, config3.SEARCH_EXPANSION_FALLBACK_MODEL),
|
|
232334
|
+
providerConfig(config3.SEARCH_EXPANSION_FALLBACK_2_BASE_URL, config3.SEARCH_EXPANSION_FALLBACK_2_API_KEY, config3.SEARCH_EXPANSION_FALLBACK_2_MODEL)
|
|
232335
|
+
]);
|
|
232336
|
+
const providerTimeoutMs = Math.max(250, Math.floor(config3.SEARCH_EXPANSION_TIMEOUT_MS / Math.max(1, chain3.length)));
|
|
232337
|
+
for (const provider of chain3)
|
|
232338
|
+
provider.timeoutMs = providerTimeoutMs;
|
|
232339
|
+
const [primary, ...fallbacks] = chain3;
|
|
232340
|
+
if (!primary)
|
|
232341
|
+
return null;
|
|
232342
|
+
const key = await expansionCacheKey(tenantScope, plan.normalized, primary, fallbacks);
|
|
232251
232343
|
const ttl = config3.SEARCH_EXPANSION_CACHE_TTL_SECONDS;
|
|
232252
232344
|
if (ttl > 0) {
|
|
232253
232345
|
try {
|
|
@@ -232272,7 +232364,7 @@ async function expandQuery(plan, scope) {
|
|
|
232272
232364
|
}
|
|
232273
232365
|
const result = await requestStructuredChat({
|
|
232274
232366
|
primary,
|
|
232275
|
-
|
|
232367
|
+
fallbacks,
|
|
232276
232368
|
messages: [
|
|
232277
232369
|
{
|
|
232278
232370
|
role: "system",
|
|
@@ -232352,12 +232444,15 @@ var cachedResultSchema = exports_external2.object({
|
|
|
232352
232444
|
}),
|
|
232353
232445
|
model: exports_external2.string()
|
|
232354
232446
|
});
|
|
232355
|
-
async function expansionCacheKey(tenantScope, normalizedQuery, primary =
|
|
232447
|
+
async function expansionCacheKey(tenantScope, normalizedQuery, primary = {
|
|
232448
|
+
baseUrl: config3.SEARCH_EXPANSION_BASE_URL,
|
|
232449
|
+
model: config3.SEARCH_EXPANSION_MODEL,
|
|
232450
|
+
timeoutMs: config3.SEARCH_EXPANSION_TIMEOUT_MS
|
|
232451
|
+
}, fallbacks = []) {
|
|
232356
232452
|
const modelProfile = [
|
|
232357
232453
|
primary.baseUrl,
|
|
232358
232454
|
primary.model,
|
|
232359
|
-
|
|
232360
|
-
fallback?.model ?? ""
|
|
232455
|
+
...fallbacks.flatMap((provider) => [provider.baseUrl, provider.model])
|
|
232361
232456
|
].join("|");
|
|
232362
232457
|
const input = [
|
|
232363
232458
|
tenantScope,
|
|
@@ -232369,6 +232464,8 @@ async function expansionCacheKey(tenantScope, normalizedQuery, primary = provide
|
|
|
232369
232464
|
return `hiai-docs:search:expansion:${digest4}`;
|
|
232370
232465
|
}
|
|
232371
232466
|
function providerConfig(baseUrl, explicitKey, model) {
|
|
232467
|
+
if (!baseUrl || !model)
|
|
232468
|
+
return;
|
|
232372
232469
|
return {
|
|
232373
232470
|
baseUrl,
|
|
232374
232471
|
model,
|
|
@@ -232396,6 +232493,174 @@ function normalizeVariant(value) {
|
|
|
232396
232493
|
return value.normalize("NFKC").trim().replace(/\s+/g, " ").toLocaleLowerCase();
|
|
232397
232494
|
}
|
|
232398
232495
|
|
|
232496
|
+
// ../../backend/src/search/rerank.ts
|
|
232497
|
+
var DEFAULT_RERANK_MAX_CHARS = 1500;
|
|
232498
|
+
function normalizeRerankText(title, content50, maxChars = DEFAULT_RERANK_MAX_CHARS) {
|
|
232499
|
+
const cleaned = `${title.trim()}
|
|
232500
|
+
${content50}`.replace(/data:[^\s)>]+/gi, " ").replace(/\s+/g, " ").trim();
|
|
232501
|
+
if (cleaned.length <= maxChars)
|
|
232502
|
+
return cleaned;
|
|
232503
|
+
return cleaned.slice(0, maxChars);
|
|
232504
|
+
}
|
|
232505
|
+
function applyRerankOrder(ranked, hits, topN) {
|
|
232506
|
+
if (ranked.length === 0 || hits.length === 0 || topN < 1)
|
|
232507
|
+
return ranked;
|
|
232508
|
+
const window2 = ranked.slice(0, Math.min(topN, ranked.length));
|
|
232509
|
+
const rest = ranked.slice(window2.length);
|
|
232510
|
+
const byId = new Map(window2.map((item) => [item.documentId, item]));
|
|
232511
|
+
const seen = new Set;
|
|
232512
|
+
const reordered = [];
|
|
232513
|
+
const sortedHits = [...hits].sort((left, right) => {
|
|
232514
|
+
if (right.score !== left.score)
|
|
232515
|
+
return right.score - left.score;
|
|
232516
|
+
return left.rank - right.rank;
|
|
232517
|
+
});
|
|
232518
|
+
for (const hit of sortedHits) {
|
|
232519
|
+
const item = byId.get(hit.id);
|
|
232520
|
+
if (!item || seen.has(hit.id))
|
|
232521
|
+
continue;
|
|
232522
|
+
seen.add(hit.id);
|
|
232523
|
+
reordered.push(item);
|
|
232524
|
+
}
|
|
232525
|
+
for (const item of window2) {
|
|
232526
|
+
if (seen.has(item.documentId))
|
|
232527
|
+
continue;
|
|
232528
|
+
reordered.push(item);
|
|
232529
|
+
}
|
|
232530
|
+
return [...reordered, ...rest];
|
|
232531
|
+
}
|
|
232532
|
+
|
|
232533
|
+
// ../../backend/src/search/rerank-provider.ts
|
|
232534
|
+
init_config();
|
|
232535
|
+
init_logger3();
|
|
232536
|
+
function configuredSlots() {
|
|
232537
|
+
const slots = [
|
|
232538
|
+
{
|
|
232539
|
+
baseUrl: config3.SEARCH_RERANK_BASE_URL,
|
|
232540
|
+
apiKey: config3.SEARCH_RERANK_API_KEY,
|
|
232541
|
+
model: config3.SEARCH_RERANK_MODEL
|
|
232542
|
+
},
|
|
232543
|
+
{
|
|
232544
|
+
baseUrl: config3.SEARCH_RERANK_FALLBACK_BASE_URL,
|
|
232545
|
+
apiKey: config3.SEARCH_RERANK_FALLBACK_API_KEY,
|
|
232546
|
+
model: config3.SEARCH_RERANK_FALLBACK_MODEL
|
|
232547
|
+
},
|
|
232548
|
+
{
|
|
232549
|
+
baseUrl: config3.SEARCH_RERANK_FALLBACK_2_BASE_URL,
|
|
232550
|
+
apiKey: config3.SEARCH_RERANK_FALLBACK_2_API_KEY,
|
|
232551
|
+
model: config3.SEARCH_RERANK_FALLBACK_2_MODEL
|
|
232552
|
+
}
|
|
232553
|
+
];
|
|
232554
|
+
const seen = new Set;
|
|
232555
|
+
const unique = [];
|
|
232556
|
+
for (const slot of slots) {
|
|
232557
|
+
if (!slot.baseUrl || !slot.model)
|
|
232558
|
+
continue;
|
|
232559
|
+
const identity = `${slot.baseUrl}|${slot.model}`;
|
|
232560
|
+
if (seen.has(identity))
|
|
232561
|
+
continue;
|
|
232562
|
+
seen.add(identity);
|
|
232563
|
+
unique.push({
|
|
232564
|
+
baseUrl: slot.baseUrl,
|
|
232565
|
+
model: slot.model,
|
|
232566
|
+
apiKey: resolveChatProviderKey(slot.baseUrl, slot.apiKey, config3.OPENROUTER_API_KEY)
|
|
232567
|
+
});
|
|
232568
|
+
}
|
|
232569
|
+
return unique;
|
|
232570
|
+
}
|
|
232571
|
+
function rerankUrl(baseUrl) {
|
|
232572
|
+
const normalized = baseUrl.replace(/\/$/, "");
|
|
232573
|
+
return normalized.endsWith("/rerank") ? normalized : `${normalized}/rerank`;
|
|
232574
|
+
}
|
|
232575
|
+
async function requestRerank(input) {
|
|
232576
|
+
const documents2 = input.candidates.map((candidate) => candidate.text.trim()).filter((text4) => text4.length > 0);
|
|
232577
|
+
if (documents2.length === 0)
|
|
232578
|
+
return null;
|
|
232579
|
+
const ids = input.candidates.map((candidate) => candidate.id);
|
|
232580
|
+
const topN = Math.min(input.topN ?? config3.SEARCH_RERANK_TOP_N, documents2.length);
|
|
232581
|
+
const timeoutMs = config3.SEARCH_RERANK_TIMEOUT_MS;
|
|
232582
|
+
const started = performance.now();
|
|
232583
|
+
incrementCounterBy(METRIC_NAMES.SEARCH_RERANK_CANDIDATES_TOTAL, documents2.length);
|
|
232584
|
+
try {
|
|
232585
|
+
for (const slot of configuredSlots()) {
|
|
232586
|
+
try {
|
|
232587
|
+
const hits = await callRerankSlot(slot, input.query, documents2, ids, {
|
|
232588
|
+
topN,
|
|
232589
|
+
timeoutMs
|
|
232590
|
+
});
|
|
232591
|
+
if (hits.length === 0)
|
|
232592
|
+
continue;
|
|
232593
|
+
incrementCounter(METRIC_NAMES.SEARCH_RERANK_SUCCESS_TOTAL);
|
|
232594
|
+
return { hits, model: slot.model };
|
|
232595
|
+
} catch (error53) {
|
|
232596
|
+
logger3.warn({ err: error53, model: slot.model }, "Rerank provider failed, trying next configured provider");
|
|
232597
|
+
}
|
|
232598
|
+
}
|
|
232599
|
+
incrementCounter(METRIC_NAMES.SEARCH_RERANK_FALLBACK_TOTAL);
|
|
232600
|
+
return null;
|
|
232601
|
+
} finally {
|
|
232602
|
+
recordDuration(METRIC_NAMES.SEARCH_RERANK_DURATION_MS, performance.now() - started);
|
|
232603
|
+
}
|
|
232604
|
+
}
|
|
232605
|
+
async function callRerankSlot(slot, query, documents2, ids, options) {
|
|
232606
|
+
const controller = new AbortController;
|
|
232607
|
+
const timer2 = setTimeout(() => controller.abort(), options.timeoutMs);
|
|
232608
|
+
const headers = {
|
|
232609
|
+
"Content-Type": "application/json"
|
|
232610
|
+
};
|
|
232611
|
+
if (slot.apiKey)
|
|
232612
|
+
headers.Authorization = `Bearer ${slot.apiKey}`;
|
|
232613
|
+
try {
|
|
232614
|
+
const response = await fetch(rerankUrl(slot.baseUrl), {
|
|
232615
|
+
method: "POST",
|
|
232616
|
+
headers,
|
|
232617
|
+
signal: controller.signal,
|
|
232618
|
+
body: JSON.stringify({
|
|
232619
|
+
model: slot.model,
|
|
232620
|
+
query,
|
|
232621
|
+
documents: documents2,
|
|
232622
|
+
top_n: options.topN
|
|
232623
|
+
})
|
|
232624
|
+
});
|
|
232625
|
+
if (!response.ok) {
|
|
232626
|
+
throw new Error(`rerank provider returned ${response.status}`);
|
|
232627
|
+
}
|
|
232628
|
+
const body = await response.json();
|
|
232629
|
+
const hits = [];
|
|
232630
|
+
for (const [rank, row] of (body.results ?? []).entries()) {
|
|
232631
|
+
if (typeof row.index !== "number" || row.index < 0)
|
|
232632
|
+
continue;
|
|
232633
|
+
const id3 = ids[row.index];
|
|
232634
|
+
if (!id3)
|
|
232635
|
+
continue;
|
|
232636
|
+
const score = typeof row.relevance_score === "number" && Number.isFinite(row.relevance_score) ? row.relevance_score : 0;
|
|
232637
|
+
hits.push({ id: id3, score, rank: rank + 1 });
|
|
232638
|
+
}
|
|
232639
|
+
return hits;
|
|
232640
|
+
} finally {
|
|
232641
|
+
clearTimeout(timer2);
|
|
232642
|
+
}
|
|
232643
|
+
}
|
|
232644
|
+
|
|
232645
|
+
// ../../backend/src/search/rerank-texts.ts
|
|
232646
|
+
init_schema2();
|
|
232647
|
+
init_drizzle_orm();
|
|
232648
|
+
init_config();
|
|
232649
|
+
async function loadRerankTexts(ctx, documentIds, maxChars = config3.SEARCH_RERANK_MAX_CHARS) {
|
|
232650
|
+
const texts = new Map;
|
|
232651
|
+
if (documentIds.length === 0)
|
|
232652
|
+
return texts;
|
|
232653
|
+
const rows = await withTenant(ctx, async (tx) => tx.select({
|
|
232654
|
+
id: documents.id,
|
|
232655
|
+
title: documents.title,
|
|
232656
|
+
content: documents.content
|
|
232657
|
+
}).from(documents).where(and(isNull(documents.deletedAt), inArray(documents.id, documentIds))));
|
|
232658
|
+
for (const row of rows) {
|
|
232659
|
+
texts.set(row.id, normalizeRerankText(row.title, row.content ?? "", maxChars));
|
|
232660
|
+
}
|
|
232661
|
+
return texts;
|
|
232662
|
+
}
|
|
232663
|
+
|
|
232399
232664
|
// ../../backend/src/search/retrievers.ts
|
|
232400
232665
|
init_drizzle_orm();
|
|
232401
232666
|
init_config();
|
|
@@ -232714,6 +232979,8 @@ async function searchDocuments(ctx, request, adapters = {}) {
|
|
|
232714
232979
|
const embeddingProvider = adapters.getEmbedding ?? getEmbedding;
|
|
232715
232980
|
const expand2 = adapters.expand ?? expandQuery;
|
|
232716
232981
|
const retrieveGraph = adapters.retrieveGraph ?? retrieveGraphCandidates;
|
|
232982
|
+
const rerankDocuments = adapters.rerank ?? requestRerank;
|
|
232983
|
+
const loadTexts = adapters.loadRerankTexts ?? loadRerankTexts;
|
|
232717
232984
|
const filterRanked = adapters.filterRanked ?? applySearchFilters;
|
|
232718
232985
|
const channelErrors = {};
|
|
232719
232986
|
const embeddingCache = new Map;
|
|
@@ -232807,27 +233074,36 @@ async function searchDocuments(ctx, request, adapters = {}) {
|
|
|
232807
233074
|
model: expansionModel,
|
|
232808
233075
|
primaryModel: config3.SEARCH_EXPANSION_MODEL,
|
|
232809
233076
|
fallbackModel: config3.SEARCH_EXPANSION_FALLBACK_MODEL,
|
|
233077
|
+
fallback2Model: config3.SEARCH_EXPANSION_FALLBACK_2_MODEL,
|
|
232810
233078
|
estimatedCostMicrounits: expansionUsed ? config3.SEARCH_EXPANSION_ESTIMATED_COST_MICROUNITS : 0
|
|
232811
233079
|
});
|
|
232812
233080
|
recordDuration(METRIC_NAMES.SEARCH_EXPANDED_DURATION_MS, performance.now() - expandedStarted);
|
|
232813
233081
|
}
|
|
232814
|
-
const
|
|
233082
|
+
const rrfOptions = {
|
|
232815
233083
|
rrfK: config3.SEARCH_RRF_K,
|
|
232816
233084
|
exactBoost: config3.SEARCH_EXACT_BOOST,
|
|
232817
233085
|
channelAgreementBoost: config3.SEARCH_CHANNEL_AGREEMENT_BOOST,
|
|
232818
233086
|
graphMaxContribution: config3.SEARCH_GRAPH_MAX_CONTRIBUTION,
|
|
232819
233087
|
vectorMinSimilarity: config3.SEARCH_VECTOR_MIN_SIMILARITY
|
|
232820
|
-
}
|
|
233088
|
+
};
|
|
233089
|
+
const direct = fuseCandidates([...fast, ...expanded].flatMap((result) => result.candidates), rrfOptions);
|
|
232821
233090
|
const graphPlan = expandedPlan ?? plan;
|
|
232822
|
-
const
|
|
233091
|
+
const graphAttempted = request.graphEnabled !== false;
|
|
233092
|
+
const rerankEnabled = request.rerankEnabled ?? config3.SEARCH_RERANK_ENABLED;
|
|
233093
|
+
const graphPosition = request.rerankGraphPosition ?? config3.SEARCH_RERANK_GRAPH_POSITION;
|
|
232823
233094
|
let graph = [];
|
|
232824
233095
|
let graphFailed = false;
|
|
232825
|
-
|
|
232826
|
-
|
|
233096
|
+
let ranked;
|
|
233097
|
+
let rerankAttempted = false;
|
|
233098
|
+
let rerankUsed = false;
|
|
233099
|
+
let rerankFallback = false;
|
|
233100
|
+
let rerankModel;
|
|
233101
|
+
const runGraph = async (seeds) => {
|
|
232827
233102
|
const graphStarted = performance.now();
|
|
233103
|
+
let next = [];
|
|
232828
233104
|
try {
|
|
232829
|
-
|
|
232830
|
-
documentSeeds:
|
|
233105
|
+
next = await retrieveGraph(ctx, {
|
|
233106
|
+
documentSeeds: seeds,
|
|
232831
233107
|
queryPlan: graphPlan,
|
|
232832
233108
|
limit: Math.min(limit * 2, config3.SEARCH_GRAPH_RESULT_LIMIT),
|
|
232833
233109
|
maxHops: request.maxGraphHops ?? config3.SEARCH_GRAPH_MAX_HOPS,
|
|
@@ -232837,25 +233113,82 @@ async function searchDocuments(ctx, request, adapters = {}) {
|
|
|
232837
233113
|
} catch {
|
|
232838
233114
|
graphFailed = true;
|
|
232839
233115
|
}
|
|
232840
|
-
|
|
233116
|
+
next = restrictCandidates(next, request.documentIds);
|
|
232841
233117
|
recordSearchChannelMetrics({
|
|
232842
233118
|
channel: "graph",
|
|
232843
233119
|
durationMs: performance.now() - graphStarted,
|
|
232844
|
-
candidateCount:
|
|
233120
|
+
candidateCount: next.length,
|
|
232845
233121
|
errorCode: graphFailed ? "query_failed" : undefined
|
|
232846
233122
|
});
|
|
233123
|
+
return next;
|
|
233124
|
+
};
|
|
233125
|
+
const seedIds = (items2) => items2.slice(0, config3.SEARCH_GRAPH_SEED_LIMIT).map((item) => item.documentId);
|
|
233126
|
+
const applyOptionalRerank = async (items2) => {
|
|
233127
|
+
if (!rerankEnabled || items2.length === 0)
|
|
233128
|
+
return items2;
|
|
233129
|
+
rerankAttempted = true;
|
|
233130
|
+
try {
|
|
233131
|
+
const topN = Math.min(config3.SEARCH_RERANK_TOP_N, items2.length);
|
|
233132
|
+
const window2 = items2.slice(0, topN);
|
|
233133
|
+
const texts = await loadTexts(ctx, window2.map((item) => item.documentId));
|
|
233134
|
+
const candidates = window2.map((item, index2) => ({
|
|
233135
|
+
id: item.documentId,
|
|
233136
|
+
text: texts.get(item.documentId) ?? "",
|
|
233137
|
+
sourceRank: index2 + 1,
|
|
233138
|
+
rrfScore: item.score
|
|
233139
|
+
}));
|
|
233140
|
+
if (candidates.every((candidate) => candidate.text.length === 0)) {
|
|
233141
|
+
rerankFallback = true;
|
|
233142
|
+
return items2;
|
|
233143
|
+
}
|
|
233144
|
+
const result = await rerankDocuments({
|
|
233145
|
+
query: plan.original,
|
|
233146
|
+
candidates,
|
|
233147
|
+
topN
|
|
233148
|
+
});
|
|
233149
|
+
if (!result || result.hits.length === 0) {
|
|
233150
|
+
rerankFallback = true;
|
|
233151
|
+
return items2;
|
|
233152
|
+
}
|
|
233153
|
+
rerankUsed = true;
|
|
233154
|
+
rerankModel = result.model;
|
|
233155
|
+
return applyRerankOrder(items2, result.hits, topN);
|
|
233156
|
+
} catch {
|
|
233157
|
+
rerankFallback = true;
|
|
233158
|
+
return items2;
|
|
233159
|
+
}
|
|
233160
|
+
};
|
|
233161
|
+
if (!rerankEnabled) {
|
|
233162
|
+
if (graphAttempted)
|
|
233163
|
+
graph = await runGraph(seedIds(direct));
|
|
233164
|
+
ranked = fuseCandidates([
|
|
233165
|
+
...fast.flatMap((result) => result.candidates),
|
|
233166
|
+
...expanded.flatMap((result) => result.candidates),
|
|
233167
|
+
...graph
|
|
233168
|
+
], rrfOptions);
|
|
233169
|
+
} else if (graphPosition === "before") {
|
|
233170
|
+
if (graphAttempted)
|
|
233171
|
+
graph = await runGraph(seedIds(direct));
|
|
233172
|
+
ranked = fuseCandidates([
|
|
233173
|
+
...fast.flatMap((result) => result.candidates),
|
|
233174
|
+
...expanded.flatMap((result) => result.candidates),
|
|
233175
|
+
...graph
|
|
233176
|
+
], rrfOptions);
|
|
233177
|
+
ranked = await applyOptionalRerank(ranked);
|
|
233178
|
+
} else {
|
|
233179
|
+
ranked = await applyOptionalRerank(direct);
|
|
233180
|
+
if (graphAttempted)
|
|
233181
|
+
graph = await runGraph(seedIds(ranked));
|
|
233182
|
+
ranked = annotateGraphChannels(ranked, graph);
|
|
233183
|
+
const present = new Set(ranked.map((item) => item.documentId));
|
|
233184
|
+
const extra = graph.filter((candidate) => !present.has(candidate.documentId));
|
|
233185
|
+
if (extra.length > 0) {
|
|
233186
|
+
ranked = [
|
|
233187
|
+
...ranked,
|
|
233188
|
+
...fuseCandidates(extra, rrfOptions).filter((item) => !present.has(item.documentId))
|
|
233189
|
+
];
|
|
233190
|
+
}
|
|
232847
233191
|
}
|
|
232848
|
-
const ranked = fuseCandidates([
|
|
232849
|
-
...fast.flatMap((result) => result.candidates),
|
|
232850
|
-
...expanded.flatMap((result) => result.candidates),
|
|
232851
|
-
...graph
|
|
232852
|
-
], {
|
|
232853
|
-
rrfK: config3.SEARCH_RRF_K,
|
|
232854
|
-
exactBoost: config3.SEARCH_EXACT_BOOST,
|
|
232855
|
-
channelAgreementBoost: config3.SEARCH_CHANNEL_AGREEMENT_BOOST,
|
|
232856
|
-
graphMaxContribution: config3.SEARCH_GRAPH_MAX_CONTRIBUTION,
|
|
232857
|
-
vectorMinSimilarity: config3.SEARCH_VECTOR_MIN_SIMILARITY
|
|
232858
|
-
});
|
|
232859
233192
|
const filters = request.filters;
|
|
232860
233193
|
const hasFilters = Boolean(filters && Object.values(filters).some((value) => Array.isArray(value) ? value.length > 0 : value !== undefined));
|
|
232861
233194
|
const filtered = hasFilters || request.visibilityScope !== undefined ? await filterRanked(ctx, ranked, filters ?? {}, request.visibilityScope) : ranked;
|
|
@@ -232881,7 +233214,11 @@ async function searchDocuments(ctx, request, adapters = {}) {
|
|
|
232881
233214
|
graphAttempted,
|
|
232882
233215
|
graphFailed,
|
|
232883
233216
|
graphContribution,
|
|
232884
|
-
confidenceReasons: confidence.reasons
|
|
233217
|
+
confidenceReasons: confidence.reasons,
|
|
233218
|
+
rerankAttempted,
|
|
233219
|
+
rerankUsed,
|
|
233220
|
+
rerankFallback,
|
|
233221
|
+
...rerankModel ? { rerankModel } : {}
|
|
232885
233222
|
};
|
|
232886
233223
|
return {
|
|
232887
233224
|
items,
|
|
@@ -233049,6 +233386,24 @@ function clampPage(value) {
|
|
|
233049
233386
|
return 1;
|
|
233050
233387
|
return Math.max(1, Math.floor(value));
|
|
233051
233388
|
}
|
|
233389
|
+
function annotateGraphChannels(items, graphCandidates) {
|
|
233390
|
+
if (graphCandidates.length === 0)
|
|
233391
|
+
return items;
|
|
233392
|
+
const graphIds = new Set(graphCandidates.map((candidate) => candidate.documentId));
|
|
233393
|
+
return items.map((item) => {
|
|
233394
|
+
if (!graphIds.has(item.documentId) || item.channels.includes("graph")) {
|
|
233395
|
+
return item;
|
|
233396
|
+
}
|
|
233397
|
+
return {
|
|
233398
|
+
...item,
|
|
233399
|
+
channels: [...item.channels, "graph"],
|
|
233400
|
+
explanations: [
|
|
233401
|
+
...item.explanations,
|
|
233402
|
+
{ channel: "graph", label: "Related document" }
|
|
233403
|
+
]
|
|
233404
|
+
};
|
|
233405
|
+
});
|
|
233406
|
+
}
|
|
233052
233407
|
|
|
233053
233408
|
// ../../backend/src/api/routes/search.ts
|
|
233054
233409
|
var BENCHMARK_SEARCH_PROFILE_HEADER = "x-docsmint-search-profile";
|
|
@@ -235564,7 +235919,42 @@ async function extractEntities(chunkText2, documentId, options = {}) {
|
|
|
235564
235919
|
return { status: "ready", entities };
|
|
235565
235920
|
}
|
|
235566
235921
|
function hasConfiguredGraphProvider(options) {
|
|
235567
|
-
return Boolean(options.llmBaseUrl || config3.GRAPH_EXTRACT_BASE_URL || config3.GRAPH_EXTRACT_FALLBACK_BASE_URL);
|
|
235922
|
+
return Boolean(options.llmBaseUrl || config3.GRAPH_EXTRACT_BASE_URL || config3.GRAPH_EXTRACT_FALLBACK_BASE_URL || config3.GRAPH_EXTRACT_FALLBACK_2_BASE_URL);
|
|
235923
|
+
}
|
|
235924
|
+
function graphExtractProviders(options) {
|
|
235925
|
+
const timeoutMs = config3.GRAPH_EXTRACT_TIMEOUT_MS;
|
|
235926
|
+
const primaryModel = options.llmModel ?? config3.GRAPH_EXTRACT_MODEL ?? "gpt-4o-mini";
|
|
235927
|
+
const slots = [
|
|
235928
|
+
{
|
|
235929
|
+
baseUrl: options.llmBaseUrl ?? config3.GRAPH_EXTRACT_BASE_URL,
|
|
235930
|
+
explicitKey: options.llmApiKey ?? config3.GRAPH_EXTRACT_API_KEY,
|
|
235931
|
+
model: primaryModel,
|
|
235932
|
+
reasoningEffort: options.reasoningEffort ?? config3.GRAPH_EXTRACT_REASONING_EFFORT
|
|
235933
|
+
},
|
|
235934
|
+
{
|
|
235935
|
+
baseUrl: config3.GRAPH_EXTRACT_FALLBACK_BASE_URL,
|
|
235936
|
+
explicitKey: config3.GRAPH_EXTRACT_FALLBACK_API_KEY,
|
|
235937
|
+
model: config3.GRAPH_EXTRACT_FALLBACK_MODEL ?? primaryModel,
|
|
235938
|
+
reasoningEffort: config3.GRAPH_EXTRACT_REASONING_EFFORT
|
|
235939
|
+
},
|
|
235940
|
+
{
|
|
235941
|
+
baseUrl: config3.GRAPH_EXTRACT_FALLBACK_2_BASE_URL,
|
|
235942
|
+
explicitKey: config3.GRAPH_EXTRACT_FALLBACK_2_API_KEY,
|
|
235943
|
+
model: config3.GRAPH_EXTRACT_FALLBACK_2_MODEL,
|
|
235944
|
+
reasoningEffort: config3.GRAPH_EXTRACT_REASONING_EFFORT
|
|
235945
|
+
}
|
|
235946
|
+
];
|
|
235947
|
+
return uniqueChatProviders(slots.map((slot) => {
|
|
235948
|
+
if (!slot.baseUrl || !slot.model)
|
|
235949
|
+
return;
|
|
235950
|
+
return {
|
|
235951
|
+
baseUrl: slot.baseUrl,
|
|
235952
|
+
apiKey: resolveGraphProviderKey(slot.baseUrl, slot.explicitKey),
|
|
235953
|
+
model: slot.model,
|
|
235954
|
+
timeoutMs,
|
|
235955
|
+
reasoningEffort: slot.reasoningEffort
|
|
235956
|
+
};
|
|
235957
|
+
}));
|
|
235568
235958
|
}
|
|
235569
235959
|
var SYSTEM_PROMPT = [
|
|
235570
235960
|
"You are an entity-extraction assistant for a knowledge-base system.",
|
|
@@ -235600,39 +235990,13 @@ var extractionOutputSchema = exports_external2.object({
|
|
|
235600
235990
|
entities: exports_external2.array(exports_external2.unknown())
|
|
235601
235991
|
});
|
|
235602
235992
|
async function callEntityExtractionLLM(text4, options) {
|
|
235603
|
-
const
|
|
235604
|
-
const
|
|
235605
|
-
const primaryModel = options.llmModel ?? config3.GRAPH_EXTRACT_MODEL ?? "gpt-4o-mini";
|
|
235606
|
-
const primaryKey2 = primaryBase ? resolveGraphProviderKey(primaryBase, primaryExplicitKey) : "";
|
|
235607
|
-
const fallbackBase = config3.GRAPH_EXTRACT_FALLBACK_BASE_URL;
|
|
235608
|
-
const fallbackExplicitKey = config3.GRAPH_EXTRACT_FALLBACK_API_KEY;
|
|
235609
|
-
const fallbackModel = config3.GRAPH_EXTRACT_FALLBACK_MODEL ?? primaryModel;
|
|
235610
|
-
const fallbackKey = fallbackBase ? resolveGraphProviderKey(fallbackBase, fallbackExplicitKey) : "";
|
|
235611
|
-
const providers = [];
|
|
235612
|
-
if (primaryBase) {
|
|
235613
|
-
providers.push({
|
|
235614
|
-
baseUrl: primaryBase,
|
|
235615
|
-
apiKey: primaryKey2,
|
|
235616
|
-
model: primaryModel,
|
|
235617
|
-
timeoutMs: config3.GRAPH_EXTRACT_TIMEOUT_MS,
|
|
235618
|
-
reasoningEffort: options.reasoningEffort ?? config3.GRAPH_EXTRACT_REASONING_EFFORT
|
|
235619
|
-
});
|
|
235620
|
-
}
|
|
235621
|
-
if (fallbackBase && (fallbackBase !== primaryBase || fallbackKey !== primaryKey2 || fallbackModel !== primaryModel)) {
|
|
235622
|
-
providers.push({
|
|
235623
|
-
baseUrl: fallbackBase,
|
|
235624
|
-
apiKey: fallbackKey,
|
|
235625
|
-
model: fallbackModel,
|
|
235626
|
-
timeoutMs: config3.GRAPH_EXTRACT_TIMEOUT_MS,
|
|
235627
|
-
reasoningEffort: config3.GRAPH_EXTRACT_REASONING_EFFORT
|
|
235628
|
-
});
|
|
235629
|
-
}
|
|
235630
|
-
const [primary, fallback] = providers;
|
|
235993
|
+
const providers = graphExtractProviders(options);
|
|
235994
|
+
const [primary, ...fallbacks] = providers;
|
|
235631
235995
|
if (!primary)
|
|
235632
235996
|
return null;
|
|
235633
235997
|
const result = await requestStructuredChat({
|
|
235634
235998
|
primary,
|
|
235635
|
-
|
|
235999
|
+
fallbacks,
|
|
235636
236000
|
messages: [
|
|
235637
236001
|
{ role: "system", content: SYSTEM_PROMPT },
|
|
235638
236002
|
{ role: "user", content: text4 }
|
|
@@ -235890,10 +236254,13 @@ async function requestKnowledgeSummary(document2) {
|
|
|
235890
236254
|
const primary = provider(config3.GRAPH_EXTRACT_BASE_URL, config3.GRAPH_EXTRACT_API_KEY, config3.GRAPH_EXTRACT_MODEL);
|
|
235891
236255
|
if (!primary)
|
|
235892
236256
|
return null;
|
|
235893
|
-
const
|
|
236257
|
+
const fallbacks = [
|
|
236258
|
+
provider(config3.GRAPH_EXTRACT_FALLBACK_BASE_URL, config3.GRAPH_EXTRACT_FALLBACK_API_KEY, config3.GRAPH_EXTRACT_FALLBACK_MODEL),
|
|
236259
|
+
provider(config3.GRAPH_EXTRACT_FALLBACK_2_BASE_URL, config3.GRAPH_EXTRACT_FALLBACK_2_API_KEY, config3.GRAPH_EXTRACT_FALLBACK_2_MODEL)
|
|
236260
|
+
].filter((item) => Boolean(item));
|
|
235894
236261
|
const result = await requestStructuredChat({
|
|
235895
236262
|
primary,
|
|
235896
|
-
|
|
236263
|
+
fallbacks,
|
|
235897
236264
|
messages: [
|
|
235898
236265
|
{ role: "system", content: systemPrompt },
|
|
235899
236266
|
{
|
|
@@ -236328,7 +236695,7 @@ function providerProfile(name) {
|
|
|
236328
236695
|
};
|
|
236329
236696
|
}
|
|
236330
236697
|
function configuredEmbeddingProvider() {
|
|
236331
|
-
const model = config3.EMBEDDING_MODEL ?? config3.EMBEDDING_FALLBACK_MODEL ?? "";
|
|
236698
|
+
const model = config3.EMBEDDING_MODEL ?? config3.EMBEDDING_FALLBACK_MODEL ?? config3.EMBEDDING_FALLBACK_2_MODEL ?? "";
|
|
236332
236699
|
return {
|
|
236333
236700
|
model,
|
|
236334
236701
|
profile: embeddingProfileId(model, EMBEDDING_DIMENSIONS, "v1"),
|
|
@@ -236339,6 +236706,10 @@ function configuredEmbeddingProvider() {
|
|
|
236339
236706
|
config3.EMBEDDING_FALLBACK_BASE_URL ?? "",
|
|
236340
236707
|
config3.EMBEDDING_FALLBACK_MODEL ?? ""
|
|
236341
236708
|
],
|
|
236709
|
+
fallback_2: [
|
|
236710
|
+
config3.EMBEDDING_FALLBACK_2_BASE_URL ?? "",
|
|
236711
|
+
config3.EMBEDDING_FALLBACK_2_MODEL ?? ""
|
|
236712
|
+
],
|
|
236342
236713
|
dimensions: EMBEDDING_DIMENSIONS,
|
|
236343
236714
|
profileVersion: "v1"
|
|
236344
236715
|
})
|
|
@@ -237571,7 +237942,7 @@ var swaggerConfig = {
|
|
|
237571
237942
|
},
|
|
237572
237943
|
info: {
|
|
237573
237944
|
title: "DocsMint API",
|
|
237574
|
-
version: "0.
|
|
237945
|
+
version: "0.8.1",
|
|
237575
237946
|
description: "Self-hosted AI-native knowledge workspace and installable PWA with hybrid search, GraphRAG, REST, SDK, CLI, and MCP access for people and AI agents.",
|
|
237576
237947
|
contact: { name: "HiAi-gg", url: "https://github.com/HiAi-gg/docsmint" },
|
|
237577
237948
|
license: {
|
|
@@ -237632,7 +238003,7 @@ app.listen({
|
|
|
237632
238003
|
idleTimeout: 30,
|
|
237633
238004
|
maxRequestBodySize: MAX_BODY_SIZE_BYTES
|
|
237634
238005
|
});
|
|
237635
|
-
logger3.info({ port: config3.API_PORT }, "
|
|
238006
|
+
logger3.info({ port: config3.API_PORT }, "DocsMint API started");
|
|
237636
238007
|
var stopDocsMintApi = async () => {
|
|
237637
238008
|
logger3.info("Shutting down...");
|
|
237638
238009
|
attachmentUploadCleanup.stop();
|
package/dist/client.d.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
/**
|
|
2
|
-
*
|
|
2
|
+
* DocsMint REST API client.
|
|
3
3
|
*
|
|
4
4
|
* Bun-native `fetch` wrapper. All authenticated requests send
|
|
5
5
|
* `Authorization: Bearer <apiKey>`. Non-OK responses throw
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
*/
|
|
10
10
|
import type { DocsApiKeyCreated, DocsApiKeyListResponse, DocsAttachment, DocsAttachmentConfirmInput, DocsAttachmentListResponse, DocsAttachmentPresignInput, DocsAttachmentPresignResponse, DocsCategory, DocsCategoryInput, DocsCategoryUpdate, DocsDocument, DocsDocumentCreateInput, DocsDocumentCursorPage, DocsDocumentIndexRefresh, DocsDocumentIndexStatus, DocsDocumentKnowledgeSummary, DocsDocumentListResponse, DocsDocumentPipeline, DocsDocumentUpdateInput, DocsFolder, DocsFolderCreateInput, DocsFolderUpdateInput, DocsGraphEntitiesResponse, DocsGraphRelatedResponse, DocsGraphSearchResponse, DocsHealthResponse, DocsRequestContext, DocsSearchOptions, DocsSearchResponse, DocsSearchSuggestItem, DocsSharedContent, DocsShareLink, DocsShareListResponse, DocsShareRole, DocsTag, DocsVersion, DocsVersionDiff } from "./types.js";
|
|
11
11
|
export interface DocsClientConfig {
|
|
12
|
-
/** Base URL of the
|
|
12
|
+
/** Base URL of the DocsMint API, e.g. `http://localhost:50700`. */
|
|
13
13
|
baseUrl: string;
|
|
14
14
|
/** Bearer token used as `Authorization: Bearer <apiKey>`. */
|
|
15
15
|
apiKey?: string;
|
package/dist/client.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
/**
|
|
2
|
-
*
|
|
2
|
+
* DocsMint REST API client.
|
|
3
3
|
*
|
|
4
4
|
* Bun-native `fetch` wrapper. All authenticated requests send
|
|
5
5
|
* `Authorization: Bearer <apiKey>`. Non-OK responses throw
|
|
@@ -18,7 +18,7 @@ export class DocsApiError extends Error {
|
|
|
18
18
|
url;
|
|
19
19
|
requestId;
|
|
20
20
|
constructor(status, body, message, metadata, code) {
|
|
21
|
-
super(message ?? `
|
|
21
|
+
super(message ?? `DocsMint API error ${status}`);
|
|
22
22
|
this.name = "DocsApiError";
|
|
23
23
|
this.status = status;
|
|
24
24
|
this.code = code ?? DocsApiError.codeFromBody(status, body);
|
|
@@ -65,7 +65,7 @@ export class DocsNetworkError extends Error {
|
|
|
65
65
|
export class DocsTimeoutError extends DocsNetworkError {
|
|
66
66
|
timeout;
|
|
67
67
|
constructor(timeout, options) {
|
|
68
|
-
super(`
|
|
68
|
+
super(`DocsMint request timed out after ${timeout}ms`, options);
|
|
69
69
|
this.name = "DocsTimeoutError";
|
|
70
70
|
this.timeout = timeout;
|
|
71
71
|
}
|
|
@@ -587,7 +587,7 @@ export class DocsClient {
|
|
|
587
587
|
"message" in body &&
|
|
588
588
|
typeof body.message === "string"
|
|
589
589
|
? body.message
|
|
590
|
-
: `
|
|
590
|
+
: `DocsMint API error ${res.status}`;
|
|
591
591
|
return new DocsApiError(res.status, body, message, {
|
|
592
592
|
url: res.url || undefined,
|
|
593
593
|
requestId: res.headers.get("x-request-id") ?? undefined,
|
|
@@ -595,12 +595,12 @@ export class DocsClient {
|
|
|
595
595
|
}
|
|
596
596
|
wrapNetworkError(err, requestId) {
|
|
597
597
|
if (err instanceof Error) {
|
|
598
|
-
return new DocsNetworkError(`
|
|
598
|
+
return new DocsNetworkError(`DocsMint network error: ${err.message}`, {
|
|
599
599
|
cause: err,
|
|
600
600
|
requestId,
|
|
601
601
|
});
|
|
602
602
|
}
|
|
603
|
-
return new DocsNetworkError(`
|
|
603
|
+
return new DocsNetworkError(`DocsMint network error: ${String(err)}`, {
|
|
604
604
|
requestId,
|
|
605
605
|
});
|
|
606
606
|
}
|
|
@@ -54571,6 +54571,19 @@ function date4(params) {
|
|
|
54571
54571
|
|
|
54572
54572
|
// ../../node_modules/.bun/zod@4.4.3/node_modules/zod/v4/classic/external.js
|
|
54573
54573
|
config(en_default());
|
|
54574
|
+
// ../../backend/src/lib/openrouter-public-matrix.ts
|
|
54575
|
+
var OPENROUTER_PUBLIC_BASE_URL = "https://openrouter.ai/api/v1";
|
|
54576
|
+
var OPENROUTER_PUBLIC_EXPANSION_MODELS = {
|
|
54577
|
+
primary: "poolside/laguna-xs-2.1",
|
|
54578
|
+
fallback: "openai/gpt-5.6-luna",
|
|
54579
|
+
fallback_2: "google/gemini-3.5-flash-lite"
|
|
54580
|
+
};
|
|
54581
|
+
var OPENROUTER_PUBLIC_RERANK_MODELS = {
|
|
54582
|
+
primary: "voyageai/rerank-2.5",
|
|
54583
|
+
fallback: "cohere/rerank-v3.5",
|
|
54584
|
+
fallback_2: "nvidia/llama-nemotron-rerank-vl-1b-v2:free"
|
|
54585
|
+
};
|
|
54586
|
+
|
|
54574
54587
|
// ../../backend/src/lib/config-schema.ts
|
|
54575
54588
|
var envSchema = exports_external.object({
|
|
54576
54589
|
DATABASE_URL: exports_external.string().default("postgresql://hiai_app:changeme@localhost:5437/hiai_docs"),
|
|
@@ -54627,6 +54640,9 @@ var envSchema = exports_external.object({
|
|
|
54627
54640
|
EMBEDDING_FALLBACK_BASE_URL: exports_external.string().optional(),
|
|
54628
54641
|
EMBEDDING_FALLBACK_API_KEY: exports_external.string().optional(),
|
|
54629
54642
|
EMBEDDING_FALLBACK_MODEL: exports_external.string().optional(),
|
|
54643
|
+
EMBEDDING_FALLBACK_2_BASE_URL: exports_external.string().optional(),
|
|
54644
|
+
EMBEDDING_FALLBACK_2_API_KEY: exports_external.string().optional(),
|
|
54645
|
+
EMBEDDING_FALLBACK_2_MODEL: exports_external.string().optional(),
|
|
54630
54646
|
API_PORT: exports_external.coerce.number().default(50700),
|
|
54631
54647
|
NODE_ENV: exports_external.enum(["development", "production", "test"]).default("development"),
|
|
54632
54648
|
LOG_LEVEL: exports_external.enum(["trace", "debug", "info", "warn", "error", "fatal"]).default("info"),
|
|
@@ -54653,14 +54669,34 @@ var envSchema = exports_external.object({
|
|
|
54653
54669
|
GRAPH_EXTRACT_FALLBACK_BASE_URL: exports_external.string().optional(),
|
|
54654
54670
|
GRAPH_EXTRACT_FALLBACK_API_KEY: exports_external.string().optional(),
|
|
54655
54671
|
GRAPH_EXTRACT_FALLBACK_MODEL: exports_external.string().optional(),
|
|
54672
|
+
GRAPH_EXTRACT_FALLBACK_2_BASE_URL: exports_external.string().optional(),
|
|
54673
|
+
GRAPH_EXTRACT_FALLBACK_2_API_KEY: exports_external.string().optional(),
|
|
54674
|
+
GRAPH_EXTRACT_FALLBACK_2_MODEL: exports_external.string().optional(),
|
|
54656
54675
|
GRAPH_EXTRACT_MIN_CONFIDENCE: exports_external.coerce.number().min(0).max(1).default(0.5),
|
|
54657
54676
|
SEARCH_EXPANSION_ENABLED: exports_external.string().optional().default("true").transform((v2) => v2 === "true"),
|
|
54658
|
-
SEARCH_EXPANSION_BASE_URL: exports_external.string().default(
|
|
54677
|
+
SEARCH_EXPANSION_BASE_URL: exports_external.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
54659
54678
|
SEARCH_EXPANSION_API_KEY: exports_external.string().optional(),
|
|
54660
|
-
SEARCH_EXPANSION_MODEL: exports_external.string().default(
|
|
54661
|
-
SEARCH_EXPANSION_FALLBACK_BASE_URL: exports_external.string().default(
|
|
54679
|
+
SEARCH_EXPANSION_MODEL: exports_external.string().default(OPENROUTER_PUBLIC_EXPANSION_MODELS.primary),
|
|
54680
|
+
SEARCH_EXPANSION_FALLBACK_BASE_URL: exports_external.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
54662
54681
|
SEARCH_EXPANSION_FALLBACK_API_KEY: exports_external.string().optional(),
|
|
54663
|
-
SEARCH_EXPANSION_FALLBACK_MODEL: exports_external.string().default(
|
|
54682
|
+
SEARCH_EXPANSION_FALLBACK_MODEL: exports_external.string().default(OPENROUTER_PUBLIC_EXPANSION_MODELS.fallback),
|
|
54683
|
+
SEARCH_EXPANSION_FALLBACK_2_BASE_URL: exports_external.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
54684
|
+
SEARCH_EXPANSION_FALLBACK_2_API_KEY: exports_external.string().optional(),
|
|
54685
|
+
SEARCH_EXPANSION_FALLBACK_2_MODEL: exports_external.string().default(OPENROUTER_PUBLIC_EXPANSION_MODELS.fallback_2),
|
|
54686
|
+
SEARCH_RERANK_ENABLED: exports_external.string().optional().default("true").transform((v2) => v2 === "true"),
|
|
54687
|
+
SEARCH_RERANK_BASE_URL: exports_external.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
54688
|
+
SEARCH_RERANK_API_KEY: exports_external.string().optional(),
|
|
54689
|
+
SEARCH_RERANK_MODEL: exports_external.string().default(OPENROUTER_PUBLIC_RERANK_MODELS.primary),
|
|
54690
|
+
SEARCH_RERANK_FALLBACK_BASE_URL: exports_external.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
54691
|
+
SEARCH_RERANK_FALLBACK_API_KEY: exports_external.string().optional(),
|
|
54692
|
+
SEARCH_RERANK_FALLBACK_MODEL: exports_external.string().default(OPENROUTER_PUBLIC_RERANK_MODELS.fallback),
|
|
54693
|
+
SEARCH_RERANK_FALLBACK_2_BASE_URL: exports_external.string().default(OPENROUTER_PUBLIC_BASE_URL),
|
|
54694
|
+
SEARCH_RERANK_FALLBACK_2_API_KEY: exports_external.string().optional(),
|
|
54695
|
+
SEARCH_RERANK_FALLBACK_2_MODEL: exports_external.string().default(OPENROUTER_PUBLIC_RERANK_MODELS.fallback_2),
|
|
54696
|
+
SEARCH_RERANK_TIMEOUT_MS: exports_external.coerce.number().int().min(250).max(30000).default(3000),
|
|
54697
|
+
SEARCH_RERANK_TOP_N: exports_external.coerce.number().int().min(1).max(100).default(20),
|
|
54698
|
+
SEARCH_RERANK_MAX_CHARS: exports_external.coerce.number().int().min(120).max(8000).default(1500),
|
|
54699
|
+
SEARCH_RERANK_GRAPH_POSITION: exports_external.enum(["before", "after"]).default("after"),
|
|
54664
54700
|
SEARCH_EXPANSION_TIMEOUT_MS: exports_external.coerce.number().int().min(1000).max(300000).default(6000),
|
|
54665
54701
|
SEARCH_VECTOR_PROVIDER_TIMEOUT_MS: exports_external.coerce.number().int().min(250).max(30000).default(2500),
|
|
54666
54702
|
SEARCH_EXPANSION_CACHE_TTL_SECONDS: exports_external.coerce.number().int().min(0).max(2592000).default(86400),
|
package/dist/mcp-cli.js
CHANGED
|
@@ -32,7 +32,7 @@ class DocsApiError extends Error {
|
|
|
32
32
|
url;
|
|
33
33
|
requestId;
|
|
34
34
|
constructor(status, body, message, metadata, code) {
|
|
35
|
-
super(message ?? `
|
|
35
|
+
super(message ?? `DocsMint API error ${status}`);
|
|
36
36
|
this.name = "DocsApiError";
|
|
37
37
|
this.status = status;
|
|
38
38
|
this.code = code ?? DocsApiError.codeFromBody(status, body);
|
|
@@ -64,7 +64,7 @@ class DocsNetworkError extends Error {
|
|
|
64
64
|
class DocsTimeoutError extends DocsNetworkError {
|
|
65
65
|
timeout;
|
|
66
66
|
constructor(timeout, options) {
|
|
67
|
-
super(`
|
|
67
|
+
super(`DocsMint request timed out after ${timeout}ms`, options);
|
|
68
68
|
this.name = "DocsTimeoutError";
|
|
69
69
|
this.timeout = timeout;
|
|
70
70
|
}
|
|
@@ -501,7 +501,7 @@ class DocsClient {
|
|
|
501
501
|
} catch {
|
|
502
502
|
body = null;
|
|
503
503
|
}
|
|
504
|
-
const message = body && typeof body === "object" && "error" in body && typeof body.error === "string" ? body.error : body && typeof body === "object" && ("message" in body) && typeof body.message === "string" ? body.message : `
|
|
504
|
+
const message = body && typeof body === "object" && "error" in body && typeof body.error === "string" ? body.error : body && typeof body === "object" && ("message" in body) && typeof body.message === "string" ? body.message : `DocsMint API error ${res.status}`;
|
|
505
505
|
return new DocsApiError(res.status, body, message, {
|
|
506
506
|
url: res.url || undefined,
|
|
507
507
|
requestId: res.headers.get("x-request-id") ?? undefined
|
|
@@ -509,12 +509,12 @@ class DocsClient {
|
|
|
509
509
|
}
|
|
510
510
|
wrapNetworkError(err, requestId) {
|
|
511
511
|
if (err instanceof Error) {
|
|
512
|
-
return new DocsNetworkError(`
|
|
512
|
+
return new DocsNetworkError(`DocsMint network error: ${err.message}`, {
|
|
513
513
|
cause: err,
|
|
514
514
|
requestId
|
|
515
515
|
});
|
|
516
516
|
}
|
|
517
|
-
return new DocsNetworkError(`
|
|
517
|
+
return new DocsNetworkError(`DocsMint network error: ${String(err)}`, {
|
|
518
518
|
requestId
|
|
519
519
|
});
|
|
520
520
|
}
|
|
@@ -1046,7 +1046,7 @@ function registerDocsmintMcpCapabilities(server, client2) {
|
|
|
1046
1046
|
registerExtendedCapabilities(server, client2, (handler11) => wrapHandler("extended", handler11));
|
|
1047
1047
|
}
|
|
1048
1048
|
function createDocsmintMcpServer(options = {}) {
|
|
1049
|
-
const server = new McpServer({ name: "docsmint", version: "0.
|
|
1049
|
+
const server = new McpServer({ name: "docsmint", version: "0.8.1" });
|
|
1050
1050
|
const client2 = options.docsClient ? createMcpDocsClient(options.docsClient, options.requestContext) : options.client ?? createMcpDocsClient(createDefaultDocsClient(), options.requestContext);
|
|
1051
1051
|
registerDocsmintMcpCapabilities(server, client2);
|
|
1052
1052
|
return server;
|
package/dist/mcp-server.js
CHANGED
|
@@ -28,7 +28,7 @@ class DocsApiError extends Error {
|
|
|
28
28
|
url;
|
|
29
29
|
requestId;
|
|
30
30
|
constructor(status, body, message, metadata, code) {
|
|
31
|
-
super(message ?? `
|
|
31
|
+
super(message ?? `DocsMint API error ${status}`);
|
|
32
32
|
this.name = "DocsApiError";
|
|
33
33
|
this.status = status;
|
|
34
34
|
this.code = code ?? DocsApiError.codeFromBody(status, body);
|
|
@@ -60,7 +60,7 @@ class DocsNetworkError extends Error {
|
|
|
60
60
|
class DocsTimeoutError extends DocsNetworkError {
|
|
61
61
|
timeout;
|
|
62
62
|
constructor(timeout, options) {
|
|
63
|
-
super(`
|
|
63
|
+
super(`DocsMint request timed out after ${timeout}ms`, options);
|
|
64
64
|
this.name = "DocsTimeoutError";
|
|
65
65
|
this.timeout = timeout;
|
|
66
66
|
}
|
|
@@ -497,7 +497,7 @@ class DocsClient {
|
|
|
497
497
|
} catch {
|
|
498
498
|
body = null;
|
|
499
499
|
}
|
|
500
|
-
const message = body && typeof body === "object" && "error" in body && typeof body.error === "string" ? body.error : body && typeof body === "object" && ("message" in body) && typeof body.message === "string" ? body.message : `
|
|
500
|
+
const message = body && typeof body === "object" && "error" in body && typeof body.error === "string" ? body.error : body && typeof body === "object" && ("message" in body) && typeof body.message === "string" ? body.message : `DocsMint API error ${res.status}`;
|
|
501
501
|
return new DocsApiError(res.status, body, message, {
|
|
502
502
|
url: res.url || undefined,
|
|
503
503
|
requestId: res.headers.get("x-request-id") ?? undefined
|
|
@@ -505,12 +505,12 @@ class DocsClient {
|
|
|
505
505
|
}
|
|
506
506
|
wrapNetworkError(err, requestId) {
|
|
507
507
|
if (err instanceof Error) {
|
|
508
|
-
return new DocsNetworkError(`
|
|
508
|
+
return new DocsNetworkError(`DocsMint network error: ${err.message}`, {
|
|
509
509
|
cause: err,
|
|
510
510
|
requestId
|
|
511
511
|
});
|
|
512
512
|
}
|
|
513
|
-
return new DocsNetworkError(`
|
|
513
|
+
return new DocsNetworkError(`DocsMint network error: ${String(err)}`, {
|
|
514
514
|
requestId
|
|
515
515
|
});
|
|
516
516
|
}
|
|
@@ -1042,7 +1042,7 @@ function registerDocsmintMcpCapabilities(server, client2) {
|
|
|
1042
1042
|
registerExtendedCapabilities(server, client2, (handler11) => wrapHandler("extended", handler11));
|
|
1043
1043
|
}
|
|
1044
1044
|
function createDocsmintMcpServer(options = {}) {
|
|
1045
|
-
const server = new McpServer({ name: "docsmint", version: "0.
|
|
1045
|
+
const server = new McpServer({ name: "docsmint", version: "0.8.1" });
|
|
1046
1046
|
const client2 = options.docsClient ? createMcpDocsClient(options.docsClient, options.requestContext) : options.client ?? createMcpDocsClient(createDefaultDocsClient(), options.requestContext);
|
|
1047
1047
|
registerDocsmintMcpCapabilities(server, client2);
|
|
1048
1048
|
return server;
|
package/package.json
CHANGED
|
@@ -25,7 +25,7 @@ import { registerSearch } from './commands/search.js';
|
|
|
25
25
|
import { registerSnapshot } from './commands/snapshot.js';
|
|
26
26
|
import { registerUpdate } from './commands/update.js';
|
|
27
27
|
|
|
28
|
-
const VERSION = '0.
|
|
28
|
+
const VERSION = '0.8.1';
|
|
29
29
|
|
|
30
30
|
const program = new Command();
|
|
31
31
|
program.name('docsmint').description('CLI for the DocsMint knowledge workspace').version(VERSION);
|
package/server.json
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"id": "1249550690",
|
|
11
11
|
"subfolder": "packages/mcp-server"
|
|
12
12
|
},
|
|
13
|
-
"version": "0.
|
|
13
|
+
"version": "0.8.1",
|
|
14
14
|
"icons": [
|
|
15
15
|
{
|
|
16
16
|
"src": "https://raw.githubusercontent.com/HiAi-gg/docsmint/main/frontend/static/logo-dark.png",
|
|
@@ -30,7 +30,7 @@
|
|
|
30
30
|
"registryType": "npm",
|
|
31
31
|
"registryBaseUrl": "https://registry.npmjs.org",
|
|
32
32
|
"identifier": "@hiai-gg/docsmint",
|
|
33
|
-
"version": "0.
|
|
33
|
+
"version": "0.8.1",
|
|
34
34
|
"runtimeHint": "npx",
|
|
35
35
|
"transport": {
|
|
36
36
|
"type": "stdio"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: docsmint-document-manager
|
|
3
|
-
description: Manage and research DocsMint documents through its scoped MCP tools, including categories, folders, hybrid search, GraphRAG, and index refresh.
|
|
3
|
+
description: Manage and research DocsMint documents through its scoped MCP tools, including categories, folders, hybrid search, GraphRAG, rerank, and index refresh.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# DocsMint Document Manager
|
|
@@ -30,4 +30,4 @@ Use this skill when an agent must create, organize, edit, or research documents
|
|
|
30
30
|
4. After an update, inspect `get_document_index_status`; use `refresh_document_index` only when indexing is stale or failed.
|
|
31
31
|
5. Use snapshots before a substantial rewrite when version history is important.
|
|
32
32
|
|
|
33
|
-
DocsMint owns chunking, embeddings, multilingual hybrid retrieval, entity extraction, and GraphRAG indexing. Agents must use the MCP tools and never write those persistence layers directly.
|
|
33
|
+
DocsMint owns chunking, embeddings, multilingual hybrid retrieval, cross-encoder rerank, entity extraction, and GraphRAG indexing. Agents must use the MCP tools and never write those persistence layers directly.
|