dsh-library 0.1.3 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.es.md +3 -1
- package/README.hi.md +3 -1
- package/README.md +4 -2
- package/README.pt.md +3 -1
- package/README.zh.md +4 -2
- package/lib/index.js +211 -17
- package/lib/types/config.d.ts +19 -1
- package/lib/types/config.d.ts.map +1 -1
- package/lib/types/embedding.d.ts +110 -0
- package/lib/types/embedding.d.ts.map +1 -1
- package/lib/types/index.d.ts +6 -5
- package/lib/types/index.d.ts.map +1 -1
- package/package.json +2 -1
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,18 @@ All notable changes to this project are documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.2.0] - 2026-08-26
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- Embedder provider seam with an optional local Ollama backend.
|
|
13
|
+
|
|
14
|
+
## [0.1.4] - 2026-08-23
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
|
|
18
|
+
- Declared `@deepseek-ai/dsh-commands` as a peer dependency: the `/library` command hard-injects the `commands` service, so its provider package must be declared alongside `@deepseek-ai/dsh-tools` and `@deepseek-ai/dsh-storage-domain` instead of being a dev-only dependency.
|
|
19
|
+
|
|
8
20
|
## [0.1.3] - 2026-08-22
|
|
9
21
|
|
|
10
22
|
### Changed
|
package/README.es.md
CHANGED
|
@@ -88,7 +88,9 @@ Todos los ajustes son campos `Config` de Schemastery (modificables desde cordis.
|
|
|
88
88
|
| `chunkOverlap` | `120` | Solapamiento entre ventanas; debe ser menor que `chunkSize` |
|
|
89
89
|
| `maxFileBytes` | `5242880` | Archivos mayores se rechazan en `library_add` |
|
|
90
90
|
| `embedding.dims` | `256` | Dimensionalidad del hash embedding (≥ 8) |
|
|
91
|
-
| `embedding.
|
|
91
|
+
| `embedding.provider` | `hash` | Backend del embedder: `hash` (integrado, cero descargas), `command` (subproceso externo, requiere `embedding.command`), `ollama` (Ollama local, probado y degradado a `hash` si es inalcanzable) |
|
|
92
|
+
| `embedding.command` | `''` | Comando de embedder externo opcional (argv separado por espacios, sin shell) sobre `ctx.subprocess`; configurarlo selecciona el backend `command` |
|
|
93
|
+
| `embedding.ollamaUrl` / `ollamaModel` | `http://127.0.0.1:11434` / `nomic-embed-text` | Endpoint y modelo de Ollama local para el backend `ollama` (cero nube) |
|
|
92
94
|
| `embedding.timeoutMs` / `graceMs` / `maxOutputBytes` / `maxBatchItems` | `30000` / `1000` / `1048576` / `64` | Presupuesto del subproceso del embedder |
|
|
93
95
|
| `search.topK` | `8` | Resultados devueltos tras el pipeline completo |
|
|
94
96
|
| `search.hybridWeight` | `0.6` | 0 = solo palabras clave, 1 = solo semántica |
|
package/README.hi.md
CHANGED
|
@@ -88,7 +88,9 @@ dsh --profile web --dump-config | grep -A2 'id: dsh-library'
|
|
|
88
88
|
| `chunkOverlap` | `120` | लगातार विंडो के बीच ओवरलैप; `chunkSize` से छोटा होना चाहिए |
|
|
89
89
|
| `maxFileBytes` | `5242880` | इससे बड़ी फ़ाइलें `library_add` पर अस्वीकार होती हैं |
|
|
90
90
|
| `embedding.dims` | `256` | हैश-एम्बेडिंग आयाम (≥ 8) |
|
|
91
|
-
| `embedding.
|
|
91
|
+
| `embedding.provider` | `hash` | एम्बेडर बैकएंड: `hash` (अंतर्निहित, शून्य डाउनलोड), `command` (बाहरी उपप्रक्रिया, `embedding.command` आवश्यक), `ollama` (स्थानीय Ollama, अनुपलब्ध होने पर `hash` में डिग्रेड) |
|
|
92
|
+
| `embedding.command` | `''` | वैकल्पिक बाहरी एम्बेडर कमांड (स्पेस-सेपरेटेड argv, कोई शेल नहीं) `ctx.subprocess` से; सेट करने पर `command` बैकएंड चयनित होता है |
|
|
93
|
+
| `embedding.ollamaUrl` / `ollamaModel` | `http://127.0.0.1:11434` / `nomic-embed-text` | `ollama` बैकएंड के लिए स्थानीय Ollama एंडपॉइंट व मॉडल (शून्य क्लाउड) |
|
|
92
94
|
| `embedding.timeoutMs` / `graceMs` / `maxOutputBytes` / `maxBatchItems` | `30000` / `1000` / `1048576` / `64` | एम्बेडर उपप्रक्रिया बजट |
|
|
93
95
|
| `search.topK` | `8` | पूरी पाइपलाइन के बाद लौटे परिणाम |
|
|
94
96
|
| `search.hybridWeight` | `0.6` | 0 = केवल कीवर्ड, 1 = केवल सिमेंटिक |
|
package/README.md
CHANGED
|
@@ -89,7 +89,9 @@ All tunables are Schemastery `Config` fields (changeable from cordis.yml). An id
|
|
|
89
89
|
| `chunkOverlap` | `120` | Overlap between consecutive windows; must be smaller than `chunkSize` |
|
|
90
90
|
| `maxFileBytes` | `5242880` | Files larger than this are rejected on `library_add` |
|
|
91
91
|
| `embedding.dims` | `256` | Hash-embedding dimensionality (≥ 8) |
|
|
92
|
-
| `embedding.
|
|
92
|
+
| `embedding.provider` | `hash` | Embedder backend: `hash` (built-in, zero downloads), `command` (external subprocess, requires `embedding.command`), or `ollama` (local Ollama, probed and degraded to `hash` when unreachable) |
|
|
93
|
+
| `embedding.command` | `''` | Optional external embedder command (space-separated argv, no shell) over `ctx.subprocess`; setting it selects the `command` provider |
|
|
94
|
+
| `embedding.ollamaUrl` / `ollamaModel` | `http://127.0.0.1:11434` / `nomic-embed-text` | Local Ollama endpoint + model for the `ollama` provider (zero cloud) |
|
|
93
95
|
| `embedding.timeoutMs` / `graceMs` / `maxOutputBytes` / `maxBatchItems` | `30000` / `1000` / `1048576` / `64` | Embedder subprocess budget |
|
|
94
96
|
| `search.topK` | `8` | Results returned after the full pipeline |
|
|
95
97
|
| `search.hybridWeight` | `0.6` | 0 = keyword-only, 1 = semantic-only |
|
|
@@ -129,7 +131,7 @@ All tunables are Schemastery `Config` fields (changeable from cordis.yml). An id
|
|
|
129
131
|
|
|
130
132
|
## Known limitations
|
|
131
133
|
|
|
132
|
-
- **Lexical-grade embeddings.** The built-in hash embedder scores surface similarity, not meaning; retrieval quality on paraphrases is lower than a real embedding model — configure `embedding.command` for stronger semantics.
|
|
134
|
+
- **Lexical-grade embeddings.** The built-in hash embedder scores surface similarity, not meaning; retrieval quality on paraphrases is lower than a real embedding model — configure `embedding.command` (any subprocess embedder) or `embedding.provider: ollama` (a local Ollama embedding model) for stronger semantics.
|
|
133
135
|
- **Local citation model.** `library_cite_check` validates against the search result page (the `[n]` numbering), not against free-form source names; the fuzzy score is a bounded token-sequence partial ratio.
|
|
134
136
|
- **No ingestion pipeline.** Documents must be imported by path (`md`/`txt`); PDF/docx extraction is out of scope for v0.1.0.
|
|
135
137
|
|
package/README.pt.md
CHANGED
|
@@ -88,7 +88,9 @@ Todos os ajustes são campos `Config` do Schemastery (alteráveis pelo cordis.ym
|
|
|
88
88
|
| `chunkOverlap` | `120` | Sobreposição entre janelas; deve ser menor que `chunkSize` |
|
|
89
89
|
| `maxFileBytes` | `5242880` | Arquivos maiores são rejeitados no `library_add` |
|
|
90
90
|
| `embedding.dims` | `256` | Dimensionalidade do hash embedding (≥ 8) |
|
|
91
|
-
| `embedding.
|
|
91
|
+
| `embedding.provider` | `hash` | Backend do embedder: `hash` (integrado, zero downloads), `command` (subprocesso externo, requer `embedding.command`), `ollama` (Ollama local, sondado e degradado para `hash` se inalcançável) |
|
|
92
|
+
| `embedding.command` | `''` | Comando de embedder externo opcional (argv separado por espaços, sem shell) via `ctx.subprocess`; configurá-lo seleciona o backend `command` |
|
|
93
|
+
| `embedding.ollamaUrl` / `ollamaModel` | `http://127.0.0.1:11434` / `nomic-embed-text` | Endpoint e modelo do Ollama local para o backend `ollama` (zero nuvem) |
|
|
92
94
|
| `embedding.timeoutMs` / `graceMs` / `maxOutputBytes` / `maxBatchItems` | `30000` / `1000` / `1048576` / `64` | Orçamento do subprocesso do embedder |
|
|
93
95
|
| `search.topK` | `8` | Resultados devolvidos após o pipeline completo |
|
|
94
96
|
| `search.hybridWeight` | `0.6` | 0 = só palavras-chave, 1 = só semântica |
|
package/README.zh.md
CHANGED
|
@@ -88,7 +88,9 @@ dsh --profile web --dump-config | grep -A2 'id: dsh-library'
|
|
|
88
88
|
| `chunkOverlap` | `120` | 相邻窗口重叠;必须小于 `chunkSize` |
|
|
89
89
|
| `maxFileBytes` | `5242880` | 超过该大小的文件在 `library_add` 被拒绝 |
|
|
90
90
|
| `embedding.dims` | `256` | 哈希嵌入维度(≥ 8) |
|
|
91
|
-
| `embedding.
|
|
91
|
+
| `embedding.provider` | `hash` | 嵌入后端:`hash`(内置,零下载)、`command`(外部子进程,需 `embedding.command`)、`ollama`(本地 Ollama,探测不可用时降级回 `hash`) |
|
|
92
|
+
| `embedding.command` | `''` | 可选外部嵌入命令(空格分隔 argv、无 shell)走 `ctx.subprocess`;设置后即选择 `command` 后端 |
|
|
93
|
+
| `embedding.ollamaUrl` / `ollamaModel` | `http://127.0.0.1:11434` / `nomic-embed-text` | `ollama` 后端的本地 Ollama 端点与模型(零云端) |
|
|
92
94
|
| `embedding.timeoutMs` / `graceMs` / `maxOutputBytes` / `maxBatchItems` | `30000` / `1000` / `1048576` / `64` | 嵌入子进程预算 |
|
|
93
95
|
| `search.topK` | `8` | 完整管线后返回的结果数 |
|
|
94
96
|
| `search.hybridWeight` | `0.6` | 0 = 仅关键词,1 = 仅语义 |
|
|
@@ -128,7 +130,7 @@ dsh --profile web --dump-config | grep -A2 'id: dsh-library'
|
|
|
128
130
|
|
|
129
131
|
## 已知限制
|
|
130
132
|
|
|
131
|
-
- **词法级嵌入。** 内置哈希嵌入打分的是表面相似而非语义;对改写表达的检索质量低于真实嵌入模型 —— 配置 `embedding.command`
|
|
133
|
+
- **词法级嵌入。** 内置哈希嵌入打分的是表面相似而非语义;对改写表达的检索质量低于真实嵌入模型 —— 配置 `embedding.command`(任意子进程嵌入)或 `embedding.provider: ollama`(本地 Ollama 嵌入模型)可获得更强语义。
|
|
132
134
|
- **本地引用模型。** `library_cite_check` 针对搜索结果页(`[n]` 编号)验证,不支持自由形式的来源名;模糊分数是有界的词序列部分匹配率。
|
|
133
135
|
- **无摄取管线。** 文档须按路径导入(`md`/`txt`);PDF/docx 抽取不在 v0.1.0 范围。
|
|
134
136
|
|
package/lib/index.js
CHANGED
|
@@ -4011,6 +4011,10 @@ const LIBRARY_NAME = /^[a-z0-9][a-z0-9-]{0,63}$/;
|
|
|
4011
4011
|
const DEFAULT_MAX_FILE_BYTES = 5242880;
|
|
4012
4012
|
/** Absolute safety bound on one stored chunk (chunkSize must stay below it). */
|
|
4013
4013
|
const MAX_CHUNK_CHARS = 4e3;
|
|
4014
|
+
/** Local Ollama base URL (zero cloud — only this localhost endpoint is ever contacted). */
|
|
4015
|
+
const DEFAULT_OLLAMA_URL = "http://127.0.0.1:11434";
|
|
4016
|
+
/** Default Ollama embedding model. */
|
|
4017
|
+
const DEFAULT_OLLAMA_MODEL = "nomic-embed-text";
|
|
4014
4018
|
/** Embedder subprocess budget. */
|
|
4015
4019
|
const DEFAULT_EMBEDDING_TIMEOUT_MS = 3e4;
|
|
4016
4020
|
const DEFAULT_EMBEDDING_GRACE_MS = 1e3;
|
|
@@ -4029,14 +4033,24 @@ const Config = z.object({
|
|
|
4029
4033
|
maxFileBytes: z.number().default(DEFAULT_MAX_FILE_BYTES),
|
|
4030
4034
|
embedding: z.object({
|
|
4031
4035
|
dims: z.number().default(256),
|
|
4036
|
+
provider: z.union([
|
|
4037
|
+
z.const("hash"),
|
|
4038
|
+
z.const("command"),
|
|
4039
|
+
z.const("ollama")
|
|
4040
|
+
]).default("hash"),
|
|
4032
4041
|
command: z.string(),
|
|
4042
|
+
ollamaUrl: z.string().default(DEFAULT_OLLAMA_URL),
|
|
4043
|
+
ollamaModel: z.string().default(DEFAULT_OLLAMA_MODEL),
|
|
4033
4044
|
timeoutMs: z.number().default(DEFAULT_EMBEDDING_TIMEOUT_MS),
|
|
4034
4045
|
graceMs: z.number().default(DEFAULT_EMBEDDING_GRACE_MS),
|
|
4035
4046
|
maxOutputBytes: z.number().default(DEFAULT_EMBEDDING_MAX_OUTPUT_BYTES),
|
|
4036
4047
|
maxBatchItems: z.number().default(64)
|
|
4037
4048
|
}).default({
|
|
4038
4049
|
dims: 256,
|
|
4050
|
+
provider: "hash",
|
|
4039
4051
|
command: "",
|
|
4052
|
+
ollamaUrl: DEFAULT_OLLAMA_URL,
|
|
4053
|
+
ollamaModel: DEFAULT_OLLAMA_MODEL,
|
|
4040
4054
|
timeoutMs: DEFAULT_EMBEDDING_TIMEOUT_MS,
|
|
4041
4055
|
graceMs: DEFAULT_EMBEDDING_GRACE_MS,
|
|
4042
4056
|
maxOutputBytes: DEFAULT_EMBEDDING_MAX_OUTPUT_BYTES,
|
|
@@ -4127,9 +4141,19 @@ function resolveConfig(config = {}) {
|
|
|
4127
4141
|
["embedding.maxBatchItems", embedding.maxBatchItems ?? 64]
|
|
4128
4142
|
]) assertPositiveInteger(name, value);
|
|
4129
4143
|
const rawCommand = embedding.command;
|
|
4130
|
-
const command = rawCommand === void 0 || rawCommand.trim().length === 0 ? void 0 : rawCommand;
|
|
4131
|
-
|
|
4132
|
-
|
|
4144
|
+
const command = rawCommand === void 0 || rawCommand.trim().length === 0 ? void 0 : rawCommand.trim();
|
|
4145
|
+
const rawProvider = embedding.provider;
|
|
4146
|
+
let provider;
|
|
4147
|
+
if (command !== void 0) provider = "command";
|
|
4148
|
+
else if (rawProvider === void 0 || rawProvider === "hash") provider = "hash";
|
|
4149
|
+
else if (rawProvider === "command") throw new TypeError("dsh-library: embedding.provider=command requires embedding.command to be set");
|
|
4150
|
+
else if (rawProvider === "ollama") provider = "ollama";
|
|
4151
|
+
else throw new TypeError(`dsh-library: embedding.provider must be hash|command|ollama, got ${JSON.stringify(rawProvider)}`);
|
|
4152
|
+
const ollamaUrl = (embedding.ollamaUrl ?? "http://127.0.0.1:11434").trim();
|
|
4153
|
+
const ollamaModel = (embedding.ollamaModel ?? "nomic-embed-text").trim();
|
|
4154
|
+
if (provider === "ollama") {
|
|
4155
|
+
if (ollamaUrl.length === 0) throw new TypeError("dsh-library: embedding.ollamaUrl must be a non-empty URL when provider=ollama");
|
|
4156
|
+
if (ollamaModel.length === 0) throw new TypeError("dsh-library: embedding.ollamaModel must be a non-empty model name when provider=ollama");
|
|
4133
4157
|
}
|
|
4134
4158
|
const search = config.search ?? {};
|
|
4135
4159
|
const topK = search.topK ?? 8;
|
|
@@ -4171,7 +4195,10 @@ function resolveConfig(config = {}) {
|
|
|
4171
4195
|
maxFileBytes: config.maxFileBytes ?? 5242880,
|
|
4172
4196
|
embedding: {
|
|
4173
4197
|
dims,
|
|
4198
|
+
provider,
|
|
4174
4199
|
command: command === void 0 ? void 0 : command.trim(),
|
|
4200
|
+
ollamaUrl,
|
|
4201
|
+
ollamaModel,
|
|
4175
4202
|
timeoutMs: embedding.timeoutMs ?? 3e4,
|
|
4176
4203
|
graceMs: embedding.graceMs ?? 1e3,
|
|
4177
4204
|
maxOutputBytes: embedding.maxOutputBytes ?? 1048576,
|
|
@@ -4418,6 +4445,177 @@ async function embedWithCommand(subprocess, argv, cwd, texts, dims, caps) {
|
|
|
4418
4445
|
function splitCommandLine(command) {
|
|
4419
4446
|
return command.trim().split(/\s+/u).filter((part) => part.length > 0);
|
|
4420
4447
|
}
|
|
4448
|
+
/** Built-in deterministic hash embedder: zero downloads, pure local hashing. */
|
|
4449
|
+
var HashEmbedder = class {
|
|
4450
|
+
dims;
|
|
4451
|
+
name = "hash";
|
|
4452
|
+
/**
|
|
4453
|
+
* @param dims - vector dimensionality (≥ 1).
|
|
4454
|
+
*/
|
|
4455
|
+
constructor(dims) {
|
|
4456
|
+
this.dims = dims;
|
|
4457
|
+
}
|
|
4458
|
+
async embed(texts) {
|
|
4459
|
+
return texts.map((text) => embedHash(text, this.dims));
|
|
4460
|
+
}
|
|
4461
|
+
};
|
|
4462
|
+
/** External command embedder over the JSON-lines subprocess protocol. */
|
|
4463
|
+
var CommandEmbedder = class {
|
|
4464
|
+
subprocess;
|
|
4465
|
+
argv;
|
|
4466
|
+
cwd;
|
|
4467
|
+
dims;
|
|
4468
|
+
caps;
|
|
4469
|
+
maxBatchItems;
|
|
4470
|
+
name = "command";
|
|
4471
|
+
/**
|
|
4472
|
+
* @param subprocess - the mounted subprocess seam.
|
|
4473
|
+
* @param argv - the command line split into argv (argv[0] = program).
|
|
4474
|
+
* @param cwd - working directory for the child.
|
|
4475
|
+
* @param dims - expected vector dimensionality.
|
|
4476
|
+
* @param caps - timeout, grace, and output caps.
|
|
4477
|
+
* @param maxBatchItems - texts per subprocess invocation; larger batches split.
|
|
4478
|
+
*/
|
|
4479
|
+
constructor(subprocess, argv, cwd, dims, caps, maxBatchItems) {
|
|
4480
|
+
this.subprocess = subprocess;
|
|
4481
|
+
this.argv = argv;
|
|
4482
|
+
this.cwd = cwd;
|
|
4483
|
+
this.dims = dims;
|
|
4484
|
+
this.caps = caps;
|
|
4485
|
+
this.maxBatchItems = maxBatchItems;
|
|
4486
|
+
}
|
|
4487
|
+
async embed(texts) {
|
|
4488
|
+
const out = [];
|
|
4489
|
+
for (let index = 0; index < texts.length; index += this.maxBatchItems) out.push(...await embedWithCommand(this.subprocess, this.argv, this.cwd, texts.slice(index, index + this.maxBatchItems), this.dims, this.caps));
|
|
4490
|
+
return out;
|
|
4491
|
+
}
|
|
4492
|
+
};
|
|
4493
|
+
/**
|
|
4494
|
+
* Local Ollama embedder over the `/api/embed` endpoint. Zero cloud: the model
|
|
4495
|
+
* runs on a localhost Ollama server. A non-2xx response or a malformed vector
|
|
4496
|
+
* fails the batch (misconfiguration must surface, not silently degrade
|
|
4497
|
+
* retrieval); availability is probed up front by {@link probeOllama}.
|
|
4498
|
+
*/
|
|
4499
|
+
var OllamaEmbedder = class {
|
|
4500
|
+
url;
|
|
4501
|
+
model;
|
|
4502
|
+
dims;
|
|
4503
|
+
timeoutMs;
|
|
4504
|
+
maxBatchItems;
|
|
4505
|
+
name = "ollama";
|
|
4506
|
+
/**
|
|
4507
|
+
* @param url - the Ollama base URL (e.g. `http://127.0.0.1:11434`).
|
|
4508
|
+
* @param model - the embedding model name (e.g. `nomic-embed-text`).
|
|
4509
|
+
* @param dims - the probed embedding dimensionality.
|
|
4510
|
+
* @param timeoutMs - per-request timeout.
|
|
4511
|
+
* @param maxBatchItems - texts per `/api/embed` call; larger batches split.
|
|
4512
|
+
*/
|
|
4513
|
+
constructor(url, model, dims, timeoutMs, maxBatchItems) {
|
|
4514
|
+
this.url = url;
|
|
4515
|
+
this.model = model;
|
|
4516
|
+
this.dims = dims;
|
|
4517
|
+
this.timeoutMs = timeoutMs;
|
|
4518
|
+
this.maxBatchItems = maxBatchItems;
|
|
4519
|
+
}
|
|
4520
|
+
async embed(texts) {
|
|
4521
|
+
const out = [];
|
|
4522
|
+
for (let index = 0; index < texts.length; index += this.maxBatchItems) out.push(...await this.embedBatch(texts.slice(index, index + this.maxBatchItems)));
|
|
4523
|
+
return out;
|
|
4524
|
+
}
|
|
4525
|
+
/** POST one batch to `/api/embed` and validate the returned vectors. */
|
|
4526
|
+
async embedBatch(texts) {
|
|
4527
|
+
let response;
|
|
4528
|
+
try {
|
|
4529
|
+
response = await fetch(`${this.url}/api/embed`, {
|
|
4530
|
+
method: "POST",
|
|
4531
|
+
headers: { "content-type": "application/json" },
|
|
4532
|
+
body: JSON.stringify({
|
|
4533
|
+
model: this.model,
|
|
4534
|
+
input: [...texts]
|
|
4535
|
+
}),
|
|
4536
|
+
signal: AbortSignal.timeout(this.timeoutMs)
|
|
4537
|
+
});
|
|
4538
|
+
} catch (error) {
|
|
4539
|
+
throw new Error(`dsh-library: ollama embed request failed: ${error instanceof Error ? error.message : String(error)}`);
|
|
4540
|
+
}
|
|
4541
|
+
if (!response.ok) throw new Error(`dsh-library: ollama embed failed (HTTP ${response.status})`);
|
|
4542
|
+
const embeddings = (await response.json())["embeddings"];
|
|
4543
|
+
if (!Array.isArray(embeddings) || embeddings.length !== texts.length) throw new Error(`dsh-library: ollama answered ${Array.isArray(embeddings) ? embeddings.length : 0} of ${texts.length} inputs`);
|
|
4544
|
+
return embeddings.map((vector) => {
|
|
4545
|
+
if (!Array.isArray(vector) || vector.length !== this.dims || vector.some((value) => typeof value !== "number" || !Number.isFinite(value))) throw new Error("dsh-library: ollama returned a malformed embedding vector");
|
|
4546
|
+
return normalizeVector(Float64Array.from(vector));
|
|
4547
|
+
});
|
|
4548
|
+
}
|
|
4549
|
+
};
|
|
4550
|
+
/**
|
|
4551
|
+
* Probe a local Ollama server for one embedding model. Sends a single-word
|
|
4552
|
+
* probe through `/api/embed` and returns the model's vector dimensionality, or
|
|
4553
|
+
* undefined when the server is unreachable, the model is unknown, or the
|
|
4554
|
+
* response is malformed. Zero cloud — only the configured local URL is ever
|
|
4555
|
+
* contacted.
|
|
4556
|
+
* @param url - the Ollama base URL.
|
|
4557
|
+
* @param model - the embedding model name.
|
|
4558
|
+
* @param timeoutMs - probe timeout.
|
|
4559
|
+
* @returns the model's dimensionality, or undefined when unavailable.
|
|
4560
|
+
*/
|
|
4561
|
+
async function probeOllama(url, model, timeoutMs) {
|
|
4562
|
+
try {
|
|
4563
|
+
const response = await fetch(`${url}/api/embed`, {
|
|
4564
|
+
method: "POST",
|
|
4565
|
+
headers: { "content-type": "application/json" },
|
|
4566
|
+
body: JSON.stringify({
|
|
4567
|
+
model,
|
|
4568
|
+
input: ["probe"]
|
|
4569
|
+
}),
|
|
4570
|
+
signal: AbortSignal.timeout(timeoutMs)
|
|
4571
|
+
});
|
|
4572
|
+
if (!response.ok) return void 0;
|
|
4573
|
+
const embeddings = (await response.json())["embeddings"];
|
|
4574
|
+
if (!Array.isArray(embeddings) || embeddings.length !== 1 || !Array.isArray(embeddings[0])) return void 0;
|
|
4575
|
+
return embeddings[0].length;
|
|
4576
|
+
} catch {
|
|
4577
|
+
return;
|
|
4578
|
+
}
|
|
4579
|
+
}
|
|
4580
|
+
/**
|
|
4581
|
+
* Resolve the embedder backend from the resolved config: the configured
|
|
4582
|
+
* command wins (fail loud without a subprocess seam), the `ollama` provider is
|
|
4583
|
+
* probed and degrades gracefully to the built-in hash embedder when the server
|
|
4584
|
+
* is unreachable, and the built-in hash embedder is the zero-download default.
|
|
4585
|
+
* @param config - the resolved embedding config.
|
|
4586
|
+
* @param subprocess - the optional subprocess seam (command provider only).
|
|
4587
|
+
* @param cwd - working directory for the command provider.
|
|
4588
|
+
* @returns the backend plus its degradation note.
|
|
4589
|
+
*/
|
|
4590
|
+
async function resolveEmbedder(config, subprocess, cwd) {
|
|
4591
|
+
if (config.provider === "command") {
|
|
4592
|
+
if (subprocess === void 0) throw new Error("dsh-library: embedding.provider=command but ctx.subprocess is not mounted — the external embedder cannot run");
|
|
4593
|
+
return {
|
|
4594
|
+
embedder: new CommandEmbedder(subprocess, splitCommandLine(config.command), cwd, config.dims, {
|
|
4595
|
+
timeoutMs: config.timeoutMs,
|
|
4596
|
+
graceMs: config.graceMs,
|
|
4597
|
+
maxOutputBytes: config.maxOutputBytes
|
|
4598
|
+
}, config.maxBatchItems),
|
|
4599
|
+
degraded: false
|
|
4600
|
+
};
|
|
4601
|
+
}
|
|
4602
|
+
if (config.provider === "ollama") {
|
|
4603
|
+
const dims = await probeOllama(config.ollamaUrl, config.ollamaModel, config.timeoutMs);
|
|
4604
|
+
if (dims === void 0) return {
|
|
4605
|
+
embedder: new HashEmbedder(config.dims),
|
|
4606
|
+
degraded: true,
|
|
4607
|
+
reason: `ollama unavailable at ${config.ollamaUrl} (model ${config.ollamaModel}); degraded to the built-in hash embedder`
|
|
4608
|
+
};
|
|
4609
|
+
return {
|
|
4610
|
+
embedder: new OllamaEmbedder(config.ollamaUrl, config.ollamaModel, dims, config.timeoutMs, config.maxBatchItems),
|
|
4611
|
+
degraded: false
|
|
4612
|
+
};
|
|
4613
|
+
}
|
|
4614
|
+
return {
|
|
4615
|
+
embedder: new HashEmbedder(config.dims),
|
|
4616
|
+
degraded: false
|
|
4617
|
+
};
|
|
4618
|
+
}
|
|
4421
4619
|
//#endregion
|
|
4422
4620
|
//#region src/ids.ts
|
|
4423
4621
|
/**
|
|
@@ -5310,16 +5508,9 @@ var LibraryStore = class {
|
|
|
5310
5508
|
this.chunks = domain.table("chunks");
|
|
5311
5509
|
this.purges = domain.table("purges");
|
|
5312
5510
|
}
|
|
5313
|
-
/** Embed a text batch with the
|
|
5511
|
+
/** Embed a text batch with the resolved embedder backend. */
|
|
5314
5512
|
async embed(texts) {
|
|
5315
|
-
|
|
5316
|
-
if (command === void 0) return texts.map((text) => embedHash(text, this.config.embedding.dims));
|
|
5317
|
-
if (this.deps.subprocess === void 0) throw new Error("embedding.command is configured but ctx.subprocess is not mounted — the external embedder cannot run");
|
|
5318
|
-
return embedWithCommand(this.deps.subprocess, splitCommandLine(command), process.cwd(), texts, this.config.embedding.dims, {
|
|
5319
|
-
timeoutMs: this.config.embedding.timeoutMs,
|
|
5320
|
-
graceMs: this.config.embedding.graceMs,
|
|
5321
|
-
maxOutputBytes: this.config.embedding.maxOutputBytes
|
|
5322
|
-
});
|
|
5513
|
+
return this.deps.embedder.embed(texts);
|
|
5323
5514
|
}
|
|
5324
5515
|
/**
|
|
5325
5516
|
* Import one document: chunk it, embed the chunks, and store the records.
|
|
@@ -5551,9 +5742,12 @@ var LibraryStore = class {
|
|
|
5551
5742
|
};
|
|
5552
5743
|
}
|
|
5553
5744
|
};
|
|
5554
|
-
/**
|
|
5555
|
-
function storeDepsOf(ctx) {
|
|
5556
|
-
|
|
5745
|
+
/** Resolve the embedder backend at mount; a degraded Ollama falls back to hash with a logged reason. */
|
|
5746
|
+
async function storeDepsOf(ctx, config) {
|
|
5747
|
+
const subprocess = ctx.get("subprocess");
|
|
5748
|
+
const resolution = await resolveEmbedder(config.embedding, subprocess, process.cwd());
|
|
5749
|
+
if (resolution.degraded) ctx.logger("dsh-library").warn(`embedder: ${resolution.reason}`);
|
|
5750
|
+
return { embedder: resolution.embedder };
|
|
5557
5751
|
}
|
|
5558
5752
|
/** Append one audit event; a failed append never changes the outcome. */
|
|
5559
5753
|
function audit(exec, type, event) {
|
|
@@ -5574,7 +5768,7 @@ async function apply(ctx, config = {}) {
|
|
|
5574
5768
|
const resolved = resolveConfig(config);
|
|
5575
5769
|
const domain = await ctx.storageDomain.open(libraryDomainSpec);
|
|
5576
5770
|
ctx.effect(() => () => domain.close(), "dsh-library: storage domain");
|
|
5577
|
-
const store = new LibraryStore(domain, resolved, storeDepsOf(ctx));
|
|
5771
|
+
const store = new LibraryStore(domain, resolved, await storeDepsOf(ctx, resolved));
|
|
5578
5772
|
const services = {
|
|
5579
5773
|
ctx,
|
|
5580
5774
|
config: resolved,
|
|
@@ -6071,4 +6265,4 @@ function libraryDiagnoseTool(services) {
|
|
|
6071
6265
|
});
|
|
6072
6266
|
}
|
|
6073
6267
|
//#endregion
|
|
6074
|
-
export { CHUNK_SIZE_BUCKETS, Config, LIBRARY_NAME, LibraryStore, PROBE_PREFIX, allTools, apply, avoidLostMiddle, buildProbePrompt, checkDiversity, chunkHash, chunkSizeHistogram, chunkText, cosine, embedHash, extractCitationNumbers, extractCitations, extractContext, extractSentenceWithCitation, filterDocuments, findDuplicateChunks, formatFewShotPrompt, fuzzyPartialRatio, inject, insertProbe, libraryDomainSpec, makeProbe, maximalMarginalRelevance, middlePenalty, name, positionBins, resolveConfig, sampleSignatures, scoreBatch, scoreRelevance, selectFewShot, splitCommandLine, successByPosition, validateCitation, validateCitations, validateQaPair, verifyPurge, verifyReferences };
|
|
6268
|
+
export { CHUNK_SIZE_BUCKETS, CommandEmbedder, Config, HashEmbedder, LIBRARY_NAME, LibraryStore, OllamaEmbedder, PROBE_PREFIX, allTools, apply, avoidLostMiddle, buildProbePrompt, checkDiversity, chunkHash, chunkSizeHistogram, chunkText, cosine, embedHash, extractCitationNumbers, extractCitations, extractContext, extractSentenceWithCitation, filterDocuments, findDuplicateChunks, formatFewShotPrompt, fuzzyPartialRatio, inject, insertProbe, libraryDomainSpec, makeProbe, maximalMarginalRelevance, middlePenalty, name, positionBins, probeOllama, resolveConfig, resolveEmbedder, sampleSignatures, scoreBatch, scoreRelevance, selectFewShot, splitCommandLine, successByPosition, validateCitation, validateCitations, validateQaPair, verifyPurge, verifyReferences };
|
package/lib/types/config.d.ts
CHANGED
|
@@ -17,6 +17,10 @@ export declare const DEFAULT_MAX_FILE_BYTES: number;
|
|
|
17
17
|
export declare const MAX_CHUNK_CHARS = 4000;
|
|
18
18
|
/** Hash-embedding dimensionality used by the zero-download local embedder. */
|
|
19
19
|
export declare const DEFAULT_EMBEDDING_DIMS = 256;
|
|
20
|
+
/** Local Ollama base URL (zero cloud — only this localhost endpoint is ever contacted). */
|
|
21
|
+
export declare const DEFAULT_OLLAMA_URL = "http://127.0.0.1:11434";
|
|
22
|
+
/** Default Ollama embedding model. */
|
|
23
|
+
export declare const DEFAULT_OLLAMA_MODEL = "nomic-embed-text";
|
|
20
24
|
/** Embedder subprocess budget. */
|
|
21
25
|
export declare const DEFAULT_EMBEDDING_TIMEOUT_MS = 30000;
|
|
22
26
|
export declare const DEFAULT_EMBEDDING_GRACE_MS = 1000;
|
|
@@ -44,10 +48,17 @@ export declare const DEFAULT_PURGE_MAX_PROBES = 24;
|
|
|
44
48
|
export declare const DEFAULT_DIAGNOSE_MAX_DUPLICATE_PAIRS = 24;
|
|
45
49
|
export declare const DEFAULT_DIAGNOSE_SAMPLE_CAP = 200;
|
|
46
50
|
export declare const DEFAULT_DIAGNOSE_POSITION_BINS = 5;
|
|
47
|
-
/** Embedder selection: built-in hash embedding,
|
|
51
|
+
/** Embedder selection: built-in hash embedding, an external command, or a local Ollama server. */
|
|
48
52
|
export interface EmbeddingConfig {
|
|
49
53
|
/** Hash-embedding dimensionality (built-in embedder only). Must be ≥ 8. */
|
|
50
54
|
dims?: number;
|
|
55
|
+
/**
|
|
56
|
+
* Which embedder backend to use: `hash` (built-in, zero downloads), `command`
|
|
57
|
+
* (the external subprocess protocol, requires `command`), or `ollama` (a
|
|
58
|
+
* local Ollama server, probed and degraded to `hash` when unavailable).
|
|
59
|
+
* When `command` is set, the command provider wins regardless of this field.
|
|
60
|
+
*/
|
|
61
|
+
provider?: 'hash' | 'command' | 'ollama';
|
|
51
62
|
/**
|
|
52
63
|
* Optional external embedder command line (space-separated, no shell
|
|
53
64
|
* interpretation; executed through `ctx.subprocess`). The command must read
|
|
@@ -58,6 +69,10 @@ export interface EmbeddingConfig {
|
|
|
58
69
|
* embedder runs with zero downloads.
|
|
59
70
|
*/
|
|
60
71
|
command?: string;
|
|
72
|
+
/** Local Ollama base URL for the `ollama` provider (default `http://127.0.0.1:11434`). */
|
|
73
|
+
ollamaUrl?: string;
|
|
74
|
+
/** Ollama embedding model name (default `nomic-embed-text`). */
|
|
75
|
+
ollamaModel?: string;
|
|
61
76
|
/** Cooperative timeout for one embedder invocation (ms). */
|
|
62
77
|
timeoutMs?: number;
|
|
63
78
|
/** Terminate-escalation grace handed to the subprocess seam (ms). */
|
|
@@ -143,7 +158,10 @@ export interface Config {
|
|
|
143
158
|
/** Resolved embedding config: defaults applied, `command` explicitly optional. */
|
|
144
159
|
export interface ResolvedEmbeddingConfig {
|
|
145
160
|
readonly dims: number;
|
|
161
|
+
readonly provider: 'hash' | 'command' | 'ollama';
|
|
146
162
|
readonly command: string | undefined;
|
|
163
|
+
readonly ollamaUrl: string;
|
|
164
|
+
readonly ollamaModel: string;
|
|
147
165
|
readonly timeoutMs: number;
|
|
148
166
|
readonly graceMs: number;
|
|
149
167
|
readonly maxOutputBytes: number;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../../src/config.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH,OAAO,CAAC,MAAM,0BAA0B,CAAA;AAExC,0EAA0E;AAC1E,eAAO,MAAM,YAAY,QAA8B,CAAA;AAEvD,iEAAiE;AACjE,eAAO,MAAM,kBAAkB,MAAM,CAAA;AACrC,eAAO,MAAM,qBAAqB,MAAM,CAAA;AACxC,yFAAyF;AACzF,eAAO,MAAM,sBAAsB,QAAkB,CAAA;AACrD,gFAAgF;AAChF,eAAO,MAAM,eAAe,OAAO,CAAA;AACnC,8EAA8E;AAC9E,eAAO,MAAM,sBAAsB,MAAM,CAAA;AACzC,kCAAkC;AAClC,eAAO,MAAM,4BAA4B,QAAS,CAAA;AAClD,eAAO,MAAM,0BAA0B,OAAO,CAAA;AAC9C,eAAO,MAAM,kCAAkC,QAAc,CAAA;AAC7D,eAAO,MAAM,2BAA2B,KAAK,CAAA;AAC7C,uBAAuB;AACvB,eAAO,MAAM,mBAAmB,IAAI,CAAA;AACpC,eAAO,MAAM,4BAA4B,MAAM,CAAA;AAC/C,eAAO,MAAM,4BAA4B,OAAO,CAAA;AAChD,eAAO,MAAM,+BAA+B,MAAM,CAAA;AAClD,eAAO,MAAM,+BAA+B,QAAS,CAAA;AACrD,+EAA+E;AAC/E,eAAO,MAAM,wBAAwB,IAAI,CAAA;AACzC,eAAO,MAAM,wBAAwB,IAAI,CAAA;AACzC,0BAA0B;AAC1B,eAAO,MAAM,wBAAwB,QAAS,CAAA;AAC9C,yBAAyB;AACzB,eAAO,MAAM,6BAA6B,MAAM,CAAA;AAChD,eAAO,MAAM,0BAA0B,KAAK,CAAA;AAC5C,eAAO,MAAM,6BAA6B,MAAM,CAAA;AAChD,mCAAmC;AACnC,eAAO,MAAM,8BAA8B,IAAI,CAAA;AAC/C,eAAO,MAAM,wBAAwB,KAAK,CAAA;AAC1C,yBAAyB;AACzB,eAAO,MAAM,oCAAoC,KAAK,CAAA;AACtD,eAAO,MAAM,2BAA2B,MAAM,CAAA;AAC9C,eAAO,MAAM,8BAA8B,IAAI,CAAA;AAE/C,
|
|
1
|
+
{"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../../src/config.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH,OAAO,CAAC,MAAM,0BAA0B,CAAA;AAExC,0EAA0E;AAC1E,eAAO,MAAM,YAAY,QAA8B,CAAA;AAEvD,iEAAiE;AACjE,eAAO,MAAM,kBAAkB,MAAM,CAAA;AACrC,eAAO,MAAM,qBAAqB,MAAM,CAAA;AACxC,yFAAyF;AACzF,eAAO,MAAM,sBAAsB,QAAkB,CAAA;AACrD,gFAAgF;AAChF,eAAO,MAAM,eAAe,OAAO,CAAA;AACnC,8EAA8E;AAC9E,eAAO,MAAM,sBAAsB,MAAM,CAAA;AACzC,2FAA2F;AAC3F,eAAO,MAAM,kBAAkB,2BAA2B,CAAA;AAC1D,sCAAsC;AACtC,eAAO,MAAM,oBAAoB,qBAAqB,CAAA;AACtD,kCAAkC;AAClC,eAAO,MAAM,4BAA4B,QAAS,CAAA;AAClD,eAAO,MAAM,0BAA0B,OAAO,CAAA;AAC9C,eAAO,MAAM,kCAAkC,QAAc,CAAA;AAC7D,eAAO,MAAM,2BAA2B,KAAK,CAAA;AAC7C,uBAAuB;AACvB,eAAO,MAAM,mBAAmB,IAAI,CAAA;AACpC,eAAO,MAAM,4BAA4B,MAAM,CAAA;AAC/C,eAAO,MAAM,4BAA4B,OAAO,CAAA;AAChD,eAAO,MAAM,+BAA+B,MAAM,CAAA;AAClD,eAAO,MAAM,+BAA+B,QAAS,CAAA;AACrD,+EAA+E;AAC/E,eAAO,MAAM,wBAAwB,IAAI,CAAA;AACzC,eAAO,MAAM,wBAAwB,IAAI,CAAA;AACzC,0BAA0B;AAC1B,eAAO,MAAM,wBAAwB,QAAS,CAAA;AAC9C,yBAAyB;AACzB,eAAO,MAAM,6BAA6B,MAAM,CAAA;AAChD,eAAO,MAAM,0BAA0B,KAAK,CAAA;AAC5C,eAAO,MAAM,6BAA6B,MAAM,CAAA;AAChD,mCAAmC;AACnC,eAAO,MAAM,8BAA8B,IAAI,CAAA;AAC/C,eAAO,MAAM,wBAAwB,KAAK,CAAA;AAC1C,yBAAyB;AACzB,eAAO,MAAM,oCAAoC,KAAK,CAAA;AACtD,eAAO,MAAM,2BAA2B,MAAM,CAAA;AAC9C,eAAO,MAAM,8BAA8B,IAAI,CAAA;AAE/C,kGAAkG;AAClG,MAAM,WAAW,eAAe;IAC9B,2EAA2E;IAC3E,IAAI,CAAC,EAAE,MAAM,CAAA;IACb;;;;;OAKG;IACH,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,GAAG,QAAQ,CAAA;IACxC;;;;;;;;OAQG;IACH,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,0FAA0F;IAC1F,SAAS,CAAC,EAAE,MAAM,CAAA;IAClB,gEAAgE;IAChE,WAAW,CAAC,EAAE,MAAM,CAAA;IACpB,4DAA4D;IAC5D,SAAS,CAAC,EAAE,MAAM,CAAA;IAClB,qEAAqE;IACrE,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,+DAA+D;IAC/D,cAAc,CAAC,EAAE,MAAM,CAAA;IACvB,8EAA8E;IAC9E,aAAa,CAAC,EAAE,MAAM,CAAA;CACvB;AAED,0GAA0G;AAC1G,MAAM,WAAW,YAAY;IAC3B,kEAAkE;IAClE,IAAI,CAAC,EAAE,MAAM,CAAA;IACb;;;OAGG;IACH,YAAY,CAAC,EAAE,MAAM,CAAA;IACrB,6GAA6G;IAC7G,YAAY,CAAC,EAAE,MAAM,CAAA;IACrB,8EAA8E;IAC9E,eAAe,CAAC,EAAE,MAAM,CAAA;IACxB,yFAAyF;IACzF,cAAc,CAAC,EAAE,MAAM,CAAA;IACvB,0DAA0D;IAC1D,cAAc,CAAC,EAAE,MAAM,CAAA;IACvB,mGAAmG;IACnG,cAAc,CAAC,EAAE,MAAM,CAAA;CACxB;AAED,+DAA+D;AAC/D,MAAM,WAAW,eAAe;IAC9B,mFAAmF;IACnF,OAAO,CAAC,EAAE,OAAO,CAAA;IACjB,wDAAwD;IACxD,QAAQ,CAAC,EAAE,MAAM,CAAA;CAClB;AAED,6DAA6D;AAC7D,MAAM,WAAW,cAAc;IAC7B,+EAA+E;IAC/E,WAAW,CAAC,EAAE,MAAM,CAAA;IACpB,0EAA0E;IAC1E,QAAQ,CAAC,EAAE,MAAM,CAAA;IACjB,oFAAoF;IACpF,WAAW,CAAC,EAAE,MAAM,CAAA;CACrB;AAED,yEAAyE;AACzE,MAAM,WAAW,WAAW;IAC1B,iFAAiF;IACjF,eAAe,CAAC,EAAE,MAAM,CAAA;IACxB,6FAA6F;IAC7F,SAAS,CAAC,EAAE,MAAM,CAAA;CACnB;AAED,sCAAsC;AACtC,MAAM,WAAW,cAAc;IAC7B,kDAAkD;IAClD,iBAAiB,CAAC,EAAE,MAAM,CAAA;IAC1B,4EAA4E;IAC5E,SAAS,CAAC,EAAE,MAAM,CAAA;IAClB,iEAAiE;IACjE,YAAY,CAAC,EAAE,MAAM,CAAA;CACtB;AAED,sFAAsF;AACtF,MAAM,WAAW,MAAM;IACrB,+CAA+C;IAC/C,SAAS,CAAC,EAAE,MAAM,CAAA;IAClB,2FAA2F;IAC3F,YAAY,CAAC,EAAE,MAAM,CAAA;IACrB,4EAA4E;IAC5E,YAAY,CAAC,EAAE,MAAM,CAAA;IACrB,uDAAuD;IACvD,SAAS,CAAC,EAAE,eAAe,CAAA;IAC3B,0DAA0D;IAC1D,MAAM,CAAC,EAAE,YAAY,CAAA;IACrB,uDAAuD;IACvD,SAAS,CAAC,EAAE,eAAe,CAAA;IAC3B,qDAAqD;IACrD,QAAQ,CAAC,EAAE,cAAc,CAAA;IACzB,+CAA+C;IAC/C,KAAK,CAAC,EAAE,WAAW,CAAA;IACnB,qDAAqD;IACrD,QAAQ,CAAC,EAAE,cAAc,CAAA;CAC1B;AAED,kFAAkF;AAClF,MAAM,WAAW,uBAAuB;IACtC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;IACrB,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,SAAS,GAAG,QAAQ,CAAA;IAChD,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,SAAS,CAAA;IACpC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAA;IAC5B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAA;IAC/B,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAA;CAC/B;AAED,+CAA+C;AAC/C,MAAM,WAAW,oBAAoB;IACnC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;IACrB,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;IAC7B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;IAC7B,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAA;IAChC,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAA;IAC/B,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAA;IAC/B,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAA;CAChC;AAED,kDAAkD;AAClD,MAAM,WAAW,uBAAuB;IACtC,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAA;IACzB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAA;CAC1B;AAED,iDAAiD;AACjD,MAAM,WAAW,sBAAsB;IACrC,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAA;IAC5B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAA;IACzB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAA;CAC7B;AAED,8CAA8C;AAC9C,MAAM,WAAW,mBAAmB;IAClC,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAA;IAChC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;CAC3B;AAED,iDAAiD;AACjD,MAAM,WAAW,sBAAsB;IACrC,QAAQ,CAAC,iBAAiB,EAAE,MAAM,CAAA;IAClC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;CAC9B;AAED,yFAAyF;AACzF,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;IAC7B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;IAC7B,QAAQ,CAAC,SAAS,EAAE,uBAAuB,CAAA;IAC3C,QAAQ,CAAC,MAAM,EAAE,oBAAoB,CAAA;IACrC,QAAQ,CAAC,SAAS,EAAE,uBAAuB,CAAA;IAC3C,QAAQ,CAAC,QAAQ,EAAE,sBAAsB,CAAA;IACzC,QAAQ,CAAC,KAAK,EAAE,mBAAmB,CAAA;IACnC,QAAQ,CAAC,QAAQ,EAAE,sBAAsB,CAAA;CAC1C;AAED,kFAAkF;AAClF,eAAO,MAAM,MAAM,EAAE,CAAC,CAAC,MAAM,CAoE3B,CAAA;AAgBF;;;;;;;;GAQG;AACH,wBAAgB,aAAa,CAAC,MAAM,GAAE,MAAW,GAAG,cAAc,CA6IjE"}
|
package/lib/types/embedding.d.ts
CHANGED
|
@@ -7,6 +7,7 @@
|
|
|
7
7
|
* @module dsh-library/embedding
|
|
8
8
|
*/
|
|
9
9
|
import type { SubprocessRuntime } from '@deepseek-ai/dsh-subprocess';
|
|
10
|
+
import type { ResolvedEmbeddingConfig } from './config.js';
|
|
10
11
|
/**
|
|
11
12
|
* Embed one text with the built-in hash embedder. Each word token and each
|
|
12
13
|
* character tri-gram hashes (FNV-1a) to a signed bucket: the index is the
|
|
@@ -53,4 +54,113 @@ export declare function embedWithCommand(subprocess: SubprocessRuntime, argv: re
|
|
|
53
54
|
* @returns the argv vector; an empty string yields an empty vector.
|
|
54
55
|
*/
|
|
55
56
|
export declare function splitCommandLine(command: string): string[];
|
|
57
|
+
/**
|
|
58
|
+
* The embedder provider seam: one pluggable text→vector backend. The built-in
|
|
59
|
+
* {@link HashEmbedder} is the zero-download default; {@link CommandEmbedder}
|
|
60
|
+
* and {@link OllamaEmbedder} are optional backends (probe → use → degrade for
|
|
61
|
+
* Ollama; fail loud for a configured-but-absent command).
|
|
62
|
+
*/
|
|
63
|
+
export interface Embedder {
|
|
64
|
+
/** Provider name recorded alongside the index (hash | command | ollama). */
|
|
65
|
+
readonly name: 'hash' | 'command' | 'ollama';
|
|
66
|
+
/** Vector dimensionality this backend emits. */
|
|
67
|
+
readonly dims: number;
|
|
68
|
+
/**
|
|
69
|
+
* Embed one batch of texts, returning one vector per input in the same order.
|
|
70
|
+
* @param texts - the texts to embed.
|
|
71
|
+
* @returns the L2-normalized embeddings.
|
|
72
|
+
*/
|
|
73
|
+
embed(texts: readonly string[]): Promise<number[][]>;
|
|
74
|
+
}
|
|
75
|
+
/** Built-in deterministic hash embedder: zero downloads, pure local hashing. */
|
|
76
|
+
export declare class HashEmbedder implements Embedder {
|
|
77
|
+
readonly dims: number;
|
|
78
|
+
readonly name: "hash";
|
|
79
|
+
/**
|
|
80
|
+
* @param dims - vector dimensionality (≥ 1).
|
|
81
|
+
*/
|
|
82
|
+
constructor(dims: number);
|
|
83
|
+
embed(texts: readonly string[]): Promise<number[][]>;
|
|
84
|
+
}
|
|
85
|
+
/** External command embedder over the JSON-lines subprocess protocol. */
|
|
86
|
+
export declare class CommandEmbedder implements Embedder {
|
|
87
|
+
private readonly subprocess;
|
|
88
|
+
private readonly argv;
|
|
89
|
+
private readonly cwd;
|
|
90
|
+
readonly dims: number;
|
|
91
|
+
private readonly caps;
|
|
92
|
+
private readonly maxBatchItems;
|
|
93
|
+
readonly name: "command";
|
|
94
|
+
/**
|
|
95
|
+
* @param subprocess - the mounted subprocess seam.
|
|
96
|
+
* @param argv - the command line split into argv (argv[0] = program).
|
|
97
|
+
* @param cwd - working directory for the child.
|
|
98
|
+
* @param dims - expected vector dimensionality.
|
|
99
|
+
* @param caps - timeout, grace, and output caps.
|
|
100
|
+
* @param maxBatchItems - texts per subprocess invocation; larger batches split.
|
|
101
|
+
*/
|
|
102
|
+
constructor(subprocess: SubprocessRuntime, argv: readonly string[], cwd: string, dims: number, caps: {
|
|
103
|
+
timeoutMs: number;
|
|
104
|
+
graceMs: number;
|
|
105
|
+
maxOutputBytes: number;
|
|
106
|
+
}, maxBatchItems: number);
|
|
107
|
+
embed(texts: readonly string[]): Promise<number[][]>;
|
|
108
|
+
}
|
|
109
|
+
/**
|
|
110
|
+
* Local Ollama embedder over the `/api/embed` endpoint. Zero cloud: the model
|
|
111
|
+
* runs on a localhost Ollama server. A non-2xx response or a malformed vector
|
|
112
|
+
* fails the batch (misconfiguration must surface, not silently degrade
|
|
113
|
+
* retrieval); availability is probed up front by {@link probeOllama}.
|
|
114
|
+
*/
|
|
115
|
+
export declare class OllamaEmbedder implements Embedder {
|
|
116
|
+
private readonly url;
|
|
117
|
+
private readonly model;
|
|
118
|
+
readonly dims: number;
|
|
119
|
+
private readonly timeoutMs;
|
|
120
|
+
private readonly maxBatchItems;
|
|
121
|
+
readonly name: "ollama";
|
|
122
|
+
/**
|
|
123
|
+
* @param url - the Ollama base URL (e.g. `http://127.0.0.1:11434`).
|
|
124
|
+
* @param model - the embedding model name (e.g. `nomic-embed-text`).
|
|
125
|
+
* @param dims - the probed embedding dimensionality.
|
|
126
|
+
* @param timeoutMs - per-request timeout.
|
|
127
|
+
* @param maxBatchItems - texts per `/api/embed` call; larger batches split.
|
|
128
|
+
*/
|
|
129
|
+
constructor(url: string, model: string, dims: number, timeoutMs: number, maxBatchItems: number);
|
|
130
|
+
embed(texts: readonly string[]): Promise<number[][]>;
|
|
131
|
+
/** POST one batch to `/api/embed` and validate the returned vectors. */
|
|
132
|
+
private embedBatch;
|
|
133
|
+
}
|
|
134
|
+
/**
|
|
135
|
+
* Probe a local Ollama server for one embedding model. Sends a single-word
|
|
136
|
+
* probe through `/api/embed` and returns the model's vector dimensionality, or
|
|
137
|
+
* undefined when the server is unreachable, the model is unknown, or the
|
|
138
|
+
* response is malformed. Zero cloud — only the configured local URL is ever
|
|
139
|
+
* contacted.
|
|
140
|
+
* @param url - the Ollama base URL.
|
|
141
|
+
* @param model - the embedding model name.
|
|
142
|
+
* @param timeoutMs - probe timeout.
|
|
143
|
+
* @returns the model's dimensionality, or undefined when unavailable.
|
|
144
|
+
*/
|
|
145
|
+
export declare function probeOllama(url: string, model: string, timeoutMs: number): Promise<number | undefined>;
|
|
146
|
+
/** The outcome of {@link resolveEmbedder}: the backend plus its degradation note. */
|
|
147
|
+
export interface EmbedderResolution {
|
|
148
|
+
/** The resolved backend. */
|
|
149
|
+
readonly embedder: Embedder;
|
|
150
|
+
/** True when the configured backend was unavailable and a default was substituted. */
|
|
151
|
+
readonly degraded: boolean;
|
|
152
|
+
/** Human-readable degradation reason (present only when {@link EmbedderResolution.degraded}). */
|
|
153
|
+
readonly reason?: string;
|
|
154
|
+
}
|
|
155
|
+
/**
|
|
156
|
+
* Resolve the embedder backend from the resolved config: the configured
|
|
157
|
+
* command wins (fail loud without a subprocess seam), the `ollama` provider is
|
|
158
|
+
* probed and degrades gracefully to the built-in hash embedder when the server
|
|
159
|
+
* is unreachable, and the built-in hash embedder is the zero-download default.
|
|
160
|
+
* @param config - the resolved embedding config.
|
|
161
|
+
* @param subprocess - the optional subprocess seam (command provider only).
|
|
162
|
+
* @param cwd - working directory for the command provider.
|
|
163
|
+
* @returns the backend plus its degradation note.
|
|
164
|
+
*/
|
|
165
|
+
export declare function resolveEmbedder(config: ResolvedEmbeddingConfig, subprocess: SubprocessRuntime | undefined, cwd: string): Promise<EmbedderResolution>;
|
|
56
166
|
//# sourceMappingURL=embedding.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"embedding.d.ts","sourceRoot":"","sources":["../../src/embedding.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAEH,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,6BAA6B,CAAA;
|
|
1
|
+
{"version":3,"file":"embedding.d.ts","sourceRoot":"","sources":["../../src/embedding.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAEH,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,6BAA6B,CAAA;AAEpE,OAAO,KAAK,EAAE,uBAAuB,EAAE,MAAM,aAAa,CAAA;AAG1D;;;;;;;;GAQG;AACH,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,MAAM,EAAE,CAU9D;AAcD;;;;;;GAMG;AACH,wBAAgB,MAAM,CAAC,CAAC,EAAE,SAAS,MAAM,EAAE,EAAE,CAAC,EAAE,SAAS,MAAM,EAAE,GAAG,MAAM,CAYzE;AAqBD;;;;;;;;;;;;;;;GAeG;AACH,wBAAsB,gBAAgB,CACpC,UAAU,EAAE,iBAAiB,EAC7B,IAAI,EAAE,SAAS,MAAM,EAAE,EACvB,GAAG,EAAE,MAAM,EACX,KAAK,EAAE,SAAS,MAAM,EAAE,EACxB,IAAI,EAAE,MAAM,EACZ,IAAI,EAAE;IAAE,SAAS,EAAE,MAAM,CAAC;IAAC,OAAO,EAAE,MAAM,CAAC;IAAC,cAAc,EAAE,MAAM,CAAA;CAAE,GACnE,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CA0CrB;AAED;;;;;GAKG;AACH,wBAAgB,gBAAgB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAE1D;AAID;;;;;GAKG;AACH,MAAM,WAAW,QAAQ;IACvB,4EAA4E;IAC5E,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,SAAS,GAAG,QAAQ,CAAA;IAC5C,gDAAgD;IAChD,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;IACrB;;;;OAIG;IACH,KAAK,CAAC,KAAK,EAAE,SAAS,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAA;CACrD;AAED,gFAAgF;AAChF,qBAAa,YAAa,YAAW,QAAQ;IAM/B,QAAQ,CAAC,IAAI,EAAE,MAAM;IALjC,QAAQ,CAAC,IAAI,EAAG,MAAM,CAAS;IAE/B;;OAEG;gBACkB,IAAI,EAAE,MAAM;IAE3B,KAAK,CAAC,KAAK,EAAE,SAAS,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC;CAG3D;AAED,yEAAyE;AACzE,qBAAa,eAAgB,YAAW,QAAQ;IAY5C,OAAO,CAAC,QAAQ,CAAC,UAAU;IAC3B,OAAO,CAAC,QAAQ,CAAC,IAAI;IACrB,OAAO,CAAC,QAAQ,CAAC,GAAG;IACpB,QAAQ,CAAC,IAAI,EAAE,MAAM;IACrB,OAAO,CAAC,QAAQ,CAAC,IAAI;IACrB,OAAO,CAAC,QAAQ,CAAC,aAAa;IAhBhC,QAAQ,CAAC,IAAI,EAAG,SAAS,CAAS;IAElC;;;;;;;OAOG;gBAEgB,UAAU,EAAE,iBAAiB,EAC7B,IAAI,EAAE,SAAS,MAAM,EAAE,EACvB,GAAG,EAAE,MAAM,EACnB,IAAI,EAAE,MAAM,EACJ,IAAI,EAAE;QAAE,SAAS,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAC;QAAC,cAAc,EAAE,MAAM,CAAA;KAAE,EACpE,aAAa,EAAE,MAAM;IAGlC,KAAK,CAAC,KAAK,EAAE,SAAS,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC;CAc3D;AAED;;;;;GAKG;AACH,qBAAa,cAAe,YAAW,QAAQ;IAW3C,OAAO,CAAC,QAAQ,CAAC,GAAG;IACpB,OAAO,CAAC,QAAQ,CAAC,KAAK;IACtB,QAAQ,CAAC,IAAI,EAAE,MAAM;IACrB,OAAO,CAAC,QAAQ,CAAC,SAAS;IAC1B,OAAO,CAAC,QAAQ,CAAC,aAAa;IAdhC,QAAQ,CAAC,IAAI,EAAG,QAAQ,CAAS;IAEjC;;;;;;OAMG;gBAEgB,GAAG,EAAE,MAAM,EACX,KAAK,EAAE,MAAM,EACrB,IAAI,EAAE,MAAM,EACJ,SAAS,EAAE,MAAM,EACjB,aAAa,EAAE,MAAM;IAGlC,KAAK,CAAC,KAAK,EAAE,SAAS,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC;IAQ1D,wEAAwE;YAC1D,UAAU;CA2BzB;AAED;;;;;;;;;;GAUG;AACH,wBAAsB,WAAW,CAAC,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC,CAgB5G;AAED,qFAAqF;AACrF,MAAM,WAAW,kBAAkB;IACjC,4BAA4B;IAC5B,QAAQ,CAAC,QAAQ,EAAE,QAAQ,CAAA;IAC3B,sFAAsF;IACtF,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAA;IAC1B,iGAAiG;IACjG,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAA;CACzB;AAED;;;;;;;;;GASG;AACH,wBAAsB,eAAe,CACnC,MAAM,EAAE,uBAAuB,EAC/B,UAAU,EAAE,iBAAiB,GAAG,SAAS,EACzC,GAAG,EAAE,MAAM,GACV,OAAO,CAAC,kBAAkB,CAAC,CAgC7B"}
|
package/lib/types/index.d.ts
CHANGED
|
@@ -14,15 +14,16 @@
|
|
|
14
14
|
import type { Context } from '@deepseek-ai/cordis';
|
|
15
15
|
import { type Domain } from '@deepseek-ai/dsh-storage-domain';
|
|
16
16
|
import { z as zod } from 'zod';
|
|
17
|
-
import type { SubprocessRuntime } from '@deepseek-ai/dsh-subprocess';
|
|
18
17
|
import { Config, type ResolvedConfig } from './config.js';
|
|
18
|
+
import { type Embedder } from './embedding.js';
|
|
19
19
|
import { chunkSizeHistogram } from './quality/chunk-visual.js';
|
|
20
20
|
export declare const name = "dsh-library";
|
|
21
21
|
/** Hard services: the durable index lives in the storage domain; tools and the command register here. */
|
|
22
22
|
export declare const inject: string[];
|
|
23
23
|
export { Config, resolveConfig, LIBRARY_NAME } from './config.js';
|
|
24
24
|
export type { ResolvedConfig, EmbeddingConfig, SearchConfig, InjectionConfig, CitationConfig, PurgeConfig, DiagnoseConfig } from './config.js';
|
|
25
|
-
export { embedHash, cosine, splitCommandLine } from './embedding.js';
|
|
25
|
+
export { embedHash, cosine, splitCommandLine, HashEmbedder, CommandEmbedder, OllamaEmbedder, probeOllama, resolveEmbedder } from './embedding.js';
|
|
26
|
+
export type { Embedder, EmbedderResolution } from './embedding.js';
|
|
26
27
|
export { CHUNK_SIZE_BUCKETS, chunkSizeHistogram, chunkText, chunkHash, findDuplicateChunks } from './quality/chunk-visual.js';
|
|
27
28
|
export type { Chunk, ChunkSizeHistogram, ChunkDiagnostics } from './quality/chunk-visual.js';
|
|
28
29
|
export { checkDiversity, maximalMarginalRelevance } from './quality/diversity.js';
|
|
@@ -98,9 +99,9 @@ declare module '@deepseek-ai/dsh-session/types' {
|
|
|
98
99
|
};
|
|
99
100
|
}
|
|
100
101
|
}
|
|
101
|
-
/** Optional seams
|
|
102
|
+
/** Optional seams resolved at call time; the embedder is resolved at mount (fail closed when absent). */
|
|
102
103
|
interface StoreDeps {
|
|
103
|
-
readonly
|
|
104
|
+
readonly embedder: Embedder;
|
|
104
105
|
}
|
|
105
106
|
/** One `library_add` outcome. */
|
|
106
107
|
export interface LibraryAddResult {
|
|
@@ -193,7 +194,7 @@ export declare class LibraryStore {
|
|
|
193
194
|
* @param deps - optional subprocess seam for the external embedder.
|
|
194
195
|
*/
|
|
195
196
|
constructor(domain: Domain<typeof libraryDomainSpec>, config: ResolvedConfig, deps: StoreDeps);
|
|
196
|
-
/** Embed a text batch with the
|
|
197
|
+
/** Embed a text batch with the resolved embedder backend. */
|
|
197
198
|
private embed;
|
|
198
199
|
/**
|
|
199
200
|
* Import one document: chunk it, embed the chunks, and store the records.
|
package/lib/types/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAGlD,OAAO,EAAgB,KAAK,MAAM,EAAgB,MAAM,iCAAiC,CAAA;AACzF,OAAO,EAAE,CAAC,IAAI,GAAG,EAAE,MAAM,KAAK,CAAA;
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAGlD,OAAO,EAAgB,KAAK,MAAM,EAAgB,MAAM,iCAAiC,CAAA;AACzF,OAAO,EAAE,CAAC,IAAI,GAAG,EAAE,MAAM,KAAK,CAAA;AAM9B,OAAO,EAAE,MAAM,EAA+B,KAAK,cAAc,EAAE,MAAM,aAAa,CAAA;AACtF,OAAO,EAA2B,KAAK,QAAQ,EAAE,MAAM,gBAAgB,CAAA;AAEvE,OAAO,EAAE,kBAAkB,EAA8C,MAAM,2BAA2B,CAAA;AAQ1G,eAAO,MAAM,IAAI,gBAAgB,CAAA;AAEjC,yGAAyG;AACzG,eAAO,MAAM,MAAM,UAAyC,CAAA;AAE5D,OAAO,EAAE,MAAM,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,aAAa,CAAA;AACjE,YAAY,EAAE,cAAc,EAAE,eAAe,EAAE,YAAY,EAAE,eAAe,EAAE,cAAc,EAAE,WAAW,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AAC9I,OAAO,EAAE,SAAS,EAAE,MAAM,EAAE,gBAAgB,EAAE,YAAY,EAAE,eAAe,EAAE,cAAc,EAAE,WAAW,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAA;AACjJ,YAAY,EAAE,QAAQ,EAAE,kBAAkB,EAAE,MAAM,gBAAgB,CAAA;AAClE,OAAO,EAAE,kBAAkB,EAAE,kBAAkB,EAAE,SAAS,EAAE,SAAS,EAAE,mBAAmB,EAAE,MAAM,2BAA2B,CAAA;AAC7H,YAAY,EAAE,KAAK,EAAE,kBAAkB,EAAE,gBAAgB,EAAE,MAAM,2BAA2B,CAAA;AAC5F,OAAO,EAAE,cAAc,EAAE,wBAAwB,EAAE,MAAM,wBAAwB,CAAA;AACjF,YAAY,EAAE,eAAe,EAAE,eAAe,EAAE,aAAa,EAAE,MAAM,wBAAwB,CAAA;AAC7F,OAAO,EAAE,iBAAiB,EAAE,SAAS,EAAE,WAAW,EAAE,gBAAgB,EAAE,eAAe,EAAE,YAAY,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,0BAA0B,CAAA;AAClK,YAAY,EAAE,WAAW,EAAE,aAAa,EAAE,UAAU,EAAE,WAAW,EAAE,MAAM,0BAA0B,CAAA;AACnG,OAAO,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,wBAAwB,CAAA;AACpF,YAAY,EAAE,cAAc,EAAE,MAAM,wBAAwB,CAAA;AAC5D,OAAO,EAAE,cAAc,EAAE,aAAa,EAAE,mBAAmB,EAAE,MAAM,uBAAuB,CAAA;AAC1F,YAAY,EAAE,SAAS,EAAE,eAAe,EAAE,cAAc,EAAE,MAAM,uBAAuB,CAAA;AACvF,OAAO,EAAE,gBAAgB,EAAE,cAAc,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAA;AAC3F,YAAY,EAAE,cAAc,EAAE,iBAAiB,EAAE,gBAAgB,EAAE,eAAe,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAA;AACpI,OAAO,EAAE,sBAAsB,EAAE,2BAA2B,EAAE,iBAAiB,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,MAAM,uBAAuB,CAAA;AACnJ,YAAY,EAAE,aAAa,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAA;AAC5E,OAAO,EAAE,WAAW,EAAE,gBAAgB,EAAE,MAAM,oBAAoB,CAAA;AAClE,YAAY,EAAE,cAAc,EAAE,gBAAgB,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAA;AAoCrG,4FAA4F;AAC5F,eAAO,MAAM,iBAAiB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CAQ5B,CAAA;AAEF,8EAA8E;AAC9E,OAAO,QAAQ,gCAAgC,CAAC;IAC9C,UAAU,eAAe;QACvB,4FAA4F;QAC5F,gBAAgB,EAAE;YAChB,QAAQ,EAAE,MAAM,CAAA;YAChB,OAAO,EAAE,MAAM,CAAA;YACf,KAAK,EAAE,MAAM,CAAA;YACb,MAAM,EAAE,MAAM,EAAE,CAAA;YAChB,KAAK,EAAE,MAAM,CAAA;SACd,CAAA;QACD,uDAAuD;QACvD,eAAe,EAAE;YACf,OAAO,EAAE,MAAM,CAAA;YACf,OAAO,EAAE,MAAM,CAAA;YACf,UAAU,EAAE,MAAM,CAAA;YAClB,MAAM,EAAE,OAAO,CAAA;YACf,UAAU,EAAE,MAAM,CAAA;SACnB,CAAA;KACF;CACF;AAED,yGAAyG;AACzG,UAAU,SAAS;IACjB,QAAQ,CAAC,QAAQ,EAAE,QAAQ,CAAA;CAC5B;AAED,iCAAiC;AACjC,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;IAC3B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;IACrB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;IACvB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;CACvB;AAED,sEAAsE;AACtE,MAAM,WAAW,mBAAmB;IAClC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;IAC3B,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAA;IAC9B,QAAQ,CAAC,WAAW,EAAE,OAAO,CAAA;IAC7B,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAA;CACjC;AAED,0CAA0C;AAC1C,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;IAC3B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;IACrB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;IACvB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;IACtB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;CAC3B;AAED,gCAAgC;AAChC,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;IAC3B,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAA;IACpB,qDAAqD;IACrD,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,+EAA+E;IAC/E,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;CACvB;AAED,6CAA6C;AAC7C,MAAM,WAAW,mBAAmB;IAClC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;IACtB,QAAQ,CAAC,OAAO,EAAE,gBAAgB,EAAE,CAAA;IACpC,qEAAqE;IACrE,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAA;CAC3B;AAED,iDAAiD;AACjD,MAAM,WAAW,kBAAkB;IACjC,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAA;IACzB,QAAQ,CAAC,gBAAgB,EAAE,MAAM,CAAA;IACjC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAA;IACvB,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;IAC3B,QAAQ,CAAC,kBAAkB,EAAE,MAAM,CAAA;IACnC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;CACxB;AAED,uCAAuC;AACvC,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;IACtB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;IACtB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;IACtB,QAAQ,CAAC,OAAO,EAAE,kBAAkB,EAAE,CAAA;CACvC;AAED,qCAAqC;AACrC,MAAM,WAAW,qBAAqB;IACpC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;IACxB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;IACvB,QAAQ,CAAC,SAAS,EAAE,UAAU,CAAC,OAAO,kBAAkB,CAAC,CAAA;IACzD,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAA;IAC/B,QAAQ,CAAC,aAAa,EAAE;QAAE,MAAM,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE,CAAA;IAC3D,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAA;CAC/B;AA0BD;;;;GAIG;AACH,qBAAa,YAAY;IAYrB,OAAO,CAAC,QAAQ,CAAC,MAAM;IACvB,OAAO,CAAC,QAAQ,CAAC,IAAI;IAZvB,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAiC;IAC3D,OAAO,CAAC,QAAQ,CAAC,MAAM,CAA8B;IACrD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAA8B;IAErD;;;;OAIG;gBAED,MAAM,EAAE,MAAM,CAAC,OAAO,iBAAiB,CAAC,EACvB,MAAM,EAAE,cAAc,EACtB,IAAI,EAAE,SAAS;IAOlC,6DAA6D;YAC/C,KAAK;IAInB;;;;;;OAMG;IACG,GAAG,CAAC,OAAO,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,OAAO,CAAC,gBAAgB,CAAC;IAgCvF;;;;;;OAMG;IACG,MAAM,CAAC,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,GAAG,OAAO,CAAC,mBAAmB,CAAC;IAgC/E,kFAAkF;IAClF,IAAI,CAAC,OAAO,CAAC,EAAE,MAAM,GAAG,gBAAgB,EAAE;IAkB1C,+DAA+D;IAC/D,OAAO,CAAC,QAAQ;IAQhB;;;;;;;OAOG;IACG,MAAM,CAAC,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,gBAAgB,EAAE,CAAC;IAkDxF,+EAA+E;IACzE,SAAS,CAAC,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,iBAAiB,CAAC;IAuC3F,qFAAqF;IAC/E,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,OAAO,CAAC,qBAAqB,CAAC;CAoChE;AAYD,8DAA8D;AAC9D,UAAU,eAAe;IACvB,QAAQ,CAAC,GAAG,EAAE,OAAO,CAAA;IACrB,QAAQ,CAAC,MAAM,EAAE,cAAc,CAAA;IAC/B,QAAQ,CAAC,KAAK,EAAE,YAAY,CAAA;CAC7B;AAaD;;;;;;GAMG;AACH,wBAAsB,KAAK,CAAC,GAAG,EAAE,OAAO,EAAE,MAAM,GAAE,MAAW,GAAG,OAAO,CAAC,IAAI,CAAC,CAqC5E;AAED,oDAAoD;AACpD,wBAAgB,QAAQ,CAAC,QAAQ,EAAE,eAAe,qDASjD"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-library",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.2.0",
|
|
4
4
|
"description": "Local document knowledge base for DeepSeek Harness: library_add/remove/list, hybrid semantic+keyword library_search with diversity re-ranking, relevance filtering and lost-in-the-middle avoidance, citation-aware injection, library_cite_check and library_diagnose —SQLite-backed index via the storage domain, local embedding, zero model downloads",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -103,6 +103,7 @@
|
|
|
103
103
|
"packageManager": "pnpm@11.7.0",
|
|
104
104
|
"peerDependencies": {
|
|
105
105
|
"@deepseek-ai/cordis": "^4.0.1",
|
|
106
|
+
"@deepseek-ai/dsh-commands": ">=0.1.0-rc.8 <0.2.0",
|
|
106
107
|
"@deepseek-ai/dsh-fs": ">=0.1.0-rc.8 <0.2.0",
|
|
107
108
|
"@deepseek-ai/dsh-llm": ">=0.1.0-rc.8 <0.2.0",
|
|
108
109
|
"@deepseek-ai/dsh-session": ">=0.1.0-rc.8 <0.2.0",
|