ko-qmd 2.8.3-ko.7 → 2.8.3-ko.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +52 -0
- package/README.en.md +1 -1
- package/README.md +3 -1
- package/dist/cli/build-info.json +2 -2
- package/dist/cli/qmd.js +2 -0
- package/dist/hangul.js +31 -10
- package/dist/mcp/server.js +3 -0
- package/dist/store.d.ts +59 -10
- package/dist/store.js +204 -31
- package/package.json +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,58 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
- Dependencies: `node-llama-cpp` 3.20.0 → **3.22.1**. ko-vault seam bench lines
|
|
6
|
+
are byte-identical before and after; Nix flake `qmd-node-modules` FOD hashes
|
|
7
|
+
refreshed for the new tree.
|
|
8
|
+
|
|
9
|
+
## [2.8.3-ko.9] - 2026-10-10
|
|
10
|
+
|
|
11
|
+
ko-qmd 2.8.3-ko.9 buys rank-1 accuracy and rerank latency at once. Per-document
|
|
12
|
+
coverage weighting on relaxed lex lists takes hard `hybrid_r1` from 0.4444 to
|
|
13
|
+
0.4778, and head-first rerank cascade halves rerank p50 with identical top-1
|
|
14
|
+
answers. A load timeout (default 120 s) stops one stuck model load from hanging
|
|
15
|
+
the query, as the scoring deadlines never covered the pre-ready wait.
|
|
16
|
+
|
|
17
|
+
- `runWithDeadline` now bounds the model-load wait itself (`QMD_LLM_LOAD_TIMEOUT_MS`,
|
|
18
|
+
default 120 s, `0` = wait forever). The scoring deadlines never covered a load
|
|
19
|
+
that reports ready never: the pre-ready race hung with no timer, so one stuck
|
|
20
|
+
load hung the query. Past the load bound the stage is skipped exactly like a
|
|
21
|
+
scoring deadline (same `timeouts` entry, same RRF-order / original-query
|
|
22
|
+
fallback, same abandoned-run cache fill), and the bounded wait is reported as
|
|
23
|
+
the `load` stage. Listed in `qmd doctor`.
|
|
24
|
+
- `QMD_RERANK_CASCADE=0` (or `false`/`off`/`no`) restores single-batch rerank.
|
|
25
|
+
Kill switch for measuring the cascade and for operations; unset cascades.
|
|
26
|
+
Listed in `qmd doctor`.
|
|
27
|
+
- Rerank now cascades: the RRF head (15) is scored first, and the tail is
|
|
28
|
+
skipped with score 0 (RRF position order only) when the head's top-2 margin
|
|
29
|
+
reaches 0.08 (`RERANK_CASCADE_FIRST_BATCH`/`RERANK_CASCADE_MARGIN` in
|
|
30
|
+
`src/store.ts`). A contested top scores everything, as before; deadlines and
|
|
31
|
+
the rerank cache are unchanged (a skipped tail writes no cache entries).
|
|
32
|
+
- `structuredSearch` and `hybridQuery` now weight a relaxed (any-word) lex list per
|
|
33
|
+
document by query coverage (`relaxedDocCoverage` in `src/store.ts`: 0.5 + 0.5 ×
|
|
34
|
+
matched units, mirroring the rerank chunk scorer's Hangul bigrams). A global
|
|
35
|
+
constant could not split the mirrored tie of `hmu-05` vs `sem-08` (2026-10-09
|
|
36
|
+
probe); both lead one list and take the same top-rank bonus, so only a per-doc
|
|
37
|
+
signal separates them. List weights are unchanged.
|
|
38
|
+
## [2.8.3-ko.8] - 2026-10-09
|
|
39
|
+
|
|
40
|
+
- `structuredSearch` (MCP `searches`, SDK `search({ queries })`) no longer lets a relaxed
|
|
41
|
+
(any-word) first lex list keep the 2x positional boost: it now counts 0.25 instead of 1.0.
|
|
42
|
+
Even halved, a relaxed rank 1 still tied a vector rank 1 against a lex rank 4 (`hmu-05`
|
|
43
|
+
probe), so the decoy sharing a few question words kept rank 1 on the `rerank:false` path.
|
|
44
|
+
On the ko-vault seam form, hard `hybrid_r1` goes 0.4111 → 0.4444 with `full_r1` unchanged at
|
|
45
|
+
0.6111; one easy query (`sem-08`) trades hybrid rank 1 for 2 while its `full` rank 1 and all
|
|
46
|
+
recall@5 lines hold. Non-first relaxed lists still count half, as before.
|
|
47
|
+
- Hangul loanword table gains 11 entries mined from real query-log traffic (`지연` →
|
|
48
|
+
`latency`, `병렬` → `parallel`, `게이트` → `gate`, `가드` → `guard`, `패널` → `panel`,
|
|
49
|
+
`팔레트` → `palette`, `하네스` → `harness`, `옵시디언` → `obsidian`, `백링크` → `backlink`,
|
|
50
|
+
`포즈` → `pause`, `루프` → `loop`). Native words from the same logs (`색인`·`질의`·`요약`·`문서`
|
|
51
|
+
…) stay unbridged. Query-side only, no re-index; ko-vault bench lines are unchanged.
|
|
52
|
+
- Negation routing stays strip-only by measurement: demoting X with lex `-term`s was tried
|
|
53
|
+
and reverted — the expected note often discusses X itself (`bm25-ranking.md` mentions TF-IDF),
|
|
54
|
+
so NOT removed the answer with the decoy (`hybrid_r1` 0.4111 → 0.3778). `stripHangulNegation`
|
|
55
|
+
now shares its parsing through an internal split helper; behavior is unchanged.
|
|
56
|
+
|
|
5
57
|
## [2.8.3-ko.7] - 2026-10-02
|
|
6
58
|
|
|
7
59
|
The query path is now observable and bounded. Every query-log row breaks its latency into
|
package/README.en.md
CHANGED
|
@@ -38,7 +38,7 @@ flowchart LR
|
|
|
38
38
|
(`검색을` → `검색`), a two-particle chain (`청킹에서의` → `청킹`), verb and nominalizing endings
|
|
39
39
|
(`토큰화하는` → `토큰화`, `검색하기` → `검색`), `한`/`된` endings (`필요한` → `필요`) and the 하↔해
|
|
40
40
|
contraction. Quoted phrases, Hanja and kana are left as written.
|
|
41
|
-
- **Loanword bridge** — about
|
|
41
|
+
- **Loanword bridge** — about 90 built-in Hangul spellings of technical terms (`서치`, `웹`, `코어`,
|
|
42
42
|
`마이그레이션`, …) also search their English spelling, so `하이브리드 서치` finds a `hybrid-search` note.
|
|
43
43
|
Only the query changes; no re-index is needed.
|
|
44
44
|
- **Mixed-script token split** — `SKILL.md계약의핵심` becomes `skill` AND `md` AND `계약의핵심`.
|
package/README.md
CHANGED
|
@@ -34,7 +34,7 @@ flowchart LR
|
|
|
34
34
|
- **조사·어미 제거** — 따옴표 없는 한글 단어는 어간도 매칭한다. 끝 조사 하나(`검색을` → `검색`), 두 조사 연쇄(`청킹에서의` → `청킹`),
|
|
35
35
|
동사·명사화 어미(`토큰화하는` → `토큰화`, `검색하기` → `검색`), `한`/`된` 어미(`필요한` → `필요`), 하↔해 축약(`더하` ↔ `더해`).
|
|
36
36
|
따옴표 구문·한자·가나는 그대로 둔다.
|
|
37
|
-
- **외래어 다리** — 기술 용어의 한글 표기(`서치`·`웹`·`코어`·`마이그레이션` 등 약
|
|
37
|
+
- **외래어 다리** — 기술 용어의 한글 표기(`서치`·`웹`·`코어`·`마이그레이션` 등 약 90개 내장 표)는 영문 표기도 함께 찾는다.
|
|
38
38
|
`하이브리드 서치`가 `hybrid-search` 노트를, `리액트 웹 코어`가 `react-web-core`를 찾는다. 질의 쪽만 바뀌므로 재색인은 필요 없다.
|
|
39
39
|
- **문자 체계가 섞인 토큰 분리** — `SKILL.md계약의핵심`은 `skill` AND `md` AND `계약의핵심`으로 나눈다.
|
|
40
40
|
- **긴 질의 완화** — 세 단어 이상 질의가 AND 로 한 건도 안 맞으면 같은 단어를 OR 로 다시 찾는다. 자연어 문장 질의가 0건이 되지 않는다.
|
|
@@ -169,6 +169,8 @@ API 전체: [docs/REFERENCE.md § SDK / Library Usage](docs/REFERENCE.md#sdk--li
|
|
|
169
169
|
|---|---|
|
|
170
170
|
| `QMD_QUERY_LOG` | `1`/`true`/`yes` 면 HTTP 데몬이 검색마다 질의 로그를 남긴다 |
|
|
171
171
|
| `QMD_LLM_IDLE_TIMEOUT_MS` | 모델을 내리기까지 idle ms (`0` = 내리지 않음) |
|
|
172
|
+
| `QMD_LLM_LOAD_TIMEOUT_MS` | 모델 로드 대기의 상한 ms (기본 120000, `0` = 무한 대기). 넘으면 그 단계 없이 답하고 로드는 백그라운드에서 계속된다 |
|
|
173
|
+
| `QMD_RERANK_CASCADE` | `0`/`false`/`off`/`no` 면 리랭크 cascade를 끄고 후보 전체를 한 번에 채점한다 |
|
|
172
174
|
| `QMD_RERANK_MAX_DOC_TOKENS` | 리랭커에 보내는 문서당 토큰 cap (요청의 `rerankMaxDocTokens` 가 우선) |
|
|
173
175
|
| `QMD_RERANK_CONTEXT_SIZE` · `QMD_EMBED_CONTEXT_SIZE` · `QMD_EXPAND_CONTEXT_SIZE` | 역할별 context 크기 |
|
|
174
176
|
| `QMD_EMBED_PARALLELISM` | 병렬 context 수 (높으면 RAM/VRAM 고갈) |
|
package/dist/cli/build-info.json
CHANGED
package/dist/cli/qmd.js
CHANGED
|
@@ -3583,6 +3583,8 @@ function collectEnvironmentOverrides(activeModels, configModels = {}) {
|
|
|
3583
3583
|
add("QMD_DOCTOR_DEVICE_PROBE", "controls qmd doctor native device probing; 0/off skips GPU probing");
|
|
3584
3584
|
add("QMD_EMBED_PARALLELISM", "overrides embedding parallel context count; too high can exhaust RAM/VRAM");
|
|
3585
3585
|
add("QMD_LLM_IDLE_TIMEOUT_MS", "ms of idle before models unload (0 = never); keeping them loaded holds their memory");
|
|
3586
|
+
add("QMD_LLM_LOAD_TIMEOUT_MS", "caps the model-load wait per stage (default 120000, 0 = wait forever); past it the stage is skipped");
|
|
3587
|
+
add("QMD_RERANK_CASCADE", "0/false/off/no disables head-first cascade rerank and scores all candidates at once");
|
|
3586
3588
|
add("QMD_EXPAND_CONTEXT_SIZE", "overrides query expansion context size; larger values use more memory");
|
|
3587
3589
|
add("QMD_RERANK_CONTEXT_SIZE", "overrides reranker context size; larger values use more memory");
|
|
3588
3590
|
add("QMD_RERANK_MAX_DOC_TOKENS", "caps tokens per document sent to the reranker; lower is faster but scores less text");
|
package/dist/hangul.js
CHANGED
|
@@ -65,24 +65,34 @@ export function sharesHangulBigram(a, b) {
|
|
|
65
65
|
return false;
|
|
66
66
|
}
|
|
67
67
|
/**
|
|
68
|
-
*
|
|
69
|
-
*
|
|
70
|
-
*
|
|
71
|
-
*
|
|
72
|
-
* no marker stands as its own token — a marker glued to X (`이거말고`) is not split.
|
|
68
|
+
* Split a "X 말고 Y" query at its last exclusion marker into the excluded (X)
|
|
69
|
+
* and kept (Y) token lists. Null when there is no rewrite: a `"` phrase is
|
|
70
|
+
* present, no Hangul, no standalone marker, an empty side, or a Y of only
|
|
71
|
+
* `-term` exclusions (lex needs a positive term to match).
|
|
73
72
|
*/
|
|
74
|
-
|
|
73
|
+
function splitHangulNegation(query) {
|
|
75
74
|
if (query.includes('"') || !containsHangul(query))
|
|
76
|
-
return
|
|
75
|
+
return null;
|
|
77
76
|
const tokens = query.trim().split(/\s+/);
|
|
78
77
|
const last = tokens.findLastIndex((token) => NEGATION_MARKERS.has(token));
|
|
79
78
|
if (last < 1 || last === tokens.length - 1)
|
|
80
|
-
return
|
|
79
|
+
return null;
|
|
81
80
|
const kept = tokens.slice(last + 1);
|
|
82
81
|
// A Y of only `-term` exclusions leaves lex no positive term to match.
|
|
83
82
|
if (kept.every((token) => token.startsWith("-")))
|
|
84
|
-
return
|
|
85
|
-
return
|
|
83
|
+
return null;
|
|
84
|
+
return { excluded: tokens.slice(0, last), kept };
|
|
85
|
+
}
|
|
86
|
+
/**
|
|
87
|
+
* Drop the excluded clause of a Korean negation query and keep what follows the
|
|
88
|
+
* last marker (`tf-idf 말고 순위 공식` → `순위 공식`), so retrieval and rerank
|
|
89
|
+
* never see the concept the user ruled out. Returns the query unchanged when a
|
|
90
|
+
* side would be empty or hold only `-term` exclusions, a `"` phrase is present, or
|
|
91
|
+
* no marker stands as its own token — a marker glued to X (`이거말고`) is not split.
|
|
92
|
+
*/
|
|
93
|
+
export function stripHangulNegation(query) {
|
|
94
|
+
const split = splitHangulNegation(query);
|
|
95
|
+
return split ? split.kept.join(" ") : query;
|
|
86
96
|
}
|
|
87
97
|
/**
|
|
88
98
|
* Strip one trailing particle (or 기) from a pure-Hangul word when the stem
|
|
@@ -173,7 +183,9 @@ export function hangulTermQuery(term) {
|
|
|
173
183
|
* ASCII words separated by single spaces so they are safe inside an FTS5 phrase.
|
|
174
184
|
*/
|
|
175
185
|
const LOANWORDS = {
|
|
186
|
+
"가드": ["guard"],
|
|
176
187
|
"가이드": ["guide"],
|
|
188
|
+
"게이트": ["gate"],
|
|
177
189
|
"골든": ["golden"],
|
|
178
190
|
"그래프": ["graph"],
|
|
179
191
|
"노트": ["note"],
|
|
@@ -194,14 +206,17 @@ const LOANWORDS = {
|
|
|
194
206
|
"리트리벌": ["retrieval"],
|
|
195
207
|
"리피티션": ["repetition"],
|
|
196
208
|
"린트": ["lint"],
|
|
209
|
+
"루프": ["loop"],
|
|
197
210
|
"마이그레이션": ["migration"],
|
|
198
211
|
"메타데이터": ["metadata"],
|
|
199
212
|
"모델": ["model"],
|
|
200
213
|
"미팅": ["meeting"],
|
|
201
214
|
"바이그램": ["bigram"],
|
|
215
|
+
"백링크": ["backlink"],
|
|
202
216
|
"벡터": ["vector"],
|
|
203
217
|
"벤치": ["bench"],
|
|
204
218
|
"벤치마크": ["benchmark"],
|
|
219
|
+
"병렬": ["parallel"],
|
|
205
220
|
"서버": ["server"],
|
|
206
221
|
"서치": ["search"],
|
|
207
222
|
"세션": ["session"],
|
|
@@ -215,6 +230,7 @@ const LOANWORDS = {
|
|
|
215
230
|
"엔그램": ["ngram", "n gram"],
|
|
216
231
|
"오토메이션": ["automation"],
|
|
217
232
|
"온리": ["only"],
|
|
233
|
+
"옵시디언": ["obsidian"],
|
|
218
234
|
"워크플로": ["workflow"],
|
|
219
235
|
"워크플로우": ["workflow"],
|
|
220
236
|
"웹": ["web"],
|
|
@@ -227,6 +243,7 @@ const LOANWORDS = {
|
|
|
227
243
|
"인제스트": ["ingest"],
|
|
228
244
|
"인코더": ["encoder"],
|
|
229
245
|
"임베딩": ["embedding"],
|
|
246
|
+
"지연": ["latency"],
|
|
230
247
|
"청크": ["chunk"],
|
|
231
248
|
"청킹": ["chunking"],
|
|
232
249
|
"캐시": ["cache"],
|
|
@@ -242,6 +259,9 @@ const LOANWORDS = {
|
|
|
242
259
|
"토픽": ["topic"],
|
|
243
260
|
"트래킹": ["tracking"],
|
|
244
261
|
"파이프라인": ["pipeline"],
|
|
262
|
+
"패널": ["panel"],
|
|
263
|
+
"팔레트": ["palette"],
|
|
264
|
+
"포즈": ["pause"],
|
|
245
265
|
"퓨전": ["fusion"],
|
|
246
266
|
"프로버넌스": ["provenance"],
|
|
247
267
|
"프로비넌스": ["provenance"],
|
|
@@ -250,6 +270,7 @@ const LOANWORDS = {
|
|
|
250
270
|
"프롬프트": ["prompt"],
|
|
251
271
|
"플러그인": ["plugin"],
|
|
252
272
|
"필터": ["filter"],
|
|
273
|
+
"하네스": ["harness"],
|
|
253
274
|
"하이브리드": ["hybrid"],
|
|
254
275
|
"훅": ["hook"],
|
|
255
276
|
};
|
package/dist/mcp/server.js
CHANGED
|
@@ -128,6 +128,9 @@ function stageTimer() {
|
|
|
128
128
|
* ko-qmd: how long the daemon waits for query expansion and for the reranker before answering
|
|
129
129
|
* without them. QMD_EXPAND_DEADLINE_MS / QMD_RERANK_DEADLINE_MS: 0 = wait, a positive integer =
|
|
130
130
|
* that many ms, anything else = the default. One MCP query once took 85 s.
|
|
131
|
+
* The model-load wait itself is bounded separately by QMD_LLM_LOAD_TIMEOUT_MS
|
|
132
|
+
* (default 120 s, 0 = wait forever), so a load that never reports ready also
|
|
133
|
+
* falls back instead of hanging.
|
|
131
134
|
*/
|
|
132
135
|
const DEFAULT_EXPAND_DEADLINE_MS = 5_000;
|
|
133
136
|
const DEFAULT_RERANK_DEADLINE_MS = 10_000;
|
package/dist/store.d.ts
CHANGED
|
@@ -477,6 +477,7 @@ export type RRFContributionTrace = {
|
|
|
477
477
|
query: string;
|
|
478
478
|
rank: number;
|
|
479
479
|
weight: number;
|
|
480
|
+
multiplier?: number;
|
|
480
481
|
backendScore: number;
|
|
481
482
|
rrfContribution: number;
|
|
482
483
|
};
|
|
@@ -912,11 +913,32 @@ export declare function rerank(query: string, documents: {
|
|
|
912
913
|
file: string;
|
|
913
914
|
score: number;
|
|
914
915
|
}[]>;
|
|
915
|
-
|
|
916
|
+
/**
|
|
917
|
+
* Fraction of the query's units present in the document body, used to weight a
|
|
918
|
+
* relaxed (any-word) FTS list per document instead of one global constant
|
|
919
|
+
* (BASELINE.md 2026-10-09 follow-up: a global weight cannot split the mirrored
|
|
920
|
+
* tie of `hmu-05` vs `sem-08`).
|
|
921
|
+
*
|
|
922
|
+
* Units mirror the rerank chunk scorer: one per Hangul run of two or more
|
|
923
|
+
* syllables (a run of three or more also matches on any shared syllable
|
|
924
|
+
* bigram, so a particle-suffixed word meets its stem), one per Latin/digit run
|
|
925
|
+
* longer than two characters. Single syllables and short tokens carry no
|
|
926
|
+
* signal and are skipped. A query with no units covers everything (1.0) so it
|
|
927
|
+
* never punishes.
|
|
928
|
+
*/
|
|
929
|
+
export declare function relaxedDocCoverage(query: string, body: string): number;
|
|
930
|
+
/**
|
|
931
|
+
* Per-document RRF multipliers, aligned with `rankedLists`: a relaxed list
|
|
932
|
+
* scales each document by `0.5 + 0.5 × coverage`, every other list is untouched
|
|
933
|
+
* (`undefined` entry = multiplier 1.0). The 0.5 floor keeps a zero-coverage
|
|
934
|
+
* candidate in the pool instead of erasing it.
|
|
935
|
+
*/
|
|
936
|
+
export declare function rrfDocMultipliers(rankedLists: RankedResult[][], rankedListMeta: RankedListMeta[]): Array<Map<string, number> | undefined>;
|
|
937
|
+
export declare function reciprocalRankFusion(resultLists: RankedResult[][], weights?: number[], k?: number, docMultipliers?: Array<Map<string, number> | undefined>): RankedResult[];
|
|
916
938
|
/**
|
|
917
939
|
* Build per-document RRF contribution traces for explain/debug output.
|
|
918
940
|
*/
|
|
919
|
-
export declare function buildRrfTrace(resultLists: RankedResult[][], weights?: number[], listMeta?: RankedListMeta[], k?: number): Map<string, RRFScoreTrace>;
|
|
941
|
+
export declare function buildRrfTrace(resultLists: RankedResult[][], weights?: number[], listMeta?: RankedListMeta[], k?: number, docMultipliers?: Array<Map<string, number> | undefined>): Map<string, RRFScoreTrace>;
|
|
920
942
|
/**
|
|
921
943
|
* Find a document by filename/path, docid (#hash), or with fuzzy matching.
|
|
922
944
|
* Returns document metadata without body by default.
|
|
@@ -1080,14 +1102,6 @@ export type RankedListMeta = {
|
|
|
1080
1102
|
* original vector boost.
|
|
1081
1103
|
*/
|
|
1082
1104
|
export declare function getHybridRrfWeights(rankedListMeta: RankedListMeta[]): number[];
|
|
1083
|
-
/**
|
|
1084
|
-
* RRF list weights for structuredSearch: the first list gets 2x (the caller ordered by
|
|
1085
|
-
* importance), the rest 1x.
|
|
1086
|
-
*
|
|
1087
|
-
* ko-qmd: a relaxed FTS list counts half here too, as in getHybridRrfWeights. At a flat 2.0 a
|
|
1088
|
-
* lex line that only matched some of a question's words outranked the vector list, and a decoy
|
|
1089
|
-
* sharing a few of those words took rank 1 (test/fixtures/ko-vault/BASELINE.md 2026-09-27).
|
|
1090
|
-
*/
|
|
1091
1105
|
export declare function getStructuredRrfWeights(rankedListMeta: RankedListMeta[]): number[];
|
|
1092
1106
|
/**
|
|
1093
1107
|
* Blend a reranker score with a linear retrieval-position tiebreak.
|
|
@@ -1112,6 +1126,41 @@ export declare function getStructuredRrfWeights(rankedListMeta: RankedListMeta[]
|
|
|
1112
1126
|
*/
|
|
1113
1127
|
export declare function blendRerankScore(rrfRank: number, rerankScore: number, candidateLimit: number): number;
|
|
1114
1128
|
export type DeadlineStage = "expand" | "rerank";
|
|
1129
|
+
/**
|
|
1130
|
+
* Bound on the model-load wait inside runWithDeadline: a load that never
|
|
1131
|
+
* reports ready must not hang the query. QMD_LLM_LOAD_TIMEOUT_MS sets it
|
|
1132
|
+
* (positive ms, 0 = wait forever, anything else = the default).
|
|
1133
|
+
*/
|
|
1134
|
+
export declare const DEFAULT_LOAD_TIMEOUT_MS = 120000;
|
|
1135
|
+
/** Load-timeout resolver for runWithDeadline; exported for tests. */
|
|
1136
|
+
export declare function resolveLoadTimeoutMs(): number;
|
|
1137
|
+
/** Rerank `chunks`, or null when the reranker misses `deadlineMs` (the caller then answers in RRF order). */
|
|
1138
|
+
export declare const RERANK_CASCADE_FIRST_BATCH = 15;
|
|
1139
|
+
/** Skip the tail past the first batch when the top-2 rerank margin reaches this. */
|
|
1140
|
+
export declare const RERANK_CASCADE_MARGIN = 0.08;
|
|
1141
|
+
/**
|
|
1142
|
+
* QMD_RERANK_CASCADE=0 (or false/off/no) restores single-batch rerank;
|
|
1143
|
+
* anything else (including unset) cascades. Kill switch for measurement and
|
|
1144
|
+
* operations; exported for tests.
|
|
1145
|
+
*/
|
|
1146
|
+
export declare function isRerankCascadeEnabled(): boolean;
|
|
1147
|
+
/**
|
|
1148
|
+
* True when the first cascade batch's top-2 margin is too tight to skip the
|
|
1149
|
+
* tail: a contested top means an unscored document could still win the blend.
|
|
1150
|
+
* A batch of fewer than two holds nothing to contest.
|
|
1151
|
+
*/
|
|
1152
|
+
export declare function cascadeNeedsSecondBatch(sortedScores: number[], margin?: number): boolean;
|
|
1153
|
+
export declare function rerankWithDeadline(store: Store, query: string, chunks: {
|
|
1154
|
+
file: string;
|
|
1155
|
+
text: string;
|
|
1156
|
+
}[], intent: string | undefined, options: {
|
|
1157
|
+
rerankMaxDocTokens?: number;
|
|
1158
|
+
rerankDeadlineMs?: number;
|
|
1159
|
+
hooks?: SearchHooks;
|
|
1160
|
+
} | undefined): Promise<{
|
|
1161
|
+
file: string;
|
|
1162
|
+
score: number;
|
|
1163
|
+
}[] | null>;
|
|
1115
1164
|
/**
|
|
1116
1165
|
* Hybrid search: BM25 + vector + query expansion + RRF + chunked reranking.
|
|
1117
1166
|
*
|
package/dist/store.js
CHANGED
|
@@ -3889,18 +3889,88 @@ export async function rerank(query, documents, model = DEFAULT_RERANK_MODEL, db,
|
|
|
3889
3889
|
// =============================================================================
|
|
3890
3890
|
// Reciprocal Rank Fusion
|
|
3891
3891
|
// =============================================================================
|
|
3892
|
-
|
|
3892
|
+
const COVERAGE_HANGUL_RUN_PATTERN = /\p{Script=Hangul}+/gu;
|
|
3893
|
+
const COVERAGE_LATIN_RUN_PATTERN = /[a-z0-9]+/gi;
|
|
3894
|
+
/**
|
|
3895
|
+
* Fraction of the query's units present in the document body, used to weight a
|
|
3896
|
+
* relaxed (any-word) FTS list per document instead of one global constant
|
|
3897
|
+
* (BASELINE.md 2026-10-09 follow-up: a global weight cannot split the mirrored
|
|
3898
|
+
* tie of `hmu-05` vs `sem-08`).
|
|
3899
|
+
*
|
|
3900
|
+
* Units mirror the rerank chunk scorer: one per Hangul run of two or more
|
|
3901
|
+
* syllables (a run of three or more also matches on any shared syllable
|
|
3902
|
+
* bigram, so a particle-suffixed word meets its stem), one per Latin/digit run
|
|
3903
|
+
* longer than two characters. Single syllables and short tokens carry no
|
|
3904
|
+
* signal and are skipped. A query with no units covers everything (1.0) so it
|
|
3905
|
+
* never punishes.
|
|
3906
|
+
*/
|
|
3907
|
+
export function relaxedDocCoverage(query, body) {
|
|
3908
|
+
const lower = body.toLowerCase();
|
|
3909
|
+
let total = 0;
|
|
3910
|
+
let matched = 0;
|
|
3911
|
+
for (const token of query.toLowerCase().split(/\s+/)) {
|
|
3912
|
+
if (!token)
|
|
3913
|
+
continue;
|
|
3914
|
+
for (const run of token.match(COVERAGE_HANGUL_RUN_PATTERN) ?? []) {
|
|
3915
|
+
const syllables = Array.from(run);
|
|
3916
|
+
if (syllables.length < 2)
|
|
3917
|
+
continue;
|
|
3918
|
+
total++;
|
|
3919
|
+
if (lower.includes(run)) {
|
|
3920
|
+
matched++;
|
|
3921
|
+
continue;
|
|
3922
|
+
}
|
|
3923
|
+
if (syllables.length >= 3) {
|
|
3924
|
+
for (let i = 0; i + 1 < syllables.length; i++) {
|
|
3925
|
+
if (lower.includes(syllables[i] + syllables[i + 1])) {
|
|
3926
|
+
matched++;
|
|
3927
|
+
break;
|
|
3928
|
+
}
|
|
3929
|
+
}
|
|
3930
|
+
}
|
|
3931
|
+
}
|
|
3932
|
+
for (const run of token.match(COVERAGE_LATIN_RUN_PATTERN) ?? []) {
|
|
3933
|
+
if (run.length <= 2)
|
|
3934
|
+
continue;
|
|
3935
|
+
total++;
|
|
3936
|
+
if (lower.includes(run))
|
|
3937
|
+
matched++;
|
|
3938
|
+
}
|
|
3939
|
+
}
|
|
3940
|
+
return total === 0 ? 1 : matched / total;
|
|
3941
|
+
}
|
|
3942
|
+
/**
|
|
3943
|
+
* Per-document RRF multipliers, aligned with `rankedLists`: a relaxed list
|
|
3944
|
+
* scales each document by `0.5 + 0.5 × coverage`, every other list is untouched
|
|
3945
|
+
* (`undefined` entry = multiplier 1.0). The 0.5 floor keeps a zero-coverage
|
|
3946
|
+
* candidate in the pool instead of erasing it.
|
|
3947
|
+
*/
|
|
3948
|
+
export function rrfDocMultipliers(rankedLists, rankedListMeta) {
|
|
3949
|
+
return rankedLists.map((list, i) => {
|
|
3950
|
+
const meta = rankedListMeta[i];
|
|
3951
|
+
if (!meta?.relaxed)
|
|
3952
|
+
return undefined;
|
|
3953
|
+
const multipliers = new Map();
|
|
3954
|
+
for (const r of list) {
|
|
3955
|
+
multipliers.set(r.file, 0.5 + 0.5 * relaxedDocCoverage(meta.query, r.body));
|
|
3956
|
+
}
|
|
3957
|
+
return multipliers;
|
|
3958
|
+
});
|
|
3959
|
+
}
|
|
3960
|
+
export function reciprocalRankFusion(resultLists, weights = [], k = 60, docMultipliers = []) {
|
|
3893
3961
|
const scores = new Map();
|
|
3894
3962
|
for (let listIdx = 0; listIdx < resultLists.length; listIdx++) {
|
|
3895
3963
|
const list = resultLists[listIdx];
|
|
3896
3964
|
if (!list)
|
|
3897
3965
|
continue;
|
|
3898
3966
|
const weight = weights[listIdx] ?? 1.0;
|
|
3967
|
+
const multipliers = docMultipliers[listIdx];
|
|
3899
3968
|
for (let rank = 0; rank < list.length; rank++) {
|
|
3900
3969
|
const result = list[rank];
|
|
3901
3970
|
if (!result)
|
|
3902
3971
|
continue;
|
|
3903
|
-
const
|
|
3972
|
+
const multiplier = multipliers?.get(result.file) ?? 1.0;
|
|
3973
|
+
const rrfContribution = (weight * multiplier) / (k + rank + 1);
|
|
3904
3974
|
const existing = scores.get(result.file);
|
|
3905
3975
|
if (existing) {
|
|
3906
3976
|
existing.rrfScore += rrfContribution;
|
|
@@ -3931,13 +4001,14 @@ export function reciprocalRankFusion(resultLists, weights = [], k = 60) {
|
|
|
3931
4001
|
/**
|
|
3932
4002
|
* Build per-document RRF contribution traces for explain/debug output.
|
|
3933
4003
|
*/
|
|
3934
|
-
export function buildRrfTrace(resultLists, weights = [], listMeta = [], k = 60) {
|
|
4004
|
+
export function buildRrfTrace(resultLists, weights = [], listMeta = [], k = 60, docMultipliers = []) {
|
|
3935
4005
|
const traces = new Map();
|
|
3936
4006
|
for (let listIdx = 0; listIdx < resultLists.length; listIdx++) {
|
|
3937
4007
|
const list = resultLists[listIdx];
|
|
3938
4008
|
if (!list)
|
|
3939
4009
|
continue;
|
|
3940
4010
|
const weight = weights[listIdx] ?? 1.0;
|
|
4011
|
+
const multipliers = docMultipliers[listIdx];
|
|
3941
4012
|
const meta = listMeta[listIdx] ?? {
|
|
3942
4013
|
source: "fts",
|
|
3943
4014
|
queryType: "original",
|
|
@@ -3948,7 +4019,8 @@ export function buildRrfTrace(resultLists, weights = [], listMeta = [], k = 60)
|
|
|
3948
4019
|
if (!result)
|
|
3949
4020
|
continue;
|
|
3950
4021
|
const rank = rank0 + 1; // 1-indexed rank for explain output
|
|
3951
|
-
const
|
|
4022
|
+
const multiplier = multipliers?.get(result.file) ?? 1.0;
|
|
4023
|
+
const contribution = (weight * multiplier) / (k + rank);
|
|
3952
4024
|
const existing = traces.get(result.file);
|
|
3953
4025
|
const detail = {
|
|
3954
4026
|
listIndex: listIdx,
|
|
@@ -3957,6 +4029,7 @@ export function buildRrfTrace(resultLists, weights = [], listMeta = [], k = 60)
|
|
|
3957
4029
|
query: meta.query,
|
|
3958
4030
|
rank,
|
|
3959
4031
|
weight,
|
|
4032
|
+
multiplier,
|
|
3960
4033
|
backendScore: result.score,
|
|
3961
4034
|
rrfContribution: contribution,
|
|
3962
4035
|
};
|
|
@@ -4601,11 +4674,15 @@ export function getHybridRrfWeights(rankedListMeta) {
|
|
|
4601
4674
|
* ko-qmd: a relaxed FTS list counts half here too, as in getHybridRrfWeights. At a flat 2.0 a
|
|
4602
4675
|
* lex line that only matched some of a question's words outranked the vector list, and a decoy
|
|
4603
4676
|
* sharing a few of those words took rank 1 (test/fixtures/ko-vault/BASELINE.md 2026-09-27).
|
|
4677
|
+
* ko-qmd: a relaxed first list forfeits the positional boost and halves again (0.25) — even
|
|
4678
|
+
* halved, its rank 1 still tied a vector rank 1 with a lex rank 4 (hmu-05, 2026-10-09 probe).
|
|
4604
4679
|
*/
|
|
4680
|
+
const RELAXED_FIRST_LIST_WEIGHT = 0.25;
|
|
4605
4681
|
export function getStructuredRrfWeights(rankedListMeta) {
|
|
4606
4682
|
return rankedListMeta.map((meta, i) => {
|
|
4607
|
-
|
|
4608
|
-
|
|
4683
|
+
if (meta.relaxed)
|
|
4684
|
+
return i === 0 ? RELAXED_FIRST_LIST_WEIGHT : RELAXED_LIST_WEIGHT;
|
|
4685
|
+
return i === 0 ? 2.0 : 1.0;
|
|
4609
4686
|
});
|
|
4610
4687
|
}
|
|
4611
4688
|
/**
|
|
@@ -4663,26 +4740,45 @@ async function withDeadline(promise, deadlineMs) {
|
|
|
4663
4740
|
* skipped instead: this caps the backlog at one abandoned call per stage instead of one per query.
|
|
4664
4741
|
*/
|
|
4665
4742
|
const abandoned = new WeakMap();
|
|
4743
|
+
/**
|
|
4744
|
+
* Bound on the model-load wait inside runWithDeadline: a load that never
|
|
4745
|
+
* reports ready must not hang the query. QMD_LLM_LOAD_TIMEOUT_MS sets it
|
|
4746
|
+
* (positive ms, 0 = wait forever, anything else = the default).
|
|
4747
|
+
*/
|
|
4748
|
+
export const DEFAULT_LOAD_TIMEOUT_MS = 120_000;
|
|
4749
|
+
/** Load-timeout resolver for runWithDeadline; exported for tests. */
|
|
4750
|
+
export function resolveLoadTimeoutMs() {
|
|
4751
|
+
const raw = process.env.QMD_LLM_LOAD_TIMEOUT_MS?.trim();
|
|
4752
|
+
if (raw === undefined || raw === "")
|
|
4753
|
+
return DEFAULT_LOAD_TIMEOUT_MS;
|
|
4754
|
+
return /^\d+$/.test(raw) ? Number(raw) : DEFAULT_LOAD_TIMEOUT_MS;
|
|
4755
|
+
}
|
|
4666
4756
|
/**
|
|
4667
4757
|
* Run one LLM stage under a deadline. The clock starts once the stage's model is loaded
|
|
4668
4758
|
* (`onModelReady`): the deadline bounds computation, not a cold load, which is reported through
|
|
4669
|
-
* `onModelLoad`.
|
|
4670
|
-
*
|
|
4759
|
+
* `onModelLoad`. The load wait itself is bounded separately by resolveLoadTimeoutMs, so a load
|
|
4760
|
+
* that never reports ready times out instead of hanging. TIMED_OUT when either bound passes,
|
|
4761
|
+
* or at once while an earlier abandoned call of the stage is still running.
|
|
4762
|
+
* Unset or 0 deadline = plain await (still under the load bound).
|
|
4671
4763
|
*/
|
|
4672
4764
|
async function runWithDeadline(store, stage, deadlineMs, hooks, start) {
|
|
4673
4765
|
const started = Date.now();
|
|
4674
4766
|
let loadedAt;
|
|
4675
4767
|
const reportLoad = () => { if (loadedAt !== undefined)
|
|
4676
4768
|
hooks?.onModelLoad?.(stage, loadedAt - started); };
|
|
4677
|
-
if (!deadlineMs || deadlineMs <= 0) {
|
|
4678
|
-
const value = await start(() => { loadedAt = Date.now(); });
|
|
4679
|
-
reportLoad();
|
|
4680
|
-
return value;
|
|
4681
|
-
}
|
|
4682
4769
|
const byStage = abandoned.get(store) ?? { expand: { running: 0, warned: false }, rerank: { running: 0, warned: false } };
|
|
4683
4770
|
abandoned.set(store, byStage);
|
|
4684
4771
|
const state = byStage[stage];
|
|
4685
|
-
|
|
4772
|
+
const abandon = () => {
|
|
4773
|
+
state.running++;
|
|
4774
|
+
const settle = () => {
|
|
4775
|
+
state.running--;
|
|
4776
|
+
if (state.running === 0)
|
|
4777
|
+
state.warned = false;
|
|
4778
|
+
};
|
|
4779
|
+
pending.then(settle, settle);
|
|
4780
|
+
};
|
|
4781
|
+
if (deadlineMs && deadlineMs > 0 && state.running > 0) {
|
|
4686
4782
|
if (!state.warned) {
|
|
4687
4783
|
state.warned = true;
|
|
4688
4784
|
process.stderr.write(`qmd: a ${stage} abandoned at its deadline is still running — answering without ${stage} until it finishes\n`);
|
|
@@ -4693,30 +4789,101 @@ async function runWithDeadline(store, stage, deadlineMs, hooks, start) {
|
|
|
4693
4789
|
let markReady = () => { };
|
|
4694
4790
|
const ready = new Promise((resolve) => { markReady = () => { loadedAt = Date.now(); resolve(); }; });
|
|
4695
4791
|
const pending = start(markReady);
|
|
4696
|
-
//
|
|
4697
|
-
|
|
4792
|
+
// Bound the load wait itself: neither a stuck load nor a call that finishes
|
|
4793
|
+
// without reporting ready may hold the query past the load timeout. The
|
|
4794
|
+
// timer is cleared on every settle so idle processes keep no handle.
|
|
4795
|
+
const loadTimeoutMs = resolveLoadTimeoutMs();
|
|
4796
|
+
let loadTimer;
|
|
4797
|
+
const loadTimedOut = new Promise((resolve) => {
|
|
4798
|
+
if (loadTimeoutMs > 0) {
|
|
4799
|
+
loadTimer = setTimeout(() => resolve("load-timeout"), loadTimeoutMs);
|
|
4800
|
+
loadTimer.unref?.();
|
|
4801
|
+
}
|
|
4802
|
+
});
|
|
4803
|
+
const settled = await Promise.race([
|
|
4804
|
+
ready.then(() => "ready"),
|
|
4805
|
+
loadTimedOut,
|
|
4806
|
+
pending.then(() => "done", () => "done"),
|
|
4807
|
+
]);
|
|
4808
|
+
clearTimeout(loadTimer);
|
|
4809
|
+
if (settled === "load-timeout") {
|
|
4810
|
+
abandon();
|
|
4811
|
+
hooks?.onModelLoad?.(stage, Date.now() - started);
|
|
4812
|
+
hooks?.onDeadline?.(stage, Date.now() - started);
|
|
4813
|
+
return TIMED_OUT;
|
|
4814
|
+
}
|
|
4698
4815
|
reportLoad();
|
|
4699
|
-
if (
|
|
4816
|
+
if (settled === "done")
|
|
4817
|
+
return pending;
|
|
4818
|
+
if (!deadlineMs || deadlineMs <= 0)
|
|
4700
4819
|
return pending;
|
|
4701
4820
|
const result = await withDeadline(pending, deadlineMs);
|
|
4702
4821
|
if (result === TIMED_OUT) {
|
|
4703
|
-
|
|
4704
|
-
const settle = () => {
|
|
4705
|
-
state.running--;
|
|
4706
|
-
if (state.running === 0)
|
|
4707
|
-
state.warned = false;
|
|
4708
|
-
};
|
|
4709
|
-
pending.then(settle, settle);
|
|
4822
|
+
abandon();
|
|
4710
4823
|
hooks?.onDeadline?.(stage, Date.now() - (loadedAt ?? started));
|
|
4711
4824
|
}
|
|
4712
4825
|
return result;
|
|
4713
4826
|
}
|
|
4714
4827
|
/** Rerank `chunks`, or null when the reranker misses `deadlineMs` (the caller then answers in RRF order). */
|
|
4715
|
-
|
|
4828
|
+
export const RERANK_CASCADE_FIRST_BATCH = 15;
|
|
4829
|
+
/** Skip the tail past the first batch when the top-2 rerank margin reaches this. */
|
|
4830
|
+
export const RERANK_CASCADE_MARGIN = 0.08;
|
|
4831
|
+
/**
|
|
4832
|
+
* QMD_RERANK_CASCADE=0 (or false/off/no) restores single-batch rerank;
|
|
4833
|
+
* anything else (including unset) cascades. Kill switch for measurement and
|
|
4834
|
+
* operations; exported for tests.
|
|
4835
|
+
*/
|
|
4836
|
+
export function isRerankCascadeEnabled() {
|
|
4837
|
+
const raw = process.env.QMD_RERANK_CASCADE?.trim().toLowerCase();
|
|
4838
|
+
return raw !== "0" && raw !== "false" && raw !== "off" && raw !== "no";
|
|
4839
|
+
}
|
|
4840
|
+
/**
|
|
4841
|
+
* True when the first cascade batch's top-2 margin is too tight to skip the
|
|
4842
|
+
* tail: a contested top means an unscored document could still win the blend.
|
|
4843
|
+
* A batch of fewer than two holds nothing to contest.
|
|
4844
|
+
*/
|
|
4845
|
+
export function cascadeNeedsSecondBatch(sortedScores, margin = RERANK_CASCADE_MARGIN) {
|
|
4846
|
+
if (sortedScores.length < 2)
|
|
4847
|
+
return false;
|
|
4848
|
+
const [top, second] = [sortedScores[0], sortedScores[1]];
|
|
4849
|
+
if (!Number.isFinite(top) || !Number.isFinite(second))
|
|
4850
|
+
return true;
|
|
4851
|
+
return top - second < margin;
|
|
4852
|
+
}
|
|
4853
|
+
export async function rerankWithDeadline(store, query, chunks, intent, options) {
|
|
4716
4854
|
const hooks = options?.hooks;
|
|
4717
4855
|
hooks?.onRerankStart?.(chunks.length);
|
|
4718
4856
|
const start = Date.now();
|
|
4719
|
-
const
|
|
4857
|
+
const scoreBatch = (docs, deadlineMs) => runWithDeadline(store, "rerank", deadlineMs, hooks, (onModelReady) => store.rerank(query, docs, undefined, intent, { maxDocTokens: options?.rerankMaxDocTokens, onModelReady }));
|
|
4858
|
+
// Cascade: score the RRF head first. On a decisive top margin the tail keeps
|
|
4859
|
+
// score 0 (RRF position tiebreak only) instead of paying for 25 more
|
|
4860
|
+
// cross-encoder pairs; a contested top scores everything, as before.
|
|
4861
|
+
if (isRerankCascadeEnabled() && chunks.length > RERANK_CASCADE_FIRST_BATCH) {
|
|
4862
|
+
const first = chunks.slice(0, RERANK_CASCADE_FIRST_BATCH);
|
|
4863
|
+
const rest = chunks.slice(RERANK_CASCADE_FIRST_BATCH);
|
|
4864
|
+
const firstScored = await scoreBatch(first, options?.rerankDeadlineMs);
|
|
4865
|
+
if (firstScored === TIMED_OUT)
|
|
4866
|
+
return null;
|
|
4867
|
+
const headroom = [...firstScored].sort((a, b) => b.score - a.score).map((r) => r.score);
|
|
4868
|
+
if (!cascadeNeedsSecondBatch(headroom)) {
|
|
4869
|
+
hooks?.onRerankDone?.(Date.now() - start);
|
|
4870
|
+
return [...firstScored, ...rest.map((d) => ({ file: d.file, score: 0 }))];
|
|
4871
|
+
}
|
|
4872
|
+
// The second batch shares the caller's deadline: it gets only the remaining
|
|
4873
|
+
// budget, so the worst case stays one deadline, not two. An exhausted budget
|
|
4874
|
+
// is the same as a timeout — the caller falls back to RRF order.
|
|
4875
|
+
// Same "no deadline" rule as runWithDeadline: unset or <= 0 waits unbounded.
|
|
4876
|
+
const deadlineMs = options?.rerankDeadlineMs;
|
|
4877
|
+
const remaining = deadlineMs && deadlineMs > 0 ? deadlineMs - (Date.now() - start) : undefined;
|
|
4878
|
+
if (remaining !== undefined && remaining <= 0)
|
|
4879
|
+
return null;
|
|
4880
|
+
const restScored = await scoreBatch(rest, remaining);
|
|
4881
|
+
if (restScored === TIMED_OUT)
|
|
4882
|
+
return null;
|
|
4883
|
+
hooks?.onRerankDone?.(Date.now() - start);
|
|
4884
|
+
return [...firstScored, ...restScored];
|
|
4885
|
+
}
|
|
4886
|
+
const reranked = await scoreBatch(chunks, options?.rerankDeadlineMs);
|
|
4720
4887
|
if (reranked === TIMED_OUT)
|
|
4721
4888
|
return null;
|
|
4722
4889
|
hooks?.onRerankDone?.(Date.now() - start);
|
|
@@ -4760,6 +4927,9 @@ const rerankDocuments = (candidates, docChunkMap) => candidates.flatMap((cand) =
|
|
|
4760
4927
|
*/
|
|
4761
4928
|
export async function hybridQuery(store, rawQuery, options) {
|
|
4762
4929
|
// ko-qmd: "X 말고 Y" names X to exclude — every list and the reranker see only Y.
|
|
4930
|
+
// (Tried demoting X with lex -terms: falsified 2026-10-09 — the expected note
|
|
4931
|
+
// often discusses X itself, e.g. bm25-ranking.md mentions TF-IDF, so NOT
|
|
4932
|
+
// removes the answer along with the decoy.)
|
|
4763
4933
|
const query = stripHangulNegation(rawQuery);
|
|
4764
4934
|
const limit = options?.limit ?? 10;
|
|
4765
4935
|
const minScore = options?.minScore ?? 0;
|
|
@@ -4899,9 +5069,11 @@ export async function hybridQuery(store, rawQuery, options) {
|
|
|
4899
5069
|
}
|
|
4900
5070
|
// Step 4: RRF fusion — original-query FTS and vector lists get 2x weight;
|
|
4901
5071
|
// expansion-derived lists stay at 1x independent of insertion order.
|
|
5072
|
+
// A relaxed list additionally scales each document by its query coverage.
|
|
4902
5073
|
const weights = getHybridRrfWeights(rankedListMeta);
|
|
4903
|
-
const
|
|
4904
|
-
const
|
|
5074
|
+
const docMultipliers = rrfDocMultipliers(rankedLists, rankedListMeta);
|
|
5075
|
+
const fused = reciprocalRankFusion(rankedLists, weights, 60, docMultipliers);
|
|
5076
|
+
const rrfTraceByFile = explain ? buildRrfTrace(rankedLists, weights, rankedListMeta, 60, docMultipliers) : null;
|
|
4905
5077
|
const candidates = fused.slice(0, candidateLimit);
|
|
4906
5078
|
if (candidates.length === 0)
|
|
4907
5079
|
return [];
|
|
@@ -5197,10 +5369,11 @@ export async function structuredSearch(store, rawSearches, options) {
|
|
|
5197
5369
|
if (rankedLists.length === 0)
|
|
5198
5370
|
return [];
|
|
5199
5371
|
// Step 3: RRF fusion — first list gets 2x weight (assume caller ordered by importance),
|
|
5200
|
-
// a relaxed FTS list half of its slot's weight
|
|
5372
|
+
// a relaxed FTS list half of its slot's weight, scaled per document by query coverage
|
|
5201
5373
|
const weights = getStructuredRrfWeights(rankedListMeta);
|
|
5202
|
-
const
|
|
5203
|
-
const
|
|
5374
|
+
const docMultipliers = rrfDocMultipliers(rankedLists, rankedListMeta);
|
|
5375
|
+
const fused = reciprocalRankFusion(rankedLists, weights, 60, docMultipliers);
|
|
5376
|
+
const rrfTraceByFile = explain ? buildRrfTrace(rankedLists, weights, rankedListMeta, 60, docMultipliers) : null;
|
|
5204
5377
|
const candidates = fused.slice(0, candidateLimit);
|
|
5205
5378
|
if (candidates.length === 0)
|
|
5206
5379
|
return [];
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ko-qmd",
|
|
3
|
-
"version": "2.8.3-ko.
|
|
3
|
+
"version": "2.8.3-ko.9",
|
|
4
4
|
"description": "Local hybrid search for Korean Markdown notes and Obsidian vaults — BM25 with Korean particle stripping, Hangul bigram FTS, Qwen3 vector search and LLM reranking, as CLI, MCP server and SDK",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -60,7 +60,7 @@
|
|
|
60
60
|
"@modelcontextprotocol/server": "2.0.0",
|
|
61
61
|
"better-sqlite3": "^13.0.3",
|
|
62
62
|
"fast-glob": "3.3.3",
|
|
63
|
-
"node-llama-cpp": "3.
|
|
63
|
+
"node-llama-cpp": "3.22.1",
|
|
64
64
|
"picomatch": "4.0.5",
|
|
65
65
|
"sqlite-vec": "0.1.9",
|
|
66
66
|
"tree-sitter-go": "0.25.0",
|