agent-knowledge 1.4.2 → 1.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +66 -0
- package/README.md +3 -3
- package/agent-desk-plugin.json +1 -1
- package/dist/knowledge/search.js +2 -2
- package/dist/knowledge/search.js.map +1 -1
- package/dist/search/bm25.d.ts +28 -0
- package/dist/search/bm25.d.ts.map +1 -0
- package/dist/search/bm25.js +216 -0
- package/dist/search/bm25.js.map +1 -0
- package/dist/sessions/scopes.d.ts.map +1 -1
- package/dist/sessions/scopes.js +3 -2
- package/dist/sessions/scopes.js.map +1 -1
- package/dist/sessions/search.d.ts.map +1 -1
- package/dist/sessions/search.js +3 -2
- package/dist/sessions/search.js.map +1 -1
- package/package.json +3 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,71 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.5.1 (2026-04-09)
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- **Streaming JSON parser in `bench/longmemeval.ts`**. The benchmark now uses an async iterator (`stream-json`) for files >384 MB so the 2.6 GB `longmemeval_m` split can be benchmarked without hitting Node's ~512 MB string-length limit. Files under the threshold still use the original `readFileSync + JSON.parse` fast path with byte-identical results.
|
|
8
|
+
- **`bench/longmemeval_m` results.** First time we've measured the harder split (500 distractor sessions per question, 10× more than `_s`):
|
|
9
|
+
|
|
10
|
+
| Mode | R@1 | R@5 | R@10 |
|
|
11
|
+
| ---------------------------- | ----- | --------- | ----- |
|
|
12
|
+
| 1.5 — BM25 + boosts (sparse) | 65.6% | **86.0%** | 92.4% |
|
|
13
|
+
| 1.5 — BM25 + semantic hybrid | 65.4% | **88.4%** | 92.2% |
|
|
14
|
+
|
|
15
|
+
- **Validation against the LongMemEval paper's official `flat-bm25`** (`bench/paper_bm25_eval.py`). We re-ran the paper's exact corpus construction (user-only text per session), tokenization (`doc.split(" ")`), and `rank_bm25.BM25Okapi` defaults on the same data, with the paper's own `recall_any` evaluator from `src/retrieval/eval_utils.py`:
|
|
16
|
+
|
|
17
|
+
| Split | Paper `flat-bm25` | agent-knowledge 1.5 BM25 | agent-knowledge 1.5 hybrid |
|
|
18
|
+
| --------------- | ----------------- | ------------------------ | -------------------------- |
|
|
19
|
+
| `longmemeval_s` | 88.6% | **97.2% (+8.6pp)** | **98.8% (+10.2pp)** |
|
|
20
|
+
| `longmemeval_m` | 75.2% | **86.0% (+10.8pp)** | **88.4% (+13.2pp)** |
|
|
21
|
+
|
|
22
|
+
The improvement comes from three small but cumulative differences vs the paper's BM25 setup: (1) we index both user and assistant turns instead of user-only, (2) we lowercase + strip stopwords instead of splitting on whitespace, (3) we use `k1 = 1.2` instead of the `BM25Okapi` default `1.5`.
|
|
23
|
+
|
|
24
|
+
- `stream-json` and `@types/stream-json` as devDependencies (used by the bench only; not in the runtime).
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- `bench/longmemeval.ts` main loop is now an `async for await` over a streaming dataset iterator. The earlier eager-load path (`readFileSync` then `JSON.parse`) is preserved for files under 384 MB so existing reproduction commands stay byte-identical.
|
|
29
|
+
|
|
30
|
+
## 1.5.0 (2026-04-08)
|
|
31
|
+
|
|
32
|
+
### Changed
|
|
33
|
+
|
|
34
|
+
- **Sparse ranker swapped from naive TF-IDF cosine to Okapi BM25** (`src/search/bm25.ts`, `k1=1.2`, `b=0.75`). The TF-IDF cosine ranker under-weighted long sessions because cosine normalization punishes document length. BM25's length normalization fixes this for the long, multi-turn sessions LongMemEval is built on. Wired into all three production search call sites: `src/sessions/search.ts`, `src/sessions/scopes.ts`, `src/knowledge/search.ts`. `src/knowledge/consolidate.ts` keeps TF-IDF for doc-to-doc duplicate detection (a different use case).
|
|
35
|
+
|
|
36
|
+
### Benchmark — LongMemEval (full 500 questions, `longmemeval_s_cleaned`)
|
|
37
|
+
|
|
38
|
+
| Mode | R@1 | R@5 | R@10 |
|
|
39
|
+
| ----------------------------------------------- | --------- | --------- | --------- |
|
|
40
|
+
| 1.4.2 — TF-IDF + boosts (sparse) | 59.8% | 83.8% | 91.2% |
|
|
41
|
+
| **1.5.0 — BM25 + boosts (sparse)** | **87.6%** | **97.2%** | **98.4%** |
|
|
42
|
+
| **1.5.0 — BM25 + semantic (hybrid, alpha=0.3)** | **89.6%** | **98.8%** | **99.6%** |
|
|
43
|
+
|
|
44
|
+
Per-category R@5 deltas (sparse, vs 1.4.2):
|
|
45
|
+
|
|
46
|
+
| Category | 1.4.2 | 1.5.0 | Δ |
|
|
47
|
+
| ------------------------- | --------- | --------- | ----------- |
|
|
48
|
+
| single-session-user | 90.0% | 100.0% | +10.0pp |
|
|
49
|
+
| single-session-assistant | 87.5% | 100.0% | +12.5pp |
|
|
50
|
+
| single-session-preference | 33.3% | 86.7% | **+53.4pp** |
|
|
51
|
+
| multi-session | 84.2% | 97.0% | +12.8pp |
|
|
52
|
+
| temporal-reasoning | 84.2% | 95.5% | +11.3pp |
|
|
53
|
+
| knowledge-update | 93.6% | 100.0% | +6.4pp |
|
|
54
|
+
| **OVERALL** | **83.8%** | **97.2%** | **+13.4pp** |
|
|
55
|
+
|
|
56
|
+
The biggest jump is `single-session-preference` (33.3% → 86.7%), which had been the dead category in 1.4.2. Preference questions are about long sessions where the user states a habit indirectly; cosine normalization had been ranking those long documents too low.
|
|
57
|
+
|
|
58
|
+
Reproduce: `npx tsx bench/longmemeval.ts --boosts --ranker bm25` (sparse) or `--hybrid --boosts --ranker bm25` (hybrid, ~70 min).
|
|
59
|
+
|
|
60
|
+
### Added
|
|
61
|
+
|
|
62
|
+
- `bench/longmemeval.ts --ranker {tfidf|bm25}` flag (default `tfidf` so the historical baseline stays reproducible).
|
|
63
|
+
- `tests/bm25.test.ts` — 9 unit tests for `BM25Index` covering ranking, length normalization, dedup on re-add, and edge cases.
|
|
64
|
+
|
|
65
|
+
### Notes
|
|
66
|
+
|
|
67
|
+
- Two changes were attempted and reverted because they failed their measured gates: a regex-based preference-extraction synthetic-chunk pass (no movement on the preference category — the haystack distractors also contain preference-shaped statements, so vocabulary injection added noise without changing rank) and Reciprocal Rank Fusion as the hybrid blender (alpha-weighted blend beat RRF by 1pp on a 100q sample). Both removed; no dead code shipped.
|
|
68
|
+
|
|
3
69
|
## 1.4.2 (2026-04-08)
|
|
4
70
|
|
|
5
71
|
### Added
|
package/README.md
CHANGED
|
@@ -2,13 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
[](LICENSE)
|
|
4
4
|
[](https://nodejs.org)
|
|
5
|
-
[]()
|
|
6
6
|
[]()
|
|
7
|
-
[]()
|
|
8
8
|
|
|
9
9
|
**Cross-session memory and recall for AI coding assistants** -- works with Claude Code, Cursor, OpenCode, Cline, Continue.dev, and Aider out of the box. Git-synced knowledge base, hybrid semantic+TF-IDF search, auto-distillation with secrets scrubbing.
|
|
10
10
|
|
|
11
|
-
**Benchmark:** **R@5 =
|
|
11
|
+
**Benchmark:** **R@5 = 97.2% (sparse) / 98.8% (hybrid)** on `longmemeval_s` and **86.0% (sparse) / 88.4% (hybrid)** on the harder `longmemeval_m` split — the public LongMemEval academic benchmark (Wu et al. 2024, ICLR 2025), full 500 questions per split, no LLM, no API key, runs entirely offline. **+8.6pp to +13.2pp R@5 over the paper's official `flat-bm25` baseline** in apples-to-apples reproduction. Full per-category table, reproduction instructions, and paper-comparison details in [`bench/README.md`](bench/README.md).
|
|
12
12
|
|
|
13
13
|
<table>
|
|
14
14
|
<tr>
|
package/agent-desk-plugin.json
CHANGED
package/dist/knowledge/search.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { BM25Index } from '../search/bm25.js';
|
|
2
2
|
import { getEntryScoring, computeFinalScore } from './scoring.js';
|
|
3
3
|
import { buildExcerpt } from '../search/excerpt.js';
|
|
4
4
|
import { listEntries, readEntry } from './store.js';
|
|
@@ -29,7 +29,7 @@ function getOrBuildKnowledgeIndex(dir, category) {
|
|
|
29
29
|
continue;
|
|
30
30
|
}
|
|
31
31
|
}
|
|
32
|
-
const index = new
|
|
32
|
+
const index = new BM25Index();
|
|
33
33
|
for (const doc of documents) {
|
|
34
34
|
index.addDocument(doc.entry.path, doc.content);
|
|
35
35
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"search.js","sourceRoot":"","sources":["../../src/knowledge/search.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"search.js","sourceRoot":"","sources":["../../src/knowledge/search.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,MAAM,mBAAmB,CAAC;AAC9C,OAAO,EAAE,eAAe,EAAE,iBAAiB,EAAE,MAAM,cAAc,CAAC;AAClE,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,WAAW,EAAE,SAAS,EAAkB,MAAM,YAAY,CAAC;AAkBpE,kFAAkF;AAClF,MAAM,CAAC,MAAM,yBAAyB,GAAG,IAAI,CAAC;AAkB9C,IAAI,oBAAoB,GAA+B,IAAI,CAAC;AAC5D,MAAM,mBAAmB,GAAG,MAAM,CAAC,CAAC,4CAA4C;AAEhF,0EAA0E;AAC1E,MAAM,UAAU,6BAA6B;IAC3C,oBAAoB,GAAG,IAAI,CAAC;AAC9B,CAAC;AAED,SAAS,wBAAwB,CAC/B,GAAW,EACX,QAAiB;IAEjB,MAAM,QAAQ,GAAG,GAAG,GAAG,IAAI,QAAQ,IAAI,EAAE,EAAE,CAAC;IAC5C,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;IAEvB,IACE,oBAAoB;QACpB,GAAG,GAAG,oBAAoB,CAAC,SAAS,GAAG,mBAAmB;QAC1D,oBAAoB,CAAC,QAAQ,KAAK,QAAQ,EAC1C,CAAC;QACD,OAAO,oBAAoB,CAAC;IAC9B,CAAC;IAED,MAAM,OAAO,GAAG,WAAW,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;IAC3C,MAAM,SAAS,GAAsD,EAAE,CAAC;IAExE,KAAK,MAAM,KAAK,IAAI,OAAO,EAAE,CAAC;QAC5B,IAAI,CAAC;YACH,MAAM,EAAE,OAAO,EAAE,GAAG,SAAS,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,CAAC,CAAC;YAC/C,SAAS,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,EAAE,GAAG,KAAK,EAAE,OAAO,EAAE,EAAE,OAAO,EAAE,CAAC,CAAC;QAC5D,CAAC;QAAC,MAAM,CAAC;YACP,SAAS;QACX,CAAC;IACH,CAAC;IAED,MAAM,KAAK,GAAG,IAAI,SAAS,EAAE,CAAC;IAC9B,KAAK,MAAM,GAAG,IAAI,SAAS,EAAE,CAAC;QAC5B,KAAK,CAAC,WAAW,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,EAAE,GAAG,CAAC,OAAO,CAAC,CAAC;IACjD,CAAC;IAED,oBAAoB,GAAG,EAAE,KAAK,EAAE,SAAS,EAAE,SAAS,EAAE,GAAG,EAAE,QAAQ,EAAE,CAAC;IACtE,OAAO,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC;AAC9B,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,eAAe,CAC7B,GAAW,EACX,KAAa,EACb,UAAyB,EAAE;IAE3B,MAAM,EAAE,QAAQ,EAAE,UAAU,GAAG,EAAE,EAAE,aAAa,GAAG,KAAK,EAAE,YAAY,GAAG,QAAQ,EAAE,GAAG,OAAO,CAAC;IAE9F,mFAAmF;IACnF,8FAA8F;IAC9F,MAAM,aAAa,GAAG,YAAY,KAAK,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,QAAQ,CAAC;IACtE,MAAM,EAAE,KAAK,EAAE,SAAS,EAAE,GAAG,wBAAwB,CAAC,GAAG,EAAE,aAAa,CAAC,CAAC;IAE1E,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IAEtC,sBAAsB;IACtB,MAAM,YAAY,GAAG,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,UAAU,CAAC,CAAC;IAErD,IAAI,YAAY,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC5B,MAAM,OAAO,GAAmB,EAAE,CAAC;QACnC,MAAM,OAAO,GAAG,eAAe,EAAE,CAAC;QAClC,MAAM,UAAU,GAAG,YAAY;aAC5B,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,SAAS,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,KAAK,CAAC,CAAC,EAAE,CAAC,EAAE,KAAK,CAAC,IAAI,CAAC;aACpE,MAAM,CAAC,CAAC,CAAC,EAAe,EAAE,CAAC,CAAC,KAAK,SAAS,CAAC,CAAC;QAC/C,MAAM,MAAM,GAAG,OAAO,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC;QAE7C,KAAK,MAAM,MAAM,IAAI,YAAY,EAAE,CAAC;YAClC,MAAM,GAAG,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,KAAK,MAAM,CAAC,EAAE,CAAC,CAAC;YAC9D,IAAI,CAAC,GAAG;gBAAE,SAAS;YAEnB,MAAM,SAAS,GAAG,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;YAC7C,MAAM,QAAQ,GAAG,CAAC,SAAS,EAAE,QAAQ,IAAI,WAAW,CAGxC,CAAC;YACb,MAAM,YAAY,GAAG,SAAS,EAAE,aAAa,IAAI,IAAI,CAAC;YACtD,IAAI,UAAU,GAAG,iBAAiB,CAAC,MAAM,CAAC,KAAK,EAAE,YAAY,EAAE,QAAQ,CAAC,CAAC;YAEzE,qEAAqE;YACrE,iDAAiD;YACjD,IAAI,YAAY,KAAK,OAAO,IAAI,QAAQ,IAAI,GAAG,CAAC,KAAK,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;gBAC5E,UAAU,IAAI,yBAAyB,CAAC;YAC1C,CAAC;YAED,OAAO,CAAC,IAAI,CAAC;gBACX,KAAK,EAAE,GAAG,CAAC,KAAK;gBAChB,KAAK,EAAE,UAAU;gBACjB,OAAO,EAAE,YAAY,CAAC,GAAG,CAAC,OAAO,EAAE,KAAK,EAAE,EAAE,aAAa,EAAE,YAAY,EAAE,GAAG,EAAE,CAAC;aAChF,CAAC,CAAC;QACL,CAAC;QAED,wDAAwD;QACxD,MAAM,QAAQ,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC;QACpD,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;QAC3C,OAAO,QAAQ,CAAC;IAClB,CAAC;IAED,kDAAkD;IAClD,MAAM,KAAK,GAAG,aAAa,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC;IACzC,IAAI,KAAa,CAAC;IAClB,IAAI,CAAC;QACH,KAAK,GAAG,IAAI,MAAM,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;IACnC,CAAC;IAAC,MAAM,CAAC;QACP,KAAK,GAAG,IAAI,MAAM,CAAC,KAAK,CAAC,OAAO,CAAC,qBAAqB,EAAE,MAAM,CAAC,EAAE,KAAK,CAAC,CAAC;IAC1E,CAAC;IAED,MAAM,YAAY,GAAmB,EAAE,CAAC;IACxC,KAAK,MAAM,GAAG,IAAI,SAAS,EAAE,CAAC;QAC5B,KAAK,CAAC,SAAS,GAAG,CAAC,CAAC,CAAC,iDAAiD;QACtE,IAAI,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,EAAE,CAAC;YAC5B,YAAY,CAAC,IAAI,CAAC;gBAChB,KAAK,EAAE,GAAG,CAAC,KAAK;gBAChB,KAAK,EAAE,CAAC;gBACR,OAAO,EAAE,YAAY,CAAC,GAAG,CAAC,OAAO,EAAE,KAAK,EAAE,EAAE,aAAa,EAAE,YAAY,EAAE,GAAG,EAAE,CAAC;aAChF,CAAC,CAAC;YACH,IAAI,YAAY,CAAC,MAAM,IAAI,UAAU;gBAAE,MAAM;QAC/C,CAAC;IACH,CAAC;IAED,OAAO,YAAY,CAAC;AACtB,CAAC"}
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* BM25 ranker — Okapi BM25 over a tokenized corpus.
|
|
3
|
+
*
|
|
4
|
+
* Same shape as TfIdfIndex (addDocument / search / clear) so callers can
|
|
5
|
+
* swap rankers via configuration. Tokenizer + stopword list match
|
|
6
|
+
* tfidf.ts so the two rankers are directly comparable on the same corpus.
|
|
7
|
+
*/
|
|
8
|
+
export interface BM25Options {
|
|
9
|
+
k1?: number;
|
|
10
|
+
b?: number;
|
|
11
|
+
}
|
|
12
|
+
export declare class BM25Index {
|
|
13
|
+
private docs;
|
|
14
|
+
private docFreq;
|
|
15
|
+
private invertedIndex;
|
|
16
|
+
private totalDocs;
|
|
17
|
+
private totalLength;
|
|
18
|
+
private readonly k1;
|
|
19
|
+
private readonly b;
|
|
20
|
+
constructor(opts?: BM25Options);
|
|
21
|
+
addDocument(id: string, text: string): void;
|
|
22
|
+
search(query: string, maxResults?: number): Array<{
|
|
23
|
+
id: string;
|
|
24
|
+
score: number;
|
|
25
|
+
}>;
|
|
26
|
+
clear(): void;
|
|
27
|
+
}
|
|
28
|
+
//# sourceMappingURL=bm25.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"bm25.d.ts","sourceRoot":"","sources":["../../src/search/bm25.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAkIH,MAAM,WAAW,WAAW;IAC1B,EAAE,CAAC,EAAE,MAAM,CAAC;IACZ,CAAC,CAAC,EAAE,MAAM,CAAC;CACZ;AAED,qBAAa,SAAS;IACpB,OAAO,CAAC,IAAI,CAAoC;IAChD,OAAO,CAAC,OAAO,CAAkC;IACjD,OAAO,CAAC,aAAa,CAAuC;IAC5D,OAAO,CAAC,SAAS,CAAa;IAC9B,OAAO,CAAC,WAAW,CAAa;IAChC,OAAO,CAAC,QAAQ,CAAC,EAAE,CAAS;IAC5B,OAAO,CAAC,QAAQ,CAAC,CAAC,CAAS;gBAEf,IAAI,GAAE,WAAgB;IAKlC,WAAW,CAAC,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,IAAI;IAqC3C,MAAM,CAAC,KAAK,EAAE,MAAM,EAAE,UAAU,CAAC,EAAE,MAAM,GAAG,KAAK,CAAC;QAAE,EAAE,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC;IAgChF,KAAK,IAAI,IAAI;CAOd"}
|
|
@@ -0,0 +1,216 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* BM25 ranker — Okapi BM25 over a tokenized corpus.
|
|
3
|
+
*
|
|
4
|
+
* Same shape as TfIdfIndex (addDocument / search / clear) so callers can
|
|
5
|
+
* swap rankers via configuration. Tokenizer + stopword list match
|
|
6
|
+
* tfidf.ts so the two rankers are directly comparable on the same corpus.
|
|
7
|
+
*/
|
|
8
|
+
const STOPWORDS = new Set([
|
|
9
|
+
'a',
|
|
10
|
+
'an',
|
|
11
|
+
'the',
|
|
12
|
+
'and',
|
|
13
|
+
'or',
|
|
14
|
+
'but',
|
|
15
|
+
'not',
|
|
16
|
+
'is',
|
|
17
|
+
'are',
|
|
18
|
+
'was',
|
|
19
|
+
'were',
|
|
20
|
+
'be',
|
|
21
|
+
'been',
|
|
22
|
+
'being',
|
|
23
|
+
'have',
|
|
24
|
+
'has',
|
|
25
|
+
'had',
|
|
26
|
+
'having',
|
|
27
|
+
'do',
|
|
28
|
+
'does',
|
|
29
|
+
'did',
|
|
30
|
+
'doing',
|
|
31
|
+
'will',
|
|
32
|
+
'would',
|
|
33
|
+
'could',
|
|
34
|
+
'should',
|
|
35
|
+
'shall',
|
|
36
|
+
'may',
|
|
37
|
+
'might',
|
|
38
|
+
'can',
|
|
39
|
+
'must',
|
|
40
|
+
'to',
|
|
41
|
+
'of',
|
|
42
|
+
'in',
|
|
43
|
+
'for',
|
|
44
|
+
'on',
|
|
45
|
+
'with',
|
|
46
|
+
'at',
|
|
47
|
+
'by',
|
|
48
|
+
'from',
|
|
49
|
+
'as',
|
|
50
|
+
'into',
|
|
51
|
+
'through',
|
|
52
|
+
'during',
|
|
53
|
+
'before',
|
|
54
|
+
'after',
|
|
55
|
+
'above',
|
|
56
|
+
'below',
|
|
57
|
+
'between',
|
|
58
|
+
'out',
|
|
59
|
+
'off',
|
|
60
|
+
'over',
|
|
61
|
+
'under',
|
|
62
|
+
'again',
|
|
63
|
+
'further',
|
|
64
|
+
'then',
|
|
65
|
+
'once',
|
|
66
|
+
'here',
|
|
67
|
+
'there',
|
|
68
|
+
'when',
|
|
69
|
+
'where',
|
|
70
|
+
'why',
|
|
71
|
+
'how',
|
|
72
|
+
'all',
|
|
73
|
+
'each',
|
|
74
|
+
'every',
|
|
75
|
+
'both',
|
|
76
|
+
'few',
|
|
77
|
+
'more',
|
|
78
|
+
'most',
|
|
79
|
+
'other',
|
|
80
|
+
'some',
|
|
81
|
+
'such',
|
|
82
|
+
'no',
|
|
83
|
+
'nor',
|
|
84
|
+
'only',
|
|
85
|
+
'own',
|
|
86
|
+
'same',
|
|
87
|
+
'so',
|
|
88
|
+
'than',
|
|
89
|
+
'too',
|
|
90
|
+
'very',
|
|
91
|
+
'just',
|
|
92
|
+
'about',
|
|
93
|
+
'up',
|
|
94
|
+
'down',
|
|
95
|
+
'if',
|
|
96
|
+
'it',
|
|
97
|
+
'its',
|
|
98
|
+
'he',
|
|
99
|
+
'she',
|
|
100
|
+
'they',
|
|
101
|
+
'them',
|
|
102
|
+
'his',
|
|
103
|
+
'her',
|
|
104
|
+
'their',
|
|
105
|
+
'we',
|
|
106
|
+
'me',
|
|
107
|
+
'him',
|
|
108
|
+
'my',
|
|
109
|
+
'your',
|
|
110
|
+
'our',
|
|
111
|
+
'this',
|
|
112
|
+
'that',
|
|
113
|
+
'these',
|
|
114
|
+
'those',
|
|
115
|
+
'i',
|
|
116
|
+
'you',
|
|
117
|
+
'what',
|
|
118
|
+
'which',
|
|
119
|
+
'who',
|
|
120
|
+
'whom',
|
|
121
|
+
'am',
|
|
122
|
+
]);
|
|
123
|
+
function tokenize(text) {
|
|
124
|
+
return text
|
|
125
|
+
.toLowerCase()
|
|
126
|
+
.split(/[^a-z0-9]+/)
|
|
127
|
+
.filter((token) => token.length > 0 && !STOPWORDS.has(token));
|
|
128
|
+
}
|
|
129
|
+
export class BM25Index {
|
|
130
|
+
docs = new Map();
|
|
131
|
+
docFreq = new Map();
|
|
132
|
+
invertedIndex = new Map();
|
|
133
|
+
totalDocs = 0;
|
|
134
|
+
totalLength = 0;
|
|
135
|
+
k1;
|
|
136
|
+
b;
|
|
137
|
+
constructor(opts = {}) {
|
|
138
|
+
this.k1 = opts.k1 ?? 1.2;
|
|
139
|
+
this.b = opts.b ?? 0.75;
|
|
140
|
+
}
|
|
141
|
+
addDocument(id, text) {
|
|
142
|
+
if (this.docs.has(id)) {
|
|
143
|
+
const existing = this.docs.get(id);
|
|
144
|
+
for (const term of existing.termFreqs.keys()) {
|
|
145
|
+
const count = this.docFreq.get(term) ?? 0;
|
|
146
|
+
if (count <= 1) {
|
|
147
|
+
this.docFreq.delete(term);
|
|
148
|
+
this.invertedIndex.delete(term);
|
|
149
|
+
}
|
|
150
|
+
else {
|
|
151
|
+
this.docFreq.set(term, count - 1);
|
|
152
|
+
this.invertedIndex.get(term)?.delete(id);
|
|
153
|
+
}
|
|
154
|
+
}
|
|
155
|
+
this.totalDocs--;
|
|
156
|
+
this.totalLength -= existing.length;
|
|
157
|
+
}
|
|
158
|
+
const tokens = tokenize(text);
|
|
159
|
+
const termFreqs = new Map();
|
|
160
|
+
for (const token of tokens) {
|
|
161
|
+
termFreqs.set(token, (termFreqs.get(token) ?? 0) + 1);
|
|
162
|
+
}
|
|
163
|
+
this.docs.set(id, { termFreqs, length: tokens.length });
|
|
164
|
+
for (const term of termFreqs.keys()) {
|
|
165
|
+
this.docFreq.set(term, (this.docFreq.get(term) ?? 0) + 1);
|
|
166
|
+
if (!this.invertedIndex.has(term)) {
|
|
167
|
+
this.invertedIndex.set(term, new Set());
|
|
168
|
+
}
|
|
169
|
+
this.invertedIndex.get(term).add(id);
|
|
170
|
+
}
|
|
171
|
+
this.totalDocs++;
|
|
172
|
+
this.totalLength += tokens.length;
|
|
173
|
+
}
|
|
174
|
+
search(query, maxResults) {
|
|
175
|
+
const queryTokens = tokenize(query);
|
|
176
|
+
if (queryTokens.length === 0 || this.totalDocs === 0)
|
|
177
|
+
return [];
|
|
178
|
+
const avgdl = this.totalLength / this.totalDocs;
|
|
179
|
+
const candidates = new Set();
|
|
180
|
+
for (const term of queryTokens) {
|
|
181
|
+
const posting = this.invertedIndex.get(term);
|
|
182
|
+
if (posting)
|
|
183
|
+
for (const id of posting)
|
|
184
|
+
candidates.add(id);
|
|
185
|
+
}
|
|
186
|
+
const results = [];
|
|
187
|
+
for (const id of candidates) {
|
|
188
|
+
const doc = this.docs.get(id);
|
|
189
|
+
if (!doc)
|
|
190
|
+
continue;
|
|
191
|
+
let score = 0;
|
|
192
|
+
for (const term of queryTokens) {
|
|
193
|
+
const tf = doc.termFreqs.get(term) ?? 0;
|
|
194
|
+
if (tf === 0)
|
|
195
|
+
continue;
|
|
196
|
+
const df = this.docFreq.get(term) ?? 0;
|
|
197
|
+
// BM25 IDF (Robertson-Sparck-Jones, with +1 to avoid negatives)
|
|
198
|
+
const idf = Math.log(1 + (this.totalDocs - df + 0.5) / (df + 0.5));
|
|
199
|
+
const denom = tf + this.k1 * (1 - this.b + this.b * (doc.length / avgdl));
|
|
200
|
+
score += idf * ((tf * (this.k1 + 1)) / denom);
|
|
201
|
+
}
|
|
202
|
+
if (score > 0)
|
|
203
|
+
results.push({ id, score });
|
|
204
|
+
}
|
|
205
|
+
results.sort((a, b) => b.score - a.score);
|
|
206
|
+
return maxResults && maxResults > 0 ? results.slice(0, maxResults) : results;
|
|
207
|
+
}
|
|
208
|
+
clear() {
|
|
209
|
+
this.docs.clear();
|
|
210
|
+
this.docFreq.clear();
|
|
211
|
+
this.invertedIndex.clear();
|
|
212
|
+
this.totalDocs = 0;
|
|
213
|
+
this.totalLength = 0;
|
|
214
|
+
}
|
|
215
|
+
}
|
|
216
|
+
//# sourceMappingURL=bm25.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"bm25.js","sourceRoot":"","sources":["../../src/search/bm25.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH,MAAM,SAAS,GAAG,IAAI,GAAG,CAAC;IACxB,GAAG;IACH,IAAI;IACJ,KAAK;IACL,KAAK;IACL,IAAI;IACJ,KAAK;IACL,KAAK;IACL,IAAI;IACJ,KAAK;IACL,KAAK;IACL,MAAM;IACN,IAAI;IACJ,MAAM;IACN,OAAO;IACP,MAAM;IACN,KAAK;IACL,KAAK;IACL,QAAQ;IACR,IAAI;IACJ,MAAM;IACN,KAAK;IACL,OAAO;IACP,MAAM;IACN,OAAO;IACP,OAAO;IACP,QAAQ;IACR,OAAO;IACP,KAAK;IACL,OAAO;IACP,KAAK;IACL,MAAM;IACN,IAAI;IACJ,IAAI;IACJ,IAAI;IACJ,KAAK;IACL,IAAI;IACJ,MAAM;IACN,IAAI;IACJ,IAAI;IACJ,MAAM;IACN,IAAI;IACJ,MAAM;IACN,SAAS;IACT,QAAQ;IACR,QAAQ;IACR,OAAO;IACP,OAAO;IACP,OAAO;IACP,SAAS;IACT,KAAK;IACL,KAAK;IACL,MAAM;IACN,OAAO;IACP,OAAO;IACP,SAAS;IACT,MAAM;IACN,MAAM;IACN,MAAM;IACN,OAAO;IACP,MAAM;IACN,OAAO;IACP,KAAK;IACL,KAAK;IACL,KAAK;IACL,MAAM;IACN,OAAO;IACP,MAAM;IACN,KAAK;IACL,MAAM;IACN,MAAM;IACN,OAAO;IACP,MAAM;IACN,MAAM;IACN,IAAI;IACJ,KAAK;IACL,MAAM;IACN,KAAK;IACL,MAAM;IACN,IAAI;IACJ,MAAM;IACN,KAAK;IACL,MAAM;IACN,MAAM;IACN,OAAO;IACP,IAAI;IACJ,MAAM;IACN,IAAI;IACJ,IAAI;IACJ,KAAK;IACL,IAAI;IACJ,KAAK;IACL,MAAM;IACN,MAAM;IACN,KAAK;IACL,KAAK;IACL,OAAO;IACP,IAAI;IACJ,IAAI;IACJ,KAAK;IACL,IAAI;IACJ,MAAM;IACN,KAAK;IACL,MAAM;IACN,MAAM;IACN,OAAO;IACP,OAAO;IACP,GAAG;IACH,KAAK;IACL,MAAM;IACN,OAAO;IACP,KAAK;IACL,MAAM;IACN,IAAI;CACL,CAAC,CAAC;AAEH,SAAS,QAAQ,CAAC,IAAY;IAC5B,OAAO,IAAI;SACR,WAAW,EAAE;SACb,KAAK,CAAC,YAAY,CAAC;SACnB,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC;AAClE,CAAC;AAYD,MAAM,OAAO,SAAS;IACZ,IAAI,GAA0B,IAAI,GAAG,EAAE,CAAC;IACxC,OAAO,GAAwB,IAAI,GAAG,EAAE,CAAC;IACzC,aAAa,GAA6B,IAAI,GAAG,EAAE,CAAC;IACpD,SAAS,GAAW,CAAC,CAAC;IACtB,WAAW,GAAW,CAAC,CAAC;IACf,EAAE,CAAS;IACX,CAAC,CAAS;IAE3B,YAAY,OAAoB,EAAE;QAChC,IAAI,CAAC,EAAE,GAAG,IAAI,CAAC,EAAE,IAAI,GAAG,CAAC;QACzB,IAAI,CAAC,CAAC,GAAG,IAAI,CAAC,CAAC,IAAI,IAAI,CAAC;IAC1B,CAAC;IAED,WAAW,CAAC,EAAU,EAAE,IAAY;QAClC,IAAI,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE,CAAC;YACtB,MAAM,QAAQ,GAAG,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAE,CAAC;YACpC,KAAK,MAAM,IAAI,IAAI,QAAQ,CAAC,SAAS,CAAC,IAAI,EAAE,EAAE,CAAC;gBAC7C,MAAM,KAAK,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBAC1C,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;oBACf,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;oBAC1B,IAAI,CAAC,aAAa,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;gBAClC,CAAC;qBAAM,CAAC;oBACN,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,GAAG,CAAC,CAAC,CAAC;oBAClC,IAAI,CAAC,aAAa,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC,EAAE,CAAC,CAAC;gBAC3C,CAAC;YACH,CAAC;YACD,IAAI,CAAC,SAAS,EAAE,CAAC;YACjB,IAAI,CAAC,WAAW,IAAI,QAAQ,CAAC,MAAM,CAAC;QACtC,CAAC;QAED,MAAM,MAAM,GAAG,QAAQ,CAAC,IAAI,CAAC,CAAC;QAC9B,MAAM,SAAS,GAAG,IAAI,GAAG,EAAkB,CAAC;QAC5C,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;YAC3B,SAAS,CAAC,GAAG,CAAC,KAAK,EAAE,CAAC,SAAS,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;QACxD,CAAC;QAED,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,EAAE,SAAS,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC;QAExD,KAAK,MAAM,IAAI,IAAI,SAAS,CAAC,IAAI,EAAE,EAAE,CAAC;YACpC,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAC1D,IAAI,CAAC,IAAI,CAAC,aAAa,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC;gBAClC,IAAI,CAAC,aAAa,CAAC,GAAG,CAAC,IAAI,EAAE,IAAI,GAAG,EAAE,CAAC,CAAC;YAC1C,CAAC;YACD,IAAI,CAAC,aAAa,CAAC,GAAG,CAAC,IAAI,CAAE,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QACxC,CAAC;QAED,IAAI,CAAC,SAAS,EAAE,CAAC;QACjB,IAAI,CAAC,WAAW,IAAI,MAAM,CAAC,MAAM,CAAC;IACpC,CAAC;IAED,MAAM,CAAC,KAAa,EAAE,UAAmB;QACvC,MAAM,WAAW,GAAG,QAAQ,CAAC,KAAK,CAAC,CAAC;QACpC,IAAI,WAAW,CAAC,MAAM,KAAK,CAAC,IAAI,IAAI,CAAC,SAAS,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAEhE,MAAM,KAAK,GAAG,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC,SAAS,CAAC;QAChD,MAAM,UAAU,GAAG,IAAI,GAAG,EAAU,CAAC;QACrC,KAAK,MAAM,IAAI,IAAI,WAAW,EAAE,CAAC;YAC/B,MAAM,OAAO,GAAG,IAAI,CAAC,aAAa,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;YAC7C,IAAI,OAAO;gBAAE,KAAK,MAAM,EAAE,IAAI,OAAO;oBAAE,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QAC5D,CAAC;QAED,MAAM,OAAO,GAAyC,EAAE,CAAC;QACzD,KAAK,MAAM,EAAE,IAAI,UAAU,EAAE,CAAC;YAC5B,MAAM,GAAG,GAAG,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;YAC9B,IAAI,CAAC,GAAG;gBAAE,SAAS;YACnB,IAAI,KAAK,GAAG,CAAC,CAAC;YACd,KAAK,MAAM,IAAI,IAAI,WAAW,EAAE,CAAC;gBAC/B,MAAM,EAAE,GAAG,GAAG,CAAC,SAAS,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBACxC,IAAI,EAAE,KAAK,CAAC;oBAAE,SAAS;gBACvB,MAAM,EAAE,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBACvC,gEAAgE;gBAChE,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,GAAG,EAAE,GAAG,GAAG,CAAC,GAAG,CAAC,EAAE,GAAG,GAAG,CAAC,CAAC,CAAC;gBACnE,MAAM,KAAK,GAAG,EAAE,GAAG,IAAI,CAAC,EAAE,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,CAAC,GAAG,IAAI,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,GAAG,KAAK,CAAC,CAAC,CAAC;gBAC1E,KAAK,IAAI,GAAG,GAAG,CAAC,CAAC,EAAE,GAAG,CAAC,IAAI,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,GAAG,KAAK,CAAC,CAAC;YAChD,CAAC;YACD,IAAI,KAAK,GAAG,CAAC;gBAAE,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,KAAK,EAAE,CAAC,CAAC;QAC7C,CAAC;QAED,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;QAC1C,OAAO,UAAU,IAAI,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC;IAC/E,CAAC;IAED,KAAK;QACH,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAC;QAClB,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;QACrB,IAAI,CAAC,aAAa,CAAC,KAAK,EAAE,CAAC;QAC3B,IAAI,CAAC,SAAS,GAAG,CAAC,CAAC;QACnB,IAAI,CAAC,WAAW,GAAG,CAAC,CAAC;IACvB,CAAC;CACF"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"scopes.d.ts","sourceRoot":"","sources":["../../src/sessions/scopes.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"scopes.d.ts","sourceRoot":"","sources":["../../src/sessions/scopes.ts"],"names":[],"mappings":"AAWA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAOvD,MAAM,MAAM,WAAW,GAAG,QAAQ,GAAG,OAAO,GAAG,SAAS,GAAG,OAAO,GAAG,OAAO,GAAG,WAAW,GAAG,KAAK,CAAC;AAEnG,MAAM,WAAW,mBAAmB;IAClC,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,EAAE,OAAO,CAAC;CACpB;AA8BD;;;;;;GAMG;AACH,wBAAsB,YAAY,CAChC,KAAK,EAAE,WAAW,EAClB,KAAK,EAAE,MAAM,EACb,OAAO,GAAE,mBAAwB,GAChC,OAAO,CAAC,YAAY,EAAE,CAAC,CAuIzB"}
|
package/dist/sessions/scopes.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { getProjectDirs, getSessionFiles, parseSessionFile, extractMessages, getSessionMeta, } from './parser.js';
|
|
2
|
-
import {
|
|
2
|
+
import { recencyDecay } from '../search/tfidf.js';
|
|
3
|
+
import { BM25Index } from '../search/bm25.js';
|
|
3
4
|
import { buildExcerpt } from '../search/excerpt.js';
|
|
4
5
|
import { getEmbeddingProvider } from '../embeddings/index.js';
|
|
5
6
|
import { VectorStore } from '../vectorstore/index.js';
|
|
@@ -31,7 +32,7 @@ const SCOPE_PATTERNS = {
|
|
|
31
32
|
export async function scopedSearch(scope, query, options = {}) {
|
|
32
33
|
const { project, maxResults = 20, semantic = true } = options;
|
|
33
34
|
const projects = getProjectDirs().filter((p) => !project || p.name.toLowerCase().includes(project.toLowerCase()));
|
|
34
|
-
const index = new
|
|
35
|
+
const index = new BM25Index();
|
|
35
36
|
const docMap = new Map();
|
|
36
37
|
let docCounter = 0;
|
|
37
38
|
for (const proj of projects) {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"scopes.js","sourceRoot":"","sources":["../../src/sessions/scopes.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,cAAc,EACd,eAAe,EACf,gBAAgB,EAChB,eAAe,EACf,cAAc,GAEf,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"scopes.js","sourceRoot":"","sources":["../../src/sessions/scopes.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,cAAc,EACd,eAAe,EACf,gBAAgB,EAChB,eAAe,EACf,cAAc,GAEf,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClD,OAAO,EAAE,SAAS,EAAE,MAAM,mBAAmB,CAAC;AAC9C,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AAEpD,OAAO,EAAE,oBAAoB,EAAE,MAAM,wBAAwB,CAAC;AAC9D,OAAO,EAAE,WAAW,EAAE,MAAM,yBAAyB,CAAC;AACtD,OAAO,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AAYxC,+EAA+E;AAE/E,IAAI,YAAY,GAAuB,IAAI,CAAC;AAE5C,SAAS,cAAc;IACrB,IAAI,CAAC,YAAY,EAAE,CAAC;QAClB,YAAY,GAAG,IAAI,WAAW,EAAE,CAAC;IACnC,CAAC;IACD,OAAO,YAAY,CAAC;AACtB,CAAC;AAED,+EAA+E;AAE/E,MAAM,cAAc,GAA0D;IAC5E,MAAM,EACJ,+KAA+K;IACjL,KAAK,EACH,+HAA+H;IACjI,OAAO,EACL,kIAAkI;IACpI,KAAK,EACH,kJAAkJ;IACpJ,SAAS,EACP,kKAAkK;CACrK,CAAC;AAEF,+EAA+E;AAE/E;;;;;;GAMG;AACH,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,KAAkB,EAClB,KAAa,EACb,UAA+B,EAAE;IAEjC,MAAM,EAAE,OAAO,EAAE,UAAU,GAAG,EAAE,EAAE,QAAQ,GAAG,IAAI,EAAE,GAAG,OAAO,CAAC;IAE9D,MAAM,QAAQ,GAAG,cAAc,EAAE,CAAC,MAAM,CACtC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,OAAO,IAAI,CAAC,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,QAAQ,CAAC,OAAO,CAAC,WAAW,EAAE,CAAC,CACxE,CAAC;IAEF,MAAM,KAAK,GAAG,IAAI,SAAS,EAAE,CAAC;IAC9B,MAAM,MAAM,GAAG,IAAI,GAAG,EAUnB,CAAC;IACJ,IAAI,UAAU,GAAG,CAAC,CAAC;IAEnB,KAAK,MAAM,IAAI,IAAI,QAAQ,EAAE,CAAC;QAC5B,MAAM,QAAQ,GAAG,eAAe,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAC5C,KAAK,MAAM,IAAI,IAAI,QAAQ,EAAE,CAAC;YAC5B,IAAI,CAAC;gBACH,MAAM,OAAO,GAAG,gBAAgB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBAC5C,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;oBAAE,SAAS;gBACnC,MAAM,IAAI,GAAG,cAAc,CAAC,OAAO,CAAC,CAAC;gBACrC,MAAM,QAAQ,GAAG,eAAe,CAAC,OAAO,CAAC,CAAC;gBAE1C,MAAM,QAAQ,GAAG,aAAa,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAC;gBAEhD,KAAK,MAAM,GAAG,IAAI,QAAQ,EAAE,CAAC;oBAC3B,MAAM,KAAK,GAAG,OAAO,UAAU,EAAE,EAAE,CAAC;oBACpC,KAAK,CAAC,WAAW,CAAC,KAAK,EAAE,GAAG,CAAC,OAAO,CAAC,CAAC;oBACtC,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE;wBAChB,OAAO,EAAE,IAAI,CAAC,IAAI;wBAClB,SAAS,EAAE,IAAI,CAAC,EAAE;wBAClB,IAAI,EAAE,GAAG,CAAC,IAAI;wBACd,SAAS,EAAE,GAAG,CAAC,SAAS;wBACxB,OAAO,EAAE,GAAG,CAAC,OAAO;wBACpB,WAAW,EAAE,IAAI,CAAC,SAAS;qBAC5B,CAAC,CAAC;gBACL,CAAC;YACH,CAAC;YAAC,MAAM,CAAC;gBACP,oBAAoB;YACtB,CAAC;QACH,CAAC;IACH,CAAC;IAED,MAAM,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,CAAC,CAAC;IAEnD,MAAM,YAAY,GAAG,MAAM;SACxB,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,KAAK,EAAE,EAAE,EAAE;QACrB,MAAM,GAAG,GAAG,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QAC3B,IAAI,CAAC,GAAG;YAAE,OAAO,IAAI,CAAC;QACtB,MAAM,OAAO,GAAG,YAAY,CAAC,GAAG,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;QACjD,MAAM,EAAE,GAAG,GAAG,CAAC,SAAS,IAAI,GAAG,CAAC,WAAW,CAAC;QAC5C,MAAM,KAAK,GAAG,YAAY,CAAC,EAAE,CAAC,CAAC;QAC/B,MAAM,aAAa,GAAG,KAAK,GAAG,KAAK,CAAC;QAEpC,OAAO;YACL,MAAM,EAAE,SAAkB;YAC1B,EAAE,EAAE,GAAG,CAAC,SAAS;YACjB,OAAO,EAAE,GAAG,CAAC,OAAO;YACpB,IAAI,EAAE,GAAG,CAAC,IAAI;YACd,SAAS,EAAE,EAAE;YACb,OAAO;YACP,KAAK,EAAE,aAAa;YACpB,QAAQ,EAAE;gBACR,KAAK;gBACL,WAAW,EAAE,GAAG,CAAC,WAAW;gBAC5B,cAAc,EAAE,KAAK;gBACrB,iBAAiB,EAAE,IAAI,CAAC,KAAK,CAAC,KAAK,GAAG,GAAG,CAAC,GAAG,GAAG;aACjD;SACF,CAAC;IACJ,CAAC,CAAC;SACD,MAAM,CAAC,CAAC,CAAC,EAA8B,EAAE,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC;IAEzD,YAAY,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;IAC/C,MAAM,YAAY,GAAG,YAAY,CAAC,KAAK,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC;IAEvD,IAAI,CAAC,QAAQ,EAAE,CAAC;QACd,OAAO,YAAY,CAAC;IACtB,CAAC;IAED,IAAI,CAAC;QACH,MAAM,QAAQ,GAAG,MAAM,oBAAoB,EAAE,CAAC;QAC9C,IAAI,CAAC,QAAQ;YAAE,OAAO,YAAY,CAAC;QAEnC,MAAM,WAAW,GAAG,MAAM,QAAQ,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC;QACnD,MAAM,KAAK,GAAG,cAAc,EAAE,CAAC;QAC/B,MAAM,aAAa,GAAG,KAAK,CAAC,cAAc,CAAC,WAAW,EAAE,SAAS,EAAE,UAAU,GAAG,CAAC,CAAC,CAAC;QAEnF,IAAI,aAAa,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,YAAY,CAAC;QAEpD,MAAM,MAAM,GAAG,SAAS,EAAE,CAAC;QAC3B,MAAM,KAAK,GAAG,MAAM,CAAC,cAAc,CAAC;QAEpC,MAAM,cAAc,GAAG,IAAI,GAAG,EAAkB,CAAC;QACjD,KAAK,MAAM,EAAE,IAAI,aAAa,EAAE,CAAC;YAC/B,MAAM,QAAQ,GAAG,cAAc,CAAC,GAAG,CAAC,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;YACtD,IAAI,EAAE,CAAC,KAAK,GAAG,QAAQ,EAAE,CAAC;gBACxB,cAAc,CAAC,GAAG,CAAC,EAAE,CAAC,QAAQ,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC;YAC5C,CAAC;QACH,CAAC;QAED,MAAM,QAAQ,GAAG,YAAY,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QAC7F,MAAM,UAAU,GAAG,QAAQ,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;QAE/C,MAAM,OAAO,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE;YAC1C,MAAM,eAAe,GAAG,MAAM,CAAC,KAAK,GAAG,UAAU,CAAC;YAClD,MAAM,aAAa,GAAG,cAAc,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC;YACzD,MAAM,OAAO,GAAG,KAAK,GAAG,eAAe,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC,GAAG,aAAa,CAAC;YACtE,MAAM,EAAE,GAAG,MAAM,CAAC,SAAS,IAAK,MAAM,CAAC,QAAQ,EAAE,WAAkC,CAAC;YACpF,MAAM,KAAK,GAAG,EAAE,CAAC,CAAC,CAAC,YAAY,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;YACxC,MAAM,UAAU,GAAG,OAAO,GAAG,KAAK,CAAC;YAEnC,OAAO;gBACL,GAAG,MAAM;gBACT,KAAK,EAAE,UAAU;gBACjB,QAAQ,EAAE;oBACR,GAAG,MAAM,CAAC,QAAQ;oBAClB,UAAU,EAAE,eAAe;oBAC3B,aAAa;oBACb,UAAU,EAAE,KAAK;iBAClB;aACF,CAAC;QACJ,CAAC,CAAC,CAAC;QAEH,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;QAC1C,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC;IACtC,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,YAAY,CAAC;IACtB,CAAC;AACH,CAAC;AAED,+EAA+E;AAE/E,SAAS,aAAa,CAAC,KAAkB,EAAE,QAA0B;IACnE,IAAI,KAAK,KAAK,KAAK,EAAE,CAAC;QACpB,OAAO,QAAQ,CAAC;IAClB,CAAC;IAED,IAAI,KAAK,KAAK,OAAO,EAAE,CAAC;QACtB,OAAO,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,UAAU,IAAI,CAAC,CAAC,IAAI,KAAK,aAAa,CAAC,CAAC;IACnF,CAAC;IAED,MAAM,OAAO,GAAG,cAAc,CAAC,KAAK,CAAC,CAAC;IACtC,OAAO,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC;AACzD,CAAC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"search.d.ts","sourceRoot":"","sources":["../../src/sessions/search.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"search.d.ts","sourceRoot":"","sources":["../../src/sessions/search.ts"],"names":[],"mappings":"AAWA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAKvD,MAAM,WAAW,oBAAoB;IACnC,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,GAAG,WAAW,GAAG,KAAK,CAAC;IACpC,aAAa,CAAC,EAAE,OAAO,CAAC;IACxB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,+EAA+E;IAC/E,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,4FAA4F;IAC5F,QAAQ,CAAC,EAAE,OAAO,CAAC;CACpB;AAaD;;;;;;;;;;;;GAYG;AACH,wBAAsB,cAAc,CAClC,KAAK,EAAE,MAAM,EACb,OAAO,GAAE,oBAAyB,GACjC,OAAO,CAAC,YAAY,EAAE,CAAC,CAoCzB"}
|
package/dist/sessions/search.js
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { recencyDecay } from '../search/tfidf.js';
|
|
2
|
+
import { BM25Index } from '../search/bm25.js';
|
|
2
3
|
import { buildExcerpt } from '../search/excerpt.js';
|
|
3
4
|
import { buildBoostContext, applyBoosts, hasAnyBoostSignal } from '../search/boosts.js';
|
|
4
5
|
import { getProjectDirs, getSessionFiles, parseSessionFile, extractMessages, getSessionMeta, } from './parser.js';
|
|
@@ -153,7 +154,7 @@ function getOrBuildIndex(projects, role) {
|
|
|
153
154
|
_indexCache.role === role) {
|
|
154
155
|
return _indexCache;
|
|
155
156
|
}
|
|
156
|
-
const index = new
|
|
157
|
+
const index = new BM25Index();
|
|
157
158
|
const docMap = new Map();
|
|
158
159
|
let docCounter = 0;
|
|
159
160
|
for (const proj of projects) {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"search.js","sourceRoot":"","sources":["../../src/sessions/search.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,
|
|
1
|
+
{"version":3,"file":"search.js","sourceRoot":"","sources":["../../src/sessions/search.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClD,OAAO,EAAE,SAAS,EAAE,MAAM,mBAAmB,CAAC;AAC9C,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,iBAAiB,EAAE,WAAW,EAAE,iBAAiB,EAAE,MAAM,qBAAqB,CAAC;AACxF,OAAO,EACL,cAAc,EACd,eAAe,EACf,gBAAgB,EAChB,eAAe,EACf,cAAc,GACf,MAAM,aAAa,CAAC;AAErB,OAAO,EAAE,oBAAoB,EAAE,MAAM,wBAAwB,CAAC;AAC9D,OAAO,EAAE,WAAW,EAAE,MAAM,yBAAyB,CAAC;AACtD,OAAO,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AAaxC,+EAA+E;AAE/E,IAAI,YAAY,GAAuB,IAAI,CAAC;AAE5C,SAAS,cAAc;IACrB,IAAI,CAAC,YAAY,EAAE,CAAC;QAClB,YAAY,GAAG,IAAI,WAAW,EAAE,CAAC;IACnC,CAAC;IACD,OAAO,YAAY,CAAC;AACtB,CAAC;AAED;;;;;;;;;;;;GAYG;AACH,MAAM,CAAC,KAAK,UAAU,cAAc,CAClC,KAAa,EACb,UAAgC,EAAE;IAElC,MAAM,EACJ,OAAO,EACP,IAAI,GAAG,KAAK,EACZ,aAAa,GAAG,KAAK,EACrB,UAAU,GAAG,EAAE,EACf,MAAM,GAAG,IAAI,EACb,QAAQ,GAAG,IAAI,GAChB,GAAG,OAAO,CAAC;IAEZ,MAAM,QAAQ,GAAG,cAAc,EAAE,CAAC,MAAM,CACtC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,OAAO,IAAI,CAAC,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,QAAQ,CAAC,OAAO,CAAC,WAAW,EAAE,CAAC,CACxE,CAAC;IAEF,IAAI,CAAC,MAAM,EAAE,CAAC;QACZ,OAAO,WAAW,CAAC,KAAK,EAAE,QAAQ,EAAE,IAAI,EAAE,aAAa,EAAE,UAAU,CAAC,CAAC;IACvE,CAAC;IAED,MAAM,YAAY,GAAG,YAAY,CAAC,KAAK,EAAE,QAAQ,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC;IAErE,IAAI,QAAQ,EAAE,CAAC;QACb,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,SAAS,EAAE,CAAC;YAC3B,MAAM,aAAa,GAAG,MAAM,YAAY,CACtC,KAAK,EACL,YAAY,EACZ,UAAU,EACV,MAAM,CAAC,cAAc,CACtB,CAAC;YACF,OAAO,aAAa,CAAC;QACvB,CAAC;QAAC,MAAM,CAAC;YACP,sDAAsD;QACxD,CAAC;IACH,CAAC;IAED,OAAO,YAAY,CAAC;AACtB,CAAC;AAED,+EAA+E;AAE/E,KAAK,UAAU,YAAY,CACzB,KAAa,EACb,YAA4B,EAC5B,UAAkB,EAClB,KAAa;IAEb,MAAM,QAAQ,GAAG,MAAM,oBAAoB,EAAE,CAAC;IAC9C,IAAI,CAAC,QAAQ,EAAE,CAAC;QACd,OAAO,YAAY,CAAC;IACtB,CAAC;IAED,IAAI,WAAqB,CAAC;IAC1B,IAAI,CAAC;QACH,WAAW,GAAG,MAAM,QAAQ,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC;IAC/C,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,YAAY,CAAC;IACtB,CAAC;IAED,MAAM,KAAK,GAAG,cAAc,EAAE,CAAC;IAC/B,IAAI,aAAa,CAAC;IAClB,IAAI,CAAC;QACH,aAAa,GAAG,KAAK,CAAC,cAAc,CAAC,WAAW,EAAE,SAAS,EAAE,UAAU,GAAG,CAAC,CAAC,CAAC;IAC/E,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,YAAY,CAAC;IACtB,CAAC;IAED,IAAI,aAAa,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC/B,OAAO,YAAY,CAAC;IACtB,CAAC;IAED,oDAAoD;IACpD,sEAAsE;IACtE,MAAM,cAAc,GAAG,IAAI,GAAG,EAAkB,CAAC;IACjD,KAAK,MAAM,EAAE,IAAI,aAAa,EAAE,CAAC;QAC/B,MAAM,QAAQ,GAAG,cAAc,CAAC,GAAG,CAAC,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QACtD,IAAI,EAAE,CAAC,KAAK,GAAG,QAAQ,EAAE,CAAC;YACxB,cAAc,CAAC,GAAG,CAAC,EAAE,CAAC,QAAQ,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC;QAC5C,CAAC;IACH,CAAC;IAED,uCAAuC;IACvC,MAAM,QAAQ,GAAG,YAAY,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAC7F,MAAM,UAAU,GAAG,QAAQ,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;IAE/C,gFAAgF;IAChF,mEAAmE;IACnE,MAAM,QAAQ,GAAG,iBAAiB,CAAC,KAAK,CAAC,CAAC;IAC1C,MAAM,YAAY,GAAG,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAEjD,6DAA6D;IAC7D,MAAM,MAAM,GAAG,IAAI,GAAG,EAAwB,CAAC;IAE/C,KAAK,MAAM,MAAM,IAAI,YAAY,EAAE,CAAC;QAClC,MAAM,GAAG,GAAG,GAAG,MAAM,CAAC,EAAE,IAAI,MAAM,CAAC,SAAS,IAAI,EAAE,EAAE,CAAC;QACrD,MAAM,eAAe,GAAG,MAAM,CAAC,KAAK,GAAG,UAAU,CAAC;QAClD,MAAM,aAAa,GAAG,cAAc,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC;QACzD,MAAM,OAAO,GAAG,KAAK,GAAG,eAAe,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC,GAAG,aAAa,CAAC;QAEtE,8CAA8C;QAC9C,MAAM,EAAE,GAAG,MAAM,CAAC,SAAS,IAAK,MAAM,CAAC,QAAQ,EAAE,WAAkC,CAAC;QACpF,MAAM,KAAK,GAAG,EAAE,CAAC,CAAC,CAAC,YAAY,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACxC,MAAM,OAAO,GAAG,OAAO,GAAG,KAAK,CAAC;QAChC,MAAM,UAAU,GAAG,YAAY;YAC7B,CAAC,CAAC,WAAW,CAAC,OAAO,EAAE,QAAQ,EAAE,EAAE,OAAO,EAAE,MAAM,CAAC,OAAO,EAAE,YAAY,EAAE,EAAE,IAAI,IAAI,EAAE,CAAC;YACvF,CAAC,CAAC,OAAO,CAAC;QAEZ,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE;YACd,GAAG,MAAM;YACT,KAAK,EAAE,UAAU;YACjB,QAAQ,EAAE;gBACR,GAAG,MAAM,CAAC,QAAQ;gBAClB,UAAU,EAAE,eAAe;gBAC3B,aAAa;gBACb,UAAU,EAAE,KAAK;aAClB;SACF,CAAC,CAAC;IACL,CAAC;IAED,KAAK,MAAM,EAAE,IAAI,aAAa,EAAE,CAAC;QAC/B,MAAM,cAAc,GAAG,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,QAAQ,CAAC,CAAC;QACrF,IAAI,CAAC,cAAc,EAAE,CAAC;YACpB,MAAM,aAAa,GAAG,EAAE,CAAC,KAAK,CAAC;YAC/B,MAAM,OAAO,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC,GAAG,aAAa,CAAC;YAC5C,qCAAqC;YACrC,MAAM,OAAO,GAAG,EAAE,CAAC,SAAS,CAAC,MAAM,GAAG,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,GAAG,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,SAAS,CAAC;YAC9F,MAAM,SAAS,GAAI,EAAE,CAAC,QAAQ,EAAE,SAAgC,IAAI,SAAS,CAAC;YAC9E,MAAM,KAAK,GAAG,SAAS,CAAC,CAAC,CAAC,YAAY,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC;YACxD,MAAM,UAAU,GAAG,OAAO,GAAG,KAAK,CAAC;YAEnC,MAAM,GAAG,GAAG,GAAG,EAAE,CAAC,QAAQ,IAAI,SAAS,IAAI,EAAE,EAAE,CAAC;YAChD,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,CAAC;gBACrB,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE;oBACd,MAAM,EAAE,SAAS;oBACjB,EAAE,EAAE,EAAE,CAAC,QAAQ;oBACf,IAAI,EAAG,EAAE,CAAC,QAAQ,EAAE,IAA2B,IAAI,SAAS;oBAC5D,SAAS;oBACT,OAAO;oBACP,KAAK,EAAE,UAAU;oBACjB,QAAQ,EAAE;wBACR,UAAU,EAAE,CAAC;wBACb,aAAa;wBACb,UAAU,EAAE,KAAK;qBAClB;iBACF,CAAC,CAAC;YACL,CAAC;QACH,CAAC;IACH,CAAC;IAED,MAAM,OAAO,GAAG,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC;IAC5C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;IAC1C,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC;AACtC,CAAC;AAqBD,IAAI,WAAW,GAAsB,IAAI,CAAC;AAC1C,MAAM,SAAS,GAAG,MAAM,CAAC,CAAC,aAAa;AAEvC,SAAS,eAAe,CACtB,QAA+C,EAC/C,IAAY;IAEZ,MAAM,WAAW,GAAG,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IAC1D,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;IAEvB,IACE,WAAW;QACX,GAAG,GAAG,WAAW,CAAC,SAAS,GAAG,SAAS;QACvC,WAAW,CAAC,WAAW,KAAK,WAAW;QACvC,WAAW,CAAC,IAAI,KAAK,IAAI,EACzB,CAAC;QACD,OAAO,WAAW,CAAC;IACrB,CAAC;IAED,MAAM,KAAK,GAAG,IAAI,SAAS,EAAE,CAAC;IAC9B,MAAM,MAAM,GAAG,IAAI,GAAG,EAAkB,CAAC;IACzC,IAAI,UAAU,GAAG,CAAC,CAAC;IAEnB,KAAK,MAAM,IAAI,IAAI,QAAQ,EAAE,CAAC;QAC5B,MAAM,QAAQ,GAAG,eAAe,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAC5C,KAAK,MAAM,IAAI,IAAI,QAAQ,EAAE,CAAC;YAC5B,IAAI,CAAC;gBACH,MAAM,OAAO,GAAG,gBAAgB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBAC5C,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;oBAAE,SAAS;gBACnC,MAAM,IAAI,GAAG,cAAc,CAAC,OAAO,CAAC,CAAC;gBACrC,MAAM,QAAQ,GAAG,eAAe,CAAC,OAAO,CAAC,CAAC;gBAE1C,KAAK,MAAM,GAAG,IAAI,QAAQ,EAAE,CAAC;oBAC3B,IAAI,IAAI,KAAK,KAAK,IAAI,GAAG,CAAC,IAAI,KAAK,IAAI;wBAAE,SAAS;oBAElD,MAAM,KAAK,GAAG,OAAO,UAAU,EAAE,EAAE,CAAC;oBACpC,KAAK,CAAC,WAAW,CAAC,KAAK,EAAE,GAAG,CAAC,OAAO,CAAC,CAAC;oBACtC,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE;wBAChB,OAAO,EAAE,IAAI,CAAC,IAAI;wBAClB,SAAS,EAAE,IAAI,CAAC,EAAE;wBAClB,IAAI,EAAE,GAAG,CAAC,IAAI;wBACd,SAAS,EAAE,GAAG,CAAC,SAAS;wBACxB,OAAO,EAAE,GAAG,CAAC,OAAO;wBACpB,WAAW,EAAE,IAAI,CAAC,SAAS;qBAC5B,CAAC,CAAC;gBACL,CAAC;YACH,CAAC;YAAC,MAAM,CAAC;gBACP,oBAAoB;YACtB,CAAC;QACH,CAAC;IACH,CAAC;IAED,WAAW,GAAG,EAAE,KAAK,EAAE,MAAM,EAAE,SAAS,EAAE,GAAG,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC;IACnE,OAAO,WAAW,CAAC;AACrB,CAAC;AAED,SAAS,YAAY,CACnB,KAAa,EACb,QAA+C,EAC/C,IAAY,EACZ,UAAkB;IAElB,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,eAAe,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC;IAC1D,MAAM,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,CAAC,CAAC;IAEnD,MAAM,OAAO,GAAG,MAAM;SACnB,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,KAAK,EAAE,EAAE,EAAE;QACrB,MAAM,GAAG,GAAG,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QAC3B,IAAI,CAAC,GAAG;YAAE,OAAO,IAAI,CAAC;QACtB,MAAM,OAAO,GAAG,YAAY,CAAC,GAAG,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;QACjD,MAAM,EAAE,GAAG,GAAG,CAAC,SAAS,IAAI,GAAG,CAAC,WAAW,CAAC;QAC5C,MAAM,KAAK,GAAG,YAAY,CAAC,EAAE,CAAC,CAAC;QAC/B,MAAM,aAAa,GAAG,KAAK,GAAG,KAAK,CAAC;QAEpC,OAAO;YACL,MAAM,EAAE,SAAkB;YAC1B,EAAE,EAAE,GAAG,CAAC,SAAS;YACjB,OAAO,EAAE,GAAG,CAAC,OAAO;YACpB,IAAI,EAAE,GAAG,CAAC,IAAI;YACd,SAAS,EAAE,EAAE;YACb,OAAO;YACP,KAAK,EAAE,aAAa;YACpB,QAAQ,EAAE;gBACR,WAAW,EAAE,GAAG,CAAC,WAAW;gBAC5B,cAAc,EAAE,KAAK;gBACrB,iBAAiB,EAAE,IAAI,CAAC,KAAK,CAAC,KAAK,GAAG,GAAG,CAAC,GAAG,GAAG;aACjD;SACF,CAAC;IACJ,CAAC,CAAC;SACD,MAAM,CAAC,CAAC,CAAC,EAA8B,EAAE,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC;IAEzD,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;IAC1C,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC;AACtC,CAAC;AAED,+EAA+E;AAE/E,SAAS,WAAW,CAClB,KAAa,EACb,QAA+C,EAC/C,IAAY,EACZ,aAAsB,EACtB,UAAkB;IAElB,MAAM,KAAK,GAAG,aAAa,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC;IACzC,IAAI,KAAa,CAAC;IAClB,IAAI,CAAC;QACH,KAAK,GAAG,IAAI,MAAM,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;IACnC,CAAC;IAAC,MAAM,CAAC;QACP,KAAK,GAAG,IAAI,MAAM,CAAC,KAAK,CAAC,OAAO,CAAC,qBAAqB,EAAE,MAAM,CAAC,EAAE,KAAK,CAAC,CAAC;IAC1E,CAAC;IAED,MAAM,OAAO,GAAmB,EAAE,CAAC;IAEnC,KAAK,MAAM,IAAI,IAAI,QAAQ,EAAE,CAAC;QAC5B,MAAM,QAAQ,GAAG,eAAe,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAC5C,KAAK,MAAM,IAAI,IAAI,QAAQ,EAAE,CAAC;YAC5B,IAAI,CAAC;gBACH,MAAM,OAAO,GAAG,gBAAgB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBAC5C,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;oBAAE,SAAS;gBACnC,MAAM,IAAI,GAAG,cAAc,CAAC,OAAO,CAAC,CAAC;gBACrC,MAAM,QAAQ,GAAG,eAAe,CAAC,OAAO,CAAC,CAAC;gBAE1C,KAAK,MAAM,GAAG,IAAI,QAAQ,EAAE,CAAC;oBAC3B,IAAI,IAAI,KAAK,KAAK,IAAI,GAAG,CAAC,IAAI,KAAK,IAAI;wBAAE,SAAS;oBAClD,2DAA2D;oBAC3D,KAAK,CAAC,SAAS,GAAG,CAAC,CAAC;oBACpB,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC;wBAAE,SAAS;oBAEvC,MAAM,OAAO,GAAG,YAAY,CAAC,GAAG,CAAC,OAAO,EAAE,KAAK,EAAE,EAAE,aAAa,EAAE,CAAC,CAAC;oBAEpE,OAAO,CAAC,IAAI,CAAC;wBACX,MAAM,EAAE,SAAS;wBACjB,EAAE,EAAE,IAAI,CAAC,EAAE;wBACX,OAAO,EAAE,IAAI,CAAC,IAAI;wBAClB,IAAI,EAAE,GAAG,CAAC,IAAI;wBACd,SAAS,EAAE,GAAG,CAAC,SAAS,IAAI,IAAI,CAAC,SAAS;wBAC1C,OAAO;wBACP,KAAK,EAAE,CAAC,EAAE,2BAA2B;wBACrC,QAAQ,EAAE;4BACR,WAAW,EAAE,IAAI,CAAC,SAAS;yBAC5B;qBACF,CAAC,CAAC;oBAEH,IAAI,OAAO,CAAC,MAAM,IAAI,UAAU;wBAAE,MAAM;gBAC1C,CAAC;YACH,CAAC;YAAC,MAAM,CAAC;gBACP,OAAO;YACT,CAAC;YACD,IAAI,OAAO,CAAC,MAAM,IAAI,UAAU;gBAAE,MAAM;QAC1C,CAAC;QACD,IAAI,OAAO,CAAC,MAAM,IAAI,UAAU;YAAE,MAAM;IAC1C,CAAC;IAED,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,kDAAkD"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-knowledge",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.5.1",
|
|
4
4
|
"description": "Cross-session memory and recall for AI agents — git-synced knowledge base, knowledge graph, confidence scoring, hybrid semantic+TF-IDF search, auto-distillation with secrets scrubbing",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -85,11 +85,13 @@
|
|
|
85
85
|
"@eslint/js": "^10.0.1",
|
|
86
86
|
"@types/better-sqlite3": "^7.6.0",
|
|
87
87
|
"@types/node": "^25.5.0",
|
|
88
|
+
"@types/stream-json": "^1.7.8",
|
|
88
89
|
"@types/ws": "^8.18.1",
|
|
89
90
|
"eslint": "^10.1.0",
|
|
90
91
|
"husky": "^9.0.0",
|
|
91
92
|
"lint-staged": "^16.0.0",
|
|
92
93
|
"prettier": "^3.8.1",
|
|
94
|
+
"stream-json": "^2.1.0",
|
|
93
95
|
"tsx": "^4.21.0",
|
|
94
96
|
"typescript": "^5.7.0",
|
|
95
97
|
"typescript-eslint": "^8.57.2",
|