agent-knowledge 1.5.0 → 1.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -0
- package/README.md +1 -1
- package/agent-desk-plugin.json +1 -1
- package/package.json +3 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,32 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.5.1 (2026-04-09)
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- **Streaming JSON parser in `bench/longmemeval.ts`**. The benchmark now uses an async iterator (`stream-json`) for files >384 MB so the 2.6 GB `longmemeval_m` split can be benchmarked without hitting Node's ~512 MB string-length limit. Files under the threshold still use the original `readFileSync + JSON.parse` fast path with byte-identical results.
|
|
8
|
+
- **`bench/longmemeval_m` results.** First time we've measured the harder split (500 distractor sessions per question, 10× more than `_s`):
|
|
9
|
+
|
|
10
|
+
| Mode | R@1 | R@5 | R@10 |
|
|
11
|
+
| ---------------------------- | ----- | --------- | ----- |
|
|
12
|
+
| 1.5 — BM25 + boosts (sparse) | 65.6% | **86.0%** | 92.4% |
|
|
13
|
+
| 1.5 — BM25 + semantic hybrid | 65.4% | **88.4%** | 92.2% |
|
|
14
|
+
|
|
15
|
+
- **Validation against the LongMemEval paper's official `flat-bm25`** (`bench/paper_bm25_eval.py`). We re-ran the paper's exact corpus construction (user-only text per session), tokenization (`doc.split(" ")`), and `rank_bm25.BM25Okapi` defaults on the same data, with the paper's own `recall_any` evaluator from `src/retrieval/eval_utils.py`:
|
|
16
|
+
|
|
17
|
+
| Split | Paper `flat-bm25` | agent-knowledge 1.5 BM25 | agent-knowledge 1.5 hybrid |
|
|
18
|
+
| --------------- | ----------------- | ------------------------ | -------------------------- |
|
|
19
|
+
| `longmemeval_s` | 88.6% | **97.2% (+8.6pp)** | **98.8% (+10.2pp)** |
|
|
20
|
+
| `longmemeval_m` | 75.2% | **86.0% (+10.8pp)** | **88.4% (+13.2pp)** |
|
|
21
|
+
|
|
22
|
+
The improvement comes from three small but cumulative differences vs the paper's BM25 setup: (1) we index both user and assistant turns instead of user-only, (2) we lowercase + strip stopwords instead of splitting on whitespace, (3) we use `k1 = 1.2` instead of the `BM25Okapi` default `1.5`.
|
|
23
|
+
|
|
24
|
+
- `stream-json` and `@types/stream-json` as devDependencies (used by the bench only; not in the runtime).
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- `bench/longmemeval.ts` main loop is now an `async for await` over a streaming dataset iterator. The earlier eager-load path (`readFileSync` then `JSON.parse`) is preserved for files under 384 MB so existing reproduction commands stay byte-identical.
|
|
29
|
+
|
|
3
30
|
## 1.5.0 (2026-04-08)
|
|
4
31
|
|
|
5
32
|
### Changed
|
package/README.md
CHANGED
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
|
|
9
9
|
**Cross-session memory and recall for AI coding assistants** -- works with Claude Code, Cursor, OpenCode, Cline, Continue.dev, and Aider out of the box. Git-synced knowledge base, hybrid semantic+TF-IDF search, auto-distillation with secrets scrubbing.
|
|
10
10
|
|
|
11
|
-
**Benchmark:** **R@5 = 97.2% (sparse) / 98.8% (hybrid)** on the public LongMemEval academic benchmark (Wu et al. 2024, ICLR 2025), full 500 questions, no LLM, no API key, runs entirely offline. R@
|
|
11
|
+
**Benchmark:** **R@5 = 97.2% (sparse) / 98.8% (hybrid)** on `longmemeval_s` and **86.0% (sparse) / 88.4% (hybrid)** on the harder `longmemeval_m` split — the public LongMemEval academic benchmark (Wu et al. 2024, ICLR 2025), full 500 questions per split, no LLM, no API key, runs entirely offline. **+8.6pp to +13.2pp R@5 over the paper's official `flat-bm25` baseline** in apples-to-apples reproduction. Full per-category table, reproduction instructions, and paper-comparison details in [`bench/README.md`](bench/README.md).
|
|
12
12
|
|
|
13
13
|
<table>
|
|
14
14
|
<tr>
|
package/agent-desk-plugin.json
CHANGED
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-knowledge",
|
|
3
|
-
"version": "1.5.
|
|
3
|
+
"version": "1.5.1",
|
|
4
4
|
"description": "Cross-session memory and recall for AI agents — git-synced knowledge base, knowledge graph, confidence scoring, hybrid semantic+TF-IDF search, auto-distillation with secrets scrubbing",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -85,11 +85,13 @@
|
|
|
85
85
|
"@eslint/js": "^10.0.1",
|
|
86
86
|
"@types/better-sqlite3": "^7.6.0",
|
|
87
87
|
"@types/node": "^25.5.0",
|
|
88
|
+
"@types/stream-json": "^1.7.8",
|
|
88
89
|
"@types/ws": "^8.18.1",
|
|
89
90
|
"eslint": "^10.1.0",
|
|
90
91
|
"husky": "^9.0.0",
|
|
91
92
|
"lint-staged": "^16.0.0",
|
|
92
93
|
"prettier": "^3.8.1",
|
|
94
|
+
"stream-json": "^2.1.0",
|
|
93
95
|
"tsx": "^4.21.0",
|
|
94
96
|
"typescript": "^5.7.0",
|
|
95
97
|
"typescript-eslint": "^8.57.2",
|