klew 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. klew-0.1.0/.gitignore +17 -0
  2. klew-0.1.0/LICENSE +21 -0
  3. klew-0.1.0/PKG-INFO +290 -0
  4. klew-0.1.0/README.md +246 -0
  5. klew-0.1.0/klew/__init__.py +1 -0
  6. klew-0.1.0/klew/adapters/__init__.py +0 -0
  7. klew-0.1.0/klew/adapters/git_miner.py +113 -0
  8. klew-0.1.0/klew/adapters/hydra_store.py +150 -0
  9. klew-0.1.0/klew/adapters/memory_store.py +158 -0
  10. klew-0.1.0/klew/adapters/queries.py +91 -0
  11. klew-0.1.0/klew/adapters/source_reader.py +270 -0
  12. klew-0.1.0/klew/adapters/tokens.py +22 -0
  13. klew-0.1.0/klew/api.py +216 -0
  14. klew-0.1.0/klew/baselines/__init__.py +0 -0
  15. klew-0.1.0/klew/baselines/chunks.py +55 -0
  16. klew-0.1.0/klew/baselines/lexical.py +34 -0
  17. klew-0.1.0/klew/baselines/vector.py +69 -0
  18. klew-0.1.0/klew/cli.py +142 -0
  19. klew-0.1.0/klew/config.py +33 -0
  20. klew-0.1.0/klew/domain/__init__.py +0 -0
  21. klew-0.1.0/klew/domain/bm25.py +33 -0
  22. klew-0.1.0/klew/domain/learning.py +76 -0
  23. klew-0.1.0/klew/domain/packer.py +182 -0
  24. klew-0.1.0/klew/domain/seeder.py +126 -0
  25. klew-0.1.0/klew/domain/walker.py +96 -0
  26. klew-0.1.0/klew/eval/__init__.py +0 -0
  27. klew-0.1.0/klew/eval/plots.py +90 -0
  28. klew-0.1.0/klew/eval/replay.py +285 -0
  29. klew-0.1.0/klew/eval/stats.py +41 -0
  30. klew-0.1.0/klew/hydra_local.py +74 -0
  31. klew-0.1.0/klew/ids.py +27 -0
  32. klew-0.1.0/klew/mcp_server.py +127 -0
  33. klew-0.1.0/klew/models.py +71 -0
  34. klew-0.1.0/klew/ports.py +67 -0
  35. klew-0.1.0/klew/runtime.py +100 -0
  36. klew-0.1.0/klew/schema.py +78 -0
  37. klew-0.1.0/klew/services/__init__.py +0 -0
  38. klew-0.1.0/klew/services/engine.py +349 -0
  39. klew-0.1.0/klew/services/index.py +215 -0
  40. klew-0.1.0/klew/text.py +66 -0
  41. klew-0.1.0/pyproject.toml +69 -0
klew-0.1.0/.gitignore ADDED
@@ -0,0 +1,17 @@
1
+ .vendor/
2
+ .hydradb/
3
+ .venv/
4
+ __pycache__/
5
+ *.pyc
6
+ .pytest_cache/
7
+ .mypy_cache/
8
+ .ruff_cache/
9
+ node_modules/
10
+ web/dist/
11
+ data/
12
+ eval/cache/
13
+ .klew/
14
+ .DS_Store
15
+ deploy/hydra_token
16
+ *.tgz
17
+ *.mcpb
klew-0.1.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Klew contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
klew-0.1.0/PKG-INFO ADDED
@@ -0,0 +1,290 @@
1
+ Metadata-Version: 2.5
2
+ Name: klew
3
+ Version: 0.1.0
4
+ Summary: Self-improving code context for coding agents: an MCP server that walks a code graph in HydraDB and learns from every merged PR.
5
+ Project-URL: Homepage, https://github.com/IronicRayquaza/klew
6
+ Project-URL: Repository, https://github.com/IronicRayquaza/klew
7
+ Project-URL: Documentation, https://github.com/IronicRayquaza/klew#readme
8
+ Project-URL: Issues, https://github.com/IronicRayquaza/klew/issues
9
+ Author: Klew contributors
10
+ License: MIT
11
+ License-File: LICENSE
12
+ Keywords: claude,code-graph,coding-agents,context,hydradb,mcp,model-context-protocol,rag
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: License :: OSI Approved :: MIT License
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
21
+ Classifier: Topic :: Software Development
22
+ Requires-Python: >=3.11
23
+ Requires-Dist: fastapi>=0.111
24
+ Requires-Dist: mcp>=2.3
25
+ Requires-Dist: neo4j>=5.20
26
+ Requires-Dist: numpy>=1.26
27
+ Requires-Dist: pydantic>=2.7
28
+ Requires-Dist: structlog>=24.1
29
+ Requires-Dist: tiktoken>=0.7
30
+ Requires-Dist: tree-sitter-python>=0.23
31
+ Requires-Dist: tree-sitter>=0.23
32
+ Requires-Dist: typer>=0.12
33
+ Requires-Dist: uvicorn>=0.30
34
+ Provides-Extra: dev
35
+ Requires-Dist: httpx>=0.27; extra == 'dev'
36
+ Requires-Dist: mypy>=1.10; extra == 'dev'
37
+ Requires-Dist: pytest>=8; extra == 'dev'
38
+ Requires-Dist: ruff>=0.5; extra == 'dev'
39
+ Provides-Extra: eval
40
+ Requires-Dist: matplotlib>=3.8; extra == 'eval'
41
+ Provides-Extra: vector
42
+ Requires-Dist: fastembed>=0.3; extra == 'vector'
43
+ Description-Content-Type: text/markdown
44
+
45
+ # Klew
46
+
47
+ **A self-improving context layer for coding agents, built on [HydraDB](https://github.com/hydra-db/hydradb).**
48
+
49
+ Every session, a coding agent rediscovers your repository by grepping and pasting whole files into its
50
+ context. Klew gives it a small map instead: the functions, callers, tests and files that matter for
51
+ *this* task, each with the path through the code that connects it to the task. It builds that map by
52
+ walking a code graph in HydraDB, not by pulling the most similar-looking chunks. Every merged PR teaches
53
+ the graph which paths actually led to the code that changed, so the next map is sharper.
54
+
55
+ > A *clew* is the ball of thread Ariadne gave Theseus to find his way through the labyrinth; it's where
56
+ > the word *clue* comes from. Klew is that thread for your codebase, and every merge makes it stronger.
57
+
58
+ <!-- RESULTS:START -->
59
+ ## Results
60
+
61
+ **In one line:** on every repository, Klew puts the right file in the agent's context 31–47 points more often
62
+ than vector RAG at the same budget (statistically significant everywhere), and learning from the repo's history
63
+ significantly improves ranking on all three. On ranking alone, Klew with vector seeds beats vector RAG
64
+ significantly on httpx and flask; on ODS only its MRR gain is significant.
65
+
66
+ **How to read these numbers.** *Right file in context*: share of changed files inside the 2,000-token
67
+ pack, which is what the agent reads (our headline). *Tokens per correct file*: context tokens spent per
68
+ changed file found (lower is cheaper). *Recall@5*: share of changed files in the top five. *Top-1*: how
69
+ often the #1 file changed. *MRR*: 1 if the first correct file is #1, ½ at #2, ⅓ at #3, and so on.
70
+
71
+ Every number below is on **held-out commits**: the first 100 commits of each window were used for
72
+ tuning (`scripts/tune.py`), and these are the rest. Every approach gets the same 2,000-token budget.
73
+
74
+ ### httpx: 199 held-out commits (first 100 commits used for tuning)
75
+
76
+ | Approach | Right file in context | Tokens per correct file | Recall@5 | Top-1 | MRR |
77
+ |---|---|---|---|---|---|
78
+ | Vector RAG | 58.1% | 3,392 | 63.9% | 47.2% | 62.2% |
79
+ | BM25 RAG | 49.1% | 3,685 | 56.2% | 39.2% | 52.3% |
80
+ | Klew, learning off | 83.4% | 1,624 | 56.0% | 44.7% | 56.9% |
81
+ | Klew without graph traversal | 87.6% | 675 | 65.5% | 49.2% | 62.8% |
82
+ | **Klew** | 88.9% | 2,025 | 65.4% | 50.7% | 63.6% |
83
+ | **Klew + vector seeds** | 95.3% | 1,986 | 67.3% | 56.8% | 68.8% |
84
+
85
+ **Key comparisons** (paired 95% bootstrap intervals over these commits; ✓ = the interval excludes 0):
86
+
87
+ | Comparison | Right file in context | Recall@5 | Top-1 | MRR |
88
+ |---|---|---|---|---|
89
+ | Klew + vector seeds vs vector RAG | +37.2 [+31.7, +43.1] ✓ | +3.4 [+0.1, +6.9] ✓ | +9.5 [+2.5, +16.6] ✓ | +6.6 [+2.5, +10.9] ✓ |
90
+ | Klew vs vector RAG | +30.9 [+24.8, +36.8] ✓ | +1.5 [-3.9, +6.3] | +3.5 [-4.0, +11.1] | +1.4 [-3.7, +6.7] |
91
+ | Learning on vs off | +5.6 [+1.7, +9.7] ✓ | +9.4 [+4.8, +13.9] ✓ | +6.0 [+1.5, +10.1] ✓ | +6.7 [+3.4, +10.1] ✓ |
92
+ | Graph walk on vs off | +1.4 [-0.7, +3.7] | -0.1 [-0.8, +0.5] | +1.5 [+0.0, +3.5] | +0.8 [-0.2, +2.0] |
93
+
94
+ - **Learning** (same graph, learning off vs on): +5.6 pts right file in context.
95
+ - **Graph traversal** (Klew with vs without the walk): +1.4 pts right file in context, +1.5 pts top-1.
96
+
97
+ **Accuracy as the graph learns** (learn from N past commits, then freeze; held-out): right file in context 82.0% → 89.1%, recall@5 55.7% → 64.3%, top-1 45.2% → 50.2%, MRR 57.0% → 63.0% from 0 to 600 commits learned.
98
+
99
+ ![accuracy as the graph learns](docs/img/httpx-learning.png)
100
+
101
+ ![learning curve over the replay](docs/img/httpx-curve.png)
102
+
103
+ ### flask: 196 held-out commits (never used for tuning; every setting came from httpx)
104
+
105
+ | Approach | Right file in context | Tokens per correct file | Recall@5 | Top-1 | MRR |
106
+ |---|---|---|---|---|---|
107
+ | Vector RAG | 45.1% | 4,393 | 48.8% | 36.7% | 49.1% |
108
+ | BM25 RAG | 53.2% | 3,311 | 55.0% | 39.3% | 51.5% |
109
+ | Klew, learning off | 84.7% | 1,472 | 48.5% | 37.8% | 49.7% |
110
+ | Klew without graph traversal | 84.1% | 703 | 60.0% | 44.4% | 56.8% |
111
+ | **Klew** | 89.0% | 2,020 | 60.6% | 43.9% | 57.1% |
112
+ | **Klew + vector seeds** | 91.6% | 2,087 | 60.1% | 46.4% | 58.4% |
113
+
114
+ **Key comparisons** (paired 95% bootstrap intervals over these commits; ✓ = the interval excludes 0):
115
+
116
+ | Comparison | Right file in context | Recall@5 | Top-1 | MRR |
117
+ |---|---|---|---|---|
118
+ | Klew + vector seeds vs vector RAG | +46.5 [+40.4, +52.2] ✓ | +11.3 [+6.7, +16.2] ✓ | +9.7 [+3.1, +16.3] ✓ | +9.3 [+4.7, +14.1] ✓ |
119
+ | Klew vs vector RAG | +43.9 [+37.9, +50.4] ✓ | +11.8 [+5.4, +18.4] ✓ | +7.1 [+0.0, +14.3] | +8.0 [+2.5, +13.3] ✓ |
120
+ | Learning on vs off | +4.3 [+1.1, +7.8] ✓ | +12.1 [+6.9, +17.6] ✓ | +6.1 [+0.5, +11.7] ✓ | +7.4 [+3.6, +11.4] ✓ |
121
+ | Graph walk on vs off | +4.9 [+1.5, +8.6] ✓ | +0.6 [-0.5, +2.0] | -0.5 [-1.5, +0.0] | +0.3 [-0.5, +0.9] |
122
+
123
+ - **Learning** (same graph, learning off vs on): +4.3 pts right file in context.
124
+ - **Graph traversal** (Klew with vs without the walk): +4.9 pts right file in context, -0.5 pts top-1.
125
+
126
+ **Accuracy as the graph learns** (learn from N past commits, then freeze; held-out): right file in context 85.0% → 87.9%, recall@5 48.5% → 58.3%, top-1 37.8% → 42.4%, MRR 49.4% → 55.0% from 0 to 600 commits learned.
127
+
128
+ ![accuracy as the graph learns](docs/img/flask-learning.png)
129
+
130
+ ![learning curve over the replay](docs/img/flask-curve.png)
131
+
132
+ ### ods: 166 held-out commits (never used for tuning; every setting came from httpx)
133
+
134
+ | Approach | Right file in context | Tokens per correct file | Recall@5 | Top-1 | MRR |
135
+ |---|---|---|---|---|---|
136
+ | Vector RAG | 35.0% | 5,557 | 45.2% | 39.2% | 51.3% |
137
+ | BM25 RAG | 30.7% | 6,282 | 41.8% | 33.7% | 47.0% |
138
+ | Klew, learning off | 69.3% | 2,642 | 41.3% | 33.7% | 46.6% |
139
+ | Klew without graph traversal | 76.9% | 1,206 | 46.6% | 41.6% | 53.3% |
140
+ | **Klew** | 72.4% | 2,702 | 47.3% | 41.6% | 53.5% |
141
+ | **Klew + vector seeds** | 71.9% | 2,753 | 48.3% | 45.8% | 57.2% |
142
+
143
+ **Key comparisons** (paired 95% bootstrap intervals over these commits; ✓ = the interval excludes 0):
144
+
145
+ | Comparison | Right file in context | Recall@5 | Top-1 | MRR |
146
+ |---|---|---|---|---|
147
+ | Klew + vector seeds vs vector RAG | +36.9 [+31.8, +42.3] ✓ | +3.1 [-0.5, +6.8] | +6.6 [-0.6, +13.9] | +5.9 [+1.3, +10.2] ✓ |
148
+ | Klew vs vector RAG | +37.5 [+30.3, +44.1] ✓ | +2.1 [-5.2, +9.4] | +2.4 [-6.0, +10.8] | +2.2 [-3.8, +8.4] |
149
+ | Learning on vs off | +3.1 [-2.1, +8.4] | +6.0 [+2.3, +10.2] ✓ | +7.8 [+4.2, +12.0] ✓ | +6.9 [+4.2, +9.6] ✓ |
150
+ | Graph walk on vs off | -4.4 [-9.0, +0.1] | +0.6 [-0.6, +2.1] | +0.0 [+0.0, +0.0] | +0.1 [-0.7, +1.0] |
151
+
152
+ - **Learning** (same graph, learning off vs on): +3.1 pts right file in context.
153
+ - **Graph traversal** (Klew with vs without the walk): -4.4 pts right file in context, +0.0 pts top-1.
154
+
155
+ **Accuracy as the graph learns** (learn from N past commits, then freeze; held-out): right file in context 68.6% → 69.5%, recall@5 41.7% → 43.8%, top-1 35.5% → 40.4%, MRR 48.3% → 52.0% from 0 to 300 commits learned.
156
+
157
+ ![accuracy as the graph learns](docs/img/ods-learning.png)
158
+
159
+ ![learning curve over the replay](docs/img/ods-curve.png)
160
+
161
+ Latency: Klew answers in about 0.2–0.4 s per request versus about 15 ms for a vector lookup.
162
+
163
+ <!-- RESULTS:END -->
164
+
165
+ ## How it works
166
+
167
+ ```
168
+ repo ──► tree-sitter indexer ──┐ ┌──────────── HydraDB ────────────┐
169
+ git history ──► co-change ─────┼── UNWIND/MERGE ─────► │ files · symbols · calls · tests │
170
+ │ │ imports · inheritance │
171
+ │ │ CO_CHANGED · POINTS_TO (learned)│
172
+ │ └───────▲──────────────▲──────────┘
173
+ agent ─MCP─► get_context ─► seed ─► walk (1-hop expansions) ───┘ │
174
+ │ rank ─► pack (signatures + snippet, │
175
+ │ why-path per file) │
176
+ merged PR / record_outcome ──┴──► learning rule ─► reinforce / decay ─────────┘
177
+ ```
178
+
179
+ 1. **Map.** Tree-sitter parses the repo into files and symbols, with calls, imports, inheritance and tests
180
+ between them, resolved statically through each file's imports. Git history adds `CO_CHANGED` edges between
181
+ files that change together. All of it lives in HydraDB.
182
+ 2. **Walk.** For a task, Klew seeds from names in the task: BM25 over symbol names, docstrings and body
183
+ terms, exact identifiers, plus anything it has learned. It then propagates relevance hop by hop through
184
+ HydraDB (calls, callers, tests, inheritance, imports, co-change), keeping the shortest path to every
185
+ node. Files are ranked, then packed into the token budget as signatures plus the best snippet, each with
186
+ its why-path.
187
+ 3. **Learn.** When a run or merged PR lands, the files that actually changed reinforce `POINTS_TO` edges from
188
+ the task's words to those files, and files that were served but not touched decay. Files touched together
189
+ reinforce `CO_CHANGED`. Structural edges are facts and never change; experience is learned on top, so
190
+ re-indexing never erases what the graph has learned.
191
+
192
+ ### Why a graph and not a vector store
193
+
194
+ - **The file you need often doesn't look like the task.** It's the caller, the test, or the config two hops
195
+ away. Similarity can't follow a call edge.
196
+ - **It learns without re-embedding.** Outcomes update edge weights in place.
197
+ - **Every item says why it's there.** A path through the code is auditable; a cosine score isn't.
198
+ - **It spends fewer tokens.** Structure tells us which signature and which body to include.
199
+
200
+ ### How it uses HydraDB
201
+
202
+ - **Typed graph:** `KlewFile`, `KlewSymbol`, `KlewTerm`, `KlewRun`, `KlewRepo` nodes. `CONTAINS`, `IMPORTS`, `CALLS`,
203
+ `TESTS`, `INHERITS` (structural), `CO_CHANGED`, `POINTS_TO` (learned), `SERVED`, `TOUCHED` (provenance).
204
+ - **Traversal where the graph lives:** each hop is one anchored 1-hop expansion per (edge type, direction),
205
+ e.g. `MATCH (a:KlewSymbol)<-[r:CALLS]-(b) WHERE a.id IN $ids RETURN a.id, b.id, r.w`. That's about 34
206
+ queries per context pack, run in parallel within each hop: about 0.2–0.3 s on httpx (warm-started
207
+ graph, laptop, Docker).
208
+ - **Learning writes back in place:** batched `UNWIND … MERGE (s)-[r:POINTS_TO {id: row.eid}]->(d) SET r.w = row.w`.
209
+ - **Native path search for explanations:** `explain` calls HydraDB's `algo.MSpaths` (GraphBLAS) to find the paths
210
+ through the code from a task's best matches to a file, e.g. `_get_proxy_map` → calls → `get_environment_proxies`
211
+ → contained in → `httpx/_utils.py`.
212
+ - **Run history as a graph:** every context request becomes a `KlewRun` with `SERVED` and `TOUCHED` edges, an
213
+ audit trail of what the agent was shown and what it changed.
214
+ - All Cypher lives in [`klew/adapters/queries.py`](klew/adapters/queries.py), with the dialect
215
+ rules we verified against HydraDB v0.4.0.
216
+
217
+ ## Install
218
+
219
+ Klew needs Docker (for a local HydraDB node) and [uv](https://docs.astral.sh/uv/) or pip.
220
+
221
+ ```bash
222
+ pip install klew # or run without installing: uvx klew …
223
+ klew hydra up # local HydraDB node in Docker
224
+ klew index ~/code/your-repo # build the graph and learn from recent history
225
+ klew context --repo your-repo "what you want to change"
226
+ ```
227
+
228
+ **Claude Code** (plugin with the MCP server and a skill that tells Claude when to use it):
229
+
230
+ ```text
231
+ /plugin install klew --marketplace IronicRayquaza/klew
232
+ ```
233
+
234
+ **Claude Desktop**: download `klew.mcpb` from the [latest release](https://github.com/IronicRayquaza/klew/releases)
235
+ and double-click it. It asks for your repository folder.
236
+
237
+ **Cursor, Windsurf and other MCP clients**:
238
+
239
+ ```json
240
+ { "mcpServers": { "klew": { "command": "npx", "args": ["-y", "klew", "mcp"],
241
+ "env": { "KLEW_REPO_ROOT": "/absolute/path/to/your/repo" } } } }
242
+ ```
243
+
244
+ MCP tools: `get_context(task, budget_tokens, cutoff)`, `record_outcome(run_id, touched_files?)` (defaults to the
245
+ working tree's changed files), `explain(task, path)`, and `graph_stats()`. There is deliberately no raw-Cypher
246
+ tool.
247
+
248
+ **Learn from merged PRs:** `klew learn-git <merge-sha>`, or the GitHub Action in [`action/`](action/action.yml).
249
+
250
+ **From source / dashboard:** `docker compose up -d`, `uv pip install -e ".[dev]"`, `klew serve` (API on :8000)
251
+ and `cd web && npm install && npm run dev` (UI on :5173).
252
+
253
+ ## Evaluation
254
+
255
+ `klew eval <repo> --window 300` replays the last 300 commits that modified 1–8 Python files, in order:
256
+
257
+ - **task** = the commit message; **truth** = the `.py` files it modified that exist in the snapshot.
258
+ - Every arm gets the **same 2,000-token budget**.
259
+ - The code is **re-indexed at the parent commit every 50 commits**, so nothing ever sees the future.
260
+ - The learning arm learns from each commit **after** answering it, exactly as it would from a merged PR.
261
+
262
+ Arms: **vector RAG** (bge-small-en-v1.5 over 40-line chunks, local via fastembed), **BM25 RAG** (same
263
+ chunks), **graph without learning**, **Klew**, and **Klew + vector seeds**.
264
+
265
+ Metrics: **right file in context** (share of changed files inside the 2k-token pack, which is what an agent
266
+ reads), **recall@5 / recall@10 / hit@1 / MRR** of the file ranking, **tokens per correct file**, and latency.
267
+
268
+ Limits, stated plainly:
269
+ - Commit messages are terser than real agent prompts.
270
+ - We score Python files only.
271
+ - Static call resolution is best-effort.
272
+ - Parameters were tuned on the first 50 commits of the first repository only (`scripts/tune.py`).
273
+
274
+ ## Repository map
275
+
276
+ | Path | What |
277
+ |---|---|
278
+ | `klew/schema.py` | labels, edge types, walk steps and priors |
279
+ | `klew/adapters/` | HydraDB + in-memory stores, Cypher, tree-sitter reader, git miner |
280
+ | `klew/domain/` | pure logic: seeder, walker, packer, learning rule, BM25 |
281
+ | `klew/services/` | indexing and the per-repo engine |
282
+ | `klew/{mcp_server,cli,api}.py` | interfaces |
283
+ | `klew/baselines/`, `klew/eval/` | RAG baselines, replay harness, plots |
284
+ | `web/` | landing page, live dashboard, results (Vite + React) |
285
+ | `tests/` | domain tests + a store contract test run against both stores |
286
+
287
+ ## License
288
+
289
+ MIT for this repository. HydraDB is licensed separately under the Elastic License 2.0; we use its
290
+ published container image and do not redistribute it.
klew-0.1.0/README.md ADDED
@@ -0,0 +1,246 @@
1
+ # Klew
2
+
3
+ **A self-improving context layer for coding agents, built on [HydraDB](https://github.com/hydra-db/hydradb).**
4
+
5
+ Every session, a coding agent rediscovers your repository by grepping and pasting whole files into its
6
+ context. Klew gives it a small map instead: the functions, callers, tests and files that matter for
7
+ *this* task, each with the path through the code that connects it to the task. It builds that map by
8
+ walking a code graph in HydraDB, not by pulling the most similar-looking chunks. Every merged PR teaches
9
+ the graph which paths actually led to the code that changed, so the next map is sharper.
10
+
11
+ > A *clew* is the ball of thread Ariadne gave Theseus to find his way through the labyrinth; it's where
12
+ > the word *clue* comes from. Klew is that thread for your codebase, and every merge makes it stronger.
13
+
14
+ <!-- RESULTS:START -->
15
+ ## Results
16
+
17
+ **In one line:** on every repository, Klew puts the right file in the agent's context 31–47 points more often
18
+ than vector RAG at the same budget (statistically significant everywhere), and learning from the repo's history
19
+ significantly improves ranking on all three. On ranking alone, Klew with vector seeds beats vector RAG
20
+ significantly on httpx and flask; on ODS only its MRR gain is significant.
21
+
22
+ **How to read these numbers.** *Right file in context*: share of changed files inside the 2,000-token
23
+ pack, which is what the agent reads (our headline). *Tokens per correct file*: context tokens spent per
24
+ changed file found (lower is cheaper). *Recall@5*: share of changed files in the top five. *Top-1*: how
25
+ often the #1 file changed. *MRR*: 1 if the first correct file is #1, ½ at #2, ⅓ at #3, and so on.
26
+
27
+ Every number below is on **held-out commits**: the first 100 commits of each window were used for
28
+ tuning (`scripts/tune.py`), and these are the rest. Every approach gets the same 2,000-token budget.
29
+
30
+ ### httpx: 199 held-out commits (first 100 commits used for tuning)
31
+
32
+ | Approach | Right file in context | Tokens per correct file | Recall@5 | Top-1 | MRR |
33
+ |---|---|---|---|---|---|
34
+ | Vector RAG | 58.1% | 3,392 | 63.9% | 47.2% | 62.2% |
35
+ | BM25 RAG | 49.1% | 3,685 | 56.2% | 39.2% | 52.3% |
36
+ | Klew, learning off | 83.4% | 1,624 | 56.0% | 44.7% | 56.9% |
37
+ | Klew without graph traversal | 87.6% | 675 | 65.5% | 49.2% | 62.8% |
38
+ | **Klew** | 88.9% | 2,025 | 65.4% | 50.7% | 63.6% |
39
+ | **Klew + vector seeds** | 95.3% | 1,986 | 67.3% | 56.8% | 68.8% |
40
+
41
+ **Key comparisons** (paired 95% bootstrap intervals over these commits; ✓ = the interval excludes 0):
42
+
43
+ | Comparison | Right file in context | Recall@5 | Top-1 | MRR |
44
+ |---|---|---|---|---|
45
+ | Klew + vector seeds vs vector RAG | +37.2 [+31.7, +43.1] ✓ | +3.4 [+0.1, +6.9] ✓ | +9.5 [+2.5, +16.6] ✓ | +6.6 [+2.5, +10.9] ✓ |
46
+ | Klew vs vector RAG | +30.9 [+24.8, +36.8] ✓ | +1.5 [-3.9, +6.3] | +3.5 [-4.0, +11.1] | +1.4 [-3.7, +6.7] |
47
+ | Learning on vs off | +5.6 [+1.7, +9.7] ✓ | +9.4 [+4.8, +13.9] ✓ | +6.0 [+1.5, +10.1] ✓ | +6.7 [+3.4, +10.1] ✓ |
48
+ | Graph walk on vs off | +1.4 [-0.7, +3.7] | -0.1 [-0.8, +0.5] | +1.5 [+0.0, +3.5] | +0.8 [-0.2, +2.0] |
49
+
50
+ - **Learning** (same graph, learning off vs on): +5.6 pts right file in context.
51
+ - **Graph traversal** (Klew with vs without the walk): +1.4 pts right file in context, +1.5 pts top-1.
52
+
53
+ **Accuracy as the graph learns** (learn from N past commits, then freeze; held-out): right file in context 82.0% → 89.1%, recall@5 55.7% → 64.3%, top-1 45.2% → 50.2%, MRR 57.0% → 63.0% from 0 to 600 commits learned.
54
+
55
+ ![accuracy as the graph learns](docs/img/httpx-learning.png)
56
+
57
+ ![learning curve over the replay](docs/img/httpx-curve.png)
58
+
59
+ ### flask: 196 held-out commits (never used for tuning; every setting came from httpx)
60
+
61
+ | Approach | Right file in context | Tokens per correct file | Recall@5 | Top-1 | MRR |
62
+ |---|---|---|---|---|---|
63
+ | Vector RAG | 45.1% | 4,393 | 48.8% | 36.7% | 49.1% |
64
+ | BM25 RAG | 53.2% | 3,311 | 55.0% | 39.3% | 51.5% |
65
+ | Klew, learning off | 84.7% | 1,472 | 48.5% | 37.8% | 49.7% |
66
+ | Klew without graph traversal | 84.1% | 703 | 60.0% | 44.4% | 56.8% |
67
+ | **Klew** | 89.0% | 2,020 | 60.6% | 43.9% | 57.1% |
68
+ | **Klew + vector seeds** | 91.6% | 2,087 | 60.1% | 46.4% | 58.4% |
69
+
70
+ **Key comparisons** (paired 95% bootstrap intervals over these commits; ✓ = the interval excludes 0):
71
+
72
+ | Comparison | Right file in context | Recall@5 | Top-1 | MRR |
73
+ |---|---|---|---|---|
74
+ | Klew + vector seeds vs vector RAG | +46.5 [+40.4, +52.2] ✓ | +11.3 [+6.7, +16.2] ✓ | +9.7 [+3.1, +16.3] ✓ | +9.3 [+4.7, +14.1] ✓ |
75
+ | Klew vs vector RAG | +43.9 [+37.9, +50.4] ✓ | +11.8 [+5.4, +18.4] ✓ | +7.1 [+0.0, +14.3] | +8.0 [+2.5, +13.3] ✓ |
76
+ | Learning on vs off | +4.3 [+1.1, +7.8] ✓ | +12.1 [+6.9, +17.6] ✓ | +6.1 [+0.5, +11.7] ✓ | +7.4 [+3.6, +11.4] ✓ |
77
+ | Graph walk on vs off | +4.9 [+1.5, +8.6] ✓ | +0.6 [-0.5, +2.0] | -0.5 [-1.5, +0.0] | +0.3 [-0.5, +0.9] |
78
+
79
+ - **Learning** (same graph, learning off vs on): +4.3 pts right file in context.
80
+ - **Graph traversal** (Klew with vs without the walk): +4.9 pts right file in context, -0.5 pts top-1.
81
+
82
+ **Accuracy as the graph learns** (learn from N past commits, then freeze; held-out): right file in context 85.0% → 87.9%, recall@5 48.5% → 58.3%, top-1 37.8% → 42.4%, MRR 49.4% → 55.0% from 0 to 600 commits learned.
83
+
84
+ ![accuracy as the graph learns](docs/img/flask-learning.png)
85
+
86
+ ![learning curve over the replay](docs/img/flask-curve.png)
87
+
88
+ ### ods: 166 held-out commits (never used for tuning; every setting came from httpx)
89
+
90
+ | Approach | Right file in context | Tokens per correct file | Recall@5 | Top-1 | MRR |
91
+ |---|---|---|---|---|---|
92
+ | Vector RAG | 35.0% | 5,557 | 45.2% | 39.2% | 51.3% |
93
+ | BM25 RAG | 30.7% | 6,282 | 41.8% | 33.7% | 47.0% |
94
+ | Klew, learning off | 69.3% | 2,642 | 41.3% | 33.7% | 46.6% |
95
+ | Klew without graph traversal | 76.9% | 1,206 | 46.6% | 41.6% | 53.3% |
96
+ | **Klew** | 72.4% | 2,702 | 47.3% | 41.6% | 53.5% |
97
+ | **Klew + vector seeds** | 71.9% | 2,753 | 48.3% | 45.8% | 57.2% |
98
+
99
+ **Key comparisons** (paired 95% bootstrap intervals over these commits; ✓ = the interval excludes 0):
100
+
101
+ | Comparison | Right file in context | Recall@5 | Top-1 | MRR |
102
+ |---|---|---|---|---|
103
+ | Klew + vector seeds vs vector RAG | +36.9 [+31.8, +42.3] ✓ | +3.1 [-0.5, +6.8] | +6.6 [-0.6, +13.9] | +5.9 [+1.3, +10.2] ✓ |
104
+ | Klew vs vector RAG | +37.5 [+30.3, +44.1] ✓ | +2.1 [-5.2, +9.4] | +2.4 [-6.0, +10.8] | +2.2 [-3.8, +8.4] |
105
+ | Learning on vs off | +3.1 [-2.1, +8.4] | +6.0 [+2.3, +10.2] ✓ | +7.8 [+4.2, +12.0] ✓ | +6.9 [+4.2, +9.6] ✓ |
106
+ | Graph walk on vs off | -4.4 [-9.0, +0.1] | +0.6 [-0.6, +2.1] | +0.0 [+0.0, +0.0] | +0.1 [-0.7, +1.0] |
107
+
108
+ - **Learning** (same graph, learning off vs on): +3.1 pts right file in context.
109
+ - **Graph traversal** (Klew with vs without the walk): -4.4 pts right file in context, +0.0 pts top-1.
110
+
111
+ **Accuracy as the graph learns** (learn from N past commits, then freeze; held-out): right file in context 68.6% → 69.5%, recall@5 41.7% → 43.8%, top-1 35.5% → 40.4%, MRR 48.3% → 52.0% from 0 to 300 commits learned.
112
+
113
+ ![accuracy as the graph learns](docs/img/ods-learning.png)
114
+
115
+ ![learning curve over the replay](docs/img/ods-curve.png)
116
+
117
+ Latency: Klew answers in about 0.2–0.4 s per request versus about 15 ms for a vector lookup.
118
+
119
+ <!-- RESULTS:END -->
120
+
121
+ ## How it works
122
+
123
+ ```
124
+ repo ──► tree-sitter indexer ──┐ ┌──────────── HydraDB ────────────┐
125
+ git history ──► co-change ─────┼── UNWIND/MERGE ─────► │ files · symbols · calls · tests │
126
+ │ │ imports · inheritance │
127
+ │ │ CO_CHANGED · POINTS_TO (learned)│
128
+ │ └───────▲──────────────▲──────────┘
129
+ agent ─MCP─► get_context ─► seed ─► walk (1-hop expansions) ───┘ │
130
+ │ rank ─► pack (signatures + snippet, │
131
+ │ why-path per file) │
132
+ merged PR / record_outcome ──┴──► learning rule ─► reinforce / decay ─────────┘
133
+ ```
134
+
135
+ 1. **Map.** Tree-sitter parses the repo into files and symbols, with calls, imports, inheritance and tests
136
+ between them, resolved statically through each file's imports. Git history adds `CO_CHANGED` edges between
137
+ files that change together. All of it lives in HydraDB.
138
+ 2. **Walk.** For a task, Klew seeds from names in the task: BM25 over symbol names, docstrings and body
139
+ terms, exact identifiers, plus anything it has learned. It then propagates relevance hop by hop through
140
+ HydraDB (calls, callers, tests, inheritance, imports, co-change), keeping the shortest path to every
141
+ node. Files are ranked, then packed into the token budget as signatures plus the best snippet, each with
142
+ its why-path.
143
+ 3. **Learn.** When a run or merged PR lands, the files that actually changed reinforce `POINTS_TO` edges from
144
+ the task's words to those files, and files that were served but not touched decay. Files touched together
145
+ reinforce `CO_CHANGED`. Structural edges are facts and never change; experience is learned on top, so
146
+ re-indexing never erases what the graph has learned.
147
+
148
+ ### Why a graph and not a vector store
149
+
150
+ - **The file you need often doesn't look like the task.** It's the caller, the test, or the config two hops
151
+ away. Similarity can't follow a call edge.
152
+ - **It learns without re-embedding.** Outcomes update edge weights in place.
153
+ - **Every item says why it's there.** A path through the code is auditable; a cosine score isn't.
154
+ - **It spends fewer tokens.** Structure tells us which signature and which body to include.
155
+
156
+ ### How it uses HydraDB
157
+
158
+ - **Typed graph:** `KlewFile`, `KlewSymbol`, `KlewTerm`, `KlewRun`, `KlewRepo` nodes. `CONTAINS`, `IMPORTS`, `CALLS`,
159
+ `TESTS`, `INHERITS` (structural), `CO_CHANGED`, `POINTS_TO` (learned), `SERVED`, `TOUCHED` (provenance).
160
+ - **Traversal where the graph lives:** each hop is one anchored 1-hop expansion per (edge type, direction),
161
+ e.g. `MATCH (a:KlewSymbol)<-[r:CALLS]-(b) WHERE a.id IN $ids RETURN a.id, b.id, r.w`. That's about 34
162
+ queries per context pack, run in parallel within each hop: about 0.2–0.3 s on httpx (warm-started
163
+ graph, laptop, Docker).
164
+ - **Learning writes back in place:** batched `UNWIND … MERGE (s)-[r:POINTS_TO {id: row.eid}]->(d) SET r.w = row.w`.
165
+ - **Native path search for explanations:** `explain` calls HydraDB's `algo.MSpaths` (GraphBLAS) to find the paths
166
+ through the code from a task's best matches to a file, e.g. `_get_proxy_map` → calls → `get_environment_proxies`
167
+ → contained in → `httpx/_utils.py`.
168
+ - **Run history as a graph:** every context request becomes a `KlewRun` with `SERVED` and `TOUCHED` edges, an
169
+ audit trail of what the agent was shown and what it changed.
170
+ - All Cypher lives in [`klew/adapters/queries.py`](klew/adapters/queries.py), with the dialect
171
+ rules we verified against HydraDB v0.4.0.
172
+
173
+ ## Install
174
+
175
+ Klew needs Docker (for a local HydraDB node) and [uv](https://docs.astral.sh/uv/) or pip.
176
+
177
+ ```bash
178
+ pip install klew # or run without installing: uvx klew …
179
+ klew hydra up # local HydraDB node in Docker
180
+ klew index ~/code/your-repo # build the graph and learn from recent history
181
+ klew context --repo your-repo "what you want to change"
182
+ ```
183
+
184
+ **Claude Code** (plugin with the MCP server and a skill that tells Claude when to use it):
185
+
186
+ ```text
187
+ /plugin install klew --marketplace IronicRayquaza/klew
188
+ ```
189
+
190
+ **Claude Desktop**: download `klew.mcpb` from the [latest release](https://github.com/IronicRayquaza/klew/releases)
191
+ and double-click it. It asks for your repository folder.
192
+
193
+ **Cursor, Windsurf and other MCP clients**:
194
+
195
+ ```json
196
+ { "mcpServers": { "klew": { "command": "npx", "args": ["-y", "klew", "mcp"],
197
+ "env": { "KLEW_REPO_ROOT": "/absolute/path/to/your/repo" } } } }
198
+ ```
199
+
200
+ MCP tools: `get_context(task, budget_tokens, cutoff)`, `record_outcome(run_id, touched_files?)` (defaults to the
201
+ working tree's changed files), `explain(task, path)`, and `graph_stats()`. There is deliberately no raw-Cypher
202
+ tool.
203
+
204
+ **Learn from merged PRs:** `klew learn-git <merge-sha>`, or the GitHub Action in [`action/`](action/action.yml).
205
+
206
+ **From source / dashboard:** `docker compose up -d`, `uv pip install -e ".[dev]"`, `klew serve` (API on :8000)
207
+ and `cd web && npm install && npm run dev` (UI on :5173).
208
+
209
+ ## Evaluation
210
+
211
+ `klew eval <repo> --window 300` replays the last 300 commits that modified 1–8 Python files, in order:
212
+
213
+ - **task** = the commit message; **truth** = the `.py` files it modified that exist in the snapshot.
214
+ - Every arm gets the **same 2,000-token budget**.
215
+ - The code is **re-indexed at the parent commit every 50 commits**, so nothing ever sees the future.
216
+ - The learning arm learns from each commit **after** answering it, exactly as it would from a merged PR.
217
+
218
+ Arms: **vector RAG** (bge-small-en-v1.5 over 40-line chunks, local via fastembed), **BM25 RAG** (same
219
+ chunks), **graph without learning**, **Klew**, and **Klew + vector seeds**.
220
+
221
+ Metrics: **right file in context** (share of changed files inside the 2k-token pack, which is what an agent
222
+ reads), **recall@5 / recall@10 / hit@1 / MRR** of the file ranking, **tokens per correct file**, and latency.
223
+
224
+ Limits, stated plainly:
225
+ - Commit messages are terser than real agent prompts.
226
+ - We score Python files only.
227
+ - Static call resolution is best-effort.
228
+ - Parameters were tuned on the first 50 commits of the first repository only (`scripts/tune.py`).
229
+
230
+ ## Repository map
231
+
232
+ | Path | What |
233
+ |---|---|
234
+ | `klew/schema.py` | labels, edge types, walk steps and priors |
235
+ | `klew/adapters/` | HydraDB + in-memory stores, Cypher, tree-sitter reader, git miner |
236
+ | `klew/domain/` | pure logic: seeder, walker, packer, learning rule, BM25 |
237
+ | `klew/services/` | indexing and the per-repo engine |
238
+ | `klew/{mcp_server,cli,api}.py` | interfaces |
239
+ | `klew/baselines/`, `klew/eval/` | RAG baselines, replay harness, plots |
240
+ | `web/` | landing page, live dashboard, results (Vite + React) |
241
+ | `tests/` | domain tests + a store contract test run against both stores |
242
+
243
+ ## License
244
+
245
+ MIT for this repository. HydraDB is licensed separately under the Elastic License 2.0; we use its
246
+ published container image and do not redistribute it.
@@ -0,0 +1 @@
1
+ """Klew: a self-improving code-context layer for coding agents, on HydraDB."""
File without changes