dsh-knit 0.18.0 → 0.19.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +229 -7
- package/README.en.md +291 -180
- package/README.md +248 -112
- package/SECURITY.md +4 -3
- package/package.json +9 -4
- package/src/client/client.js +451 -47
- package/src/host/classification.js +250 -0
- package/src/host/context.js +31 -9
- package/src/host/index.js +336 -77
- package/src/host/tool.js +29 -13
- package/test/classification.test.mjs +237 -0
- package/test/client.test.mjs +431 -3
- package/test/code-context.test.mjs +567 -0
- package/test/eval/fixture.mjs +309 -6
- package/test/eval.test.mjs +117 -1
- package/test/host.test.mjs +63 -3
- package/test/path.test.mjs +97 -5
- package/test/tool.test.mjs +23 -15
package/README.en.md
CHANGED
|
@@ -1,17 +1,51 @@
|
|
|
1
1
|
# Knit
|
|
2
2
|
|
|
3
|
-
>
|
|
4
|
-
> Knit puts them next to the conversation — whatever you're talking about, the relevant doc is on top.
|
|
3
|
+
> **Task-aware workspace context retrieval and lifecycle tracking for AI coding agents.**
|
|
5
4
|
>
|
|
6
|
-
>
|
|
5
|
+
> **面向 AI Coding Agent 的任务感知工作区上下文检索与生命周期追踪。**
|
|
6
|
+
|
|
7
|
+
Knit is the layer between **the task you are on** and **your whole workspace**: it finds the most
|
|
8
|
+
relevant documents, source files and media, organises them into **Primary / Supporting / Related**
|
|
9
|
+
context (the Context Pack) for both the panel and the agent, and keeps tracking **whether they were
|
|
10
|
+
read, and whether they changed after being read**.
|
|
11
|
+
|
|
12
|
+
**Find → Organise → Track**, all of it plain string work on your own machine — no model calls, no
|
|
13
|
+
network, no latency, nothing leaves the machine.
|
|
14
|
+
|
|
15
|
+
```text
|
|
16
|
+
Current task
|
|
17
|
+
│
|
|
18
|
+
Workspace retrieval
|
|
19
|
+
┌───────────────┼───────────────┐
|
|
20
|
+
▼ ▼ ▼
|
|
21
|
+
Primary Supporting Related
|
|
22
|
+
└───────────────┼───────────────┘
|
|
23
|
+
▼
|
|
24
|
+
Context Pack ──► the `knit_docs` tool (for the agent)
|
|
25
|
+
▼
|
|
26
|
+
Agent reads
|
|
27
|
+
▼
|
|
28
|
+
Read Evidence · Context Epoch
|
|
29
|
+
▼
|
|
30
|
+
Workspace changes
|
|
31
|
+
▼
|
|
32
|
+
Re-read · Lifecycle
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
> **It is not a smarter recent-files list, and it is not a code browser.**
|
|
36
|
+
> A recent-files list records what you happened to open and what is newest; Knit answers
|
|
37
|
+
> **which things in this project are worth looking at first for the task at hand — and what
|
|
38
|
+
> happened to them after they were read.**
|
|
7
39
|
|
|
8
|
-

|
|
9
41
|
|
|
10
|
-
*Screenshot from a real machine, not a mock-up:
|
|
11
|
-
|
|
12
|
-
*The
|
|
13
|
-
|
|
14
|
-
*
|
|
42
|
+
*Screenshot from a real machine, not a mock-up: the same workspace, this one left on the **Code tab** —
|
|
43
|
+
since v0.19 source files are first-class citizens, just like Markdown.*
|
|
44
|
+
*The "why" line under each row says how it matched: a direct hit on the task text, a path hit, or a
|
|
45
|
+
workspace hit.*
|
|
46
|
+
*Click a row to expand the preview **in place**: here `knit/src/host/classification.js`, with line
|
|
47
|
+
numbers; the preview header holds only the path and four actions (copy / full screen / open locally / close).*
|
|
48
|
+
*The header holds only the path, one count and two toggles; whether the list runs by
|
|
15
49
|
relevance or by newest-first is decided by that pair of tabs on the left.*
|
|
16
50
|
*Since v0.14 the list is always one column: the rank number sits in its own column on the far left
|
|
17
51
|
(vertically centred on the title's first line), then Primary dot + title + relative time (time on the
|
|
@@ -35,104 +69,128 @@ dsh plugin --profile web add dsh-knit
|
|
|
35
69
|
|
|
36
70
|
---
|
|
37
71
|
|
|
38
|
-
##
|
|
72
|
+
## Why it exists
|
|
73
|
+
|
|
74
|
+
> An agent produces 20 documents a day, and you cannot find the one from a minute ago.
|
|
39
75
|
|
|
40
|
-
**
|
|
76
|
+
That is the problem it started from: **output got faster than anyone can browse directories.**
|
|
77
|
+
DSH's built-in "recent files" only records what you opened; once a turn ends, the earlier ones sink
|
|
78
|
+
out of sight.
|
|
41
79
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
80
|
+
Knit asks a different question: **not "what is newest", but "what is relevant to what I am doing".**
|
|
81
|
+
It scans the **whole workspace** (not just what this turn produced) and ranks by what you are
|
|
82
|
+
talking about — restart DSH, open a new session, come back the next day, and it is still there.
|
|
45
83
|
|
|
46
|
-
|
|
84
|
+
Then the problem grew one step further: when you actually change a feature, the first thing worth
|
|
85
|
+
reading is usually not a document but **a few lines of source**. That is what v0.19 added. And after
|
|
86
|
+
that, "ranked first" stopped being enough — **of the things it ranked, did the agent actually read
|
|
87
|
+
them? Did they change afterwards?** v0.15–v0.18 filled in that layer.
|
|
47
88
|
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
2. The order **follows the conversation**: talk about architecture and the architecture doc
|
|
51
|
-
floats up; talk about competitors and the competitor analysis floats up.
|
|
52
|
-
3. **No model calls, no network egress**: plain local string matching — zero latency,
|
|
53
|
-
zero cost, your documents never leave the machine.
|
|
89
|
+
Three words: **Find** (scan the workspace, rank by the current task) → **Organise** (three tiers,
|
|
90
|
+
each with its reason) → **Track** (read or not, changed or not).
|
|
54
91
|
|
|
55
92
|
---
|
|
56
93
|
|
|
57
|
-
##
|
|
94
|
+
## It is not a recent-files list, and not a code browser
|
|
58
95
|
|
|
59
|
-
|
|
96
|
+
Both comparisons catch one **surface** of Knit and miss the question it answers.
|
|
60
97
|
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
98
|
+
| | Recent-files list | Code browser | **Knit** |
|
|
99
|
+
|---|---|---|---|
|
|
100
|
+
| Scope | What you happened to open | A directory tree | **The whole workspace**: documents / code / images / video |
|
|
101
|
+
| Order | Modified time | You scroll it yourself | **Relevance to the current task** (BM25 + IDF, computed locally) |
|
|
102
|
+
| Output | One flat list | The one file you opened | **Primary / Supporting / Related**, each saying why it is there |
|
|
103
|
+
| For whom | You | You | You **and** the agent — the same Context Pack, through `knit_docs` |
|
|
104
|
+
| Afterwards | Nothing | Nothing | **Was it read? Did it change after being read?** (Read Evidence + lifecycle) |
|
|
64
105
|
|
|
65
|
-
|
|
106
|
+
The three differences that matter:
|
|
66
107
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
(「图片」, 「排序」) out of the query entirely
|
|
74
|
-
- stop-word filtering plus greedy de-overlap (picking「相关性排序」drops「相关性」and「排序」)
|
|
108
|
+
1. **Scope**: a recent-files list only records the files you **opened**; Knit scans **the whole project
|
|
109
|
+
folder** — restart DSH, open a new session, come back days later, and it is still there.
|
|
110
|
+
2. **Order**: it sorts by time; Knit sorts by **what you are talking about** — discuss architecture and
|
|
111
|
+
the architecture docs rise; discuss a loader bug and `plugin-loader.ts` rises.
|
|
112
|
+
3. **It does not stop at finding**: after finding comes **organising** (three tiers, each with a
|
|
113
|
+
verifiable reason) and **tracking** (which were really read, and whether they changed).
|
|
75
114
|
|
|
76
|
-
**
|
|
115
|
+
**No model calls, no network**: all plain string work, no latency, no cost, nothing leaves the machine.
|
|
77
116
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## Install
|
|
120
|
+
|
|
121
|
+
```sh
|
|
122
|
+
dsh plugin --profile web add dsh-knit
|
|
84
123
|
```
|
|
85
124
|
|
|
86
|
-
**
|
|
125
|
+
Then **restart DSH** and hard-refresh the browser (`Cmd + Shift + R`).
|
|
87
126
|
|
|
88
|
-
|
|
89
|
-
rare one — so the frequent term discriminates nothing and dilutes the rare ones
|
|
90
|
-
- with no length normalisation, **a long document wins by piling up hits**
|
|
91
|
-
- capping hits at 6 was a hand-drawn knee; `k1` / `b` exist precisely for this
|
|
127
|
+
**How to open it:**
|
|
92
128
|
|
|
93
|
-
|
|
129
|
+
- The **Knit icon button in the session header** (right next to the sidebar toggle) — one click
|
|
130
|
+
- Or the right sidebar's tab bar「+」→「Knit recent docs」
|
|
94
131
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
| old (weighted hits) | 76.2% | 0.830 |
|
|
98
|
-
| **BM25** | **95.2%** | **0.976** |
|
|
132
|
+
**Optional**: if `dsh-better-sidebar` is installed, the panel also registers
|
|
133
|
+
as one of its tabs. Each host is an independent optional dependency; missing one doesn't affect the other.
|
|
99
134
|
|
|
100
|
-
|
|
101
|
-
engine frozen in `test/eval/legacy.mjs` — so "the new engine must be clearly better" is
|
|
102
|
-
verified automatically rather than asserted against a hard-coded number.
|
|
135
|
+
**Upgrading**: same command as installing, and the same restart + hard-refresh afterwards.
|
|
103
136
|
|
|
104
|
-
|
|
105
|
-
the corpus is built with a real trap — 12 short, focused topic documents, plus a pile of long
|
|
106
|
-
distractors that mention every topic five times without explaining any of them (which is what a
|
|
107
|
-
real project's CHANGELOG looks like). Half the topic documents have descriptive filenames, half
|
|
108
|
-
are opaque.
|
|
137
|
+
---
|
|
109
138
|
|
|
110
|
-
|
|
111
|
-
|---|---|---|---|
|
|
112
|
-
| **Knit (BM25)** | **100%** | **100%** | **1.000 (flat)** |
|
|
113
|
-
| `grep -c` keyword counting | 17% | **0%** | 0.313 → **0.089** |
|
|
114
|
-
| Filenames only | 100% | **0%** | 0.602 |
|
|
139
|
+
## Features
|
|
115
140
|
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
141
|
+
| | |
|
|
142
|
+
|---|---|
|
|
143
|
+
| **Code Context** (v0.19): source files join the **same** retrieval pipeline as documents — classify, BM25, Context Pack, lifecycle. A dedicated classification layer decides what a file *is* and what it may do, so extension knowledge lives in exactly one place | ✅ |
|
|
144
|
+
| **Sorted by relevance to the current conversation** (BM25 + IDF, fully local, no model) | ✅ |
|
|
145
|
+
| **The `knit_docs` tool for the agent**: the model can look up this project's most relevant documents itself | ✅ |
|
|
146
|
+
| **Document lifecycle** (v0.17): each recommended document is shown as `Unread / Read / Updated after read / Re-read after update`, alongside the most recent read, how many reads fell outside the pack (expandable to which ones), and the latest context change | ✅ |
|
|
147
|
+
| One-click toggle between relevance / modification time (preference kept in localStorage) | ✅ |
|
|
148
|
+
| Scans **documents and code** in the session workspace (recursive, depth ≤ 6, skips `node_modules` / `.git` / `dist` / `build` / `out` / `coverage`) | ✅ |
|
|
149
|
+
| Each row shows H1 title (or filename) + relative time + first-paragraph summary | ✅ |
|
|
150
|
+
| The document list is **always one column** (v0.14 — the multi-column layout was deleted outright, not switched off); the **number sits in its own column on the far left** and is **vertically centred with the title line**, everything else stacks to its right starting with **one line: Primary dot + title + relative time** (the time is pushed to the right edge), then the summary. **The path is no longer shown in the list** — it duplicates the clickable **path** in the preview header, which is the one that stays | ✅ |
|
|
151
|
+
| Click to preview inline, click again to collapse | ✅ |
|
|
152
|
+
| Relative-path images actually render (`./img/a.png`, `../assets/b.png`) | ✅ |
|
|
153
|
+
| One-click switch between **Docs / Code / Media / All** (v0.19 adds Code; remembered, defaults to Docs); the selected tab is an **underline tab**, deliberately different from the list row's grey fill | ✅ |
|
|
154
|
+
| Images & video: square thumbnail grid — cells have a **fixed 104px baseline independent of the item count**, columns come from CSS `repeat(auto-fill, minmax(104px, 1fr))`, and **as many rows as there are items are laid out** (no max height, the grid never scrolls itself — scrolling belongs to the list); videos auto-grab the first frame with a play glyph and duration badge (no deps, no transcoding) | ✅ |
|
|
155
|
+
| **"Web preview" for `.html`** (v0.19): the preview header hands the file to DSH's own document-preview tab (`openResource`) instead of showing source; the inline preview still shows source, so the two do not duplicate. Knit itself sends no request | ✅ |
|
|
156
|
+
| Click an image / video to preview **inline**: large image, playable & seekable video streamed over HTTP Range (no full download) | ✅ |
|
|
157
|
+
| The **All** view splits into **three stacked sections**: docs (max 4, with "View all →" when truncated), **code** (max 4), then images & video (**never truncated**, count only) | ✅ |
|
|
158
|
+
| Preview pane is height-draggable (20%–80%, remembered), fullscreen-able, `Esc` to exit | ✅ |
|
|
159
|
+
| **"Open locally"** opens the document in your default app; the **path in the preview header is clickable** and now opens the system file manager with the file **selected** (`action:'reveal'`), and its tooltip carries the full relative path | ✅ |
|
|
160
|
+
| Double-click opens in a new tab (the official document preview, with its PDF renderer and renderer switching) | ✅ |
|
|
161
|
+
| Filter box over title / summary / path | ✅ |
|
|
162
|
+
| **Click the workspace path** to open the project folder in your file manager | ✅ |
|
|
163
|
+
| **Hover the entry button to peek**: a read-only floating list of the 5 most recent docs; click to open the right sidebar (doesn't push the layout; the popover is just header + list — no extra divider or hint line) | ✅ |
|
|
164
|
+
| Keyboard: `↑` `↓` move-and-preview, `Enter` toggle, `Esc` collapse (`←` `→` span rows in the **media grid only** — the document list is always one column, so they have no spatial meaning there) | ✅ |
|
|
165
|
+
| Auto-refresh every 5s plus a manual button; docs changed in the last 2 min get 🆕 | ✅ |
|
|
166
|
+
| Bilingual (zh/en), follows the DSH language live — no plugin reload needed | ✅ |
|
|
167
|
+
| Zero model calls, zero network egress | ✅ |
|
|
168
|
+
| **A `knit_docs` tool for the agent** — read-only, so the model can find this project's relevant docs itself | ✅ |
|
|
119
169
|
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
include fragments that straddle two words (「项目文档」 yields `项目文` / `目文档`), and showing
|
|
123
|
-
those renders as gibberish — merging their spans and slicing the original text recovers `项目文档`.
|
|
124
|
-
The label is **your own wording**, so its casing is preserved (type `BM25`, see `BM25`).
|
|
170
|
+
> Relevance is deliberately **not visualised** (no percentages, no bars) — the ranking itself is
|
|
171
|
+
> the answer; position is relevance.
|
|
125
172
|
|
|
126
|
-
|
|
173
|
+
### The other two views
|
|
127
174
|
|
|
128
|
-
|
|
175
|
+

|
|
129
176
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
177
|
+
*Media: square thumbnails, with the **column count following the panel width continuously** —
|
|
178
|
+
six columns at this width, cells about 110px. These are the workspace's real image and SVG files; this
|
|
179
|
+
particular workspace happens to hold several screenshots, one demo GIF and one single-colour SVG icon
|
|
180
|
+
(the solid black square is that icon, not a failed load). Videos get their first frame as a poster with
|
|
181
|
+
a play glyph and a duration badge — this workspace simply has no video, so none shows here.*
|
|
182
|
+
|
|
183
|
+

|
|
184
|
+
|
|
185
|
+
*All: the documents section shows at most 4, the **code section** shows at most 4 (each gets its own
|
|
186
|
+
"View all →" when truncated — here "4 / 19" and "4 / 17"), and the media grid comes last.*
|
|
187
|
+
|
|
188
|
+
⚠️ All three sections share the **same 40-item window** the host returns — a category that ranks past
|
|
189
|
+
40th by relevance drops out of the All view entirely. This screenshot uses the **Newest** sort: in this
|
|
190
|
+
workspace, under the default Relevance sort, media rank outside that window every time (at `limit=40`
|
|
191
|
+
those 40 rows are 34 documents + 6 code files). The **Media** and **Code** tabs are unaffected — they
|
|
192
|
+
request their own kind. A known defect, with reproduction and the fix recorded in
|
|
193
|
+
[`docs/README.md`](docs/README.md).*
|
|
136
194
|
|
|
137
195
|
---
|
|
138
196
|
|
|
@@ -217,6 +275,41 @@ There is **no** "the AI thinks this matters" and **no** percentages, stars or co
|
|
|
217
275
|
|
|
218
276
|
---
|
|
219
277
|
|
|
278
|
+
## Code as context (v0.19)
|
|
279
|
+
|
|
280
|
+
Until v0.18 Knit understood Markdown, images and video. v0.19's product concept becomes
|
|
281
|
+
**documents / code / media** — not "a few more extensions", but letting Knit put the source files an
|
|
282
|
+
AI-coding task actually needs into the same Context Pack, on the same main line:
|
|
283
|
+
|
|
284
|
+
```text
|
|
285
|
+
Current task -> Workspace Retrieval -> Document + Code Candidates
|
|
286
|
+
-> Primary / Supporting / Related -> Context Pack -> Agent Read
|
|
287
|
+
-> Read Evidence -> File Changed -> Lifecycle -> Context Epoch / Delta -> Usage Lens
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
- **One classification layer, not scattered `if (ext === '.js')`** — `src/host/classification.js`
|
|
291
|
+
turns a filename into `{ kind, language, previewable, searchable, contextual, generated,
|
|
292
|
+
sourceMap }`. The scanner, retrieval, preview and UI all just read that verdict.
|
|
293
|
+
- **Supported in v0.19**: `.js .mjs .cjs .ts .tsx .jsx .py .json .html .htm .css .scss .yaml .yml
|
|
294
|
+
.sh .bash .zsh`. **Not yet**: `.go .rs .java .kt .c .cpp .h .hpp .cs .php .rb .swift .sql`.
|
|
295
|
+
- **Code gets its own field weights** — filename x4, path x2, body (first 8 KB) x1, instead of the
|
|
296
|
+
document weights (title x4 / summary x2 / body x1). A filename like `context-manager.ts` carries
|
|
297
|
+
the task signal.
|
|
298
|
+
- **Generated files are not context**: `.map`, `*.min.js`, `*.min.css`, `*.bundle.js`,
|
|
299
|
+
`*.generated.js`, `*.gen.js`, `*.lock` classify as `generated` — previewable, but never in
|
|
300
|
+
retrieval, the Context Pack or the Usage Lens. When `app.js` and `app.js.map` both exist, the
|
|
301
|
+
`app.js` preview offers one low-key **"Source map"** entry; the `.map` never shows up as a row.
|
|
302
|
+
**Context eligibility is not file-system visibility** — the built-in file tree still opens them.
|
|
303
|
+
- **Bounded and capped**: at most `MAX_CODE = 300` code candidates and a head-limited body read, so
|
|
304
|
+
a large repo cannot flood the Markdown corpus. Corpus pressure is the thing to watch in this
|
|
305
|
+
version, not recall.
|
|
306
|
+
- **The type tabs become four**: `Docs / Code / Media / All`. The Code tab is **still driven by the
|
|
307
|
+
current task** — it is not a file tree of the project.
|
|
308
|
+
- **No second engine**: no second retrieval path, no second lifecycle, no second store, no new
|
|
309
|
+
dependencies, no AST, no LSP, no embedding, no model call, no network, no IDE.
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
220
313
|
## Also for the agent: the `knit_docs` tool
|
|
221
314
|
|
|
222
315
|
The same ranking that you see in the panel is also exposed to the model.
|
|
@@ -254,99 +347,6 @@ Three details:
|
|
|
254
347
|
|
|
255
348
|
---
|
|
256
349
|
|
|
257
|
-
## Install
|
|
258
|
-
|
|
259
|
-
```sh
|
|
260
|
-
dsh plugin --profile web add dsh-knit
|
|
261
|
-
```
|
|
262
|
-
|
|
263
|
-
Then **restart DSH** and hard-refresh the browser (`Cmd + Shift + R`).
|
|
264
|
-
|
|
265
|
-
**How to open it:**
|
|
266
|
-
|
|
267
|
-
- The **Knit icon button in the session header** (right next to the sidebar toggle) — one click
|
|
268
|
-
- Or the right sidebar's tab bar「+」→「Knit recent docs」
|
|
269
|
-
|
|
270
|
-
**Optional**: if `dsh-better-sidebar` is installed, the panel also registers
|
|
271
|
-
as one of its tabs. Each host is an independent optional dependency; missing one doesn't affect the other.
|
|
272
|
-
|
|
273
|
-
**Upgrading**: same command as installing, and the same restart + hard-refresh afterwards.
|
|
274
|
-
|
|
275
|
-
---
|
|
276
|
-
|
|
277
|
-
## Feedback
|
|
278
|
-
|
|
279
|
-
**Right now this project is not about adding features — it's about finding out whether
|
|
280
|
-
"rank project docs by the current conversation" is something anyone actually uses.**
|
|
281
|
-
So the most valuable thing you can send is not "could you add X" but **how you work around
|
|
282
|
-
this today** — including "I installed it and never opened it", which is more useful than a
|
|
283
|
-
feature request.
|
|
284
|
-
|
|
285
|
-
- 💬 [Tell us when you actually open it](https://github.com/PolinniZhong/dsh-knit/issues/new?template=feature.yml) — two fields, thirty seconds
|
|
286
|
-
- 🐞 [Won't install / panel won't open / ranking looks wrong](https://github.com/PolinniZhong/dsh-knit/issues/new?template=bug.yml)
|
|
287
|
-
- 📖 Check [known limitations](#known-limitations) first — short conversations degrade to time order and Chinese uses n-gram approximation; those are deliberate trade-offs, not bugs
|
|
288
|
-
|
|
289
|
-
> A single issue is treated as a real signal. So far this plugin has **no trace of a single
|
|
290
|
-
> real user** — the npm download count is automated version enumeration, not people
|
|
291
|
-
> (only 14% of it is `latest`, and a human install only ever pulls `latest`). One human reply
|
|
292
|
-
> changes what gets built next.
|
|
293
|
-
|
|
294
|
-
---
|
|
295
|
-
|
|
296
|
-
## Features
|
|
297
|
-
|
|
298
|
-
| | |
|
|
299
|
-
|---|---|
|
|
300
|
-
| **Sorted by relevance to the current conversation** (BM25 + IDF, fully local, no model) | ✅ |
|
|
301
|
-
| **The `knit_docs` tool for the agent**: the model can look up this project's most relevant documents itself | ✅ |
|
|
302
|
-
| **Document lifecycle** (v0.17): each recommended document is shown as `Unread / Read / Updated after read / Re-read after update`, alongside the most recent read, how many reads fell outside the pack (expandable to which ones), and the latest context change | ✅ |
|
|
303
|
-
| One-click toggle between relevance / modification time (preference kept in localStorage) | ✅ |
|
|
304
|
-
| Scans `.md` in the session workspace (recursive, depth ≤ 6, skips `node_modules` / `.git` / `dist`) | ✅ |
|
|
305
|
-
| Each row shows H1 title (or filename) + relative time + first-paragraph summary | ✅ |
|
|
306
|
-
| The document list is **always one column** (v0.14 — the multi-column layout was deleted outright, not switched off); the **number sits in its own column on the far left** and is **vertically centred with the title line**, everything else stacks to its right starting with **one line: Primary dot + title + relative time** (the time is pushed to the right edge), then the summary. **The path is no longer shown in the list** — it duplicates the clickable **path** in the preview header, which is the one that stays | ✅ |
|
|
307
|
-
| Click to preview inline, click again to collapse | ✅ |
|
|
308
|
-
| Relative-path images actually render (`./img/a.png`, `../assets/b.png`) | ✅ |
|
|
309
|
-
| One-click switch between **Docs / Media / All** (remembered; defaults to Docs, unchanged); the selected tab is a **neutral grey fill**, with no coloured outline (the tab label became "Media" on 2026-10-06, was "Images & video") | ✅ |
|
|
310
|
-
| Images & video: square thumbnail grid — **at least 3 columns, more only as the pane gets wider**; cells start at 64px and the baseline is **8 per screen**; past 8 nothing is hidden, the whole grid scales down proportionally; videos auto-grab the first frame with a play glyph and duration badge (no deps, no transcoding) | ✅ |
|
|
311
|
-
| Click an image / video to preview **inline**: large image, playable & seekable video streamed over HTTP Range (no full download) | ✅ |
|
|
312
|
-
| The **All** view splits into **two stacked sections**: docs (max 4, with "View all →" when truncated) and images & video (**never truncated**, count only) | ✅ |
|
|
313
|
-
| Preview pane is height-draggable (20%–80%, remembered), fullscreen-able, `Esc` to exit | ✅ |
|
|
314
|
-
| **"Open locally"**: opens the previewed document in your default app (the path in the preview header is clickable too, and its tooltip carries the full relative path) | ✅ |
|
|
315
|
-
| Double-click opens in a new tab (the official document preview, with its PDF renderer and renderer switching) | ✅ |
|
|
316
|
-
| Filter box over title / summary / path | ✅ |
|
|
317
|
-
| **Click the workspace path** to open the project folder in your file manager | ✅ |
|
|
318
|
-
| **Hover the entry button to peek**: a read-only floating list of the 5 most recent docs; click to open the right sidebar (doesn't push the layout; the popover is just header + list — no extra divider or hint line) | ✅ |
|
|
319
|
-
| Keyboard: `↑` `↓` move-and-preview, `Enter` toggle, `Esc` collapse (`←` `→` span rows in the **media grid only** — the document list is always one column, so they have no spatial meaning there) | ✅ |
|
|
320
|
-
| Auto-refresh every 5s plus a manual button; docs changed in the last 2 min get 🆕 | ✅ |
|
|
321
|
-
| Bilingual (zh/en), follows the DSH language live — no plugin reload needed | ✅ |
|
|
322
|
-
| Zero model calls, zero network egress | ✅ |
|
|
323
|
-
| **A `knit_docs` tool for the agent** — read-only, so the model can find this project's relevant docs itself | ✅ |
|
|
324
|
-
|
|
325
|
-
> Relevance is deliberately **not visualised** (no percentages, no bars) — the ranking itself is
|
|
326
|
-
> the answer; position is relevance.
|
|
327
|
-
|
|
328
|
-
### The other two views
|
|
329
|
-
|
|
330
|
-

|
|
331
|
-
|
|
332
|
-
*Media: square thumbnails, with the **column count following the panel width continuously** —
|
|
333
|
-
six columns at this width, cells about 112px. These are the workspace's real image and SVG files; this
|
|
334
|
-
particular workspace happens to hold several screenshots, one demo GIF and one single-colour SVG icon
|
|
335
|
-
(the solid black square is that icon, not a failed load). Videos get their first frame as a poster with
|
|
336
|
-
a play glyph and a duration badge — this workspace simply has no video, so none shows here.*
|
|
337
|
-
|
|
338
|
-

|
|
339
|
-
|
|
340
|
-
*All: the documents section shows at most 4, with a "View all →" link when truncated (here "4 / 39"),
|
|
341
|
-
and the media grid below it.*
|
|
342
|
-
|
|
343
|
-
⚠️ Both sections share the **same 40-item window** the host returns — when media rank past 40th by
|
|
344
|
-
relevance, the All view keeps only the few inside that window (this screen keeps just one; the Media
|
|
345
|
-
tab is unaffected, because it requests media on its own). A known defect, with reproduction and the
|
|
346
|
-
fix recorded in [`docs/README.md`](docs/README.md).*
|
|
347
|
-
|
|
348
|
-
---
|
|
349
|
-
|
|
350
350
|
## Did it actually get used? (v0.16 → v0.17)
|
|
351
351
|
|
|
352
352
|
Knit orders "what to read first right now" — but **whether that order was right could only be felt,
|
|
@@ -509,10 +509,100 @@ The `knit_docs` tool output **did not grow**: it still only finds context.
|
|
|
509
509
|
|
|
510
510
|
---
|
|
511
511
|
|
|
512
|
+
## How retrieval works (implementation, not the pitch)
|
|
513
|
+
|
|
514
|
+
> This section is **evidence**, not the sales pitch: it is what backs the claim "ranked by the
|
|
515
|
+
> current task". The product question is answered on the first screen; this answers "why is it
|
|
516
|
+
> any good at ranking".
|
|
517
|
+
|
|
518
|
+
No magic, just string operations. Three steps:
|
|
519
|
+
|
|
520
|
+
**1. Read the conversation.** The host takes the last 6 user/assistant messages, and only
|
|
521
|
+
counts real user messages (`agent.inject()` synthetic context would drag the topic off course).
|
|
522
|
+
Newer messages weigh more: 3 / 2 / 1 / 1 …
|
|
523
|
+
|
|
524
|
+
**2. Extract keywords.**
|
|
525
|
+
|
|
526
|
+
- **ASCII words** — high value, one occurrence is enough (`chokidar`, `mtime`)
|
|
527
|
+
- **Chinese 2/3-grams** — either occurring twice, or appearing in the newest message
|
|
528
|
+
- **Drop fragments that straddle a word boundary.** Chinese has no word boundaries, so
|
|
529
|
+
n-grams glue the last character of one word to the first of the next (「图片和」, 「个插」,
|
|
530
|
+
「的排」). They share one trait — **the first or last character is a pure function word** —
|
|
531
|
+
and are dropped. Leave them in and they fill every candidate slot, pushing the real words
|
|
532
|
+
(「图片」, 「排序」) out of the query entirely
|
|
533
|
+
- stop-word filtering plus greedy de-overlap (picking「相关性排序」drops「相关性」and「排序」)
|
|
534
|
+
|
|
535
|
+
**3. Score the documents — BM25.**
|
|
536
|
+
|
|
537
|
+
```
|
|
538
|
+
IDF per term first: rarer in the corpus means more valuable
|
|
539
|
+
ln(1 + (N - df + 0.5) / (df + 0.5))
|
|
540
|
+
then a weighted sum over fields: title ×4 + summary ×2 + first 2500 chars of body ×1
|
|
541
|
+
each field saturated and length-normalised (k1 = 1.2, b = 0.3 / 0.5 / 0.75)
|
|
542
|
+
plus a 10% recency nudge (relevance still dominates)
|
|
543
|
+
```
|
|
544
|
+
|
|
545
|
+
**Why BM25 and not "hits × weight"** (the original approach, since replaced):
|
|
546
|
+
|
|
547
|
+
- with no IDF, a term that appears **everywhere** (the project name) is worth as much as a
|
|
548
|
+
rare one — so the frequent term discriminates nothing and dilutes the rare ones
|
|
549
|
+
- with no length normalisation, **a long document wins by piling up hits**
|
|
550
|
+
- capping hits at 6 was a hand-drawn knee; `k1` / `b` exist precisely for this
|
|
551
|
+
|
|
552
|
+
Measured on `test/eval/fixture.mjs` (21 cases, both engines on the same corpus):
|
|
553
|
+
|
|
554
|
+
| | top-1 | MRR |
|
|
555
|
+
|---|---|---|
|
|
556
|
+
| old (weighted hits) | 76.2% | 0.830 |
|
|
557
|
+
| **BM25** | **95.2%** | **0.976** |
|
|
558
|
+
|
|
559
|
+
That eval runs inside `npm test`, and the baseline is **recomputed each run** from the old
|
|
560
|
+
engine frozen in `test/eval/legacy.mjs` — so "the new engine must be clearly better" is
|
|
561
|
+
verified automatically rather than asserted against a hard-coded number.
|
|
562
|
+
|
|
563
|
+
**Against counting keywords yourself** (`knit/tools/scale-benchmark.mjs`, N = 20/60/180/540):
|
|
564
|
+
the corpus is built with a real trap — 12 short, focused topic documents, plus a pile of long
|
|
565
|
+
distractors that mention every topic five times without explaining any of them (which is what a
|
|
566
|
+
real project's CHANGELOG looks like). Half the topic documents have descriptive filenames, half
|
|
567
|
+
are opaque.
|
|
568
|
+
|
|
569
|
+
| Approach | Descriptive filenames | Opaque filenames | MRR vs. scale |
|
|
570
|
+
|---|---|---|---|
|
|
571
|
+
| **Knit (BM25)** | **100%** | **100%** | **1.000 (flat)** |
|
|
572
|
+
| `grep -c` keyword counting | 17% | **0%** | 0.313 → **0.089** |
|
|
573
|
+
| Filenames only | 100% | **0%** | 0.602 |
|
|
574
|
+
|
|
575
|
+
Three things: **the ranking beats counting keywords yourself** (so having the agent recompute it
|
|
576
|
+
is irrational); **filename matching only works when names describe content**, and Knit is the
|
|
577
|
+
only approach that scores 100% either way; and **self-counting degrades as the corpus grows**.
|
|
578
|
+
|
|
579
|
+
**About the "sorted by「xxx」" line**: it shows the **span of your own text** that the matched
|
|
580
|
+
terms cover, not the raw candidate tokens. Chinese has no word boundaries, so candidates always
|
|
581
|
+
include fragments that straddle two words (「项目文档」 yields `项目文` / `目文档`), and showing
|
|
582
|
+
those renders as gibberish — merging their spans and slicing the original text recovers `项目文档`.
|
|
583
|
+
The label is **your own wording**, so its casing is preserved (type `BM25`, see `BM25`).
|
|
584
|
+
|
|
585
|
+
All of it is string arithmetic — **no embeddings, no model calls**.
|
|
586
|
+
|
|
587
|
+
**And the honest boundaries**:
|
|
588
|
+
|
|
589
|
+
- **With only one or two messages** there are too few keywords, so it falls back to sorting by
|
|
590
|
+
modification time and says so in the panel — it does not pretend to rank
|
|
591
|
+
- **With only three to five documents IDF barely does anything**: `df` only takes a few values,
|
|
592
|
+
so its dynamic range collapses. The more documents, the better this ranking gets
|
|
593
|
+
- **It can only rank documents that share vocabulary with the conversation**: if no term matches,
|
|
594
|
+
every document scores the same and the order degrades to time
|
|
595
|
+
|
|
596
|
+
---
|
|
597
|
+
|
|
512
598
|
## What it reads, and what it doesn't
|
|
513
599
|
|
|
514
|
-
- Scans
|
|
515
|
-
outside is rejected); for media it reads metadata only,
|
|
600
|
+
- Scans Markdown, **supported code files** (see the v0.19 list above) and media **inside the current
|
|
601
|
+
session workspace** (anything resolving outside is rejected); for media it reads metadata only,
|
|
602
|
+
never the pixels, and for code it reads a **bounded head** of the body, never the whole file
|
|
603
|
+
- **Generated / noisy files are excluded from context** (`.map`, `*.min.*`, `*.bundle.*`,
|
|
604
|
+
`*.generated.*`, `*.lock`): they classify as `generated`, so they never enter retrieval, the
|
|
605
|
+
Context Pack or the Usage Lens — but `/knit/api/doc` still serves them when you open one on purpose
|
|
516
606
|
- Reads only **the current session's** conversation events (used for ranking)
|
|
517
607
|
- The `knit_docs` tool is **read-only**: it writes no files and persists no index
|
|
518
608
|
- v0.15 usage accounting is **off by default**: when enabled it reads only the **current session's
|
|
@@ -521,9 +611,10 @@ The `knit_docs` tool output **did not grow**: it still only finds context.
|
|
|
521
611
|
the plugin's own same-origin routes
|
|
522
612
|
- **No install-time scripts** (no `install` / `postinstall`)
|
|
523
613
|
- **Zero dependencies** — nothing to build, no build-authorisation prompt
|
|
524
|
-
- The
|
|
614
|
+
- The media route only allows an **image/video-extension allowlist** (images ≤ 12MB,
|
|
525
615
|
video ≤ 256MB), serves video over HTTP Range, and responds with
|
|
526
|
-
`nosniff` plus `default-src 'none'; sandbox
|
|
616
|
+
`nosniff` plus `default-src 'none'; sandbox`; the **text route serves documents, code and
|
|
617
|
+
generated artifacts**, and refuses media and unknown extensions
|
|
527
618
|
|
|
528
619
|
The relevance figure only affects ordering — it is **never displayed and never sent anywhere**.
|
|
529
620
|
|
|
@@ -558,13 +649,32 @@ The relevance figure only affects ordering — it is **never displayed and never
|
|
|
558
649
|
|
|
559
650
|
---
|
|
560
651
|
|
|
652
|
+
## Feedback
|
|
653
|
+
|
|
654
|
+
**Right now this project is not about adding features — it's about finding out whether
|
|
655
|
+
"rank project docs by the current conversation" is something anyone actually uses.**
|
|
656
|
+
So the most valuable thing you can send is not "could you add X" but **how you work around
|
|
657
|
+
this today** — including "I installed it and never opened it", which is more useful than a
|
|
658
|
+
feature request.
|
|
659
|
+
|
|
660
|
+
- 💬 [Tell us when you actually open it](https://github.com/PolinniZhong/dsh-knit/issues/new?template=feature.yml) — two fields, thirty seconds
|
|
661
|
+
- 🐞 [Won't install / panel won't open / ranking looks wrong](https://github.com/PolinniZhong/dsh-knit/issues/new?template=bug.yml)
|
|
662
|
+
- 📖 Check [known limitations](#known-limitations) first — short conversations degrade to time order and Chinese uses n-gram approximation; those are deliberate trade-offs, not bugs
|
|
663
|
+
|
|
664
|
+
> A single issue is treated as a real signal. So far this plugin has **no trace of a single
|
|
665
|
+
> real user** — the npm download count is automated version enumeration, not people
|
|
666
|
+
> (only 14% of it is `latest`, and a human install only ever pulls `latest`). One human reply
|
|
667
|
+
> changes what gets built next.
|
|
668
|
+
|
|
669
|
+
---
|
|
670
|
+
|
|
561
671
|
## Development
|
|
562
672
|
|
|
563
673
|
```sh
|
|
564
674
|
git clone https://github.com/PolinniZhong/dsh-knit.git
|
|
565
675
|
cd dsh-knit
|
|
566
676
|
|
|
567
|
-
npm test #
|
|
677
|
+
npm test # 588 tests, zero dependencies, no npm install needed
|
|
568
678
|
```
|
|
569
679
|
|
|
570
680
|
**How changes take effect**: the host half (`src/host/`) **requires a DSH restart** (no hot reload);
|
|
@@ -584,11 +694,12 @@ knit/
|
|
|
584
694
|
│ ├── host/index.js # /knit/api/recent · /doc · /raw · /links · /context
|
|
585
695
|
│ ├── host/relevance.js # the relevance engine: BM25 + keyword extraction
|
|
586
696
|
│ ├── host/links.js # reference parsing (pure; never touches ranking)
|
|
697
|
+
│ ├── host/classification.js # v0.19 file classification: kind / language / searchable / contextual
|
|
587
698
|
│ ├── host/context.js # context assembly: the three-tier Context Pack (pure, zero I/O)
|
|
588
699
|
│ ├── host/feedback.js # usage feedback: read-time attribution / Context Epoch / delta (pure)
|
|
589
700
|
│ ├── host/tool.js # the agent tool knit_docs (hand-written ToolDefinition)
|
|
590
701
|
│ └── client/client.js # dual-host registration + panel UI
|
|
591
|
-
└── test/ #
|
|
702
|
+
└── test/ # 588 tests
|
|
592
703
|
├── eval/ # retrieval quality: corpus + cases + frozen v0.5.2 baseline
|
|
593
704
|
└── context/ # context tiering: 24-doc corpus + 12 real-task cases
|
|
594
705
|
```
|