dsh-knit 0.19.0 → 0.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +285 -0
- package/README.en.md +298 -217
- package/README.md +184 -113
- package/SECURITY.md +2 -2
- package/package.json +7 -3
- package/src/client/client.js +742 -63
- package/src/host/context.js +29 -1
- package/src/host/index.js +124 -13
- package/src/host/passage.js +408 -0
- package/src/host/tool.js +104 -23
- package/test/client.test.mjs +793 -12
- package/test/context/fixture.mjs +2 -5
- package/test/deep-context.test.mjs +447 -0
- package/test/deep-eval.test.mjs +90 -0
- package/test/deep-scale.test.mjs +64 -0
- package/test/eval/deep.mjs +322 -0
- package/test/eval/fixture.mjs +6 -9
- package/test/fixture.mjs +21 -1
- package/test/harness.mjs +63 -4
- package/test/host.test.mjs +2 -2
- package/test/icon.test.mjs +11 -8
- package/test/tool.test.mjs +247 -18
package/README.en.md
CHANGED
|
@@ -1,33 +1,67 @@
|
|
|
1
1
|
# Knit
|
|
2
2
|
|
|
3
|
-
>
|
|
4
|
-
> Knit puts them next to the conversation — whatever you're talking about, the relevant doc is on top.
|
|
3
|
+
> **Task-aware workspace context retrieval and lifecycle tracking for AI coding agents.**
|
|
5
4
|
>
|
|
6
|
-
>
|
|
5
|
+
> **面向 AI Coding Agent 的任务感知工作区上下文检索与生命周期追踪。**
|
|
7
6
|
|
|
8
|
-
|
|
7
|
+
Knit is the layer between **the task you are on** and **your whole workspace**: it finds the most
|
|
8
|
+
relevant documents, source files and media, organises them into **Primary / Supporting / Related**
|
|
9
|
+
context (the Context Pack) for both the panel and the agent, and keeps tracking **whether they were
|
|
10
|
+
read, and whether they changed after being read**.
|
|
11
|
+
|
|
12
|
+
**Find → Organise → Track**, all of it plain string work on your own machine — no model calls, no
|
|
13
|
+
network, no latency, nothing leaves the machine.
|
|
14
|
+
|
|
15
|
+
```text
|
|
16
|
+
Current task
|
|
17
|
+
│
|
|
18
|
+
Workspace retrieval
|
|
19
|
+
┌───────────────┼───────────────┐
|
|
20
|
+
▼ ▼ ▼
|
|
21
|
+
Primary Supporting Related
|
|
22
|
+
└───────────────┼───────────────┘
|
|
23
|
+
▼
|
|
24
|
+
Context Pack ──► the `knit_docs` tool (for the agent)
|
|
25
|
+
▼
|
|
26
|
+
Agent reads
|
|
27
|
+
▼
|
|
28
|
+
Read Evidence · Context Epoch
|
|
29
|
+
▼
|
|
30
|
+
Workspace changes
|
|
31
|
+
▼
|
|
32
|
+
Re-read · Lifecycle
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
> **It is not a smarter recent-files list, and it is not a code browser.**
|
|
36
|
+
> A recent-files list records what you happened to open and what is newest; Knit answers
|
|
37
|
+
> **which things in this project are worth looking at first for the task at hand — and what
|
|
38
|
+
> happened to them after they were read.**
|
|
39
|
+
|
|
40
|
+

|
|
9
41
|
|
|
10
42
|
*Screenshot from a real machine, not a mock-up: the same workspace, this one left on the **Code tab** —
|
|
11
43
|
since v0.19 source files are first-class citizens, just like Markdown.*
|
|
12
|
-
*The
|
|
13
|
-
|
|
14
|
-
*Click a row to expand the preview **in place
|
|
15
|
-
numbers
|
|
44
|
+
*The sub-row under each row says how it matched (a filename hit, a path hit, …) and **which passage**
|
|
45
|
+
of that file matched this query (`命中段 24–38`).*
|
|
46
|
+
*Click a row to expand the preview **in place**, stopped right on that passage: here
|
|
47
|
+
`knit/src/host/passage.js`, line numbers plus syntax highlighting, with the matched lines under a grey
|
|
48
|
+
band; the preview header holds only the path and four actions (copy / full screen / open locally / close).*
|
|
16
49
|
*The header holds only the path, one count and two toggles; whether the list runs by
|
|
17
50
|
relevance or by newest-first is decided by that pair of tabs on the left.*
|
|
18
51
|
*Since v0.14 the list is always one column: the rank number sits in its own column on the far left
|
|
19
52
|
(vertically centred on the title's first line), then Primary dot + title + relative time (time on the
|
|
20
53
|
right), then the summary. **The path lives in the preview header only** — the directory collapses to a `…/` placeholder (hover the path for the full relative path); rows no longer repeat it.*
|
|
21
54
|
|
|
22
|
-

|
|
23
56
|
|
|
24
|
-
*Order follows the conversation — same workspace, in a
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
*⚠️ The panel polls **every 5 seconds**, so the re-rank
|
|
30
|
-
|
|
57
|
+
*Order follows the conversation — same workspace, in a separate session: the conversation had been
|
|
58
|
+
about a release checklist, so the top of the list was that family of documents; ask
|
|
59
|
+
"how is BM25 weighted for the ranking?" and the list swaps over to the BM25 family, keeping the tier
|
|
60
|
+
headings and the **continuous rank numbers across all tiers**, with a one-line "why" under each row
|
|
61
|
+
(what matched, how, and which passage).*
|
|
62
|
+
*⚠️ The panel polls **every 5 seconds**, so the re-rank is not instantaneous: in this take the top rows
|
|
63
|
+
only swapped once that turn had moved along (the tens of seconds in between were cut out).
|
|
64
|
+
The GIF runs faster than real time.*
|
|
31
65
|
|
|
32
66
|
Scans Markdown / images / video across the project folder · Order follows the conversation · No model calls, no network
|
|
33
67
|
|
|
@@ -37,104 +71,129 @@ dsh plugin --profile web add dsh-knit
|
|
|
37
71
|
|
|
38
72
|
---
|
|
39
73
|
|
|
40
|
-
##
|
|
74
|
+
## Why it exists
|
|
41
75
|
|
|
42
|
-
|
|
76
|
+
> An agent produces 20 documents a day, and you cannot find the one from a minute ago.
|
|
43
77
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
78
|
+
That is the problem it started from: **output got faster than anyone can browse directories.**
|
|
79
|
+
DSH's built-in "recent files" only records what you opened; once a turn ends, the earlier ones sink
|
|
80
|
+
out of sight.
|
|
47
81
|
|
|
48
|
-
|
|
82
|
+
Knit asks a different question: **not "what is newest", but "what is relevant to what I am doing".**
|
|
83
|
+
It scans the **whole workspace** (not just what this turn produced) and ranks by what you are
|
|
84
|
+
talking about — restart DSH, open a new session, come back the next day, and it is still there.
|
|
49
85
|
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
86
|
+
Then the problem grew one step further: when you actually change a feature, the first thing worth
|
|
87
|
+
reading is usually not a document but **a few lines of source**. That is what v0.19 added. And after
|
|
88
|
+
that, "ranked first" stopped being enough — **of the things it ranked, did the agent actually read
|
|
89
|
+
them? Did they change afterwards?** v0.15–v0.18 filled in that layer.
|
|
90
|
+
|
|
91
|
+
Three words: **Find** (scan the workspace, rank by the current task) → **Organise** (three tiers,
|
|
92
|
+
each with its reason) → **Track** (read or not, changed or not).
|
|
56
93
|
|
|
57
94
|
---
|
|
58
95
|
|
|
59
|
-
##
|
|
96
|
+
## It is not a recent-files list, and not a code browser
|
|
60
97
|
|
|
61
|
-
|
|
98
|
+
Both comparisons catch one **surface** of Knit and miss the question it answers.
|
|
62
99
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
100
|
+
| | Recent-files list | Code browser | **Knit** |
|
|
101
|
+
|---|---|---|---|
|
|
102
|
+
| Scope | What you happened to open | A directory tree | **The whole workspace**: documents / code / images / video |
|
|
103
|
+
| Order | Modified time | You scroll it yourself | **Relevance to the current task** (BM25 + IDF, computed locally) |
|
|
104
|
+
| Output | One flat list | The one file you opened | **Primary / Supporting / Related**, each saying why it is there |
|
|
105
|
+
| For whom | You | You | You **and** the agent — the same Context Pack, through `knit_docs` |
|
|
106
|
+
| Afterwards | Nothing | Nothing | **Was it read? Did it change after being read?** (Read Evidence + lifecycle) |
|
|
66
107
|
|
|
67
|
-
|
|
108
|
+
The three differences that matter:
|
|
68
109
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
(「图片」, 「排序」) out of the query entirely
|
|
76
|
-
- stop-word filtering plus greedy de-overlap (picking「相关性排序」drops「相关性」and「排序」)
|
|
110
|
+
1. **Scope**: a recent-files list only records the files you **opened**; Knit scans **the whole project
|
|
111
|
+
folder** — restart DSH, open a new session, come back days later, and it is still there.
|
|
112
|
+
2. **Order**: it sorts by time; Knit sorts by **what you are talking about** — discuss architecture and
|
|
113
|
+
the architecture docs rise; discuss a loader bug and `plugin-loader.ts` rises.
|
|
114
|
+
3. **It does not stop at finding**: after finding comes **organising** (three tiers, each with a
|
|
115
|
+
verifiable reason) and **tracking** (which were really read, and whether they changed).
|
|
77
116
|
|
|
78
|
-
**
|
|
117
|
+
**No model calls, no network**: all plain string work, no latency, no cost, nothing leaves the machine.
|
|
79
118
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## Install
|
|
122
|
+
|
|
123
|
+
```sh
|
|
124
|
+
dsh plugin --profile web add dsh-knit
|
|
86
125
|
```
|
|
87
126
|
|
|
88
|
-
**
|
|
127
|
+
Then **restart DSH** and hard-refresh the browser (`Cmd + Shift + R`).
|
|
89
128
|
|
|
90
|
-
|
|
91
|
-
rare one — so the frequent term discriminates nothing and dilutes the rare ones
|
|
92
|
-
- with no length normalisation, **a long document wins by piling up hits**
|
|
93
|
-
- capping hits at 6 was a hand-drawn knee; `k1` / `b` exist precisely for this
|
|
129
|
+
**How to open it:**
|
|
94
130
|
|
|
95
|
-
|
|
131
|
+
- The **Knit icon button in the session header** (right next to the sidebar toggle) — one click
|
|
132
|
+
- Or the right sidebar's tab bar「+」→「Knit recent docs」
|
|
96
133
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
| old (weighted hits) | 76.2% | 0.830 |
|
|
100
|
-
| **BM25** | **95.2%** | **0.976** |
|
|
134
|
+
**Optional**: if `dsh-better-sidebar` is installed, the panel also registers
|
|
135
|
+
as one of its tabs. Each host is an independent optional dependency; missing one doesn't affect the other.
|
|
101
136
|
|
|
102
|
-
|
|
103
|
-
engine frozen in `test/eval/legacy.mjs` — so "the new engine must be clearly better" is
|
|
104
|
-
verified automatically rather than asserted against a hard-coded number.
|
|
137
|
+
**Upgrading**: same command as installing, and the same restart + hard-refresh afterwards.
|
|
105
138
|
|
|
106
|
-
|
|
107
|
-
the corpus is built with a real trap — 12 short, focused topic documents, plus a pile of long
|
|
108
|
-
distractors that mention every topic five times without explaining any of them (which is what a
|
|
109
|
-
real project's CHANGELOG looks like). Half the topic documents have descriptive filenames, half
|
|
110
|
-
are opaque.
|
|
139
|
+
---
|
|
111
140
|
|
|
112
|
-
|
|
113
|
-
|---|---|---|---|
|
|
114
|
-
| **Knit (BM25)** | **100%** | **100%** | **1.000 (flat)** |
|
|
115
|
-
| `grep -c` keyword counting | 17% | **0%** | 0.313 → **0.089** |
|
|
116
|
-
| Filenames only | 100% | **0%** | 0.602 |
|
|
141
|
+
## Features
|
|
117
142
|
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
143
|
+
| | |
|
|
144
|
+
|---|---|
|
|
145
|
+
| **Code Context** (v0.19): source files join the **same** retrieval pipeline as documents — classify, BM25, Context Pack, lifecycle. A dedicated classification layer decides what a file *is* and what it may do, so extension knowledge lives in exactly one place | ✅ |
|
|
146
|
+
| **Sorted by relevance to the current conversation** (BM25 + IDF, fully local, no model) | ✅ |
|
|
147
|
+
| **The `knit_docs` tool for the agent**: the model can look up this project's most relevant documents itself | ✅ |
|
|
148
|
+
| **Document lifecycle** (v0.17): each recommended document is shown as `Unread / Read / Updated after read / Re-read after update`, alongside the most recent read, how many reads fell outside the pack (expandable to which ones), and the latest context change | ✅ |
|
|
149
|
+
| One-click toggle between relevance / modification time (preference kept in localStorage) | ✅ |
|
|
150
|
+
| Scans **documents and code** in the session workspace (recursive, depth ≤ 6, skips `node_modules` / `.git` / `dist` / `build` / `out` / `coverage`) | ✅ |
|
|
151
|
+
| Each row shows H1 title (or filename) + relative time + first-paragraph summary | ✅ |
|
|
152
|
+
| The document list is **always one column** (v0.14 — the multi-column layout was deleted outright, not switched off); the **number sits in its own column on the far left** and is **vertically centred with the title line**, everything else stacks to its right starting with **one line: Primary dot + title + relative time** (the time is pushed to the right edge), then the summary. **The path is no longer shown in the list** — it duplicates the clickable **path** in the preview header, which is the one that stays | ✅ |
|
|
153
|
+
| Click to preview inline, click again to collapse | ✅ |
|
|
154
|
+
| Relative-path images actually render (`./img/a.png`, `../assets/b.png`) | ✅ |
|
|
155
|
+
| One-click switch between **Docs / Code / Media / All** (v0.19 adds Code; remembered, defaults to Docs); the selected tab is an **underline tab**, deliberately different from the list row's grey fill | ✅ |
|
|
156
|
+
| Images & video: square thumbnail grid — cells have a **fixed 104px baseline independent of the item count**, columns come from CSS `repeat(auto-fill, minmax(104px, 1fr))`, and **as many rows as there are items are laid out** (no max height, the grid never scrolls itself — scrolling belongs to the list); videos auto-grab the first frame with a play glyph and duration badge (no deps, no transcoding) | ✅ |
|
|
157
|
+
| **"Web preview" for `.html`** (v0.19): the preview header hands the file to DSH's own document-preview tab (`openResource`) instead of showing source; the inline preview still shows source, so the two do not duplicate. Knit itself sends no request | ✅ |
|
|
158
|
+
| Click an image / video to preview **inline**: large image, playable & seekable video streamed over HTTP Range (no full download) | ✅ |
|
|
159
|
+
| The **All** view splits into **three stacked sections**: docs (max 4, with "View all →" when truncated), **code** (max 4), then images & video (**never truncated**, count only) | ✅ |
|
|
160
|
+
| Preview pane is height-draggable (20%–80%, remembered), fullscreen-able, `Esc` to exit | ✅ |
|
|
161
|
+
| **"Open locally"** opens the document in your default app; the **path in the preview header is clickable** and now opens the system file manager with the file **selected** (`action:'reveal'`), and its tooltip carries the full relative path | ✅ |
|
|
162
|
+
| Double-click opens in a new tab (the official document preview, with its PDF renderer and renderer switching) | ✅ |
|
|
163
|
+
| Filter box over title / summary / path | ✅ |
|
|
164
|
+
| **Click the workspace path** to open the project folder in your file manager | ✅ |
|
|
165
|
+
| **Hover the entry button to peek**: a read-only floating list of the 5 most recent docs; click to open the right sidebar (doesn't push the layout; the popover is just header + list — no extra divider or hint line) | ✅ |
|
|
166
|
+
| Keyboard: `↑` `↓` move-and-preview, `Enter` toggle, `Esc` collapse (`←` `→` span rows in the **media grid only** — the document list is always one column, so they have no spatial meaning there) | ✅ |
|
|
167
|
+
| Auto-refresh every 5s plus a manual button; docs changed in the last 2 min get 🆕 | ✅ |
|
|
168
|
+
| Bilingual (zh/en), follows the DSH language live — no plugin reload needed | ✅ |
|
|
169
|
+
| Zero model calls, zero network egress | ✅ |
|
|
170
|
+
| **A `knit_docs` tool for the agent** — read-only, so the model can find this project's relevant docs itself | ✅ |
|
|
121
171
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
include fragments that straddle two words (「项目文档」 yields `项目文` / `目文档`), and showing
|
|
125
|
-
those renders as gibberish — merging their spans and slicing the original text recovers `项目文档`.
|
|
126
|
-
The label is **your own wording**, so its casing is preserved (type `BM25`, see `BM25`).
|
|
172
|
+
> Relevance is deliberately **not visualised** (no percentages, no bars) — the ranking itself is
|
|
173
|
+
> the answer; position is relevance.
|
|
127
174
|
|
|
128
|
-
|
|
175
|
+
### The other two views
|
|
129
176
|
|
|
130
|
-
|
|
177
|
+

|
|
131
178
|
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
179
|
+
*Media: square thumbnails, with the **column count following the panel width continuously** —
|
|
180
|
+
six columns at this width, cells about 110px. These are the workspace's real image and SVG files
|
|
181
|
+
(18 of them in this take); this particular workspace holds several screenshots, a demo GIF and one
|
|
182
|
+
single-colour SVG icon (the solid black square is that icon, not a failed load). Videos get their first
|
|
183
|
+
frame as a poster with a play glyph and a duration badge — this workspace simply has no video, so none
|
|
184
|
+
shows here.*
|
|
185
|
+
|
|
186
|
+

|
|
187
|
+
|
|
188
|
+
*All: the documents section shows at most 4, the **code section** shows at most 4 (each gets its own
|
|
189
|
+
"View all →" when truncated — here "4 / 21" and "4 / 8"), and the media grid (11 rows) comes last.*
|
|
190
|
+
|
|
191
|
+
⚠️ All three sections share the **same 40-item window** the host returns — a category that ranks past
|
|
192
|
+
40th by relevance drops out of the All view entirely. This screenshot uses the **Newest** sort: in this
|
|
193
|
+
workspace, under the default Relevance sort, media rank outside that window every time (at `limit=40`
|
|
194
|
+
those 40 rows are documents + code files). The **Media** and **Code** tabs are unaffected — they
|
|
195
|
+
request their own kind. A known defect, with reproduction and the fix recorded in
|
|
196
|
+
[`docs/README.md`](docs/README.md).*
|
|
138
197
|
|
|
139
198
|
---
|
|
140
199
|
|
|
@@ -219,6 +278,48 @@ There is **no** "the AI thinks this matters" and **no** percentages, stars or co
|
|
|
219
278
|
|
|
220
279
|
---
|
|
221
280
|
|
|
281
|
+
## Code as context (v0.19)
|
|
282
|
+
|
|
283
|
+
Until v0.18 Knit understood Markdown, images and video. v0.19's product concept becomes
|
|
284
|
+
**documents / code / media** — not "a few more extensions", but letting Knit put the source files an
|
|
285
|
+
AI-coding task actually needs into the same Context Pack, on the same main line:
|
|
286
|
+
|
|
287
|
+
```text
|
|
288
|
+
Current task -> Workspace Retrieval -> Document + Code Candidates
|
|
289
|
+
-> Primary / Supporting / Related -> Context Pack -> Agent Read
|
|
290
|
+
-> Read Evidence -> File Changed -> Lifecycle -> Context Epoch / Delta -> Usage Lens
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
- **One classification layer, not scattered `if (ext === '.js')`** — `src/host/classification.js`
|
|
294
|
+
turns a filename into `{ kind, language, previewable, searchable, contextual, generated,
|
|
295
|
+
sourceMap }`. The scanner, retrieval, preview and UI all just read that verdict.
|
|
296
|
+
- **Supported in v0.19**: `.js .mjs .cjs .ts .tsx .jsx .py .json .html .htm .css .scss .yaml .yml
|
|
297
|
+
.sh .bash .zsh`. **Not yet**: `.go .rs .java .kt .c .cpp .h .hpp .cs .php .rb .swift .sql`.
|
|
298
|
+
- **Code gets its own field weights** — filename x4, path x2, body (first 8 KB) x1, instead of the
|
|
299
|
+
document weights (title x4 / summary x2 / body x1). A filename like `context-manager.ts` carries
|
|
300
|
+
the task signal.
|
|
301
|
+
- **Generated files are not context**: `.map`, `*.min.js`, `*.min.css`, `*.bundle.js`,
|
|
302
|
+
`*.generated.js`, `*.gen.js`, `*.lock` classify as `generated` — previewable, but never in
|
|
303
|
+
retrieval, the Context Pack or the Usage Lens. When `app.js` and `app.js.map` both exist, the
|
|
304
|
+
`app.js` preview offers one low-key **"Source map"** entry; the `.map` never shows up as a row.
|
|
305
|
+
**Context eligibility is not file-system visibility** — the built-in file tree still opens them.
|
|
306
|
+
- **Bounded and capped**: at most `MAX_CODE = 300` code candidates and a head-limited body read, so
|
|
307
|
+
a large repo cannot flood the Markdown corpus. Corpus pressure is the thing to watch in this
|
|
308
|
+
version, not recall.
|
|
309
|
+
- **The type tabs become four**: `Docs / Code / Media / All`. The Code tab is **still driven by the
|
|
310
|
+
current task** — it is not a file tree of the project.
|
|
311
|
+
- **Syntax highlighting is DSH's own (v0.20)** — the client reuses
|
|
312
|
+
`useCodeHighlighter` from `@deepseek-ai/dsh-client-ui-primitives` and takes every colour from the
|
|
313
|
+
official `--shiki-*` tokens, so light and dark match DSH instead of inventing a palette. Only the
|
|
314
|
+
matched stretch is coloured (60 lines either side, window capped at 800) and the rest stays plain
|
|
315
|
+
text — a file with no match gets no colour at all (an un-capped 4k-line file
|
|
316
|
+
costs ~2 s); if the host does not export the hook (older DSH) or the line counts disagree, it falls
|
|
317
|
+
back to plain text silently.
|
|
318
|
+
- **No second engine**: no second retrieval path, no second lifecycle, no second store, no new
|
|
319
|
+
dependencies, no AST, no LSP, no embedding, no model call, no network, no IDE.
|
|
320
|
+
|
|
321
|
+
---
|
|
322
|
+
|
|
222
323
|
## Also for the agent: the `knit_docs` tool
|
|
223
324
|
|
|
224
325
|
The same ranking that you see in the panel is also exposed to the model.
|
|
@@ -228,7 +329,8 @@ project are most relevant to what we're discussing?"* and get back **the same Co
|
|
|
228
329
|
panel shows — the `primary` / `supporting` / `related` tiers, each item carrying its path, title,
|
|
229
330
|
summary, a **structured reason** (`direct` / `titleMatch` / `summaryMatch` / `bodyMatch` /
|
|
230
331
|
`linkTarget` / `linkSource` / `related`), its **project role** (`impl` / `test` / `config` /
|
|
231
|
-
`design` / `doc`) and the matching snippet
|
|
332
|
+
`design` / `doc`) and the matching snippet **with its line range** (`match: lines 147–163 — …`) —
|
|
333
|
+
then open one of them with its own `read` tool.
|
|
232
334
|
|
|
233
335
|
**Why it helps**: to cite a document that already exists, an agent can only guess paths or glob
|
|
234
336
|
and `read` them one by one. Knit has **already computed that ranking** every refresh; this tool
|
|
@@ -244,7 +346,12 @@ Three details:
|
|
|
244
346
|
- **No relevance score in the result.** It is a *relative* score (something is always 100%, and it
|
|
245
347
|
may be a different document on the next refresh); shown to a model it reads as absolute confidence.
|
|
246
348
|
**The order is the relevance** — the same rule the panel follows.
|
|
247
|
-
- **
|
|
349
|
+
- **The whole file is searched** (since v0.20). Every file is split into structural passages and each
|
|
350
|
+
passage is scored, so "the relevant part is past the first 2,500 characters" no longer means
|
|
351
|
+
"unfindable"; a file over 256 KB is indexed only up to that size, and media has no text — both
|
|
352
|
+
boundaries are stated in the output rather than hidden.
|
|
353
|
+
- **No document bodies.** Only the matched passage (one per file at most, ≤200 characters) — the agent
|
|
354
|
+
has its own `read` tool; Knit *finds*, it does not *carry*.
|
|
248
355
|
- **No workspace means an error, never a fallback.** The HTTP route falls back to the process cwd
|
|
249
356
|
when a session can't be resolved (for older clients that send no session id); the tool has no such
|
|
250
357
|
baggage — falling back would scan an unrelated project and return *its* documents.
|
|
@@ -256,103 +363,6 @@ Three details:
|
|
|
256
363
|
|
|
257
364
|
---
|
|
258
365
|
|
|
259
|
-
## Install
|
|
260
|
-
|
|
261
|
-
```sh
|
|
262
|
-
dsh plugin --profile web add dsh-knit
|
|
263
|
-
```
|
|
264
|
-
|
|
265
|
-
Then **restart DSH** and hard-refresh the browser (`Cmd + Shift + R`).
|
|
266
|
-
|
|
267
|
-
**How to open it:**
|
|
268
|
-
|
|
269
|
-
- The **Knit icon button in the session header** (right next to the sidebar toggle) — one click
|
|
270
|
-
- Or the right sidebar's tab bar「+」→「Knit recent docs」
|
|
271
|
-
|
|
272
|
-
**Optional**: if `dsh-better-sidebar` is installed, the panel also registers
|
|
273
|
-
as one of its tabs. Each host is an independent optional dependency; missing one doesn't affect the other.
|
|
274
|
-
|
|
275
|
-
**Upgrading**: same command as installing, and the same restart + hard-refresh afterwards.
|
|
276
|
-
|
|
277
|
-
---
|
|
278
|
-
|
|
279
|
-
## Feedback
|
|
280
|
-
|
|
281
|
-
**Right now this project is not about adding features — it's about finding out whether
|
|
282
|
-
"rank project docs by the current conversation" is something anyone actually uses.**
|
|
283
|
-
So the most valuable thing you can send is not "could you add X" but **how you work around
|
|
284
|
-
this today** — including "I installed it and never opened it", which is more useful than a
|
|
285
|
-
feature request.
|
|
286
|
-
|
|
287
|
-
- 💬 [Tell us when you actually open it](https://github.com/PolinniZhong/dsh-knit/issues/new?template=feature.yml) — two fields, thirty seconds
|
|
288
|
-
- 🐞 [Won't install / panel won't open / ranking looks wrong](https://github.com/PolinniZhong/dsh-knit/issues/new?template=bug.yml)
|
|
289
|
-
- 📖 Check [known limitations](#known-limitations) first — short conversations degrade to time order and Chinese uses n-gram approximation; those are deliberate trade-offs, not bugs
|
|
290
|
-
|
|
291
|
-
> A single issue is treated as a real signal. So far this plugin has **no trace of a single
|
|
292
|
-
> real user** — the npm download count is automated version enumeration, not people
|
|
293
|
-
> (only 14% of it is `latest`, and a human install only ever pulls `latest`). One human reply
|
|
294
|
-
> changes what gets built next.
|
|
295
|
-
|
|
296
|
-
---
|
|
297
|
-
|
|
298
|
-
## Features
|
|
299
|
-
|
|
300
|
-
| | |
|
|
301
|
-
|---|---|
|
|
302
|
-
| **Code Context** (v0.19): source files join the **same** retrieval pipeline as documents — classify, BM25, Context Pack, lifecycle. A dedicated classification layer decides what a file *is* and what it may do, so extension knowledge lives in exactly one place | ✅ |
|
|
303
|
-
| **Sorted by relevance to the current conversation** (BM25 + IDF, fully local, no model) | ✅ |
|
|
304
|
-
| **The `knit_docs` tool for the agent**: the model can look up this project's most relevant documents itself | ✅ |
|
|
305
|
-
| **Document lifecycle** (v0.17): each recommended document is shown as `Unread / Read / Updated after read / Re-read after update`, alongside the most recent read, how many reads fell outside the pack (expandable to which ones), and the latest context change | ✅ |
|
|
306
|
-
| One-click toggle between relevance / modification time (preference kept in localStorage) | ✅ |
|
|
307
|
-
| Scans **documents and code** in the session workspace (recursive, depth ≤ 6, skips `node_modules` / `.git` / `dist` / `build` / `out` / `coverage`) | ✅ |
|
|
308
|
-
| Each row shows H1 title (or filename) + relative time + first-paragraph summary | ✅ |
|
|
309
|
-
| The document list is **always one column** (v0.14 — the multi-column layout was deleted outright, not switched off); the **number sits in its own column on the far left** and is **vertically centred with the title line**, everything else stacks to its right starting with **one line: Primary dot + title + relative time** (the time is pushed to the right edge), then the summary. **The path is no longer shown in the list** — it duplicates the clickable **path** in the preview header, which is the one that stays | ✅ |
|
|
310
|
-
| Click to preview inline, click again to collapse | ✅ |
|
|
311
|
-
| Relative-path images actually render (`./img/a.png`, `../assets/b.png`) | ✅ |
|
|
312
|
-
| One-click switch between **Docs / Code / Media / All** (v0.19 adds Code; remembered, defaults to Docs); the selected tab is an **underline tab**, deliberately different from the list row's grey fill | ✅ |
|
|
313
|
-
| Images & video: square thumbnail grid — cells have a **fixed 104px baseline independent of the item count**, columns come from CSS `repeat(auto-fill, minmax(104px, 1fr))`, and **as many rows as there are items are laid out** (no max height, the grid never scrolls itself — scrolling belongs to the list); videos auto-grab the first frame with a play glyph and duration badge (no deps, no transcoding) | ✅ |
|
|
314
|
-
| **"Web preview" for `.html`** (v0.19): the preview header hands the file to DSH's own document-preview tab (`openResource`) instead of showing source; the inline preview still shows source, so the two do not duplicate. Knit itself sends no request | ✅ |
|
|
315
|
-
| Click an image / video to preview **inline**: large image, playable & seekable video streamed over HTTP Range (no full download) | ✅ |
|
|
316
|
-
| The **All** view splits into **three stacked sections**: docs (max 4, with "View all →" when truncated), **code** (max 4), then images & video (**never truncated**, count only) | ✅ |
|
|
317
|
-
| Preview pane is height-draggable (20%–80%, remembered), fullscreen-able, `Esc` to exit | ✅ |
|
|
318
|
-
| **"Open locally"** opens the document in your default app; the **path in the preview header is clickable** and now opens the system file manager with the file **selected** (`action:'reveal'`), and its tooltip carries the full relative path | ✅ |
|
|
319
|
-
| Double-click opens in a new tab (the official document preview, with its PDF renderer and renderer switching) | ✅ |
|
|
320
|
-
| Filter box over title / summary / path | ✅ |
|
|
321
|
-
| **Click the workspace path** to open the project folder in your file manager | ✅ |
|
|
322
|
-
| **Hover the entry button to peek**: a read-only floating list of the 5 most recent docs; click to open the right sidebar (doesn't push the layout; the popover is just header + list — no extra divider or hint line) | ✅ |
|
|
323
|
-
| Keyboard: `↑` `↓` move-and-preview, `Enter` toggle, `Esc` collapse (`←` `→` span rows in the **media grid only** — the document list is always one column, so they have no spatial meaning there) | ✅ |
|
|
324
|
-
| Auto-refresh every 5s plus a manual button; docs changed in the last 2 min get 🆕 | ✅ |
|
|
325
|
-
| Bilingual (zh/en), follows the DSH language live — no plugin reload needed | ✅ |
|
|
326
|
-
| Zero model calls, zero network egress | ✅ |
|
|
327
|
-
| **A `knit_docs` tool for the agent** — read-only, so the model can find this project's relevant docs itself | ✅ |
|
|
328
|
-
|
|
329
|
-
> Relevance is deliberately **not visualised** (no percentages, no bars) — the ranking itself is
|
|
330
|
-
> the answer; position is relevance.
|
|
331
|
-
|
|
332
|
-
### The other two views
|
|
333
|
-
|
|
334
|
-

|
|
335
|
-
|
|
336
|
-
*Media: square thumbnails, with the **column count following the panel width continuously** —
|
|
337
|
-
six columns at this width, cells about 110px. These are the workspace's real image and SVG files; this
|
|
338
|
-
particular workspace happens to hold several screenshots, one demo GIF and one single-colour SVG icon
|
|
339
|
-
(the solid black square is that icon, not a failed load). Videos get their first frame as a poster with
|
|
340
|
-
a play glyph and a duration badge — this workspace simply has no video, so none shows here.*
|
|
341
|
-
|
|
342
|
-

|
|
343
|
-
|
|
344
|
-
*All: the documents section shows at most 4, the **code section** shows at most 4 (each gets its own
|
|
345
|
-
"View all →" when truncated — here "4 / 19" and "4 / 17"), and the media grid comes last.*
|
|
346
|
-
|
|
347
|
-
⚠️ All three sections share the **same 40-item window** the host returns — a category that ranks past
|
|
348
|
-
40th by relevance drops out of the All view entirely. This screenshot uses the **Newest** sort: in this
|
|
349
|
-
workspace, under the default Relevance sort, media rank outside that window every time (at `limit=40`
|
|
350
|
-
those 40 rows are 34 documents + 6 code files). The **Media** and **Code** tabs are unaffected — they
|
|
351
|
-
request their own kind. A known defect, with reproduction and the fix recorded in
|
|
352
|
-
[`docs/README.md`](docs/README.md).*
|
|
353
|
-
|
|
354
|
-
---
|
|
355
|
-
|
|
356
366
|
## Did it actually get used? (v0.16 → v0.17)
|
|
357
367
|
|
|
358
368
|
Knit orders "what to read first right now" — but **whether that order was right could only be felt,
|
|
@@ -515,38 +525,89 @@ The `knit_docs` tool output **did not grow**: it still only finds context.
|
|
|
515
525
|
|
|
516
526
|
---
|
|
517
527
|
|
|
518
|
-
|
|
528
|
+
## How retrieval works (implementation, not the pitch)
|
|
519
529
|
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
530
|
+
> This section is **evidence**, not the sales pitch: it is what backs the claim "ranked by the
|
|
531
|
+
> current task". The product question is answered on the first screen; this answers "why is it
|
|
532
|
+
> any good at ranking".
|
|
523
533
|
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
534
|
+
No magic, just string operations. Three steps:
|
|
535
|
+
|
|
536
|
+
**1. Read the conversation.** The host takes the last 6 user/assistant messages, and only
|
|
537
|
+
counts real user messages (`agent.inject()` synthetic context would drag the topic off course).
|
|
538
|
+
Newer messages weigh more: 3 / 2 / 1 / 1 …
|
|
539
|
+
|
|
540
|
+
**2. Extract keywords.**
|
|
541
|
+
|
|
542
|
+
- **ASCII words** — high value, one occurrence is enough (`chokidar`, `mtime`)
|
|
543
|
+
- **Chinese 2/3-grams** — either occurring twice, or appearing in the newest message
|
|
544
|
+
- **Drop fragments that straddle a word boundary.** Chinese has no word boundaries, so
|
|
545
|
+
n-grams glue the last character of one word to the first of the next (「图片和」, 「个插」,
|
|
546
|
+
「的排」). They share one trait — **the first or last character is a pure function word** —
|
|
547
|
+
and are dropped. Leave them in and they fill every candidate slot, pushing the real words
|
|
548
|
+
(「图片」, 「排序」) out of the query entirely
|
|
549
|
+
- stop-word filtering plus greedy de-overlap (picking「相关性排序」drops「相关性」and「排序」)
|
|
550
|
+
|
|
551
|
+
**3. Score the documents — BM25.**
|
|
552
|
+
|
|
553
|
+
```
|
|
554
|
+
IDF per term first: rarer in the corpus means more valuable
|
|
555
|
+
ln(1 + (N - df + 0.5) / (df + 0.5))
|
|
556
|
+
then a weighted sum over fields: title ×4 + summary ×2 + the whole body ×1
|
|
557
|
+
each field saturated and length-normalised (k1 = 1.2, b = 0.3 / 0.5 / 0.75)
|
|
558
|
+
plus a 10% recency nudge (relevance still dominates)
|
|
528
559
|
```
|
|
529
560
|
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
-
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
561
|
+
**Why BM25 and not "hits × weight"** (the original approach, since replaced):
|
|
562
|
+
|
|
563
|
+
- with no IDF, a term that appears **everywhere** (the project name) is worth as much as a
|
|
564
|
+
rare one — so the frequent term discriminates nothing and dilutes the rare ones
|
|
565
|
+
- with no length normalisation, **a long document wins by piling up hits**
|
|
566
|
+
- capping hits at 6 was a hand-drawn knee; `k1` / `b` exist precisely for this
|
|
567
|
+
|
|
568
|
+
Measured on `test/eval/fixture.mjs` (21 cases, both engines on the same corpus):
|
|
569
|
+
|
|
570
|
+
| | top-1 | MRR |
|
|
571
|
+
|---|---|---|
|
|
572
|
+
| old (weighted hits) | 76.2% | 0.830 |
|
|
573
|
+
| **BM25** | **95.2%** | **0.976** |
|
|
574
|
+
|
|
575
|
+
That eval runs inside `npm test`, and the baseline is **recomputed each run** from the old
|
|
576
|
+
engine frozen in `test/eval/legacy.mjs` — so "the new engine must be clearly better" is
|
|
577
|
+
verified automatically rather than asserted against a hard-coded number.
|
|
578
|
+
|
|
579
|
+
**Against counting keywords yourself** (`knit/tools/scale-benchmark.mjs`, N = 20/60/180/540):
|
|
580
|
+
the corpus is built with a real trap — 12 short, focused topic documents, plus a pile of long
|
|
581
|
+
distractors that mention every topic five times without explaining any of them (which is what a
|
|
582
|
+
real project's CHANGELOG looks like). Half the topic documents have descriptive filenames, half
|
|
583
|
+
are opaque.
|
|
584
|
+
|
|
585
|
+
| Approach | Descriptive filenames | Opaque filenames | MRR vs. scale |
|
|
586
|
+
|---|---|---|---|
|
|
587
|
+
| **Knit (BM25)** | **100%** | **100%** | **1.000 (flat)** |
|
|
588
|
+
| `grep -c` keyword counting | 17% | **0%** | 0.313 → **0.089** |
|
|
589
|
+
| Filenames only | 100% | **0%** | 0.602 |
|
|
590
|
+
|
|
591
|
+
Three things: **the ranking beats counting keywords yourself** (so having the agent recompute it
|
|
592
|
+
is irrational); **filename matching only works when names describe content**, and Knit is the
|
|
593
|
+
only approach that scores 100% either way; and **self-counting degrades as the corpus grows**.
|
|
594
|
+
|
|
595
|
+
**About the "sorted by「xxx」" line**: it shows the **span of your own text** that the matched
|
|
596
|
+
terms cover, not the raw candidate tokens. Chinese has no word boundaries, so candidates always
|
|
597
|
+
include fragments that straddle two words (「项目文档」 yields `项目文` / `目文档`), and showing
|
|
598
|
+
those renders as gibberish — merging their spans and slicing the original text recovers `项目文档`.
|
|
599
|
+
The label is **your own wording**, so its casing is preserved (type `BM25`, see `BM25`).
|
|
600
|
+
|
|
601
|
+
All of it is string arithmetic — **no embeddings, no model calls**.
|
|
602
|
+
|
|
603
|
+
**And the honest boundaries**:
|
|
604
|
+
|
|
605
|
+
- **With only one or two messages** there are too few keywords, so it falls back to sorting by
|
|
606
|
+
modification time and says so in the panel — it does not pretend to rank
|
|
607
|
+
- **With only three to five documents IDF barely does anything**: `df` only takes a few values,
|
|
608
|
+
so its dynamic range collapses. The more documents, the better this ranking gets
|
|
609
|
+
- **It can only rank documents that share vocabulary with the conversation**: if no term matches,
|
|
610
|
+
every document scores the same and the order degrades to time
|
|
550
611
|
|
|
551
612
|
---
|
|
552
613
|
|
|
@@ -604,13 +665,32 @@ The relevance figure only affects ordering — it is **never displayed and never
|
|
|
604
665
|
|
|
605
666
|
---
|
|
606
667
|
|
|
668
|
+
## Feedback
|
|
669
|
+
|
|
670
|
+
**Right now this project is not about adding features — it's about finding out whether
|
|
671
|
+
"rank project docs by the current conversation" is something anyone actually uses.**
|
|
672
|
+
So the most valuable thing you can send is not "could you add X" but **how you work around
|
|
673
|
+
this today** — including "I installed it and never opened it", which is more useful than a
|
|
674
|
+
feature request.
|
|
675
|
+
|
|
676
|
+
- 💬 [Tell us when you actually open it](https://github.com/PolinniZhong/dsh-knit/issues/new?template=feature.yml) — two fields, thirty seconds
|
|
677
|
+
- 🐞 [Won't install / panel won't open / ranking looks wrong](https://github.com/PolinniZhong/dsh-knit/issues/new?template=bug.yml)
|
|
678
|
+
- 📖 Check [known limitations](#known-limitations) first — short conversations degrade to time order and Chinese uses n-gram approximation; those are deliberate trade-offs, not bugs
|
|
679
|
+
|
|
680
|
+
> A single issue is treated as a real signal. So far this plugin has **no trace of a single
|
|
681
|
+
> real user** — the npm download count is automated version enumeration, not people
|
|
682
|
+
> (only 14% of it is `latest`, and a human install only ever pulls `latest`). One human reply
|
|
683
|
+
> changes what gets built next.
|
|
684
|
+
|
|
685
|
+
---
|
|
686
|
+
|
|
607
687
|
## Development
|
|
608
688
|
|
|
609
689
|
```sh
|
|
610
690
|
git clone https://github.com/PolinniZhong/dsh-knit.git
|
|
611
691
|
cd dsh-knit
|
|
612
692
|
|
|
613
|
-
npm test #
|
|
693
|
+
npm test # 641 tests, zero dependencies, no npm install needed
|
|
614
694
|
```
|
|
615
695
|
|
|
616
696
|
**How changes take effect**: the host half (`src/host/`) **requires a DSH restart** (no hot reload);
|
|
@@ -631,11 +711,12 @@ knit/
|
|
|
631
711
|
│ ├── host/relevance.js # the relevance engine: BM25 + keyword extraction
|
|
632
712
|
│ ├── host/links.js # reference parsing (pure; never touches ranking)
|
|
633
713
|
│ ├── host/classification.js # v0.19 file classification: kind / language / searchable / contextual
|
|
714
|
+
│ ├── host/passage.js # passage layer: chunking / passage scoring / line mapping (pure, zero I/O)
|
|
634
715
|
│ ├── host/context.js # context assembly: the three-tier Context Pack (pure, zero I/O)
|
|
635
716
|
│ ├── host/feedback.js # usage feedback: read-time attribution / Context Epoch / delta (pure)
|
|
636
717
|
│ ├── host/tool.js # the agent tool knit_docs (hand-written ToolDefinition)
|
|
637
718
|
│ └── client/client.js # dual-host registration + panel UI
|
|
638
|
-
└── test/ #
|
|
719
|
+
└── test/ # 641 tests
|
|
639
720
|
├── eval/ # retrieval quality: corpus + cases + frozen v0.5.2 baseline
|
|
640
721
|
└── context/ # context tiering: 24-doc corpus + 12 real-task cases
|
|
641
722
|
```
|