@rahmantech/knl-view 0.3.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +178 -261
- package/bin/knl-view.js +12 -6
- package/package.json +1 -1
- package/server/consensus.js +0 -4
- package/server/graph/modules_builder.js +18 -8
- package/server/graph/modules_catalog.js +99 -58
- package/server/graph/sources.js +1 -2
- package/server/scaffold.js +4 -3
package/README.md
CHANGED
|
@@ -1,79 +1,43 @@
|
|
|
1
1
|
# knl-view
|
|
2
2
|
|
|
3
|
-
A local viewer for `.knowledges` knowledge bases: a CLI that serves a small web
|
|
4
|
-
app over a plain folder of Markdown files. The disk is the source of truth;
|
|
5
|
-
`map.json` is optional decoration.
|
|
3
|
+
A local viewer and graph engine for `.knowledges` knowledge bases: a CLI that serves a small web app over a plain folder of Markdown files and tracks project architecture lineage. The disk is the source of truth; `map.json` is optional decoration.
|
|
6
4
|
|
|
7
|
-
```
|
|
5
|
+
```sh
|
|
8
6
|
knl-view --port 8086 --path=.knowledges
|
|
9
7
|
```
|
|
10
8
|
|
|
11
|
-
##
|
|
9
|
+
## Collaboration Model: Human, AI, and AnchorDB
|
|
12
10
|
|
|
13
|
-
-
|
|
14
|
-
step for your content, no account.
|
|
15
|
-
- Sidebar tree with a fixed `index` entry (the root `README.md`) and a
|
|
16
|
-
`contents` entry that lists the whole base on one page.
|
|
17
|
-
- Folders open their own `README.md`, rendered the same way as a normal document.
|
|
18
|
-
- `map.json` (optional) relabels, reorders, or hides entries in the menu. It
|
|
19
|
-
never creates, moves, or deletes files. Entries pointing at paths that are not
|
|
20
|
-
on disk are ignored silently.
|
|
21
|
-
- Markdown with GFM tables, and triple-backtick fences tagged `mermaid` rendered
|
|
22
|
-
as diagrams. The mermaid bundle is served locally and loaded only when a
|
|
23
|
-
document contains a diagram; nothing is fetched from a CDN.
|
|
24
|
-
- Per-diagram controls: toggle between diagram and source, and a zoom modal
|
|
25
|
-
(wheel zoom, drag to pan, +/−/Reset, Esc to close).
|
|
26
|
-
- Table of contents per document, breadcrumb, and a filter box in the sidebar
|
|
27
|
-
that narrows the index by file title or path.
|
|
28
|
-
- Interface in English or Indonesian, switchable in the top bar. English is the
|
|
29
|
-
default.
|
|
30
|
-
- Whole-project dependency and symbol call graph extraction via WebAssembly
|
|
31
|
-
Tree-Sitter for JavaScript, TypeScript, Go, Python, and Rust.
|
|
32
|
-
|
|
33
|
-
## Requirements
|
|
11
|
+
`knl-view` is built around a practical three-way collaboration model that aligns AI coding agents with human intent while keeping context windows clean and efficient:
|
|
34
12
|
|
|
35
|
-
|
|
36
|
-
-
|
|
37
|
-
|
|
13
|
+
### 1. The Human Foundation (`.knowledges`)
|
|
14
|
+
- **Single Source of Truth**: The human architect defines the system architecture, domain constraints, feature roadmaps, and audit notes in `.knowledges/`.
|
|
15
|
+
- **Human-in-the-Loop**: AI agents can assist in drafting and organizing documentation, but the human validates and approves business logic and architectural boundaries. This ensures coding agents have an unambiguous reference rather than guessing intent.
|
|
38
16
|
|
|
39
|
-
|
|
17
|
+
### 2. Decision History (`decision-history`)
|
|
18
|
+
- **Traceable Decision Records**: Major architectural choices, technical trade-offs, and strategic pivots are recorded as append-only records using the [`decision-history`](https://github.com/rachmanzz/decisionskills) skill.
|
|
19
|
+
- **Context Continuity**: Future AI agent sessions read these records to understand *why* specific patterns exist, preventing agents from re-introducing rejected approaches or modifying agreed-upon architectures without context.
|
|
40
20
|
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
21
|
+
### 3. Graph Storage & Dependency Efficiency (`AnchorDB`)
|
|
22
|
+
- **Structured Dependency Graphs**: AnchorDB stores indexed relationships for project source code (`sources`) and third-party libraries (`modules-npm`, `modules-go`, etc.) as nodes and edges.
|
|
23
|
+
- **Targeted Lineage Queries**: Instead of dumping large code files or entire dependency trees into LLM context, AI agents query specific caller/callee and import relationships via `knl-view sources trace` and `knl-view modules trace`.
|
|
24
|
+
- **Pre-Edit Impact Analysis**: Agents identify the exact blast radius of changes before modifying code, speeding up dependency analysis and conserving context tokens without guessing.
|
|
44
25
|
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
```sh
|
|
50
|
-
ln -s "$(npm prefix -g)/bin/knl-view" ~/.local/bin/knl-view
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
The published tarball ships the built `dist/`, so the install compiles
|
|
54
|
-
nothing.
|
|
55
|
-
|
|
56
|
-
Run it once without installing:
|
|
26
|
+
### 4. Autonomous Agent Execution
|
|
27
|
+
- Standardized operating rules are seeded into `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, and project skills via `knl-view setup`.
|
|
28
|
+
- AI coding agents (Antigravity, Claude Code, Cursor, OpenCode) execute graph traces before editing, build module graphs on-demand, and sync changes back to AnchorDB autonomously. Developers assign tasks normally while agents manage graph operations in the background.
|
|
57
29
|
|
|
58
|
-
|
|
59
|
-
npx @rahmantech/knl-view
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
Or install from source, for a local checkout or an unreleased commit:
|
|
63
|
-
|
|
64
|
-
```sh
|
|
65
|
-
git clone https://github.com/rachmanzz/knl-view.git
|
|
66
|
-
cd knl-view
|
|
67
|
-
npm install
|
|
68
|
-
npm run build
|
|
69
|
-
npm link
|
|
70
|
-
```
|
|
30
|
+
## What it does
|
|
71
31
|
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
`
|
|
75
|
-
|
|
76
|
-
|
|
32
|
+
- Serves a web viewer over any folder of Markdown files. No database, no build step for your content, no account.
|
|
33
|
+
- Sidebar tree with a fixed `index` entry (the root `README.md`) and a `contents` entry that lists the whole base on one page.
|
|
34
|
+
- Folders open their own `README.md`, rendered the same way as a normal document.
|
|
35
|
+
- `map.json` (optional) relabels, reorders, or hides entries in the menu. It never creates, moves, or deletes files. Entries pointing at paths that are not on disk are ignored silently.
|
|
36
|
+
- Markdown with GFM tables, and triple-backtick fences tagged `mermaid` rendered as diagrams. The mermaid bundle is served locally and loaded only when a document contains a diagram; nothing is fetched from a CDN.
|
|
37
|
+
- Per-diagram controls: toggle between diagram and source, and a zoom modal (wheel zoom, drag to pan, +/−/Reset, Esc to close).
|
|
38
|
+
- Table of contents per document, breadcrumb, and a filter box in the sidebar that narrows the index by file title or path.
|
|
39
|
+
- Interface in English or Indonesian, switchable in the top bar. English is the default.
|
|
40
|
+
- Whole-project dependency and symbol call graph extraction via WebAssembly Tree-Sitter for JavaScript, TypeScript, Go, Python, and Rust.
|
|
77
41
|
|
|
78
42
|
## How to Set Up
|
|
79
43
|
|
|
@@ -104,6 +68,45 @@ After setup, start the viewer immediately:
|
|
|
104
68
|
knl-view
|
|
105
69
|
```
|
|
106
70
|
|
|
71
|
+
## Requirements & Installation
|
|
72
|
+
|
|
73
|
+
### Requirements
|
|
74
|
+
|
|
75
|
+
- Node.js 20.19 or newer.
|
|
76
|
+
- A built frontend. `dist/` is committed to this repository, so an install from GitHub already has it. The CLI refuses to start without it and tells you so.
|
|
77
|
+
|
|
78
|
+
### Install
|
|
79
|
+
|
|
80
|
+
```sh
|
|
81
|
+
npm install -g @rahmantech/knl-view
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
That puts the `knl-view` command into Node's global bin, `$(npm prefix -g)/bin`. If that directory is not on your PATH, link the command somewhere that is:
|
|
85
|
+
|
|
86
|
+
```sh
|
|
87
|
+
ln -s "$(npm prefix -g)/bin/knl-view" ~/.local/bin/knl-view
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
The published tarball ships the built `dist/`, so the install compiles nothing.
|
|
91
|
+
|
|
92
|
+
Run it once without installing:
|
|
93
|
+
|
|
94
|
+
```sh
|
|
95
|
+
npx @rahmantech/knl-view
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Or install from source, for a local checkout or an unreleased commit:
|
|
99
|
+
|
|
100
|
+
```sh
|
|
101
|
+
git clone https://github.com/rachmanzz/knl-view.git
|
|
102
|
+
cd knl-view
|
|
103
|
+
npm install
|
|
104
|
+
npm run build
|
|
105
|
+
npm link
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
`npm link` puts `knl-view` on your PATH from the checkout. To install a GitHub commit without cloning, run `npm install -g github:rachmanzz/knl-view --install-links`; without `--install-links`, npm 11 links against a temporary clone it deletes afterwards, which leaves a broken install.
|
|
109
|
+
|
|
107
110
|
## Usage
|
|
108
111
|
|
|
109
112
|
```sh
|
|
@@ -136,107 +139,9 @@ knl-view --seed-skill=knl-view --dir=global --yes
|
|
|
136
139
|
| `--help`, `-h` | | Show help |
|
|
137
140
|
| `--version`, `-v` | | Show version |
|
|
138
141
|
|
|
139
|
-
The server binds to `127.0.0.1` by default, so it is reachable only from your
|
|
140
|
-
machine unless you pass `--host` yourself. There is no authentication; treat
|
|
141
|
-
`--host 0.0.0.0` as a decision you are making on purpose.
|
|
142
|
-
|
|
143
|
-
## Configuration
|
|
144
|
-
|
|
145
|
-
Two optional config files. A missing file is never an error; a broken file is
|
|
146
|
-
ignored with one warning on stderr. CLI flags always win over config values.
|
|
147
|
-
|
|
148
|
-
**Global (`~/.knlview/knl-config.json`)**: shared by every project on the
|
|
149
|
-
machine. Holds the `anchordb` block. Without it, AnchorDB stays completely
|
|
150
|
-
off and the viewer runs fully local:
|
|
151
|
-
|
|
152
|
-
```json
|
|
153
|
-
{
|
|
154
|
-
"anchordb": {
|
|
155
|
-
"url": "wire://127.0.0.1:17448",
|
|
156
|
-
"token": "your-token",
|
|
157
|
-
"sync": true,
|
|
158
|
-
"deamon": "podman",
|
|
159
|
-
"container_name": "anchordb-knlview",
|
|
160
|
-
"container_id": "56519e7acbaa...",
|
|
161
|
-
"container_created_on": "2026-10-31 14:08:48..."
|
|
162
|
-
}
|
|
163
|
-
}
|
|
164
|
-
```
|
|
165
|
-
|
|
166
|
-
**Per project: `knl-config.json` at the project root** (gitignored):
|
|
167
|
-
`path`, `port`, `host`, `models`, and optional `include` / `exclude` patterns:
|
|
168
|
-
|
|
169
|
-
```json
|
|
170
|
-
{
|
|
171
|
-
"port": 8091,
|
|
172
|
-
"models": {
|
|
173
|
-
"knowledges": "vez0vqrp",
|
|
174
|
-
"consensus": "n0qrnmsz"
|
|
175
|
-
},
|
|
176
|
-
"include": ["dist/bundle.js"],
|
|
177
|
-
"exclude": ["legacy/**", "*.tmp"]
|
|
178
|
-
}
|
|
179
|
-
```
|
|
180
|
-
|
|
181
|
-
- `exclude`: Hard exclude that strictly filters files and folders from graph analysis and knowledge tree. Always takes precedence over defaults and includes.
|
|
182
|
-
- `include`: Whitelists files and folders. Overrides default directory exclusions (e.g. including files from `dist/` or dot-folders).
|
|
183
|
-
|
|
184
|
-
Model names are owned by the `models` map: role → name, and the name can
|
|
185
|
-
be anything (random names are fine; the map records what each name is
|
|
186
|
-
*for*). Names stay stable even if the project folder is renamed. knl-view
|
|
187
|
-
never invents names by itself: create them with
|
|
188
|
-
`knl-view model new <role>` (writes the map and the global registry), and
|
|
189
|
-
inspect ownership with `knl-view model check`.
|
|
142
|
+
The server binds to `127.0.0.1` by default, so it is reachable only from your machine unless you pass `--host` yourself. There is no authentication; treat `--host 0.0.0.0` as a decision you are making on purpose.
|
|
190
143
|
|
|
191
|
-
|
|
192
|
-
(project, role, path) and warns when two projects claim the same name. A
|
|
193
|
-
project whose map has no entry for a role runs fully local: AnchorDB is
|
|
194
|
-
touched only once the role is mapped.
|
|
195
|
-
|
|
196
|
-
## Operating System Support & Installing AnchorDB
|
|
197
|
-
|
|
198
|
-
### Operating System Support
|
|
199
|
-
|
|
200
|
-
| OS | Core Viewer (`knl-view`) | Global Config Location | AnchorDB Backend |
|
|
201
|
-
| --- | --- | --- | --- |
|
|
202
|
-
| **Linux** | Native (Node.js >= 20.19) | `~/.knlview/knl-config.json` | **Native Binary** (`amd64`, `arm64`) or Container |
|
|
203
|
-
| **macOS** | Native (Node.js >= 20.19) | `~/.knlview/knl-config.json` | **Container**: **Podman (Recommended)** or Docker |
|
|
204
|
-
| **Windows** | Native (Node.js >= 20.19) | `%APPDATA%\knlview\knl-config.json` | **Container**: **Podman (Recommended)** or Docker |
|
|
205
|
-
|
|
206
|
-
---
|
|
207
|
-
|
|
208
|
-
### Installing AnchorDB (Optional)
|
|
209
|
-
|
|
210
|
-
AnchorDB is an optional layer. Without it, the viewer runs 100% locally with zero external dependencies: document browsing, full-text search, the Markdown graph engine, and `knl-view graph trace` all work immediately. Installing AnchorDB adds persistent multi-hop causal tracing across sessions, whole-project source code dependency syncing, and shared multi-machine graphs.
|
|
211
|
-
|
|
212
|
-
Run `knl-view install anchordb` to install AnchorDB and write the `anchordb` block into your global configuration:
|
|
213
|
-
|
|
214
|
-
#### 1. Linux (Native Standalone Binary)
|
|
215
|
-
- Native binaries are published on [GitHub Releases](https://github.com/rachmanzz/anchordb/releases) for Linux architectures:
|
|
216
|
-
- `linux_amd64` (x86_64)
|
|
217
|
-
- `linux_arm64` (aarch64)
|
|
218
|
-
- On Linux, `knl-view install anchordb` automatically:
|
|
219
|
-
- Fetches the latest release asset and verifies it against the official `SHA256SUMS.txt`.
|
|
220
|
-
- Installs `anchordb-server` and `anchordb` into `~/.knlview/bin/` with secure file permissions (`0o700` / `0o755`).
|
|
221
|
-
- Starts `anchordb-server` detached as a background process daemon (logs in `~/.knlview/anchordb.log`, data in `~/.knlview/anchordb-data`).
|
|
222
|
-
- You can also explicitly opt for containers on Linux using `--method=container`.
|
|
223
|
-
|
|
224
|
-
#### 2. Windows & macOS (Container via Podman or Docker)
|
|
225
|
-
- AnchorDB relies on Linux kernel primitives (`mmap` memory mapping and POSIX file locking). Native binaries are not yet published for Windows or macOS.
|
|
226
|
-
- For Windows and macOS, AnchorDB is currently deployed via a lightweight container image (`ghcr.io/rachmanzz/anchordb`):
|
|
227
|
-
- **Podman (Recommended)**: Rootless, daemonless, lightweight, and requires no administrative privileges or background daemons.
|
|
228
|
-
- **Docker**: Supported via Docker Desktop or Docker Engine (`--runtime=docker`).
|
|
229
|
-
- `knl-view install anchordb` automatically detects installed container runtimes. If both are found, it prompts you to select **Podman (Recommended)** or **Docker**. If neither is installed, it halts with clear installation guidance.
|
|
230
|
-
|
|
231
|
-
#### Flags & Automation (CI / Scripts)
|
|
232
|
-
- Non-interactive mode: add `--yes` (accepts all defaults).
|
|
233
|
-
- Force install method: `--method=binary` or `--method=container`.
|
|
234
|
-
- Specify container engine: `--runtime=podman` (recommended) or `--runtime=docker`.
|
|
235
|
-
- Custom token: `--token=<text>` (default: cryptographic 32-byte hex token).
|
|
236
|
-
- Port conflict handling: automatically detects if port 17448 is occupied and suggests alternative free ports.
|
|
237
|
-
- Data safety: if a container named `anchordb-knlview` already exists, the installer stops rather than overwriting existing data.
|
|
238
|
-
|
|
239
|
-
## How a base is read
|
|
144
|
+
## How a Knowledge Base is Read
|
|
240
145
|
|
|
241
146
|
A base is an ordinary folder:
|
|
242
147
|
|
|
@@ -251,9 +156,7 @@ A base is an ordinary folder:
|
|
|
251
156
|
auth.md
|
|
252
157
|
```
|
|
253
158
|
|
|
254
|
-
Reading order: the root `README.md`, then `map.json` when present, then the
|
|
255
|
-
structure on disk. Without `map.json`, menu labels fall back to the first
|
|
256
|
-
`# heading` of each file, then to the plain file name (files before folders).
|
|
159
|
+
Reading order: the root `README.md`, then `map.json` when present, then the structure on disk. Without `map.json`, menu labels fall back to the first `# heading` of each file, then to the plain file name (files before folders).
|
|
257
160
|
|
|
258
161
|
### map.json
|
|
259
162
|
|
|
@@ -285,12 +188,9 @@ Every entry is decoration. Nothing in it can invent a file that is not on disk.
|
|
|
285
188
|
| `order` | any entry | Ascending menu order; entries without `order` follow, files before folders |
|
|
286
189
|
| `ignore` (or `hidden`) | any entry | `true` hides it from the menu, lists, search, home, and contents |
|
|
287
190
|
|
|
288
|
-
Paths are relative to the `.knowledges` root, use `/` separators, and must
|
|
289
|
-
match the on-disk path exactly, for example `features/auth.md`. `ignore` is
|
|
290
|
-
view-only: the file stays on disk. Remove the flag to bring it back.
|
|
191
|
+
Paths are relative to the `.knowledges` root, use `/` separators, and must match the on-disk path exactly, for example `features/auth.md`. `ignore` is view-only: the file stays on disk. Remove the flag to bring it back.
|
|
291
192
|
|
|
292
|
-
The full field reference lives in the bundled skill:
|
|
293
|
-
`.agents/skills/knl-view/SKILL.md`.
|
|
193
|
+
The full field reference lives in the bundled skill: `.agents/skills/knl-view/SKILL.md`.
|
|
294
194
|
|
|
295
195
|
## AI Graph & Decision Tracing
|
|
296
196
|
|
|
@@ -300,53 +200,28 @@ The full field reference lives in the bundled skill:
|
|
|
300
200
|
- **Decision Lineage Tracking**: Automatically discovers `supersedes`, `related_to`, and `affects_knowledge` edges from `decisionskills` decision blocks.
|
|
301
201
|
- **Low-Token Subgraphs for AI**: Instead of loading an entire knowledge base into LLM context, query exact $N$-hop subgraphs in sub-milliseconds (< 1 ms).
|
|
302
202
|
|
|
303
|
-
This engine is fully local: it needs no AnchorDB and no network. AnchorDB
|
|
304
|
-
(see the optional install above) adds persistence on top: edges survive
|
|
305
|
-
across sessions, are synced incrementally, and can be traced from any
|
|
306
|
-
machine pointing at the same server.
|
|
203
|
+
This engine is fully local: it needs no AnchorDB and no network. AnchorDB (see the optional install below) adds persistence on top: edges survive across sessions, are synced incrementally, and can be traced from any machine pointing at the same server.
|
|
307
204
|
|
|
308
205
|
### Two-way lineage
|
|
309
206
|
|
|
310
|
-
Traces walk in a mechanically defined direction relative to the edges,
|
|
311
|
-
never interpreted as cause and effect (`depends_on` edges *leave* a
|
|
312
|
-
decision while pointing at its upstream, so up/down labels would lie):
|
|
207
|
+
Traces walk in a mechanically defined direction relative to the edges, never interpreted as cause and effect (`depends_on` edges *leave* a decision while pointing at its upstream, so up/down labels would lie):
|
|
313
208
|
|
|
314
|
-
- `direction=out`: edges leaving the target: what it supersedes,
|
|
315
|
-
|
|
316
|
-
- `direction=
|
|
317
|
-
replaces it.
|
|
318
|
-
- `direction=both`: the default for the local graph trace; the full
|
|
319
|
-
neighborhood.
|
|
209
|
+
- `direction=out`: edges leaving the target: what it supersedes, `depends_on`, or affects.
|
|
210
|
+
- `direction=in`: edges entering the target: what references or replaces it.
|
|
211
|
+
- `direction=both`: the default for the local graph trace; the full neighborhood.
|
|
320
212
|
|
|
321
|
-
Every returned node carries `via` (`center`, `in`, `out`, or `in+out`)
|
|
322
|
-
telling how it connects to the trace center.
|
|
213
|
+
Every returned node carries `via` (`center`, `in`, `out`, or `in+out`) telling how it connects to the trace center.
|
|
323
214
|
|
|
324
|
-
CLI: `knl-view graph trace --target=<id> --direction=out`
|
|
215
|
+
CLI: `knl-view graph trace --target=<id> --direction=out`
|
|
325
216
|
HTTP: `GET /api/graph/trace?target=<id>&direction=in`
|
|
326
217
|
|
|
327
|
-
The AnchorDB endpoint adds the same parameter,
|
|
328
|
-
`GET /api/anchordb/trace?target=<id>&direction=in|out|both` (default
|
|
329
|
-
`out`): `out` is scored server-side via `causalChain`; `in` is walked
|
|
330
|
-
backwards from the local sync mirror; the wire protocol has no reverse
|
|
331
|
-
query, and the mirror holds every edge this project ever linked, so it
|
|
332
|
-
is exactly the inbound map for this project's model. Mirror results
|
|
333
|
-
carry score 1 and the response reports `mirrored: true`/`false` so
|
|
334
|
-
consumers know which half came from where.
|
|
218
|
+
The AnchorDB endpoint adds the same parameter, `GET /api/anchordb/trace?target=<id>&direction=in|out|both` (default `out`): `out` is scored server-side via `causalChain`; `in` is walked backwards from the local sync mirror; the wire protocol has no reverse query, and the mirror holds every edge this project ever linked, so it is exactly the inbound map for this project's model. Mirror results carry score 1 and the response reports `mirrored: true`/`false` so consumers know which half came from where.
|
|
335
219
|
|
|
336
220
|
### Consensus (claims)
|
|
337
221
|
|
|
338
|
-
A second AnchorDB model, the `consensus` role in the project's models map,
|
|
339
|
-
holds agreement and conflict between *claims*: short statements that are
|
|
340
|
-
content-addressed (`c-` + 16 hex chars of the statement's hash), so the same
|
|
341
|
-
text is the same claim on every machine. Claims relate only to other claims,
|
|
342
|
-
through two fixed relation types: `agrees_with` and `contradicts`. The
|
|
343
|
-
consensus structure lives purely in AnchorDB; the local artifacts (claim
|
|
344
|
-
index in `~/.knlview/claims/`, edge mirror in `~/.knlview/sync-state/`) are
|
|
345
|
-
caches that degrade convenience, never the data.
|
|
222
|
+
A second AnchorDB model, the `consensus` role in the project's models map, holds agreement and conflict between *claims*: short statements that are content-addressed (`c-` + 16 hex chars of the statement's hash), so the same text is the same claim on every machine. Claims relate only to other claims, through two fixed relation types: `agrees_with` and `contradicts`. The consensus structure lives purely in AnchorDB; the local artifacts (claim index in `~/.knlview/claims/`, edge mirror in `~/.knlview/sync-state/`) are caches that degrade convenience, never the data.
|
|
346
223
|
|
|
347
|
-
A claim's status is derived from who points at it: any incoming `contradicts`
|
|
348
|
-
makes it `disputed`, otherwise any incoming `agrees_with` makes it
|
|
349
|
-
`consensus`, otherwise it is `uncontested`.
|
|
224
|
+
A claim's status is derived from who points at it: any incoming `contradicts` makes it `disputed`, otherwise any incoming `agrees_with` makes it `consensus`, otherwise it is `uncontested`.
|
|
350
225
|
|
|
351
226
|
```sh
|
|
352
227
|
knl-view consensus claim "We use bcrypt for password hashing"
|
|
@@ -357,48 +232,23 @@ knl-view consensus status c-437ff9f10f83edde
|
|
|
357
232
|
knl-view consensus list
|
|
358
233
|
```
|
|
359
234
|
|
|
360
|
-
Mutations are CLI-only on purpose: one write path, one mirror kept honest.
|
|
361
|
-
The read-only HTTP endpoint answers "is this claim settled?":
|
|
235
|
+
Mutations are CLI-only on purpose: one write path, one mirror kept honest. The read-only HTTP endpoint answers "is this claim settled?":
|
|
362
236
|
|
|
363
|
-
`GET /api/consensus?target=<claimId>` returns the claim's `status`,
|
|
364
|
-
`supporters`, `objectors`, its own `stance`, `mirrored` (whether a valid
|
|
365
|
-
local mirror backs the inbound half), and a `statements` map of the
|
|
366
|
-
claim texts involved. Returns `503` when AnchorDB is off or the
|
|
367
|
-
`consensus` role is not mapped.
|
|
237
|
+
`GET /api/consensus?target=<claimId>` returns the claim's `status`, `supporters`, `objectors`, its own `stance`, `mirrored` (whether a valid local mirror backs the inbound half), and a `statements` map of the claim texts involved. Returns `503` when AnchorDB is off or the `consensus` role is not mapped.
|
|
368
238
|
|
|
369
239
|
### Sources graph (whole project)
|
|
370
240
|
|
|
371
|
-
A third AnchorDB model, the `sources` role in the models map, holds the
|
|
372
|
-
dependency graph of the whole project, not just `.knowledges`. Every file
|
|
373
|
-
becomes a node (id = repo-relative path; hidden folders are skipped except
|
|
374
|
-
`.github`, and `node_modules`, `dist`, and `.gitignore` patterns are
|
|
375
|
-
ignored). Import edges are extracted with tree-sitter syntax parsing
|
|
376
|
-
(JavaScript, TypeScript/TSX, Go, Python, and Rust, one grammar package per
|
|
377
|
-
language family) and stored in both directions (`imports`, `imported_by`) so
|
|
378
|
-
lineage works without any local mirror: sync is server-authoritative, enumerating
|
|
379
|
-
the server's own records and edges before applying the difference from a fresh
|
|
380
|
-
disk scan.
|
|
241
|
+
A third AnchorDB model, the `sources` role in the models map, holds the dependency graph of the whole project, not just `.knowledges`. Every file becomes a node (id = repo-relative path; hidden folders are skipped except `.github`, and `node_modules`, `dist`, and `.gitignore` patterns are ignored). Import edges are extracted with tree-sitter syntax parsing (JavaScript, TypeScript/TSX, Go, Python, and Rust, one grammar package per language family) and stored in both directions (`imports`, `imported_by`) so lineage works without any local mirror: sync is server-authoritative, enumerating the server's own records and edges before applying the difference from a fresh disk scan.
|
|
381
242
|
|
|
382
243
|
Each file also defines **symbol nodes**, extracted from the same parse:
|
|
383
244
|
|
|
384
|
-
- Symbol ids are `<path>#<name>`; class methods are `Class.method`. A name
|
|
385
|
-
defined more than once in one file gets a line suffix
|
|
386
|
-
(`<path>#<name>@<line>`), so duplicate helpers stay distinct records.
|
|
245
|
+
- Symbol ids are `<path>#<name>`; class methods are `Class.method`. A name defined more than once in one file gets a line suffix (`<path>#<name>@<line>`), so duplicate helpers stay distinct records.
|
|
387
246
|
- Kinds: `function`, `class`, `method`, `type`, `interface`, `enum`, `struct`, `trait`.
|
|
388
|
-
- Edges: `defines`/`defined_in` (file → symbol) and `calls`/`called_by`
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
project-wide (same package directory preferred in Go). A receiver-qualified
|
|
394
|
-
call (`obj.Method()`) resolves only when the receiver is an import
|
|
395
|
-
binding of the file, otherwise it is skipped, never guessed.
|
|
396
|
-
- Label text (symbol names and kinds; AnchorDB attributes are
|
|
397
|
-
numeric-only) is written to `.knowledges/.knl-data/<path>.md`, one file
|
|
398
|
-
per source file that has symbols; override the location with `labelDir`
|
|
399
|
-
in the project config. Labels are regenerated and pruned on every sync.
|
|
400
|
-
|
|
401
|
-
#### Supported Languages (v0.3.0)
|
|
247
|
+
- Edges: `defines`/`defined_in` (file → symbol) and `calls`/`called_by` (symbol → symbol), each stored in both directions because causalChain walks forward only.
|
|
248
|
+
- Call resolution is conservative: a call belongs to its innermost enclosing symbol, and the callee must match exactly one definition project-wide (same package directory preferred in Go). A receiver-qualified call (`obj.Method()`) resolves only when the receiver is an import binding of the file, otherwise it is skipped, never guessed.
|
|
249
|
+
- Label text (symbol names and kinds; AnchorDB attributes are numeric-only) is written to `.knowledges/.knl-data/<path>.md`, one file per source file that has symbols; override the location with `labelDir` in the project config. Labels are regenerated and pruned on every sync.
|
|
250
|
+
|
|
251
|
+
#### Supported Languages (v0.3.1)
|
|
402
252
|
|
|
403
253
|
Tree-sitter WASM grammars run in-memory within Node.js without requiring host compilers:
|
|
404
254
|
|
|
@@ -428,20 +278,7 @@ knl-view sources trace --target=server/index.js --direction=out --hops=2
|
|
|
428
278
|
knl-view sources trace --target=server/index.js#handleTrace --direction=in
|
|
429
279
|
```
|
|
430
280
|
|
|
431
|
-
`sync` is idempotent: run it again after refactoring and only the
|
|
432
|
-
difference is applied. A target containing `#` traces symbol lineage over
|
|
433
|
-
`calls`/`called_by`; a plain path traces file lineage over
|
|
434
|
-
`imports`/`imported_by`. Records the disk no longer has are archived with
|
|
435
|
-
the native lifecycle opcode 2 (TOMBSTONE: the server hides them from list
|
|
436
|
-
and lookups) after their edges are unlinked, and the `tombstone = 2`
|
|
437
|
-
attribute stays as a marker for servers that predate opcode support.
|
|
438
|
-
`sync` then asks the REST collector to purge those records physically
|
|
439
|
-
(scoped to the model); run `knl-view sources collect` by hand if the REST
|
|
440
|
-
endpoint was unreachable at that moment. A reappearing file revives the
|
|
441
|
-
record with a normal save and relinks its edges. Sync reports
|
|
442
|
-
`ORPHAN EDGES` when an edge endpoint failed to save, so a broken sync is
|
|
443
|
-
never silent. The read-only HTTP endpoint answers the same queries:
|
|
444
|
-
`GET /api/sources/trace?target=<id>&direction=in|out|both&hops=N`.
|
|
281
|
+
`sync` is idempotent: run it again after refactoring and only the difference is applied. A target containing `#` traces symbol lineage over `calls`/`called_by`; a plain path traces file lineage over `imports`/`imported_by`. Records the disk no longer has are archived with the native lifecycle opcode 2 (TOMBSTONE: the server hides them from list and lookups) after their edges are unlinked, and the `tombstone = 2` attribute stays as a marker for servers that predate opcode support. `sync` then asks the REST collector to purge those records physically (scoped to the model); run `knl-view sources collect` by hand if the REST endpoint was unreachable at that moment. A reappearing file revives the record with a normal save and relinks its edges. Sync reports `ORPHAN EDGES` when an edge endpoint failed to save, so a broken sync is never silent. The read-only HTTP endpoint answers the same queries: `GET /api/sources/trace?target=<id>&direction=in|out|both&hops=N`.
|
|
445
282
|
|
|
446
283
|
### External Modules Graph (On-Demand Module Boundary)
|
|
447
284
|
|
|
@@ -476,24 +313,104 @@ When `knl-view` is running as a local server:
|
|
|
476
313
|
- `GET /api/consensus?target=<claimId>`: Read-only consensus status for one claim (requires AnchorDB and a mapped `consensus` role). Returns `503` when AnchorDB is off or the role is unmapped, `400` without a target.
|
|
477
314
|
- `GET /api/sources/trace?target=<path>&direction=in|out&hops=2`: Read-only file lineage over the whole-project sources graph (requires AnchorDB and a mapped `sources` role). Returns `503` when unavailable, `400` without a target or with a bad direction.
|
|
478
315
|
|
|
316
|
+
## Operating System Support & Installing AnchorDB
|
|
317
|
+
|
|
318
|
+
### Operating System Support
|
|
319
|
+
|
|
320
|
+
| OS | Core Viewer (`knl-view`) | Global Config Location | AnchorDB Backend |
|
|
321
|
+
| --- | --- | --- | --- |
|
|
322
|
+
| **Linux** | Native (Node.js >= 20.19) | `~/.knlview/knl-config.json` | **Native Binary** (`amd64`, `arm64`) or Container |
|
|
323
|
+
| **macOS** | Native (Node.js >= 20.19) | `~/.knlview/knl-config.json` | **Container**: **Podman (Recommended)** or Docker |
|
|
324
|
+
| **Windows** | Native (Node.js >= 20.19) | `%APPDATA%\knlview\knl-config.json` | **Container**: **Podman (Recommended)** or Docker |
|
|
325
|
+
|
|
326
|
+
---
|
|
327
|
+
|
|
328
|
+
### Installing AnchorDB (Optional)
|
|
329
|
+
|
|
330
|
+
AnchorDB is an optional layer. Without it, the viewer runs 100% locally with zero external dependencies: document browsing, full-text search, the Markdown graph engine, and `knl-view graph trace` all work immediately. Installing AnchorDB adds persistent multi-hop causal tracing across sessions, whole-project source code dependency syncing, and shared multi-machine graphs.
|
|
331
|
+
|
|
332
|
+
Run `knl-view install anchordb` to install AnchorDB and write the `anchordb` block into your global configuration:
|
|
333
|
+
|
|
334
|
+
#### 1. Linux (Native Standalone Binary)
|
|
335
|
+
- Native binaries are published on [GitHub Releases](https://github.com/rachmanzz/anchordb/releases) for Linux architectures:
|
|
336
|
+
- `linux_amd64` (x86_64)
|
|
337
|
+
- `linux_arm64` (aarch64)
|
|
338
|
+
- On Linux, `knl-view install anchordb` automatically:
|
|
339
|
+
- Fetches the latest release asset and verifies it against the official `SHA256SUMS.txt`.
|
|
340
|
+
- Installs `anchordb-server` and `anchordb` into `~/.knlview/bin/` with secure file permissions (`0o700` / `0o755`).
|
|
341
|
+
- Starts `anchordb-server` detached as a background process daemon (logs in `~/.knlview/anchordb.log`, data in `~/.knlview/anchordb-data`).
|
|
342
|
+
- You can also explicitly opt for containers on Linux using `--method=container`.
|
|
343
|
+
|
|
344
|
+
#### 2. Windows & macOS (Container via Podman or Docker)
|
|
345
|
+
- AnchorDB relies on Linux kernel primitives (`mmap` memory mapping and POSIX file locking). Native binaries are not yet published for Windows or macOS.
|
|
346
|
+
- For Windows and macOS, AnchorDB is currently deployed via a lightweight container image (`ghcr.io/rachmanzz/anchordb`):
|
|
347
|
+
- **Podman (Recommended)**: Rootless, daemonless, lightweight, and requires no administrative privileges or background daemons.
|
|
348
|
+
- **Docker**: Supported via Docker Desktop or Docker Engine (`--runtime=docker`).
|
|
349
|
+
- `knl-view install anchordb` automatically detects installed container runtimes. If both are found, it prompts you to select **Podman (Recommended)** or **Docker**. If neither is installed, it halts with clear installation guidance.
|
|
350
|
+
|
|
351
|
+
#### Flags & Automation (CI / Scripts)
|
|
352
|
+
- Non-interactive mode: add `--yes` (accepts all defaults).
|
|
353
|
+
- Force install method: `--method=binary` or `--method=container`.
|
|
354
|
+
- Specify container engine: `--runtime=podman` (recommended) or `--runtime=docker`.
|
|
355
|
+
- Custom token: `--token=<text>` (default: cryptographic 32-byte hex token).
|
|
356
|
+
- Port conflict handling: automatically detects if port 17448 is occupied and suggests alternative free ports.
|
|
357
|
+
- Data safety: if a container named `anchordb-knlview` already exists, the installer stops rather than overwriting existing data.
|
|
358
|
+
|
|
359
|
+
## Configuration
|
|
360
|
+
|
|
361
|
+
Two optional config files. A missing file is never an error; a broken file is ignored with one warning on stderr. CLI flags always win over config values.
|
|
362
|
+
|
|
363
|
+
**Global (`~/.knlview/knl-config.json`)**: shared by every project on the machine. Holds the `anchordb` block. Without it, AnchorDB stays completely off and the viewer runs fully local:
|
|
364
|
+
|
|
365
|
+
```json
|
|
366
|
+
{
|
|
367
|
+
"anchordb": {
|
|
368
|
+
"url": "wire://127.0.0.1:17448",
|
|
369
|
+
"token": "your-token",
|
|
370
|
+
"sync": true,
|
|
371
|
+
"deamon": "podman",
|
|
372
|
+
"container_name": "anchordb-knlview",
|
|
373
|
+
"container_id": "56519e7acbaa...",
|
|
374
|
+
"container_created_on": "2026-10-31 14:08:48..."
|
|
375
|
+
}
|
|
376
|
+
}
|
|
377
|
+
```
|
|
378
|
+
|
|
379
|
+
**Per project: `knl-config.json` at the project root** (gitignored):
|
|
380
|
+
`path`, `port`, `host`, `models`, and optional `include` / `exclude` patterns:
|
|
381
|
+
|
|
382
|
+
```json
|
|
383
|
+
{
|
|
384
|
+
"port": 8091,
|
|
385
|
+
"models": {
|
|
386
|
+
"knowledges": "vez0vqrp",
|
|
387
|
+
"consensus": "n0qrnmsz"
|
|
388
|
+
},
|
|
389
|
+
"include": ["dist/bundle.js"],
|
|
390
|
+
"exclude": ["legacy/**", "*.tmp"]
|
|
391
|
+
}
|
|
392
|
+
```
|
|
393
|
+
|
|
394
|
+
- `exclude`: Hard exclude that strictly filters files and folders from graph analysis and knowledge tree. Always takes precedence over defaults and includes.
|
|
395
|
+
- `include`: Whitelists files and folders. Overrides default directory exclusions (e.g. including files from `dist/` or dot-folders).
|
|
396
|
+
|
|
397
|
+
Model names are owned by the `models` map: role → name, and the name can be anything (random names are fine; the map records what each name is *for*). Names stay stable even if the project folder is renamed. knl-view never invents names by itself: create them with `knl-view model new <role>` (writes the map and the global registry), and inspect ownership with `knl-view model check`.
|
|
398
|
+
|
|
399
|
+
The global registry `~/.knlview/models.json` records every name's owner (project, role, path) and warns when two projects claim the same name. A project whose map has no entry for a role runs fully local: AnchorDB is touched only once the role is mapped.
|
|
400
|
+
|
|
479
401
|
## Development
|
|
480
402
|
|
|
481
403
|
```sh
|
|
482
404
|
npm run dev
|
|
483
405
|
```
|
|
484
406
|
|
|
485
|
-
Starts the API on port 8085 (override with `PORT`) and the Vite dev server on
|
|
486
|
-
5173 (override with `WEB_PORT`). Pass a base folder as the first argument:
|
|
407
|
+
Starts the API on port 8085 (override with `PORT`) and the Vite dev server on 5173 (override with `WEB_PORT`). Pass a base folder as the first argument:
|
|
487
408
|
|
|
488
409
|
```sh
|
|
489
410
|
npm run dev -- ../some-project/.knowledges
|
|
490
411
|
```
|
|
491
412
|
|
|
492
|
-
`dist/` is committed to this repository so that installs straight from GitHub
|
|
493
|
-
keep working, because npm runs no build step for a git install without a
|
|
494
|
-
`prepare` script. After changing the frontend, run `npm run build` and commit
|
|
495
|
-
the result together with the source, or those installs stay on the previous
|
|
496
|
-
build.
|
|
413
|
+
`dist/` is committed to this repository so that installs straight from GitHub keep working, because npm runs no build step for a git install without a `prepare` script. After changing the frontend, run `npm run build` and commit the result together with the source, or those installs stay on the previous build.
|
|
497
414
|
|
|
498
415
|
## License
|
|
499
416
|
|
package/bin/knl-view.js
CHANGED
|
@@ -31,7 +31,9 @@ import {
|
|
|
31
31
|
import {
|
|
32
32
|
readModulesCatalog,
|
|
33
33
|
defaultEcosystemModel,
|
|
34
|
-
detectProjectManifests
|
|
34
|
+
detectProjectManifests,
|
|
35
|
+
isModuleSynced,
|
|
36
|
+
SUPPORTED_ECOSYSTEMS
|
|
35
37
|
} from '../server/graph/modules_catalog.js'
|
|
36
38
|
import { readModelRegistry, registerModel, generateModelName, modelRegistryPath } from '../server/models_registry.js'
|
|
37
39
|
import {
|
|
@@ -1282,7 +1284,10 @@ async function runSourcesCollector(base, model, { quiet }) {
|
|
|
1282
1284
|
async function runModulesCommand(sub, args, { values, settings }) {
|
|
1283
1285
|
if (sub === 'list') {
|
|
1284
1286
|
const ecoFilter = args?.[0]?.toLowerCase()
|
|
1285
|
-
|
|
1287
|
+
if (ecoFilter && !SUPPORTED_ECOSYSTEMS.has(ecoFilter)) {
|
|
1288
|
+
fail(`Unknown ecosystem "${args[0]}". Supported: ${[...SUPPORTED_ECOSYSTEMS].join(', ')}`)
|
|
1289
|
+
}
|
|
1290
|
+
const ecosystems = ecoFilter ? [ecoFilter] : [...SUPPORTED_ECOSYSTEMS]
|
|
1286
1291
|
const all = {}
|
|
1287
1292
|
for (const eco of ecosystems) {
|
|
1288
1293
|
all[eco] = await readModulesCatalog(eco)
|
|
@@ -1311,13 +1316,14 @@ async function runModulesCommand(sub, args, { values, settings }) {
|
|
|
1311
1316
|
const base = anchordbClientOptions(settings.anchordb)
|
|
1312
1317
|
|
|
1313
1318
|
if (!values.force) {
|
|
1314
|
-
const catalog = await readModulesCatalog(ecosystem)
|
|
1315
1319
|
const manifestVersions = await detectProjectManifests(process.cwd())
|
|
1316
1320
|
const expectedVersion = manifestVersions[ecosystem]?.[moduleName]
|
|
1317
1321
|
if (expectedVersion) {
|
|
1318
|
-
const
|
|
1319
|
-
if (
|
|
1320
|
-
|
|
1322
|
+
const already = await isModuleSynced(ecosystem, moduleName, expectedVersion)
|
|
1323
|
+
if (already) {
|
|
1324
|
+
const catalog = await readModulesCatalog(ecosystem)
|
|
1325
|
+
const key = `${moduleName}@${expectedVersion}`
|
|
1326
|
+
process.stdout.write(`module ${ecosystem}::${key} is already synced in model ${model} (${catalog[key]?.symbolCount || 0} symbols). Use --force to rebuild.\n`)
|
|
1321
1327
|
return
|
|
1322
1328
|
}
|
|
1323
1329
|
}
|
package/package.json
CHANGED
package/server/consensus.js
CHANGED
|
@@ -48,10 +48,6 @@ export function claimsIndexPath(model, home = null, options = {}) {
|
|
|
48
48
|
return path.join(knlGlobalDir(home, options), 'claims', `${model}.json`)
|
|
49
49
|
}
|
|
50
50
|
|
|
51
|
-
export function consensusMirrorPath(model, home = null, options = {}) {
|
|
52
|
-
return path.join(knlGlobalDir(home, options), 'consensus', `${model}.json`)
|
|
53
|
-
}
|
|
54
|
-
|
|
55
51
|
/**
|
|
56
52
|
* Read the claim index. Missing file = empty index, not an error.
|
|
57
53
|
* @returns {Promise<{claims: object, error: string|null}>}
|
|
@@ -8,7 +8,9 @@ import {
|
|
|
8
8
|
formatExternalPackageNodeId,
|
|
9
9
|
formatExternalSymbolNodeId,
|
|
10
10
|
defaultEcosystemModel,
|
|
11
|
-
updateModulesCatalog
|
|
11
|
+
updateModulesCatalog,
|
|
12
|
+
SUPPORTED_ECOSYSTEMS,
|
|
13
|
+
isValidModuleName
|
|
12
14
|
} from './modules_catalog.js'
|
|
13
15
|
|
|
14
16
|
const MODULE_DEFAULT_IGNORES = [
|
|
@@ -27,11 +29,14 @@ const MODULE_DEFAULT_IGNORES = [
|
|
|
27
29
|
]
|
|
28
30
|
|
|
29
31
|
/**
|
|
30
|
-
* Scan source files of an external module directory.
|
|
32
|
+
* Scan source files of an external module directory with depth and file count guard-rails.
|
|
31
33
|
*/
|
|
32
|
-
export async function scanModuleFiles(moduleDir, ecosystem) {
|
|
34
|
+
export async function scanModuleFiles(moduleDir, ecosystem, { maxDepth = 20, maxFiles = 5000 } = {}) {
|
|
33
35
|
const files = {}
|
|
34
|
-
|
|
36
|
+
let totalFiles = 0
|
|
37
|
+
|
|
38
|
+
const walk = async (dirAbs, rel, depth = 0) => {
|
|
39
|
+
if (depth > maxDepth || totalFiles >= maxFiles) return
|
|
35
40
|
let entries
|
|
36
41
|
try {
|
|
37
42
|
entries = await fs.readdir(dirAbs, { withFileTypes: true })
|
|
@@ -39,10 +44,11 @@ export async function scanModuleFiles(moduleDir, ecosystem) {
|
|
|
39
44
|
return
|
|
40
45
|
}
|
|
41
46
|
for (const entry of entries) {
|
|
47
|
+
if (totalFiles >= maxFiles) break
|
|
42
48
|
const childRel = rel ? `${rel}/${entry.name}` : entry.name
|
|
43
49
|
if (entry.isDirectory()) {
|
|
44
50
|
if (MODULE_DEFAULT_IGNORES.includes(entry.name.toLowerCase())) continue
|
|
45
|
-
await walk(path.join(dirAbs, entry.name), childRel)
|
|
51
|
+
await walk(path.join(dirAbs, entry.name), childRel, depth + 1)
|
|
46
52
|
continue
|
|
47
53
|
}
|
|
48
54
|
if (!entry.isFile()) continue
|
|
@@ -51,14 +57,16 @@ export async function scanModuleFiles(moduleDir, ecosystem) {
|
|
|
51
57
|
if (ecosystem === 'go' && entry.name.endsWith('_test.go')) continue
|
|
52
58
|
if (ecosystem === 'pypi' && entry.name.startsWith('test_')) continue
|
|
53
59
|
files[childRel] = path.join(dirAbs, entry.name)
|
|
60
|
+
totalFiles += 1
|
|
54
61
|
}
|
|
55
62
|
}
|
|
56
|
-
await walk(moduleDir, '')
|
|
63
|
+
await walk(moduleDir, '', 0)
|
|
57
64
|
return files
|
|
58
65
|
}
|
|
59
66
|
|
|
60
67
|
/**
|
|
61
68
|
* Parse an eco::name string into components: { ecosystem, moduleName }
|
|
69
|
+
* Strictly validates ecosystem against SUPPORTED_ECOSYSTEMS and moduleName against path traversal.
|
|
62
70
|
*/
|
|
63
71
|
export function parseModuleSpecifier(target) {
|
|
64
72
|
if (!target || typeof target !== 'string') return null
|
|
@@ -66,7 +74,9 @@ export function parseModuleSpecifier(target) {
|
|
|
66
74
|
if (idx <= 0) return null
|
|
67
75
|
const ecosystem = target.slice(0, idx).toLowerCase()
|
|
68
76
|
const moduleName = target.slice(idx + 2)
|
|
69
|
-
if (!moduleName)
|
|
77
|
+
if (!moduleName || !SUPPORTED_ECOSYSTEMS.has(ecosystem) || !isValidModuleName(moduleName)) {
|
|
78
|
+
return null
|
|
79
|
+
}
|
|
70
80
|
return { ecosystem, moduleName }
|
|
71
81
|
}
|
|
72
82
|
|
|
@@ -233,7 +243,7 @@ export async function syncModuleToAnchorDB(client, moduleGraph, options = {}) {
|
|
|
233
243
|
ecosystem: moduleGraph.ecosystem,
|
|
234
244
|
model: moduleGraph.model,
|
|
235
245
|
nodeId: moduleGraph.moduleNode.id,
|
|
236
|
-
|
|
246
|
+
symbolCount: moduleGraph.symbols.length
|
|
237
247
|
}
|
|
238
248
|
}, options)
|
|
239
249
|
|
|
@@ -2,6 +2,21 @@ import { promises as fs } from 'node:fs'
|
|
|
2
2
|
import path from 'node:path'
|
|
3
3
|
import { knlGlobalDir } from '../config.js'
|
|
4
4
|
|
|
5
|
+
export const SUPPORTED_ECOSYSTEMS = new Set(['npm', 'go', 'cargo', 'pypi'])
|
|
6
|
+
|
|
7
|
+
/**
|
|
8
|
+
* Validate that a module name does not contain traversal tokens, null bytes, or invalid paths.
|
|
9
|
+
*/
|
|
10
|
+
export function isValidModuleName(name) {
|
|
11
|
+
if (!name || typeof name !== 'string') return false
|
|
12
|
+
if (name.includes('\0')) return false
|
|
13
|
+
const segments = name.replace(/\\/g, '/').split('/')
|
|
14
|
+
for (const seg of segments) {
|
|
15
|
+
if (seg === '.' || seg === '..' || !seg) return false
|
|
16
|
+
}
|
|
17
|
+
return true
|
|
18
|
+
}
|
|
19
|
+
|
|
5
20
|
/**
|
|
6
21
|
* Returns the path to the global modules catalog directory (~/.knlview/modules).
|
|
7
22
|
*/
|
|
@@ -14,7 +29,11 @@ export function modulesCatalogDir(home = null, options = {}) {
|
|
|
14
29
|
* Default global AnchorDB model name for an ecosystem.
|
|
15
30
|
*/
|
|
16
31
|
export function defaultEcosystemModel(ecosystem) {
|
|
17
|
-
|
|
32
|
+
const eco = String(ecosystem || '').toLowerCase()
|
|
33
|
+
if (!SUPPORTED_ECOSYSTEMS.has(eco)) {
|
|
34
|
+
throw new Error(`Unsupported ecosystem "${ecosystem}". Supported: ${[...SUPPORTED_ECOSYSTEMS].join(', ')}`)
|
|
35
|
+
}
|
|
36
|
+
return `modules-${eco}`
|
|
18
37
|
}
|
|
19
38
|
|
|
20
39
|
export function formatModuleKey(pkgName, version) {
|
|
@@ -33,8 +52,10 @@ export function formatExternalSymbolNodeId(ecosystem, pkgName, version, symbol)
|
|
|
33
52
|
* Read the catalog for a specific ecosystem ('npm', 'go', 'cargo', 'pypi').
|
|
34
53
|
*/
|
|
35
54
|
export async function readModulesCatalog(ecosystem, options = {}) {
|
|
55
|
+
const eco = String(ecosystem || '').toLowerCase()
|
|
56
|
+
if (!SUPPORTED_ECOSYSTEMS.has(eco)) return {}
|
|
36
57
|
const dir = modulesCatalogDir(options.home, options)
|
|
37
|
-
const file = path.join(dir, `${
|
|
58
|
+
const file = path.join(dir, `${eco}.json`)
|
|
38
59
|
try {
|
|
39
60
|
const raw = await fs.readFile(file, 'utf8')
|
|
40
61
|
const parsed = JSON.parse(raw)
|
|
@@ -55,31 +76,32 @@ export async function isModuleSynced(ecosystem, pkgName, version, options = {})
|
|
|
55
76
|
|
|
56
77
|
/**
|
|
57
78
|
* Record discovered modules and symbols to ~/.knlview/modules/<ecosystem>.json.
|
|
79
|
+
* Stores lightweight metadata and symbolCount to avoid bloat in the local JSON catalog.
|
|
58
80
|
*/
|
|
59
81
|
export async function updateModulesCatalog(ecosystem, entries, options = {}) {
|
|
82
|
+
const eco = String(ecosystem || '').toLowerCase()
|
|
83
|
+
if (!SUPPORTED_ECOSYSTEMS.has(eco)) return
|
|
60
84
|
if (!entries || !Object.keys(entries).length) return
|
|
61
85
|
const dir = modulesCatalogDir(options.home, options)
|
|
62
|
-
const file = path.join(dir, `${
|
|
86
|
+
const file = path.join(dir, `${eco}.json`)
|
|
63
87
|
try {
|
|
64
88
|
await fs.mkdir(dir, { recursive: true })
|
|
65
|
-
const existing = await readModulesCatalog(
|
|
89
|
+
const existing = await readModulesCatalog(eco, options)
|
|
66
90
|
for (const [key, item] of Object.entries(entries)) {
|
|
67
|
-
const prev = existing[key] || {
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
}
|
|
75
|
-
const symSet = new Set([...(prev.symbols || []), ...(item.symbols || [])])
|
|
91
|
+
const prev = existing[key] || {}
|
|
92
|
+
const symbolCount = item.symbolCount !== undefined
|
|
93
|
+
? item.symbolCount
|
|
94
|
+
: Array.isArray(item.symbols)
|
|
95
|
+
? item.symbols.length
|
|
96
|
+
: (prev.symbolCount || 0)
|
|
97
|
+
|
|
76
98
|
existing[key] = {
|
|
77
99
|
name: item.name,
|
|
78
100
|
version: item.version,
|
|
79
|
-
ecosystem,
|
|
80
|
-
model: item.model || prev.model || defaultEcosystemModel(
|
|
81
|
-
nodeId: item.nodeId || prev.nodeId || formatExternalPackageNodeId(
|
|
82
|
-
|
|
101
|
+
ecosystem: eco,
|
|
102
|
+
model: item.model || prev.model || defaultEcosystemModel(eco),
|
|
103
|
+
nodeId: item.nodeId || prev.nodeId || formatExternalPackageNodeId(eco, item.name, item.version),
|
|
104
|
+
symbolCount,
|
|
83
105
|
syncedAt: item.syncedAt || prev.syncedAt || Date.now()
|
|
84
106
|
}
|
|
85
107
|
}
|
|
@@ -183,22 +205,27 @@ export async function detectProjectManifests(rootAbs) {
|
|
|
183
205
|
* and Python (venv site-packages).
|
|
184
206
|
*/
|
|
185
207
|
export async function resolveModuleLocation(ecosystem, moduleName, options = {}) {
|
|
208
|
+
const eco = String(ecosystem || '').toLowerCase()
|
|
209
|
+
if (!SUPPORTED_ECOSYSTEMS.has(eco) || !isValidModuleName(moduleName)) return null
|
|
186
210
|
const root = path.resolve(options.projectRoot || process.cwd())
|
|
187
211
|
const manifests = options.manifests || await detectProjectManifests(root)
|
|
188
212
|
|
|
189
|
-
if (
|
|
213
|
+
if (eco === 'npm') {
|
|
190
214
|
// 1. Check node_modules in projectRoot and walk up if necessary
|
|
191
215
|
let curr = root
|
|
192
216
|
let foundDir = null
|
|
193
217
|
while (true) {
|
|
194
|
-
const
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
218
|
+
const baseModules = path.join(curr, 'node_modules')
|
|
219
|
+
const candidate = path.resolve(baseModules, moduleName)
|
|
220
|
+
if (candidate.startsWith(baseModules + path.sep)) {
|
|
221
|
+
try {
|
|
222
|
+
const stat = await fs.stat(candidate)
|
|
223
|
+
if (stat.isDirectory()) {
|
|
224
|
+
foundDir = candidate
|
|
225
|
+
break
|
|
226
|
+
}
|
|
227
|
+
} catch {}
|
|
228
|
+
}
|
|
202
229
|
const parent = path.dirname(curr)
|
|
203
230
|
if (parent === curr) break
|
|
204
231
|
curr = parent
|
|
@@ -211,39 +238,48 @@ export async function resolveModuleLocation(ecosystem, moduleName, options = {})
|
|
|
211
238
|
const pkg = JSON.parse(pkgText)
|
|
212
239
|
if (pkg.version) version = pkg.version
|
|
213
240
|
} catch {}
|
|
214
|
-
return { location: foundDir, version, name: moduleName, ecosystem }
|
|
241
|
+
return { location: foundDir, version, name: moduleName, ecosystem: eco }
|
|
215
242
|
}
|
|
216
|
-
} else if (
|
|
243
|
+
} else if (eco === 'go') {
|
|
217
244
|
// 1. Check local vendor
|
|
218
|
-
const
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
245
|
+
const vendorBase = path.join(root, 'vendor')
|
|
246
|
+
const vendorDir = path.resolve(vendorBase, moduleName)
|
|
247
|
+
if (vendorDir.startsWith(vendorBase + path.sep)) {
|
|
248
|
+
try {
|
|
249
|
+
const stat = await fs.stat(vendorDir)
|
|
250
|
+
if (stat.isDirectory()) {
|
|
251
|
+
return { location: vendorDir, version: manifests.go[moduleName] || 'vendor', name: moduleName, ecosystem: eco }
|
|
252
|
+
}
|
|
253
|
+
} catch {}
|
|
254
|
+
}
|
|
225
255
|
|
|
226
256
|
// 2. Check GOPATH
|
|
227
257
|
const gopath = process.env.GOPATH || path.join(options.home || process.env.HOME || '', 'go')
|
|
228
258
|
const version = manifests.go[moduleName]
|
|
229
259
|
if (version) {
|
|
230
|
-
const
|
|
260
|
+
const modBase = path.join(gopath, 'pkg', 'mod')
|
|
261
|
+
const modDir = path.resolve(modBase, `${moduleName}@${version}`)
|
|
262
|
+
if (modDir.startsWith(modBase + path.sep)) {
|
|
263
|
+
try {
|
|
264
|
+
const stat = await fs.stat(modDir)
|
|
265
|
+
if (stat.isDirectory()) {
|
|
266
|
+
return { location: modDir, version, name: moduleName, ecosystem: eco }
|
|
267
|
+
}
|
|
268
|
+
} catch {}
|
|
269
|
+
}
|
|
270
|
+
}
|
|
271
|
+
} else if (eco === 'cargo') {
|
|
272
|
+
// 1. Check local vendor
|
|
273
|
+
const vendorBase = path.join(root, 'vendor')
|
|
274
|
+
const vendorDir = path.resolve(vendorBase, moduleName)
|
|
275
|
+
if (vendorDir.startsWith(vendorBase + path.sep)) {
|
|
231
276
|
try {
|
|
232
|
-
const stat = await fs.stat(
|
|
277
|
+
const stat = await fs.stat(vendorDir)
|
|
233
278
|
if (stat.isDirectory()) {
|
|
234
|
-
return { location:
|
|
279
|
+
return { location: vendorDir, version: manifests.cargo[moduleName] || 'vendor', name: moduleName, ecosystem: eco }
|
|
235
280
|
}
|
|
236
281
|
} catch {}
|
|
237
282
|
}
|
|
238
|
-
} else if (ecosystem === 'cargo') {
|
|
239
|
-
// 1. Check local vendor
|
|
240
|
-
const vendorDir = path.join(root, 'vendor', moduleName)
|
|
241
|
-
try {
|
|
242
|
-
const stat = await fs.stat(vendorDir)
|
|
243
|
-
if (stat.isDirectory()) {
|
|
244
|
-
return { location: vendorDir, version: manifests.cargo[moduleName] || 'vendor', name: moduleName, ecosystem }
|
|
245
|
-
}
|
|
246
|
-
} catch {}
|
|
247
283
|
|
|
248
284
|
// 2. Check cargo registry cache (~/.cargo/registry/src)
|
|
249
285
|
const cargoHome = process.env.CARGO_HOME || path.join(options.home || process.env.HOME || '', '.cargo')
|
|
@@ -257,13 +293,15 @@ export async function resolveModuleLocation(ecosystem, moduleName, options = {})
|
|
|
257
293
|
const entries = await fs.readdir(regDir)
|
|
258
294
|
const match = entries.find((e) => version ? e === targetPrefix : e.startsWith(targetPrefix))
|
|
259
295
|
if (match) {
|
|
260
|
-
const location = path.
|
|
261
|
-
|
|
262
|
-
|
|
296
|
+
const location = path.resolve(regDir, match)
|
|
297
|
+
if (location.startsWith(regDir + path.sep)) {
|
|
298
|
+
const matchedVersion = version || match.slice(moduleName.length + 1)
|
|
299
|
+
return { location, version: matchedVersion, name: moduleName, ecosystem: eco }
|
|
300
|
+
}
|
|
263
301
|
}
|
|
264
302
|
}
|
|
265
303
|
} catch {}
|
|
266
|
-
} else if (
|
|
304
|
+
} else if (eco === 'pypi') {
|
|
267
305
|
const version = manifests.pypi[moduleName.toLowerCase()] || 'unknown'
|
|
268
306
|
// 1. Check .venv and venv site-packages
|
|
269
307
|
const venvDirs = ['.venv', 'venv']
|
|
@@ -272,13 +310,16 @@ export async function resolveModuleLocation(ecosystem, moduleName, options = {})
|
|
|
272
310
|
try {
|
|
273
311
|
const pyDirs = await fs.readdir(libDir)
|
|
274
312
|
for (const py of pyDirs) {
|
|
275
|
-
const
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
313
|
+
const spBase = path.join(libDir, py, 'site-packages')
|
|
314
|
+
const sp = path.resolve(spBase, moduleName)
|
|
315
|
+
if (sp.startsWith(spBase + path.sep)) {
|
|
316
|
+
try {
|
|
317
|
+
const stat = await fs.stat(sp)
|
|
318
|
+
if (stat.isDirectory()) {
|
|
319
|
+
return { location: sp, version, name: moduleName, ecosystem: eco }
|
|
320
|
+
}
|
|
321
|
+
} catch {}
|
|
322
|
+
}
|
|
282
323
|
}
|
|
283
324
|
} catch {}
|
|
284
325
|
}
|
package/server/graph/sources.js
CHANGED
|
@@ -27,8 +27,7 @@ import { createPathMatcher } from '../config.js'
|
|
|
27
27
|
|
|
28
28
|
export const SOURCE_RELATIONS = ['imports', 'imported_by']
|
|
29
29
|
export const SOURCE_TOMBSTONE = 2
|
|
30
|
-
// OpStatus on the wire:
|
|
31
|
-
export const OP_ACTIVE = 1
|
|
30
|
+
// OpStatus on the wire: 2 = Archived (TOMBSTONE).
|
|
32
31
|
export const OP_ARCHIVED = 2
|
|
33
32
|
// Symbol layer, same AnchorDB model as the files: a file defines symbols,
|
|
34
33
|
// symbols call symbols. Mirrored both directions like imports, for the same
|
package/server/scaffold.js
CHANGED
|
@@ -90,9 +90,10 @@ export function buildAgentInstructionBlock(hasGraphify = false) {
|
|
|
90
90
|
|
|
91
91
|
return `<!-- knl-view:start -->
|
|
92
92
|
# Knowledge graph tools
|
|
93
|
-
- KNL-View is the primary graph tool. For \`.knowledges\`, source graphs, and project graph questions, read the project's \`knl-view\` skill and use the relevant KNL-View feature first.
|
|
93
|
+
- KNL-View is the primary graph tool. For \`.knowledges\`, source graphs, external module graphs, and project graph questions, read the project's \`knl-view\` skill and use the relevant KNL-View feature first.
|
|
94
94
|
${fallback}## KNL-View graph access
|
|
95
|
-
- Use the KNL-View feature that fits the task (\`graph trace\`, \`sources trace/sync/collect\`, \`model check/new\`, consensus commands, or the viewer). Do not approximate a supported KNL-View operation with ad-hoc graph queries.
|
|
95
|
+
- Use the KNL-View feature that fits the task (\`graph trace\`, \`sources trace/sync/collect\`, \`modules build-graph/trace/list\`, \`model check/new\`, consensus commands, or the viewer). Do not approximate a supported KNL-View operation with ad-hoc graph queries.
|
|
96
|
+
- For third-party libraries and dependencies (\`node_modules\`, \`go mod\`, \`site-packages\`, \`crates\`): NEVER read raw library files directly. Build and query the module graph on-demand: run \`knl-view modules build-graph <eco>::<module_name>\` (e.g. \`npm::express\`, \`go::github.com/gin-gonic/gin\`), then inspect API definitions and call hierarchies via \`knl-view modules trace <eco>::<module_name> [symbol]\` to save context tokens.
|
|
96
97
|
- Route all graph reads and writes through KNL-View's documented CLI, viewer, or application endpoints. Never connect to AnchorDB directly through AnchorWire, its CLI, direct database endpoints, files, SQL, or custom clients/scripts. If KNL-View cannot perform a requested operation, explain the limitation; do not bypass it.
|
|
97
98
|
- Keep graph work targeted: specify the relevant node/path, direction, and smallest useful hop count; summarize only relevant nodes and edges; avoid dumping whole graphs or large raw JSON.
|
|
98
99
|
- Follow the KNL-View skill's trace-before-edit and \`sources sync\`-after-change rules. Treat edge direction as lineage; do not claim causality without separate evidence.
|
|
@@ -196,7 +197,7 @@ async function statOrNull(abs) {
|
|
|
196
197
|
// structure.
|
|
197
198
|
const SKILL_MD = `---
|
|
198
199
|
name: knl-view
|
|
199
|
-
description: Read and maintain .knowledges knowledge bases. Use whenever a task involves
|
|
200
|
+
description: Read and maintain .knowledges knowledge bases, project source dependency graphs, and on-demand external module boundary graphs (npm, go, cargo, pypi). Use whenever a task involves project architecture, dependencies, third-party libraries, .knowledges, or the knl-view viewer.
|
|
200
201
|
---
|
|
201
202
|
|
|
202
203
|
# Knowledges base
|