@dzhechkov/skills-book-digitizer 0.2.0 → 0.2.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +80 -2
- package/book-pack-assemble/SKILL.md +19 -2
- package/package.json +3 -2
- package/scripts/book-ingest.mjs +0 -0
- package/source-brain-ingest/SKILL.md +129 -0
package/README.md
CHANGED
|
@@ -12,7 +12,7 @@ Knowledge Base behind them. dz-original; design & rationale in
|
|
|
12
12
|
methods at the decisions it's about — e.g. DDIA → `ddia-replication-topology-choice`,
|
|
13
13
|
`ddia-partitioning-strategy`, `ddia-consistency-model-selection`.
|
|
14
14
|
|
|
15
|
-
## The
|
|
15
|
+
## The skills (a checkpointed pipeline + one sibling)
|
|
16
16
|
|
|
17
17
|
| Skill | Stage |
|
|
18
18
|
|-------|-------|
|
|
@@ -20,8 +20,9 @@ methods at the decisions it's about — e.g. DDIA → `ddia-replication-topology
|
|
|
20
20
|
| `book-ingest` | PDF → chunked, page-anchored corpus + manifest (deterministic: poppler + heuristics, no LLM) |
|
|
21
21
|
| `book-knowledge-extract` | corpus → **verified** Knowledge Units (map-reduce, dedup, KU-verify entailment, FACTS-verbatim/PROSE-paraphrase regime) |
|
|
22
22
|
| `book-skill-distill` | verified KUs → **decision-moment** skills (slug-prefixed, routing evals + activation gate, KU→skill traceability) |
|
|
23
|
-
| `book-pack-assemble` | skills → installable pack (`references/` deep-lookup shipped inside, **private-by-default** distribution, collision gate, Copilot policy) |
|
|
23
|
+
| `book-pack-assemble` | skills → installable pack (`references/` deep-lookup + the per-book `brain/<slug>.sqlite` KB slice shipped inside, **private-by-default** distribution, collision gate, Copilot policy) |
|
|
24
24
|
| `book-kb-index` | KUs → the memory layer: `.dz/memory/books.sqlite` (lexical) + shared agentdb (vector) |
|
|
25
|
+
| `source-brain-ingest` | **Sibling (not a book stage, ADR §6)** — deep-walk a **repo/dir** → the *same* Knowledge Units → the *same* brain via `dz brain add --from-kus … --kind repo`. One brain, two ingest front-ends (books + code). |
|
|
25
26
|
|
|
26
27
|
## Install & use
|
|
27
28
|
|
|
@@ -33,6 +34,83 @@ dz init --target claude-code --select digitize-book
|
|
|
33
34
|
Prerequisites: `poppler-utils` (ingest) and, for the KB, `dz setup --memory agentdb`
|
|
34
35
|
(installs `agentdb` + `better-sqlite3`).
|
|
35
36
|
|
|
37
|
+
## Usage scenarios
|
|
38
|
+
|
|
39
|
+
The digitizer produces three durable artifacts from one book: (1) an installable **skill pack**, (2) a
|
|
40
|
+
per-project **two-store KB** (`book-kb-index` → `.dz/memory/books.sqlite` lexical + shared agentdb
|
|
41
|
+
vector), and (3) a promote path into a durable cross-project **Knowledge Brain** (`~/.dz/brain/`).
|
|
42
|
+
Full design + the RU version: [ADR-001 Knowledge Brain](https://github.com/djd1m/dz-harness-hub/blob/main/features/book-brain/ADR-001-knowledge-brain.md).
|
|
43
|
+
|
|
44
|
+
### 1. Digitize a book → skills + KB
|
|
45
|
+
|
|
46
|
+
The core pipeline. Install the orchestrator, then hand it a PDF; it runs `book-ingest → extract →
|
|
47
|
+
distill → assemble → book-kb-index` through 5 human checkpoints.
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
dz init --target claude-code --select digitize-book
|
|
51
|
+
# then, in your agent: «оцифруй вот эту книгу — хочу, чтобы агент проектировал по её методикам»
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Output: a `skills-book-<slug>` pack (decision-moment skills) **and** the two-store KB behind it. Live
|
|
55
|
+
run: DDIA → «Высоконагруженные приложения» digitized to **116 Knowledge Units**.
|
|
56
|
+
|
|
57
|
+
### 2. Query one book's knowledge
|
|
58
|
+
|
|
59
|
+
Lexical (FTS5) recall over a single digitized book's KB, in the project that ingested it. `--book`
|
|
60
|
+
narrows the search to one book slug:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
dz recall --books --book <slug> "replication topology"
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Pull-based: you ask, you get KU hits back with source + page anchors. No hook, no injection.
|
|
67
|
+
|
|
68
|
+
### 3. Build a cross-project brain from N books
|
|
69
|
+
|
|
70
|
+
Promote each project's digitized-book KUs into **one shared brain** (`~/.dz/brain/` — lexical
|
|
71
|
+
`books.sqlite` + vector `agentdb.db`), then query across every ingested book at once:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
dz brain add --source <slug> # promote this project's KUs into the brain (idempotent)
|
|
75
|
+
dz brain list # what's in the brain: slug, kind, KU count, corpus version
|
|
76
|
+
dz brain query "replication topology" # cross-source recall over ALL books
|
|
77
|
+
dz brain query "replication topology" --source <slug> # optional: narrow to one book
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
One shared store, not per-book packages: you do **not** install anything to query the brain — it
|
|
81
|
+
accretes on `dz brain add`. The *right* book surfaces by **relevance** (KUs rank on one common scale,
|
|
82
|
+
each hit labelled with its source book + page); `--source` is a label filter, not a manual routing
|
|
83
|
+
pick. Cross-source is the default — multiple books scoring on a query is a feature (richer grounding),
|
|
84
|
+
not a bug.
|
|
85
|
+
|
|
86
|
+
### 4. Grounded answering (the hook)
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
dz brain init # opt-in: wire the grounding UserPromptSubmit hook (once)
|
|
90
|
+
dz brain ground "<prompt>" # what the hook runs: retrieve + emit citations (or nothing)
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
`dz brain init` (opt-in) wires a single `UserPromptSubmit` hook that runs `dz brain ground` on each
|
|
94
|
+
prompt. On a *relevant* prompt (relevance-gated, so it stays quiet on unrelated turns) the hook
|
|
95
|
+
mechanically retrieves top-K brain KUs and injects them — with source + page citations — into that
|
|
96
|
+
turn's context, so the agent answers **from the book, not from training data**. Off by default; a
|
|
97
|
+
prompt with no lexical overlap (e.g. "what's the weather") injects nothing.
|
|
98
|
+
|
|
99
|
+
Honesty boundary: the injection is **mechanical** (the hook forces the grounding block into context and
|
|
100
|
+
the agent literally cannot start the turn without it); "answering from it" is **agent discipline** — the
|
|
101
|
+
hook can force the context but not the completion. The hook is **generic and wired once** (one hook over
|
|
102
|
+
the whole brain across all sources — *not* per book), and needs **no edits to `CLAUDE.md` or governance
|
|
103
|
+
shards** — the injection channel is the hook's stdout (`hookSpecificOutput.additionalContext`).
|
|
104
|
+
|
|
105
|
+
### 5. Sharing a digitized book
|
|
106
|
+
|
|
107
|
+
The `skills-book-<slug>` **pack** (via GitHub / `npx`) carries the behavior — the decision-moment
|
|
108
|
+
skills and `references/` deep-lookup. The design (ADR §8.1, P2) also ships the book's **KB slice inside
|
|
109
|
+
the pack** under the same CP5 licence decision that gates the pack itself: copyrighted books stay
|
|
110
|
+
`private: true` (structural, mechanically enforced — no auto-publish, no preset entry), permissive
|
|
111
|
+
sources are publishable. Your whole personal **brain is never published** — it is a local accretion you
|
|
112
|
+
back up, not a distributable.
|
|
113
|
+
|
|
36
114
|
## Design guarantees — structural vs. protocol (stated honestly)
|
|
37
115
|
|
|
38
116
|
Two kinds of guarantee, and the difference matters:
|
|
@@ -34,6 +34,7 @@ Distilled skills + their KUs + routing evals from `book-skill-distill`. `dz benc
|
|
|
34
34
|
<slug>-<decision>/SKILL.md
|
|
35
35
|
<slug>-<decision>/references/*.md ← the deep-lookup, shipped IN the pack
|
|
36
36
|
<slug>-<decision>/evals/routing.yaml
|
|
37
|
+
brain/<slug>.sqlite ← the per-book KB slice, shipped IN the pack (step 9)
|
|
37
38
|
package.json sources.json README.md LICENSE
|
|
38
39
|
```
|
|
39
40
|
2. **`references/` ships the deep-lookup inside the pack** — each skill's consumed KUs in full
|
|
@@ -41,8 +42,8 @@ Distilled skills + their KUs + routing evals from `book-skill-distill`. `dz benc
|
|
|
41
42
|
every target/machine, not a call to a KB the recipient doesn't have. `sources.json` declares
|
|
42
43
|
which lookup tiers exist per skill: `references/` = always, corpus = owner-local, agentdb =
|
|
43
44
|
owner-local — the skill degrades honestly.
|
|
44
|
-
3. **`files[]` whitelist** every skill dir + `references`/`evals` + `
|
|
45
|
-
`LICENSE` (forget one → npm silently drops it). Verify the tarball after.
|
|
45
|
+
3. **`files[]` whitelist** every skill dir + `references`/`evals` + `brain/` (the KB slice, step 9) +
|
|
46
|
+
`sources.json` + `README.md` + `LICENSE` (forget one → npm silently drops it). Verify the tarball after.
|
|
46
47
|
4. **`package.json` distribution guardrails (structural, not policy):**
|
|
47
48
|
`"private": true` + `"publishConfig": { "access": "restricted" }`. **No automatic
|
|
48
49
|
`harness-presets` entry** (presets are the public install surface). Flipping to publishable
|
|
@@ -68,6 +69,19 @@ Distilled skills + their KUs + routing evals from `book-skill-distill`. `dz benc
|
|
|
68
69
|
(re-paraphrase) and re-run. Cited ≤25-word quotes with a page anchor are exempt. Only once it
|
|
69
70
|
passes: `dz benchmark <pack> --all` (honest L0 grade per skill — never fabricate schemas/evals to
|
|
70
71
|
game it) → `dz init --target claude-code --select <skills>` install smoke → **CP4** report.
|
|
72
|
+
9. **Ship the per-book KB slice inside the pack (ADR §8.1)** — once skills + `references/` are
|
|
73
|
+
assembled, export the book's KUs as a standalone slice:
|
|
74
|
+
`dz brain export --source <slug> --out brain/<slug>.sqlite` (this book's rows only —
|
|
75
|
+
`WHERE book=<slug>`, **lexical-only** for portability — the recipient's `dz brain add --from-pack`
|
|
76
|
+
re-embeds the vectors into their own brain on import). Whitelist `brain/` in `files[]` (step 3)
|
|
77
|
+
and pass the slice **through the same shingling IP gate** (step 8) as the rest of the pack — a
|
|
78
|
+
verbatim leak in the KB slice fails the pack just like one in a SKILL.md. The slice rides the pack
|
|
79
|
+
under the **same CP5 licence decision**: a copyrighted book stays `private: true` (slice never
|
|
80
|
+
auto-published); a permissive source is publishable (slice ships). A recipient loads it with
|
|
81
|
+
`dz brain add --from-pack @dzhechkov/skills-book-<slug>` — an idempotent upsert on
|
|
82
|
+
`(book, ku_id, corpus_version)` into their `~/.dz/brain/` — so one `npx`/github install carries
|
|
83
|
+
BOTH the behavior (skills) AND the knowledge (the brain slice), inseparably, under one decision.
|
|
84
|
+
Document the `--from-pack` load in the pack README.
|
|
71
85
|
|
|
72
86
|
## Anti-patterns
|
|
73
87
|
|
|
@@ -79,6 +93,8 @@ Distilled skills + their KUs + routing evals from `book-skill-distill`. `dz benc
|
|
|
79
93
|
| `origin` block in sources.json | sync-upstream would chase a non-repo | documentation-only, no origin |
|
|
80
94
|
| Fat SKILL.md bodies on Copilot | floods every request | lean body + `references/` |
|
|
81
95
|
| Shipping a pack without the shingling gate | uncited verbatim runs = IP violation | run `shingling-check.mjs` before the L0 benchmark; re-paraphrase any hit |
|
|
96
|
+
| Publishing the KB slice separately from the pack's CP5 decision | book knowledge leaks out from under the licence gate | slice rides the pack: `private` book → private slice, permissive → publishable |
|
|
97
|
+
| KB slice skipping the shingling gate | a verbatim leak in the slice = same IP violation | pass `brain/<slug>.sqlite` through the shingling gate like the rest of the pack |
|
|
82
98
|
|
|
83
99
|
## Self-check
|
|
84
100
|
|
|
@@ -88,6 +104,7 @@ Distilled skills + their KUs + routing evals from `book-skill-distill`. `dz benc
|
|
|
88
104
|
- [ ] Collision gate clean; slug-prefixed unique ids?
|
|
89
105
|
- [ ] Faithfulness gate run (each shipped skill's claims supported by its `derived_from` KUs)?
|
|
90
106
|
- [ ] Shingling gate passed (no uncited verbatim run ≥8 words; cited ≤25-word quotes anchored)?
|
|
107
|
+
- [ ] KB slice `brain/<slug>.sqlite` exported, whitelisted in `files[]`, passed the shingling gate, and follows the pack's CP5 licence (private book → private slice); `--from-pack` load documented in README?
|
|
91
108
|
- [ ] L0 grades honest; install smoke passed; Copilot bytes shown if relevant?
|
|
92
109
|
|
|
93
110
|
## Examples
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@dzhechkov/skills-book-digitizer",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.4",
|
|
4
4
|
"description": "Canonical book-digitizer skill pack — 6 skills that turn an engineering book/textbook/monograph into an installable skill pack whose skills make AI coders work within the book's methodologies: digitize-book (orchestrator), book-ingest, book-knowledge-extract, book-skill-distill, book-pack-assemble, book-kb-index.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "dzhechko",
|
|
@@ -22,6 +22,7 @@
|
|
|
22
22
|
"book-skill-distill",
|
|
23
23
|
"book-pack-assemble",
|
|
24
24
|
"book-kb-index",
|
|
25
|
+
"source-brain-ingest",
|
|
25
26
|
"scripts",
|
|
26
27
|
"sources.json",
|
|
27
28
|
"README.md",
|
|
@@ -36,4 +37,4 @@
|
|
|
36
37
|
"directory": "packages/@dzhechkov/skills-book-digitizer"
|
|
37
38
|
},
|
|
38
39
|
"homepage": "https://github.com/djd1m/dz-harness-hub/tree/main/packages/@dzhechkov/skills-book-digitizer#readme"
|
|
39
|
-
}
|
|
40
|
+
}
|
package/scripts/book-ingest.mjs
CHANGED
|
File without changes
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: source-brain-ingest
|
|
3
|
+
description: >
|
|
4
|
+
The repo/dir SIBLING of the book pipeline (ADR-001 §6): deep-walk a codebase or directory
|
|
5
|
+
(respect .gitignore, skip binaries/node_modules/dist, cap size), then distill DECISION-GRADE
|
|
6
|
+
Knowledge Units in the SAME BookKU shape the book pipeline emits — and feed them into the SAME
|
|
7
|
+
shared brain register via `dz brain add --from-kus`, WITHOUT the book-only stages (no pdftotext
|
|
8
|
+
ingest, no skill distillation). Same KU schema = free reuse of the whole brain (query, primer,
|
|
9
|
+
grounding). Records the source's SPDX license and gates on it. Triggers on: "ingest a repo into
|
|
10
|
+
the brain", "source-brain-ingest", "add this codebase to my knowledge brain".
|
|
11
|
+
trust_tier: 1
|
|
12
|
+
trust_tier_label: "Structured"
|
|
13
|
+
trust_tier_path: "Run /bto-test to promote to Tier 2"
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# source-brain-ingest — repo/dir → brain KUs (the non-book front-end)
|
|
17
|
+
|
|
18
|
+
One brain, **two ingest front-ends** (ADR §6): the book pipeline turns PDFs into KUs; this turns a
|
|
19
|
+
**repo or directory** into KUs — the *same* KU shape, the *same* register stage, the *same* query /
|
|
20
|
+
primer / grounding surface. A repo plugs into everything the book path already built, for free. This
|
|
21
|
+
is a **sibling** skill — it does not touch, and must not contort, the intact book pipeline.
|
|
22
|
+
|
|
23
|
+
## When to use / NOT
|
|
24
|
+
|
|
25
|
+
- **Use** to distill a codebase/dir into decision-grade Knowledge Units and promote them into the
|
|
26
|
+
durable brain (`~/.dz/brain/`) so `dz brain query` / primers / the grounding hook cover repos too.
|
|
27
|
+
- **NOT for books/PDFs** → use `digitize-book` (pdftotext ingest is book-shaped and would be
|
|
28
|
+
contorted by repo logic; keeping it pure protects the proven book path).
|
|
29
|
+
- **NOT to author skills** → that's `book-skill-distill`. This stage emits KUs into the brain only;
|
|
30
|
+
it does not produce a `skills-<slug>` pack (repos may later return whole files, §8/G5, where books
|
|
31
|
+
return distilled KUs for IP reasons).
|
|
32
|
+
- **NOT to extract/verify book KUs** → that's `book-knowledge-extract`.
|
|
33
|
+
|
|
34
|
+
## Prerequisites
|
|
35
|
+
|
|
36
|
+
- A local repo or directory (a checkout, or a `git clone` you control).
|
|
37
|
+
- `git` (to honour `.gitignore` and capture the commit for provenance) — degrade gracefully on a
|
|
38
|
+
non-git dir (walk the tree; `commit: null`).
|
|
39
|
+
- The brain stores (`dz setup --memory agentdb`) — the same `~/.dz/brain/` the book path promotes into.
|
|
40
|
+
- The source's **SPDX license identifier** (read `LICENSE`/`package.json`; pass `--license <spdx>`).
|
|
41
|
+
|
|
42
|
+
## The Knowledge Unit schema (identical to the book pipeline — that's the whole point)
|
|
43
|
+
|
|
44
|
+
Emit the **same BookKU shape** `book-knowledge-extract` emits, so a repo is just another `source`:
|
|
45
|
+
|
|
46
|
+
```yaml
|
|
47
|
+
id: <repo-slug>-<path-hash>-kuNN # GLOBALLY UNIQUE: <path-hash> discriminates the file/chunk
|
|
48
|
+
# (short hash of the repo-relative path) so two files never
|
|
49
|
+
# collide on kuNN — the brain upsert keys on (source, ku_id,
|
|
50
|
+
# corpus_version) and silently drops id collisions otherwise.
|
|
51
|
+
type: methodology | decision-framework | formula | heuristic | checklist |
|
|
52
|
+
tradeoff-table | case-pattern | definition
|
|
53
|
+
name: "Retry-with-jitter backoff policy"
|
|
54
|
+
problem: the decision/task this knowledge serves (when/why it applies)
|
|
55
|
+
content: the paraphrased procedure / framework / restructured table / verbatim signature-or-formula
|
|
56
|
+
applicability: preconditions, scale ranges, contexts where it holds
|
|
57
|
+
limits: tradeoffs, failure modes, when NOT to apply
|
|
58
|
+
chapter: src/net/backoff.ts # PATH takes the book's `chapter` slot
|
|
59
|
+
pages: [40, 88] # LINE-RANGE takes the book's `pages` slot
|
|
60
|
+
sources: [{ repo: <repo-slug>, path: src/net/backoff.ts, lines: [40, 88], commit: <sha> }]
|
|
61
|
+
metadata: { license: <spdx>, corpus_version: <sha-or-ts>, lang: <detected> }
|
|
62
|
+
verified: true | false | partial # entailment against the cited lines
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Because the shape is identical, the brain treats books and repos on one common scale (query, primer,
|
|
66
|
+
grounding all work unchanged). `kind: repo` in the registry lets the brain treat repos **distinctly**
|
|
67
|
+
where it matters (e.g. return whole files, §8/G5) without a second schema.
|
|
68
|
+
|
|
69
|
+
## Protocol
|
|
70
|
+
|
|
71
|
+
1. **Walk** — deep-walk the repo/dir. **Respect `.gitignore`** (use `git ls-files` when it's a git
|
|
72
|
+
repo — free `.gitignore` honouring + tracked-only). **Skip** `node_modules/`, `dist/`, `build/`,
|
|
73
|
+
`.git/`, vendored trees, lockfiles, and **binaries** (detect by extension + a NUL-byte sniff).
|
|
74
|
+
**Cap** per-file size (skip/point at oversized generated files) and total bytes scanned; log what
|
|
75
|
+
was skipped so coverage is honest.
|
|
76
|
+
2. **Chunk** — split each kept file into page-analogue units (a function/class/section, or a
|
|
77
|
+
line-window for prose docs). Anchor every chunk to `path` + `[startLine–endLine]` — the analogue
|
|
78
|
+
of the book's `[p.N]` anchor; these anchors make the verify step a cheap lookup.
|
|
79
|
+
3. **Distill KUs** — emit **decision-grade** Knowledge Units in the schema above (a decision the code
|
|
80
|
+
encodes, a policy, a tradeoff, a reusable pattern) — NOT a file summary. **Fidelity regime** (same
|
|
81
|
+
as the book path): FACTS (signatures, constants, config values, exact code) **verbatim** with a
|
|
82
|
+
`path:line` anchor; PROSE (rationale, README narrative) **paraphrased** — no uncited verbatim run
|
|
83
|
+
≥ 8 words (this is what a downstream shingling gate checks). Then a fresh checker re-opens each
|
|
84
|
+
KU's cited `path:line` range and sets `verified` (entailment gate; verify all, lookup ≪ generation).
|
|
85
|
+
4. **License-check (the IP gate)** — record the source's SPDX license in every KU's `metadata.license`
|
|
86
|
+
and in what you pass to the register. The register **refuses** an unknown / incompatible license
|
|
87
|
+
without an explicit `--override` — so an unlicensed or GPL-incompatible repo cannot silently land
|
|
88
|
+
in the brain. Repo license travels with the KUs; if the brain ever returns whole files (§8/G5),
|
|
89
|
+
that license binds any downstream reuse.
|
|
90
|
+
5. **Promote** — write the verified KUs to a JSON file in the BookKU shape and hand them to the SAME
|
|
91
|
+
register the book path uses:
|
|
92
|
+
```bash
|
|
93
|
+
dz brain add --from-kus <file.json> --slug <repo-slug> --kind repo --license <spdx>
|
|
94
|
+
```
|
|
95
|
+
This idempotently upserts on `(source, ku_id, corpus_version)` into `~/.dz/brain/` (lexical
|
|
96
|
+
`books.sqlite` + vector `agentdb.db`), writes the `brain.json` registry entry (`kind: repo`,
|
|
97
|
+
license, corpus_version, ku_count), and generates the primer — **no book-ingest, no
|
|
98
|
+
book-skill-distill in the path**. Verify: `dz brain query "<a KU name>"` returns it labeled with
|
|
99
|
+
the repo source.
|
|
100
|
+
|
|
101
|
+
## Anti-patterns
|
|
102
|
+
|
|
103
|
+
| Anti-pattern | Why it fails | Instead |
|
|
104
|
+
|--------------|--------------|---------|
|
|
105
|
+
| Inventing a repo-specific KU schema | breaks the shared brain (query/primer/grounding assume one shape) | emit the exact BookKU schema — path→chapter, lines→pages |
|
|
106
|
+
| Ignoring `.gitignore` / walking `node_modules` | ingests junk + secrets, blows the token budget | `git ls-files`; skip binaries/vendored/dist; cap size |
|
|
107
|
+
| `kuNN` id without a path discriminator | two files collide → upsert-by-id drops KUs | `<repo-slug>-<path-hash>-kuNN` |
|
|
108
|
+
| Routing a repo through `book-ingest`/`digitize-book` | pdftotext + book stages don't fit code | this sibling → `dz brain add --from-kus` |
|
|
109
|
+
| Distilling repo KUs into a skill pack | that's the book path's job; repos may return whole files (§8) | KUs into the brain only |
|
|
110
|
+
| Promoting an unknown/incompatible license | leaks IP-encumbered code into the durable brain | license-check gate; `--override` is an explicit decision |
|
|
111
|
+
| File-summary "KUs" | RAG noise, not decisions | decision-grade KUs (policy/tradeoff/pattern) |
|
|
112
|
+
|
|
113
|
+
## Self-check
|
|
114
|
+
|
|
115
|
+
- [ ] Walk respected `.gitignore`, skipped binaries/`node_modules`/`dist`, capped size — skips logged?
|
|
116
|
+
- [ ] Every KU in the BookKU schema (id `<repo-slug>-<path-hash>-kuNN`; path→chapter; lines→pages; `sources` carry repo+path+commit)?
|
|
117
|
+
- [ ] FACTS verbatim + `path:line`, PROSE paraphrased (no uncited ≥8-word run); all KUs entailment-verified?
|
|
118
|
+
- [ ] Source SPDX license recorded in `metadata.license`; license-check gate passed (or explicit `--override`)?
|
|
119
|
+
- [ ] Promoted via `dz brain add --from-kus … --kind repo --license <spdx>` (NOT book-ingest/distill); registry shows `kind: repo`; `dz brain query` returns a KU labeled with the repo?
|
|
120
|
+
|
|
121
|
+
## Examples
|
|
122
|
+
|
|
123
|
+
- «Add this backoff library to my brain» → deep-walk → KUs like `retry-lib-a1b2c3-ku02`
|
|
124
|
+
(decision-framework «Retry-with-jitter backoff policy», `src/net/backoff.ts:40–88`, verified,
|
|
125
|
+
license MIT) → `dz brain add --from-kus retry-lib.kus.json --slug retry-lib --kind repo --license MIT`.
|
|
126
|
+
- «Почему этот репозиторий не попал в brain?» → license unknown → the register refused it; re-run with
|
|
127
|
+
a recorded `--override` decision, or add a `LICENSE`.
|
|
128
|
+
- «Ground my design questions on our infra repo AND the DDIA book» → both are just `source`s in the one
|
|
129
|
+
brain; `dz brain query "replication topology"` ranks KUs from both, each labeled with its source.
|