@msareen/knowledge-hub-builder 0.1.7 → 0.2.0

package/.bundle_template/sources.yaml

@@ -4,6 +4,7 @@
 sources: []
 # - type: folder
 #   path: /abs/path/to/some-folder
+#   exclude: []          # optional — skip these paths/globs before ingesting
 # - type: web
 #   urls: []
 # - type: confluence

package/AGENTS.md

@@ -73,8 +73,10 @@ One hard boundary governs every workflow and every future change to the tooling:
 
 - **`khb` converts bytes to text.** Hashing, caching, file plumbing, ledger-keeping, and
   *every* local extractor: PDF/DOCX/ODT/XLSX/PPTX libraries, tesseract OCR for scans and
-  images, whisper for audio and video. All of it deterministic, offline, and free of
-  charge. It **never contacts a model** — not directly, not by shelling out.
+  images, whisper.cpp (via `vno`) or whisper for audio and video, and the caption reader
+  that spares them the job whenever a `.vtt`/`.srt` sits beside the recording. All of it
+  deterministic, offline, and free of charge. It **never contacts a model** — not directly,
+  not by shelling out.
 - **The agent decides what the text means.** Splitting a document into concepts, labeling
   and linking them, curating `raw/` into the wiki, escalating a bad OCR to a vision read,
   and judging when a query has produced a new concept worth keeping.
@@ -136,11 +138,27 @@ From outside, pass `--hub <dir>` or set `$KHB_HUB`.
 |---|---|
 | `khb lint` | validate structure against `skills/lint/SKILL.md` |
 | `khb upgrade` | refresh this hub's package-owned contract docs — runs by itself whenever `khb.json`'s stamped version differs from the installed khb, so these docs always match the CLI |
-| `khb visualize` | regenerate `visualizer/graph.html` |
+| `khb visualize [--port N] [--no-open]` | serve the live bundle graph in your browser; aliases `vis`, `viz` |
 | `khb new-bundle <name>` | scaffold + register a bundle |
 | `khb ingest <bundle>` | acquire + extract every source in `sources.yaml` → `raw/`; maintains `log.md`. Name the bundle — with none it lists the hub's bundles and stops, unless there is nothing to choose between (no bundles, or only `default`), where it uses `default` |
 | `khb export <bundle> [dest]` | standalone copy: bundle + common patterns, shareable alone |
 
+These five work **outside** any hub, against a per-machine shortcut list at
+`~/.khb/hubs-config.json` (`%USERPROFILE%\.khb` on Windows). They hold no knowledge — only
+paths — and fill themselves in as hubs are used, so they need no maintenance.
+
+| Command | Purpose |
+|---|---|
+| `khb list` | every hub on this machine, with its description and path |
+| `khb go [name\|N]` | open a hub — prints its path, then launches your agent there. Bare `khb` is this: one hub asks, several offer a list, none walks the user through creating the first. `--path` prints only the path |
+| `khb agent [name]` | which agent `khb go` launches — `claude`, `codex`, a custom `--command`, or `none` |
+| `khb update [new-path]` | repair the hub: `--path`/`-p` repoints the list and rewrites every old-path reference after a move (needs no arguments — the move is detected and announced by any khb command run there); `--schema`/`-s` backfills a bundle's `sources.yaml` to the current schema (e.g. a newly added `exclude:` field). No flag runs both. `--dry-run` first. Unrelated to `khb upgrade`, which only touches package-owned contract docs, never bundle content |
+| `khb forget <name>` | drop a shortcut; the hub folder is untouched |
+
+A hub's `name` and `description` in that list come from its own `khb.json`, so they travel
+with it. Set them at creation with `khb init --name --description`, or edit the marker's
+`name`/`description` keys — those two are yours, unlike the rest of the file.
+
 There is no `khb catalog` command — cataloging is entirely a judgement pass.
 
 ## Ingest, then catalog — two steps, in that order
@@ -148,8 +166,10 @@ There is no `khb catalog` command — cataloging is entirely a judgement pass.
 **Ingest** (`skills/ingest/SKILL.md`) is mechanical and flat: `khb ingest <bundle>` pulls
 every declared source into `raw/` as markdown with a provenance header, extracting
 everything it can locally — text, PDF, DOCX, ODT, XLSX, PPTX, images by OCR, audio and
-video by whisper. Sources behind an authenticated API (Confluence, ADO, git hosts) you pull
-yourself via MCP/CLI into the same `raw/` shape. Ingest never interprets content.
+video by whisper — or, where a recording has a `.vtt`/`.srt` beside it, from those captions,
+the pair acquired as a single source. Sources behind an authenticated API (Confluence, ADO,
+git hosts) you pull yourself via MCP/CLI into the same `raw/` shape. Ingest never interprets
+content.
 
 When the user has not named a bundle, **ask** which existing bundle owns the material or
 whether to start a new one, and for an existing bundle ask whether to re-ingest what its

package/README.md

@@ -27,8 +27,17 @@ KHB requires [Bun](https://bun.sh).
 
 ```bash
 bun install -g @msareen/knowledge-hub-builder
+khb
+```
+
+On a machine with no hubs yet, `khb` walks you through the first one — where it lives, what
+to call it, which agent opens it, and a first bundle. Every question has a default, so
+holding Enter produces a working hub.
+
+To do the same by hand:
 
-khb init ~/my-knowledge
+```bash
+khb init ~/my-knowledge --description "Personal knowledge"
 cd ~/my-knowledge
 
 khb new-bundle personal "My accounts, plans, records, and reference material"
@@ -40,6 +49,7 @@ Add sources to `bundles/personal/sources.yaml`:
 sources:
   - type: folder
     path: /absolute/path/to/documents
+    exclude: [drafts/]      # optional — skip paths/globs before ingesting
   - type: files
     paths:
       - /absolute/path/to/one.pdf
@@ -62,6 +72,9 @@ or ask the agent:
 The agent discovers the KHB workflow skills from the hub and performs the catalog pass
 after ingestion.
 
+Later, from any terminal, `khb` on its own takes you back to the hub and starts your agent
+there — see [Moving between hubs](#moving-between-hubs).
+
 ## How It Works
 
 ### 1. Ingest
@@ -75,6 +88,8 @@ Supported out of the box, with no system tools and no further installation:
 - PDF and DOCX
 - ODT, XLSX, and PPTX
 - OCR for images and scanned PDFs, applied automatically when a PDF has no text layer
+- subtitle files (`.vtt`, `.srt`) — and a video or audio file with one beside it is read
+  from the captions rather than transcribed, as one source rather than two
 
 OCR ships with KHB: `@hyzyla/pdfium`, `sharp`, and `tesseract.js` are ordinary dependencies,
 so installing KHB pulls them down whether or not you ever ingest a scan. That costs roughly
@@ -83,8 +98,14 @@ waiting for a setup step.
 
 One extractor is genuinely optional:
 
-- audio and video transcription needs a `whisper` or `faster-whisper` executable on `PATH`
-  (`pip install -U openai-whisper`)
+- audio and video transcription needs a transcriber on `PATH`. KHB prefers
+  [`vno`](https://www.npmjs.com/package/@msareen/voice-notes-organizer)
+  (`npm install -g @msareen/voice-notes-organizer`), which wraps whisper.cpp and installs
+  its own ffmpeg and model, and falls back to `whisper` / `faster-whisper`
+  (`pip install -U openai-whisper`). KHB checks `vno status` first: a vno that is installed
+  but not yet set up leaves recordings pending with `run: vno setup` and holds up nothing
+  else in the run. A recording that already has a `.vtt` or `.srt` beside
+  it needs neither — those captions are read instead
 
 Without it, KHB leaves a pending row in `log.md` and prints the required setup rather than
 failing the run.
@@ -225,10 +246,10 @@ Commands can be run directly or requested through the matching agent skill.
 
 | Command | Purpose |
 |---|---|
-| `khb init [dir]` | Create a hub |
+| `khb init [dir] [--name N] [--description "…"]` | Create a hub |
 | `khb upgrade` | Refresh package-owned contracts and skills (also runs automatically on version drift) |
 | `khb new-bundle <name> ["scope"]` | Create and register a bundle |
-| `khb ingest [bundle] [--force]` | Acquire and extract declared sources |
+| `khb ingest [bundle] [--force] [--skip-ocr] [--skip-audio]` | Acquire and extract declared sources |
 | `khb lint` | Validate routing, bundle structure, and OKF metadata |
 | `khb visualize [--port N] [--no-open]` (aliases: `vis`, `viz`) | Serve the live bundle graph on a random free port and open it in your default browser — pan/zoomable cross-bundle map, drill into a bundle for its folder-clustered concepts, rebuild-on-refresh, exits when you close the tab |
 | `khb export <bundle> [dest]` | Export one standalone bundle |
@@ -240,6 +261,84 @@ Additional ingest flags:
 
 Run against a hub outside the current directory with `--hub <dir>` or `$KHB_HUB`.
 
+### Moving between hubs
+
+These four run **outside** any hub, from any terminal:
+
+| Command | Purpose |
+|---|---|
+| `khb` | Open a hub. One registered hub asks; several show a list and take a pick; none walks you through creating the first |
+| `khb list [--json]` | Every hub on this machine, with its description and path |
+| `khb go <name\|N> [--path]` | Open one by name or list position. `--path` prints just the path, for `cd "$(khb go --path work)"` |
+| `khb agent [name] [--command X] [--args "…"]` | Which agent `khb go` launches — `claude`, `codex`, anything on your PATH, or `none` to just print the path |
+| `khb update [new-path] [--path\|-p] [--schema\|-s] [--from <old>] [--dry-run]` | Repair the hub: path references after a move, and/or backfill `sources.yaml` to the current schema. No flag runs both |
+| `khb forget <name>` | Drop a hub from the list. The folder is untouched |
+
+`khb go` prints the hub's path and then starts your agent there, so a bare `khb` from a
+cold terminal ends with an agent open on the right folder. No program can change its
+parent shell's directory, which is why the `cd` line is printed rather than performed.
+
+The list lives in `~/.khb/hubs-config.json` (`%USERPROFILE%\.khb` on Windows). It holds
+paths and one launch command — no knowledge — and fills itself in: any khb command run
+inside a hub registers it, so hubs you already had show up without a migration step.
+Delete the file and the next command in each hub puts it back.
+
+A hub's name and description come from its own `khb.json`, so they travel with the hub
+rather than living in one machine's list. Set them with `khb init --name --description`,
+or edit those two keys in the marker later.
+
+### `khb update` — repairing a hub
+
+Two independent repairs, run together or apart:
+
+```bash
+khb update --dry-run    # what would change, both halves
+khb update               # apply both
+khb update --path        # only repoint paths after a move
+khb update --schema      # only backfill sources.yaml to the current schema
+```
+
+**`--path`/`-p`** — after you move the folder, this repoints the shortcut list (a moved hub
+shows as `MISSING` in `khb list` until you do) and rewrites the absolute paths recorded
+*inside* the hub that named the old location: `sources.yaml` entries, `source:` headers in
+`raw/`, `log.md` rows, `resource:` front matter. It needs no old path: the hub records where
+it lives in its own `khb.json` (`path`), so any khb command run in a moved hub notices and
+says so —
+
+```
+khb: this hub was at D:\kb\old and is now at D:\kb\new.
+khb:   repair them:  khb update --path            (--dry-run to preview)
+```
+
+— and `--path` reads the old location straight out of the marker. That record travels with
+the folder, so it works after `~/.khb` is deleted, on a second machine, or on a hub a
+colleague handed you. Move a hub twice before repairing it and both former homes are
+rewritten in one pass. For a hub last touched by a khb too old to have recorded a location,
+the old path is worked out from the registry entry that now points at nothing, matched on
+the identity stamp in `khb.json`. If it can't prove which entry that is, it asks; `--from
+<old-path>` tells it outright.
+
+The rewrite is a literal substitution with three guards: every spelling of the path moves
+(native, forward-slashed, and JSON-escaped as `raw/` headers store it), each rewritten to
+the same spelling of the new path; matches must end at a path boundary, so moving `…/old`
+never touches a sibling `…/older`; and the new path is matched too and rewritten to itself,
+which is what makes overlapping moves safe — lifting a hub out of its parent, or pushing it
+down into a subdirectory of where it stood — and makes a second run a no-op.
+`.git/`, `node_modules/` and the `inbox/` cache are skipped, as are binaries.
+
+**`--schema`/`-s`** — a bundle's `sources.yaml` can predate a field khb's since learned about
+(e.g. `exclude:`), with no way to discover it short of reading the docs. This backfills
+missing optional fields with their default, per bundle, preserving your comments and
+formatting. `khb upgrade` mentions when either half of `update` has something pending, as a
+printed hint — it never runs `update` for you.
+
+Both `khb update` and `khb ingest` state their whole plan before writing anything, then
+report as they go — a position per file for work slow enough to matter, a live counter for
+work that isn't. The counter goes to stderr, so redirected stdout stays clean.
+
+Not to be confused with `khb upgrade`, which refreshes a hub's package-owned contract docs
+and has nothing to do with moving anything.
+
 To update the installed package:
 
 ```bash

package/SPEC.md

@@ -45,7 +45,8 @@ KHB's own contribution is the bundle-of-bundles layer over both — see §1.
    every concept traces back through a provenance header to the original file.
 6. **Extract to markdown, locally** — binary/opaque formats (PDF, DOCX, XLSX, images,
    audio) are converted to markdown so every bundle's knowledge is plain, greppable text.
-   Every extractor is local and deterministic: pure-JS libraries, tesseract WASM, whisper.
+   Every extractor is local and deterministic: pure-JS libraries, tesseract WASM,
+   whisper.cpp.
    Lossy routes (OCR, ASR) are marked `quality: low` rather than hidden.
 7. **Bun** is the scripting language for all tooling and third-party interfaces.
 
@@ -115,6 +116,8 @@ acting on them** — an agent can never read a protocol the CLI no longer implem
 ├── scripts/
 │   ├── cli.ts                 # subcommand dispatch; --hub flag; the version drift check
 │   ├── init.ts                # khb init / khb upgrade
+│   ├── hubs.ts                # khb list / go / agent / update / forget — the only
+│   │                          #   commands that run outside a hub
 │   ├── new-bundle.ts          # scaffold from .bundle_template, register in outer.index.md
 │   ├── export.ts              # bundle + common patterns → standalone shareable folder
 │   ├── lint.ts                # enforce skills/lint/SKILL.md across the hub
@@ -124,6 +127,9 @@ acting on them** — an agent can never read a protocol the CLI no longer implem
 │       ├── extract.ts         # every local extractor + the content-hash cache
 │       ├── ledger.ts          # log.md read/write
 │       ├── paths.ts           # package-side paths — importing it never needs a hub
+│       ├── registry.ts        # ~/.khb/hubs-config.json: where this machine's hubs are
+│       ├── relocate.ts        # khb update --path's path rewriter — pure text, no judgement
+│       ├── schema.ts          # khb update --schema: sources.yaml field diff/apply
 │       ├── upgrade.ts         # the refresh itself: `khb upgrade` and the drift check
 │       └── util.ts            # hub resolution + shared helpers
 ├── .bundle_template/          # copied by `khb new-bundle`
@@ -143,6 +149,210 @@ Rule 3 is the normal path: `cd` anywhere inside the hub and run `khb lint`. One
 consequence worth stating — a hub is identified by its marker file, not its name, so
 hubs may be renamed or moved freely, and nested hubs resolve to the innermost one.
 
+### 2d. The machine registry — `~/.khb/hubs-config.json`
+
+Rules 1–3 all assume you already know where the hub is. From a cold terminal in an
+unrelated folder, you don't — and a person with a personal hub, a work hub and a client
+hub has three paths to remember. So khb keeps **one file per machine** recording where the
+hubs on it are:
+
+```
+~/.khb/                        # %USERPROFILE%\.khb on Windows; $KHB_HOME overrides
+└── hubs-config.json
+```
+
+```json
+{
+  "version": 1,
+  "defaultAgent": "claude",
+  "agents": { "claude": { "command": "claude", "args": [] },
+              "codex":  { "command": "codex",  "args": [] } },
+  "hubs": [
+    { "name": "my-knowledge", "description": "Personal — work + home",
+      "path": "D:\\code\\my-knowledge", "added": "…", "lastUsed": "…" }
+  ]
+}
+```
+
+Three properties define it:
+
+- **It is a shortcut list, never knowledge.** It holds paths and one launch command.
+  Delete it and nothing is lost — the next command run inside each hub puts it back.
+- **It fills itself in.** Every khb command that resolves a hub registers it, so hubs made
+  before the registry existed appear the first time anything is run in them. There is no
+  migration and no `register` command to remember. `khb forget <name>` drops a shortcut and
+  never touches the folder.
+- **The hub is the authority on its own identity.** `name` and `description` are read out
+  of the hub's `khb.json` (`khb init --name --description`, or edit the file), so a hub
+  moved to another machine or cloned by a colleague describes itself the same way there.
+  Only when the marker says nothing does khb fall back to the folder name and a summary of
+  the bundles inside. `khb upgrade` merges rather than replaces the marker, so keys khb
+  does not own survive an upgrade.
+
+The commands over it are `khb list`, `khb go`, `khb agent`, `khb update` and `khb forget` —
+the only ones that run **outside** a hub, and therefore the only ones that skip hub
+resolution and the version drift check. A bare `khb` is `khb go`: one hub asks to open it,
+several show the list and take a pick, none prints the help.
+
+`khb go` ends by launching your configured agent with the hub as its working directory.
+No process can change its parent shell's directory, so `khb go` prints the `cd` line for
+the human and passes the path to the agent as cwd — `khb go --path <name>` prints only the
+path, for `cd "$(khb go --path work)"`. `khb agent none` turns the launch off entirely.
+
+Every path stored in or compared against the registry is canonicalized first (`realpath`,
+case-folded on Windows). One directory has several true names — `C:\Users\MANASV~1\…` and
+`C:\Users\Manasvi Sareen\…` are the same folder, as is anything reached through a symlink —
+and without this a single hub lists twice and `khb update --path` cannot tell it has already
+been repaired.
+
+### 2e. `khb update` — repairing a hub
+
+Two independent repairs, selectable together or apart:
+
+```
+khb update [new-path] [--path|-p] [--schema|-s] [--from <old-path>] [--dry-run]
+```
+
+No flag runs both. `--path`/`-p` repairs a moved hub; `--schema`/`-s` backfills a bundle's
+`sources.yaml` to the current field set (§ scripts/lib/schema.ts). They are unrelated repairs
+that happen to share a plan-then-write-then-report shape, so one command with two switches
+beat two commands that would each need the same hub-resolution and dry-run plumbing.
+
+#### `--path` — a hub that moved
+
+Moving a hub folder breaks two things, and this half fixes both:
+
+1. **The shortcut list** still points at the old folder. `khb list` marks it `MISSING`.
+2. **Absolute paths recorded inside the hub** that named the old location — `sources.yaml`
+   entries, `source:` headers in `raw/`, `log.md` rows, `resource:` front matter — are now
+   dangling.
+
+Run from inside the moved hub, or name it. Nothing else is needed, because **the hub records
+its own location**. `khb.json` carries a `path` key — canonical, plus `pathAs` for the
+spelling khb was invoked through when the two differ — and any khb command run in a hub
+compares it against where the hub actually is. A mismatch is the move, stated by the only
+witness that travelled with the folder:
+
+```
+khb: this hub was at D:\kb\old and is now at D:\kb\new.
+khb:   absolute paths recorded inside it still name the old location.
+khb:   repair them:  khb update --path            (--dry-run to preview)
+```
+
+The old location moves into `movedFrom`, where it waits until `--path` works it off.
+A hub moved twice before anyone repaired it lists both former homes and all of them are
+rewritten in the one pass. The key costs a string and is inert — nothing reads it but this
+repair — but it is what makes the repair need no arguments, and it survives what the registry
+does not: a deleted `~/.khb`, a first run on a second machine, a folder handed to a colleague.
+
+Failing that — a hub last touched by a khb too old to have recorded a location — the old path
+is *inferred, on proof only*: each registry entry records the `created` stamp from the hub's
+marker, minted once at `khb init`, so the dead entry carrying this hub's stamp is provably the
+same hub. A mere name match is circumstantial and is put to the user as a question rather than
+acted on. `--from` settles it either way, and is still the answer when the move predates all
+of this.
+
+The rewrite itself is a **conversion, not an interpretation**, which is what keeps it in the
+CLI rather than in an agent pass (§ AGENTS.md, division of labor): the same substring in
+and out, no judgement about what a path means. Three properties make it safe to run
+unattended:
+
+- **Every spelling moves.** A path appears natively (`D:\a\b`), forward-slashed (`D:/a/b`),
+  and backslash-escaped inside JSON (`D:\\a\\b`, how `raw/` headers and `log.md` store a
+  source). All are matched, and each is rewritten to *the same spelling* of the new path, so
+  a JSON-escaped source stays JSON-escaped.
+- **Matches end at a path boundary.** Moving `…/old` never touches `…/older`, a sibling
+  whose name merely starts the same way.
+- **The new path is shielded from itself.** It is matched too, and rewritten to itself. That
+  is what makes an *overlapping* move safe — a hub lifted out of its parent (`…/kb/hub` →
+  `…/kb`) or pushed down into a subdirectory of where it stood (`…/kb` → `…/kb/hub`). In the
+  second case the old path occurs inside every already-correct reference, and without the
+  shield each would have the move applied to it a second time. Claiming those matches for an
+  identity rewrite also makes the whole command idempotent: run it twice and the second run
+  changes nothing. Only old and new naming *the same directory* is refused, there being no
+  move to repair.
+
+`.git/`, `node_modules/` and the `inbox/` extraction cache are not walked; binary files and
+anything over 8 MB are skipped. `--dry-run` reports the file-by-file hit count and writes
+nothing.
+
+Like `khb ingest`, it states the whole plan before writing anything and reports progress as
+it walks (§2g).
+
+#### `--schema` — a bundle's `sources.yaml` predates a field
+
+A `sources.yaml` written before a field existed (e.g. `folder`/`files` sources gained
+`exclude:`) has no way to discover it short of reading the docs. `scripts/lib/schema.ts`
+holds the current, optional, backfillable field list per source `type`; this half diffs
+every bundle's `sources.yaml` against it and, for anything missing, stages a default value —
+or, for a field the schema has since deprecated, stages its removal. Comment-preserving:
+it edits the parsed YAML document node-by-node (the `yaml` package's `Document` API) rather
+than reserializing from scratch, so hand-written comments and structure in an untouched part
+of the file survive. The one caveat is cosmetic, not correctness: re-serializing the whole
+document can normalize whitespace in *other*, unrelated flow-style content in the same file.
+
+There is no persisted schema version anywhere — the "schema" is just this file's current
+shape, diffed fresh on every run, the same way `khb upgrade`'s `MANAGED`/`RETIRED` lists are
+static and re-checked rather than tracked historically. `--dry-run` prints the changes and
+writes nothing; otherwise it applies immediately, same convention as `--path`.
+
+`khb upgrade` prints a one-line hint when either half of `update` has something pending —
+never a prompt, since `upgrade` already runs unattended inside unrelated commands on version
+drift, and a blocking question there would interrupt work that has nothing to do with either
+repair.
+
+The name says what it does, now that it does two things belonging to neither `khb upgrade`
+(package-owned contract docs only) nor to a single-purpose `update-path`. The original
+one-letter-from-`upgrade` objection to the shorter name held while `update` meant only path
+repair; a second, unrelated repair under the same verb resolves it — `update` repairs what
+the user's own bundles record, `upgrade` refreshes what the package owns, and the two no
+longer read as near-synonyms.
+
+### 2f. First run — the wizard
+
+A bare `khb` on a machine with no hubs has a terminal in front of it and knows the one
+thing the user needs to hear, so it asks rather than referring them to the docs. Five
+questions, every one with a default that Enter accepts: where the hub goes, what to call
+it, a one-line description, which agent opens it, and a first bundle.
+
+Three things keep it honest:
+
+- **It asks only what `khb init` takes as flags**, and calls the same `createHub()`. A hub
+  born in the wizard is byte-identical to one made by hand — there is no second creation
+  path to drift.
+- **Agents are detected, not guessed.** `claude` and `codex` are probed with `--version`
+  and the ones present are marked; the first found is the default. Any other command can be
+  typed instead, or `none`.
+- **A path that is already a hub is adopted, not overwritten.** The machine simply had not
+  heard of it, so it joins the list under its own name and the wizard stops there.
+
+No terminal, or `--path` (a script asking for a path), falls back to three lines of
+guidance. `khb list` on an empty machine likewise reports rather than starts a
+conversation.
+
+### 2g. Progress on the mechanical passes
+
+`khb`'s half of the split is the deterministic half, and some of it is slow: a scanned PDF
+is seconds per page, a video minutes per file, and a hub that has accumulated thousands of
+raw documents takes a visible moment to walk. A silent process is indistinguishable from a
+hung one, so every mechanical pass reports (`scripts/lib/log.ts`):
+
+- **The plan, before any work.** Which hub, which bundle, which extractors are armed, how
+  many sources — or for `update --path`, the old path, the new path, and whether anything
+  will be written; for `update --schema`, which bundles and fields. A `--hub`/`$KHB_HUB`
+  run can target a folder you did not expect, and that must be visible before the first
+  write, not after.
+- **Position within the run.** `[3/57]` per unit where a unit is slow enough to deserve its
+  own line (ingest's per-file extraction), and a single rewritten counter line where the
+  units are fast and numerous (`update --path` checking files).
+- **An outcome per unit, and a closing summary** with wall-clock time.
+
+The transient counter writes to **stderr** and repaints at ~12fps; with no terminal it
+degrades to a milestone line every 500 units. So `khb … > out.txt` keeps clean, complete
+stdout, and a CI log still shows the walk moving. There is no `--quiet`: the per-unit line
+is the audit trail for a pass that rewrites files, and a run you have to repeat to find out
+what it did is worse than a noisy one.
+
 ## 3. Routing model
 
 - `outer.index.md` lists every bundle with a one-line scope and "route here when" hints.
@@ -195,6 +405,7 @@ Each bundle declares its sources in `sources.yaml`:
 sources:
   - type: folder      # local disk
     path: /abs/path/to/project-x
+    exclude: [drafts/]      # optional — skip paths/globs before ingesting
   - type: web
     urls:
       - https://example.com/design-doc
@@ -284,11 +495,23 @@ puts them on the CLI side of the §Division-of-labor line.
 | XLSX | `fflate` → one markdown table per sheet | bundled | high |
 | scanned PDF | `pdfium` + `tesseract.js` (WASM) | bundled, ~75 MB | low |
 | Images (png/jpg/webp/tif) | `tesseract.js` | bundled, ~75 MB | low |
-| Audio, video | `whisper` / `faster-whisper` | opt-in, pip | low |
+| Audio, video | `vno` (whisper.cpp), else `whisper` / `faster-whisper` | opt-in, npm or pip | low |
+| Captions (vtt/srt) | built-in reader | bundled | high |
+
+A recording that has a caption sidecar beside it — `talk.vtt`, `talk.en.vtt`, `talk.srt` —
+is read from the sidecar rather than transcribed, and the two are acquired as one source:
+one ledger row under the recording, no row and no `raw/` file for the sidecar. The pair's
+content hash covers both files, so correcting a caption re-ingests the recording. Where two
+sidecars disagree about language khb transcribes instead: choosing an audience is not a
+conversion decision, and §Division-of-labor puts choices on the agent's side of the line.
 
 The OCR stack is bundled rather than opt-in: an ingest that stops to ask for an install is
-worse than an install that carries WASM nobody uses. Transcription stays opt-in because it is
-a Python executable, not something a JS package manager can pull down.
+worse than an install that carries WASM nobody uses. Transcription stays opt-in because it
+is an external executable, not something khb's own dependency tree can carry: `vno`
+(@msareen/voice-notes-organizer) where `vno status` reports it ready, since whisper.cpp is
+faster than the Python whisper and hands back WebVTT the caption reader can anchor, else
+`whisper` / `faster-whisper`. A vno that is installed but not set up degrades to the
+fallback, or to pending rows, and never to a failed run.
 
 A missing dep degrades to a ledger row with an empty `raw` and a printed install
 hint — never to a failed run. `quality: low` output is a standing invitation for the catalog

package/document/faq.md

@@ -90,6 +90,7 @@ For example:
 sources:
   - type: folder
     path: /absolute/path/to/documents
+    exclude: [drafts/]      # optional — skip paths/globs before ingesting
   - type: files
     paths:
       - /absolute/path/to/one.pdf

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@msareen/knowledge-hub-builder",
-  "version": "0.1.7",
+  "version": "0.2.0",
   "description": "Knowledge Hub Builder — a bundle-of-bundles knowledge base you build with an agent",
   "keywords": ["knowledge-base", "okf", "agent", "wiki", "bundles", "claude", "codex", "rag"],
   "license": "MIT",