@msareen/knowledge-hub-builder 0.1.5 → 0.1.8

package/.bundle_template/sources.yaml

@@ -4,6 +4,7 @@
 sources: []
 # - type: folder
 #   path: /abs/path/to/some-folder
+#   exclude: []          # optional — skip these paths/globs before ingesting
 # - type: web
 #   urls: []
 # - type: confluence

package/AGENTS.md

@@ -141,6 +141,22 @@ From outside, pass `--hub <dir>` or set `$KHB_HUB`.
 | `khb ingest <bundle>` | acquire + extract every source in `sources.yaml` → `raw/`; maintains `log.md`. Name the bundle — with none it lists the hub's bundles and stops, unless there is nothing to choose between (no bundles, or only `default`), where it uses `default` |
 | `khb export <bundle> [dest]` | standalone copy: bundle + common patterns, shareable alone |
 
+These five work **outside** any hub, against a per-machine shortcut list at
+`~/.khb/hubs-config.json` (`%USERPROFILE%\.khb` on Windows). They hold no knowledge — only
+paths — and fill themselves in as hubs are used, so they need no maintenance.
+
+| Command | Purpose |
+|---|---|
+| `khb list` | every hub on this machine, with its description and path |
+| `khb go [name\|N]` | open a hub — prints its path, then launches your agent there. Bare `khb` is this: one hub asks, several offer a list, none walks the user through creating the first. `--path` prints only the path |
+| `khb agent [name]` | which agent `khb go` launches — `claude`, `codex`, a custom `--command`, or `none` |
+| `khb update [new-path]` | repair the hub: `--path`/`-p` repoints the list and rewrites every old-path reference after a move (needs no arguments — the move is detected and announced by any khb command run there); `--schema`/`-s` backfills a bundle's `sources.yaml` to the current schema (e.g. a newly added `exclude:` field). No flag runs both. `--dry-run` first. Unrelated to `khb upgrade`, which only touches package-owned contract docs, never bundle content |
+| `khb forget <name>` | drop a shortcut; the hub folder is untouched |
+
+A hub's `name` and `description` in that list come from its own `khb.json`, so they travel
+with it. Set them at creation with `khb init --name --description`, or edit the marker's
+`name`/`description` keys — those two are yours, unlike the rest of the file.
+
 There is no `khb catalog` command — cataloging is entirely a judgement pass.
 
 ## Ingest, then catalog — two steps, in that order
@@ -158,6 +174,10 @@ The single exception is a hub with nothing to choose between: with no bundles at
 unnamed ingest lands in `default`, created on the spot, and where `default` is the only
 bundle it lands there as it stands.
 
+When a run finishes, **offer the catalog pass** for the bundle just ingested — report what
+landed and how many rows are uncurated, then ask before curating anything. Neither stopping
+silently at the summary nor cataloging unasked is right.
+
 **Catalog** (`skills/catalog/SKILL.md`) is the judgement half, one bundle at a time: read
 each `raw/` file, split it into concepts, give each OKF frontmatter, link them, register
 them in `index.md`. When the runtime supports parallel agents, fan them out over the raw

package/README.md

@@ -1,7 +1,7 @@
 # knowledge-hub-builder
 
 <p align="center">
-  <img src="images/demo.gif" alt="KHB creates a hub, ingests sources, routes a question to a concept, and validates the result" width="544">
+  <img src="images/demo.gif" alt="khb init in the shell, then an agent session: the ingest skill asks which bundle owns the material, runs khb ingest as a tool call, offers to catalog, writes the concept docs, then answers a question from them" width="600">
 </p>
 
 **KHB (Knowledge Hub Builder)** is a local, markdown-based knowledge base maintained with
@@ -27,8 +27,17 @@ KHB requires [Bun](https://bun.sh).
 
 ```bash
 bun install -g @msareen/knowledge-hub-builder
+khb
+```
+
+On a machine with no hubs yet, `khb` walks you through the first one — where it lives, what
+to call it, which agent opens it, and a first bundle. Every question has a default, so
+holding Enter produces a working hub.
+
+To do the same by hand:
 
-khb init ~/my-knowledge
+```bash
+khb init ~/my-knowledge --description "Personal knowledge"
 cd ~/my-knowledge
 
 khb new-bundle personal "My accounts, plans, records, and reference material"
@@ -40,6 +49,7 @@ Add sources to `bundles/personal/sources.yaml`:
 sources:
   - type: folder
     path: /absolute/path/to/documents
+    exclude: [drafts/]      # optional — skip paths/globs before ingesting
   - type: files
     paths:
       - /absolute/path/to/one.pdf
@@ -62,6 +72,9 @@ or ask the agent:
 The agent discovers the KHB workflow skills from the hub and performs the catalog pass
 after ingestion.
 
+Later, from any terminal, `khb` on its own takes you back to the hub and starts your agent
+there — see [Moving between hubs](#moving-between-hubs).
+
 ## How It Works
 
 ### 1. Ingest
@@ -69,20 +82,25 @@ after ingestion.
 `khb ingest <bundle>` reads `sources.yaml` and writes extracted markdown under
 `bundles/<bundle>/raw/`.
 
-Supported without additional system tools:
+Supported out of the box, with no system tools and no further installation:
 
 - text, markdown, CSV, JSON, and YAML
 - PDF and DOCX
 - ODT, XLSX, and PPTX
+- OCR for images and scanned PDFs, applied automatically when a PDF has no text layer
+
+OCR ships with KHB: `@hyzyla/pdfium`, `sharp`, and `tesseract.js` are ordinary dependencies,
+so installing KHB pulls them down whether or not you ever ingest a scan. That costs roughly
+75 MB of WASM plus `sharp`'s native binaries — the price of an ingest that never stalls
+waiting for a setup step.
 
-Optional local tools add:
+One extractor is genuinely optional:
 
-- OCR for images and scanned PDFs: `@hyzyla/pdfium`, `sharp`, and `tesseract.js`
-- audio and video transcription: a `whisper` or `faster-whisper` executable on `PATH`
+- audio and video transcription needs a `whisper` or `faster-whisper` executable on `PATH`
+  (`pip install -U openai-whisper`)
 
-If an optional extractor is missing, KHB leaves a pending row in `log.md` and prints the
-required setup. OCR packages must be installed where the KHB package resolves modules; the
-CLI prints that location.
+Without it, KHB leaves a pending row in `log.md` and prints the required setup rather than
+failing the run.
 
 Every raw markdown file carries provenance:
 
@@ -220,7 +238,7 @@ Commands can be run directly or requested through the matching agent skill.
 
 | Command | Purpose |
 |---|---|
-| `khb init [dir]` | Create a hub |
+| `khb init [dir] [--name N] [--description "…"]` | Create a hub |
 | `khb upgrade` | Refresh package-owned contracts and skills (also runs automatically on version drift) |
 | `khb new-bundle <name> ["scope"]` | Create and register a bundle |
 | `khb ingest [bundle] [--force]` | Acquire and extract declared sources |
@@ -235,6 +253,84 @@ Additional ingest flags:
 
 Run against a hub outside the current directory with `--hub <dir>` or `$KHB_HUB`.
 
+### Moving between hubs
+
+These four run **outside** any hub, from any terminal:
+
+| Command | Purpose |
+|---|---|
+| `khb` | Open a hub. One registered hub asks; several show a list and take a pick; none walks you through creating the first |
+| `khb list [--json]` | Every hub on this machine, with its description and path |
+| `khb go <name\|N> [--path]` | Open one by name or list position. `--path` prints just the path, for `cd "$(khb go --path work)"` |
+| `khb agent [name] [--command X] [--args "…"]` | Which agent `khb go` launches — `claude`, `codex`, anything on your PATH, or `none` to just print the path |
+| `khb update [new-path] [--path\|-p] [--schema\|-s] [--from <old>] [--dry-run]` | Repair the hub: path references after a move, and/or backfill `sources.yaml` to the current schema. No flag runs both |
+| `khb forget <name>` | Drop a hub from the list. The folder is untouched |
+
+`khb go` prints the hub's path and then starts your agent there, so a bare `khb` from a
+cold terminal ends with an agent open on the right folder. No program can change its
+parent shell's directory, which is why the `cd` line is printed rather than performed.
+
+The list lives in `~/.khb/hubs-config.json` (`%USERPROFILE%\.khb` on Windows). It holds
+paths and one launch command — no knowledge — and fills itself in: any khb command run
+inside a hub registers it, so hubs you already had show up without a migration step.
+Delete the file and the next command in each hub puts it back.
+
+A hub's name and description come from its own `khb.json`, so they travel with the hub
+rather than living in one machine's list. Set them with `khb init --name --description`,
+or edit those two keys in the marker later.
+
+### `khb update` — repairing a hub
+
+Two independent repairs, run together or apart:
+
+```bash
+khb update --dry-run    # what would change, both halves
+khb update               # apply both
+khb update --path        # only repoint paths after a move
+khb update --schema      # only backfill sources.yaml to the current schema
+```
+
+**`--path`/`-p`** — after you move the folder, this repoints the shortcut list (a moved hub
+shows as `MISSING` in `khb list` until you do) and rewrites the absolute paths recorded
+*inside* the hub that named the old location: `sources.yaml` entries, `source:` headers in
+`raw/`, `log.md` rows, `resource:` front matter. It needs no old path: the hub records where
+it lives in its own `khb.json` (`path`), so any khb command run in a moved hub notices and
+says so —
+
+```
+khb: this hub was at D:\kb\old and is now at D:\kb\new.
+khb:   repair them:  khb update --path            (--dry-run to preview)
+```
+
+— and `--path` reads the old location straight out of the marker. That record travels with
+the folder, so it works after `~/.khb` is deleted, on a second machine, or on a hub a
+colleague handed you. Move a hub twice before repairing it and both former homes are
+rewritten in one pass. For a hub last touched by a khb too old to have recorded a location,
+the old path is worked out from the registry entry that now points at nothing, matched on
+the identity stamp in `khb.json`. If it can't prove which entry that is, it asks; `--from
+<old-path>` tells it outright.
+
+The rewrite is a literal substitution with three guards: every spelling of the path moves
+(native, forward-slashed, and JSON-escaped as `raw/` headers store it), each rewritten to
+the same spelling of the new path; matches must end at a path boundary, so moving `…/old`
+never touches a sibling `…/older`; and the new path is matched too and rewritten to itself,
+which is what makes overlapping moves safe — lifting a hub out of its parent, or pushing it
+down into a subdirectory of where it stood — and makes a second run a no-op.
+`.git/`, `node_modules/` and the `inbox/` cache are skipped, as are binaries.
+
+**`--schema`/`-s`** — a bundle's `sources.yaml` can predate a field khb's since learned about
+(e.g. `exclude:`), with no way to discover it short of reading the docs. This backfills
+missing optional fields with their default, per bundle, preserving your comments and
+formatting. `khb upgrade` mentions when either half of `update` has something pending, as a
+printed hint — it never runs `update` for you.
+
+Both `khb update` and `khb ingest` state their whole plan before writing anything, then
+report as they go — a position per file for work slow enough to matter, a live counter for
+work that isn't. The counter goes to stderr, so redirected stdout stays clean.
+
+Not to be confused with `khb upgrade`, which refreshes a hub's package-owned contract docs
+and has nothing to do with moving anything.
+
 To update the installed package:
 
 ```bash

package/SPEC.md

@@ -115,6 +115,8 @@ acting on them** — an agent can never read a protocol the CLI no longer implem
 ├── scripts/
 │   ├── cli.ts                 # subcommand dispatch; --hub flag; the version drift check
 │   ├── init.ts                # khb init / khb upgrade
+│   ├── hubs.ts                # khb list / go / agent / update / forget — the only
+│   │                          #   commands that run outside a hub
 │   ├── new-bundle.ts          # scaffold from .bundle_template, register in outer.index.md
 │   ├── export.ts              # bundle + common patterns → standalone shareable folder
 │   ├── lint.ts                # enforce skills/lint/SKILL.md across the hub
@@ -124,6 +126,9 @@ acting on them** — an agent can never read a protocol the CLI no longer implem
 │       ├── extract.ts         # every local extractor + the content-hash cache
 │       ├── ledger.ts          # log.md read/write
 │       ├── paths.ts           # package-side paths — importing it never needs a hub
+│       ├── registry.ts        # ~/.khb/hubs-config.json: where this machine's hubs are
+│       ├── relocate.ts        # khb update --path's path rewriter — pure text, no judgement
+│       ├── schema.ts          # khb update --schema: sources.yaml field diff/apply
 │       ├── upgrade.ts         # the refresh itself: `khb upgrade` and the drift check
 │       └── util.ts            # hub resolution + shared helpers
 ├── .bundle_template/          # copied by `khb new-bundle`
@@ -143,6 +148,210 @@ Rule 3 is the normal path: `cd` anywhere inside the hub and run `khb lint`. One
 consequence worth stating — a hub is identified by its marker file, not its name, so
 hubs may be renamed or moved freely, and nested hubs resolve to the innermost one.
 
+### 2d. The machine registry — `~/.khb/hubs-config.json`
+
+Rules 1–3 all assume you already know where the hub is. From a cold terminal in an
+unrelated folder, you don't — and a person with a personal hub, a work hub and a client
+hub has three paths to remember. So khb keeps **one file per machine** recording where the
+hubs on it are:
+
+```
+~/.khb/                        # %USERPROFILE%\.khb on Windows; $KHB_HOME overrides
+└── hubs-config.json
+```
+
+```json
+{
+  "version": 1,
+  "defaultAgent": "claude",
+  "agents": { "claude": { "command": "claude", "args": [] },
+              "codex":  { "command": "codex",  "args": [] } },
+  "hubs": [
+    { "name": "my-knowledge", "description": "Personal — work + home",
+      "path": "D:\\code\\my-knowledge", "added": "…", "lastUsed": "…" }
+  ]
+}
+```
+
+Three properties define it:
+
+- **It is a shortcut list, never knowledge.** It holds paths and one launch command.
+  Delete it and nothing is lost — the next command run inside each hub puts it back.
+- **It fills itself in.** Every khb command that resolves a hub registers it, so hubs made
+  before the registry existed appear the first time anything is run in them. There is no
+  migration and no `register` command to remember. `khb forget <name>` drops a shortcut and
+  never touches the folder.
+- **The hub is the authority on its own identity.** `name` and `description` are read out
+  of the hub's `khb.json` (`khb init --name --description`, or edit the file), so a hub
+  moved to another machine or cloned by a colleague describes itself the same way there.
+  Only when the marker says nothing does khb fall back to the folder name and a summary of
+  the bundles inside. `khb upgrade` merges rather than replaces the marker, so keys khb
+  does not own survive an upgrade.
+
+The commands over it are `khb list`, `khb go`, `khb agent`, `khb update` and `khb forget` —
+the only ones that run **outside** a hub, and therefore the only ones that skip hub
+resolution and the version drift check. A bare `khb` is `khb go`: one hub asks to open it,
+several show the list and take a pick, none prints the help.
+
+`khb go` ends by launching your configured agent with the hub as its working directory.
+No process can change its parent shell's directory, so `khb go` prints the `cd` line for
+the human and passes the path to the agent as cwd — `khb go --path <name>` prints only the
+path, for `cd "$(khb go --path work)"`. `khb agent none` turns the launch off entirely.
+
+Every path stored in or compared against the registry is canonicalized first (`realpath`,
+case-folded on Windows). One directory has several true names — `C:\Users\MANASV~1\…` and
+`C:\Users\Manasvi Sareen\…` are the same folder, as is anything reached through a symlink —
+and without this a single hub lists twice and `khb update --path` cannot tell it has already
+been repaired.
+
+### 2e. `khb update` — repairing a hub
+
+Two independent repairs, selectable together or apart:
+
+```
+khb update [new-path] [--path|-p] [--schema|-s] [--from <old-path>] [--dry-run]
+```
+
+No flag runs both. `--path`/`-p` repairs a moved hub; `--schema`/`-s` backfills a bundle's
+`sources.yaml` to the current field set (§ scripts/lib/schema.ts). They are unrelated repairs
+that happen to share a plan-then-write-then-report shape, so one command with two switches
+beat two commands that would each need the same hub-resolution and dry-run plumbing.
+
+#### `--path` — a hub that moved
+
+Moving a hub folder breaks two things, and this half fixes both:
+
+1. **The shortcut list** still points at the old folder. `khb list` marks it `MISSING`.
+2. **Absolute paths recorded inside the hub** that named the old location — `sources.yaml`
+   entries, `source:` headers in `raw/`, `log.md` rows, `resource:` front matter — are now
+   dangling.
+
+Run from inside the moved hub, or name it. Nothing else is needed, because **the hub records
+its own location**. `khb.json` carries a `path` key — canonical, plus `pathAs` for the
+spelling khb was invoked through when the two differ — and any khb command run in a hub
+compares it against where the hub actually is. A mismatch is the move, stated by the only
+witness that travelled with the folder:
+
+```
+khb: this hub was at D:\kb\old and is now at D:\kb\new.
+khb:   absolute paths recorded inside it still name the old location.
+khb:   repair them:  khb update --path            (--dry-run to preview)
+```
+
+The old location moves into `movedFrom`, where it waits until `--path` works it off.
+A hub moved twice before anyone repaired it lists both former homes and all of them are
+rewritten in the one pass. The key costs a string and is inert — nothing reads it but this
+repair — but it is what makes the repair need no arguments, and it survives what the registry
+does not: a deleted `~/.khb`, a first run on a second machine, a folder handed to a colleague.
+
+Failing that — a hub last touched by a khb too old to have recorded a location — the old path
+is *inferred, on proof only*: each registry entry records the `created` stamp from the hub's
+marker, minted once at `khb init`, so the dead entry carrying this hub's stamp is provably the
+same hub. A mere name match is circumstantial and is put to the user as a question rather than
+acted on. `--from` settles it either way, and is still the answer when the move predates all
+of this.
+
+The rewrite itself is a **conversion, not an interpretation**, which is what keeps it in the
+CLI rather than in an agent pass (§ AGENTS.md, division of labor): the same substring in
+and out, no judgement about what a path means. Three properties make it safe to run
+unattended:
+
+- **Every spelling moves.** A path appears natively (`D:\a\b`), forward-slashed (`D:/a/b`),
+  and backslash-escaped inside JSON (`D:\\a\\b`, how `raw/` headers and `log.md` store a
+  source). All are matched, and each is rewritten to *the same spelling* of the new path, so
+  a JSON-escaped source stays JSON-escaped.
+- **Matches end at a path boundary.** Moving `…/old` never touches `…/older`, a sibling
+  whose name merely starts the same way.
+- **The new path is shielded from itself.** It is matched too, and rewritten to itself. That
+  is what makes an *overlapping* move safe — a hub lifted out of its parent (`…/kb/hub` →
+  `…/kb`) or pushed down into a subdirectory of where it stood (`…/kb` → `…/kb/hub`). In the
+  second case the old path occurs inside every already-correct reference, and without the
+  shield each would have the move applied to it a second time. Claiming those matches for an
+  identity rewrite also makes the whole command idempotent: run it twice and the second run
+  changes nothing. Only old and new naming *the same directory* is refused, there being no
+  move to repair.
+
+`.git/`, `node_modules/` and the `inbox/` extraction cache are not walked; binary files and
+anything over 8 MB are skipped. `--dry-run` reports the file-by-file hit count and writes
+nothing.
+
+Like `khb ingest`, it states the whole plan before writing anything and reports progress as
+it walks (§2g).
+
+#### `--schema` — a bundle's `sources.yaml` predates a field
+
+A `sources.yaml` written before a field existed (e.g. `folder`/`files` sources gained
+`exclude:`) has no way to discover it short of reading the docs. `scripts/lib/schema.ts`
+holds the current, optional, backfillable field list per source `type`; this half diffs
+every bundle's `sources.yaml` against it and, for anything missing, stages a default value —
+or, for a field the schema has since deprecated, stages its removal. Comment-preserving:
+it edits the parsed YAML document node-by-node (the `yaml` package's `Document` API) rather
+than reserializing from scratch, so hand-written comments and structure in an untouched part
+of the file survive. The one caveat is cosmetic, not correctness: re-serializing the whole
+document can normalize whitespace in *other*, unrelated flow-style content in the same file.
+
+There is no persisted schema version anywhere — the "schema" is just this file's current
+shape, diffed fresh on every run, the same way `khb upgrade`'s `MANAGED`/`RETIRED` lists are
+static and re-checked rather than tracked historically. `--dry-run` prints the changes and
+writes nothing; otherwise it applies immediately, same convention as `--path`.
+
+`khb upgrade` prints a one-line hint when either half of `update` has something pending —
+never a prompt, since `upgrade` already runs unattended inside unrelated commands on version
+drift, and a blocking question there would interrupt work that has nothing to do with either
+repair.
+
+The name says what it does, now that it does two things belonging to neither `khb upgrade`
+(package-owned contract docs only) nor to a single-purpose `update-path`. The original
+one-letter-from-`upgrade` objection to the shorter name held while `update` meant only path
+repair; a second, unrelated repair under the same verb resolves it — `update` repairs what
+the user's own bundles record, `upgrade` refreshes what the package owns, and the two no
+longer read as near-synonyms.
+
+### 2f. First run — the wizard
+
+A bare `khb` on a machine with no hubs has a terminal in front of it and knows the one
+thing the user needs to hear, so it asks rather than referring them to the docs. Five
+questions, every one with a default that Enter accepts: where the hub goes, what to call
+it, a one-line description, which agent opens it, and a first bundle.
+
+Three things keep it honest:
+
+- **It asks only what `khb init` takes as flags**, and calls the same `createHub()`. A hub
+  born in the wizard is byte-identical to one made by hand — there is no second creation
+  path to drift.
+- **Agents are detected, not guessed.** `claude` and `codex` are probed with `--version`
+  and the ones present are marked; the first found is the default. Any other command can be
+  typed instead, or `none`.
+- **A path that is already a hub is adopted, not overwritten.** The machine simply had not
+  heard of it, so it joins the list under its own name and the wizard stops there.
+
+No terminal, or `--path` (a script asking for a path), falls back to three lines of
+guidance. `khb list` on an empty machine likewise reports rather than starts a
+conversation.
+
+### 2g. Progress on the mechanical passes
+
+`khb`'s half of the split is the deterministic half, and some of it is slow: a scanned PDF
+is seconds per page, a video minutes per file, and a hub that has accumulated thousands of
+raw documents takes a visible moment to walk. A silent process is indistinguishable from a
+hung one, so every mechanical pass reports (`scripts/lib/log.ts`):
+
+- **The plan, before any work.** Which hub, which bundle, which extractors are armed, how
+  many sources — or for `update --path`, the old path, the new path, and whether anything
+  will be written; for `update --schema`, which bundles and fields. A `--hub`/`$KHB_HUB`
+  run can target a folder you did not expect, and that must be visible before the first
+  write, not after.
+- **Position within the run.** `[3/57]` per unit where a unit is slow enough to deserve its
+  own line (ingest's per-file extraction), and a single rewritten counter line where the
+  units are fast and numerous (`update --path` checking files).
+- **An outcome per unit, and a closing summary** with wall-clock time.
+
+The transient counter writes to **stderr** and repaints at ~12fps; with no terminal it
+degrades to a milestone line every 500 units. So `khb … > out.txt` keeps clean, complete
+stdout, and a CI log still shows the walk moving. There is no `--quiet`: the per-unit line
+is the audit trail for a pass that rewrites files, and a run you have to repeat to find out
+what it did is worse than a noisy one.
+
 ## 3. Routing model
 
 - `outer.index.md` lists every bundle with a one-line scope and "route here when" hints.
@@ -195,6 +404,7 @@ Each bundle declares its sources in `sources.yaml`:
 sources:
   - type: folder      # local disk
     path: /abs/path/to/project-x
+    exclude: [drafts/]      # optional — skip paths/globs before ingesting
   - type: web
     urls:
       - https://example.com/design-doc
@@ -282,11 +492,15 @@ puts them on the CLI side of the §Division-of-labor line.
 | DOCX | `mammoth`, `pandoc` if present | bundled | high |
 | ODT, PPTX | `fflate` + XML | bundled | high |
 | XLSX | `fflate` → one markdown table per sheet | bundled | high |
-| scanned PDF | `pdfium` + `tesseract.js` (WASM) | opt-in, ~75 MB | low |
-| Images (png/jpg/webp/tif) | `tesseract.js` | opt-in, ~75 MB | low |
+| scanned PDF | `pdfium` + `tesseract.js` (WASM) | bundled, ~75 MB | low |
+| Images (png/jpg/webp/tif) | `tesseract.js` | bundled, ~75 MB | low |
 | Audio, video | `whisper` / `faster-whisper` | opt-in, pip | low |
 
-Missing optional deps degrade to a ledger row with an empty `raw` and a printed install
+The OCR stack is bundled rather than opt-in: an ingest that stops to ask for an install is
+worse than an install that carries WASM nobody uses. Transcription stays opt-in because it is
+a Python executable, not something a JS package manager can pull down.
+
+A missing dep degrades to a ledger row with an empty `raw` and a printed install
 hint — never to a failed run. `quality: low` output is a standing invitation for the catalog
 pass to re-read the original: a vision read of a chart or a scanned table recovers what OCR
 drops, and rewrites the raw file with `extract_tool: claude-vision`.

package/document/faq.md

@@ -90,6 +90,7 @@ For example:
 sources:
   - type: folder
     path: /absolute/path/to/documents
+    exclude: [drafts/]      # optional — skip paths/globs before ingesting
   - type: files
     paths:
       - /absolute/path/to/one.pdf

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@msareen/knowledge-hub-builder",
-  "version": "0.1.5",
+  "version": "0.1.8",
   "description": "Knowledge Hub Builder — a bundle-of-bundles knowledge base you build with an agent",
   "keywords": ["knowledge-base", "okf", "agent", "wiki", "bundles", "claude", "codex", "rag"],
   "license": "MIT",

package/scripts/cli.ts

@@ -3,16 +3,39 @@
 // so nothing that resolves a hub may be imported at module scope.
 import { version, findHub, markerIn, MARKER } from "./lib/paths";
 
-const COMMANDS: Record<string, { load: () => Promise<unknown>; help: string }> = {
-  init: { load: () => import("./init"), help: "khb init [dir]                  create a hub here (or in dir)" },
-  upgrade: { load: () => import("./init"), help: "khb upgrade                     refresh this hub's contract docs" },
-  "new-bundle": { load: () => import("./new-bundle"), help: 'khb new-bundle <name> ["scope"]  scaffold a bundle + register it' },
-  ingest: { load: () => import("./ingest/index"), help: "khb ingest <bundle> [--force]   acquire + extract declared sources → raw/ (name required once the hub has a bundle other than 'default')" },
-  lint: { load: () => import("./lint"), help: "khb lint                        validate the hub against skills/lint/SKILL.md" },
-  visualize: { load: () => import("./visualize"), help: "khb visualize [--port N] [--no-open]  serve the live bundle graph in your browser; aliases: vis, viz" },
-  export: { load: () => import("./export"), help: "khb export <bundle> [dest]      standalone copy of one bundle" },
+const COMMANDS: Record<string, { load: () => Promise<unknown>; usage: string; desc: string }> = {
+  init: { load: () => import("./init"), usage: 'khb init [dir] [--name N] [--description "…"]', desc: "create a hub here (or in dir)" },
+  upgrade: { load: () => import("./init"), usage: "khb upgrade", desc: "refresh this hub's contract docs" },
+  "new-bundle": { load: () => import("./new-bundle"), usage: 'khb new-bundle <name> ["scope"]', desc: "scaffold a bundle + register it" },
+  ingest: {
+    load: () => import("./ingest/index"),
+    usage: "khb ingest <bundle> [--force]",
+    desc: "acquire + extract declared sources → raw/ (name required once the hub has a bundle other than 'default')",
+  },
+  lint: { load: () => import("./lint"), usage: "khb lint", desc: "validate the hub against skills/lint/SKILL.md" },
+  visualize: {
+    load: () => import("./visualize"),
+    usage: "khb visualize [--port N] [--no-open]",
+    desc: "serve the live bundle graph in your browser; aliases: vis, viz",
+  },
+  export: { load: () => import("./export"), usage: "khb export <bundle> [dest]", desc: "standalone copy of one bundle" },
+  list: { load: () => import("./hubs"), usage: "khb list [--json]", desc: "every hub on this machine" },
+  go: { load: () => import("./hubs"), usage: "khb go [name|N] [--path]", desc: "open a hub with your agent (bare 'khb' picks one)" },
+  agent: { load: () => import("./hubs"), usage: "khb agent [name] [--command X]", desc: "which agent 'khb go' launches" },
+  update: {
+    load: () => import("./hubs"),
+    usage: "khb update [new] [--path|-p] [--schema|-s] [--dry-run]",
+    desc: "repair a moved hub's paths and/or backfill sources.yaml to the current schema",
+  },
+  forget: { load: () => import("./hubs"), usage: "khb forget <name>", desc: "drop a hub from the list (folder untouched)" },
 };
 
+/**
+ * Commands that work *outside* a hub, against ~/.khb/hubs-config.json. They must not
+ * resolve or upgrade a hub — their whole job is running before you are in one.
+ */
+const REGISTRY_COMMANDS = new Set(["list", "go", "agent", "forget", "update"]);
+
 // Short forms that just resolve to a canonical command above — kept out of COMMANDS
 // itself so help text lists each command once. `-v` is taken by --version, so
 // `visualize` gets word-shaped aliases instead of a letter one.
@@ -34,15 +57,39 @@ if (hubAt >= 0) {
 }
 
 const cmd0 = argv.shift();
-const cmd = cmd0 && ALIASES[cmd0] ? ALIASES[cmd0] : cmd0;
+// Bare `khb` is the way in from a cold terminal: pick a hub and open it. Help stays one
+// word away, and is what you get when no hub has ever been registered.
+const cmd = !cmd0 ? "go" : (ALIASES[cmd0] ?? cmd0);
+
+if (cmd === "help" || cmd === "--help" || cmd === "-h") {
+  const width = Math.max(...Object.values(COMMANDS).map((c) => c.usage.length));
+  const printSection = (title: string, names: string[]) => {
+    console.log(title);
+    for (const name of names) {
+      const c = COMMANDS[name];
+      console.log(`  ${c.usage.padEnd(width)}   ${c.desc}`);
+    }
+  };
 
-if (!cmd || cmd === "help" || cmd === "--help" || cmd === "-h") {
-  console.log(`khb ${version()} — Knowledge Hub Builder\n`);
+  console.log(`khb ${version()} — Knowledge Hub Builder`);
+  console.log();
   console.log(`khb is the supporting tool: it handles deterministic extraction, file plumbing,`);
   console.log(`validation, and export. Your AI agent — Claude, Codex, Gemini, or another`);
-  console.log(`compatible agent — follows the workflow skills and orchestrates the knowledge work.\n`);
-  for (const c of Object.values(COMMANDS)) console.log("  " + c.help);
-  console.log(`\nGlobal:  --hub <dir>   operate on that hub instead of searching upward from cwd`);
+  console.log(`compatible agent — follows the workflow skills and orchestrates the knowledge work.`);
+  console.log();
+
+  printSection(
+    "In a hub:",
+    Object.keys(COMMANDS).filter((name) => !REGISTRY_COMMANDS.has(name)),
+  );
+  console.log();
+  printSection(
+    "Anywhere on this machine (no hub needed) — config and maintenance:",
+    Object.keys(COMMANDS).filter((name) => REGISTRY_COMMANDS.has(name)),
+  );
+  console.log();
+
+  console.log(`Global:  --hub <dir>   operate on that hub instead of searching upward from cwd`);
   console.log(`Docs:    https://github.com/msareen/knowledge-hub-builder`);
   process.exit(0);
 }
@@ -63,11 +110,31 @@ if (!entry) {
 // stamped at an older version than the installed khb is stating an older contract than
 // the one the CLI now implements. Rather than let the two disagree, refresh the hub in
 // place before running the command — `khb upgrade` touches nothing the user wrote.
-// `init` has no hub yet, `upgrade` does this itself, and $KHB_NO_AUTO_UPGRADE opts out.
-if (cmd !== "init" && cmd !== "upgrade" && !process.env.KHB_NO_AUTO_UPGRADE) {
+// `init` has no hub yet, `upgrade` does this itself, the registry commands run outside
+// any hub, and $KHB_NO_AUTO_UPGRADE opts out.
+if (cmd !== "init" && cmd !== "upgrade" && !REGISTRY_COMMANDS.has(cmd)) {
   const hub = findHub();
   if (hub) {
-    const { hubVersion, upgradeHub } = await import("./lib/upgrade");
+    // Any command run in a hub is proof the hub exists and is in use — record it, so the
+    // machine registry fills itself in without a migration or a `khb register` to recall.
+    const { registerHub, touchHub } = await import("./lib/registry");
+    registerHub(hub);
+    touchHub(hub);
+
+    // …and proof of where it is. The hub records its own location in its marker, so a move
+    // is noticed by the hub itself rather than inferred from a registry that may have been
+    // deleted or never have seen this machine. Must run before the drift check below:
+    // that restamps the marker, and would overwrite the old location before anyone read it.
+    const { recordLocation } = await import("./lib/upgrade");
+    const { moved } = recordLocation(hub);
+    if (moved) {
+      console.error(`khb: this hub was at ${moved} and is now at ${hub}.`);
+      console.error(`khb:   absolute paths recorded inside it still name the old location.`);
+      console.error(`khb:   repair them:  khb update --path           (--dry-run to preview)`);
+    }
+  }
+  if (hub && !process.env.KHB_NO_AUTO_UPGRADE) {
+    const { hubVersion, upgradeHub, updateHint } = await import("./lib/upgrade");
     // A marker under a pre-rename name is drift too, even at a matching version.
     if (hubVersion(hub) !== version() || markerIn(hub) !== MARKER) {
       const { from, to, pruned, renamed } = upgradeHub(hub);
@@ -77,6 +144,8 @@ if (cmd !== "init" && cmd !== "upgrade" && !process.env.KHB_NO_AUTO_UPGRADE) {
       );
       if (renamed) console.error(`khb:   renamed ${renamed} -> khb.json`);
       if (pruned.length) console.error(`khb:   removed (no longer part of the contract): ${pruned.join(", ")}`);
+      const hint = updateHint(hub);
+      if (hint) console.error(hint);
     }
   }
 }