@msareen/knowledge-hub-builder 0.1.3

package/SPEC.md

@@ -0,0 +1,354 @@
+# KHB — Specification
+
+**KHB — Knowledge Hub Builder.** Distributed as
+`@msareen/knowledge-hub-builder`. Install the tooling once; run `khb init` wherever the knowledge should
+live (OneDrive, a shared drive, a private repo) to create a **hub**. KHB supplies the
+rules and tooling and holds no knowledge; the hub holds all of it and is yours.
+
+A personal knowledge system built as a **bundle of bundles**: independent knowledge bundles
+joined by a thin router, navigable by any coding/knowledge agent (Claude, Codex, or other).
+The common contract and canonical workflows are agent-agnostic. Runtime-specific shims
+only handle discovery: Codex reads `AGENTS.md` and `.agents/skills/`; Claude imports the
+contract through `CLAUDE.md` and discovers `.claude/skills/`.
+
+Lineage: the operating model (immutable raw sources → LLM-curated wiki → schema, driven
+by ingest/query/lint) is [Karpathy's LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f);
+the file format is [Google's OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/okf) v0.1.
+KHB's own contribution is the bundle-of-bundles layer over both — see §1.
+
+## 1. Core ideas
+
+1. **Bundle** — a self-contained *logical* unit owning its content and ingestion sources.
+   What makes a bundle is a human decision about ownership, not a classification of subject
+   matter: a person, a team, a client, a project. One bundle holds as many topics as its
+   owner has — a team bundle carries its roadmap, its incidents and its vendor notes
+   together, grouped by subdirectory, because they share a custodian and a context.
+   Bundles are never split automatically; splitting is an explicit instruction, never a
+   default behavior of ingest, catalog or query.
+   Bundles are *lean*: the common rules (AGENTS.md and the `skills/` protocols) live once
+   at the hub root, not duplicated per bundle. When a bundle must travel alone,
+   `khb export <bundle>` produces a standalone folder with the common patterns
+   injected — independence on demand rather than boilerplate everywhere.
+2. **Bundle of bundles** — the hub root is itself a bundle whose content is *routing*, not
+   knowledge. `outer.index.md` is the router: it points at bundles and larger topics, and
+   details route downward into a bundle's own `index.md`.
+3. **No collation by copying** — when two topics must be combined, content is never merged
+   across bundles. Instead: the requesting bundle records a **reference** (`refs.md`), and the
+   query resolves by *querying into* the other bundle. This keeps bundles independent and
+   collision-free. Two agents can work the two bundles in parallel and join results at the end.
+4. **Ingest, don't wiki-sprawl** — external material (folders on disk, Confluence, Azure
+   DevOps, internet links) is pulled *into* a bundle by ingesters rather than being linked as
+   living parts of the wiki. The wiki stays canonical; sources are provenance.
+5. **Acquisition and interpretation are separate steps.** `khb ingest` converts bytes to
+   markdown in `raw/` and stops. `catalog` — an agent pass, no command — turns that text
+   into concepts. Splitting them means a bad extraction is a re-run, not a re-think, and
+   every concept traces back through a provenance header to the original file.
+6. **Extract to markdown, locally** — binary/opaque formats (PDF, DOCX, XLSX, images,
+   audio) are converted to markdown so every bundle's knowledge is plain, greppable text.
+   Every extractor is local and deterministic: pure-JS libraries, tesseract WASM, whisper.
+   Lossy routes (OCR, ASR) are marked `quality: low` rather than hidden.
+7. **Bun** is the scripting language for all tooling and third-party interfaces.
+
+## 2. Two directories, one of them yours
+
+KHB is two things that must not be confused: an installed **package** (tooling, no
+knowledge) and a **hub** (knowledge, no tooling). Nothing you write ever lands in the
+package, and the package is never a place to keep bundles.
+
+### 2a. The hub — created by `khb init`, lives wherever you want
+
+```
+my-knowledge/                  # ~/OneDrive/my-knowledge, a private repo, a share…
+├── khb.json            # THE MARKER — how `khb` finds this hub; records its version
+├── outer.index.md             # THE router — bundle-of-bundles index
+├── bundles/
+│   └── <bundle>/              # OKF bundle, lean: content + routing, no agent boilerplate
+│       ├── index.md           # OKF index: progressive disclosure of this bundle
+│       ├── refs.md            # cross-bundle references (the ONLY way out of a bundle)
+│       ├── sources.yaml       # ingestion sources for this bundle
+│       ├── log.md             # ingest ledger: source → sha256 → raw → curated
+│       ├── <group>/           # free-form subdirectories of concept .md files
+│       │   ├── index.md       #   (each may carry its own index)
+│       │   └── <concept>.md
+│       └── raw/               # ingested/extracted material, pre-curation (gitignored)
+├── inbox/extracted/           # hub-wide extraction cache, keyed by content hash (gitignored)
+│
+│   ── below: package-owned copies, refreshed by `khb upgrade`, never hand-edited ──
+├── AGENTS.md                  # common contract; Codex discovers this directly
+├── CLAUDE.md                  # Claude shim — imports AGENTS.md
+├── .agents/skills/<name>/     # Codex discovery adapter → canonical skill
+├── .claude/skills/<name>/     # Claude discovery adapter → canonical skill
+├── SPEC.md                    # this file
+└── skills/<name>/SKILL.md     # one self-contained workflow protocol per folder:
+                               #   query      routing, reading, and query-time learning
+                               #   ingest     acquire + extract → raw/ (mechanical)
+                               #   catalog    raw/ → concept docs (subagent fan-out)
+                               #   lint       structural rules L1–L9
+                               #   new-bundle / export / visualize
+```
+
+The contract docs are *copied into* the hub rather than read from the package because an
+agent is opened on the hub folder and must find its rules there, without knowing where
+`khb` is installed. They are package-owned: `khb upgrade` overwrites them in place and
+leaves `bundles/` and `outer.index.md` alone.
+
+### 2b. The package — `@msareen/knowledge-hub-builder`, installed once
+
+> **None of the following is in your hub.** This section describes the *installed tool*,
+> which lives in your global package directory — not in the folder you are working in. A
+> hub contains exactly what §2a lists: `khb.json`, `outer.index.md`, `bundles/`, and the
+> package-owned contract docs. There is no `scripts/` and no `package.json` in a hub, so
+> paths below are package-internal and will not resolve from the hub root. To run the tool,
+> call `khb <command>` — never reach for a file in this tree.
+
+```
+@msareen/knowledge-hub-builder/     # installed once, globally — NOT part of a hub
+├── package.json               # bin: khb → scripts/cli.ts
+├── scripts/
+│   ├── cli.ts                 # subcommand dispatch; owns the global --hub flag
+│   ├── init.ts                # khb init / khb upgrade
+│   ├── new-bundle.ts          # scaffold from .bundle_template, register in outer.index.md
+│   ├── export.ts              # bundle + common patterns → standalone shareable folder
+│   ├── lint.ts                # enforce skills/lint/SKILL.md across the hub
+│   ├── visualize.ts           # serve the live bundle graph from indexes + refs
+│   ├── ingest/                # folder.ts / files.ts / web.ts → acquire.ts → bundle/raw
+│   └── lib/
+│       ├── extract.ts         # every local extractor + the content-hash cache
+│       ├── ledger.ts          # log.md read/write
+│       ├── paths.ts           # package-side paths — importing it never needs a hub
+│       └── util.ts            # hub resolution + shared helpers
+├── .bundle_template/          # copied by `khb new-bundle`
+├── templates/hub/             # copied by `khb init`
+└── AGENTS.md, skills/, …      # the masters that `khb init`/`upgrade` copy into hubs
+```
+
+### 2c. Hub resolution
+
+`khb` locates the hub in this order, and refuses to guess if none is found:
+
+1. `--hub <dir>` on the command line;
+2. `$KHB_HUB`;
+3. the nearest ancestor of the working directory containing `khb.json`.
+
+Rule 3 is the normal path: `cd` anywhere inside the hub and run `khb lint`. One
+consequence worth stating — a hub is identified by its marker file, not its name, so
+hubs may be renamed or moved freely, and nested hubs resolve to the innermost one.
+
+## 3. Routing model
+
+- `outer.index.md` lists every bundle with a one-line scope and "route here when" hints.
+  It contains **no knowledge**, only routing. An agent always starts at `AGENTS.md`, which
+  sends it to `outer.index.md`, which sends it into exactly one bundle.
+- Inside a bundle, `index.md` routes to concept docs and subdirectory indexes
+  (progressive disclosure, OKF §6). Same rule: index = routing only.
+- Keep hops shallow: outer index → bundle index → (subdirectory index →) concept.
+  If a bundle needs deep index chains, consider splitting it into two bundles.
+
+## 3b. Concept model (OKF v0.1)
+
+Bundle content follows the [Open Knowledge Format (OKF)](https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/okf) v0.1:
+
+- A **concept** is any non-reserved `.md` file in the bundle — not a special folder.
+  Concept ID = its path without `.md`. It may describe a tangible asset (table, API)
+  or an abstract idea (metric, decision, playbook).
+- **Front matter**: `type` required (free-form, e.g. `Metric`, `Playbook`); `title`,
+  `description`, `resource`, `tags`, `timestamp` recommended; extra keys allowed.
+- **Grouping** is free-form subdirectories chosen per domain (`tables/`,
+  `references/metrics/`, `notes/`, …) — structure carries no fixed semantics.
+- **Indexes**: `index.md` may appear in any directory for progressive disclosure
+  (`* [Title](path.md) - description`). `log.md` optionally records history.
+- **Links** between concepts are plain markdown links (bundle-root form
+  `/tables/x.md` preferred); relationship semantics live in prose, not link types.
+  Broken links are tolerated (not-yet-written knowledge).
+- Conventional body headings: `# Schema`, `# Examples`, `# Citations`.
+
+KHB additions on top of OKF: `refs.md` (reserved; the only cross-bundle
+pointer), `sources.yaml` (ingestion provenance), `raw/` (uncurated ingested material,
+exempt from OKF conformance), and the outer bundle-of-bundles router.
+
+## 4. Cross-bundle references and collation
+
+- A bundle may only reach outside itself through its `refs.md`. Each entry names the target
+  bundle, the reason, and optionally the specific note.
+- **Collation protocol** (also in `skills/query/SKILL.md`): to answer a question spanning bundles A and B:
+  1. Resolve in A. Note where A's `refs.md` points to B.
+  2. Open B **through its own index.md** — never jump straight to a B note from A's text.
+  3. Join the two answers in the response, not in the files.
+  Parallelizable: agent 1 handles A, agent 2 handles B, results merged by the orchestrator.
+- Inline links from a note in A directly into a note in B are a lint error. Notes may say
+  "see refs → <bundle>", nothing more specific. This is what keeps bundles independent.
+
+## 5. Ingestion
+
+Each bundle declares its sources in `sources.yaml`:
+
+```yaml
+sources:
+  - type: folder      # local disk
+    path: /abs/path/to/project-x
+  - type: web
+    urls:
+      - https://example.com/design-doc
+  # Confluence and ADO are listed for the record but ingested by the agent
+  # via MCP/CLI, not by scripts — see "scripting philosophy" below.
+  - type: confluence
+    space: PROJX
+  - type: ado
+    org: myorg
+    project: ProjectX
+```
+
+`khb ingest <bundle>` is **one flat phase**: it walks every scripted source, extracts what it
+can, and writes `raw/<type>/<file>.md` with a provenance header. It does not interpret
+content — that is §5b. Re-running is incremental: a source whose content hash is unchanged
+and whose `raw/` copy still exists is skipped; `--force` re-acquires.
+
+**A source is identified by its bytes, not its path.** When a file turns up at an unseen
+path, khb looks for a row with the same hash whose own path has since vanished; exactly one
+match means the file moved, and that row is re-pointed at the new path — keeping its `raw/`
+file (never renamed, because concepts cite it) and its `curated` value, and correcting the
+raw file's `source:` header. Without this, moving a file reads as a deletion plus an
+unrelated arrival: a duplicate raw file, a duplicate row with an empty `curated`, and
+eventually a duplicate concept for material already cataloged. A source's `raw/` filename is
+likewise fixed for life, so a file that moves and *then* changes re-extracts over the same
+file rather than stranding the one its citations point at.
+
+Two look-alikes are deliberately excluded, since both would rewire provenance on a guess: a
+**copy** (the twin's path still exists — two real sources, both ingested) and an **ambiguous**
+match (several vanished rows share the hash). Each is reported and left to judgement.
+
+The bundle argument is optional. With none, ingest targets `default` and scaffolds it if the
+hub has no such bundle, so a hub with zero bundles still has a landing path. Only that name
+is auto-created — an explicit name that doesn't resolve is an error, not a scaffold request.
+`default` is a holding area, not a tier: its contents are cataloged like any bundle's. Its
+material moves only when a human says which bundle should own it — a topic emerging inside
+it is not itself a reason to move anything.
+
+Every raw file carries its origin, so a lossy extraction is always recoverable:
+
+```yaml
+---
+source: /abs/path/to/original.pdf
+fetched: 2026-07-23T09:14:02Z
+sha256: db2ee470c95d
+extract_tool: tesseract.js
+quality: low          # high = real text; low = OCR or transcript, verify against source
+---
+```
+
+**Scripting philosophy: script every deterministic conversion, script no API wrappers.**
+Folder, files and web are scripted, as is every extractor (§6). Confluence, ADO and git
+hosts are reached through their MCP servers or official CLIs by the agent, which writes into
+the same `raw/` shape — no API wrapper code to maintain here. If agent-free refresh of one
+of those ever becomes a real need, promote it to a script then.
+
+## 5b. Catalog
+
+Turning `raw/` into concept docs is a separate, agent-only step — `skills/catalog/SKILL.md`,
+with no CLI command, because nothing about it is mechanical. Per bundle: read each raw file,
+split it into concepts, give each OKF frontmatter, link them, register them in `index.md`,
+and fill the `curated` column of `log.md`. Parallelized by fanning cheap subagents over the
+raw files, with one hard rule — subagents write concept docs, the orchestrator alone writes
+`index.md`, `log.md` and `refs.md`.
+
+## 6. Extraction
+
+Extraction is deterministic, so `khb` owns all of it. Common formats use bundled pure-JS
+libraries with no system install; results are cached hub-wide by content hash
+(`inbox/extracted/<sha256>.md`) and reused across bundles. Nothing here contacts a model:
+tesseract and whisper are local binaries, expensive in CPU but reproducible, which is what
+puts them on the CLI side of the §Division-of-labor line.
+
+| Format | Tool | Deps | Quality |
+|---|---|---|---|
+| PDF | `unpdf` (pdf.js), `pdftotext` if present | bundled | high |
+| DOCX | `mammoth`, `pandoc` if present | bundled | high |
+| ODT, PPTX | `fflate` + XML | bundled | high |
+| XLSX | `fflate` → one markdown table per sheet | bundled | high |
+| scanned PDF | `pdfium` + `tesseract.js` (WASM) | opt-in, ~75 MB | low |
+| Images (png/jpg/webp/tif) | `tesseract.js` | opt-in, ~75 MB | low |
+| Audio, video | `whisper` / `faster-whisper` | opt-in, pip | low |
+
+Missing optional deps degrade to a ledger row with an empty `raw` and a printed install
+hint — never to a failed run. `quality: low` output is a standing invitation for the catalog
+pass to re-read the original: a vision read of a chart or a scanned table recovers what OCR
+drops, and rewrites the raw file with `extract_tool: claude-vision`.
+
+## 7. Lint
+
+`khb lint` enforces (details in `skills/lint/SKILL.md`):
+- every bundle has `index.md`, `refs.md`, `sources.yaml`
+- every bundle is registered in `outer.index.md`; nothing in `outer.index.md` is dangling
+- every concept doc is listed in an index and carries OKF frontmatter (`type` required)
+- `refs.md` targets exist; no cross-bundle inline links from concept docs
+- index files contain links only (routing, not content)
+
+## 8. Visualizer
+
+`khb visualize [--port N]` scans `outer.index.md`, every bundle `index.md`/`refs.md`, and
+every concept doc's markdown links, and serves an interactive graph from a local server —
+`aliases: vis, viz` (`-v` is taken by `khb --version`). Two zoom levels: bundles as nodes
+with refs as directed edges (note counts as node size), and — click a bundle — its
+concepts as nodes with the markdown links between them as edges. Click a concept to open
+a panel that fetches its full body from `/api/file`. A refresh button rescans the hub and
+refetches the graph over `/api/graph`, for watching a hub change live during a catalog
+pass.
+
+The inner view is **clustered by folder, not free-floating**: each concept is anchored to
+the region of its top-level subdirectory (`tables/`, `notes/`, …), drawn as a labelled
+hull, and links that cross folders pull far more weakly than links inside one. The
+bundle's own organisation is therefore the visible structure, and the graph reads as
+traffic between regions instead of a single hairball. Deeper nesting collapses into its
+top-level folder — a handful of labelled regions beats one region per directory.
+
+The canvas **pans and zooms** (wheel to zoom at the cursor, drag the background to pan,
+`fit` / `F` to reframe). The layout is settled before the first paint and then framed by a
+fit pass with a floor on the zoom, so a small hub opens zoomed in rather than as three dots
+in an empty canvas, and a large one opens whole.
+
+The outer view seats bundles on a ring **just wide enough to hold them side by side**, with
+short-range repulsion and a firm pull to the centre: the hub opens as one compact group
+that reads at a glance, and zoom — not distance — is what makes it usable.
+
+Labels are drawn in **screen space at a fixed size**, not in world space, and any label
+whose box would collide with one already drawn is dropped — larger nodes claim their name
+first, and the hovered node always keeps its own. Text therefore stays legible and sparse
+at every zoom level instead of piling into an unreadable puddle; zooming in reveals the
+labels that were culled. A node's label is its front-matter `title`, clipped short; the
+untruncated title and full path appear in the panel when it is clicked, and in the hover
+strip at the bottom.
+
+A concept's `type` is encoded as **colour**, never as sub-text, and is spelled out in words
+in the hover strip and the panel chip — there is no legend and no shape vocabulary to
+learn. Hovering a node dims everything it isn't linked to. The top bar is deliberately
+thin: back, fit, a dark/light toggle remembered in `localStorage`, and refresh. Every
+colour the canvas draws comes from the active theme so the graph and the surrounding chrome
+stay in step.
+
+Folder regions are laid out on an **ellipse sized from the folders themselves** — each
+folder's disc grows with its file count, and the ring is only wide enough for neighbouring
+discs to clear each other — so two folders sit side by side rather than a screen and a half
+apart. The fit pass has a zoom ceiling but no floor: a large graph zooms out until it is
+whole rather than opening cropped.
+
+Unpinned, it binds a random free port (and picks another if `--port N` is already taken)
+and opens the URL in the default browser; `--no-open` just prints it. The page heartbeats
+the server and beacons on unload, so the server exits on its own once the browser tab
+closes rather than lingering as a background process.
+
+## 9. Agent workflow summary
+
+```
+build:  sources.yaml → khb ingest → raw/*.md (+ provenance, + log.md row)
+                     → catalog (agent) → concept docs + index.md entries + curated column
+                     → khb lint
+
+query:  question → AGENTS.md → outer.index.md → bundle/index.md → concept docs
+                           ↘ (spanning?) refs.md → other bundle via ITS index
+                           ↘ (durable synthesis?) propose a new concept → on confirm,
+                             write it, link it both ways, index it, log it
+```
+
+The query arm writing back is what makes the hub denser with use rather than merely larger:
+a question answered by joining two concepts leaves that join behind for the next one. It is
+always proposed and never silent — see `skills/query/SKILL.md`.

package/document/faq.md

@@ -0,0 +1,156 @@
+# Frequently Asked Questions
+
+## What is KHB?
+
+KHB (Knowledge Hub Builder) is a local, markdown-based system for building and maintaining
+a bundle-of-bundles knowledge base with an AI agent.
+
+The `khb` CLI is the supporting tool. It handles deterministic work such as extraction,
+file plumbing, validation, visualization, and export. The bundled workflow skills tell an
+AI agent—such as Claude, Codex, Gemini, or another compatible agent—how to ingest, catalog,
+query, and maintain the knowledge. The agent follows those skills and orchestrates the
+overall workflow.
+
+The hub itself remains an ordinary folder of markdown files, indexes, source declarations,
+and provenance records. The CLI holds no knowledge and does not call a model.
+
+## What are the main KHB workflow stages?
+
+KHB has three main stages:
+
+1. **Ingest** converts declared sources into provenance-bearing markdown under
+   `bundles/<bundle>/raw/`.
+2. **Catalog** interprets one bundle's raw material, creates canonical concept documents,
+   and updates that bundle's index and ledger.
+3. **Query** routes a question through the indexes and answers from curated concept
+   documents.
+
+Ingest is mechanical and local. Catalog and query require an agent because they involve
+judgement.
+
+## Does cataloging move everything into one main bundle?
+
+No. Ingest and catalog operate on the same owner-defined bundle. For example, material
+ingested into `client-a` is cataloged into concepts inside `bundles/client-a/`.
+
+Bundles are units of ownership, such as a person, team, project, or client. They are not
+subject categories, and an agent must not create, split, merge, or reorganize them without
+explicit instruction.
+
+## What happens when there are multiple bundles?
+
+Cataloging handles exactly one bundle at a time. Name the bundle in the request, such as:
+
+> Catalog the `client-a` bundle.
+
+If multiple bundles are possible and none was identified, the agent asks which bundle to
+use instead of guessing.
+
+During a query, the agent starts at `outer.index.md` and selects one bundle. If the
+question genuinely spans bundles, it follows `refs.md`, enters each additional bundle
+through its own `index.md`, and combines the results only in the answer.
+
+## What does `khb ingest [bundle] [--force]` do?
+
+The command reads the selected bundle's `sources.yaml`, acquires every supported declared
+source, extracts it into markdown under `raw/`, and updates `log.md`.
+
+It does not summarize, label, organize, or create concepts. Unchanged content hashes are
+skipped by default; `--force` reacquires everything.
+
+When no bundle is named, KHB uses the `default` bundle and creates it if necessary. An
+explicitly named bundle must already exist.
+
+## Should I run ingestion manually or ask an agent?
+
+Either works. Running `khb ingest client-a` manually performs the deterministic extraction.
+You can instead ask:
+
+> Ingest the `client-a` bundle.
+
+The agent invokes the same command, checks its summary, handles supported authenticated
+sources, and reports anything pending. Manual execution is useful in scripts or when only
+extraction is needed.
+
+## Where are ingestion sources declared?
+
+Each bundle has its own declaration file:
+
+```text
+bundles/<bundle>/sources.yaml
+```
+
+For example:
+
+```yaml
+sources:
+  - type: folder
+    path: /absolute/path/to/documents
+  - type: files
+    paths:
+      - /absolute/path/to/one.pdf
+      - /absolute/path/to/two.xlsx
+  - type: web
+    urls:
+      - https://en.wikipedia.org/wiki/Example
+  - type: confluence
+    space: PROJX
+```
+
+The `sources` list may contain one or many source declarations of different types.
+
+## When are sources added to `sources.yaml`?
+
+Sources are declared before ingestion. The usual sequence is:
+
+1. Create or select the owner bundle.
+2. Add its source locations to `sources.yaml`.
+3. Run ingestion.
+4. Catalog the resulting raw material.
+
+Declaring a source records where material should come from; it does not copy anything by
+itself.
+
+## What if I ask for ingestion without identifying the sources?
+
+The ingest workflow inspects the bundle's existing `sources.yaml` and asks which files,
+folders, URLs, or services to ingest. It includes existing declarations so they can be
+confirmed or replaced.
+
+The agent does not infer sources from nearby files, change `sources.yaml`, or run ingestion
+until the user answers.
+
+## Can one bundle ingest Wikipedia, Confluence, and Azure DevOps material?
+
+Yes. Public pages such as Wikipedia can be declared as `web` sources and fetched by
+`khb ingest`.
+
+Authenticated systems such as Confluence and Azure DevOps are also declared in
+`sources.yaml`, but the KHB CLI does not authenticate to them. The agent pulls their data
+through an available MCP connector or official CLI, writes the same provenance-bearing
+markdown shape under `raw/`, and updates `log.md`. If access is unavailable, the agent
+reports the blocked source.
+
+## Does cataloging delete the raw files?
+
+No. Cataloging reads raw files and creates canonical concept documents elsewhere in the
+same bundle. The raw material remains derived evidence and is not used directly to answer
+queries.
+
+The bundle's `log.md` connects each source and raw file to the concept documents curated
+from it.
+
+## Can a query create or update concepts?
+
+Yes, but never silently. If answering produces durable, reusable knowledge that existing
+concepts do not cover, the agent offers to:
+
+- fold the knowledge into an existing concept; or
+- create a new concept and link it to the concepts from which it was derived.
+
+The agent explains what it would write and where, then waits for confirmation. After
+approval, it updates the concept links, `index.md`, and `log.md`, and runs `khb lint`.
+One-off conversational answers are not saved.
+
+If a query reveals that an existing concept is wrong or stale, the agent similarly offers
+to repair it in place before making any change.

package/package.json

@@ -0,0 +1,52 @@
+{
+  "name": "@msareen/knowledge-hub-builder",
+  "version": "0.1.3",
+  "description": "Knowledge Hub Builder — a bundle-of-bundles knowledge base you build with an agent",
+  "keywords": ["knowledge-base", "okf", "agent", "wiki", "bundles", "claude", "codex", "rag"],
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/msareen/knowledge-hub-builder.git"
+  },
+  "homepage": "https://github.com/msareen/knowledge-hub-builder#readme",
+  "type": "module",
+  "bin": {
+    "khb": "./scripts/cli.ts"
+  },
+  "files": [
+    "scripts/",
+    "skills/",
+    "templates/",
+    ".bundle_template/",
+    ".agents/skills/",
+    ".claude/skills/",
+    "AGENTS.md",
+    "CLAUDE.md",
+    "SPEC.md",
+    "README.md",
+    "document/"
+  ],
+  "engines": {
+    "bun": ">=1.0.0"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "khb": "bun scripts/cli.ts",
+    "lint": "bun scripts/cli.ts lint",
+    "visualize": "bun scripts/cli.ts visualize",
+    "new-bundle": "bun scripts/cli.ts new-bundle",
+    "export": "bun scripts/cli.ts export",
+    "ingest": "bun scripts/cli.ts ingest"
+  },
+  "dependencies": {
+    "@hyzyla/pdfium": "^2.1.13",
+    "fflate": "^0.8.3",
+    "mammoth": "^1.12.0",
+    "sharp": "^0.35.3",
+    "tesseract.js": "^7.0.0",
+    "unpdf": "^1.6.2",
+    "yaml": "^2.4.0"
+  }
+}

package/scripts/cli.ts

@@ -0,0 +1,66 @@
+#!/usr/bin/env bun
+// khb — the CLI. Subcommands are loaded lazily: `init` must run before a hub exists,
+// so nothing that resolves a hub may be imported at module scope.
+import { version } from "./lib/paths";
+
+const COMMANDS: Record<string, { load: () => Promise<unknown>; help: string }> = {
+  init: { load: () => import("./init"), help: "khb init [dir]                  create a hub here (or in dir)" },
+  upgrade: { load: () => import("./init"), help: "khb upgrade                     refresh this hub's contract docs" },
+  "new-bundle": { load: () => import("./new-bundle"), help: 'khb new-bundle <name> ["scope"]  scaffold a bundle + register it' },
+  ingest: { load: () => import("./ingest/index"), help: "khb ingest [bundle] [--force]   acquire + extract declared sources → raw/ (default: 'default')" },
+  lint: { load: () => import("./lint"), help: "khb lint                        validate the hub against skills/lint/SKILL.md" },
+  visualize: { load: () => import("./visualize"), help: "khb visualize [--port N] [--no-open]  serve the live bundle graph in your browser; aliases: vis, viz" },
+  export: { load: () => import("./export"), help: "khb export <bundle> [dest]      standalone copy of one bundle" },
+};
+
+// Short forms that just resolve to a canonical command above — kept out of COMMANDS
+// itself so help text lists each command once. `-v` is taken by --version, so
+// `visualize` gets word-shaped aliases instead of a letter one.
+const ALIASES: Record<string, string> = { vis: "visualize", viz: "visualize" };
+
+const argv = process.argv.slice(2);
+
+// --hub <dir> is global: strip it here so subcommands never see it, and hand it to
+// util.ts through the environment (same channel as $KHB_HUB).
+const hubAt = argv.indexOf("--hub");
+if (hubAt >= 0) {
+  const dir = argv[hubAt + 1];
+  if (!dir) {
+    console.error("--hub needs a directory");
+    process.exit(1);
+  }
+  process.env.KHB_HUB = dir;
+  argv.splice(hubAt, 2);
+}
+
+const cmd0 = argv.shift();
+const cmd = cmd0 && ALIASES[cmd0] ? ALIASES[cmd0] : cmd0;
+
+if (!cmd || cmd === "help" || cmd === "--help" || cmd === "-h") {
+  console.log(`khb ${version()} — Knowledge Hub Builder\n`);
+  console.log(`khb is the supporting tool: it handles deterministic extraction, file plumbing,`);
+  console.log(`validation, and export. Your AI agent — Claude, Codex, Gemini, or another`);
+  console.log(`compatible agent — follows the workflow skills and orchestrates the knowledge work.\n`);
+  for (const c of Object.values(COMMANDS)) console.log("  " + c.help);
+  console.log(`\nGlobal:  --hub <dir>   operate on that hub instead of searching upward from cwd`);
+  console.log(`Docs:    https://github.com/msareen/knowledge-hub-builder`);
+  process.exit(0);
+}
+
+if (cmd === "--version" || cmd === "-v") {
+  console.log(version());
+  process.exit(0);
+}
+
+const entry = COMMANDS[cmd];
+if (!entry) {
+  console.error(`Unknown command: ${cmd}`);
+  console.error(`Try: khb help`);
+  process.exit(1);
+}
+
+// Subcommand modules parse process.argv.slice(2) themselves — reshape it so they see
+// their own arguments and not the subcommand name.
+process.argv = [process.argv[0], process.argv[1], ...argv];
+process.env.KHB_SUBCOMMAND = cmd;
+await entry.load();

package/scripts/export.ts

@@ -0,0 +1,42 @@
+// khb export <bundle> [dest] — export a bundle as a standalone, shareable unit.
+// Bundles stay lean in the hub (common patterns live at hub root); export injects those
+// patterns so the exported folder works alone with any agent.
+import { cpSync, writeFileSync, mkdirSync, existsSync, readFileSync } from "node:fs";
+import { HUB, bundleDir, join } from "./lib/util";
+import { detail, totalElapsed } from "./lib/log";
+
+const [name, destArg] = process.argv.slice(2);
+if (!name) { console.error("Usage: khb export <bundle> [dest]"); process.exit(1); }
+
+const src = bundleDir(name);
+const dest = destArg ?? join(HUB, "export", name);
+if (existsSync(dest)) { console.error(`Destination exists: ${dest}`); process.exit(1); }
+
+console.log(`khb export → ${name}`);
+detail(`from: ${src}`);
+detail(`to:   ${dest}`);
+
+mkdirSync(dest, { recursive: true });
+detail(`copying bundle/ …`);
+cpSync(src, join(dest, "bundle"), { recursive: true });
+
+// Inject the hub's package-owned copies (kept current by `khb upgrade`). AGENTS.md and
+// skills/ are canonical; the shims/adapters make them discoverable by Claude and Codex.
+for (const item of ["AGENTS.md", "CLAUDE.md", "skills", ".agents/skills", ".claude/skills"]) {
+  const target = join(dest, item);
+  mkdirSync(join(target, ".."), { recursive: true });
+  cpSync(join(HUB, item), target, { recursive: true });
+  detail(`copying ${item}`);
+}
+
+// standalone router: one-bundle outer index
+detail("writing outer.index.md + README.md");
+const scope = (readFileSync(join(HUB, "outer.index.md"), "utf8")
+  .split("\n").find((l) => l.includes(`[${name}]`)) ?? "").split("|")[2]?.trim() ?? "";
+writeFileSync(join(dest, "outer.index.md"),
+  `# outer.index — exported bundle\n\n| Bundle | Scope | Route here when |\n|---|---|---|\n| [${name}](bundle/index.md) | ${scope} | always — single-bundle export |\n`);
+
+writeFileSync(join(dest, "README.md"),
+  `# ${name} (exported KHB bundle)\n\nExported: ${new Date().toISOString()}\nOrigin: KHB bundle-of-bundles repo.\n\nStandalone unit: start at AGENTS.md → outer.index.md → bundle/index.md.\nWorkflow protocols (query, ingest, lint, …) live in skills/<name>/SKILL.md and are discoverable by Claude and Codex.\nNote: refs.md entries pointing at other bundles will not resolve here.\n`);
+
+console.log(`\nExported to ${dest} in ${totalElapsed()} (bundle + agent contracts, skills, single-bundle router)`);