@msareen/knowledge-hub-builder 0.1.3 → 0.1.4

package/AGENTS.md

@@ -138,7 +138,7 @@ From outside, pass `--hub <dir>` or set `$KHB_HUB`.
 | `khb upgrade` | refresh this hub's package-owned contract docs |
 | `khb visualize` | regenerate `visualizer/graph.html` |
 | `khb new-bundle <name>` | scaffold + register a bundle |
-| `khb ingest [bundle]` | acquire + extract every source in `sources.yaml` → `raw/`; maintains `log.md`. No bundle named → `default`, created if absent |
+| `khb ingest <bundle>` | acquire + extract every source in `sources.yaml` → `raw/`; maintains `log.md`. Name the bundle — with none it lists the hub's bundles and stops, unless there is nothing to choose between (no bundles, or only `default`), where it uses `default` |
 | `khb export <bundle> [dest]` | standalone copy: bundle + common patterns, shareable alone |
 
 There is no `khb catalog` command — cataloging is entirely a judgement pass.
@@ -147,10 +147,17 @@ There is no `khb catalog` command — cataloging is entirely a judgement pass.
 
 **Ingest** (`skills/ingest/SKILL.md`) is mechanical and flat: `khb ingest <bundle>` pulls
 every declared source into `raw/` as markdown with a provenance header, extracting
-everything it can locally — into the named bundle, or into `default` when none is named — text, PDF, DOCX, ODT, XLSX, PPTX, images by OCR, audio and
+everything it can locally — text, PDF, DOCX, ODT, XLSX, PPTX, images by OCR, audio and
 video by whisper. Sources behind an authenticated API (Confluence, ADO, git hosts) you pull
 yourself via MCP/CLI into the same `raw/` shape. Ingest never interprets content.
 
+When the user has not named a bundle, **ask** which existing bundle owns the material or
+whether to start a new one, and for an existing bundle ask whether to re-ingest what its
+`sources.yaml` already declares or to take a new path. Never choose a destination silently.
+The single exception is a hub with nothing to choose between: with no bundles at all an
+unnamed ingest lands in `default`, created on the spot, and where `default` is the only
+bundle it lands there as it stands.
+
 **Catalog** (`skills/catalog/SKILL.md`) is the judgement half, one bundle at a time: read
 each `raw/` file, split it into concepts, give each OKF frontmatter, link them, register
 them in `index.md`. When the runtime supports parallel agents, fan them out over the raw

package/README.md

@@ -180,9 +180,13 @@ Create one only when you intend to:
 khb new-bundle team-payments "Payments team roadmap, incidents, and vendor decisions"
 ```
 
-KHB never creates, splits, or merges named bundles based on their contents. When
-`khb ingest` is run without a bundle name, it uses a `default` landing bundle and creates it
-if needed.
+KHB never creates, splits, or merges bundles based on their contents. Name the bundle when
+you ingest: run `khb ingest` without one and it lists the hub's bundles and stops. Ask an
+agent to ingest without naming a bundle and it asks you — existing bundle or new one, and for
+an existing one whether to reuse the paths in its `sources.yaml` or take a new path. An
+unnamed ingest lands somewhere by itself only when there is nothing to choose between: a hub
+with no bundles gets a `default` bundle created on the spot, and a hub whose only bundle is
+`default` uses it. Material moves out of `default` when you say who owns it.
 
 Cross-bundle relationships belong in `refs.md`; concept documents must not link directly
 into another bundle.

package/SPEC.md

@@ -217,12 +217,20 @@ Two look-alikes are deliberately excluded, since both would rewire provenance on
 **copy** (the twin's path still exists — two real sources, both ingested) and an **ambiguous**
 match (several vanished rows share the hash). Each is reported and left to judgement.
 
-The bundle argument is optional. With none, ingest targets `default` and scaffolds it if the
-hub has no such bundle, so a hub with zero bundles still has a landing path. Only that name
-is auto-created — an explicit name that doesn't resolve is an error, not a scaffold request.
-`default` is a holding area, not a tier: its contents are cataloged like any bundle's. Its
-material moves only when a human says which bundle should own it — a topic emerging inside
-it is not itself a reason to move anything.
+A bundle is a logical unit its owner defines, so the destination is a human decision. An
+explicit name that doesn't resolve is an error, not a scaffold request; a bare `khb ingest`
+in a hub that has real bundles prints them and stops, because the CLI cannot ask. The agent
+asks instead (`skills/ingest/SKILL.md` §1–2): which existing bundle, or a new one; then, for
+an existing bundle, whether to re-ingest its declared `sources.yaml` paths or take a new path.
+
+`default` survives as the fallback for a hub with nothing to choose between: a bare
+`khb ingest` scaffolds it and lands there when there are **no bundles at all**, and uses it
+when it is the **only** bundle — a one-option question is not a choice, and the first ingest
+anyone runs should not fail for want of a destination. It is not an option once a real bundle
+exists — that was the earlier design, and it bought a pile of material whose ownership
+nobody had decided, which is exactly the decision cataloging then has to make blind.
+`default` is a holding area, not a tier: its contents are cataloged like any bundle's, and
+they move only when a human says which bundle should own them.
 
 Every raw file carries its origin, so a lossy extraction is always recoverable:
 

package/document/faq.md

@@ -58,8 +58,12 @@ source, extracts it into markdown under `raw/`, and updates `log.md`.
 It does not summarize, label, organize, or create concepts. Unchanged content hashes are
 skipped by default; `--force` reacquires everything.
 
-When no bundle is named, KHB uses the `default` bundle and creates it if necessary. An
-explicitly named bundle must already exist.
+A named bundle must already exist. Run without a name and the command lists the hub's bundles
+and stops — unless there is nothing to choose between: a hub with no bundles gets a `default`
+landing bundle created so a first ingest still works, and a hub whose only bundle is `default`
+uses it. An agent asked to ingest without a named bundle asks you which
+existing bundle owns the material or whether to create a new one, and for an existing bundle
+whether to re-ingest the paths already in its `sources.yaml` or take a new path.
 
 ## Should I run ingestion manually or ask an agent?
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@msareen/knowledge-hub-builder",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "description": "Knowledge Hub Builder — a bundle-of-bundles knowledge base you build with an agent",
   "keywords": ["knowledge-base", "okf", "agent", "wiki", "bundles", "claude", "codex", "rag"],
   "license": "MIT",

package/scripts/cli.ts

@@ -7,7 +7,7 @@ const COMMANDS: Record<string, { load: () => Promise<unknown>; help: string }> =
   init: { load: () => import("./init"), help: "khb init [dir]                  create a hub here (or in dir)" },
   upgrade: { load: () => import("./init"), help: "khb upgrade                     refresh this hub's contract docs" },
   "new-bundle": { load: () => import("./new-bundle"), help: 'khb new-bundle <name> ["scope"]  scaffold a bundle + register it' },
-  ingest: { load: () => import("./ingest/index"), help: "khb ingest [bundle] [--force]   acquire + extract declared sources → raw/ (default: 'default')" },
+  ingest: { load: () => import("./ingest/index"), help: "khb ingest <bundle> [--force]   acquire + extract declared sources → raw/ (name required once the hub has a bundle other than 'default')" },
   lint: { load: () => import("./lint"), help: "khb lint                        validate the hub against skills/lint/SKILL.md" },
   visualize: { load: () => import("./visualize"), help: "khb visualize [--port N] [--no-open]  serve the live bundle graph in your browser; aliases: vis, viz" },
   export: { load: () => import("./export"), help: "khb export <bundle> [dest]      standalone copy of one bundle" },

package/scripts/ingest/index.ts

@@ -12,7 +12,7 @@
 import { parse } from "yaml";
 import { read, join, HUB } from "../lib/util";
 import { detail, section, totalElapsed } from "../lib/log";
-import { bundleForIngest, DEFAULT_BUNDLE } from "../lib/scaffold";
+import { bundleForIngest, listBundles, DEFAULT_BUNDLE } from "../lib/scaffold";
 import { readLedger, writeLedger } from "../lib/ledger";
 import { takeFlag } from "../lib/args";
 import { ingestFolder } from "./folder";
@@ -43,10 +43,23 @@ if (positional.length > 1) {
   console.error("Usage: khb ingest [bundle] [--force] [--skip-ocr] [--skip-audio]");
   process.exit(1);
 }
-// No bundle named → `default`, created on the spot if the hub has none. Bytes always have
-// somewhere to land; sorting them into real bundles is a later, cheaper decision (a concept
-// is one file, and moving it is a `git mv`). Naming a bundle explicitly stays the norm.
-const bundle = positional[0] ?? DEFAULT_BUNDLE;
+// No bundle named: which one owns the material is a human decision (AGENTS.md) and a CLI
+// cannot ask, so it stops and shows what the hub has, leaving the choice to whoever is
+// driving. The exception is a hub with nothing to choose between — no bundles at all, or
+// only the landing bundle — where bytes go to `default` rather than the ingest failing.
+let bundle = positional[0];
+if (!bundle) {
+  const have = listBundles();
+  const onlyLanding = have.length === 1 && have[0] === DEFAULT_BUNDLE;
+  if (have.length && !onlyLanding) {
+    console.error("Usage: khb ingest [bundle] [--force] [--skip-ocr] [--skip-audio]");
+    console.error(`\nBundles in this hub: ${have.join(", ")}`);
+    console.error(`Name the one that owns this material, or start a new one:`);
+    console.error(`  khb new-bundle <name> "<scope>"`);
+    process.exit(1);
+  }
+  bundle = DEFAULT_BUNDLE;
+}
 
 const dir = bundleForIngest(bundle);
 let cfg: unknown;

package/scripts/lib/scaffold.ts

@@ -1,9 +1,12 @@
-// Bundle creation, shared by `khb new-bundle` and by ingest's default-bundle fallback.
-// One implementation so a bundle born from a bare `khb ingest` is indistinguishable from
-// one the user named: same template, same {{name}} substitution, same outer.index.md row.
+// Bundle creation and lookup, shared by `khb new-bundle` and `khb ingest`.
+// One implementation so every bundle is born the same way: same template, same {{name}}
+// substitution, same outer.index.md row — and nothing ever conjures one implicitly.
 import { cpSync, readFileSync, writeFileSync, readdirSync, statSync, existsSync } from "node:fs";
 import { HUB, BUNDLES, TEMPLATE, join } from "./util";
 
+// The landing bundle: where a bare `khb ingest` goes in a hub with nothing to choose between
+// — conjured when the hub has no bundles at all, reused when it is the only one. With a real
+// bundle present the destination is a choice and the user makes it, so nothing is auto-created.
 export const DEFAULT_BUNDLE = "default";
 
 export const VALID_NAME = /^[a-z0-9][a-z0-9-]*$/;
@@ -33,24 +36,34 @@ export function createBundle(name: string, scope: string): string {
   return dest;
 }
 
+/** Names of the bundles that exist in this hub, alphabetically. */
+export function listBundles(): string[] {
+  if (!existsSync(BUNDLES)) return [];
+  return readdirSync(BUNDLES)
+    .filter((n) => statSync(join(BUNDLES, n)).isDirectory())
+    .sort();
+}
+
 /**
- * Resolve the bundle to ingest into, creating `default` if that is the target and it does
- * not exist yet. A hub with no bundles must still have somewhere for bytes to land — the
- * alternative is refusing the first ingest anyone ever runs. Only `default` is ever
- * conjured this way: a misspelled explicit name is a mistake, not a request to scaffold.
+ * Resolve the bundle to ingest into. A name the user gave must already exist — which bundle
+ * owns material is their decision, so an unresolvable name is a typo, not a scaffold request.
+ * The one exception is `default` in a hub with no bundles: the first ingest anywhere must
+ * still have somewhere to land, and there is no choice to put to the user yet.
  */
 export function bundleForIngest(name: string): string {
   const dir = join(BUNDLES, name);
   if (existsSync(dir)) return dir;
-  if (name !== DEFAULT_BUNDLE) {
-    console.error(`No such bundle: ${name}`);
-    console.error(`Create it:   khb new-bundle ${name} "<scope>"`);
-    process.exit(1);
+  const have = listBundles();
+  if (name === DEFAULT_BUNDLE && !have.length) {
+    // The scope line lands in outer.index.md, where every agent reads it — so it must not
+    // read as an instruction to reorganize the hub. Splitting `default` into real bundles is
+    // the user's call, exactly like any other bundle decision.
+    createBundle(DEFAULT_BUNDLE, "Unsorted material — where a first ingest lands before any bundle exists; moves out when you say which bundle owns it");
+    console.log(`Created bundles/${DEFAULT_BUNDLE}/ — this hub had no bundles to land in.`);
+    return dir;
   }
-  // The scope line lands in outer.index.md, where every agent reads it — so it must not
-  // read as an instruction to reorganize the hub. Splitting `default` into real bundles is
-  // the user's call, exactly like any other bundle decision.
-  createBundle(DEFAULT_BUNDLE, "Unsorted material — where an ingest with no named bundle lands; moves out when you say which bundle owns it");
-  console.log(`Created bundles/${DEFAULT_BUNDLE}/ — the landing bundle for unrouted material.`);
-  return dir;
+  console.error(`No such bundle: ${name}`);
+  if (have.length) console.error(`This hub has: ${have.join(", ")}`);
+  console.error(`Create it:   khb new-bundle ${name} "<scope>"`);
+  process.exit(1);
 }

package/skills/ingest/SKILL.md

@@ -11,14 +11,60 @@ text exists. Deciding what the text *means* — splitting it into concepts, titl
 tagging, linking, indexing — is the [catalog skill](../catalog/SKILL.md), a separate step
 you run afterwards.
 
-Do not curate here. Do not create, split, or merge bundles here — routing material to a
-bundle that exists is fine, reshaping the hub is not. If you find yourself reading a document
-to understand it, you have left this skill.
-
-## 1. Declare the sources
-
-Ingest is bundle-first: material lands in one bundle, and you say where it comes from. Edit
-`bundles/<bundle>/sources.yaml`:
+Do not curate here. Do not split or merge bundles here, and create one only as the user's
+answer to the question in step 1 — never on your own initiative. If you find yourself reading
+a document to understand it, you have left this skill.
+
+## 1. Settle the bundle — ask, never assume
+
+Ingest is bundle-first: material lands in exactly one bundle, and which bundle owns it is the
+user's decision, not yours. Whenever there is a choice to make, put it to them.
+
+**Take the first of these that applies:**
+
+1. **The user named a bundle** — "re-ingest the real-estate bundle" — → use it, no question
+   asked. If the name does not resolve to a bundle in `bundles/`, say so and ask whether to
+   create it; never scaffold on a guess, and never silently fall back to a similar name.
+2. **No bundle named, and the hub has real bundles** → ask, always, offering the choice
+   explicitly: **an existing bundle** (list them, from `outer.index.md`, with their scope
+   lines so the user can tell them apart) **or a new bundle**. Do not pick for the user,
+   however plainly one bundle seems to own the material — say which you would pick if you
+   have a view, then wait for the answer. Do not offer `default` here and do not mention it;
+   with real bundles on the table it is not one of the options.
+3. **The user answered "a new bundle"** → they name it and give its scope; you run
+   `khb new-bundle <name> "<scope>"`. Creating a bundle is a human decision (`AGENTS.md`), so
+   this branch only ever runs on an explicit answer to the question above.
+4. **No bundle named and nothing to choose between** → `default`, without asking. Two shapes
+   of hub qualify:
+   - **no bundles at all** → `default` is created on the spot; a first ingest should not fail
+     for want of a destination.
+   - **`default` is the only bundle** → it is used as it stands. A one-option question is not
+     a choice, so do not put it to the user.
+
+**The `default` bundle** is only that last case — a landing place in a hub that has no other,
+not an option to fall back on once a real bundle exists. The moment one does, `default` stops
+being a destination for unnamed ingests and case 2 applies. What lands there is ordinary
+bundle content: catalog it like any other. Do **not** graduate it into new bundles on your
+own; material leaves `default` when the user says which bundle owns it.
+
+## 2. Declare the sources — reuse or replace
+
+Once the bundle is settled, settle where the material comes from. If the user already named
+the files, folders or URLs, that is the answer — you still ask the add-or-replace question
+below when the bundle has declarations of its own. **Otherwise, for an existing bundle, read
+its `sources.yaml` first and ask which you are doing:**
+
+- **use what's declared** — re-ingest the paths already in `sources.yaml` (this is what
+  "re-ingest the real-estate bundle" usually means), or
+- **a new path** — the user gives files, folders or URLs; ask whether they are *added* to
+  the declarations or *replace* them before you edit the file.
+
+Quote the current declarations in the question so the answer is informed. A bundle with an
+empty `sources.yaml` has nothing to re-ingest, so there the only answer is a new path — ask
+for it. Do not infer sources from nearby files, do not edit `sources.yaml`, and do not run
+`khb ingest` until the user has answered.
+
+Sources live in `bundles/<bundle>/sources.yaml`:
 
 ```yaml
 sources:
@@ -32,46 +78,17 @@ sources:
     urls:
       - https://example.com/design-doc
   # Types with no scripted ingester are still declared here, for the record —
-  # you pull them yourself in step 3.
+  # you pull them yourself in step 4.
   - type: confluence
     space: PROJX
 ```
 
-If the user has not explicitly named the source locations, inspect the bundle's current
-`sources.yaml`, then ask which files, folders, URLs, or services to ingest. Include any
-existing declarations in the question so the user can confirm or replace them. Do not
-infer sources from nearby files, edit `sources.yaml`, or run `khb ingest` until the user
-answers.
-
 Nothing is copied by declaring a source.
 
-**Which bundle — take the first of these that applies, and do not go further:**
-
-1. **The user named a bundle** → use it. A named bundle that does not exist is an error,
-   not an invitation to create one.
-2. **The hub has bundles and exactly one plainly owns the material** → use it, and say
-   which you picked. If several could own it, ask which — this is the only bundle question
-   ingest ever asks.
-3. **Anything else** — no bundle named, or the hub has no bundles at all → `default`,
-   created on the spot, without asking.
-
-Never ask the user to name or create a bundle *for the ingest to land in*. `default` exists
-so that question never has to be asked at this stage: bytes always have somewhere to go, and
-which bundle owns them is a cheaper decision later, once the text exists and the user can see
-what they actually have.
-
-**The `default` bundle.** When no bundle is named, ingest targets `default` and creates it
-if the hub has none — a first `khb ingest` never fails for want of a destination. It is not a
-way around step 2: when a bundle in the hub plainly owns the material, that bundle wins. What
-lands there is ordinary bundle content: catalog it like any other. Do **not** graduate it into new
-bundles on your own — a bundle is a logical unit the user defines (a person, a team, a
-project), so material leaves `default` only when the user says which bundle owns it. An
-explicitly named bundle that doesn't exist is still an error — only `default` is conjured.
-
-## 2. Run it
+## 3. Run it
 
 ```
-khb ingest                          # no bundle named → the 'default' bundle
+khb ingest                          # only where 'default' is the sole bundle, or none is
 khb ingest <bundle>                 # incremental: unchanged content hashes are skipped
 khb ingest <bundle> --force         # re-acquire everything
 khb ingest <bundle> --skip-ocr      # leave scans and images unread
@@ -135,7 +152,7 @@ pip install -U openai-whisper                # transcription (faster-whisper als
 Install them where `khb` resolves modules from — for a global install that is the khb
 package directory, not your hub. khb prints the exact `cd … && bun add …` to use.
 
-## 3. Sources khb cannot reach
+## 4. Sources khb cannot reach
 
 Anything behind an authenticated API has no scripted ingester, because maintaining API
 wrappers is not what this tool is for. Pull those yourself with the site's MCP server or
@@ -175,7 +192,7 @@ and the text reads thin, garbled, or contradictory, **open the `source:` file an
 directly** — a vision pass over a chart or a scanned table recovers what OCR drops. Rewrite
 the `raw/` file with `extract_tool: claude-vision` and `quality: high` when you do.
 
-## 4. The ledger — `log.md`
+## 5. The ledger — `log.md`
 
 Every bundle keeps its ingest ledger in `log.md` (OKF-reserved, so it is never mistaken for
 a concept doc, and committed, so it survives `raw/` being deleted and re-derived).