@msareen/knowledge-hub-builder 0.1.3 → 0.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md CHANGED
@@ -135,10 +135,10 @@ From outside, pass `--hub <dir>` or set `$KHB_HUB`.
135
135
  | Command | Purpose |
136
136
  |---|---|
137
137
  | `khb lint` | validate structure against `skills/lint/SKILL.md` |
138
- | `khb upgrade` | refresh this hub's package-owned contract docs |
138
+ | `khb upgrade` | refresh this hub's package-owned contract docs — runs by itself whenever `khb.json`'s stamped version differs from the installed khb, so these docs always match the CLI |
139
139
  | `khb visualize` | regenerate `visualizer/graph.html` |
140
140
  | `khb new-bundle <name>` | scaffold + register a bundle |
141
- | `khb ingest [bundle]` | acquire + extract every source in `sources.yaml` → `raw/`; maintains `log.md`. No bundle named `default`, created if absent |
141
+ | `khb ingest <bundle>` | acquire + extract every source in `sources.yaml` → `raw/`; maintains `log.md`. Name the bundle with none it lists the hub's bundles and stops, unless there is nothing to choose between (no bundles, or only `default`), where it uses `default` |
142
142
  | `khb export <bundle> [dest]` | standalone copy: bundle + common patterns, shareable alone |
143
143
 
144
144
  There is no `khb catalog` command — cataloging is entirely a judgement pass.
@@ -147,10 +147,17 @@ There is no `khb catalog` command — cataloging is entirely a judgement pass.
147
147
 
148
148
  **Ingest** (`skills/ingest/SKILL.md`) is mechanical and flat: `khb ingest <bundle>` pulls
149
149
  every declared source into `raw/` as markdown with a provenance header, extracting
150
- everything it can locally — into the named bundle, or into `default` when none is named — text, PDF, DOCX, ODT, XLSX, PPTX, images by OCR, audio and
150
+ everything it can locally — text, PDF, DOCX, ODT, XLSX, PPTX, images by OCR, audio and
151
151
  video by whisper. Sources behind an authenticated API (Confluence, ADO, git hosts) you pull
152
152
  yourself via MCP/CLI into the same `raw/` shape. Ingest never interprets content.
153
153
 
154
+ When the user has not named a bundle, **ask** which existing bundle owns the material or
155
+ whether to start a new one, and for an existing bundle ask whether to re-ingest what its
156
+ `sources.yaml` already declares or to take a new path. Never choose a destination silently.
157
+ The single exception is a hub with nothing to choose between: with no bundles at all an
158
+ unnamed ingest lands in `default`, created on the spot, and where `default` is the only
159
+ bundle it lands there as it stands.
160
+
154
161
  **Catalog** (`skills/catalog/SKILL.md`) is the judgement half, one bundle at a time: read
155
162
  each `raw/` file, split it into concepts, give each OKF frontmatter, link them, register
156
163
  them in `index.md`. When the runtime supports parallel agents, fan them out over the raw
package/README.md CHANGED
@@ -180,9 +180,13 @@ Create one only when you intend to:
180
180
  khb new-bundle team-payments "Payments team roadmap, incidents, and vendor decisions"
181
181
  ```
182
182
 
183
- KHB never creates, splits, or merges named bundles based on their contents. When
184
- `khb ingest` is run without a bundle name, it uses a `default` landing bundle and creates it
185
- if needed.
183
+ KHB never creates, splits, or merges bundles based on their contents. Name the bundle when
184
+ you ingest: run `khb ingest` without one and it lists the hub's bundles and stops. Ask an
185
+ agent to ingest without naming a bundle and it asks you — existing bundle or new one, and for
186
+ an existing one whether to reuse the paths in its `sources.yaml` or take a new path. An
187
+ unnamed ingest lands somewhere by itself only when there is nothing to choose between: a hub
188
+ with no bundles gets a `default` bundle created on the spot, and a hub whose only bundle is
189
+ `default` uses it. Material moves out of `default` when you say who owns it.
186
190
 
187
191
  Cross-bundle relationships belong in `refs.md`; concept documents must not link directly
188
192
  into another bundle.
@@ -204,6 +208,12 @@ files under `skills/` when developing KHB.
204
208
  changing `bundles/` or `outer.index.md`. `khb export` includes the same compatibility
205
209
  layout in a standalone bundle export.
206
210
 
211
+ The refresh is also automatic: every command that works on a hub first compares the `khb`
212
+ version stamped in `khb.json` against the installed package, and upgrades the hub in place
213
+ if they differ, printing one line to stderr. So updating the package is enough — the hub's
214
+ contract docs can never be a version behind the CLI acting on them. Set
215
+ `KHB_NO_AUTO_UPGRADE=1` to suppress the check and leave the hub as it is.
216
+
207
217
  ## Commands
208
218
 
209
219
  Commands can be run directly or requested through the matching agent skill.
@@ -211,7 +221,7 @@ Commands can be run directly or requested through the matching agent skill.
211
221
  | Command | Purpose |
212
222
  |---|---|
213
223
  | `khb init [dir]` | Create a hub |
214
- | `khb upgrade` | Refresh package-owned contracts and skills |
224
+ | `khb upgrade` | Refresh package-owned contracts and skills (also runs automatically on version drift) |
215
225
  | `khb new-bundle <name> ["scope"]` | Create and register a bundle |
216
226
  | `khb ingest [bundle] [--force]` | Acquire and extract declared sources |
217
227
  | `khb lint` | Validate routing, bundle structure, and OKF metadata |
@@ -225,14 +235,15 @@ Additional ingest flags:
225
235
 
226
236
  Run against a hub outside the current directory with `--hub <dir>` or `$KHB_HUB`.
227
237
 
228
- To update the installed package and then refresh a hub:
238
+ To update the installed package:
229
239
 
230
240
  ```bash
231
241
  bun update -g @msareen/knowledge-hub-builder
232
- cd ~/my-knowledge
233
- khb upgrade
234
242
  ```
235
243
 
244
+ Each hub refreshes itself the next time you run any command in it. `khb upgrade` does the
245
+ same thing on demand, with a fuller report.
246
+
236
247
  ## Hub Layout
237
248
 
238
249
  ```text
package/SPEC.md CHANGED
@@ -92,6 +92,14 @@ agent is opened on the hub folder and must find its rules there, without knowing
92
92
  `khb` is installed. They are package-owned: `khb upgrade` overwrites them in place and
93
93
  leaves `bundles/` and `outer.index.md` alone.
94
94
 
95
+ Copies drift, so the copy is kept honest by the CLI rather than by the user's memory.
96
+ `khb.json` stamps the version that wrote the hub's copies; before running any command that
97
+ touches a hub, `khb` compares that stamp to the installed package version and, if they
98
+ differ, performs the upgrade in place and says so on stderr. Only `init` (no hub yet) and
99
+ `upgrade` (which is the operation) skip the check, and `KHB_NO_AUTO_UPGRADE=1` disables it.
100
+ The invariant it buys: **a hub's contract docs always state the same version as the `khb`
101
+ acting on them** — an agent can never read a protocol the CLI no longer implements.
102
+
95
103
  ### 2b. The package — `@msareen/knowledge-hub-builder`, installed once
96
104
 
97
105
  > **None of the following is in your hub.** This section describes the *installed tool*,
@@ -105,7 +113,7 @@ leaves `bundles/` and `outer.index.md` alone.
105
113
  @msareen/knowledge-hub-builder/ # installed once, globally — NOT part of a hub
106
114
  ├── package.json # bin: khb → scripts/cli.ts
107
115
  ├── scripts/
108
- │ ├── cli.ts # subcommand dispatch; owns the global --hub flag
116
+ │ ├── cli.ts # subcommand dispatch; --hub flag; the version drift check
109
117
  │ ├── init.ts # khb init / khb upgrade
110
118
  │ ├── new-bundle.ts # scaffold from .bundle_template, register in outer.index.md
111
119
  │ ├── export.ts # bundle + common patterns → standalone shareable folder
@@ -116,6 +124,7 @@ leaves `bundles/` and `outer.index.md` alone.
116
124
  │ ├── extract.ts # every local extractor + the content-hash cache
117
125
  │ ├── ledger.ts # log.md read/write
118
126
  │ ├── paths.ts # package-side paths — importing it never needs a hub
127
+ │ ├── upgrade.ts # the refresh itself: `khb upgrade` and the drift check
119
128
  │ └── util.ts # hub resolution + shared helpers
120
129
  ├── .bundle_template/ # copied by `khb new-bundle`
121
130
  ├── templates/hub/ # copied by `khb init`
@@ -217,12 +226,20 @@ Two look-alikes are deliberately excluded, since both would rewire provenance on
217
226
  **copy** (the twin's path still exists — two real sources, both ingested) and an **ambiguous**
218
227
  match (several vanished rows share the hash). Each is reported and left to judgement.
219
228
 
220
- The bundle argument is optional. With none, ingest targets `default` and scaffolds it if the
221
- hub has no such bundle, so a hub with zero bundles still has a landing path. Only that name
222
- is auto-created an explicit name that doesn't resolve is an error, not a scaffold request.
223
- `default` is a holding area, not a tier: its contents are cataloged like any bundle's. Its
224
- material moves only when a human says which bundle should own it a topic emerging inside
225
- it is not itself a reason to move anything.
229
+ A bundle is a logical unit its owner defines, so the destination is a human decision. An
230
+ explicit name that doesn't resolve is an error, not a scaffold request; a bare `khb ingest`
231
+ in a hub that has real bundles prints them and stops, because the CLI cannot ask. The agent
232
+ asks instead (`skills/ingest/SKILL.md` §1–2): which existing bundle, or a new one; then, for
233
+ an existing bundle, whether to re-ingest its declared `sources.yaml` paths or take a new path.
234
+
235
+ `default` survives as the fallback for a hub with nothing to choose between: a bare
236
+ `khb ingest` scaffolds it and lands there when there are **no bundles at all**, and uses it
237
+ when it is the **only** bundle — a one-option question is not a choice, and the first ingest
238
+ anyone runs should not fail for want of a destination. It is not an option once a real bundle
239
+ exists — that was the earlier design, and it bought a pile of material whose ownership
240
+ nobody had decided, which is exactly the decision cataloging then has to make blind.
241
+ `default` is a holding area, not a tier: its contents are cataloged like any bundle's, and
242
+ they move only when a human says which bundle should own them.
226
243
 
227
244
  Every raw file carries its origin, so a lossy extraction is always recoverable:
228
245
 
package/document/faq.md CHANGED
@@ -58,8 +58,12 @@ source, extracts it into markdown under `raw/`, and updates `log.md`.
58
58
  It does not summarize, label, organize, or create concepts. Unchanged content hashes are
59
59
  skipped by default; `--force` reacquires everything.
60
60
 
61
- When no bundle is named, KHB uses the `default` bundle and creates it if necessary. An
62
- explicitly named bundle must already exist.
61
+ A named bundle must already exist. Run without a name and the command lists the hub's bundles
62
+ and stops unless there is nothing to choose between: a hub with no bundles gets a `default`
63
+ landing bundle created so a first ingest still works, and a hub whose only bundle is `default`
64
+ uses it. An agent asked to ingest without a named bundle asks you which
65
+ existing bundle owns the material or whether to create a new one, and for an existing bundle
66
+ whether to re-ingest the paths already in its `sources.yaml` or take a new path.
63
67
 
64
68
  ## Should I run ingestion manually or ask an agent?
65
69
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@msareen/knowledge-hub-builder",
3
- "version": "0.1.3",
3
+ "version": "0.1.5",
4
4
  "description": "Knowledge Hub Builder — a bundle-of-bundles knowledge base you build with an agent",
5
5
  "keywords": ["knowledge-base", "okf", "agent", "wiki", "bundles", "claude", "codex", "rag"],
6
6
  "license": "MIT",
package/scripts/cli.ts CHANGED
@@ -1,13 +1,13 @@
1
1
  #!/usr/bin/env bun
2
2
  // khb — the CLI. Subcommands are loaded lazily: `init` must run before a hub exists,
3
3
  // so nothing that resolves a hub may be imported at module scope.
4
- import { version } from "./lib/paths";
4
+ import { version, findHub, markerIn, MARKER } from "./lib/paths";
5
5
 
6
6
  const COMMANDS: Record<string, { load: () => Promise<unknown>; help: string }> = {
7
7
  init: { load: () => import("./init"), help: "khb init [dir] create a hub here (or in dir)" },
8
8
  upgrade: { load: () => import("./init"), help: "khb upgrade refresh this hub's contract docs" },
9
9
  "new-bundle": { load: () => import("./new-bundle"), help: 'khb new-bundle <name> ["scope"] scaffold a bundle + register it' },
10
- ingest: { load: () => import("./ingest/index"), help: "khb ingest [bundle] [--force] acquire + extract declared sources → raw/ (default: 'default')" },
10
+ ingest: { load: () => import("./ingest/index"), help: "khb ingest <bundle> [--force] acquire + extract declared sources → raw/ (name required once the hub has a bundle other than 'default')" },
11
11
  lint: { load: () => import("./lint"), help: "khb lint validate the hub against skills/lint/SKILL.md" },
12
12
  visualize: { load: () => import("./visualize"), help: "khb visualize [--port N] [--no-open] serve the live bundle graph in your browser; aliases: vis, viz" },
13
13
  export: { load: () => import("./export"), help: "khb export <bundle> [dest] standalone copy of one bundle" },
@@ -59,6 +59,28 @@ if (!entry) {
59
59
  process.exit(1);
60
60
  }
61
61
 
62
+ // Version drift: a hub carries package-owned copies of the agent contract, and a hub
63
+ // stamped at an older version than the installed khb is stating an older contract than
64
+ // the one the CLI now implements. Rather than let the two disagree, refresh the hub in
65
+ // place before running the command — `khb upgrade` touches nothing the user wrote.
66
+ // `init` has no hub yet, `upgrade` does this itself, and $KHB_NO_AUTO_UPGRADE opts out.
67
+ if (cmd !== "init" && cmd !== "upgrade" && !process.env.KHB_NO_AUTO_UPGRADE) {
68
+ const hub = findHub();
69
+ if (hub) {
70
+ const { hubVersion, upgradeHub } = await import("./lib/upgrade");
71
+ // A marker under a pre-rename name is drift too, even at a matching version.
72
+ if (hubVersion(hub) !== version() || markerIn(hub) !== MARKER) {
73
+ const { from, to, pruned, renamed } = upgradeHub(hub);
74
+ // stderr, so a command's own output stays pipeable.
75
+ console.error(
76
+ `khb: hub was built by ${from ?? "an unknown version"}, khb is ${to} — refreshed its contract docs.`,
77
+ );
78
+ if (renamed) console.error(`khb: renamed ${renamed} -> khb.json`);
79
+ if (pruned.length) console.error(`khb: removed (no longer part of the contract): ${pruned.join(", ")}`);
80
+ }
81
+ }
82
+ }
83
+
62
84
  // Subcommand modules parse process.argv.slice(2) themselves — reshape it so they see
63
85
  // their own arguments and not the subcommand name.
64
86
  process.argv = [process.argv[0], process.argv[1], ...argv];
@@ -12,7 +12,7 @@
12
12
  import { parse } from "yaml";
13
13
  import { read, join, HUB } from "../lib/util";
14
14
  import { detail, section, totalElapsed } from "../lib/log";
15
- import { bundleForIngest, DEFAULT_BUNDLE } from "../lib/scaffold";
15
+ import { bundleForIngest, listBundles, DEFAULT_BUNDLE } from "../lib/scaffold";
16
16
  import { readLedger, writeLedger } from "../lib/ledger";
17
17
  import { takeFlag } from "../lib/args";
18
18
  import { ingestFolder } from "./folder";
@@ -43,10 +43,23 @@ if (positional.length > 1) {
43
43
  console.error("Usage: khb ingest [bundle] [--force] [--skip-ocr] [--skip-audio]");
44
44
  process.exit(1);
45
45
  }
46
- // No bundle named `default`, created on the spot if the hub has none. Bytes always have
47
- // somewhere to land; sorting them into real bundles is a later, cheaper decision (a concept
48
- // is one file, and moving it is a `git mv`). Naming a bundle explicitly stays the norm.
49
- const bundle = positional[0] ?? DEFAULT_BUNDLE;
46
+ // No bundle named: which one owns the material is a human decision (AGENTS.md) and a CLI
47
+ // cannot ask, so it stops and shows what the hub has, leaving the choice to whoever is
48
+ // driving. The exception is a hub with nothing to choose between no bundles at all, or
49
+ // only the landing bundle where bytes go to `default` rather than the ingest failing.
50
+ let bundle = positional[0];
51
+ if (!bundle) {
52
+ const have = listBundles();
53
+ const onlyLanding = have.length === 1 && have[0] === DEFAULT_BUNDLE;
54
+ if (have.length && !onlyLanding) {
55
+ console.error("Usage: khb ingest [bundle] [--force] [--skip-ocr] [--skip-audio]");
56
+ console.error(`\nBundles in this hub: ${have.join(", ")}`);
57
+ console.error(`Name the one that owns this material, or start a new one:`);
58
+ console.error(` khb new-bundle <name> "<scope>"`);
59
+ process.exit(1);
60
+ }
61
+ bundle = DEFAULT_BUNDLE;
62
+ }
50
63
 
51
64
  const dir = bundleForIngest(bundle);
52
65
  let cfg: unknown;
package/scripts/init.ts CHANGED
@@ -5,63 +5,23 @@
5
5
  // copied INTO the hub — an agent opened on the hub folder must be able to read them
6
6
  // without knowing where khb is installed. Those copies are package-owned: `upgrade`
7
7
  // overwrites them.
8
- import { cpSync, mkdirSync, writeFileSync, existsSync, statSync, rmSync } from "node:fs";
9
- import { join, resolve, basename, dirname } from "node:path";
10
- import { PKG, HUB_TEMPLATE, MANAGED, RETIRED, MARKER, markerIn, version } from "./lib/paths";
8
+ //
9
+ // The mechanism itself lives in lib/upgrade.ts, because cli.ts also runs it on version
10
+ // drift before any hub command.
11
+ import { cpSync, mkdirSync, existsSync } from "node:fs";
12
+ import { join, resolve, basename } from "node:path";
13
+ import { HUB_TEMPLATE, MARKER, markerIn } from "./lib/paths";
14
+ import { upgradeHub, syncManaged, stamp } from "./lib/upgrade";
11
15
 
12
16
  const upgrading = process.env.KHB_SUBCOMMAND === "upgrade";
13
17
  const [dirArg] = process.argv.slice(2);
14
18
 
15
- /** Copy every package-owned contract file into the hub, replacing what is there. */
16
- function syncManaged(hub: string): string[] {
17
- const done: string[] = [];
18
- for (const f of MANAGED) {
19
- const src = join(PKG, f);
20
- if (!existsSync(src)) continue;
21
- const dest = join(hub, f);
22
- mkdirSync(dirname(dest), { recursive: true });
23
- cpSync(src, dest, { recursive: true, force: true });
24
- done.push(statSync(src).isDirectory() ? `${f}/` : f);
25
- }
26
- return done;
27
- }
28
-
29
- /** Drop package-owned files that later versions stopped shipping. */
30
- function pruneRetired(hub: string): string[] {
31
- const gone: string[] = [];
32
- for (const f of RETIRED) {
33
- const p = join(hub, f);
34
- if (!existsSync(p)) continue;
35
- rmSync(p, { recursive: true, force: true });
36
- gone.push(f);
37
- }
38
- return gone;
39
- }
40
-
41
- function stamp(hub: string, created?: string) {
42
- writeFileSync(
43
- join(hub, MARKER),
44
- JSON.stringify(
45
- { khb: version(), created: created ?? new Date().toISOString(), upgraded: new Date().toISOString() },
46
- null,
47
- 2,
48
- ) + "\n",
49
- );
50
- }
51
-
52
19
  if (upgrading) {
53
20
  const { HUB } = await import("./lib/util"); // resolves the hub, or exits with guidance
54
- // The hub may still carry a marker name from an older version; stamp() writes MARKER,
55
- // so drop the old file rather than leaving the hub with two.
56
- const found = markerIn(HUB)!;
57
- const before = JSON.parse(await Bun.file(join(HUB, found)).text());
58
- if (found !== MARKER) rmSync(join(HUB, found));
59
- const synced = syncManaged(HUB);
60
- const pruned = pruneRetired(HUB);
61
- stamp(HUB, before.created);
62
- console.log(`Upgraded ${HUB}: ${before.khb ?? before.bkr ?? "?"} -> ${version()}`);
21
+ const { from, to, synced, pruned, renamed } = upgradeHub(HUB);
22
+ console.log(`Upgraded ${HUB}: ${from ?? "?"} -> ${to}`);
63
23
  console.log(` refreshed: ${synced.join(", ")}`);
64
- if (found !== MARKER) console.log(` renamed: ${found} -> ${MARKER}`);
24
+ if (renamed) console.log(` renamed: ${renamed} -> ${MARKER}`);
65
25
  if (pruned.length) console.log(` removed (no longer part of the contract): ${pruned.join(", ")}`);
66
26
  console.log(`Your bundles/ and outer.index.md were not touched. Next: khb lint`);
67
27
  } else {
@@ -1,7 +1,7 @@
1
1
  // Package-side paths. Importing this must never require a hub to exist — `khb init`
2
2
  // runs before there is one. Hub-side paths live in util.ts.
3
3
  import { fileURLToPath } from "node:url";
4
- import { join } from "node:path";
4
+ import { join, dirname, resolve } from "node:path";
5
5
  import { readFileSync, existsSync } from "node:fs";
6
6
 
7
7
  /** Root of the installed @msareen/knowledge-hub-builder package (NOT the user's hub). */
@@ -29,6 +29,26 @@ export const LEGACY_MARKERS = ["bkr.json"];
29
29
  export const markerIn = (dir: string): string | undefined =>
30
30
  [MARKER, ...LEGACY_MARKERS].find((m) => existsSync(join(dir, m)));
31
31
 
32
+ /**
33
+ * The hub this invocation acts on, or undefined if there is none. Precedence:
34
+ * $KHB_HUB (set from --hub by cli.ts) > nearest ancestor of cwd holding a marker.
35
+ * Soft by design — util.ts turns "none" into an error with guidance, while the version
36
+ * drift check in cli.ts must stay silent for commands that run outside a hub.
37
+ */
38
+ export function findHub(): string | undefined {
39
+ const explicit = process.env.KHB_HUB;
40
+ if (explicit) {
41
+ const dir = resolve(explicit);
42
+ return markerIn(dir) ? dir : undefined;
43
+ }
44
+ for (let dir = process.cwd(); ; ) {
45
+ if (markerIn(dir)) return dir;
46
+ const up = dirname(dir);
47
+ if (up === dir) return undefined;
48
+ dir = up;
49
+ }
50
+ }
51
+
32
52
  /**
33
53
  * Package-owned files copied into every hub by `khb init` and refreshed by
34
54
  * `khb upgrade`. These are the agent contract — the hub needs its own copies so an
@@ -1,9 +1,12 @@
1
- // Bundle creation, shared by `khb new-bundle` and by ingest's default-bundle fallback.
2
- // One implementation so a bundle born from a bare `khb ingest` is indistinguishable from
3
- // one the user named: same template, same {{name}} substitution, same outer.index.md row.
1
+ // Bundle creation and lookup, shared by `khb new-bundle` and `khb ingest`.
2
+ // One implementation so every bundle is born the same way: same template, same {{name}}
3
+ // substitution, same outer.index.md row — and nothing ever conjures one implicitly.
4
4
  import { cpSync, readFileSync, writeFileSync, readdirSync, statSync, existsSync } from "node:fs";
5
5
  import { HUB, BUNDLES, TEMPLATE, join } from "./util";
6
6
 
7
+ // The landing bundle: where a bare `khb ingest` goes in a hub with nothing to choose between
8
+ // — conjured when the hub has no bundles at all, reused when it is the only one. With a real
9
+ // bundle present the destination is a choice and the user makes it, so nothing is auto-created.
7
10
  export const DEFAULT_BUNDLE = "default";
8
11
 
9
12
  export const VALID_NAME = /^[a-z0-9][a-z0-9-]*$/;
@@ -33,24 +36,34 @@ export function createBundle(name: string, scope: string): string {
33
36
  return dest;
34
37
  }
35
38
 
39
+ /** Names of the bundles that exist in this hub, alphabetically. */
40
+ export function listBundles(): string[] {
41
+ if (!existsSync(BUNDLES)) return [];
42
+ return readdirSync(BUNDLES)
43
+ .filter((n) => statSync(join(BUNDLES, n)).isDirectory())
44
+ .sort();
45
+ }
46
+
36
47
  /**
37
- * Resolve the bundle to ingest into, creating `default` if that is the target and it does
38
- * not exist yet. A hub with no bundles must still have somewhere for bytes to land — the
39
- * alternative is refusing the first ingest anyone ever runs. Only `default` is ever
40
- * conjured this way: a misspelled explicit name is a mistake, not a request to scaffold.
48
+ * Resolve the bundle to ingest into. A name the user gave must already exist which bundle
49
+ * owns material is their decision, so an unresolvable name is a typo, not a scaffold request.
50
+ * The one exception is `default` in a hub with no bundles: the first ingest anywhere must
51
+ * still have somewhere to land, and there is no choice to put to the user yet.
41
52
  */
42
53
  export function bundleForIngest(name: string): string {
43
54
  const dir = join(BUNDLES, name);
44
55
  if (existsSync(dir)) return dir;
45
- if (name !== DEFAULT_BUNDLE) {
46
- console.error(`No such bundle: ${name}`);
47
- console.error(`Create it: khb new-bundle ${name} "<scope>"`);
48
- process.exit(1);
56
+ const have = listBundles();
57
+ if (name === DEFAULT_BUNDLE && !have.length) {
58
+ // The scope line lands in outer.index.md, where every agent reads it so it must not
59
+ // read as an instruction to reorganize the hub. Splitting `default` into real bundles is
60
+ // the user's call, exactly like any other bundle decision.
61
+ createBundle(DEFAULT_BUNDLE, "Unsorted material — where a first ingest lands before any bundle exists; moves out when you say which bundle owns it");
62
+ console.log(`Created bundles/${DEFAULT_BUNDLE}/ — this hub had no bundles to land in.`);
63
+ return dir;
49
64
  }
50
- // The scope line lands in outer.index.md, where every agent reads it — so it must not
51
- // read as an instruction to reorganize the hub. Splitting `default` into real bundles is
52
- // the user's call, exactly like any other bundle decision.
53
- createBundle(DEFAULT_BUNDLE, "Unsorted material — where an ingest with no named bundle lands; moves out when you say which bundle owns it");
54
- console.log(`Created bundles/${DEFAULT_BUNDLE}/ — the landing bundle for unrouted material.`);
55
- return dir;
65
+ console.error(`No such bundle: ${name}`);
66
+ if (have.length) console.error(`This hub has: ${have.join(", ")}`);
67
+ console.error(`Create it: khb new-bundle ${name} "<scope>"`);
68
+ process.exit(1);
56
69
  }
@@ -0,0 +1,93 @@
1
+ // The upgrade mechanism: refresh a hub's package-owned contract docs to match the
2
+ // installed khb. Lives here rather than in init.ts because two callers need it — the
3
+ // explicit `khb upgrade`, and the drift check cli.ts runs before every hub command.
4
+ //
5
+ // Nothing here may import util.ts: the drift check runs before a hub is resolved, and
6
+ // util.ts resolves one or exits.
7
+ import { cpSync, mkdirSync, writeFileSync, readFileSync, existsSync, statSync, rmSync } from "node:fs";
8
+ import { join, dirname } from "node:path";
9
+ import { PKG, MANAGED, RETIRED, MARKER, markerIn, version } from "./paths";
10
+
11
+ export type UpgradeResult = {
12
+ /** Version recorded in the hub's marker before the upgrade, if it recorded one. */
13
+ from?: string;
14
+ to: string;
15
+ synced: string[];
16
+ pruned: string[];
17
+ /** Set when a legacy marker name was renamed to khb.json. */
18
+ renamed?: string;
19
+ };
20
+
21
+ /** Read the khb version a hub was last stamped with. Undefined if it records none. */
22
+ export function hubVersion(hub: string): string | undefined {
23
+ const found = markerIn(hub);
24
+ if (!found) return undefined;
25
+ try {
26
+ const marker = JSON.parse(readFileSync(join(hub, found), "utf8"));
27
+ return marker.khb ?? marker.bkr; // bkr: the pre-rename field name
28
+ } catch {
29
+ return undefined; // unreadable marker — treat as drifted, upgrade will restamp it
30
+ }
31
+ }
32
+
33
+ /** Copy every package-owned contract file into the hub, replacing what is there. */
34
+ export function syncManaged(hub: string): string[] {
35
+ const done: string[] = [];
36
+ for (const f of MANAGED) {
37
+ const src = join(PKG, f);
38
+ if (!existsSync(src)) continue;
39
+ const dest = join(hub, f);
40
+ mkdirSync(dirname(dest), { recursive: true });
41
+ cpSync(src, dest, { recursive: true, force: true });
42
+ done.push(statSync(src).isDirectory() ? `${f}/` : f);
43
+ }
44
+ return done;
45
+ }
46
+
47
+ /** Drop package-owned files that later versions stopped shipping. */
48
+ function pruneRetired(hub: string): string[] {
49
+ const gone: string[] = [];
50
+ for (const f of RETIRED) {
51
+ const p = join(hub, f);
52
+ if (!existsSync(p)) continue;
53
+ rmSync(p, { recursive: true, force: true });
54
+ gone.push(f);
55
+ }
56
+ return gone;
57
+ }
58
+
59
+ /** Write khb.json with the installed version, preserving the hub's creation date. */
60
+ export function stamp(hub: string, created?: string) {
61
+ writeFileSync(
62
+ join(hub, MARKER),
63
+ JSON.stringify(
64
+ { khb: version(), created: created ?? new Date().toISOString(), upgraded: new Date().toISOString() },
65
+ null,
66
+ 2,
67
+ ) + "\n",
68
+ );
69
+ }
70
+
71
+ /**
72
+ * Bring `hub`'s package-owned files up to the installed version. Touches only what the
73
+ * package owns: bundles/, outer.index.md and anything else the user wrote are untouched.
74
+ */
75
+ export function upgradeHub(hub: string): UpgradeResult {
76
+ // The hub may still carry a marker name from an older version; stamp() writes MARKER,
77
+ // so drop the old file rather than leaving the hub with two.
78
+ const found = markerIn(hub)!;
79
+ let created: string | undefined;
80
+ let from: string | undefined;
81
+ try {
82
+ const before = JSON.parse(readFileSync(join(hub, found), "utf8"));
83
+ created = before.created;
84
+ from = before.khb ?? before.bkr;
85
+ } catch {
86
+ /* unreadable marker: rewritten below with today's date */
87
+ }
88
+ if (found !== MARKER) rmSync(join(hub, found));
89
+ const synced = syncManaged(hub);
90
+ const pruned = pruneRetired(hub);
91
+ stamp(hub, created);
92
+ return { from, to: version(), synced, pruned, renamed: found === MARKER ? undefined : found };
93
+ }
@@ -1,7 +1,7 @@
1
1
  import { readdirSync, readFileSync, existsSync, statSync, mkdirSync, writeFileSync } from "node:fs";
2
2
  import { createHash } from "node:crypto";
3
- import { join, basename, dirname, resolve } from "node:path";
4
- import { MARKER, markerIn } from "./paths";
3
+ import { join, basename, resolve } from "node:path";
4
+ import { MARKER, markerIn, findHub } from "./paths";
5
5
 
6
6
  /**
7
7
  * Find the hub root — the folder holding khb.json, outer.index.md and bundles/.
@@ -10,20 +10,12 @@ import { MARKER, markerIn } from "./paths";
10
10
  * ancestor of cwd containing the marker.
11
11
  */
12
12
  function resolveHub(): string {
13
+ const found = findHub();
14
+ if (found) return found;
13
15
  const explicit = process.env.KHB_HUB;
14
16
  if (explicit) {
15
- const dir = resolve(explicit);
16
- if (!markerIn(dir)) {
17
- console.error(`Not a KHB hub (no ${MARKER}): ${dir}`);
18
- process.exit(1);
19
- }
20
- return dir;
21
- }
22
- for (let dir = process.cwd(); ; ) {
23
- if (markerIn(dir)) return dir;
24
- const up = dirname(dir);
25
- if (up === dir) break;
26
- dir = up;
17
+ console.error(`Not a KHB hub (no ${MARKER}): ${resolve(explicit)}`);
18
+ process.exit(1);
27
19
  }
28
20
  console.error(`No KHB hub found in ${process.cwd()} or any parent directory.`);
29
21
  console.error(`Create one: khb init <dir>`);
@@ -11,14 +11,60 @@ text exists. Deciding what the text *means* — splitting it into concepts, titl
11
11
  tagging, linking, indexing — is the [catalog skill](../catalog/SKILL.md), a separate step
12
12
  you run afterwards.
13
13
 
14
- Do not curate here. Do not create, split, or merge bundles here routing material to a
15
- bundle that exists is fine, reshaping the hub is not. If you find yourself reading a document
16
- to understand it, you have left this skill.
17
-
18
- ## 1. Declare the sources
19
-
20
- Ingest is bundle-first: material lands in one bundle, and you say where it comes from. Edit
21
- `bundles/<bundle>/sources.yaml`:
14
+ Do not curate here. Do not split or merge bundles here, and create one only as the user's
15
+ answer to the question in step 1 never on your own initiative. If you find yourself reading
16
+ a document to understand it, you have left this skill.
17
+
18
+ ## 1. Settle the bundle — ask, never assume
19
+
20
+ Ingest is bundle-first: material lands in exactly one bundle, and which bundle owns it is the
21
+ user's decision, not yours. Whenever there is a choice to make, put it to them.
22
+
23
+ **Take the first of these that applies:**
24
+
25
+ 1. **The user named a bundle** — "re-ingest the real-estate bundle" — → use it, no question
26
+ asked. If the name does not resolve to a bundle in `bundles/`, say so and ask whether to
27
+ create it; never scaffold on a guess, and never silently fall back to a similar name.
28
+ 2. **No bundle named, and the hub has real bundles** → ask, always, offering the choice
29
+ explicitly: **an existing bundle** (list them, from `outer.index.md`, with their scope
30
+ lines so the user can tell them apart) **or a new bundle**. Do not pick for the user,
31
+ however plainly one bundle seems to own the material — say which you would pick if you
32
+ have a view, then wait for the answer. Do not offer `default` here and do not mention it;
33
+ with real bundles on the table it is not one of the options.
34
+ 3. **The user answered "a new bundle"** → they name it and give its scope; you run
35
+ `khb new-bundle <name> "<scope>"`. Creating a bundle is a human decision (`AGENTS.md`), so
36
+ this branch only ever runs on an explicit answer to the question above.
37
+ 4. **No bundle named and nothing to choose between** → `default`, without asking. Two shapes
38
+ of hub qualify:
39
+ - **no bundles at all** → `default` is created on the spot; a first ingest should not fail
40
+ for want of a destination.
41
+ - **`default` is the only bundle** → it is used as it stands. A one-option question is not
42
+ a choice, so do not put it to the user.
43
+
44
+ **The `default` bundle** is only that last case — a landing place in a hub that has no other,
45
+ not an option to fall back on once a real bundle exists. The moment one does, `default` stops
46
+ being a destination for unnamed ingests and case 2 applies. What lands there is ordinary
47
+ bundle content: catalog it like any other. Do **not** graduate it into new bundles on your
48
+ own; material leaves `default` when the user says which bundle owns it.
49
+
50
+ ## 2. Declare the sources — reuse or replace
51
+
52
+ Once the bundle is settled, settle where the material comes from. If the user already named
53
+ the files, folders or URLs, that is the answer — you still ask the add-or-replace question
54
+ below when the bundle has declarations of its own. **Otherwise, for an existing bundle, read
55
+ its `sources.yaml` first and ask which you are doing:**
56
+
57
+ - **use what's declared** — re-ingest the paths already in `sources.yaml` (this is what
58
+ "re-ingest the real-estate bundle" usually means), or
59
+ - **a new path** — the user gives files, folders or URLs; ask whether they are *added* to
60
+ the declarations or *replace* them before you edit the file.
61
+
62
+ Quote the current declarations in the question so the answer is informed. A bundle with an
63
+ empty `sources.yaml` has nothing to re-ingest, so there the only answer is a new path — ask
64
+ for it. Do not infer sources from nearby files, do not edit `sources.yaml`, and do not run
65
+ `khb ingest` until the user has answered.
66
+
67
+ Sources live in `bundles/<bundle>/sources.yaml`:
22
68
 
23
69
  ```yaml
24
70
  sources:
@@ -32,46 +78,17 @@ sources:
32
78
  urls:
33
79
  - https://example.com/design-doc
34
80
  # Types with no scripted ingester are still declared here, for the record —
35
- # you pull them yourself in step 3.
81
+ # you pull them yourself in step 4.
36
82
  - type: confluence
37
83
  space: PROJX
38
84
  ```
39
85
 
40
- If the user has not explicitly named the source locations, inspect the bundle's current
41
- `sources.yaml`, then ask which files, folders, URLs, or services to ingest. Include any
42
- existing declarations in the question so the user can confirm or replace them. Do not
43
- infer sources from nearby files, edit `sources.yaml`, or run `khb ingest` until the user
44
- answers.
45
-
46
86
  Nothing is copied by declaring a source.
47
87
 
48
- **Which bundle take the first of these that applies, and do not go further:**
49
-
50
- 1. **The user named a bundle** → use it. A named bundle that does not exist is an error,
51
- not an invitation to create one.
52
- 2. **The hub has bundles and exactly one plainly owns the material** → use it, and say
53
- which you picked. If several could own it, ask which — this is the only bundle question
54
- ingest ever asks.
55
- 3. **Anything else** — no bundle named, or the hub has no bundles at all → `default`,
56
- created on the spot, without asking.
57
-
58
- Never ask the user to name or create a bundle *for the ingest to land in*. `default` exists
59
- so that question never has to be asked at this stage: bytes always have somewhere to go, and
60
- which bundle owns them is a cheaper decision later, once the text exists and the user can see
61
- what they actually have.
62
-
63
- **The `default` bundle.** When no bundle is named, ingest targets `default` and creates it
64
- if the hub has none — a first `khb ingest` never fails for want of a destination. It is not a
65
- way around step 2: when a bundle in the hub plainly owns the material, that bundle wins. What
66
- lands there is ordinary bundle content: catalog it like any other. Do **not** graduate it into new
67
- bundles on your own — a bundle is a logical unit the user defines (a person, a team, a
68
- project), so material leaves `default` only when the user says which bundle owns it. An
69
- explicitly named bundle that doesn't exist is still an error — only `default` is conjured.
70
-
71
- ## 2. Run it
88
+ ## 3. Run it
72
89
 
73
90
  ```
74
- khb ingest # no bundle named the 'default' bundle
91
+ khb ingest # only where 'default' is the sole bundle, or none is
75
92
  khb ingest <bundle> # incremental: unchanged content hashes are skipped
76
93
  khb ingest <bundle> --force # re-acquire everything
77
94
  khb ingest <bundle> --skip-ocr # leave scans and images unread
@@ -135,7 +152,7 @@ pip install -U openai-whisper # transcription (faster-whisper als
135
152
  Install them where `khb` resolves modules from — for a global install that is the khb
136
153
  package directory, not your hub. khb prints the exact `cd … && bun add …` to use.
137
154
 
138
- ## 3. Sources khb cannot reach
155
+ ## 4. Sources khb cannot reach
139
156
 
140
157
  Anything behind an authenticated API has no scripted ingester, because maintaining API
141
158
  wrappers is not what this tool is for. Pull those yourself with the site's MCP server or
@@ -175,7 +192,7 @@ and the text reads thin, garbled, or contradictory, **open the `source:` file an
175
192
  directly** — a vision pass over a chart or a scanned table recovers what OCR drops. Rewrite
176
193
  the `raw/` file with `extract_tool: claude-vision` and `quality: high` when you do.
177
194
 
178
- ## 4. The ledger — `log.md`
195
+ ## 5. The ledger — `log.md`
179
196
 
180
197
  Every bundle keeps its ingest ledger in `log.md` (OKF-reserved, so it is never mistaken for
181
198
  a concept doc, and committed, so it survives `raw/` being deleted and re-derived).