boost-skill-cli 1.2.43__tar.gz → 1.2.45__tar.gz

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/.vale/styles/config/vocabularies/boost/accept.txt

@@ -69,6 +69,7 @@ idempotently
 JetBrains
 jonnyeclectic
 jonnyeclectic's
+keyless
 Kotlin
 langchain
 Langfuse

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/CLAUDE.md

@@ -306,7 +306,7 @@ for both — edit items, run `build_roadmap.py`, never touch the HTML by hand.
 ## Architecture
 
 **CLI dispatch.** `boost_cli/cli.py` holds `COMMANDS`, the single
-source-of-truth list of `(name, group, module, summary)` for all 80 commands.
+source-of-truth list of `(name, group, module, summary)` for all 81 commands.
 Each command is implemented as `def cmd_<name_with_underscores>(argv) -> int`
 inside `boost_cli/commands/<module>.py`, and `_dispatch` imports that module
 lazily on invocation — so `boost --help` stays instant and command modules are
@@ -419,6 +419,26 @@ of the three item kinds (see Non-obvious rules above); `core/store.py` owns
 install, uninstall and sync — copying into the canonical store, symlinking into
 each agent's skills dir, and updating the lock file.
 
+- **A published shard is only importable while three things match, and every
+  mismatch is silent if waved through.** `core/shards.py` fetches
+  `manifest.json` from a rolling `shards-latest` **prerelease** on this repo —
+  prerelease and `--latest=false` deliberately, so release-drafter (which
+  resolves the next version from published non-prereleases), the shields badge
+  in the README and setuptools-scm's `--match *[0-9]*` all keep ignoring it.
+  The manifest carries the embedding *space* once at the top, so
+  `shards.incompatible()` can refuse before downloading 129 MB; each row pins
+  the registry *commit*, checked in `shards.sync` and again in
+  `dense.import_shard`, because a stale shard makes `dense.build` mark that tap
+  "reused" forever; and each row's *sha256* is verified over the bytes written.
+  A shard URL off the manifest's own host is refused — a manifest names what
+  boost downloads, so it must not widen where the download goes. `boost tap
+  --at <sha>` exists for the commit rule: `boost quickstart` pins each registry
+  to the commit its vectors describe rather than tapping HEAD and hoping.
+  **Publish keyless vectors only** — a shard exported from a machine holding
+  `VOYAGE_API_KEY` is 1024-d `voyage-4` and can only be imported *or queried*
+  by another key-holder; `scripts/publish_shards.py manifest` refuses to
+  describe two spaces in one file.
+
 **Search has two engines**, both in `core/`: `rag.py` is the always-on,
 zero-dependency BM25 engine (full-content index, auto-builds on first
 search — this is what the required `eval` gate floors at recall@k ≥ 0.78).

{boost_skill_cli-1.2.43/boost_skill_cli.egg-info → boost_skill_cli-1.2.45}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boost-skill-cli
-Version: 1.2.43
+Version: 1.2.45
 Summary: boost — Homebrew for AI coding skills
 Author: Jonathan Reyes
 License-Expression: Apache-2.0
@@ -58,8 +58,8 @@ Cursor and Gemini CLI in a single pass.
 
 ```bash
 pipx install boost-skill-cli
-boost tap --defaults          # 7 curated registries: skills, rules, workflows
-boost search tdd              # full-content BM25 over every skill
+boost quickstart              # taps the 7 starter registries, loads prebuilt vectors
+boost search tdd              # keyword + semantic, fused
 boost install tdd-workflow    # → every agent, version-pinned, one lock file
 ```
 
@@ -172,14 +172,26 @@ ln -s ~/.boost-src/boost ~/bin/boost        # anywhere on PATH works
 itself on your first search, so there is nothing to set up.
 
 BM25 matches words, which fails when your query shares no vocabulary with the
-skill you want. Semantic search fixes that, needs no API key, and takes two
-commands:
+skill you want. Semantic search fixes that and needs no API key — only the
+extra:
 
 ```bash
 pipx inject boost-skill-cli "boost-skill-cli[rag]"   # pip install "boost-skill-cli[rag]" if you used pip
-boost reindex --dense
+boost quickstart                                     # downloads prebuilt vectors
 ```
 
+`boost quickstart` is the fast path: it taps the starter registries **pinned to
+the commits the published vectors were built from**, then downloads and imports
+those vectors. Embedding is ~1.2 s/chunk on CPU — hours for a real corpus —
+and importing the same rows takes 0.12 s, so the difference between the two is
+the difference between semantic search being available and being reachable.
+
+Already tapped? `boost reindex --fetch-shards` does the download half alone,
+and `boost reindex --dense` embeds locally whatever has no published shard.
+Every download is checked against the sha256 in the manifest and refused on a
+mismatch, and shards for a registry that has moved since publication are
+refused rather than merged — stale vectors would otherwise look fresh forever.
+
 When both indexes exist boost fuses the rankings rather than choosing between
 them. See [docs/semantic-search.md](docs/semantic-search.md) for the whole
 setup, what it costs, and how to tell which engine is actually serving.
@@ -362,7 +374,7 @@ at least 95% alike, fold into one badged `×5`, with `^D` to show them again.
 Identity is the description and never the name: `code-reviewer` appears 75 times
 with 42 different descriptions, and those are 42 real skills.
 
-## 80 commands, organized into 8 groups
+## 81 commands, organized into 8 groups
 
 `boost --help` prints the full grouped list. For every flag of every command see
 [`docs/commands.html`](docs/commands.html); for a visual tour,

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/README.md

@@ -16,8 +16,8 @@ Cursor and Gemini CLI in a single pass.
 
 ```bash
 pipx install boost-skill-cli
-boost tap --defaults          # 7 curated registries: skills, rules, workflows
-boost search tdd              # full-content BM25 over every skill
+boost quickstart              # taps the 7 starter registries, loads prebuilt vectors
+boost search tdd              # keyword + semantic, fused
 boost install tdd-workflow    # → every agent, version-pinned, one lock file
 ```
 
@@ -130,14 +130,26 @@ ln -s ~/.boost-src/boost ~/bin/boost        # anywhere on PATH works
 itself on your first search, so there is nothing to set up.
 
 BM25 matches words, which fails when your query shares no vocabulary with the
-skill you want. Semantic search fixes that, needs no API key, and takes two
-commands:
+skill you want. Semantic search fixes that and needs no API key — only the
+extra:
 
 ```bash
 pipx inject boost-skill-cli "boost-skill-cli[rag]"   # pip install "boost-skill-cli[rag]" if you used pip
-boost reindex --dense
+boost quickstart                                     # downloads prebuilt vectors
 ```
 
+`boost quickstart` is the fast path: it taps the starter registries **pinned to
+the commits the published vectors were built from**, then downloads and imports
+those vectors. Embedding is ~1.2 s/chunk on CPU — hours for a real corpus —
+and importing the same rows takes 0.12 s, so the difference between the two is
+the difference between semantic search being available and being reachable.
+
+Already tapped? `boost reindex --fetch-shards` does the download half alone,
+and `boost reindex --dense` embeds locally whatever has no published shard.
+Every download is checked against the sha256 in the manifest and refused on a
+mismatch, and shards for a registry that has moved since publication are
+refused rather than merged — stale vectors would otherwise look fresh forever.
+
 When both indexes exist boost fuses the rankings rather than choosing between
 them. See [docs/semantic-search.md](docs/semantic-search.md) for the whole
 setup, what it costs, and how to tell which engine is actually serving.
@@ -320,7 +332,7 @@ at least 95% alike, fold into one badged `×5`, with `^D` to show them again.
 Identity is the description and never the name: `code-reviewer` appears 75 times
 with 42 different descriptions, and those are 42 real skills.
 
-## 80 commands, organized into 8 groups
+## 81 commands, organized into 8 groups
 
 `boost --help` prints the full grouped list. For every flag of every command see
 [`docs/commands.html`](docs/commands.html); for a visual tour,

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/boost_cli/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '1.2.43'
-__version_tuple__ = version_tuple = (1, 2, 43)
+__version__ = version = '1.2.45'
+__version_tuple__ = version_tuple = (1, 2, 45)
 
 __commit_id__ = commit_id = None

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/boost_cli/cli.py

@@ -1,6 +1,6 @@
 # Copyright the boost contributors.
 # SPDX-License-Identifier: Apache-2.0
-"""boost CLI dispatcher — 80 commands across 8 groups.
+"""boost CLI dispatcher — 81 commands across 8 groups.
 
 Each command lives in boost_cli/commands/<module>.py as
     def cmd_<name_with_underscores>(argv: list[str]) -> int
@@ -117,6 +117,7 @@ COMMANDS = [
     ("health",      "chk", "quality", "Dashboard of skill-environment health"),
     ("trust",       "chk", "quality", "Manage signing keys & verify tap provenance"),
     # Configuration (13)
+    ("quickstart",  "cfg", "quickstart", "Tap the starter registries and load prebuilt vectors"),
     ("config",      "cfg", "configuration", "Display or modify boost configuration"),
     ("clean",       "cfg", "configuration", "Clear stale caches & broken symlinks"),
     ("compact",     "cfg", "configuration", "Shrink tap clones to the files boost indexes"),

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/boost_cli/commands/discovery.py

@@ -270,6 +270,59 @@ def _shard_io(args) -> int:
     return 0
 
 
+def _fetch_shards(args) -> int:
+    """`--fetch-shards`: the published half of the keyless tier, on demand.
+
+    Separate from `boost quickstart` because the two answer different
+    questions. Quickstart sets a machine up; this refreshes vectors on one that
+    is already tapped, including taps quickstart never touches.
+    """
+    from ..core import dense, shards
+    if not registry.list_taps():
+        raise BoostError("no taps configured — nothing to fetch",
+                        hint="`boost quickstart` taps the defaults and fetches "
+                             "their vectors in one pass")
+    manifest = shards.fetch_manifest()
+    why = shards.incompatible(manifest)
+    if why:
+        raise BoostError("published shards cannot serve this machine — %s" % why,
+                        hint=dense.fix_hint(dense.status().get("reason", "")))
+    commits = rag._tap_commits()
+    by_name = {t.name: commits.get(t.safe_name, "") for t in registry.list_taps()}
+    results = shards.sync(list(by_name), by_name, manifest=manifest,
+                          on_event=_shard_event)
+    if args.as_json:
+        print(json.dumps({"shards": results}, indent=2))
+        return 0
+    got = [r for r in results if r["status"] == "imported"]
+    total = sum(int(r.get("chunks") or 0) for r in got)
+    if got:
+        out.ok("imported %d shard(s), %s chunks — no embedding needed"
+               % (len(got), format(total, ",")))
+    else:
+        # Not a success line. Nothing landed, and saying "imported 0" with a
+        # tick reads as a job well done to the one user who most needs to know
+        # their vectors are still missing.
+        out.warn("no published vectors matched your taps")
+    missing = [r["tap"] for r in results if r["status"] != "imported"]
+    if missing:
+        out.info(out.role("%d tap(s) without usable vectors: %s"
+                          % (len(missing), ", ".join(missing[:5])
+                             + (" …" if len(missing) > 5 else "")), "muted"))
+        out.info("embed those locally with `boost reindex --dense`")
+    return 0
+
+
+def _shard_event(tap: str, status: str, detail: str) -> None:
+    """Progress for one shard, quiet enough to run over forty taps."""
+    if status == "downloading":
+        out.info(out.role("fetching %s %s" % (tap, detail), "muted"))
+    elif status in ("failed", "refused"):
+        out.info(out.role("%s: %s%s" % (tap, status,
+                                        " (%s)" % detail if detail else ""),
+                          "muted"))
+
+
 def cmd_reindex(argv):
     """Build/refresh the full-content (RAG) search index over tapped items."""
     p = cliparse.parser(
@@ -287,11 +340,16 @@ def cmd_reindex(argv):
                    help="merge a prebuilt vector shard, skipping the embed "
                         "cost; refused unless it matches this store's backend "
                         "and the tap's current commit")
+    p.add_argument("--fetch-shards", action="store_true",
+                   help="download and import published vectors for every tap "
+                        "that has them, instead of embedding locally")
     p.add_argument("--json", action="store_true", dest="as_json",
                    help="machine-readable output")
     args = p.parse_args(argv)
     if args.export_shard or args.import_shard:
         return _shard_io(args)
+    if args.fetch_shards:
+        return _fetch_shards(args)
     if not registry.list_taps():
         raise BoostError("no taps configured — nothing to index",
                         hint="add the recommended registries with `boost tap --defaults`")

boost_skill_cli-1.2.45/boost_cli/commands/quickstart.py

@@ -0,0 +1,149 @@
+# Copyright the boost contributors.
+# SPDX-License-Identifier: Apache-2.0
+"""`boost quickstart` — one command from empty machine to working search.
+
+WHY A COMMAND AND NOT A DOC. The setup a new user needs is four steps that each
+fail quietly when skipped: tap something, build the BM25 index, install the
+`rag` extra, and embed the catalogue. The last one is the wall — ~1.2 s/chunk on
+CPU, hours for a corpus worth searching — and it is why the keyless semantic
+tier was reachable in principle and not in practice. Published shards remove it,
+but only if the tap sits at the commit its vectors were built from, so tapping
+and fetching cannot be two commands a user is trusted to order correctly.
+
+WHAT IT WILL NOT DO. It never embeds as a side effect. A user who runs a
+"quickstart" and gets an unannounced multi-hour CPU job has been ambushed; the
+taps without a published shard are named, with the one command that would embed
+them, and left alone.
+"""
+from __future__ import annotations
+
+from .. import cliparse, spin
+from ..core import (
+    catalog,
+    complete,
+    config,
+    dense,
+    journal,
+    rag,
+    registry,
+    shards,
+)
+from ..core import output as out
+from ..errors import BoostError
+
+
+def _tap_defaults(pins: dict[str, dict], dry_run: bool) -> list[str]:
+    """Tap the default registries, pinned to a shard's commit when one exists.
+
+    Returns the tap names that are configured afterwards, tapped here or not.
+    Pinning is the whole point: tapping HEAD and then fetching a shard is the
+    ordering that produces a commit mismatch on every registry that moved since
+    the last shard run.
+    """
+    existing = {t.name for t in registry.list_taps()}
+    names = []
+    for default in config.DEFAULT_TAPS:
+        name = str(default["name"])
+        names.append(name)
+        if name in existing:
+            out.info(out.role("%s already tapped" % name, "muted"))
+            continue
+        row = pins.get(name)
+        at = str(row.get("commit")) if row else None
+        if dry_run:
+            out.info("would tap %s%s" % (name, " @ %s" % at[:7] if at else ""))
+            continue
+        try:
+            with spin.Spinner("tapping %s" % name):
+                tap = registry.add(str(default["url"]), curated=True, at=at)
+                entries = catalog.rebuild_tap(tap)
+        except BoostError as exc:
+            out.warn("could not tap %s: %s" % (name, exc.message))
+            continue
+        journal.log("tap", tap.name)
+        out.ok("tapped %s (%d items)%s"
+               % (name, len(entries), " @ %s" % at[:7] if at else ""))
+    return names
+
+
+def _report(results: list[dict]) -> None:
+    """Say what each shard did, and name the remedy for what it did not do."""
+    imported = [r for r in results if r["status"] == "imported"]
+    if imported:
+        total = sum(int(r.get("chunks") or 0) for r in imported)
+        out.ok("imported %d prebuilt shard%s (%s chunks) — no embedding needed"
+               % (len(imported), "" if len(imported) == 1 else "s",
+                  format(total, ",")))
+    for kind, label in (("unpublished", "no published shard"),
+                        ("refused", "shard refused"),
+                        ("failed", "shard failed")):
+        rows = [r for r in results if r["status"] == kind]
+        if not rows:
+            continue
+        for r in rows:
+            detail = r.get("detail") or ""
+            out.info(out.role("%s: %s%s" % (r["tap"], label,
+                                            " (%s)" % detail if detail else ""),
+                              "muted"))
+    left = [r["tap"] for r in results if r["status"] != "imported"]
+    if left:
+        out.info("embed the rest locally when you want to: "
+                 "`boost reindex --dense`")
+
+
+def cmd_quickstart(argv) -> int:
+    """boost quickstart [--no-vectors] [--dry-run]"""
+    p = cliparse.parser(
+        prog="boost quickstart",
+        description="Tap the starter registries and load prebuilt vectors")
+    p.add_argument("--no-vectors", action="store_true",
+                   help="set up taps and keyword search only, skip shards")
+    p.add_argument("--dry-run", action="store_true",
+                   help="print what would happen, change nothing")
+    args = p.parse_args(argv)
+
+    manifest = None
+    pins: dict[str, dict] = {}
+    want_vectors = not args.no_vectors and dense.have_backend()
+    if want_vectors:
+        # Fetched first because it decides how the taps are pinned. A failure
+        # here is not fatal: keyword search is the documented default and works
+        # without a single vector.
+        try:
+            with spin.Spinner("reading the shard manifest"):
+                manifest = shards.fetch_manifest()
+            pins = shards.rows(manifest)
+        except BoostError as exc:
+            out.warn("no published shards: %s" % exc.message)
+            manifest = None
+
+    names = _tap_defaults(pins, args.dry_run)
+    if args.dry_run:
+        planned = [n for n in names if n in pins] if manifest else []
+        out.info("would build the keyword index, then import %d shard(s)"
+                 % len(planned))
+        return 0
+
+    with spin.Spinner("building the keyword index"):
+        stats = rag.build()
+    out.ok("indexed %s items for keyword search" % format(
+        int(stats.get("entries", 0)), ","))
+
+    if manifest is not None:
+        commits = rag._tap_commits()
+        # `_tap_commits` is keyed by safe name; `sync` speaks tap names.
+        by_name = {t.name: commits.get(t.safe_name, "")
+                   for t in registry.list_taps()}
+        results = shards.sync([n for n in names if n in by_name], by_name,
+                              manifest=manifest)
+        _report(results)
+    elif args.no_vectors:
+        out.info(out.role("skipped vectors as asked", "muted"))
+    elif not dense.have_backend():
+        out.info("semantic search needs the extra: "
+                 "`pipx inject boost-skill-cli \"boost-skill-cli[rag]\"`, "
+                 "then `boost quickstart` again")
+
+    complete.refresh_names()
+    out.ok("ready — try `boost search brainstorming`")
+    return 0

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/boost_cli/commands/taps.py

@@ -101,7 +101,12 @@ def cmd_tap(argv) -> int:
                    help="with --catalog: print what would be tapped, tap nothing")
     p.add_argument("--curated", action="store_true",
                    help="mark the tap as curated (★ in listings)")
+    p.add_argument("--at", metavar="SHA",
+                   help="pin the clone to one 40-character commit, as a "
+                        "published vector shard requires")
     args = p.parse_args(argv)
+    if args.at and not args.spec:
+        p.error("--at pins one registry, so it needs a SPEC")
     if not args.spec and not args.defaults and not args.catalog:
         p.error("provide a SPEC, --defaults, or --catalog")
 
@@ -129,10 +134,11 @@ def cmd_tap(argv) -> int:
                    % (tap.name, len(entries), default.get("focus", "")))
     if args.spec:
         with spin.Spinner("cloning %s" % args.spec):
-            tap = registry.add(args.spec, curated=args.curated)
+            tap = registry.add(args.spec, curated=args.curated, at=args.at)
             entries = catalog.rebuild_tap(tap)
         journal.log("tap", tap.name)
-        out.ok("Tapped %s (%d items)" % (tap.name, len(entries)))
+        pin = " @ %s" % args.at[:7] if args.at else ""
+        out.ok("Tapped %s (%d items)%s" % (tap.name, len(entries), pin))
     # Refresh the TAB-completion name cache so `boost install <TAB>` sees
     # whatever this call just tapped instead of the pre-tap snapshot.
     complete.refresh_names()

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/boost_cli/core/gitutil.py

@@ -284,6 +284,44 @@ def head_commit(repo: Path) -> str:
     return proc.stdout.strip() if proc.returncode == 0 else ""
 
 
+def checkout_commit(repo: Path, sha: str) -> None:
+    """Detach `repo` at `sha`, fetching that commit first if it is absent.
+
+    A tap is cloned `--depth 1`, so the commit a published shard was built from
+    is usually *not* in the clone: the fetch is the normal path, not the
+    fallback. GitHub serves a fetch by full SHA for a reachable commit, which
+    is what makes pinning possible without un-shallowing a repo that exists to
+    stay small.
+
+    The fetch keeps `--depth 1` and the blobless filter for the same reason the
+    clone does — pinning a tap must not cost the history and blobs the sparse
+    clone was built to avoid (458 taps held 12 GB before that work).
+    """
+    if not _is_sha(sha):
+        raise BoostError("%r is not a full commit SHA" % sha,
+                         hint="pin with the 40-character hash a manifest names")
+    have = run(["-C", str(repo), "cat-file", "-e", sha + "^{commit}"],
+               check=False)
+    if have.returncode != 0:
+        run(["-C", str(repo), "fetch", "--quiet", "--depth", "1",
+             "--filter=blob:none", "origin", sha])
+    run(["-C", str(repo), "checkout", "--quiet", "--detach", sha])
+    # A detached checkout leaves the sparse cone as it was, but re-applying is
+    # cheap and keeps a pinned clone byte-identical to a freshly cloned one.
+    run(["-C", str(repo), "sparse-checkout", "reapply"], check=False)
+
+
+def _is_sha(value: str) -> bool:
+    """True for a full 40-character hex commit hash.
+
+    Deliberately strict. An abbreviated hash is ambiguous across a fetch, and a
+    branch or tag name is not a pin at all — both would silently give a
+    different tree than the shard was built against, which `import_shard` would
+    then refuse with a confusing commit mismatch.
+    """
+    return len(value) == 40 and all(c in "0123456789abcdef" for c in value.lower())
+
+
 def remote_url(repo: Path) -> str:
     """Return the URL of `repo`'s `origin` remote, or "" if it has none."""
     proc = run(["-C", str(repo), "remote", "get-url", "origin"], check=False)

{boost_skill_cli-1.2.43 → boost_skill_cli-1.2.45}/boost_cli/core/registry.py

@@ -161,9 +161,15 @@ def get(name: str) -> Tap:
                     else "list taps with `boost taps`")
 
 
-def add(spec: str, curated: bool = False) -> Tap:
+def add(spec: str, curated: bool = False, at: str | None = None) -> Tap:
     """Parse `spec`, shallow-clone it, and record the tap in config.
 
+    ``at`` pins the clone to one commit. It exists for published vector shards:
+    a shard is only importable while the tap sits at the commit it was built
+    from (see ``core.shards``), so "tap this registry as the shard expects it"
+    has to be one operation — tapping HEAD and then moving would re-scan the
+    catalog for a tree the vectors do not describe.
+
     Raises BoostError if a tap with that name is already configured.
     """
     name, url = parse_spec(spec)
@@ -176,6 +182,16 @@ def add(spec: str, curated: bool = False) -> Tap:
     if tap.path.exists():
         util.rmtree(tap.path)
     gitutil.clone_shallow(url, tap.path)
+    if at:
+        try:
+            gitutil.checkout_commit(tap.path, at)
+        except BoostError:
+            # A pin that cannot be honoured must not leave a tap silently on
+            # HEAD: the caller asked for one tree and would get another, and
+            # every shard keyed to the pin would then be refused for a commit
+            # mismatch whose cause is three steps back.
+            util.rmtree(tap.path)
+            raise
     problems = policy.check_tap_signing(tap.path)
     if problems:
         util.rmtree(tap.path)  # leave no half-added tap behind