boost-skill-cli 1.2.54__tar.gz → 1.2.56__tar.gz

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/CLAUDE.md

@@ -584,6 +584,26 @@ Consequences for code you write here:
   real install are repeats and are embedded once — so a progress total taken
   from row count would over-report the work.
 
+- **Dense reuse is decided at TWO levels, and the second one is why `chunks`
+  carries a `digest`.** A tap whose git commit is unchanged is skipped whole.
+  A tap whose commit *moved* is not therefore changed: `_split_by_digest`
+  compares each entry's `content` digest against the one stored beside its
+  chunks, and only the entries that really differ are deleted and re-embedded.
+  Measured on a real 464-tap install: 19 taps drifted in 6.85 h and re-embedded
+  112,081 chunks for 659 chunks of actual change (**170x**), and **10 of the
+  19 changed nothing boost indexes at all** — badge `.json`, star-history
+  `.svg`, CI `.yml` — for 40.0% of the bill. Three rules make it safe, and
+  each was a real bug waiting: an entry is reused only when **every** stored
+  chunk carries the current digest, because a half-written entry must not be
+  served as a mixture of two versions' vectors; `_prune_stale_entries` drops
+  rows for entries the catalog dropped, which whole-tap deletion used to sweep
+  for free; and a **missing digest is never a match**, because two absences
+  are two unknowns. The digest travels in a shard too — without it an imported
+  shard re-embeds on the next build, spending the CPU the download existed to
+  save. The lookup is scoped by tap through the `chunks_tap` index: reading
+  the whole table instead measured 3.75 s cold on a 657,587-chunk store, and
+  ran even on the build where nothing had changed.
+
 - **Nothing on the search path may count `chunks`.** `SELECT COUNT(*)` scans the
   `chunks_tap` covering index — 8,419 pages / 34.5 MB, measured 1.94 s — and
   both `ready()` and `status()` did it on every search, the latter only to word

{boost_skill_cli-1.2.54/boost_skill_cli.egg-info → boost_skill_cli-1.2.56}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boost-skill-cli
-Version: 1.2.54
+Version: 1.2.56
 Summary: boost — Homebrew for AI coding skills
 Author: Jonathan Reyes
 License-Expression: Apache-2.0
@@ -448,8 +448,10 @@ purely local operation, for CI images and air-gapped machines.
 | Claude Code | `claude` | `claude mcp add` → its user-scope MCP config |
 | Gemini CLI | `gemini` | `gemini mcp add -s user` → `~/.gemini/settings.json` |
 
-Six tools are exposed: `boost_search`, `boost_list`, `boost_info`,
-`boost_install`, `boost_doctor` and `boost_discover_github`.
+Seven tools are exposed: `boost_search`, `boost_list`, `boost_info`,
+`boost_read`, `boost_install`, `boost_doctor` and
+`boost_discover_github`. `boost_read` is the one that returns an item's
+actual text — without it, installing was the only way to read a skill.
 
 The server also returns MCP `instructions`, telling the agent when boost is
 relevant. The two hosts place those differently, and it matters. Claude Code

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/README.md

@@ -406,8 +406,10 @@ purely local operation, for CI images and air-gapped machines.
 | Claude Code | `claude` | `claude mcp add` → its user-scope MCP config |
 | Gemini CLI | `gemini` | `gemini mcp add -s user` → `~/.gemini/settings.json` |
 
-Six tools are exposed: `boost_search`, `boost_list`, `boost_info`,
-`boost_install`, `boost_doctor` and `boost_discover_github`.
+Seven tools are exposed: `boost_search`, `boost_list`, `boost_info`,
+`boost_read`, `boost_install`, `boost_doctor` and
+`boost_discover_github`. `boost_read` is the one that returns an item's
+actual text — without it, installing was the only way to read a skill.
 
 The server also returns MCP `instructions`, telling the agent when boost is
 relevant. The two hosts place those differently, and it matters. Claude Code

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/boost_cli/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '1.2.54'
-__version_tuple__ = version_tuple = (1, 2, 54)
+__version__ = version = '1.2.56'
+__version_tuple__ = version_tuple = (1, 2, 56)
 
 __commit_id__ = commit_id = None

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/boost_cli/commands/configuration.py

@@ -1169,6 +1169,47 @@ def _tool_info(args: dict):
     return "\n".join(lines), False
 
 
+def _tool_read(args: dict):
+    """`boost_read` — the item's own text, so an agent can look before it acts.
+
+    The gap this closes: ``boost_info`` returns the same one-line description
+    ``boost_search`` already returned, so an agent deciding whether a procedure
+    is worth adopting had one sentence written by whoever published it, and its
+    only route to the actual steps was to install into the user's real
+    ``~/.agents/skills`` and read it off disk. That inverts the surface's own
+    pitch — ``boost_search`` spends 10-15 s of rerank so the top result is worth
+    acting on, and then nothing let the agent look at it.
+
+    The body is also the only thing separating a written skill from a generated
+    stub, and the catalogue holds both: two ranked hits share the description
+    "To optimize **Skill**, we enforce the following foundational rules:" — an
+    unfilled template — and ``superpowers-lab`` is an 868-byte SKILL.md whose
+    Instructions section is its own name echoed back. Both index, rank and
+    render exactly like real results. The eval gate cannot catch it either:
+    ``golden.jsonl`` grades by *name*, and a stub matches its own name
+    perfectly.
+
+    Read-only, and that is the whole point rather than a side note — this is
+    the tool that exists so an agent does *not* have to install to look, so it
+    must never become a second install path. It reuses ``boost cat``'s
+    resolution (``info._resolve_text``): the installed copy when there is one,
+    the tap source otherwise, with the same integrity and quarantine rules. A
+    second resolver would be a second opinion about what "this item" means,
+    disagreeing with the tool it is advising about.
+
+    ``cat`` rather than ``explain``: cat is offline, deterministic and free,
+    while ``explain`` goes through ``core/ai.py`` and costs seconds plus a key.
+    The cheap answer must not sit behind the expensive one's latency — the same
+    reason ``boost_list`` never reads the catalog.
+    """
+    from . import info
+    name = str(args.get("name", ""))
+    if not name.strip():
+        return "boost_read needs a name — pass one as boost_search returned it", True
+    text, kind, lock, _cat = info._resolve_text(name)
+    return mcp.read_reply(name, text, kind=kind, installed=lock is not None), False
+
+
 def _tool_install(args: dict):
     entry = catalog.resolve_one(str(args.get("name", "")))
     res = store.install(entry)
@@ -1369,19 +1410,54 @@ REGISTRY.register(
     _tool_list)
 REGISTRY.register(
     "boost_info",
-    "The whole picture of one skill, rule or workflow by name — what it does, "
-    "its kind, the tap it came "
-    "from, its version, and whether it is already installed — so you can "
-    "commit or move on without guessing. Reach for it when a name arrives from "
-    "somewhere else: a teammate, a README, a repo you are reading. You do not "
-    "need this between a search and an install — boost_search already returns "
-    "each match's kind and description.",
+    # This used to promise "the whole picture" and return five fields, one of
+    # which — description — is byte-identical to the one-liner boost_search
+    # already returned. So the tool added `installed: no`, which the search
+    # reply already marks, to an agent that had been told to expect what the
+    # item does. An overstated description costs a wasted call and teaches an
+    # agent to discount the rest of the surface, which is the expensive half.
+    # It now says what it returns (status and provenance) and names the tool
+    # that returns the body.
+    "Where one skill, rule or workflow came from, by name, and whether it is "
+    "already here — its kind, tap, version, install state, and which agents "
+    "it reached. It is the STATUS of an item, not its contents: the description "
+    "it echoes is the same one-liner boost_search returned, so to read what "
+    "the item actually says, call boost_read. Reach for boost_info when a "
+    "name arrives from somewhere else — a teammate, a README, a repo you are "
+    "reading — and you need to know whether this machine already has it. You "
+    "do not need this between a search and an install: boost_search already "
+    "returns each match's kind, description and installed marker.",
     {"type": "object",
      "properties": {"name": {"type": "string",
                              "description": "the item's name, as boost_search "
                              "or boost_list returned it"}},
      "required": ["name"]},
     _tool_info)
+REGISTRY.register(
+    "boost_read",
+    "Read the actual procedure before you commit to it — the item's own text, "
+    "not its one-line description. This is the tool that makes looking cheaper "
+    "than installing: without it the only way to see what a skill actually "
+    "says is to install it into the user's machine and read it off disk, which "
+    "is a change to their setup made in order to answer a question. Reach for "
+    "it on the top hit from boost_search before boost_install, and whenever a "
+    "one-liner is doing more selling than describing. What you are checking "
+    "for is specific: the catalogue is indexed rather than reviewed, so it "
+    "holds unfilled templates and stubs that echo their own name back, and "
+    "those rank and render exactly like a procedure someone debugged. The body "
+    "is the only thing that tells them apart. Read-only and offline — it "
+    "installs nothing, changes nothing, and unlike boost_search costs no "
+    "rerank, so it is fast. Long items are truncated at a stated character "
+    "count and the reply says so and names the command that returns the rest; "
+    "a body that arrives whole is whole.",
+    {"type": "object",
+     "properties": {"name": {"type": "string",
+                             "description": "the item's name, as boost_search "
+                             "or boost_list returned it; qualify with the tap "
+                             "(\"owner/repo:name\") when the same name exists "
+                             "in more than one"}},
+     "required": ["name"]},
+    _tool_read)
 REGISTRY.register(
     "boost_install",
     "Turn a skill you found with boost_search into permanent capability: copied "

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/boost_cli/commands/discovery.py

@@ -423,8 +423,29 @@ def cmd_reindex(argv):
                          "`boost reindex --dense --force`."
                          % len(dense_stats["reused"]))
             else:
-                out.ok("embedded %d passages (%s) into the dense vector store"
-                       % (dense_stats["chunks"], dense_stats["provider"]))
+                # `chunks` is the store TOTAL, not this run's work, so on an
+                # incremental run "embedded 657,587 passages" described an
+                # afternoon of CPU that did not happen. `added` is what this
+                # run actually paid for.
+                added = dense_stats.get("added", dense_stats["chunks"])
+                out.ok("embedded %d passage%s (%s) — the dense store holds %d"
+                       % (added, "" if added == 1 else "s",
+                          dense_stats["provider"], dense_stats["chunks"]))
+                # The saving the tap-level line cannot show: a tap appears in
+                # `reindexed` the moment its commit moves, so a run that reused
+                # every entry inside three moved taps otherwise reads as three
+                # taps fully re-embedded.
+                kept = dense_stats.get("reused_entries") or 0
+                if kept:
+                    out.info(out.role(
+                        "reused %d unchanged item%s inside %d moved tap%s — "
+                        "only %d item%s changed"
+                        % (kept, "" if kept == 1 else "s",
+                           len(dense_stats["reindexed"]),
+                           "" if len(dense_stats["reindexed"]) == 1 else "s",
+                           dense_stats.get("embedded_entries") or 0,
+                           "" if dense_stats.get("embedded_entries") == 1
+                           else "s"), "muted"))
             if dense_stats.get("quantized"):
                 out.ok("quantized %d existing vectors — dense search no longer "
                        "scans the whole store on every query"

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/boost_cli/commands/info.py

@@ -149,7 +149,21 @@ def _resolve_text(name: str):
     found = lockfile.find_any(bare)
     if found is None or found[0] == "skill":
         path, lock, cat = _resolve_skill_md(name)
-        return _read(path), "skill", lock, cat
+        # The lock file is the authority for an INSTALLED item, and for one
+        # that is not installed it has no opinion at all — so a rule or
+        # workflow that is only in a tap arrived here and was reported as a
+        # skill, contradicting this function's own "any kind" contract. The
+        # catalog entry knows, and `_resolve_skill_md` has already resolved it:
+        # it returns a catalog entry on exactly the branch where `lock` is
+        # None, so no second lookup is needed (and a fallback for `cat is None`
+        # here would guard a state that function cannot produce).
+        #
+        # Every caller before `boost_read` discarded this value (`_kind`), so
+        # the wrong answer was invisible: the resolution and the text were
+        # always right, only the label was not.
+        kind = ((cat.get("kind") or "skill")
+                if lock is None and cat else "skill")
+        return _read(path), kind, lock, cat
     kind, entry = found
     lock = _for_tap(entry, qualifier)
     if lock is not None:

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/boost_cli/commands/quality.py

@@ -395,7 +395,14 @@ def cmd_doctor(argv):
     if taps and tap_ok == len(taps):
         out.ok("%d tap%s cloned & cached" % (len(taps), _s(len(taps))))
     elif not taps:
-        out.info("no taps configured — add one with `boost tap owner/repo`")
+        # `boost tap --defaults` leads, and it is the same command in the same
+        # order that `boost search`'s error, `mcp.no_results` and the MCP
+        # `boost_doctor` tool all name. A user who hits two of these surfaces
+        # in one session must not see the recommendation flip and read it as
+        # two different fixes — which is exactly what happened here: search
+        # said `--defaults`, doctor said `owner/repo`.
+        out.info("no registries tapped — nothing is searchable yet; add the "
+                 "recommended ones with `boost tap --defaults`", wrap=True)
 
     lock_ok = True
     lp = paths.lockfile_path()
@@ -598,10 +605,22 @@ def cmd_doctor(argv):
     else:
         out.warn("lock file integrity or log rotation needs attention")
 
-    out.verdict(issues == 0,
-                "healthy" if not issues else
-                "%d issue%s need attention — see the suggestions above"
-                % (issues, _s(issues)))
+    # A machine with no taps has nothing to disagree about, so every check
+    # above passes and the verdict read "healthy" — directly under the line
+    # saying no registries are tapped. That is the one state where a clean bill
+    # of health actively misleads: boost cannot answer anything yet, which is a
+    # setup step rather than a fault. Reported, never fatal — the exit code
+    # still turns only on real issues, so scripts and CI are unaffected. The
+    # MCP `boost_doctor` tool already refused to say "healthy" here; this is
+    # the CLI half of the same rule.
+    if issues == 0 and not taps:
+        out.verdict(True, "ready to set up — tap a registry to make boost "
+                          "searchable")
+    else:
+        out.verdict(issues == 0,
+                    "healthy" if not issues else
+                    "%d issue%s need attention — see the suggestions above"
+                    % (issues, _s(issues)))
     return 1 if issues else 0
 
 

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/boost_cli/commands/quickstart.py

@@ -162,6 +162,26 @@ def cmd_quickstart(argv) -> int:
         planned = [n for n in names if n in pins] if manifest else []
         out.info("would build the keyword index, then import %d shard(s)"
                  % len(planned))
+        # "0 shard(s)" reads as "none are published" when the real cause is
+        # local, and --dry-run is exactly what a cautious new user runs first.
+        # The live path already explains both cases; without this the preview
+        # is the one surface that reports the symptom and withholds the reason.
+        if not planned:
+            if args.no_vectors:
+                out.info(out.role("(0 because --no-vectors was asked for)",
+                                  "muted"))
+            elif not dense.have_backend():
+                out.info("0 because semantic search needs the extra: "
+                         "`pipx inject boost-skill-cli "
+                         "\"boost-skill-cli[rag]\"` — keyword search works "
+                         "without it", wrap=True)
+            elif manifest is None:
+                out.info(out.role("(0 because the shard manifest could not be "
+                                  "read — keyword search is unaffected)",
+                                  "muted"), wrap=True)
+            else:
+                out.info(out.role("(0 because none of these registries have a "
+                                  "published shard yet)", "muted"), wrap=True)
         return 0
 
     with spin.Spinner("building the keyword index"):

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/boost_cli/core/dense.py

@@ -23,7 +23,13 @@ from ..errors import BoostError
 from . import catalog, embed, paths
 from .rag import Hit, chunk, entry_key, read_body
 
-INDEX_VERSION = 2
+# 3 -- `chunks.digest` carries each row's entry content digest, which is what
+#      lets reuse be decided per ENTRY rather than per tap. Bumping this is the
+#      migration: a v2 store has no such column, `_ensure_schema` cannot add one
+#      to an existing table (CREATE TABLE IF NOT EXISTS), and `build` already
+#      wipes on a version change. That costs one full re-embed, once — the same
+#      price v2 charged when it landed, and the last one this mechanism needs.
+INDEX_VERSION = 3
 _BATCH = 128            # texts per embedding request
 
 #: Rows between commits while storing vectors. Small enough that an interrupted
@@ -112,8 +118,14 @@ def quantized(con: sqlite3.Connection) -> bool:
 def _ensure_schema(con: sqlite3.Connection, dim: int) -> None:
     con.execute(
         "CREATE TABLE IF NOT EXISTS chunks (id INTEGER PRIMARY KEY AUTOINCREMENT,"
-        " name TEXT, tap TEXT, path TEXT, kind TEXT, cix INTEGER, snip TEXT)")
+        " name TEXT, tap TEXT, path TEXT, kind TEXT, cix INTEGER, snip TEXT,"
+        " digest TEXT)")
     con.execute("CREATE INDEX IF NOT EXISTS chunks_tap ON chunks(tap)")
+    # Keyed by (tap, path) because that is `rag.entry_key` — row identity, the
+    # thing an entry's chunks belong to. Keying it on `digest` alone would be
+    # the content question, which is a different one: two taps shipping the
+    # same file share a digest and must still be deleted independently.
+    con.execute("CREATE INDEX IF NOT EXISTS chunks_entry ON chunks(tap, path)")
     con.execute("CREATE TABLE IF NOT EXISTS meta (k TEXT PRIMARY KEY, v TEXT)")
     if _quantizable(dim):
         # Two relations, because one cannot do both jobs. `vec0` is the only
@@ -518,17 +530,38 @@ def build(entries: list[dict] | None = None, force: bool = False,
                     if commit and old_commits.get(safe) == commit:
                         reused.add(safe)
 
-        fresh = [e for e in entries
-                 if e["tap"].replace("/", "__") not in reused]
-        changed_taps = sorted({e["tap"] for e in fresh})
+        candidates = [e for e in entries
+                      if e["tap"].replace("/", "__") not in reused]
+        changed_taps = sorted({e["tap"] for e in candidates})
         # Prune taps that are gone entirely, not only the ones that changed.
         # `boost untap` removes a tap's entries, so it can never appear in
-        # `fresh` and its vectors survived every later incremental build —
+        # `candidates` and its vectors survived every later incremental build —
         # crowding the KNN pool on every query with rows `retrieve` then
         # discards for not being live, which is how a dense search quietly
         # returns fewer hits the longer an index has been in use.
         removed_taps = sorted(_indexed_taps(con) - {e["tap"] for e in entries})
-        _delete_taps(con, changed_taps + removed_taps)
+        _delete_taps(con, removed_taps)
+
+        # Second-level reuse, and the reason it is worth the column. A tap's
+        # commit moving says *something* in that clone changed — not that
+        # anything boost indexes did. Measured on a real 464-tap install: 19
+        # taps drifted in 6.85 h, and 10 of them changed nothing indexed at all
+        # (badge JSON, star-history SVGs, CI YAML, e2e TypeScript), yet cost
+        # 44,866 chunks — 40.0% of the incremental bill. Across all 19, 967
+        # changed files mapped to 39 indexed entries owning 659 chunks, against
+        # 112,081 re-embedded: 170x the actual change.
+        #
+        # The digest is not new identity machinery. `catalog._content_digest`
+        # already stamps every entry with a hash of exactly what `read_body`
+        # assembles, and `tests/unit/test_content_identity.py` pins that parity
+        # — so an entry whose digest matches the one stored beside its chunks
+        # is one whose embedded text is byte-identical.
+        fresh, kept = _split_by_digest(con, candidates)
+        _delete_entries(con, [entry_key(e) for e in fresh])
+        # An entry deleted upstream keeps answering queries otherwise: whole-tap
+        # deletion used to sweep it for free, and reuse is exactly what stops
+        # that happening.
+        _prune_stale_entries(con, changed_taps, {entry_key(e) for e in entries})
 
         added, failed_taps = _embed_and_store(con, fresh, tap_paths,
                                              on_progress=on_progress)
@@ -555,6 +588,12 @@ def build(entries: list[dict] | None = None, force: bool = False,
             "reindexed": changed_taps,
             "pruned": removed_taps,
             "reused": sorted(reused),
+            # Entries inside a moved tap whose content had not changed. This is
+            # the number the tap-level `reused` cannot show — a tap appears in
+            # `reindexed` the moment its commit moves, and on real data most of
+            # its entries are still untouched.
+            "reused_entries": len(kept),
+            "embedded_entries": len(fresh),
             "failed": sorted(failed_taps),
             "provider": prov,
             "model": mdl,
@@ -614,7 +653,8 @@ def export_shard(tap: str) -> dict:
             vt = "vec_raw v ON v.id = c.id" if quantized(con) \
                 else "vec_chunks v ON v.rowid = c.id"
             rows = con.execute(
-                "SELECT c.name, c.tap, c.path, c.kind, c.cix, c.snip, v.embedding "  # noqa: S608  relation name from a literal pair
+                "SELECT c.name, c.tap, c.path, c.kind, c.cix, c.snip, "  # noqa: S608  relation name from a literal pair
+                "c.digest, v.embedding "
                 "FROM chunks c JOIN %s WHERE c.tap = ? ORDER BY c.id" % vt,
                 (tap,)).fetchall()
         except sqlite3.Error as exc:
@@ -630,10 +670,16 @@ def export_shard(tap: str) -> dict:
             # the vector rows are gone or unlinked. Still not "never built".
             raise _unreadable_vectors(tap, expected, None)
         for row in rows:
-            name, ctap, path, kind, cix, snip, emb = row
+            name, ctap, path, kind, cix, snip, digest, emb = row
             chunks.append({
                 "name": name, "tap": ctap, "path": path, "kind": kind,
                 "cix": cix, "snip": snip,
+                # The digest travels with the vector or the shard defeats its
+                # own purpose: `_split_by_digest` reuses an entry only when the
+                # stored digest matches the catalog's, so an imported chunk
+                # without one is re-embedded on the very next build — paying
+                # in CPU exactly what downloading the shard was meant to save.
+                "digest": digest,
                 # base64 so the shard is plain JSON and can be published as a
                 # release artifact without a binary format of its own.
                 "embedding": base64.b64encode(bytes(emb)).decode("ascii"),
@@ -714,10 +760,15 @@ def import_shard(shard: dict, commit: str) -> tuple[bool, str]:
         mod = _load()
         for c in shard.get("chunks") or []:
             cur = con.execute(
-                "INSERT INTO chunks (name, tap, path, kind, cix, snip) "
-                "VALUES (?, ?, ?, ?, ?, ?)",
+                "INSERT INTO chunks (name, tap, path, kind, cix, snip, digest) "
+                "VALUES (?, ?, ?, ?, ?, ?, ?)",
                 (c.get("name"), c.get("tap"), c.get("path"), c.get("kind"),
-                 c.get("cix"), c.get("snip")))
+                 c.get("cix"), c.get("snip"),
+                 # A shard published before the digest travelled has none, and
+                 # None is the honest value: that tap re-embeds once, which is
+                 # what it did before this existed. Never a placeholder — a
+                 # wrong digest would suppress a real re-embed forever.
+                 c.get("digest")))
             blob = base64.b64decode(c["embedding"])
             _store_vector(con, cur.lastrowid, blob)
         commits = meta.get("commits")
@@ -748,12 +799,109 @@ def _indexed_taps(con: sqlite3.Connection) -> set:
         return set()          # no chunks table yet — nothing indexed, nothing stale
 
 
+def _delete_matching(con: sqlite3.Connection, where: str,
+                     params: tuple) -> int:
+    """Drop the chunk rows matching ``where``, and their vectors. Rows hit.
+
+    Deleting a chunk is two statements that must not drift apart: the vectors
+    go through :func:`_drop_vectors` (which knows both store layouts) and only
+    then does the row go. Tap-level and entry-level deletion had that pair
+    written out twice, which is one copy too many for a rule this easy to half-
+    apply — an orphan vector outlives every later deletion, because those are
+    all scoped through ``chunks``.
+
+    ``where`` is a literal from this module, never caller data; the parameters
+    are bound.
+    """
+    # `%` rather than `+`, to match every other interpolated statement in this
+    # module: those are the shapes ruff's S608 is known to flag here, so the
+    # suppression is one ruff actually consumes rather than an unused `noqa`
+    # that RUF100 would then reject.
+    ids = [r[0] for r in con.execute(
+        "SELECT id FROM chunks WHERE %s" % where, params)]  # noqa: S608  literal clause
+    if not ids:
+        return 0
+    _drop_vectors(con, ids)
+    con.execute("DELETE FROM chunks WHERE %s" % where, params)  # noqa: S608  same literal
+    return len(ids)
+
+
+def _delete_entries(con: sqlite3.Connection,
+                    keys: list[tuple[str, str]]) -> int:
+    """Drop every chunk row for these ``(tap, path)`` entries. Returns rows hit.
+
+    The entry-level counterpart to :func:`_delete_taps`, and it must stay
+    row-scoped rather than content-scoped: two taps can ship a byte-identical
+    file, so deleting by digest would take another tap's rows with it.
+    """
+    return sum(_delete_matching(con, "tap = ? AND path = ?", (tap, path))
+               for tap, path in keys)
+
+
+def _prune_stale_entries(con: sqlite3.Connection, taps: list[str],
+                         live: set[tuple[str, str]]) -> int:
+    """Drop rows in ``taps`` for entries the catalog no longer has.
+
+    Whole-tap deletion used to do this for free: a changed tap was emptied and
+    rebuilt, so an entry deleted upstream simply never came back. Reuse keeps
+    the rows instead, which means a file removed from a registry would answer
+    queries forever — the same silent-stale-row failure `_delete_taps`'s
+    `removed_taps` sweep exists to prevent, one level down.
+    """
+    stale = [(tap, path) for tap in taps
+             for (path,) in con.execute(
+                 "SELECT DISTINCT path FROM chunks WHERE tap = ?", (tap,))
+             if (tap, path) not in live]
+    return _delete_entries(con, stale)
+
+
+def _split_by_digest(con: sqlite3.Connection,
+                     entries: list[dict]) -> tuple[list[dict], list[dict]]:
+    """Partition ``entries`` into ``(to_embed, already_stored)``.
+
+    An entry is already stored when the store holds chunks for its
+    ``(tap, path)`` **and every one of them** carries its current content
+    digest. Every one, not any: a partially-written entry — an interrupted
+    build, or one whose digest changed mid-run — would otherwise be reused
+    with a mixture of two versions' vectors, which is worse than re-embedding
+    because nothing later notices.
+
+    An entry with no digest is never reused. `catalog._content_digest` stamps
+    them at scan time, but a cache written before ``CACHE_FORMAT`` has none,
+    and CLAUDE.md's rule is explicit: consumers degrade cleanly when ``content``
+    is absent and **must never treat two absences as a match**. Two entries
+    with no digest are not the same entry; they are two unknowns.
+    """
+    # Nothing to decide, and the early return is the common case rather than a
+    # guard: on a build where no tap moved, `candidates` is empty, and without
+    # this the store gets scanned end to end to answer a question nobody asked.
+    if not entries:
+        return [], []
+    # Scoped to the taps actually in play, via the `chunks_tap` index. Reading
+    # the whole table instead measured 3.75 s cold (0.17 s warm) against 0.08 s
+    # for the nineteen largest taps and under 0.01 s for nineteen typical ones,
+    # on a real 657,587-chunk store. The full read was also the wrong shape: it
+    # grows with the store while the work grows with the drift.
+    stored: dict[tuple[str, str], set] = {}
+    for tap in sorted({e["tap"] for e in entries}):
+        for path, digest in con.execute(
+                "SELECT path, digest FROM chunks WHERE tap = ?", (tap,)):
+            stored.setdefault((tap, path), set()).add(digest)
+    to_embed, kept = [], []
+    for e in entries:
+        digest = e.get("content")
+        have = stored.get(entry_key(e))
+        if digest and have == {digest}:
+            kept.append(e)
+        else:
+            to_embed.append(e)
+    return to_embed, kept
+
+
 def _delete_taps(con: sqlite3.Connection, taps: list[str]) -> None:
+    """Drop every chunk row belonging to these taps, and their vectors."""
     for tap in taps:
-        ids = [r[0] for r in
-               con.execute("SELECT id FROM chunks WHERE tap = ?", (tap,))]
-        _drop_vectors(con, ids)
-        con.execute("DELETE FROM chunks WHERE tap = ?", (tap,))
+        _delete_matching(con, "tap = ?", (tap,))
 
 
 def _embed_and_store(con: sqlite3.Connection, entries: list[dict],
@@ -825,10 +973,14 @@ def _embed_and_store(con: sqlite3.Connection, entries: list[dict],
             failed_taps.add(e["tap"])
             continue
         cur = con.execute(
-            "INSERT INTO chunks (name, tap, path, kind, cix, snip) "
-            "VALUES (?, ?, ?, ?, ?, ?)",
+            "INSERT INTO chunks (name, tap, path, kind, cix, snip, digest) "
+            "VALUES (?, ?, ?, ?, ?, ?, ?)",
             (e["name"], e["tap"], e["skill_md"],
-             e.get("kind", "skill"), ci, text[:200].strip()))
+             e.get("kind", "skill"), ci, text[:200].strip(),
+             # None where the catalog has no digest, which `_split_by_digest`
+             # then refuses to reuse — the honest answer for a cache written
+             # before CACHE_FORMAT stamped one.
+             e.get("content")))
         _store_vector(con, cur.lastrowid, mod.serialize_float32(vec))
         added += 1
         since_commit += 1

{boost_skill_cli-1.2.54 → boost_skill_cli-1.2.56}/boost_cli/core/mcp.py

@@ -266,6 +266,62 @@ def hit_line(entry: dict, *, installed: bool = False) -> str:
                                entry.get("tap", "?"))
 
 
+# `boost_read`'s budget, and it is a measured number rather than a round one.
+# An MCP reply lands directly in an agent's context and a SKILL.md is not
+# bounded above, so a tool that returns one whole is how a single lookup costs
+# a session. Measured over the 63,053 items in a real 467-tap install: median
+# 6,063 bytes, p90 16,225, p99 35,275, **max 567,484** — three orders of
+# magnitude between the middle and the tail, which is the shape that makes an
+# unbounded read a real hazard rather than a theoretical one.
+#
+# 12,000 is the knee. It delivers 80.3% of the catalogue whole against 62.8%
+# at 8,000 — two thirds of the remaining truncations bought for 50% more bytes
+# — and caps the worst case near 3,000 tokens. Raising it to 16,000 buys 9.3
+# points for another third again, which is the wrong side of the trade for a
+# tool an agent may call several times in one turn.
+#
+# The cap counts CHARACTERS, deliberately: boost cannot know the caller's
+# tokenizer, and a character count is one an agent can check against the reply
+# it is already holding. Truncation is never the failure mode this guards
+# against, because it is announced — see :func:`read_reply`.
+READ_LIMIT = 12000
+
+
+def read_reply(name: str, text: str, *, kind: str = "skill",
+               installed: bool = False, limit: int = READ_LIMIT) -> str:
+    """`boost_read`'s reply: a header, then the item's own content.
+
+    The header exists because the body alone does not say what the agent is
+    holding. A tap-qualified name resolves to one item out of several that
+    share a bare name, and whether it is already installed decides whether the
+    next call is `boost_install` or nothing at all — both are one line here and
+    neither is recoverable from Markdown.
+
+    **Truncation is announced, in the reply, at the point it happens.** A body
+    cut silently reads as a complete document that simply ends — which for a
+    procedure is worse than no body at all, because the agent acts on the half
+    it can see and never learns there was a second half. The notice names the
+    command that returns the rest, so the reply carries its own recovery path
+    the way ``tool-design`` asks agent-facing errors to.
+
+    Truncation cuts at a LINE boundary within the budget rather than mid-line.
+    A Markdown body cut mid-fence or mid-sentence is text an agent has to
+    guess about; a whole-line cut is unambiguously "the document continues".
+    A body whose first line already exceeds the budget has no such boundary,
+    so it falls back to a hard character cut — announced the same way.
+    """
+    head = ["name: %s" % name]
+    if kind != "skill":
+        head.append("kind: %s" % kind)
+    head.append("installed: %s" % ("yes" if installed else "no"))
+    body = text.strip("\n")
+    if len(body) > limit:
+        cut = body[:limit].rpartition("\n")[0] or body[:limit]
+        body = ("%s\n\n[truncated at %d of %d characters — read the rest with "
+                "`boost cat %s`]" % (cut, len(cut), len(text), name))
+    return "\n".join(head) + "\n\n" + body
+
+
 # The lock sections, in the order every boost surface names them. Fixed here
 # rather than read off the caller's dict, so the footer's column order cannot
 # drift between a machine that has rules and one that does not.