boost-skill-cli 1.0.452__tar.gz → 1.1.0__tar.gz

{boost_skill_cli-1.0.452 → boost_skill_cli-1.1.0}/CLAUDE.md

@@ -140,6 +140,31 @@ line coverage. Target `boost_cli/core` behavior with assertions, not just import
   (`chain-of-thought-design`, `guardrail-design`), so it is `ai`, not `ui`.
   `tests/unit/test_registry_categories.py::TestDesignDomain` pins both
   directions.
+- **A tap clone holds Markdown and nothing else.** Taps clone
+  `--filter=blob:none --sparse` with a cone covering exactly what
+  `catalog.scan_dir` opens (`gitutil.SPARSE_PATTERNS`, pinned against catalog's
+  own constants by `tests/unit/test_gitutil_sparse.py` — `gitutil` can't
+  import `catalog`, `catalog` imports `gitutil`). 458 taps held 12 GB to index
+  1.9 GB of Markdown; `Shopify/agent-skills` was 611 MB for its 30 SKILL.md
+  files and is 11 MB under the cone, with a byte-identical catalog. So
+  **anything reading a tap's real files must go through
+  `store.source_dir_for`**, which calls
+  `gitutil.materialize` first — a skill's own `scripts/`/`assets/` sit outside
+  the cone, and `_copy_skill` is a `copytree` that would copy the half of the
+  directory that exists and report success. Use `sparse-checkout add`, never
+  `set`: `set` replaces the pattern list and un-fetches every skill materialized
+  before it. `boost import` clones `sparse=False` — it reads the whole repo.
+  `boost compact` narrows clones that predate this (offline; measured 177 MB →
+  93 MB), and must `update-index --refresh` first, because git silently declines
+  to remove a path whose stat data looks dirty and otherwise reports success
+  having changed nothing.
+- **`~/.boost/cache/` holds boost's own indexes, not just tap catalogs.**
+  `paths.INTERNAL_CACHE_FILES` names them. `boost clean` sweeps `cache/*.json`
+  whose stem is not a configured tap, which described `rag_index.json` and
+  `discovery.json` perfectly — it deleted the BM25 index on every run, and
+  rebuilding it re-parses every tap catalog on the machine (~71k items). A new
+  derived artifact goes in that set or `tests/unit/test_clean_internal_cache.py`
+  fails.
 - **Sandbox tests via env, and export separately.** boost state lives under
   `HOME`/`BOOST_HOME`; tests point them at a tempdir. In zsh, one-line chains
   like `export A=$(...) B=$A/x` leave `B` broken — use separate `export`
@@ -233,7 +258,7 @@ for both — edit items, run `build_roadmap.py`, never touch the HTML by hand.
 ## Architecture
 
 **CLI dispatch.** `boost_cli/cli.py` holds `COMMANDS`, the single
-source-of-truth list of `(name, group, module, summary)` for all 79 commands.
+source-of-truth list of `(name, group, module, summary)` for all 80 commands.
 Each command is implemented as `def cmd_<name_with_underscores>(argv) -> int`
 inside `boost_cli/commands/<module>.py`, and `_dispatch` imports that module
 lazily on invocation — so `boost --help` stays instant and command modules are
@@ -254,7 +279,7 @@ in `boost_cli/core/paths.py`, which is what makes `HOME=<tempdir>` sandboxing
 work in tests and the dev loop:
 
 ```text
-~/.boost/repos/     shallow git clones of tapped registries
+~/.boost/repos/     blobless, sparse git clones of tapped registries
 ~/.boost/cache/     JSON catalogs built from SKILL.md/rule/workflow frontmatter
 ~/.boost/logs/      rotating diagnostic log + crash reports
 ~/.boost/state/     pins, tags, policy, profiles, pulse feed, snapshots

{boost_skill_cli-1.0.452 → boost_skill_cli-1.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boost-skill-cli
-Version: 1.0.452
+Version: 1.1.0
 Summary: boost — Homebrew for AI coding skills
 Author: Jonathan Reyes
 License: GNU GENERAL PUBLIC LICENSE
@@ -908,13 +908,30 @@ whichever agents you've got — all of it tracked in a lock file so state
 stays reproducible.
 
 ```text
-GitHub registries  ──boost update──▶  ~/.boost/repos/    (shallow clones)
+GitHub registries  ──boost update──▶  ~/.boost/repos/    (blobless sparse clones)
                                      ~/.boost/cache/    (JSON catalogs)
                    ──boost install─▶  ~/.agents/skills/ (canonical store)
                                      .skill-lock.json  (v3 lock file)
                    ────symlinks───▶  ~/.claude/skills/  ~/.windsurf/skills/  ~/.cursor/skills/
 ```
 
+A tap clone holds **only the files boost indexes** — `SKILL.md`, workflow
+Markdown, and rule files. Everything else a registry ships (`node_modules`,
+binary assets, bundled JS) is neither downloaded nor checked out, because the
+catalog is built from Markdown and nothing else. `Shopify/agent-skills` is
+611 MB as a normal clone and 11 MB as a tap, and produces an identical catalog.
+When a skill you install owns its own `scripts/` or `assets/`, those are fetched
+on demand at install time, so nothing is missing from what lands in the store.
+
+If you tapped heavily before this landed, `boost compact` narrows the clones you
+already have — offline, no re-clone, no loss of search coverage:
+
+```bash
+boost compact --dry-run     # what it would reclaim
+boost compact               # narrow every clone in place
+boost compact --reclone     # also drop already-downloaded git objects (needs network)
+```
+
 Gemini CLI is the one agent that needs no symlink: it implements the
 [Agent Skills](https://agentskills.io) standard and discovers `~/.agents/skills`
 — boost's canonical store — directly. Linking into `~/.gemini/skills` as well
@@ -971,7 +988,7 @@ boost doctor               # sanity check: broken links, lock drift, stale taps
 boost bundle dump > Boostfile     # everyone else runs: boost bundle install
 ```
 
-## 79 commands, organized into 8 groups
+## 80 commands, organized into 8 groups
 
 `boost --help` prints the full grouped command list; for a visual tour see
 [`docs/index.html`](docs/index.html), and for every flag of every command see
@@ -986,10 +1003,10 @@ install path — see [`docs/architecture/`](docs/architecture/README.md).
 | **Package Management** | install · uninstall · sync · update · reinstall · bundle · import · pin · unpin · snapshot · export · adapt · run |
 | **Discovery & Search** | search · reindex · discover · recommend · browse · index · trending · stats · count |
 | **Skill Information** | list · info · cat · edit · preview · explain · log · home · deps · tag |
-| **Registry (Taps)** | tap · untap · taps · outdated |
-| **Intelligence** | distill · simulate · infer · absorb · evolve · context · focus · impact |
+| **Registry (Taps)** | tap · untap · taps · outdated · catalog |
+| **Intelligence** | chat · distill · simulate · infer · absorb · evolve · context · focus · impact |
 | **Quality & Health** | doctor · lint · audit · verify · drift · test · fingerprint · quarantine · decay · heal · conflict · changelog · attest · health · trust |
-| **Configuration** | config · clean · create · policy · onboard · completions · schedule · serve · mcp · hooks · bmad · self-update |
+| **Configuration** | config · clean · compact · create · policy · onboard · completions · schedule · serve · mcp · hooks · bmad · self-update |
 | **Team & Collaboration** | cohort · profile · protocol · pulse · replay · who |
 
 The AI-assisted commands (`search --smart`, `explain`, `distill`, `infer`,
@@ -1185,7 +1202,7 @@ Four layers, all enforced (`make check` runs the full set; CI runs the same thin
 
 | Layer | What it does | Gate |
 |---|---|---|
-| `make test` | pytest across `tests/unit/` (every core module) and `tests/functional/` (drives all 79 commands in-process against sandboxed homes) | **≥80% line coverage** of `boost_cli` (`fail_under` in pyproject.toml) |
+| `make test` | pytest across `tests/unit/` (every core module) and `tests/functional/` (drives all 80 commands in-process against sandboxed homes) | **≥80% line coverage** of `boost_cli` (`fail_under` in pyproject.toml) |
 | `make smoke` | `tests/smoke.sh` — 175 checks run through the actual `./boost` shim (`--online` also hits real registries) | all pass |
 | `make mutation` | [mutmut] mutates `boost_cli/core` (~9,900 mutants) and reruns the unit suite against each one | **≥80% killed** (`scripts/mutation_gate.py`) |
 | `make evals` | scores the search ranker on a graded golden set — recall@5/@10, MRR, nDCG@5/@10 | **metric floors + no statistically significant regression** (`scripts/eval_gate.py`) |

{boost_skill_cli-1.0.452 → boost_skill_cli-1.1.0}/README.md

@@ -190,13 +190,30 @@ whichever agents you've got — all of it tracked in a lock file so state
 stays reproducible.
 
 ```text
-GitHub registries  ──boost update──▶  ~/.boost/repos/    (shallow clones)
+GitHub registries  ──boost update──▶  ~/.boost/repos/    (blobless sparse clones)
                                      ~/.boost/cache/    (JSON catalogs)
                    ──boost install─▶  ~/.agents/skills/ (canonical store)
                                      .skill-lock.json  (v3 lock file)
                    ────symlinks───▶  ~/.claude/skills/  ~/.windsurf/skills/  ~/.cursor/skills/
 ```
 
+A tap clone holds **only the files boost indexes** — `SKILL.md`, workflow
+Markdown, and rule files. Everything else a registry ships (`node_modules`,
+binary assets, bundled JS) is neither downloaded nor checked out, because the
+catalog is built from Markdown and nothing else. `Shopify/agent-skills` is
+611 MB as a normal clone and 11 MB as a tap, and produces an identical catalog.
+When a skill you install owns its own `scripts/` or `assets/`, those are fetched
+on demand at install time, so nothing is missing from what lands in the store.
+
+If you tapped heavily before this landed, `boost compact` narrows the clones you
+already have — offline, no re-clone, no loss of search coverage:
+
+```bash
+boost compact --dry-run     # what it would reclaim
+boost compact               # narrow every clone in place
+boost compact --reclone     # also drop already-downloaded git objects (needs network)
+```
+
 Gemini CLI is the one agent that needs no symlink: it implements the
 [Agent Skills](https://agentskills.io) standard and discovers `~/.agents/skills`
 — boost's canonical store — directly. Linking into `~/.gemini/skills` as well
@@ -253,7 +270,7 @@ boost doctor               # sanity check: broken links, lock drift, stale taps
 boost bundle dump > Boostfile     # everyone else runs: boost bundle install
 ```
 
-## 79 commands, organized into 8 groups
+## 80 commands, organized into 8 groups
 
 `boost --help` prints the full grouped command list; for a visual tour see
 [`docs/index.html`](docs/index.html), and for every flag of every command see
@@ -268,10 +285,10 @@ install path — see [`docs/architecture/`](docs/architecture/README.md).
 | **Package Management** | install · uninstall · sync · update · reinstall · bundle · import · pin · unpin · snapshot · export · adapt · run |
 | **Discovery & Search** | search · reindex · discover · recommend · browse · index · trending · stats · count |
 | **Skill Information** | list · info · cat · edit · preview · explain · log · home · deps · tag |
-| **Registry (Taps)** | tap · untap · taps · outdated |
-| **Intelligence** | distill · simulate · infer · absorb · evolve · context · focus · impact |
+| **Registry (Taps)** | tap · untap · taps · outdated · catalog |
+| **Intelligence** | chat · distill · simulate · infer · absorb · evolve · context · focus · impact |
 | **Quality & Health** | doctor · lint · audit · verify · drift · test · fingerprint · quarantine · decay · heal · conflict · changelog · attest · health · trust |
-| **Configuration** | config · clean · create · policy · onboard · completions · schedule · serve · mcp · hooks · bmad · self-update |
+| **Configuration** | config · clean · compact · create · policy · onboard · completions · schedule · serve · mcp · hooks · bmad · self-update |
 | **Team & Collaboration** | cohort · profile · protocol · pulse · replay · who |
 
 The AI-assisted commands (`search --smart`, `explain`, `distill`, `infer`,
@@ -467,7 +484,7 @@ Four layers, all enforced (`make check` runs the full set; CI runs the same thin
 
 | Layer | What it does | Gate |
 |---|---|---|
-| `make test` | pytest across `tests/unit/` (every core module) and `tests/functional/` (drives all 79 commands in-process against sandboxed homes) | **≥80% line coverage** of `boost_cli` (`fail_under` in pyproject.toml) |
+| `make test` | pytest across `tests/unit/` (every core module) and `tests/functional/` (drives all 80 commands in-process against sandboxed homes) | **≥80% line coverage** of `boost_cli` (`fail_under` in pyproject.toml) |
 | `make smoke` | `tests/smoke.sh` — 175 checks run through the actual `./boost` shim (`--online` also hits real registries) | all pass |
 | `make mutation` | [mutmut] mutates `boost_cli/core` (~9,900 mutants) and reruns the unit suite against each one | **≥80% killed** (`scripts/mutation_gate.py`) |
 | `make evals` | scores the search ranker on a graded golden set — recall@5/@10, MRR, nDCG@5/@10 | **metric floors + no statistically significant regression** (`scripts/eval_gate.py`) |

{boost_skill_cli-1.0.452 → boost_skill_cli-1.1.0}/boost_cli/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '1.0.452'
-__version_tuple__ = version_tuple = (1, 0, 452)
+__version__ = version = '1.1.0'
+__version_tuple__ = version_tuple = (1, 1, 0)
 
 __commit_id__ = commit_id = None

{boost_skill_cli-1.0.452 → boost_skill_cli-1.1.0}/boost_cli/cli.py

@@ -1,4 +1,4 @@
-"""boost CLI dispatcher — 78 commands across 8 groups.
+"""boost CLI dispatcher — 80 commands across 8 groups.
 
 Each command lives in boost_cli/commands/<module>.py as
     def cmd_<name_with_underscores>(argv: list[str]) -> int
@@ -47,7 +47,7 @@ GROUPS = {
 # (name, group, module, summary) — the single source of truth for the
 # command surface. docs/overview.html mirrors this list.
 COMMANDS = [
-    # Package Management (14)
+    # Package Management (13)
     ("install",     "pkg", "pkg", "Install a skill from a tap registry"),
     ("uninstall",   "pkg", "pkg", "Remove an installed skill, rule, workflow, or config"),
     ("sync",        "pkg", "pkg", "Reconcile installed skills & symlinks against the lock file"),
@@ -82,13 +82,13 @@ COMMANDS = [
     ("home",        "info", "info", "Open a skill's GitHub page in the browser"),
     ("deps",        "info", "info", "Show dependency & conflict relationships"),
     ("tag",         "info", "info", "Custom labels for organizing skills"),
-    # Registry (Taps) (4)
+    # Registry (Taps) (5)
     ("tap",         "tap", "taps", "Add a GitHub repo as a skill registry"),
     ("untap",       "tap", "taps", "Remove a registry tap"),
     ("taps",        "tap", "taps", "List all configured registry taps"),
     ("outdated",    "tap", "taps", "Show skills with available updates"),
     ("catalog",     "tap", "taps", "Share the tapped catalogue so others skip the clone"),
-    # Intelligence (8)
+    # Intelligence (9)
     ("chat",        "ai", "intelligence", "Ask about skills in plain language (RAG-grounded)"),
     ("distill",     "ai", "intelligence", "Merge multiple skills into one deduplicated skill"),
     ("simulate",    "ai", "intelligence", "Preview how a skill would change Claude's behavior"),
@@ -114,9 +114,10 @@ COMMANDS = [
     ("attest",      "chk", "safety", "Display/verify the install record for skills"),
     ("health",      "chk", "quality", "Dashboard of skill-environment health"),
     ("trust",       "chk", "quality", "Manage signing keys & verify tap provenance"),
-    # Configuration (12)
+    # Configuration (13)
     ("config",      "cfg", "configuration", "Display or modify boost configuration"),
     ("clean",       "cfg", "configuration", "Clear stale caches & broken symlinks"),
+    ("compact",     "cfg", "configuration", "Shrink tap clones to the files boost indexes"),
     ("create",      "cfg", "configuration", "Scaffold a new skill from a template"),
     ("policy",      "cfg", "configuration", "Manage & enforce skill governance policies"),
     ("onboard",     "cfg", "configuration", "Add skill-tracker telemetry to a repo & open a PR"),

{boost_skill_cli-1.0.452 → boost_skill_cli-1.1.0}/boost_cli/commands/configuration.py

@@ -141,10 +141,14 @@ def cmd_clean(argv) -> int:
 
     configured = {t.safe_name for t in registry.list_taps()}
     if paths.cache_dir().is_dir():
+        # Skip boost's own derived artifacts. They live in the same directory and
+        # end in .json, but no tap is named `rag_index` or `discovery`, so the
+        # stem test alone called both stale and deleted them every run — see
+        # paths.INTERNAL_CACHE_FILES.
         items.extend(
             (f, "stale tap cache", f.stat().st_size)
             for f in sorted(paths.cache_dir().glob("*.json"))
-            if f.stem not in configured
+            if f.stem not in configured and f.name not in paths.INTERNAL_CACHE_FILES
         )
 
     if paths.lock_history_dir().is_dir():
@@ -199,6 +203,106 @@ def cmd_clean(argv) -> int:
     return 0
 
 
+def _freight_bytes(tap_path: Path, keep_dirs: list[str]) -> int:
+    """Bytes that would leave `tap_path`'s working tree when the cone applies.
+
+    Mirrors what `gitutil.SPARSE_PATTERNS` keeps rather than approximating it,
+    so the dry run does not promise back the provenance files or the assets of
+    an already-installed skill — both of which survive and neither of which is
+    freed.
+    """
+    kept = tuple("%s/" % d.strip("/") for d in keep_dirs)
+    total = 0
+    for f in tap_path.rglob("*"):
+        if not f.is_file():
+            continue
+        # Relative to the tap, never absolute: every clone lives *under*
+        # ~/.boost, so testing the absolute parts for ".boost" excluded every
+        # file in every tap and reported that nothing could be freed.
+        rel = f.relative_to(tap_path)
+        if ".git" in rel.parts or ".boost" in rel.parts:
+            continue
+        if (f.suffix.lower() in (".md", ".mdc")
+                or f.name in catalog.RULE_FILENAMES):
+            continue
+        if kept and rel.as_posix().startswith(kept):
+            continue
+        total += f.stat().st_size
+    return total
+
+
+def cmd_compact(argv) -> int:
+    """boost compact [--dry-run] [--reclone] [TAP ...]"""
+    p = cliparse.parser(
+        prog="boost compact",
+        description="Shrink tap clones to the files boost indexes")
+    p.add_argument("tap", nargs="*",
+                   help="taps to compact (default: all cloned taps)")
+    p.add_argument("--dry-run", action="store_true",
+                   help="show what would be reclaimed without touching anything")
+    p.add_argument("--reclone", action="store_true",
+                   help="re-clone blobless for the smallest result (needs network)")
+    args = p.parse_args(argv)
+
+    taps = [registry.get(n) for n in args.tap] if args.tap else registry.list_taps()
+    taps = [t for t in taps if t.is_cloned]
+    if not taps:
+        out.ok("no cloned taps to compact")
+        return 0
+
+    # source_dir keeps an installed skill's own assets on disk, so routine
+    # hashing (`outdated`, `doctor`) stays offline after the freight is gone.
+    keep: dict[str, list[str]] = {}
+    for section in lockfile.all_installed().values():
+        for entry in section.values():
+            if entry.get("source_dir"):
+                keep.setdefault(entry.get("tap", ""), []).append(entry["source_dir"])
+
+    freed = 0
+    changed = 0
+    for tap in taps:
+        before = util.dir_size(tap.path)
+        if args.dry_run:
+            loose = _freight_bytes(tap.path, keep.get(tap.name, []))
+            if loose:
+                changed += 1
+                freed += loose
+                out.info("would free %s from %s"
+                         % (util.human_size(loose), tap.name))
+            continue
+        try:
+            if args.reclone:
+                util.rmtree(tap.path)
+                gitutil.clone_shallow(tap.url, tap.path)
+            else:
+                gitutil.narrow(tap.path)
+            for rel in keep.get(tap.name, []):
+                gitutil.materialize(tap.path, rel)
+        except BoostError as e:
+            out.warn("could not compact %s: %s" % (tap.name, e))
+            continue
+        after = util.dir_size(tap.path)
+        if after < before:
+            changed += 1
+            freed += before - after
+            out.info("%s  %s → %s" % (tap.name, util.human_size(before),
+                                      util.human_size(after)))
+
+    if args.dry_run:
+        out.dim("  %d tap(s) · %s would be freed"
+                % (changed, util.human_size(freed)))
+        return 0
+    journal.log("compact", "%d taps" % changed, freed=util.human_size(freed))
+    if not changed:
+        out.ok("every tap is already compact")
+        return 0
+    out.ok("compacted %d tap(s) · %s freed" % (changed, util.human_size(freed)))
+    if not args.reclone:
+        out.dim("  `boost compact --reclone` also drops already-downloaded "
+                "git objects")
+    return 0
+
+
 # ---------------------------------------------------------------- create
 
 _CREATE_BODY = """# %(title)s

{boost_skill_cli-1.0.452 → boost_skill_cli-1.1.0}/boost_cli/commands/pkg.py

@@ -1193,7 +1193,9 @@ def cmd_import(argv: list[str]) -> int:
             tmp = Path(tempfile.mkdtemp(prefix="boost-import-"))
             root = tmp / "repo"
             out.info("cloning %s …" % args.source)
-            gitutil.clone_shallow(args.source, root)
+            # Full checkout, not a tap's Markdown cone: import reads whatever the
+            # repo ships and copies it verbatim, assets included.
+            gitutil.clone_shallow(args.source, root, sparse=False)
         else:
             root = paths.expand(args.source)
             if not root.is_dir():