boost-skill-cli 1.0.340__tar.gz → 1.0.342__tar.gz

{boost_skill_cli-1.0.340/boost_skill_cli.egg-info → boost_skill_cli-1.0.342}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boost-skill-cli
-Version: 1.0.340
+Version: 1.0.342
 Summary: boost — Homebrew for AI coding skills
 Author: Jonathan Reyes
 License: GNU GENERAL PUBLIC LICENSE

{boost_skill_cli-1.0.340 → boost_skill_cli-1.0.342}/boost_cli/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '1.0.340'
-__version_tuple__ = version_tuple = (1, 0, 340)
+__version__ = version = '1.0.342'
+__version_tuple__ = version_tuple = (1, 0, 342)
 
 __commit_id__ = commit_id = None

{boost_skill_cli-1.0.340 → boost_skill_cli-1.0.342/boost_skill_cli.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boost-skill-cli
-Version: 1.0.340
+Version: 1.0.342
 Summary: boost — Homebrew for AI coding skills
 Author: Jonathan Reyes
 License: GNU GENERAL PUBLIC LICENSE

{boost_skill_cli-1.0.340 → boost_skill_cli-1.0.342}/docs/roadmap/items/keyless-semantic-search-for-everyone.md

@@ -331,3 +331,56 @@ both sides, and only the document half can be shipped ahead of time.
 <b>Still open in step 2:</b> the CI workflow that publishes shards as release artifacts, and the
 <code>boost tap</code> integration that fetches one automatically. Both are now plumbing on top of a
 verified mechanism rather than open questions. Steps 3 and 5 are untouched.
+
+<b>Step 3 works, and it did not need new code &mdash; it needed proving.</b> Local delta top-up falls
+out of the shard mechanism plus the commit-keyed reuse <code>build()</code> already had:
+<code>import_shard</code> records the tap's commit in the same <code>meta.commits</code> map
+<code>build()</code> consults, so an imported shard is indistinguishable from locally-built vectors
+as far as reuse is concerned.
+
+Measured end to end with a stubbed embedder, on a store holding an imported shard for
+<code>anthropics/skills</code> plus a freshly tapped <code>Aaronontheweb/dotnet-cursor-rules</code>:
+<b>158 chunks embedded &mdash; the new tap only &mdash; and the shard's 262 untouched</b>. The
+resulting store is one coherent embedding space: 420 vectors, both taps' commits recorded, single
+provider/model/dim. That is exactly the shape this step asks for &mdash; shards cover the popular
+registries, the local model makes the long tail self-serve, and cost scales with what a user taps
+rather than with the ecosystem.
+
+Two tests now pin it, because the coupling is the kind that breaks silently: an import that forgot to
+record the commit would still produce a <em>working</em> store, and the only symptom would be
+re-embedding the shard's chunks on every later build &mdash; minutes of wasted CPU that nothing
+reports.
+
+<b>Steps 1, 2, 3, 4 and 6 are now done; step 5 (a hosted demo) is the remainder</b>, along with the
+CI workflow that publishes shards as release artifacts, which is noted under step 2.
+
+<b>Step 2's publishing half is shipped &mdash; <code>.github/workflows/shards.yml</code>.</b> Weekly
+plus on-demand, it taps each registry, embeds it, exports a shard and uploads it as an artifact.
+
+<b>It is deliberately not chained off release, and that is the load-bearing decision.</b>
+<code>publish.yml</code> already sits at <code>ci &rarr; release &rarr; sbom</code>, GitHub's
+documented three-level <code>workflow_run</code> limit; a fourth link would silently never fire.
+<code>sbom.yml</code>'s header records exactly what that looks like in this repo &mdash; 253
+releases, 0 runs, 0 assets, a control that appeared present and produced nothing. Shards are keyed on
+a <em>registry's</em> commit anyway, not on a boost release, so coupling them to our version cadence
+would be wrong even if the chain allowed it. Same reasoning for uploading artifacts rather than
+release assets: attaching them to a release would re-publish unchanged vectors on every version bump.
+
+<b>Scale drove the shape.</b> At the measured ~1.2&nbsp;s/chunk, the 20-tap corpus is ~3.4&nbsp;h
+against the 6&nbsp;h job limit and the full 466-registry catalogue is far past it. So the workflow
+fans out <b>one job per registry</b> (<code>fail-fast: false</code>, so one bad registry cannot lose
+the others' work) rather than embedding everything in a single job.
+
+<b>Two things were caught by testing the pieces locally rather than trusting the YAML.</b> The
+matrix-planning step used <code>printf '%s\n' $repos</code>, which relies on word-splitting that does
+not survive quoting: it produced a <b>one-entry matrix holding all twenty repos as a single
+string</b> &mdash; one job attempting the entire corpus, straight past the job limit, and it would
+have looked like a plausible timeout rather than a bug. Splitting in Python fixes it. The shard
+validation step was also run against a real 262-chunk shard and against a provenance-stripped copy,
+to confirm it accepts one and rejects the other; a shard missing <code>provider</code>,
+<code>model</code>, <code>dim</code> or <code>commit</code> cannot be validated on import and would be
+refused by every consumer, silently, forever.
+
+<b>Unproven until it runs:</b> the workflow has never executed. The pieces are tested, the YAML
+parses and the pinned action SHAs match the repo's existing ones, but the first real run is the first
+end-to-end exercise. <b>Step 5 (a hosted demo) is now the only part of this epic left.</b>

{boost_skill_cli-1.0.340 → boost_skill_cli-1.0.342}/docs/roadmap.html

@@ -918,7 +918,60 @@ both sides, and only the document half can be shipped ahead of time.
 
 <b>Still open in step 2:</b> the CI workflow that publishes shards as release artifacts, and the
 <code>boost tap</code> integration that fetches one automatically. Both are now plumbing on top of a
-verified mechanism rather than open questions. Steps 3 and 5 are untouched.</p>
+verified mechanism rather than open questions. Steps 3 and 5 are untouched.
+
+<b>Step 3 works, and it did not need new code &mdash; it needed proving.</b> Local delta top-up falls
+out of the shard mechanism plus the commit-keyed reuse <code>build()</code> already had:
+<code>import_shard</code> records the tap's commit in the same <code>meta.commits</code> map
+<code>build()</code> consults, so an imported shard is indistinguishable from locally-built vectors
+as far as reuse is concerned.
+
+Measured end to end with a stubbed embedder, on a store holding an imported shard for
+<code>anthropics/skills</code> plus a freshly tapped <code>Aaronontheweb/dotnet-cursor-rules</code>:
+<b>158 chunks embedded &mdash; the new tap only &mdash; and the shard's 262 untouched</b>. The
+resulting store is one coherent embedding space: 420 vectors, both taps' commits recorded, single
+provider/model/dim. That is exactly the shape this step asks for &mdash; shards cover the popular
+registries, the local model makes the long tail self-serve, and cost scales with what a user taps
+rather than with the ecosystem.
+
+Two tests now pin it, because the coupling is the kind that breaks silently: an import that forgot to
+record the commit would still produce a <em>working</em> store, and the only symptom would be
+re-embedding the shard's chunks on every later build &mdash; minutes of wasted CPU that nothing
+reports.
+
+<b>Steps 1, 2, 3, 4 and 6 are now done; step 5 (a hosted demo) is the remainder</b>, along with the
+CI workflow that publishes shards as release artifacts, which is noted under step 2.
+
+<b>Step 2's publishing half is shipped &mdash; <code>.github/workflows/shards.yml</code>.</b> Weekly
+plus on-demand, it taps each registry, embeds it, exports a shard and uploads it as an artifact.
+
+<b>It is deliberately not chained off release, and that is the load-bearing decision.</b>
+<code>publish.yml</code> already sits at <code>ci &rarr; release &rarr; sbom</code>, GitHub's
+documented three-level <code>workflow_run</code> limit; a fourth link would silently never fire.
+<code>sbom.yml</code>'s header records exactly what that looks like in this repo &mdash; 253
+releases, 0 runs, 0 assets, a control that appeared present and produced nothing. Shards are keyed on
+a <em>registry's</em> commit anyway, not on a boost release, so coupling them to our version cadence
+would be wrong even if the chain allowed it. Same reasoning for uploading artifacts rather than
+release assets: attaching them to a release would re-publish unchanged vectors on every version bump.
+
+<b>Scale drove the shape.</b> At the measured ~1.2&nbsp;s/chunk, the 20-tap corpus is ~3.4&nbsp;h
+against the 6&nbsp;h job limit and the full 466-registry catalogue is far past it. So the workflow
+fans out <b>one job per registry</b> (<code>fail-fast: false</code>, so one bad registry cannot lose
+the others' work) rather than embedding everything in a single job.
+
+<b>Two things were caught by testing the pieces locally rather than trusting the YAML.</b> The
+matrix-planning step used <code>printf '%s\n' $repos</code>, which relies on word-splitting that does
+not survive quoting: it produced a <b>one-entry matrix holding all twenty repos as a single
+string</b> &mdash; one job attempting the entire corpus, straight past the job limit, and it would
+have looked like a plausible timeout rather than a bug. Splitting in Python fixes it. The shard
+validation step was also run against a real 262-chunk shard and against a provenance-stripped copy,
+to confirm it accepts one and rejects the other; a shard missing <code>provider</code>,
+<code>model</code>, <code>dim</code> or <code>commit</code> cannot be validated on import and would be
+refused by every consumer, silently, forever.
+
+<b>Unproven until it runs:</b> the workflow has never executed. The pieces are tested, the YAML
+parses and the pinned action SHAs match the repo's existing ones, but the first real run is the first
+end-to-end exercise. <b>Step 5 (a hosted demo) is now the only part of this epic left.</b></p>
         <div class="meta">
           <span class="m">Complexity <b>L</b></span>
           <span class="m hi">Impact <b>High</b></span>

{boost_skill_cli-1.0.340 → boost_skill_cli-1.0.342}/tests/unit/test_dense_shards.py

@@ -96,7 +96,19 @@ def with_backend(monkeypatch):
     instead of exercising the check they are named for, which is how they
     passed locally and reddened three macOS legs plus the canary.
     """
-    monkeypatch.setattr(dense, "_load", lambda: object())
+    class _FakeVec:
+        """Enough of sqlite_vec for the paths under test.
+
+        `serialize_float32` is real: tests that WRITE vectors reach it, and a
+        bare object() stub only survived while every test stopped at validation.
+        """
+
+        @staticmethod
+        def serialize_float32(vec):
+            import struct
+            return struct.pack("%df" % len(vec), *vec)
+
+    monkeypatch.setattr(dense, "_load", lambda: _FakeVec)
     monkeypatch.setattr(dense, "_connect",
                         lambda: sqlite3.connect(str(dense.db_path())))
 
@@ -240,3 +252,66 @@ class TestWorksWithoutTheExtension:
                  "commit": "c1", "chunks": []}
         ok, reason = dense.import_shard(shard, commit="c1")
         assert ok is False and "backend" in reason
+
+
+class TestDeltaTopUp:
+    """An imported shard must make `build` skip that tap and embed only the rest.
+
+    This is step 3 of the keyless epic — "when a tap runs ahead of its published
+    shard, or is a registry CI has never seen, embed just those files on the
+    spot". It works because `import_shard` records the tap's commit in the same
+    `meta.commits` map that `build` consults, so a shard is indistinguishable
+    from locally-built vectors as far as reuse is concerned.
+
+    That coupling is easy to break silently: an import that forgot to record the
+    commit would still produce a working store, and the only symptom would be
+    re-embedding the shard's chunks on the next build — minutes of wasted CPU
+    that nothing reports. Hence a test.
+    """
+
+    def _tap_seen(self, monkeypatch, commits):
+        from boost_cli.core import rag
+        monkeypatch.setattr(rag, "_tap_commits", lambda: commits)
+        monkeypatch.setattr(rag, "_tap_paths", lambda: {})
+
+    def test_an_imported_shard_is_reused_not_re_embedded(self, sandbox,
+                                                         with_backend,
+                                                         monkeypatch):
+        from boost_cli.core import embed
+        _store(rows=(("a", "acme/skills"),))
+        embedded = []
+        monkeypatch.setattr(embed, "embed",
+                            lambda texts, **kw: embedded.extend(texts) or
+                            [[0.1, 0.2, 0.3] for _ in texts])
+        monkeypatch.setattr(embed, "available", lambda: True)
+        monkeypatch.setattr(embed, "dimension", lambda: 3)
+        monkeypatch.setattr(embed, "provider", lambda: "local")
+        monkeypatch.setattr(embed, "model", lambda: "bge")
+        self._tap_seen(monkeypatch, {"acme__skills": "c1"})
+        entries = [{"name": "a", "tap": "acme/skills", "kind": "skill",
+                    "skill_md": "a/SKILL.md", "_body": "text", "description": ""}]
+        stats = dense.build(entries=entries, force=False)
+        assert stats is not None
+        assert "acme__skills" in stats["reused"]
+        assert embedded == [], "re-embedded a tap the shard already covered"
+
+    def test_a_tap_the_shard_does_not_cover_is_still_embedded(self, sandbox,
+                                                              with_backend,
+                                                              monkeypatch):
+        # The other half: shards cover the popular registries, the local model
+        # makes the long tail self-serve. If this regressed, an uncatalogued
+        # registry would be a dead end for a keyless user.
+        from boost_cli.core import embed
+        _store(rows=(("a", "acme/skills"),))
+        monkeypatch.setattr(embed, "embed",
+                            lambda texts, **kw: [[0.1, 0.2, 0.3] for _ in texts])
+        monkeypatch.setattr(embed, "available", lambda: True)
+        monkeypatch.setattr(embed, "dimension", lambda: 3)
+        monkeypatch.setattr(embed, "provider", lambda: "local")
+        monkeypatch.setattr(embed, "model", lambda: "bge")
+        self._tap_seen(monkeypatch, {"acme__skills": "c1", "new__repo": "z9"})
+        entries = [{"name": "b", "tap": "new/repo", "kind": "skill",
+                    "skill_md": "b/SKILL.md", "_body": "text", "description": ""}]
+        stats = dense.build(entries=entries, force=False)
+        assert stats is not None
+        assert "new/repo" in stats["reindexed"]