agentskills-retrieval 0.5.0__tar.gz

agentskills_retrieval-0.5.0/PKG-INFO

@@ -0,0 +1,161 @@
+Metadata-Version: 2.4
+Name: agentskills-retrieval
+Version: 0.5.0
+Summary: Query-time skill selection for the Agent Skills format (https://agentskills.io)
+License: MIT
+Author: Pratik Panda
+Requires-Python: >=3.12,<4.0
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development :: Libraries
+Requires-Dist: agentskills-core (>=0.5.0,<1.0)
+Project-URL: Homepage, https://agentskills.io
+Project-URL: Repository, https://github.com/pratikxpanda/agentskills-sdk
+Description-Content-Type: text/markdown
+
+# agentskills-retrieval
+
+[![PyPI](https://img.shields.io/pypi/v/agentskills-retrieval)](https://pypi.org/project/agentskills-retrieval/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/pratikxpanda/agentskills-sdk/blob/main/LICENSE)
+
+Query-time skill selection for the [Agent Skills](https://agentskills.io) SDK.
+
+The skills catalog is injected into the system prompt on **every turn**, so its cost is linear in the number of registered skills. Two things get worse as a registry grows, not one: the token bill, and the accuracy of the model's choice. A fifty-skill registry is both more expensive *and* worse at picking the right skill.
+
+`get_skills_catalog()` already accepts `include`, `exclude`, `tags` and `max_chars` — but every one of them requires the caller to know the answer in advance, and `max_chars` drops entries from the end, which is arbitrary with respect to relevance. This package narrows the catalog by *what was asked*.
+
+## Installation
+
+```bash
+pip install agentskills-retrieval
+```
+
+There are no dependencies beyond `agentskills-core`. The default selector is pure Python.
+
+## Usage
+
+```python
+from agentskills_retrieval import LexicalSelector, build_selected_catalog
+
+selector = LexicalSelector(registry)
+catalog = await build_selected_catalog(
+    registry, selector, "the checkout API is returning 503s"
+)
+```
+
+Or drive the two halves yourself, which is the whole integration surface:
+
+```python
+selection = await selector.select("the checkout API is returning 503s", limit=5)
+
+catalog = await registry.get_skills_catalog(
+    include=selection.skill_ids,
+    total=selection.considered,
+)
+```
+
+`include=` is applied **before any metadata is fetched**, so narrowing fifty skills to five also avoids forty-five provider round trips. That matters most for exactly the registries this package exists for.
+
+## Selectors
+
+| Selector | Ranks by | Needs |
+| --- | --- | --- |
+| `LexicalSelector` | Okapi BM25 over name, description, `when_to_use` and tags | nothing |
+| `EmbeddingSelector` | cosine similarity between query and skill vectors | an embedder you supply |
+
+`LexicalSelector` is the default because it works the moment the package is installed. An embedding ranker is better at paraphrase — it can match "the site is down" to a skill that says "service degradation", which BM25 cannot, because they share no word — but it is also an API key, a network hop and a bill. A package whose only ranker needs all three is a package most people never switch on.
+
+### Embedders
+
+No embedding SDK is a dependency of anything here. The contract is one method:
+
+```python
+class OpenAIEmbedder:
+    embedder_id = "text-embedding-3-small"
+
+    async def embed(self, texts):
+        reply = await client.embeddings.create(model=self.embedder_id, input=list(texts))
+        return [item.embedding for item in reply.data]
+```
+
+```python
+from agentskills_retrieval import EmbeddingSelector
+
+selector = EmbeddingSelector(registry, OpenAIEmbedder())
+```
+
+`load_embedder("myapp.embedders:build")` resolves the same thing from a dotted path when the name comes from configuration, mirroring how the eval harness resolves chat models.
+
+### Caching
+
+Skill vectors are cached by content hash, so re-registering an unchanged skill is free and editing one invalidates only itself. `embedder_id` is part of every cache key, because two models' vectors are not comparable and a cache that mixes them silently returns nonsense.
+
+The default cache is an in-process dict. Persistence is a two-method protocol — a hosted registry should not re-embed its corpus every time a process starts:
+
+```python
+class RedisEmbeddingCache:
+    def get(self, key: str) -> list[float] | None: ...
+    def set(self, key: str, vector: list[float]) -> None: ...
+
+selector = EmbeddingSelector(registry, embedder, cache=RedisEmbeddingCache())
+```
+
+## `when_not_to_use` is a penalty, not a match
+
+Selection metadata is indexed in two halves. `description`, `when_to_use`, the skill name and its tags are evidence *for* a skill. `when_not_to_use` is evidence *against* it, and is scored separately and subtracted.
+
+Folding both into one bag of words would make a skill match the very query its author wrote it to disclaim — "not for local test failures" would make the skill *more* likely to win on a query about a local test failure, because the words line up. The weight is `0.5` rather than `1.0` because a disclaimer is weaker evidence than a description: authors write far fewer of them, and phrase them loosely. Pass `negative_weight=0.0` to ignore them.
+
+## Selection is visible or it is not debuggable
+
+From inside an agent, a skill that was ranked out is indistinguishable from a skill that was never registered. So:
+
+- Every selection is logged at `INFO` on `agentskills.retrieval.*` with the scores that produced it.
+- `Selection.rejected` carries what did **not** make the cut, also best-first. "The right skill scored just under the floor" and "the right skill was never registered" are different bugs that look identical without it.
+- `build_selected_catalog` passes `total=` so the catalog reports the shortfall itself — `shown`/`total` on the XML root, a closing note in Markdown.
+
+## Floors, and what they can and cannot catch
+
+Returning the five best of fifty irrelevant skills is worse than returning nothing, so both selectors take a `min_score`.
+
+- **Embeddings**: cosine is bounded and comparable across corpora, so the floor is meaningful. The `0.25` default is still model-specific — some embedders put unrelated text around 0.7 — so treat it as a starting point.
+- **BM25**: scores are unbounded and corpus-relative, so there is no meaningful absolute floor above zero. The default of `0.0` catches the case that matters — the query shares no term with any skill — but it **cannot** catch a query that matches a common word and is nonetheless irrelevant. That limit is real, and it is why the recall numbers below are measured rather than assumed.
+
+When nothing clears the floor, `build_selected_catalog` returns the **full** catalog. "Selection has no opinion" is not "the agent should have no skills": a wrong prune silently removes a capability, which is a worse failure than a few wasted tokens. The fallback is logged.
+
+## What the query should be
+
+`select()` takes a string and never derives one, because the obvious default is wrong. The last user message alone fails for any conversation where the topic was established several turns ago — "try that again" ranks against nothing. Concatenating the whole history fails the other way, dragging in every topic the conversation has touched. The caller knows its own conversation shape; this package does not, and guessing on its behalf would be a silent accuracy regression rather than an obvious one.
+
+## Measured recall
+
+A ranker shipped without a measurement is a guess with an API. The fixture set in [`tests/conftest.py`](https://github.com/pratikxpanda/agentskills-sdk/blob/main/packages/retrieval/agentskills-retrieval/tests/conftest.py) pairs realistic queries with the skill that should win, and [`tests/test_recall.py`](https://github.com/pratikxpanda/agentskills-sdk/blob/main/packages/retrieval/agentskills-retrieval/tests/test_recall.py) asserts a floor that CI enforces, so a change that makes ranking worse fails the build.
+
+`LexicalSelector` over the fixture corpus:
+
+| Metric | Score |
+| --- | --- |
+| recall@1 | 0.85 |
+| recall@3 | 1.00 |
+
+The `EmbeddingSelector` figures are measured against a deterministic hashing embedder, which tests the plumbing rather than any real model's quality; a number from a stub embedder would be a claim about nothing. Run the harness against your own embedder before trusting it in production.
+
+Both numbers are over a small synthetic corpus. They are a regression guard, not a benchmark.
+
+## Not a default
+
+Nothing here is enabled unless you enable it. A registry that is not asked to select behaves exactly as it did before this package existed, byte for byte.
+
+## Security
+
+Selection reads skill metadata only, never bodies or resources. It executes nothing. An embedder you supply is your own code and your own network egress; this package neither imports nor configures one.
+
+## License
+
+MIT
+

agentskills_retrieval-0.5.0/README.md

@@ -0,0 +1,140 @@
+# agentskills-retrieval
+
+[![PyPI](https://img.shields.io/pypi/v/agentskills-retrieval)](https://pypi.org/project/agentskills-retrieval/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/pratikxpanda/agentskills-sdk/blob/main/LICENSE)
+
+Query-time skill selection for the [Agent Skills](https://agentskills.io) SDK.
+
+The skills catalog is injected into the system prompt on **every turn**, so its cost is linear in the number of registered skills. Two things get worse as a registry grows, not one: the token bill, and the accuracy of the model's choice. A fifty-skill registry is both more expensive *and* worse at picking the right skill.
+
+`get_skills_catalog()` already accepts `include`, `exclude`, `tags` and `max_chars` — but every one of them requires the caller to know the answer in advance, and `max_chars` drops entries from the end, which is arbitrary with respect to relevance. This package narrows the catalog by *what was asked*.
+
+## Installation
+
+```bash
+pip install agentskills-retrieval
+```
+
+There are no dependencies beyond `agentskills-core`. The default selector is pure Python.
+
+## Usage
+
+```python
+from agentskills_retrieval import LexicalSelector, build_selected_catalog
+
+selector = LexicalSelector(registry)
+catalog = await build_selected_catalog(
+    registry, selector, "the checkout API is returning 503s"
+)
+```
+
+Or drive the two halves yourself, which is the whole integration surface:
+
+```python
+selection = await selector.select("the checkout API is returning 503s", limit=5)
+
+catalog = await registry.get_skills_catalog(
+    include=selection.skill_ids,
+    total=selection.considered,
+)
+```
+
+`include=` is applied **before any metadata is fetched**, so narrowing fifty skills to five also avoids forty-five provider round trips. That matters most for exactly the registries this package exists for.
+
+## Selectors
+
+| Selector | Ranks by | Needs |
+| --- | --- | --- |
+| `LexicalSelector` | Okapi BM25 over name, description, `when_to_use` and tags | nothing |
+| `EmbeddingSelector` | cosine similarity between query and skill vectors | an embedder you supply |
+
+`LexicalSelector` is the default because it works the moment the package is installed. An embedding ranker is better at paraphrase — it can match "the site is down" to a skill that says "service degradation", which BM25 cannot, because they share no word — but it is also an API key, a network hop and a bill. A package whose only ranker needs all three is a package most people never switch on.
+
+### Embedders
+
+No embedding SDK is a dependency of anything here. The contract is one method:
+
+```python
+class OpenAIEmbedder:
+    embedder_id = "text-embedding-3-small"
+
+    async def embed(self, texts):
+        reply = await client.embeddings.create(model=self.embedder_id, input=list(texts))
+        return [item.embedding for item in reply.data]
+```
+
+```python
+from agentskills_retrieval import EmbeddingSelector
+
+selector = EmbeddingSelector(registry, OpenAIEmbedder())
+```
+
+`load_embedder("myapp.embedders:build")` resolves the same thing from a dotted path when the name comes from configuration, mirroring how the eval harness resolves chat models.
+
+### Caching
+
+Skill vectors are cached by content hash, so re-registering an unchanged skill is free and editing one invalidates only itself. `embedder_id` is part of every cache key, because two models' vectors are not comparable and a cache that mixes them silently returns nonsense.
+
+The default cache is an in-process dict. Persistence is a two-method protocol — a hosted registry should not re-embed its corpus every time a process starts:
+
+```python
+class RedisEmbeddingCache:
+    def get(self, key: str) -> list[float] | None: ...
+    def set(self, key: str, vector: list[float]) -> None: ...
+
+selector = EmbeddingSelector(registry, embedder, cache=RedisEmbeddingCache())
+```
+
+## `when_not_to_use` is a penalty, not a match
+
+Selection metadata is indexed in two halves. `description`, `when_to_use`, the skill name and its tags are evidence *for* a skill. `when_not_to_use` is evidence *against* it, and is scored separately and subtracted.
+
+Folding both into one bag of words would make a skill match the very query its author wrote it to disclaim — "not for local test failures" would make the skill *more* likely to win on a query about a local test failure, because the words line up. The weight is `0.5` rather than `1.0` because a disclaimer is weaker evidence than a description: authors write far fewer of them, and phrase them loosely. Pass `negative_weight=0.0` to ignore them.
+
+## Selection is visible or it is not debuggable
+
+From inside an agent, a skill that was ranked out is indistinguishable from a skill that was never registered. So:
+
+- Every selection is logged at `INFO` on `agentskills.retrieval.*` with the scores that produced it.
+- `Selection.rejected` carries what did **not** make the cut, also best-first. "The right skill scored just under the floor" and "the right skill was never registered" are different bugs that look identical without it.
+- `build_selected_catalog` passes `total=` so the catalog reports the shortfall itself — `shown`/`total` on the XML root, a closing note in Markdown.
+
+## Floors, and what they can and cannot catch
+
+Returning the five best of fifty irrelevant skills is worse than returning nothing, so both selectors take a `min_score`.
+
+- **Embeddings**: cosine is bounded and comparable across corpora, so the floor is meaningful. The `0.25` default is still model-specific — some embedders put unrelated text around 0.7 — so treat it as a starting point.
+- **BM25**: scores are unbounded and corpus-relative, so there is no meaningful absolute floor above zero. The default of `0.0` catches the case that matters — the query shares no term with any skill — but it **cannot** catch a query that matches a common word and is nonetheless irrelevant. That limit is real, and it is why the recall numbers below are measured rather than assumed.
+
+When nothing clears the floor, `build_selected_catalog` returns the **full** catalog. "Selection has no opinion" is not "the agent should have no skills": a wrong prune silently removes a capability, which is a worse failure than a few wasted tokens. The fallback is logged.
+
+## What the query should be
+
+`select()` takes a string and never derives one, because the obvious default is wrong. The last user message alone fails for any conversation where the topic was established several turns ago — "try that again" ranks against nothing. Concatenating the whole history fails the other way, dragging in every topic the conversation has touched. The caller knows its own conversation shape; this package does not, and guessing on its behalf would be a silent accuracy regression rather than an obvious one.
+
+## Measured recall
+
+A ranker shipped without a measurement is a guess with an API. The fixture set in [`tests/conftest.py`](https://github.com/pratikxpanda/agentskills-sdk/blob/main/packages/retrieval/agentskills-retrieval/tests/conftest.py) pairs realistic queries with the skill that should win, and [`tests/test_recall.py`](https://github.com/pratikxpanda/agentskills-sdk/blob/main/packages/retrieval/agentskills-retrieval/tests/test_recall.py) asserts a floor that CI enforces, so a change that makes ranking worse fails the build.
+
+`LexicalSelector` over the fixture corpus:
+
+| Metric | Score |
+| --- | --- |
+| recall@1 | 0.85 |
+| recall@3 | 1.00 |
+
+The `EmbeddingSelector` figures are measured against a deterministic hashing embedder, which tests the plumbing rather than any real model's quality; a number from a stub embedder would be a claim about nothing. Run the harness against your own embedder before trusting it in production.
+
+Both numbers are over a small synthetic corpus. They are a regression guard, not a benchmark.
+
+## Not a default
+
+Nothing here is enabled unless you enable it. A registry that is not asked to select behaves exactly as it did before this package existed, byte for byte.
+
+## Security
+
+Selection reads skill metadata only, never bodies or resources. It executes nothing. An embedder you supply is your own code and your own network egress; this package neither imports nor configures one.
+
+## License
+
+MIT

agentskills_retrieval-0.5.0/agentskills_retrieval/__init__.py

@@ -0,0 +1,77 @@
+"""Query-time skill selection for the Agent Skills SDK.
+
+The catalog is injected on every turn, so per-turn prompt cost is
+linear in the number of registered skills — and selection accuracy
+falls as the candidate list grows, so a large registry is both more
+expensive and worse at choosing. This package narrows the catalog to
+what the current query is actually about.
+
+Two selectors ship. :class:`LexicalSelector` is BM25 and needs nothing
+installed; :class:`EmbeddingSelector` takes any embedder behind a
+one-method protocol. Both return scores, and both compose with the
+``include=`` filter the registry already has::
+
+    from agentskills_retrieval import LexicalSelector, build_selected_catalog
+
+    selector = LexicalSelector(registry)
+    catalog = await build_selected_catalog(registry, selector, "checkout is down")
+
+Nothing here is on by default. A registry that is not asked to select
+behaves exactly as it did before this package existed.
+"""
+
+from agentskills_retrieval.catalog import build_selected_catalog
+from agentskills_retrieval.corpus import (
+    STOPWORDS,
+    SkillDocument,
+    build_corpus,
+    document_of,
+    tokenize,
+)
+from agentskills_retrieval.embedding import (
+    DEFAULT_MIN_SCORE as DEFAULT_EMBEDDING_MIN_SCORE,
+)
+from agentskills_retrieval.embedding import (
+    Embedder,
+    EmbeddingCache,
+    EmbeddingSelector,
+    InMemoryEmbeddingCache,
+    cosine,
+    load_embedder,
+)
+from agentskills_retrieval.lexical import (
+    DEFAULT_MIN_SCORE as DEFAULT_LEXICAL_MIN_SCORE,
+)
+from agentskills_retrieval.lexical import (
+    NEGATIVE_WEIGHT,
+    LexicalSelector,
+)
+from agentskills_retrieval.selector import (
+    DEFAULT_LIMIT,
+    ScoredSkill,
+    Selection,
+    SkillSelector,
+)
+
+__all__ = [
+    "DEFAULT_EMBEDDING_MIN_SCORE",
+    "DEFAULT_LEXICAL_MIN_SCORE",
+    "DEFAULT_LIMIT",
+    "NEGATIVE_WEIGHT",
+    "STOPWORDS",
+    "Embedder",
+    "EmbeddingCache",
+    "EmbeddingSelector",
+    "InMemoryEmbeddingCache",
+    "LexicalSelector",
+    "ScoredSkill",
+    "Selection",
+    "SkillDocument",
+    "SkillSelector",
+    "build_corpus",
+    "build_selected_catalog",
+    "cosine",
+    "document_of",
+    "load_embedder",
+    "tokenize",
+]

agentskills_retrieval-0.5.0/agentskills_retrieval/catalog.py

@@ -0,0 +1,72 @@
+"""Turning a selection into a catalog.
+
+The one function here is the whole integration surface: rank, then
+hand the winning IDs to the filter ``get_skills_catalog`` already has.
+Core learns nothing about ranking; retrieval learns nothing about
+rendering.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any, Literal
+
+from agentskills_core import get_logger
+from agentskills_retrieval.selector import DEFAULT_LIMIT
+
+if TYPE_CHECKING:
+    from agentskills_core import SkillRegistry
+    from agentskills_retrieval.selector import SkillSelector
+
+_logger = get_logger(__name__)
+
+
+async def build_selected_catalog(
+    registry: SkillRegistry,
+    selector: SkillSelector,
+    query: str,
+    *,
+    limit: int = DEFAULT_LIMIT,
+    format: Literal["xml", "markdown"] = "xml",
+    **catalog_kwargs: Any,
+) -> str:
+    """Rank the registry against *query* and build a catalog of the winners.
+
+    The result reports its own narrowing — ``shown``/``total`` on the
+    XML root, a closing note in Markdown — because from inside an agent
+    a skill that was ranked out is indistinguishable from one that was
+    never registered, and that is a miserable thing to debug.
+
+    When nothing clears the selector's floor the **full** catalog is
+    returned. "Selection has no opinion" is not "the agent should have
+    no skills": a wrong prune silently removes a capability, which is a
+    worse failure than a few wasted tokens. The fallback is logged.
+
+    Args:
+        registry: The registry to build from.
+        selector: Any :class:`~agentskills_retrieval.SkillSelector`.
+        query: Text to rank against.
+        limit: Maximum number of skills to advertise.
+        format: ``"xml"`` or ``"markdown"``, as for
+            :meth:`~agentskills_core.SkillRegistry.get_skills_catalog`.
+        **catalog_kwargs: Passed straight through to
+            :meth:`~agentskills_core.SkillRegistry.get_skills_catalog`.
+            ``include`` and ``total`` are supplied by this function.
+
+    Returns:
+        A catalog string ready for a system prompt.
+    """
+    selection = await selector.select(query, limit=limit)
+    if selection.is_empty:
+        _logger.info(
+            "Selection matched nothing for %r; falling back to the full catalog of %d skills",
+            query,
+            selection.considered,
+        )
+        return await registry.get_skills_catalog(format=format, **catalog_kwargs)
+
+    return await registry.get_skills_catalog(
+        format=format,
+        include=selection.skill_ids,
+        total=selection.considered,
+        **catalog_kwargs,
+    )

agentskills_retrieval-0.5.0/agentskills_retrieval/corpus.py

@@ -0,0 +1,136 @@
+"""The searchable text of a skill, and how it is tokenized.
+
+Both selectors rank the same corpus: whatever the catalog would have
+shown a model.  Building it here rather than in each selector means a
+lexical and an embedding ranking are answering the same question over
+the same words, so a difference between them is a difference in method
+and not in what they were allowed to read.
+
+``when_not_to_use`` is deliberately kept apart from the rest.  It is
+evidence *against* a skill, and folding it into one bag of words would
+make a skill match the very query its author wrote it to disclaim.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import re
+from dataclasses import dataclass
+from hashlib import blake2b
+from typing import TYPE_CHECKING, Any
+
+from agentskills_core import SELECTION_FIELDS, get_logger
+
+if TYPE_CHECKING:
+    from agentskills_core import Skill, SkillRegistry
+
+_logger = get_logger(__name__)
+
+_WORD = re.compile(r"[a-z0-9]+")
+
+#: Words carried by almost every skill description, so they separate nothing.
+#:
+#: Deliberately tiny.  A long list is a language model of its own that
+#: nobody here is qualified to maintain, and BM25 already discounts a
+#: term that appears in most documents.  This exists only to stop very
+#: short queries being dominated by their function words.
+STOPWORDS = frozenset(
+    [
+        "a", "an", "and", "are", "as", "at", "be", "by", "for", "from",
+        "how", "in", "into", "is", "it", "of", "on", "or", "that", "the",
+        "this", "to", "use", "used", "using", "was", "what", "when",
+        "where", "which", "who", "why", "with", "you", "your",
+    ]
+)  # fmt: skip
+
+
+def tokenize(text: str) -> list[str]:
+    """Split *text* into lowercase alphanumeric terms, minus stopwords.
+
+    No stemming: it would need a dependency, and this package's whole
+    claim is that it is useful with none.  The cost is real — "deploys"
+    will not match "deploy" — and it is the main thing an embedding
+    selector buys back.
+    """
+    return [word for word in _WORD.findall(text.casefold()) if word not in STOPWORDS]
+
+
+@dataclass(frozen=True)
+class SkillDocument:
+    """One skill, as a ranker sees it.
+
+    Attributes:
+        skill_id: The registered ID, which is what a selection returns.
+        positive_text: Name, description, ``when_to_use`` and tags.
+        negative_text: ``when_not_to_use``, scored separately.
+        content_hash: Digest of the text above, so an embedding can be
+            cached against it and invalidated by nothing else.
+    """
+
+    skill_id: str
+    positive_text: str
+    negative_text: str
+    content_hash: str
+
+    @property
+    def positive_terms(self) -> list[str]:
+        """The tokens a query is matched against."""
+        return tokenize(self.positive_text)
+
+    @property
+    def negative_terms(self) -> list[str]:
+        """The tokens a query is penalised for matching."""
+        return tokenize(self.negative_text)
+
+
+def _strings(value: Any) -> list[str]:
+    """Return *value* as a list of non-empty strings, or nothing."""
+    if not isinstance(value, list):
+        return []
+    return [item.strip() for item in value if isinstance(item, str) and item.strip()]
+
+
+def document_of(skill_id: str, meta: dict[str, Any]) -> SkillDocument:
+    """Build the searchable document for one skill's metadata."""
+    positive = [
+        str(meta.get("name") or skill_id),
+        str(meta.get("description") or ""),
+        *_strings(meta.get(SELECTION_FIELDS[0])),
+    ]
+    container = meta.get("metadata")
+    if isinstance(container, dict):
+        positive += _strings(container.get("tags"))
+
+    negative = _strings(meta.get(SELECTION_FIELDS[1]))
+
+    positive_text = "\n".join(part for part in positive if part)
+    negative_text = "\n".join(negative)
+    digest = blake2b(f"{positive_text}\x00{negative_text}".encode(), digest_size=16)
+    return SkillDocument(
+        skill_id=skill_id,
+        positive_text=positive_text,
+        negative_text=negative_text,
+        content_hash=digest.hexdigest(),
+    )
+
+
+async def build_corpus(registry: SkillRegistry, *, concurrency: int = 8) -> list[SkillDocument]:
+    """Fetch metadata for every registered skill and index it.
+
+    Args:
+        registry: The registry to read.
+        concurrency: Ceiling on simultaneous metadata fetches, which
+            matters when the provider is network-backed.
+
+    Returns:
+        One :class:`SkillDocument` per registered skill, in ID order.
+    """
+    semaphore = asyncio.Semaphore(concurrency)
+
+    async def fetch(skill: Skill) -> SkillDocument:
+        async with semaphore:
+            return document_of(skill.get_id(), await skill.get_metadata())
+
+    corpus = list(await asyncio.gather(*(fetch(skill) for skill in registry.list_skills())))
+    _logger.debug("Indexed %d skills for selection", len(corpus))
+    return corpus

agentskills_retrieval-0.5.0/agentskills_retrieval/embedding.py

@@ -0,0 +1,249 @@
+"""Embedding-based selection, with no vendor SDK anywhere in the tree.
+
+BM25 cannot match "the site is down" to a skill that says "service
+degradation", because they share no word.  Embeddings can, and the
+price is a model — so the model stays outside the package, behind a
+one-method protocol resolved from a dotted path at run time, exactly
+as the eval harness resolves chat models.  Nothing here imports
+anything a user did not ask for.
+
+Vectors are cached by content hash, so re-registering an unchanged
+skill costs nothing and editing one invalidates only itself.  The cache
+is a protocol too: a hosted registry should not re-embed its corpus
+every time a process starts.
+"""
+
+from __future__ import annotations
+
+import math
+from importlib import import_module
+from typing import TYPE_CHECKING, Protocol, runtime_checkable
+
+from agentskills_core import get_logger
+from agentskills_retrieval.corpus import SkillDocument, build_corpus
+from agentskills_retrieval.selector import DEFAULT_LIMIT, ScoredSkill, Selection
+
+if TYPE_CHECKING:
+    from collections.abc import Sequence
+
+    from agentskills_core import SkillRegistry
+
+_logger = get_logger(__name__)
+
+#: Cosine similarity below this is treated as no match.
+#:
+#: Unlike BM25 this floor is meaningful, because cosine is bounded and
+#: comparable across corpora.  It is still model-specific — some
+#: embedders put unrelated text around 0.7 — so it is a constructor
+#: argument and this is only a starting point.
+DEFAULT_MIN_SCORE = 0.25
+
+
+@runtime_checkable
+class Embedder(Protocol):
+    """The whole contract: a name, and a way to turn text into vectors.
+
+    Implement it over any client::
+
+        class OpenAIEmbedder:
+            embedder_id = "text-embedding-3-small"
+
+            async def embed(self, texts):
+                reply = await client.embeddings.create(
+                    model=self.embedder_id, input=list(texts)
+                )
+                return [item.embedding for item in reply.data]
+
+    ``embedder_id`` is part of every cache key, because two models'
+    vectors are not comparable and a cache that mixes them silently
+    returns nonsense.
+    """
+
+    embedder_id: str
+
+    async def embed(self, texts: Sequence[str]) -> list[list[float]]:
+        """Return one vector per text, in the same order."""
+        ...
+
+
+def load_embedder(spec: str) -> Embedder:
+    """Resolve ``module:factory`` to an embedder.
+
+    Args:
+        spec: A dotted module path, a colon, and the name of a
+            zero-argument callable returning an :class:`Embedder`.
+
+    Returns:
+        The embedder the factory built.
+
+    Raises:
+        ValueError: If the spec is malformed, cannot be imported, or
+            does not produce something with ``embedder_id`` and
+            ``embed``.
+    """
+    module_name, _, factory_name = spec.partition(":")
+    if not module_name or not factory_name:
+        raise ValueError(f"embedder spec must look like 'module:factory', got {spec!r}")
+
+    try:
+        module = import_module(module_name)
+    except ImportError as exc:
+        raise ValueError(f"cannot import '{module_name}': {exc}") from exc
+
+    try:
+        factory = getattr(module, factory_name)
+    except AttributeError as exc:
+        raise ValueError(f"'{module_name}' has no attribute '{factory_name}'") from exc
+
+    embedder = factory()
+    if not isinstance(embedder, Embedder):
+        raise ValueError(
+            f"{spec} returned {type(embedder).__name__}, which has no 'embedder_id' and 'embed'."
+        )
+    return embedder
+
+
+@runtime_checkable
+class EmbeddingCache(Protocol):
+    """Somewhere to keep vectors between processes.
+
+    Two methods, both synchronous, because the interesting backends
+    (a file, a table, Redis) are all fast enough that making callers
+    write an async adapter buys nothing.
+    """
+
+    def get(self, key: str) -> list[float] | None:
+        """Return the vector stored under *key*, or ``None``."""
+        ...
+
+    def set(self, key: str, vector: list[float]) -> None:
+        """Store *vector* under *key*."""
+        ...
+
+
+class InMemoryEmbeddingCache:
+    """The default cache: a dict that dies with the process."""
+
+    def __init__(self) -> None:
+        self._vectors: dict[str, list[float]] = {}
+
+    def get(self, key: str) -> list[float] | None:
+        """Return the vector stored under *key*, or ``None``."""
+        return self._vectors.get(key)
+
+    def set(self, key: str, vector: list[float]) -> None:
+        """Store *vector* under *key*."""
+        self._vectors[key] = vector
+
+    def __len__(self) -> int:
+        """Number of cached vectors, which is what tests assert on."""
+        return len(self._vectors)
+
+
+def cosine(left: Sequence[float], right: Sequence[float]) -> float:
+    """Return the cosine similarity of two vectors, or 0.0 if either is zero."""
+    dot = sum(a * b for a, b in zip(left, right, strict=True))
+    norm = math.sqrt(sum(a * a for a in left)) * math.sqrt(sum(b * b for b in right))
+    return dot / norm if norm else 0.0
+
+
+class EmbeddingSelector:
+    """Select skills by cosine similarity between query and skill vectors.
+
+    Args:
+        registry: The registry whose skills are ranked.
+        embedder: Anything satisfying :class:`Embedder`.  Build one with
+            :func:`load_embedder` when the name comes from config.
+        cache: Where skill vectors are kept.  Defaults to an in-process
+            dict; pass your own to survive a restart.
+        min_score: Cosine below this is rejected.  See
+            :data:`DEFAULT_MIN_SCORE`.
+        negative_weight: How much similarity to ``when_not_to_use``
+            subtracts.  Pass ``0.0`` to ignore disclaimers.
+    """
+
+    def __init__(
+        self,
+        registry: SkillRegistry,
+        embedder: Embedder,
+        *,
+        cache: EmbeddingCache | None = None,
+        min_score: float = DEFAULT_MIN_SCORE,
+        negative_weight: float = 0.5,
+    ) -> None:
+        self._registry = registry
+        self._embedder = embedder
+        self._cache: EmbeddingCache = cache if cache is not None else InMemoryEmbeddingCache()
+        self._min_score = min_score
+        self._negative_weight = negative_weight
+        self._corpus: list[SkillDocument] = []
+        self._positive: list[list[float]] = []
+        self._negative: list[list[float] | None] = []
+        self._indexed_ids: tuple[str, ...] = ()
+
+    def _key(self, content_hash: str, field: str) -> str:
+        return f"{self._embedder.embedder_id}:{field}:{content_hash}"
+
+    async def _vectors(self, wanted: list[tuple[str, str]]) -> list[list[float]]:
+        """Return a vector per ``(cache key, text)``, embedding only misses."""
+        cached = [self._cache.get(key) for key, _ in wanted]
+        missing = [i for i, vector in enumerate(cached) if vector is None]
+        if missing:
+            fresh = await self._embedder.embed([wanted[i][1] for i in missing])
+            if len(fresh) != len(missing):
+                raise ValueError(
+                    f"{self._embedder.embedder_id} returned {len(fresh)} vectors "
+                    f"for {len(missing)} texts"
+                )
+            for i, vector in zip(missing, fresh, strict=True):
+                self._cache.set(wanted[i][0], vector)
+                cached[i] = vector
+            _logger.debug(
+                "Embedded %d of %d texts; the rest were cached", len(missing), len(wanted)
+            )
+        return [vector for vector in cached if vector is not None]
+
+    async def index(self) -> None:
+        """Embed every registered skill, reusing anything already cached."""
+        self._corpus = await build_corpus(self._registry)
+        self._positive = await self._vectors(
+            [(self._key(doc.content_hash, "pos"), doc.positive_text) for doc in self._corpus]
+        )
+
+        negatives = [doc for doc in self._corpus if doc.negative_text]
+        vectors = await self._vectors(
+            [(self._key(doc.content_hash, "neg"), doc.negative_text) for doc in negatives]
+        )
+        by_id = dict(zip((doc.skill_id for doc in negatives), vectors, strict=True))
+        self._negative = [by_id.get(doc.skill_id) for doc in self._corpus]
+        self._indexed_ids = tuple(doc.skill_id for doc in self._corpus)
+
+    async def _ensure_index(self) -> None:
+        if self._indexed_ids != tuple(skill.get_id() for skill in self._registry.list_skills()):
+            await self.index()
+
+    async def select(self, query: str, *, limit: int = DEFAULT_LIMIT) -> Selection:
+        """Return the best *limit* skills for *query*, by cosine similarity."""
+        await self._ensure_index()
+
+        [query_vector] = await self._embedder.embed([query])
+        scored = []
+        for doc, positive, negative in zip(
+            self._corpus, self._positive, self._negative, strict=True
+        ):
+            score = cosine(query_vector, positive)
+            if negative is not None:
+                score -= self._negative_weight * max(cosine(query_vector, negative), 0.0)
+            scored.append(ScoredSkill(doc.skill_id, score))
+        scored.sort(key=lambda s: (-s.score, s.skill_id))
+
+        selected = [s for s in scored if s.score > self._min_score][:limit]
+        chosen = {s.skill_id for s in selected}
+        selection = Selection(
+            query=query,
+            selected=selected,
+            rejected=[s for s in scored if s.skill_id not in chosen],
+            considered=len(scored),
+        )
+        _logger.info("EmbeddingSelector %s", selection.describe())
+        return selection

agentskills_retrieval-0.5.0/agentskills_retrieval/lexical.py

@@ -0,0 +1,170 @@
+"""BM25 ranking over skill metadata, with no dependencies at all.
+
+This is the default selector, and it is the default because it works
+the moment the package is installed.  An embedding selector is better
+at paraphrase; it is also an API key, a network hop and a bill, and a
+package whose only ranker needs all three is a package most people
+never switch on.
+
+Scoring is Okapi BM25 over ``description``, ``when_to_use``, the skill
+name and its tags, minus a weighted BM25 over ``when_not_to_use``.  A
+skill whose author wrote "not for local test failures" should lose
+ground on a query about a local test failure, not gain it for sharing
+the vocabulary.
+"""
+
+from __future__ import annotations
+
+import math
+from collections import Counter
+from typing import TYPE_CHECKING
+
+from agentskills_core import get_logger
+from agentskills_retrieval.corpus import SkillDocument, build_corpus, tokenize
+from agentskills_retrieval.selector import DEFAULT_LIMIT, ScoredSkill, Selection
+
+if TYPE_CHECKING:
+    from agentskills_core import SkillRegistry
+
+_logger = get_logger(__name__)
+
+#: Term-frequency saturation. The standard value; nothing here justifies tuning it.
+K1 = 1.5
+
+#: Length normalisation, also the standard value.
+B = 0.75
+
+#: How much a ``when_not_to_use`` match counts against a skill.
+#:
+#: Below 1.0 on purpose: a disclaimer is weaker evidence than a
+#: description, because authors write far fewer of them and phrase them
+#: loosely.  At 1.0 a single shared word in a disclaimer could cancel a
+#: genuine description match.
+NEGATIVE_WEIGHT = 0.5
+
+#: Scores at or below this are treated as no match at all.
+#:
+#: BM25 is unbounded and corpus-relative, so there is no meaningful
+#: absolute floor above zero.  Zero still catches the case that matters
+#: — the query shares no term with any skill — but it cannot catch a
+#: query that matches a common word and is nonetheless irrelevant.
+#: That limit is real and is why recall@k is measured rather than assumed.
+DEFAULT_MIN_SCORE = 0.0
+
+
+class _Bm25Index:
+    """A scored field across the corpus."""
+
+    def __init__(self, documents: list[list[str]]) -> None:
+        self._lengths = [len(terms) for terms in documents]
+        self._avg_length = (sum(self._lengths) / len(self._lengths)) if self._lengths else 0.0
+        self._frequencies = [Counter(terms) for terms in documents]
+
+        seen: Counter[str] = Counter()
+        for frequency in self._frequencies:
+            seen.update(frequency.keys())
+        count = len(documents)
+        self._idf = {
+            # Lucene's variant, which cannot go negative for a term in
+            # most documents; the classic form can, and a negative IDF
+            # turns a match into a penalty.
+            term: math.log(1 + (count - n + 0.5) / (n + 0.5))
+            for term, n in seen.items()
+        }
+
+    def score(self, index: int, terms: list[str]) -> float:
+        """Score document *index* against the query *terms*."""
+        if not self._avg_length:
+            return 0.0
+        frequency = self._frequencies[index]
+        length = self._lengths[index]
+        total = 0.0
+        for term in terms:
+            occurrences = frequency.get(term, 0)
+            if not occurrences:
+                continue
+            denominator = occurrences + K1 * (1 - B + B * length / self._avg_length)
+            total += self._idf[term] * occurrences * (K1 + 1) / denominator
+        return total
+
+
+class LexicalSelector:
+    """Select skills by BM25 over their catalog metadata.
+
+    The corpus is built on first use and reused until the registry's
+    set of skill IDs changes, so a long-lived agent pays for indexing
+    once.
+
+    Args:
+        registry: The registry whose skills are ranked.
+        min_score: Scores at or below this are rejected outright.  See
+            :data:`DEFAULT_MIN_SCORE` for why the default is zero.
+        negative_weight: How much a ``when_not_to_use`` match subtracts.
+            Pass ``0.0`` to ignore disclaimers entirely.
+    """
+
+    def __init__(
+        self,
+        registry: SkillRegistry,
+        *,
+        min_score: float = DEFAULT_MIN_SCORE,
+        negative_weight: float = NEGATIVE_WEIGHT,
+    ) -> None:
+        self._registry = registry
+        self._min_score = min_score
+        self._negative_weight = negative_weight
+        self._corpus: list[SkillDocument] = []
+        self._positive = _Bm25Index([])
+        self._negative = _Bm25Index([])
+        self._indexed_ids: tuple[str, ...] = ()
+
+    async def index(self) -> None:
+        """Build the BM25 index, discarding any previous one."""
+        self._corpus = await build_corpus(self._registry)
+        self._positive = _Bm25Index([doc.positive_terms for doc in self._corpus])
+        self._negative = _Bm25Index([doc.negative_terms for doc in self._corpus])
+        self._indexed_ids = tuple(doc.skill_id for doc in self._corpus)
+
+    async def _ensure_index(self) -> None:
+        if self._indexed_ids != tuple(skill.get_id() for skill in self._registry.list_skills()):
+            await self.index()
+
+    async def select(self, query: str, *, limit: int = DEFAULT_LIMIT) -> Selection:
+        """Return the best *limit* skills for *query*.
+
+        Args:
+            query: Free text to rank against.  The caller decides what
+                this is; see the README on why the last user message
+                alone is a poor default.
+            limit: Maximum number of skills to return.
+
+        Returns:
+            A :class:`~agentskills_retrieval.Selection`, whose
+            ``skill_ids`` feed ``get_skills_catalog(include=...)``.
+        """
+        await self._ensure_index()
+
+        terms = tokenize(query)
+        scored = [
+            ScoredSkill(
+                doc.skill_id,
+                self._positive.score(i, terms)
+                - self._negative_weight * self._negative.score(i, terms),
+            )
+            for i, doc in enumerate(self._corpus)
+        ]
+        # Ties break by ID so the same registry and query always give
+        # the same catalog; a prompt that varies run to run is not one
+        # you can debug.
+        scored.sort(key=lambda s: (-s.score, s.skill_id))
+
+        selected = [s for s in scored if s.score > self._min_score][:limit]
+        chosen = {s.skill_id for s in selected}
+        selection = Selection(
+            query=query,
+            selected=selected,
+            rejected=[s for s in scored if s.skill_id not in chosen],
+            considered=len(scored),
+        )
+        _logger.info("LexicalSelector %s", selection.describe())
+        return selection

agentskills_retrieval-0.5.0/agentskills_retrieval/selector.py

@@ -0,0 +1,87 @@
+"""What a selector is, and what it returns.
+
+A selector answers one question — *given this query, which of these
+skills are worth putting in front of the model?* — and it answers it
+with scores, not a bare list, because a caller that cannot see why a
+skill was dropped cannot debug an agent that failed to use it.
+
+The output composes with the filter the catalog already has::
+
+    selection = await selector.select("the checkout API is down")
+    catalog = await registry.get_skills_catalog(
+        include=selection.skill_ids,
+        total=selection.considered,
+    )
+
+``include=`` is applied before any metadata is fetched, so narrowing
+fifty skills to five also avoids forty-five provider round trips.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Protocol, runtime_checkable
+
+#: Skills a selection returns unless the caller asks for more or fewer.
+DEFAULT_LIMIT = 5
+
+
+@dataclass(frozen=True)
+class ScoredSkill:
+    """One skill and what it scored, in the selector's own units."""
+
+    skill_id: str
+    score: float
+
+
+@dataclass(frozen=True)
+class Selection:
+    """The outcome of one selection, including what it rejected.
+
+    Attributes:
+        query: The text that was ranked against.
+        selected: Skills that cleared the floor, best first.
+        rejected: Skills that did not, also best first.  Kept because
+            "the right skill scored just under the floor" and "the
+            right skill was not registered" are different bugs and
+            look identical without this.
+        considered: How many skills were scored, which is the
+            denominator to report in the catalog.
+    """
+
+    query: str
+    selected: list[ScoredSkill]
+    rejected: list[ScoredSkill]
+    considered: int
+
+    @property
+    def skill_ids(self) -> list[str]:
+        """The selected IDs, ready to pass to ``include=``."""
+        return [scored.skill_id for scored in self.selected]
+
+    @property
+    def is_empty(self) -> bool:
+        """Whether nothing cleared the floor."""
+        return not self.selected
+
+    def describe(self) -> str:
+        """Return a one-line summary with scores, for logs and reports."""
+        if self.is_empty:
+            best = f", best rejected {self.rejected[0].score:.3f}" if self.rejected else ""
+            return f"no skill cleared the floor for {self.query!r} of {self.considered}{best}"
+        scores = ", ".join(f"{s.skill_id}={s.score:.3f}" for s in self.selected)
+        return f"selected {len(self.selected)} of {self.considered} for {self.query!r}: {scores}"
+
+
+@runtime_checkable
+class SkillSelector(Protocol):
+    """Rank registered skills against a query.
+
+    One method, so a caller can supply its own ranker — a hosted
+    reranking API, a hand-written routing table — without inheriting
+    anything from this package.
+    """
+
+    async def select(self, query: str, *, limit: int = DEFAULT_LIMIT) -> Selection:
+        """Return the best *limit* skills for *query*, with scores."""
+        ...

agentskills_retrieval-0.5.0/pyproject.toml

@@ -0,0 +1,26 @@
+[tool.poetry]
+name = "agentskills-retrieval"
+version = "0.5.0"
+description = "Query-time skill selection for the Agent Skills format (https://agentskills.io)"
+license = "MIT"
+authors = ["Pratik Panda"]
+readme = "README.md"
+homepage = "https://agentskills.io"
+repository = "https://github.com/pratikxpanda/agentskills-sdk"
+packages = [{include = "agentskills_retrieval"}]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Topic :: Software Development :: Libraries",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+]
+
+[tool.poetry.dependencies]
+python = ">=3.12,<4.0"
+agentskills-core = ">=0.5.0,<1.0"
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"