signalwire-sdk 3.7.2.dev132__tar.gz → 3.7.2.dev134__tar.gz

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## [Unreleased]
+
+### Added
+- `sw-search --query-stopwords` (and `IndexBuilder(query_stopwords=...)`)
+  records terms every document in the index is about, such as the product's
+  name. The search service and the native_vector_search skill drop them from
+  queries before embedding and keyword search, unless fewer than two content
+  words would remain, so "what is SignalWire" still searches for SignalWire.
+  In a corpus about one product, the name only favours the chunks that spell
+  it out: "SignalWire call pricing" ranked phone-number fees above call rates.
+  On SignalWire's corpus (58 labelled questions) hit@5 went from 0.88 to 0.95
+  and MRR@10 from 0.678 to 0.745; without the guard, a bare "SignalWire" fell
+  out of the top 10.
+- Chunks can carry a `rank_weight` between 0 and 1 in their metadata. Search
+  multiplies it into the chunk's score, so an index can mark secondary sources,
+  such as commentary indexed next to reference material, and the reference
+  ranks first when both match. It only demotes: values outside that range are
+  ignored.
+
+### Fixed
+- Search's same-file penalty now applies per document. It used to key on the
+  file a chunk was read from, so in a JSON file holding many documents every
+  chunk counted as one document and pushed the others down: in a pricing
+  collection, 50 rate cards shared one file, and the per-minute card for
+  "outbound call pricing" fell from 9th by relevance to below the top 20. A
+  chunk's document is its `document` metadata, else `url`, else `source`, else
+  its file, the same key the JSON loader already uses to tell documents apart.
+  Indexes with one document per file rank as before.
+- The penalty's diversity pass no longer raises `KeyError` on a result
+  without `filename` metadata.
+
 ## [3.7.2] - 2026-10-14
 
 ### Added

{signalwire_sdk-3.7.2.dev132/signalwire/signalwire_sdk.egg-info → signalwire_sdk-3.7.2.dev134}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: signalwire-sdk
-Version: 3.7.2.dev132
+Version: 3.7.2.dev134
 Summary: SignalWire SDK for AI voice agents, live call control (RELAY) and REST. Run sw-pydocs for its documentation.
 Author-email: SignalWire Team <info@signalwire.com>
 License: MIT

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/docs/cli_guide.md

@@ -2325,6 +2325,7 @@ sw-search <source> [<source> ...] [options]
 - `--topic-threshold FLOAT` - Threshold for topic-based chunking (default: 0.3)
 - `--index-nlp-backend BACKEND` - NLP backend for processing: `nltk` (default) or `spacy`
 - `--embed-text {content,processed}` - What the embedding model sees: the chunk as written (`content`, default), or the keyword-search text with tags appended (`processed`), as indexes were built before this option existed
+- `--query-stopwords TERMS` - Comma-separated terms every document is about, such as your product's name. Recorded in the index; search drops them from queries, unless fewer than two content words would remain (so "what is Acme" still searches for Acme)
 - `--split-newlines N` - Split on N or more consecutive newlines, in addition to sentence boundaries
 - `--languages LANGS` - Comma-separated language codes (default: en)
 

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/docs/search_indexing.md

@@ -523,7 +523,9 @@ Common edits during review:
 
 - Add descriptive chunk IDs (replace auto-generated `chunk_0`, `chunk_1`)
 - Add relevant tags for hybrid search boosting
-- Add URLs to source documentation
+- Add URLs to source documentation. In a JSON file that holds several documents, the URL (or a `document` value in the chunk's metadata) also tells search which chunks belong to the same document. Without one, every chunk in the file counts as one document and they push each other down the results. Make the value identify one document: a `source` shared by the whole collection (for example `"source": "docs-site"` on every chunk) makes every file that uses it count as a single document.
+- If every document is about the same product, build with `--query-stopwords "<product name>"`. A name that every document is about carries no information in a query, and only favours the chunks that happen to spell it out. In one product corpus, "<product> call pricing" ranked the phone-number price cards (which mention porting numbers "into <product>") above the per-minute call rates.
+- Set `rank_weight` (between 0 and 1) on secondary chunks, such as commentary next to reference material, so the reference ranks first when both match
 - Add priority levels (`high`, `normal`, `low`)
 - Link related chunks via `related_chunks`
 - Remove low-value chunks (e.g., header-only content)

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/docs/search_overview.md

@@ -327,9 +327,10 @@ The algorithm proceeds in these steps:
 2. Keyword matching, weighted metadata field matching (category, product, tags, source, section, and description each carry a different weight), and filename matching run against that same pool.
 3. Each candidate is scored. If it has a vector similarity score, that score is the base, and every additional agreeing signal adds a small boost, capped at a fixed maximum. Agreement can reorder near-ties; it cannot let a weak vector match outrank a stronger one.
 4. A candidate found only through keyword or metadata matching, with no vector score, is scaled below the pool's strongest vector match. A non-semantic match can never outrank a genuine semantic one.
-5. Near-duplicate chunks are removed, keeping the highest-scoring copy of each.
-6. Results are penalized when several chunks come from the same file, so one document cannot dominate the result list. The second chunk from a file keeps 85% of its computed score, the third 70%, the fourth 50%, and the fifth or later 40%.
-7. The top N results are returned.
+5. A chunk with a `rank_weight` in its metadata has its score multiplied by it. Only values between 0 and 1 count, so a weight can demote a chunk but never promote it. Use it to mark secondary sources, such as commentary indexed next to reference material, so the reference wins when both match.
+6. Near-duplicate chunks are removed, keeping the highest-scoring copy of each.
+7. Results are penalized when several chunks come from the same document, so one document cannot dominate the result list. The second chunk of a document keeps 85% of its computed score, the third 70%, the fourth 50%, and the fifth or later 40%. A chunk's document is its `document` metadata if set, otherwise its `url`, then its `source`, then the file it was read from. Set `document` or `url` on chunks in a JSON file that holds many documents, or the whole file is treated as one.
+8. The top N results are returned.
 
 **Concrete example:**
 
@@ -460,13 +461,14 @@ if __name__ == "__main__":
 
 When a user asks a question, the following occurs:
 
-1. The query is converted to a vector embedding.
-2. The search engine compares the query vector to all chunk vectors in the index.
-3. Chunks are scored by cosine similarity.
-4. Keyword and metadata matches boost scores (hybrid scoring).
-5. Top results (above the distance threshold) are returned.
-6. Results are formatted and provided to the LLM.
-7. The LLM generates an answer grounded in the retrieved content.
+1. If the index was built with `--query-stopwords`, those terms are removed from the query, unless fewer than two content words would remain.
+2. The query is converted to a vector embedding.
+3. The search engine compares the query vector to all chunk vectors in the index.
+4. Chunks are scored by cosine similarity.
+5. Keyword and metadata matches boost scores (hybrid scoring).
+6. Top results (above the distance threshold) are returned.
+7. Results are formatted and provided to the LLM.
+8. The LLM generates an answer grounded in the retrieved content.
 
 ### Testing Your Search
 

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "signalwire-sdk"
-version = "3.7.2.dev132"
+version = "3.7.2.dev134"
 description = "SignalWire SDK for AI voice agents, live call control (RELAY) and REST. Run sw-pydocs for its documentation."
 authors = [
     {name = "SignalWire Team", email = "info@signalwire.com"}

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/__init__.py

@@ -59,7 +59,7 @@ if TYPE_CHECKING:
     from signalwire.utils.schema_utils import SchemaValidationError
     from signalwire.web import WebService
 
-__version__ = "3.7.2.dev132"
+__version__ = "3.7.2.dev134"
 
 # §6.2-python (owner decision: NO packaging split — one package, all deps required;
 # IMPORT-TIME behavior only): the agent/web symbols are lazy-imported via module

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/agent_server.py

@@ -68,7 +68,7 @@ class AgentServer:
         self.app = FastAPI(
             title="SignalWire AI Agents",
             description="Hosted SignalWire AI Agents",
-            version="3.7.2.dev132",
+            version="3.7.2.dev134",
             redirect_slashes=False,
             docs_url=None,
             redoc_url=None,

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/cli/build_search.py

@@ -330,6 +330,17 @@ Documentation: sw-pydocs prints the SDK's installed docs, examples and API.
         ),
     )
 
+    parser.add_argument(
+        "--query-stopwords",
+        default=None,
+        help=(
+            "Comma-separated terms every document is about, such as your "
+            "product's name. Recorded in the index; search drops them from "
+            "queries unless the query would be left with fewer than two "
+            "content words."
+        ),
+    )
+
     parser.add_argument("--verbose", action="store_true", help="Enable verbose output")
 
     parser.add_argument(
@@ -355,6 +366,13 @@ Documentation: sw-pydocs prints the SDK's installed docs, examples and API.
     return parser
 
 
+def _split_terms(value: str | None) -> list[str] | None:
+    """Split a comma-separated --query-stopwords value; None when not given."""
+    if not value:
+        return None
+    return [t.strip() for t in value.split(",") if t.strip()]
+
+
 def main() -> None:
     """Main entry point for the build-search command"""
     args = _build_parser().parse_args()
@@ -531,6 +549,7 @@ def main() -> None:
                 split_newlines=args.split_newlines,
                 index_nlp_backend=args.index_nlp_backend,
                 embed_text=args.embed_text,
+                query_stopwords=_split_terms(args.query_stopwords),
                 verbose=args.verbose,
                 semantic_threshold=args.semantic_threshold,
                 topic_threshold=args.topic_threshold,
@@ -642,6 +661,7 @@ def main() -> None:
             split_newlines=args.split_newlines,
             index_nlp_backend=args.index_nlp_backend,
             embed_text=args.embed_text,
+            query_stopwords=_split_terms(args.query_stopwords),
             verbose=args.verbose,
             semantic_threshold=args.semantic_threshold,
             topic_threshold=args.topic_threshold,

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/search/document_processor.py

@@ -95,6 +95,7 @@ else:
         magic = None
 
 from signalwire.core.logging_config import get_logger
+from signalwire.search.models import document_key
 
 logger = get_logger(__name__)
 
@@ -1796,11 +1797,7 @@ class DocumentProcessor:
                 # the heading is reset each time, and a scan of 16,894 chunks
                 # found no instance -- but that is a property of those inputs,
                 # not of this loop. Reset when the document changes.
-                doc_key = (
-                    json_metadata.get("url")
-                    or json_metadata.get("source")
-                    or json_metadata.get("filename")
-                )
+                doc_key = document_key(json_metadata)
                 if doc_key != current_doc:
                     current_doc = doc_key
                     current_heading = ""

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/search/index_builder.py

@@ -57,6 +57,7 @@ class IndexBuilder:
         backend: str = "sqlite",
         connection_string: str | None = None,
         embed_text: str = "content",
+        query_stopwords: list[str] | None = None,
     ):
         """
         Initialize the index builder
@@ -96,6 +97,10 @@ class IndexBuilder:
                 'content', and 0.562 with content plus tags appended. Your
                 corpus may differ; the option exists so you can measure both.
                 Keyword search uses processed_content either way.
+            query_stopwords: Terms every document in the index is about, such
+                as the product's own name. They are recorded in the index, and
+                search drops them from queries; see
+                query_processor.drop_corpus_terms.
         """
         self.model_name = model_name
         self.chunking_strategy = chunking_strategy
@@ -111,6 +116,9 @@ class IndexBuilder:
         self.backend = backend
         self.connection_string = connection_string
         self.embed_text = embed_text
+        self.query_stopwords = [
+            t.strip() for t in (query_stopwords or []) if t and t.strip()
+        ]
         # SentenceTransformer instance, loaded lazily in _load_model(); the lib
         # ships no stubs, so this is effectively Any once populated.
         self.model: Any = None
@@ -738,6 +746,7 @@ class IndexBuilder:
                 "chunk_overlap": str(self.chunk_overlap),
                 "preprocessing_version": "1.0",
                 "embed_text": self.embed_text,
+                "query_stopwords": json.dumps(self.query_stopwords),
                 "languages": json.dumps(languages),
                 "created_at": datetime.now().isoformat(),
                 "sources": json.dumps(
@@ -962,6 +971,7 @@ class IndexBuilder:
                     "chunk_overlap": self.chunk_overlap,
                     "index_nlp_backend": self.index_nlp_backend,
                     "embed_text": self.embed_text,
+                    "query_stopwords": self.query_stopwords,
                 },
             }
 

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/search/models.py

@@ -8,6 +8,8 @@ See LICENSE file in the project root for full license information.
 """
 
 import warnings
+from collections.abc import Mapping
+from typing import Any
 
 # Embedding model configuration
 MODEL_ALIASES = {
@@ -40,3 +42,27 @@ def resolve_model_alias(model_name: str) -> str:
     if model_name == "large":
         warnings.warn(LARGE_ALIAS_DEPRECATION, DeprecationWarning, stacklevel=2)
     return MODEL_ALIASES.get(model_name, model_name)
+
+
+# Metadata keys naming the document a chunk came from, most specific first.
+# `document` lets an index state it outright. Without it, a page URL or a
+# repo:path source is the next-best identity, and the file the chunk was read
+# from is the last resort -- which is right for one-document-per-file indexes
+# and wrong for JSON collections, where one file holds many documents.
+DOCUMENT_KEYS = ("document", "url", "source", "filename")
+
+
+def document_key(metadata: Mapping[str, Any] | None) -> str | None:
+    """Identify the document a chunk belongs to, or None if nothing says.
+
+    Used wherever "the same document" matters: resetting heading context
+    between documents in a JSON file, and penalising several chunks of one
+    document in a result list. One definition, so the two cannot disagree.
+    """
+    if not metadata:
+        return None
+    for key in DOCUMENT_KEYS:
+        value = metadata.get(key)
+        if value:
+            return str(value)
+    return None

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/search/query_processor.py

@@ -370,6 +370,97 @@ stopwords_language_map = {
     "tr": "turkish",
 }
 
+# Used when NLTK's stopword corpus is unavailable, so the guard in
+# drop_corpus_terms still recognises a question with nothing left to say.
+_FALLBACK_STOPWORDS = frozenset(
+    {
+        "a",
+        "an",
+        "and",
+        "are",
+        "as",
+        "at",
+        "be",
+        "by",
+        "can",
+        "do",
+        "does",
+        "for",
+        "from",
+        "how",
+        "i",
+        "in",
+        "is",
+        "it",
+        "its",
+        "me",
+        "my",
+        "of",
+        "on",
+        "or",
+        "the",
+        "this",
+        "that",
+        "to",
+        "vs",
+        "versus",
+        "what",
+        "when",
+        "where",
+        "which",
+        "who",
+        "why",
+        "with",
+        "you",
+        "your",
+    }
+)
+
+
+def drop_corpus_terms(
+    query: str, terms: list[str] | tuple[str, ...], language: str = "en"
+) -> str:
+    """Remove terms that every document in an index is about from a query.
+
+    In a corpus about one product, the product's name carries no information:
+    everything is about it. In the query it only rewards the few chunks that
+    happen to spell the name out. In SignalWire's own corpus, "SignalWire call
+    pricing" ranked the phone-number rate cards first, because each says
+    numbers "can be ported into SignalWire", over the per-minute rates the
+    question was about. An index declares such terms when it is built
+    (`--query-stopwords`), and search drops them here, before embedding and
+    keyword matching.
+
+    A term is kept when dropping it would leave fewer than two content words.
+    "What is SignalWire" and a bare "SignalWire" are ABOUT the name, and
+    stripping it leaves nothing to search for: measured, a bare "SignalWire"
+    fell out of the top 10 without this guard.
+
+    Matching is case-insensitive, on word boundaries, and takes a possessive
+    with it. Terms may be phrases.
+    """
+    if not isinstance(terms, (list, tuple)):
+        return query
+    clean = [t.strip() for t in terms if isinstance(t, str) and t.strip()]
+    if not query or not clean:
+        return query
+    pattern = re.compile(
+        r"(?i)\b(?:"
+        + "|".join(r"\s+".join(map(re.escape, t.split())) for t in clean)
+        + r")(?:'s)?\b"
+    )
+    if not pattern.search(query):
+        return query
+    stripped = re.sub(r"\s+", " ", pattern.sub(" ", query)).strip()
+    try:
+        stop = set(
+            nltk.corpus.stopwords.words(stopwords_language_map.get(language, "english"))
+        )
+    except Exception:  # corpus not downloaded: fall back to a small fixed list
+        stop = set(_FALLBACK_STOPWORDS)
+    content = [w for w in re.findall(r"[^\W_]+", stripped.lower()) if w not in stop]
+    return stripped if len(content) >= 2 else query
+
 
 # Function to ensure NLTK resources are downloaded
 def ensure_nltk_resources() -> None:

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/search/search_engine.py

@@ -13,6 +13,7 @@ import warnings
 from typing import Any, TYPE_CHECKING
 
 from signalwire.core.logging_config import get_logger
+from signalwire.search.models import document_key
 
 # NDArray is the runtime numpy array type when available, else Any; typed Any
 # because the fallback makes the two branches incompatible to mypy.
@@ -94,6 +95,27 @@ class SearchEngine:
                 f"Invalid backend '{backend}'. Must be 'sqlite' or 'pgvector'"
             )
 
+    @property
+    def query_stopwords(self) -> list[str]:
+        """Terms the index was built to drop from queries; see drop_corpus_terms.
+
+        SQLite indexes keep them in the config table as a JSON list, pgvector
+        collections in collection_config.metadata. A missing or malformed value
+        means none, never an error: older indexes simply have no such key.
+        """
+        config: dict[str, Any] = getattr(self, "config", None) or {}
+        raw: Any = config.get("query_stopwords")
+        if raw is None and isinstance(config.get("metadata"), dict):
+            raw = config["metadata"].get("query_stopwords")
+        if isinstance(raw, str):
+            try:
+                raw = json.loads(raw)
+            except ValueError:
+                return []
+        if not isinstance(raw, list):
+            return []
+        return [t for t in raw if isinstance(t, str) and t.strip()]
+
     def _load_config(self) -> dict[str, str]:
         """Load index configuration"""
         conn = None
@@ -312,8 +334,9 @@ class SearchEngine:
           2. tag filter (optional)
           3. threshold filter on final score (redundant but safe)
           4. exact-match boost (when original_query provided)
+          4b. rank weight — the chunk's own `rank_weight` metadata, if any
           5. content dedup — collapse near-identical chunks, keep highest scorer
-          6. filename diversity — decay 2nd/3rd/4th+ chunks from the same file
+          6. document diversity — decay 2nd/3rd/4th+ chunks of one document
           7. match-type diversity — 40/40/20 hybrid/vector-only/keyword-only
           8. truncate to `count`
         """
@@ -377,6 +400,16 @@ class SearchEngine:
                         r["final_score"] = min(r["final_score"], vector_ceiling * 0.99)
             final_results.sort(key=lambda x: x["final_score"], reverse=True)
 
+        # 4b. rank weight -- see _rank_weight
+        weighted = False
+        for r in final_results:
+            w = self._rank_weight(r.get("metadata"))
+            if w != 1.0:
+                r["final_score"] *= w
+                weighted = True
+        if weighted:
+            final_results.sort(key=lambda x: x["final_score"], reverse=True)
+
         # 5. content dedup (sorted best-first, so the kept copy is the winner)
         final_results = self._dedupe_by_content(final_results)
 
@@ -1550,10 +1583,36 @@ class SearchEngine:
             deduped.append(result)
         return deduped
 
+    @staticmethod
+    def _rank_weight(metadata: dict[str, Any] | None) -> float:
+        """A chunk's declared rank weight: a number in (0, 1], else 1.0.
+
+        Lets an index say some chunks are secondary sources -- commentary next
+        to reference, say -- so that when both match, the reference wins,
+        without the query having to say which kind it wants. It multiplies the
+        final score, so it only ever demotes: nothing can be weighted above
+        what its relevance earned, which keeps the rule that non-semantic
+        signals may reorder results but not choose them. Values outside
+        (0, 1], booleans and non-numbers are ignored rather than trusted.
+        """
+        w = (metadata or {}).get("rank_weight")
+        if isinstance(w, bool) or not isinstance(w, (int, float)):
+            return 1.0
+        return float(w) if 0.0 < w <= 1.0 else 1.0
+
     def _apply_diversity_penalties(
         self, results: list[dict[str, Any]], target_count: int
     ) -> list[dict[str, Any]]:
-        """Apply penalties to prevent single-file dominance while maintaining quality"""
+        """Apply penalties to prevent single-document dominance while maintaining quality.
+
+        "Document" is document_key(): the chunk's `document`, `url` or `source`
+        metadata before its `filename`. Keying on the file alone treated every
+        chunk of a JSON collection as one document -- 50 separate rate cards in
+        one pricing file decayed each other out of the results, and the
+        per-minute card for "outbound call pricing" fell from 9th by relevance
+        to below the top 20. For one-document-per-file indexes the key is the
+        filename, as before.
+        """
         if not results:
             return results
 
@@ -1570,7 +1629,7 @@ class SearchEngine:
         }
 
         for result in results:
-            filename = (result.get("metadata") or {}).get("filename")
+            filename = document_key(result.get("metadata"))
             if filename is None:
                 # No filename signal - skip penalty logic for this result
                 penalized_result = result.copy()
@@ -1605,28 +1664,32 @@ class SearchEngine:
         # Ensure minimum diversity if we have enough results
         if len(penalized_results) > target_count:
             unique_files = len(
-                {r["metadata"]["filename"] for r in penalized_results[:target_count]}
+                {
+                    document_key(r.get("metadata"))
+                    for r in penalized_results[:target_count]
+                }
             )
 
             # If top results are too homogeneous (e.g., all from 1-2 files)
             if unique_files < min(3, target_count):
                 # Try to inject some diversity
                 selected = penalized_results[:target_count]
-                seen_files = {r["metadata"]["filename"] for r in selected}
+                seen_files = {document_key(r.get("metadata")) for r in selected}
 
                 # Look for high-quality results from other files
                 for result in penalized_results[target_count:]:
                     # Only inject a result from a new file that's reasonably good
                     # (within 50% of top score)
                     if (
-                        result["metadata"]["filename"] not in seen_files
+                        document_key(result.get("metadata")) not in seen_files
                         and result["final_score"] > 0.5 * selected[0]["final_score"]
                     ):
                         # Replace the lowest scoring result from an over-represented file
                         for i in range(len(selected) - 1, -1, -1):
-                            if file_counts[selected[i]["metadata"]["filename"]] > 2:
+                            key = document_key(selected[i].get("metadata"))
+                            if key is not None and file_counts.get(key, 0) > 2:
                                 selected[i] = result
-                                seen_files.add(result["metadata"]["filename"])
+                                seen_files.add(document_key(result.get("metadata")))
                                 break
 
                 penalized_results[:target_count] = selected

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/search/search_service.py

@@ -39,7 +39,7 @@ else:
     except ImportError:
         SentenceTransformer = None
 
-from .query_processor import preprocess_query, set_global_model
+from .query_processor import drop_corpus_terms, preprocess_query, set_global_model
 from .search_engine import SearchEngine
 from signalwire.core.security_config import SecurityConfig
 from signalwire.core.config_loader import ConfigLoader
@@ -564,17 +564,24 @@ class SearchService:
         # results for this query until the entry is evicted.
         degraded = False
 
+        # Terms the index declares every document is about (its product name,
+        # usually) carry no information and only reward the chunks that spell
+        # them out. See drop_corpus_terms.
+        query_text = drop_corpus_terms(
+            request.query, getattr(search_engine, "query_stopwords", None) or []
+        )
+
         # Enhance query
         try:
             enhanced = preprocess_query(
-                request.query,
+                query_text,
                 language=request.language or "auto",
                 vector=True,
                 model_name=model_name,  # Pass the correct model!
             )
         except Exception as e:
             logger.error(f"Error preprocessing query: {e}")
-            enhanced = {"enhanced_text": request.query, "vector": [], "language": "en"}
+            enhanced = {"enhanced_text": query_text, "vector": [], "language": "en"}
             degraded = True
 
         # Perform search

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134}/signalwire/signalwire/skills/native_vector_search/skill.py

@@ -691,7 +691,10 @@ class NativeVectorSearchSkill(SkillBase):
                 self.logger.debug("Remote search returned %d results", len(results))
             else:
                 # For local searches, preprocess the query locally
-                from signalwire.search.query_processor import preprocess_query
+                from signalwire.search.query_processor import (
+                    drop_corpus_terms,
+                    preprocess_query,
+                )
 
                 # Guaranteed non-None here: the guard above returns early when
                 # not use_remote and search_engine is falsy.
@@ -709,7 +712,10 @@ class NativeVectorSearchSkill(SkillBase):
                     extra_kwargs["model_name"] = model_for_query  # Use model from index
 
                 enhanced = preprocess_query(
-                    query,
+                    drop_corpus_terms(
+                        query,
+                        getattr(self.search_engine, "query_stopwords", None) or [],
+                    ),
                     language="en",
                     vector=True,
                     query_nlp_backend=self.query_nlp_backend,

{signalwire_sdk-3.7.2.dev132 → signalwire_sdk-3.7.2.dev134/signalwire/signalwire_sdk.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: signalwire-sdk
-Version: 3.7.2.dev132
+Version: 3.7.2.dev134
 Summary: SignalWire SDK for AI voice agents, live call control (RELAY) and REST. Run sw-pydocs for its documentation.
 Author-email: SignalWire Team <info@signalwire.com>
 License: MIT