onyx-database 2.3.0__tar.gz → 2.4.0__tar.gz

{onyx_database-2.3.0 → onyx_database-2.4.0}/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## Unreleased
+
+## 2.4.0 - 2026-08-29
+
+- Added one high-level `search` API for lexical, semantic, and hybrid retrieval
+  across sync and async table builders and database-wide facades. The new form
+  accepts an options mapping or Python keyword arguments, emits the fail-closed
+  `SEARCH` wire operator, defaults to hybrid/any retrieval, validates scores in
+  `0..1`, and remains composable with structured filters while rejecting
+  mutations.
+- Preserved the exact legacy `MATCHES` wire shape for `search(text)` and
+  `search(text, min_score)` calls.
+- Added typed `SearchOptions` overloads and a PEP 561 marker while retaining the
+  legacy numeric-score overloads on sync and async builders and database-wide
+  facades.
+- Added recursive guards for duplicate or mis-targeted `SEARCH`, mixed
+  `__full_text__` predicates, and unsupported live query streams. Database-wide
+  search no longer inherits a configured default partition.
+
 ## 2.3.0 - 2026-08-29
 
 - Added typed, hard-bounded native `CANDIDATES`, `SEARCH_CANDIDATES`, and

{onyx_database-2.3.0 → onyx_database-2.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: onyx-database
-Version: 2.3.0
+Version: 2.4.0
 Summary: Python client SDK for Onyx Cloud Database (builder-pattern API and schema helpers)
 Project-URL: Homepage, https://onyx.dev
 Project-URL: Repository, https://github.com/OnyxDevTools/onyx-database-python
@@ -199,7 +199,7 @@ db = onyx.init(
 
 #### Default partition + logging
 
-- `partition` sets a default partition for queries, `find_by_id`, and deletes by primary key.
+- `partition` sets a default partition for table-scoped queries, `find_by_id`, and deletes by primary key; database-wide `db.search(...)` omits it.
 - Save operations use the partition field on the entity itself (if present).
 - `request_logging_enabled` logs HTTP requests and JSON bodies.
 - `response_logging_enabled` logs HTTP responses and JSON bodies.
@@ -503,7 +503,30 @@ from onyx_database import (
 - Prefer `within` / `not_within` for inclusion checks (supports arrays, comma-separated strings, or inner queries).
 - `in_op` / `not_in` remain available for backward compatibility and are exact aliases.
 
-Native candidate channels are explicit and physically bounded:
+For application search, choose lexical, semantic, or hybrid mode directly:
+
+```py
+question = "how do i calculate cost per horse"
+
+lexical = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="lexical",
+    match="any",
+).list()
+
+semantic = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="semantic",
+).list()
+
+hybrid = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="hybrid",
+).list()
+```
+
+Native candidate channels remain available as explicit, physically bounded
+low-level APIs:
 
 ```py
 from onyx_database import hnsw_search_query
@@ -587,9 +610,75 @@ roles_missing_permission = (
 )
 ```
 
-### Native vector-managed search
+### Lexical, semantic, and hybrid search
+
+The high-level API sends natural-language text to the database when you pass an
+options mapping or search-option keywords. `mode` defaults to `"hybrid"`, `match`
+defaults to `"any"`, `min_score` defaults to `None`, and `max_candidates` defaults
+to `1000`.
+
+```py
+question = "how do i calculate cost per horse"
+
+# Match any normalized query term.
+lexical = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="lexical",
+    match="any",
+    min_score=0.4,
+    max_candidates=500,
+).list()
+
+# Let the database embed the query and use semantic retrieval.
+semantic = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="semantic",
+).list()
+
+# Combine lexical and semantic retrieval.
+hybrid = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="hybrid",
+).list()
+
+# A mapping is equivalent; Python snake_case and wire camelCase are accepted.
+same_hybrid = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    {"mode": "hybrid", "match": "any", "max_candidates": 500},
+).list()
+```
 
-Use `.search(...)` on a builder or the `search` predicate helper to add a `MATCHES` condition against the `__full_text__` pseudo-field. `db.search(...)` sets `table = "ALL"` and seeds a query builder with that condition (extras like `partition`, `pageSize`, `nextPage` remain querystring params when provided).
+The same options work through `db.search(...)` for database-wide search and on
+the async client. When supplied, `mode` must be `"lexical"`, `"semantic"`, or
+`"hybrid"`; `match` must be `"all"` or `"any"`; `min_score` must be finite and
+between 0 and 1; and `max_candidates` must be between 1 and 5000. Hybrid mode
+requires at least 2 candidates so both lexical and semantic channels receive a
+candidate budget.
+High-level `SEARCH` queries are read-only but may be combined with structured
+filters using `where`, `and_`, or `or_`. A query may contain only one `SEARCH`,
+it cannot contain another `__full_text__` predicate, and high-level/candidate
+search plans cannot be used for live query streams.
+
+Table-scoped high-level search spans the table's current partitions by default
+under one global candidate budget; use `in_partition(...)` to constrain it.
+The low-level candidate APIs below still require one concrete partition.
+Database-wide `db.search(...)` searches eligible unpartitioned tables only and
+does not inherit the client's configured default partition.
+
+Semantic and hybrid modes require a server-side embedding provider. The provider
+must embed saved searchable text and query text with the same model and vector
+space; marking a field searchable does not select or configure that provider.
+Rows saved before embedding was enabled must be explicitly resaved or backfilled
+before HNSW can retrieve them.
+
+### Legacy vector-managed search
+
+Use `.search(...)` on a builder or the `search` predicate helper to add a `MATCHES` condition against the `__full_text__` pseudo-field. `db.search(...)` sets `table = "ALL"` and seeds a query builder with that condition. Paging remains a query parameter; partition is deliberately omitted from `ALL` searches.
+
+For compatibility, calls with no options and calls whose only second argument is
+a numeric score retain this legacy wire contract. Pass an options mapping or a
+high-level option such as `mode`, `match`, or `max_candidates` to opt into
+`SEARCH`.
 
 ```py
 from onyx_database import onyx, search, eq
@@ -727,7 +816,7 @@ Example request bodies emitted by the SDK:
 }
 ```
 
-For table-specific lexical, semantic, or hybrid search, build a validated
+For advanced caller-supplied semantic routing, build a validated
 `VectorSearchQuery`. The helper accepts Python snake_case arguments and emits the
 Cloud API's camelCase wire fields:
 
@@ -817,8 +906,8 @@ validation are available on `AsyncQueryBuilder`.
 Condition forms are exported as `approximate_search`, `hnsw_candidates`, and
 `approximate_candidates`. Use them only as the sole condition, for example
 `.where(approximate_search("storm", max_candidates=250))`. Database-wide
-`db.search(...)` remains the text-only API because the `ALL` route performs its own
-bounded lexical fan-out.
+`db.search(...)` accepts the same high-level mode, match, score, and candidate
+options as a table builder.
 
 ---
 
@@ -1111,7 +1200,7 @@ except (OnyxConfigError, OnyxHTTPError) as err:
 
 A typical release flow for this repository:
 
-1. Update the version in `onyx_database/_version.py` (or use your preferred versioning tool).
+1. Update the project version in `pyproject.toml`.
 2. Build: `python -m build`
 3. Publish: `twine upload dist/*`
 

{onyx_database-2.3.0 → onyx_database-2.4.0}/README.md

@@ -174,7 +174,7 @@ db = onyx.init(
 
 #### Default partition + logging
 
-- `partition` sets a default partition for queries, `find_by_id`, and deletes by primary key.
+- `partition` sets a default partition for table-scoped queries, `find_by_id`, and deletes by primary key; database-wide `db.search(...)` omits it.
 - Save operations use the partition field on the entity itself (if present).
 - `request_logging_enabled` logs HTTP requests and JSON bodies.
 - `response_logging_enabled` logs HTTP responses and JSON bodies.
@@ -478,7 +478,30 @@ from onyx_database import (
 - Prefer `within` / `not_within` for inclusion checks (supports arrays, comma-separated strings, or inner queries).
 - `in_op` / `not_in` remain available for backward compatibility and are exact aliases.
 
-Native candidate channels are explicit and physically bounded:
+For application search, choose lexical, semantic, or hybrid mode directly:
+
+```py
+question = "how do i calculate cost per horse"
+
+lexical = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="lexical",
+    match="any",
+).list()
+
+semantic = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="semantic",
+).list()
+
+hybrid = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="hybrid",
+).list()
+```
+
+Native candidate channels remain available as explicit, physically bounded
+low-level APIs:
 
 ```py
 from onyx_database import hnsw_search_query
@@ -562,9 +585,75 @@ roles_missing_permission = (
 )
 ```
 
-### Native vector-managed search
+### Lexical, semantic, and hybrid search
+
+The high-level API sends natural-language text to the database when you pass an
+options mapping or search-option keywords. `mode` defaults to `"hybrid"`, `match`
+defaults to `"any"`, `min_score` defaults to `None`, and `max_candidates` defaults
+to `1000`.
+
+```py
+question = "how do i calculate cost per horse"
+
+# Match any normalized query term.
+lexical = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="lexical",
+    match="any",
+    min_score=0.4,
+    max_candidates=500,
+).list()
+
+# Let the database embed the query and use semantic retrieval.
+semantic = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="semantic",
+).list()
+
+# Combine lexical and semantic retrieval.
+hybrid = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="hybrid",
+).list()
+
+# A mapping is equivalent; Python snake_case and wire camelCase are accepted.
+same_hybrid = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    {"mode": "hybrid", "match": "any", "max_candidates": 500},
+).list()
+```
 
-Use `.search(...)` on a builder or the `search` predicate helper to add a `MATCHES` condition against the `__full_text__` pseudo-field. `db.search(...)` sets `table = "ALL"` and seeds a query builder with that condition (extras like `partition`, `pageSize`, `nextPage` remain querystring params when provided).
+The same options work through `db.search(...)` for database-wide search and on
+the async client. When supplied, `mode` must be `"lexical"`, `"semantic"`, or
+`"hybrid"`; `match` must be `"all"` or `"any"`; `min_score` must be finite and
+between 0 and 1; and `max_candidates` must be between 1 and 5000. Hybrid mode
+requires at least 2 candidates so both lexical and semantic channels receive a
+candidate budget.
+High-level `SEARCH` queries are read-only but may be combined with structured
+filters using `where`, `and_`, or `or_`. A query may contain only one `SEARCH`,
+it cannot contain another `__full_text__` predicate, and high-level/candidate
+search plans cannot be used for live query streams.
+
+Table-scoped high-level search spans the table's current partitions by default
+under one global candidate budget; use `in_partition(...)` to constrain it.
+The low-level candidate APIs below still require one concrete partition.
+Database-wide `db.search(...)` searches eligible unpartitioned tables only and
+does not inherit the client's configured default partition.
+
+Semantic and hybrid modes require a server-side embedding provider. The provider
+must embed saved searchable text and query text with the same model and vector
+space; marking a field searchable does not select or configure that provider.
+Rows saved before embedding was enabled must be explicitly resaved or backfilled
+before HNSW can retrieve them.
+
+### Legacy vector-managed search
+
+Use `.search(...)` on a builder or the `search` predicate helper to add a `MATCHES` condition against the `__full_text__` pseudo-field. `db.search(...)` sets `table = "ALL"` and seeds a query builder with that condition. Paging remains a query parameter; partition is deliberately omitted from `ALL` searches.
+
+For compatibility, calls with no options and calls whose only second argument is
+a numeric score retain this legacy wire contract. Pass an options mapping or a
+high-level option such as `mode`, `match`, or `max_candidates` to opt into
+`SEARCH`.
 
 ```py
 from onyx_database import onyx, search, eq
@@ -702,7 +791,7 @@ Example request bodies emitted by the SDK:
 }
 ```
 
-For table-specific lexical, semantic, or hybrid search, build a validated
+For advanced caller-supplied semantic routing, build a validated
 `VectorSearchQuery`. The helper accepts Python snake_case arguments and emits the
 Cloud API's camelCase wire fields:
 
@@ -792,8 +881,8 @@ validation are available on `AsyncQueryBuilder`.
 Condition forms are exported as `approximate_search`, `hnsw_candidates`, and
 `approximate_candidates`. Use them only as the sole condition, for example
 `.where(approximate_search("storm", max_candidates=250))`. Database-wide
-`db.search(...)` remains the text-only API because the `ALL` route performs its own
-bounded lexical fan-out.
+`db.search(...)` accepts the same high-level mode, match, score, and candidate
+options as a table builder.
 
 ---
 
@@ -1086,7 +1175,7 @@ except (OnyxConfigError, OnyxHTTPError) as err:
 
 A typical release flow for this repository:
 
-1. Update the version in `onyx_database/_version.py` (or use your preferred versioning tool).
+1. Update the project version in `pyproject.toml`.
 2. Build: `python -m build`
 3. Publish: `twine upload dist/*`
 

{onyx_database-2.3.0 → onyx_database-2.4.0}/onyx_database/__init__.py

@@ -66,6 +66,9 @@ from .types import (
     HnswSearchQueryInput,
     Int64WireInput,
     QueryCriteriaOperator,
+    SearchMatch,
+    SearchMode,
+    SearchOptions,
     SemanticVectorSignature,
     SemanticVectorSignatureInput,
     VectorSearchQuery,
@@ -152,6 +155,9 @@ __all__ = [
     "HnswSearchQueryInput",
     "Int64WireInput",
     "QueryCriteriaOperator",
+    "SearchMatch",
+    "SearchMode",
+    "SearchOptions",
     "SemanticVectorSignature",
     "SemanticVectorSignatureInput",
     "VectorSearchQuery",

{onyx_database-2.3.0 → onyx_database-2.4.0}/onyx_database/helpers/candidate_search.py

@@ -13,6 +13,8 @@ from ..types import (
     ApproximateIndexCandidateQuery,
     HnswSearchQuery,
     Int64WireInput,
+    SearchMatch,
+    SearchMode,
     SemanticVectorSignature,
     VectorSearchQuery,
 )
@@ -37,6 +39,16 @@ _SEMANTIC_BAND_COUNT = 4
 _MISSING = object()
 _DECIMAL_RE = re.compile(r"^-?\d+$")
 _HEX_RE = re.compile(r"^[0-9a-fA-F]+$")
+_SEARCH_MODES = {"lexical", "semantic", "hybrid"}
+_SEARCH_MATCH_POLICIES = {"all", "any"}
+_SEARCH_OPTION_FIELDS = {
+    "mode",
+    "match",
+    "minScore",
+    "min_score",
+    "maxCandidates",
+    "max_candidates",
+}
 
 
 def _mapping(input_value: Mapping[str, Any] | None, field: str) -> Mapping[str, Any]:
@@ -62,6 +74,105 @@ def _value(
     return default
 
 
+def _aliased_search_option(
+    values: Mapping[str, Any],
+    wire_name: str,
+    snake_name: str,
+    *,
+    default: Any = _MISSING,
+) -> Any:
+    present = [name for name in (wire_name, snake_name) if name in values]
+    if len(present) > 1:
+        raise TypeError(f"provide only one of {wire_name} and {snake_name}")
+    if present:
+        return values[present[0]]
+    if default is _MISSING:
+        raise TypeError(f"{snake_name} is required")
+    return default
+
+
+def high_level_search_query(
+    text: Any,
+    options: Mapping[str, Any] | None = None,
+    *,
+    mode: Any = _MISSING,
+    match: Any = _MISSING,
+    min_score: Any = _MISSING,
+    max_candidates: Any = _MISSING,
+) -> dict[str, Any]:
+    """Validate and canonicalize a text-first lexical, semantic, or hybrid query."""
+
+    if not isinstance(text, str):
+        raise TypeError("search text must be a string")
+    if not text.strip():
+        raise ValueError("search text must not be blank")
+
+    has_options_mapping = options is not None
+    values = _mapping(options, "SearchOptions")
+    unknown = sorted(set(values) - _SEARCH_OPTION_FIELDS)
+    if unknown:
+        raise TypeError(f"unsupported search option: {unknown[0]}")
+
+    keyword_values = (mode, match, min_score, max_candidates)
+    if has_options_mapping and any(
+        value is not _MISSING for value in keyword_values
+    ):
+        raise TypeError("do not combine a search options mapping with search keywords")
+
+    if has_options_mapping:
+        mode = _value(values, "mode", default="hybrid")
+        if mode is None:
+            raise TypeError("search mode must be a string")
+        match = _value(values, "match", default="any")
+        min_score = _aliased_search_option(
+            values, "minScore", "min_score", default=None
+        )
+        max_candidates = _aliased_search_option(
+            values, "maxCandidates", "max_candidates", default=1_000
+        )
+    else:
+        if mode is _MISSING or mode is None:
+            mode = "hybrid"
+        if match is _MISSING:
+            match = "any"
+        if min_score is _MISSING:
+            min_score = None
+        if max_candidates is _MISSING or max_candidates is None:
+            max_candidates = 1_000
+
+    if mode is _MISSING or mode is None:
+        mode = "hybrid"
+    if not isinstance(mode, str):
+        raise TypeError("search mode must be a string")
+    if mode not in _SEARCH_MODES:
+        raise ValueError("search mode must be lexical, semantic, or hybrid")
+    if not isinstance(match, str):
+        raise TypeError("search match must be a string")
+    if match not in _SEARCH_MATCH_POLICIES:
+        raise ValueError("search match must be all or any")
+
+    canonical_min_score = None
+    if min_score is not None:
+        canonical_min_score, _ = _finite_float32(min_score, "minScore")
+        if canonical_min_score < 0 or canonical_min_score > 1:
+            raise ValueError("minScore must be between 0 and 1")
+    candidate_count = _require_integer(max_candidates, "maxCandidates")
+    if candidate_count < 1 or candidate_count > MAX_VECTOR_SEARCH_CANDIDATES:
+        raise ValueError(
+            f"maxCandidates must be between 1 and {MAX_VECTOR_SEARCH_CANDIDATES}"
+        )
+    if mode == "hybrid" and candidate_count < 2:
+        raise ValueError("maxCandidates must be at least 2 for hybrid search")
+
+    return {
+        "text": text,
+        "mode": cast(SearchMode, mode),
+        "match": cast(SearchMatch, match),
+        "minScore": canonical_min_score,
+        "maxCandidates": candidate_count,
+    }
+
+
 def _require_integer(value: Any, field: str) -> int:
     if isinstance(value, bool) or not isinstance(value, int):
         raise TypeError(f"{field} must be an integer")

{onyx_database-2.3.0 → onyx_database-2.4.0}/onyx_database/helpers/conditions.py

@@ -5,10 +5,11 @@ from __future__ import annotations
 from collections.abc import Mapping
 from typing import Any, Optional, Sequence, Union
 
-from ..types import Condition, QueryBuilderLike
+from ..types import Condition, QueryBuilderLike, SearchMatch, SearchMode, SearchOptions
 from .candidate_search import (
     DEFAULT_APPROXIMATE_INDEX_CANDIDATES,
     approximate_index_candidate_query,
+    high_level_search_query,
     hnsw_search_query,
     vector_search_query,
 )
@@ -123,19 +124,70 @@ def not_contains_ignore_case(field: str, value: Any) -> Condition:
 
 def search(
     query_text_or_search: Any,
-    min_score: Optional[float] = None,
+    min_score: Optional[float] | SearchOptions = None,
     *,
+    mode: Optional[SearchMode] = None,
+    match: Optional[SearchMatch] = None,
     semantic: Any = None,
     nearby_bucket_radius: Optional[int] = None,
     max_candidates: Optional[int] = None,
     require_all_terms: Optional[bool] = None,
 ) -> Condition:
-    """Create an exact native full-text, semantic, or hybrid search condition.
+    """Create a legacy predicate or high-level natural-language search condition.
 
-    Strings retain the established ``queryText`` wire shape. A mapping is
-    validated and emitted as a canonical ``VectorSearchQuery``.
+    A plain string, optionally followed only by a numeric score, retains the
+    established ``queryText`` wire shape. Passing a ``SearchOptions`` mapping as
+    the second argument, or passing a high-level option keyword, emits the
+    canonical ``SEARCH`` contract. First-argument mappings remain the advanced
+    ``VectorSearchQuery`` path.
     """
 
+    options_mapping = min_score if isinstance(min_score, Mapping) else None
+    legacy_vector_options = (
+        semantic is not None
+        or nearby_bucket_radius is not None
+        or require_all_terms is not None
+    )
+    high_level_keywords = (
+        mode is not None
+        or match is not None
+        or (max_candidates is not None and not legacy_vector_options)
+    )
+    if options_mapping is not None or high_level_keywords:
+        if not isinstance(query_text_or_search, str):
+            raise TypeError("the high-level search API requires text")
+        if options_mapping is not None:
+            if (
+                high_level_keywords
+                or semantic is not None
+                or nearby_bucket_radius is not None
+                or max_candidates is not None
+                or require_all_terms is not None
+            ):
+                raise TypeError(
+                    "do not combine a search options mapping with search keywords"
+                )
+            search_value = high_level_search_query(
+                query_text_or_search, options_mapping
+            )
+        else:
+            if (
+                semantic is not None
+                or nearby_bucket_radius is not None
+                or require_all_terms is not None
+            ):
+                raise TypeError(
+                    "mode/match cannot be combined with semantic routing options"
+                )
+            search_value = high_level_search_query(
+                query_text_or_search,
+                mode="hybrid" if mode is None else mode,
+                match="any" if match is None else match,
+                min_score=min_score,
+                max_candidates=1_000 if max_candidates is None else max_candidates,
+            )
+        return _condition("__full_text__", "SEARCH", search_value)
+
     has_vector_options = (
         semantic is not None
         or nearby_bucket_radius is not None