onyx-database 2.2.0__tar.gz → 2.4.0__tar.gz

{onyx_database-2.2.0 → onyx_database-2.4.0}/CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## Unreleased
+
+## 2.4.0 - 2026-08-29
+
+- Added one high-level `search` API for lexical, semantic, and hybrid retrieval
+  across sync and async table builders and database-wide facades. The new form
+  accepts an options mapping or Python keyword arguments, emits the fail-closed
+  `SEARCH` wire operator, defaults to hybrid/any retrieval, validates scores in
+  `0..1`, and remains composable with structured filters while rejecting
+  mutations.
+- Preserved the exact legacy `MATCHES` wire shape for `search(text)` and
+  `search(text, min_score)` calls.
+- Added typed `SearchOptions` overloads and a PEP 561 marker while retaining the
+  legacy numeric-score overloads on sync and async builders and database-wide
+  facades.
+- Added recursive guards for duplicate or mis-targeted `SEARCH`, mixed
+  `__full_text__` predicates, and unsupported live query streams. Database-wide
+  search no longer inherits a configured default partition.
+
+## 2.3.0 - 2026-08-29
+
+- Added typed, hard-bounded native `CANDIDATES`, `SEARCH_CANDIDATES`, and
+  `HNSW_CANDIDATES` query helpers to sync and async builders.
+- Added lossless semantic-signature and HNSW wire validation, including signed
+  64-bit calibration identifiers, mixed-radix bucket checks, vector/work bounds,
+  and sole-root admission enforcement.
+- Added the missing public `NOT_BETWEEN` operator/helper so the Python SDK now
+  covers every Cloud query operator.
+
 ## 2.2.0 - 2026-08-29
 
 - Added sync and async `fenced_save`, `fenced_delete_where`, and `fenced_update_where` helpers for the

{onyx_database-2.2.0 → onyx_database-2.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: onyx-database
-Version: 2.2.0
+Version: 2.4.0
 Summary: Python client SDK for Onyx Cloud Database (builder-pattern API and schema helpers)
 Project-URL: Homepage, https://onyx.dev
 Project-URL: Repository, https://github.com/OnyxDevTools/onyx-database-python
@@ -199,7 +199,7 @@ db = onyx.init(
 
 #### Default partition + logging
 
-- `partition` sets a default partition for queries, `find_by_id`, and deletes by primary key.
+- `partition` sets a default partition for table-scoped queries, `find_by_id`, and deletes by primary key; database-wide `db.search(...)` omits it.
 - Save operations use the partition field on the entity itself (if present).
 - `request_logging_enabled` logs HTTP requests and JSON bodies.
 - `response_logging_enabled` logs HTTP responses and JSON bodies.
@@ -491,7 +491,7 @@ Importable helpers for conditions and sort:
 from onyx_database import (
     eq, neq, within, not_within,
     in_op, not_in,
-    between,
+    between, not_between,
     gt, gte, lt, lte,
     like, not_like, contains, not_contains,
     starts_with, not_starts_with, matches, not_matches,
@@ -503,6 +503,62 @@ from onyx_database import (
 - Prefer `within` / `not_within` for inclusion checks (supports arrays, comma-separated strings, or inner queries).
 - `in_op` / `not_in` remain available for backward compatibility and are exact aliases.
 
+For application search, choose lexical, semantic, or hybrid mode directly:
+
+```py
+question = "how do i calculate cost per horse"
+
+lexical = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="lexical",
+    match="any",
+).list()
+
+semantic = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="semantic",
+).list()
+
+hybrid = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="hybrid",
+).list()
+```
+
+Native candidate channels remain available as explicit, physically bounded
+low-level APIs:
+
+```py
+from onyx_database import hnsw_search_query
+
+lexical = (
+    db.from_table("ActiveDocumentChunk")
+    .in_partition("revision-7")
+    .approximate_search("customer success", max_candidates=128)
+    .limit(20)
+    .list()
+)
+
+semantic = (
+    db.from_table("ChunkAttentionHash")
+    .in_partition("revision-7")
+    .hnsw_candidates(
+        hnsw_search_query(
+            calibration_id=73,
+            vector=prompt_embedding,
+            max_candidates=256,
+            ef_search=1024,
+        )
+    )
+    .limit(20)
+    .list()
+)
+```
+
+`CANDIDATES`, `SEARCH_CANDIDATES`, and `HNSW_CANDIDATES` must be the sole root
+criterion. Candidate results are approximate; exactly rerank them when the
+full-precision vectors are available.
+
 Aggregate / string helpers for `select()` expressions:
 
 ```py
@@ -554,9 +610,75 @@ roles_missing_permission = (
 )
 ```
 
-### Native bounded full-text search
+### Lexical, semantic, and hybrid search
+
+The high-level API sends natural-language text to the database when you pass an
+options mapping or search-option keywords. `mode` defaults to `"hybrid"`, `match`
+defaults to `"any"`, `min_score` defaults to `None`, and `max_candidates` defaults
+to `1000`.
+
+```py
+question = "how do i calculate cost per horse"
+
+# Match any normalized query term.
+lexical = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="lexical",
+    match="any",
+    min_score=0.4,
+    max_candidates=500,
+).list()
+
+# Let the database embed the query and use semantic retrieval.
+semantic = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="semantic",
+).list()
+
+# Combine lexical and semantic retrieval.
+hybrid = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="hybrid",
+).list()
+
+# A mapping is equivalent; Python snake_case and wire camelCase are accepted.
+same_hybrid = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    {"mode": "hybrid", "match": "any", "max_candidates": 500},
+).list()
+```
 
-Use `.search(...)` on a builder or the `search` predicate helper to add a `MATCHES` condition against the `__full_text__` pseudo-field. `db.search(...)` sets `table = "ALL"` and seeds a query builder with that condition (extras like `partition`, `pageSize`, `nextPage` remain querystring params when provided).
+The same options work through `db.search(...)` for database-wide search and on
+the async client. When supplied, `mode` must be `"lexical"`, `"semantic"`, or
+`"hybrid"`; `match` must be `"all"` or `"any"`; `min_score` must be finite and
+between 0 and 1; and `max_candidates` must be between 1 and 5000. Hybrid mode
+requires at least 2 candidates so both lexical and semantic channels receive a
+candidate budget.
+High-level `SEARCH` queries are read-only but may be combined with structured
+filters using `where`, `and_`, or `or_`. A query may contain only one `SEARCH`,
+it cannot contain another `__full_text__` predicate, and high-level/candidate
+search plans cannot be used for live query streams.
+
+Table-scoped high-level search spans the table's current partitions by default
+under one global candidate budget; use `in_partition(...)` to constrain it.
+The low-level candidate APIs below still require one concrete partition.
+Database-wide `db.search(...)` searches eligible unpartitioned tables only and
+does not inherit the client's configured default partition.
+
+Semantic and hybrid modes require a server-side embedding provider. The provider
+must embed saved searchable text and query text with the same model and vector
+space; marking a field searchable does not select or configure that provider.
+Rows saved before embedding was enabled must be explicitly resaved or backfilled
+before HNSW can retrieve them.
+
+### Legacy vector-managed search
+
+Use `.search(...)` on a builder or the `search` predicate helper to add a `MATCHES` condition against the `__full_text__` pseudo-field. `db.search(...)` sets `table = "ALL"` and seeds a query builder with that condition. Paging remains a query parameter; partition is deliberately omitted from `ALL` searches.
+
+For compatibility, calls with no options and calls whose only second argument is
+a numeric score retain this legacy wire contract. Pass an options mapping or a
+high-level option such as `mode`, `match`, or `max_candidates` to opt into
+`SEARCH`.
 
 ```py
 from onyx_database import onyx, search, eq
@@ -694,6 +816,99 @@ Example request bodies emitted by the SDK:
 }
 ```
 
+For advanced caller-supplied semantic routing, build a validated
+`VectorSearchQuery`. The helper accepts Python snake_case arguments and emits the
+Cloud API's camelCase wire fields:
+
+```py
+from onyx_database import semantic_vector_signature, vector_search_query
+
+signature = semantic_vector_signature(
+    calibration_id=-7909761245221418085,
+    bucket_id=5,
+    cells=[1, 2],
+    cell_counts=[2, 3],
+    fingerprint=["0x0123456789abcdef"],
+    boundary_confidence=0.2,
+)
+
+hybrid = vector_search_query(
+    text="storm warning",
+    semantic=signature,
+    min_score=0.15,
+    nearby_bucket_radius=2,
+    max_candidates=500,
+    require_all_terms=False,
+)
+
+db.from_table(tables.Document).search(hybrid).limit(20).list()
+```
+
+`VectorSearchQuery` requires text and/or a semantic signature. Its defaults are
+`nearbyBucketRadius=1`, `maxCandidates=1000`, and `requireAllTerms=true`;
+`maxCandidates` is bounded to `1..5000`. Semantic helpers validate the mixed-radix
+bucket identifier, compute or verify exactly four fingerprint bands, keep the
+non-zero signed 64-bit calibration identifier as decimal text, and emit fingerprint
+words as fixed-width unsigned hexadecimal strings.
+
+Three explicit candidate APIs provide physically bounded admission for downstream
+reranking:
+
+```py
+from onyx_database import hnsw_search_query
+
+# Bounded lexical admission from a SEARCHABLE table.
+lexical = (
+    db.from_table(tables.Document)
+      .in_partition("corpus-a")
+      .approximate_search(
+          "storm warning",
+          min_score=0.1,
+          max_candidates=250,
+          require_all_terms=False,
+      )
+      .limit(20)
+      .list()
+)
+
+# Native HNSW nearest-neighbor admission.
+hnsw = hnsw_search_query(
+    calibration_id=-7909761245221418085,
+    vector=[0.25, -0.5, 0.75],
+    max_candidates=100,
+    ef_search=400,
+    min_score=0.2,
+)
+neighbors = (
+    db.from_table(tables.Document)
+      .in_partition("corpus-a")
+      .hnsw_candidates(hnsw)
+      .list()
+)
+
+# Bounded EQUAL/IN-style admission from an ordinary secondary index.
+routed = (
+    db.from_table(tables.Document)
+      .in_partition("corpus-a")
+      .approximate_candidates("tenantId", ["tenant-a", "tenant-b"], 200)
+      .list()
+)
+```
+
+`SEARCH_CANDIDATES`, `HNSW_CANDIDATES`, and `CANDIDATES` are positive, read-only,
+sole-root criteria. The builders reject attempts to combine them with `where`,
+`and_`, `or_`, or `search` in either call order. Partitioned tables require one
+concrete partition. HNSW vectors contain `1..16384` finite values with a non-zero
+norm; `maxCandidates` is `1..5000`, `efSearch` is at least `maxCandidates` and at
+most `20000`, and `minScore` is optional in `[-1, 1]`. The same methods and wire
+validation are available on `AsyncQueryBuilder`.
+
+Condition forms are exported as `approximate_search`, `hnsw_candidates`, and
+`approximate_candidates`. Use them only as the sole condition, for example
+`.where(approximate_search("storm", max_candidates=250))`. Database-wide
+`db.search(...)` accepts the same high-level mode, match, score, and candidate
+options as a table builder.
+
 ---
 
 ## Usage examples with `User`, `Role`, `Permission`
@@ -985,7 +1200,7 @@ except (OnyxConfigError, OnyxHTTPError) as err:
 
 A typical release flow for this repository:
 
-1. Update the version in `onyx_database/_version.py` (or use your preferred versioning tool).
+1. Update the project version in `pyproject.toml`.
 2. Build: `python -m build`
 3. Publish: `twine upload dist/*`
 

{onyx_database-2.2.0 → onyx_database-2.4.0}/README.md

@@ -174,7 +174,7 @@ db = onyx.init(
 
 #### Default partition + logging
 
-- `partition` sets a default partition for queries, `find_by_id`, and deletes by primary key.
+- `partition` sets a default partition for table-scoped queries, `find_by_id`, and deletes by primary key; database-wide `db.search(...)` omits it.
 - Save operations use the partition field on the entity itself (if present).
 - `request_logging_enabled` logs HTTP requests and JSON bodies.
 - `response_logging_enabled` logs HTTP responses and JSON bodies.
@@ -466,7 +466,7 @@ Importable helpers for conditions and sort:
 from onyx_database import (
     eq, neq, within, not_within,
     in_op, not_in,
-    between,
+    between, not_between,
     gt, gte, lt, lte,
     like, not_like, contains, not_contains,
     starts_with, not_starts_with, matches, not_matches,
@@ -478,6 +478,62 @@ from onyx_database import (
 - Prefer `within` / `not_within` for inclusion checks (supports arrays, comma-separated strings, or inner queries).
 - `in_op` / `not_in` remain available for backward compatibility and are exact aliases.
 
+For application search, choose lexical, semantic, or hybrid mode directly:
+
+```py
+question = "how do i calculate cost per horse"
+
+lexical = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="lexical",
+    match="any",
+).list()
+
+semantic = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="semantic",
+).list()
+
+hybrid = db.from_table("ActiveDocumentChunk").search(
+    question,
+    mode="hybrid",
+).list()
+```
+
+Native candidate channels remain available as explicit, physically bounded
+low-level APIs:
+
+```py
+from onyx_database import hnsw_search_query
+
+lexical = (
+    db.from_table("ActiveDocumentChunk")
+    .in_partition("revision-7")
+    .approximate_search("customer success", max_candidates=128)
+    .limit(20)
+    .list()
+)
+
+semantic = (
+    db.from_table("ChunkAttentionHash")
+    .in_partition("revision-7")
+    .hnsw_candidates(
+        hnsw_search_query(
+            calibration_id=73,
+            vector=prompt_embedding,
+            max_candidates=256,
+            ef_search=1024,
+        )
+    )
+    .limit(20)
+    .list()
+)
+```
+
+`CANDIDATES`, `SEARCH_CANDIDATES`, and `HNSW_CANDIDATES` must be the sole root
+criterion. Candidate results are approximate; exactly rerank them when the
+full-precision vectors are available.
+
 Aggregate / string helpers for `select()` expressions:
 
 ```py
@@ -529,9 +585,75 @@ roles_missing_permission = (
 )
 ```
 
-### Native bounded full-text search
+### Lexical, semantic, and hybrid search
+
+The high-level API sends natural-language text to the database when you pass an
+options mapping or search-option keywords. `mode` defaults to `"hybrid"`, `match`
+defaults to `"any"`, `min_score` defaults to `None`, and `max_candidates` defaults
+to `1000`.
+
+```py
+question = "how do i calculate cost per horse"
+
+# Match any normalized query term.
+lexical = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="lexical",
+    match="any",
+    min_score=0.4,
+    max_candidates=500,
+).list()
+
+# Let the database embed the query and use semantic retrieval.
+semantic = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="semantic",
+).list()
+
+# Combine lexical and semantic retrieval.
+hybrid = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    mode="hybrid",
+).list()
+
+# A mapping is equivalent; Python snake_case and wire camelCase are accepted.
+same_hybrid = db.from_table(tables.ActiveDocumentChunk).search(
+    question,
+    {"mode": "hybrid", "match": "any", "max_candidates": 500},
+).list()
+```
 
-Use `.search(...)` on a builder or the `search` predicate helper to add a `MATCHES` condition against the `__full_text__` pseudo-field. `db.search(...)` sets `table = "ALL"` and seeds a query builder with that condition (extras like `partition`, `pageSize`, `nextPage` remain querystring params when provided).
+The same options work through `db.search(...)` for database-wide search and on
+the async client. When supplied, `mode` must be `"lexical"`, `"semantic"`, or
+`"hybrid"`; `match` must be `"all"` or `"any"`; `min_score` must be finite and
+between 0 and 1; and `max_candidates` must be between 1 and 5000. Hybrid mode
+requires at least 2 candidates so both lexical and semantic channels receive a
+candidate budget.
+High-level `SEARCH` queries are read-only but may be combined with structured
+filters using `where`, `and_`, or `or_`. A query may contain only one `SEARCH`,
+it cannot contain another `__full_text__` predicate, and high-level/candidate
+search plans cannot be used for live query streams.
+
+Table-scoped high-level search spans the table's current partitions by default
+under one global candidate budget; use `in_partition(...)` to constrain it.
+The low-level candidate APIs below still require one concrete partition.
+Database-wide `db.search(...)` searches eligible unpartitioned tables only and
+does not inherit the client's configured default partition.
+
+Semantic and hybrid modes require a server-side embedding provider. The provider
+must embed saved searchable text and query text with the same model and vector
+space; marking a field searchable does not select or configure that provider.
+Rows saved before embedding was enabled must be explicitly resaved or backfilled
+before HNSW can retrieve them.
+
+### Legacy vector-managed search
+
+Use `.search(...)` on a builder or the `search` predicate helper to add a `MATCHES` condition against the `__full_text__` pseudo-field. `db.search(...)` sets `table = "ALL"` and seeds a query builder with that condition. Paging remains a query parameter; partition is deliberately omitted from `ALL` searches.
+
+For compatibility, calls with no options and calls whose only second argument is
+a numeric score retain this legacy wire contract. Pass an options mapping or a
+high-level option such as `mode`, `match`, or `max_candidates` to opt into
+`SEARCH`.
 
 ```py
 from onyx_database import onyx, search, eq
@@ -669,6 +791,99 @@ Example request bodies emitted by the SDK:
 }
 ```
 
+For advanced caller-supplied semantic routing, build a validated
+`VectorSearchQuery`. The helper accepts Python snake_case arguments and emits the
+Cloud API's camelCase wire fields:
+
+```py
+from onyx_database import semantic_vector_signature, vector_search_query
+
+signature = semantic_vector_signature(
+    calibration_id=-7909761245221418085,
+    bucket_id=5,
+    cells=[1, 2],
+    cell_counts=[2, 3],
+    fingerprint=["0x0123456789abcdef"],
+    boundary_confidence=0.2,
+)
+
+hybrid = vector_search_query(
+    text="storm warning",
+    semantic=signature,
+    min_score=0.15,
+    nearby_bucket_radius=2,
+    max_candidates=500,
+    require_all_terms=False,
+)
+
+db.from_table(tables.Document).search(hybrid).limit(20).list()
+```
+
+`VectorSearchQuery` requires text and/or a semantic signature. Its defaults are
+`nearbyBucketRadius=1`, `maxCandidates=1000`, and `requireAllTerms=true`;
+`maxCandidates` is bounded to `1..5000`. Semantic helpers validate the mixed-radix
+bucket identifier, compute or verify exactly four fingerprint bands, keep the
+non-zero signed 64-bit calibration identifier as decimal text, and emit fingerprint
+words as fixed-width unsigned hexadecimal strings.
+
+Three explicit candidate APIs provide physically bounded admission for downstream
+reranking:
+
+```py
+from onyx_database import hnsw_search_query
+
+# Bounded lexical admission from a SEARCHABLE table.
+lexical = (
+    db.from_table(tables.Document)
+      .in_partition("corpus-a")
+      .approximate_search(
+          "storm warning",
+          min_score=0.1,
+          max_candidates=250,
+          require_all_terms=False,
+      )
+      .limit(20)
+      .list()
+)
+
+# Native HNSW nearest-neighbor admission.
+hnsw = hnsw_search_query(
+    calibration_id=-7909761245221418085,
+    vector=[0.25, -0.5, 0.75],
+    max_candidates=100,
+    ef_search=400,
+    min_score=0.2,
+)
+neighbors = (
+    db.from_table(tables.Document)
+      .in_partition("corpus-a")
+      .hnsw_candidates(hnsw)
+      .list()
+)
+
+# Bounded EQUAL/IN-style admission from an ordinary secondary index.
+routed = (
+    db.from_table(tables.Document)
+      .in_partition("corpus-a")
+      .approximate_candidates("tenantId", ["tenant-a", "tenant-b"], 200)
+      .list()
+)
+```
+
+`SEARCH_CANDIDATES`, `HNSW_CANDIDATES`, and `CANDIDATES` are positive, read-only,
+sole-root criteria. The builders reject attempts to combine them with `where`,
+`and_`, `or_`, or `search` in either call order. Partitioned tables require one
+concrete partition. HNSW vectors contain `1..16384` finite values with a non-zero
+norm; `maxCandidates` is `1..5000`, `efSearch` is at least `maxCandidates` and at
+most `20000`, and `minScore` is optional in `[-1, 1]`. The same methods and wire
+validation are available on `AsyncQueryBuilder`.
+
+Condition forms are exported as `approximate_search`, `hnsw_candidates`, and
+`approximate_candidates`. Use them only as the sole condition, for example
+`.where(approximate_search("storm", max_candidates=250))`. Database-wide
+`db.search(...)` accepts the same high-level mode, match, score, and candidate
+options as a table builder.
+
 ---
 
 ## Usage examples with `User`, `Role`, `Permission`
@@ -960,7 +1175,7 @@ except (OnyxConfigError, OnyxHTTPError) as err:
 
 A typical release flow for this repository:
 
-1. Update the version in `onyx_database/_version.py` (or use your preferred versioning tool).
+1. Update the project version in `pyproject.toml`.
 2. Build: `python -m build`
 3. Publish: `twine upload dist/*`
 

{onyx_database-2.2.0 → onyx_database-2.4.0}/onyx.schema.json

@@ -1,5 +1,5 @@
 {
-  "databaseId": "bbabca0e-82ce-11f0-0000-a2ce78b61b6a",
+  "databaseId": "YOUR_DATABASE_ID",
   "entities": [
     {
       "name": "AuditLog",