vastdb 2.0.2__tar.gz → 2.0.3__tar.gz

{vastdb-2.0.2 → vastdb-2.0.3}/CHANGELOG.md

@@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+## [2.0.3] (2025-11-25)
+[2.0.2]: https://github.com/vast-data/vastdb_sdk/compare/v2.0.2...v2.0.3
+
+### Added
+- Support for vastdb's adbc driver
+  - Session now gets an Optional[AdbcDriver]
+  - AdbcDriver is created out of a local path to a driver `.so` (shared library) or a url
+- Vector Search capability
+  - `Table.vector_search` - Approximate Top-N closest vectors
+
 ## [2.0.2] (2025-10-22)
 [2.0.2]: https://github.com/vast-data/vastdb_sdk/compare/v2.0.1...v2.0.2
 

{vastdb-2.0.2 → vastdb-2.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: vastdb
-Version: 2.0.2
+Version: 2.0.3
 Summary: VAST Data SDK
 Home-page: https://github.com/vast-data/vastdb_sdk
 Author: VAST DATA
@@ -29,6 +29,7 @@ Requires-Dist: numpy
 Requires-Dist: requests
 Requires-Dist: xmltodict
 Requires-Dist: backoff==2.2.1
+Requires-Dist: adbc_driver_manager
 Dynamic: author
 Dynamic: author-email
 Dynamic: classifier

{vastdb-2.0.2 → vastdb-2.0.3}/README.md

@@ -19,16 +19,12 @@ For technical details about VAST Database architecture, see the [whitepaper](htt
 ### Requirements
 
 - Linux client with Python 3.10 - 3.13, and network access to the VAST Cluster
+- VAST Cluster release `5.0.0-sp10` or later
+  - If your VAST Cluster is running an older release, please contact customer.support@vastdata.com.
 - [Virtual IP pool configured with DNS service](https://support.vastdata.com/s/topic/0TOV40000000FThOAM/configuring-network-access-v50)
 - [S3 access & secret keys on the VAST cluster](https://support.vastdata.com/s/article/UUID-4d2e7e23-b2fb-7900-d98f-96c31a499626)
 - [Tabular identity policy with the proper permissions](https://support.vastdata.com/s/article/UUID-14322b60-d6a2-89ac-3df0-3dfbb6974182)
 
-### Required VAST Cluster release
-
-VAST DB Python SDK requires VAST Cluster release `5.0.0-sp10` or later.
-
-If your VAST Cluster is running an older release, please contact customer.support@vastdata.com.
-
 ### Installation
 
 ```bash
@@ -80,61 +76,81 @@ with session.transaction() as tx:
     # the transaction is automatically committed when exiting the context
 ```
 
-For configuration examples, see [here](docs/config.md).
+Note: the transaction must be remain open while the returned [pyarrow.RecordBatchReader](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatchReader.html) generator is being used.
 
 The list of supported data types can be found [here](docs/types.md).
 
-Note: the transaction must be remain open while the returned [pyarrow.RecordBatchReader](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatchReader.html) generator is being used.
+## Features
 
-## Use Cases
+### Select Performance
 
-### Filters and Projections
+The `Table.select()` method accepts a [QueryConfig](vastdb/config.py) object that modifies how the select is fulfilled.
 
-The SDK supports predicate and projection pushdown:
+The most important setting is the `data_endpoints` parameter that, when set, will allow the SDK to parallelize the select across multiple CNodes. Without this, only the CNode specified in the `connect()` will service the select.
 
 ```python
-    from ibis import _
+from vastdb.config import QueryConfig
 
-    # SELECT c1 FROM t WHERE (c2 > 2) AND (c3 IS NULL)
-    table.select(columns=['c1'],
-                 predicate=(_.c2 > 2) & _.c3.isnull())
+# load default configuration values
+cfg = QueryConfig()
 
-    # SELECT c2, c3 FROM t WHERE (c2 BETWEEN 0 AND 1) OR (c2 > 10)
-    table.select(columns=['c2', 'c3'],
-                 predicate=(_.c2.between(0, 1) | (_.c2 > 10))
+# set data_endpoints to CNode VIPs
+cfg.data_endpoints = [
+    "http://172.19.196.1",
+    "http://172.19.196.2",
+    "http://172.19.196.3",
+    "http://172.19.196.4",
+]
 
-    # SELECT * FROM t WHERE c3 LIKE '%substring%'
-    table.select(predicate=_.c3.contains('substring'))
+table.select(columns=['c1'], predicate=(_.c2 > 2), config=cfg)
 ```
 
-See [here for more details](docs/predicate.md).
+If using DNS with either TTL=0 or multi-response per the [Best Practice on Load Balancing CNodes](https://support.vastdata.com/s/document-item?bundleId=z-kb-articles-publications-prod&topicId=6058049537.html&_LANG=enus), passing in the same DNS name equal to the number of VIPs is a decent proxy.
+
+```python
+cfg.data_endpoints = ["http://vip-pool.v123-xy.VastENG.lab"] * 16  # assuming 16 VIPs in the pool
+```
 
-### Import a single Parquet file via S3 protocol
+### Filters and Projections
 
-You can efficiently create tables from Parquet files (without copying them via the client):
+The SDK supports predicate and projection pushdown using Ibis:
 
 ```python
-    with tempfile.NamedTemporaryFile() as f:
-        pa.parquet.write_table(arrow_table, f.name)
-        s3.put_object(Bucket='bucket-name', Key='staging/file.parquet', Body=f)
+from ibis import _
 
-    schema = tx.bucket('bucket-name').schema('schema-name')
-    table = util.create_table_from_files(
-        schema=schema, table_name='imported-table',
-        parquet_files=['/bucket-name/staging/file.parquet'])
+# SELECT c1 FROM t WHERE (c2 > 2) AND (c3 IS NULL)
+table.select(columns=['c1'],
+             predicate=(_.c2 > 2) & _.c3.isnull())
+
+# SELECT c2, c3 FROM t WHERE (c2 BETWEEN 0 AND 1) OR (c2 > 10)
+table.select(columns=['c2', 'c3'],
+             predicate=(_.c2.between(0, 1) | (_.c2 > 10))
+
+# SELECT * FROM t WHERE c3 LIKE '%substring%'
+table.select(predicate=_.c3.contains('substring'))
 ```
 
-### Import multiple Parquet files concurrently via S3 protocol
+See the [Predicate pushdown support document](docs/predicate.md) for more information on constructing predicates using Ibis.
 
-Import multiple files concurrently into a table (by using multiple CNodes' cores):
+### Import Parquet files via S3 protocol
+
+You can efficiently create tables from Parquet files that already exist in an S3 bucket on VAST without copying them via the client. If more than one file is included in `parquet_files` they will be loaded concurrently.
 
 ```python
-    schema = tx.bucket('bucket-name').schema('schema-name')
+with session.transaction() as tx:
+    schema = tx.bucket('database-name').schema('schema-name')
     table = util.create_table_from_files(
-        schema=schema, table_name='large-imported-table',
-        parquet_files=[f'/bucket-name/staging/file{i}.parquet' for i in range(10)])
+        schema=schema, table_name='imported-table',
+        parquet_files=['/bucket-name/staging/file.parquet'])
 ```
 
+If the table already exists, you can use the `table.import_files()` method to add more data to the table from Parquet files that already exist in an S3 bucket on VAST.
+
+```python
+with session.transaction() as tx:
+    table = tx.bucket('database-name').schema('schema-name').table('table-name')
+    table.import_files(["/bucket-name/staging/file2.parquet"])
+```
 
 ### Semi-sorted Projections
 
@@ -158,9 +174,9 @@ batches = snaps[0].schema('schema-name').table('table-name').select()
 
 ## Interactive and Non-Interactive Workflows
 
-Interactive `Table` created via the `Schema` object (`tx.bucket('..').schema('..').table('..')`) loads metadata and stats eagrly allowing for interactive development.
+A `Table` created via the `Schema` object (`tx.bucket('..').schema('..').table('..')`) loads metadata and stats eagerly allowing for interactive development. Each object (bucket, schema, table) requires one or more round-trips to the server and `.table()` will fetch the full table schema.
 
-For more efficient use-cases where the extra round-trips to the server hurt use the Non-Interactive workflow.
+It's generally more efficient to use the `TableMetadata` interface that allows for both lazy loading of the schemas as it's needed, as well as allowing reusing the metadata across transactions.
 
 ```python
 # load the table schema & stats into an object we can use across transactions
@@ -172,7 +188,7 @@ with session.transaction() as tx:
 table_md = TableMetadata(TableRef("bucket-name", "schema-name", "table-name"),
                          arrow_schema=<some-arrow-schema>)
 
-# now we can reuse without the overhead of reloading the schema and stats,
+# now we can reuse it without the overhead of reloading the schema and stats,
 # such as for inserts:
 with session.transaction() as tx:
     table = tx.table_from_metadata(table_md)
@@ -188,6 +204,15 @@ with session.transaction() as tx:
     print(results)
 ```
 
+Some table operations, like `table.import_files()`, does not require the client to know the table schema, and using the `TableMetadata` interface will bypass fetching the schema entirely.
+
+```python
+table_md = TableMetadata(TableRef("bucket-name", "schema-name", "table-name"))
+
+with session.transaction() as tx:
+    table = tx.table_from_metadata(table_md)
+    table.import_files(["/bucket-name/staging/file2.parquet"])
+```
 
 ## Post-processing
 
@@ -248,9 +273,6 @@ with session.transaction() as tx:
     print(distinct_elements)
 ```
 
-
-
-
 ## More Information
 
 See these blog posts for more examples:
@@ -259,5 +281,3 @@ See these blog posts for more examples:
 - https://vastdata.com/blog/the-vast-catalog-in-action-part-2
 
 See also the [full Vast DB Python SDK documentation](https://vastdb-sdk.readthedocs.io/en/latest/)
-
-

{vastdb-2.0.2 → vastdb-2.0.3}/requirements.txt

@@ -9,3 +9,4 @@ numpy
 requests
 xmltodict
 backoff==2.2.1
+adbc_driver_manager

{vastdb-2.0.2 → vastdb-2.0.3}/setup.py

@@ -29,7 +29,7 @@ setup(
     name='vastdb',
     python_requires='>=3.10.0',
     description='VAST Data SDK',
-    version='2.0.2' + suffix,
+    version='2.0.3' + suffix,
     url='https://github.com/vast-data/vastdb_sdk',
     author='VAST DATA',
     author_email='hello@vastdata.com',

vastdb-2.0.3/vastdb/_adbc.py

@@ -0,0 +1,194 @@
+import hashlib
+import logging
+import os
+import urllib.request
+from typing import Optional
+
+import pyarrow as pa
+import sqlglot
+from adbc_driver_manager.dbapi import Connection, Cursor, connect
+from sqlglot import exp
+
+from vastdb._internal import VectorIndex
+from vastdb._table_interface import IbisPredicate
+from vastdb.table_metadata import TableRef
+
+log = logging.getLogger(__name__)
+
+
+TXID_OVERRIDE_PROPERTY: str = "vast.db.external_txid"
+VAST_DIST_ALIAS = "vast_pysdk_vector_dist"
+DEFAULT_ADBC_DRIVER_CACHE_DIR: str = "~/.vast/adbc_drivers_cache"
+DEFAULT_ADBC_DRIVER_CACHE_BY_URL_DIR: str = f"{DEFAULT_ADBC_DRIVER_CACHE_DIR}/by_url"
+
+
+class LocalAdbcDriverNotFound(Exception):
+    """LocalAdbcDriverNotFound."""
+
+    pass
+
+
+class RemoteAdbcDriverDownloadFailed(Exception):
+    """RemoteAdbcDriverDownloadFailed."""
+
+    pass
+
+
+class AdbcDriver:
+    _local_path: str
+
+    def __init__(self, local_path: str):
+        self._local_path = local_path
+
+    @staticmethod
+    def from_local_path(local_path: str) -> "AdbcDriver":
+        """AdbcDriver from a local_path to shared-library."""
+        if not os.path.exists(local_path):
+            raise LocalAdbcDriverNotFound(local_path)
+
+        return AdbcDriver(local_path)
+
+    @staticmethod
+    def from_url(url: str) -> "AdbcDriver":
+        """AdbcDriver to be downloaded by url to shared-library (uses cache if exists)."""
+        expected_local_path = AdbcDriver._url_to_local_path(url)
+
+        if os.path.exists(expected_local_path):
+            return AdbcDriver(expected_local_path)
+
+        AdbcDriver._download_driver(url, expected_local_path)
+        return AdbcDriver(expected_local_path)
+
+    @staticmethod
+    def _url_to_local_path(url: str) -> str:
+        url_hash = hashlib.sha256(url.encode("utf-8")).hexdigest()
+        return os.path.join(DEFAULT_ADBC_DRIVER_CACHE_BY_URL_DIR, url_hash)
+
+    @staticmethod
+    def _download_driver(url: str, target_path: str):
+        os.makedirs(os.path.dirname(target_path), exist_ok=True)
+
+        try:
+            log.info(f"Downloading ADBC driver from {url} to {target_path}...")
+            urllib.request.urlretrieve(url, target_path)
+            log.info(f"Successfully downloaded driver to {target_path}.")
+        except Exception as e:
+            raise RemoteAdbcDriverDownloadFailed(
+                f"Failed to download ADBC driver from {url}: {e}"
+            ) from e
+
+    @property
+    def local_path(self) -> str:
+        return self._local_path
+
+
+def _get_adbc_connection(
+    adbc_driver_path: str, endpoint: str, access_key: str, secret_key: str, txid: int
+) -> Connection:
+    """Get an adbc connection in transaction."""
+    return connect(
+        driver=adbc_driver_path,
+        db_kwargs={
+            "vast.db.endpoint": endpoint,
+            "vast.db.access_key": access_key,
+            "vast.db.secret_key": secret_key,
+        },
+        conn_kwargs={TXID_OVERRIDE_PROPERTY: str(txid)},
+    )
+
+
+def _remove_table_qualification_from_columns(expression: exp.Expression):
+    """Goes over all columns which are fully qualified with "t0" table reference (ibis default table qualification for unbound tables.
+
+    Note: use only if one table is involved - if two tables exist in the expression columns might become ambiguous.
+    """
+    for col in expression.find_all(exp.Column):
+        col.set("table", None)
+    return expression
+
+
+def _ibis_to_qe_predicates(predicate: IbisPredicate) -> str:
+    ibis_sql = predicate.to_sql()
+    parsed = sqlglot.parse_one(ibis_sql)
+
+    # currently there is a single table
+    # removing the
+    without_table_qualification = _remove_table_qualification_from_columns(
+        parsed.expressions[0].this
+    )
+
+    return without_table_qualification.sql()
+
+
+def _vector_search_sql(
+    query_vector: list[float],
+    vector_index: VectorIndex,
+    table_ref: TableRef,
+    columns: list[str],
+    limit: int,
+    predicate: Optional[IbisPredicate] = None,
+) -> str:
+    query_vector_dim = len(query_vector)
+
+    query_vector_literal = f"{query_vector}::FLOAT[{query_vector_dim}]"
+    dist_func = f"{vector_index.sql_distance_function}({vector_index.column}::FLOAT[{query_vector_dim}], {query_vector_literal})"
+    dist_alias = f"{dist_func} as {VAST_DIST_ALIAS}"
+
+    projection_str = ",".join(columns + [dist_alias])
+
+    if predicate is not None:
+        where = f"WHERE {_ibis_to_qe_predicates(predicate)}"
+    else:
+        where = ""
+
+    return f"""
+            SELECT {projection_str}
+            FROM {table_ref.query_engine_full_path}
+            {where}
+            ORDER BY {VAST_DIST_ALIAS}
+            LIMIT {limit}"""
+
+
+class AdbcConnection:
+    def __init__(
+        self,
+        adbc_driver: AdbcDriver,
+        endpoint: str,
+        access_key: str,
+        secret_key: str,
+        txid: int,
+    ):
+        self._adbc_conn = _get_adbc_connection(
+            adbc_driver.local_path, endpoint, access_key, secret_key, txid
+        )
+
+        self._cursor = self._adbc_conn.cursor()
+
+    @property
+    def cursor(self) -> Cursor:
+        return self._cursor
+
+    def close(self):
+        self._cursor.close()
+
+    def vector_search(
+        self,
+        query_vector: list[float],
+        vector_index: VectorIndex,
+        table_ref: TableRef,
+        columns: list[str],
+        limit: int,
+        predicate: Optional[IbisPredicate] = None,
+    ) -> pa.RecordBatchReader:
+        """Top-n on vector-column."""
+        sql = _vector_search_sql(
+            query_vector=query_vector,
+            vector_index=vector_index,
+            table_ref=table_ref,
+            columns=columns,
+            limit=limit,
+            predicate=predicate,
+        )
+
+        self._cursor.execute(sql)
+        return self._cursor.fetch_record_batch()

{vastdb-2.0.2 → vastdb-2.0.3}/vastdb/_internal.py

@@ -6,6 +6,7 @@ import struct
 import time
 import urllib.parse
 from collections import defaultdict, namedtuple
+from dataclasses import dataclass
 from enum import Enum
 from typing import Any, Dict, Iterator, List, Optional, Tuple, Union, cast
 
@@ -116,6 +117,7 @@ from vastdb.vast_flatbuf.tabular.ListSchemasResponse import (
 from vastdb.vast_flatbuf.tabular.ListTablesResponse import (
     ListTablesResponse as list_tables,
 )
+from vastdb.vast_flatbuf.tabular.VectorIndexMetadata import VectorIndexMetadata
 
 from . import errors, util
 from .config import BackoffConfig
@@ -803,10 +805,40 @@ def _parse_table_info(obj, parse_properties):
                      sorting_score, write_amplification, acummulative_row_insertion_count, sorting_done)
 
 
-# Results that returns from tablestats
+@dataclass
+class VectorIndexSpec:
+    """
+    Vector Index Specification when creating a table.
+    """
+    column: str
+    distance_metric: str
+
+
+@dataclass
+class VectorIndex:
+    column: str
+    distance_metric: str
+    sql_distance_function: str
 
+    def to_vector_index_spec(self) -> VectorIndexSpec:
+        return VectorIndexSpec(self.column,
+                               self.distance_metric)
 
-TableStatsResult = namedtuple("TableStatsResult", 'num_rows size_in_bytes is_external_rowid_alloc sorting_key_enabled sorting_score write_amplification acummulative_row_inserition_count sorting_done endpoints')
+
+@dataclass
+class TableStats:
+    """Table-related information."""
+
+    num_rows: int
+    size_in_bytes: int
+    sorting_score: int
+    write_amplification: int
+    acummulative_row_inserition_count: int
+    is_external_rowid_alloc: bool = False
+    sorting_key_enabled: bool = False
+    sorting_done: bool = False
+    endpoints: Tuple[str, ...] = ()
+    vector_index: Optional[VectorIndex] = None
 
 
 _RETRIABLE_EXCEPTIONS = (
@@ -1128,11 +1160,31 @@ class VastdbApi:
     def create_table(self, bucket, schema, name, arrow_schema=None,
                      txid=0, client_tags=[], expected_retvals=[],
                      create_imports_table=False, use_external_row_ids_allocation=False, table_props=None,
-                     sorting_key=[]):
+                     sorting_key=[], vector_index: Optional[VectorIndexSpec] = None):
+        """
+        Create a table in the specified bucket and schema.
+
+        Args:
+            bucket: Name of the bucket
+            schema: Name of the schema
+            name: Name of the table
+            arrow_schema: PyArrow schema defining the table columns
+            txid: Transaction ID
+            client_tags: Client tags for the request
+            expected_retvals: Expected return values
+            create_imports_table: Whether this is an imports table
+            use_external_row_ids_allocation: Whether to use external row ID allocation
+            table_props: Table properties
+            sorting_key: List of column indices to sort by (for Elysium tables)
+            vector_index: Optional vector index
+        """
         self._create_table_internal(bucket=bucket, schema=schema, name=name, arrow_schema=arrow_schema,
-                                    txid=txid, client_tags=client_tags, expected_retvals=expected_retvals,
-                                    create_imports_table=create_imports_table, use_external_row_ids_allocation=use_external_row_ids_allocation,
-                                    table_props=table_props, sorting_key=sorting_key)
+                                    txid=txid, client_tags=client_tags,
+                                    expected_retvals=expected_retvals,
+                                    create_imports_table=create_imports_table,
+                                    use_external_row_ids_allocation=use_external_row_ids_allocation,
+                                    table_props=table_props, sorting_key=sorting_key,
+                                    vector_index=vector_index)
 
     def create_topic(self, bucket, name, topic_partitions, expected_retvals=[],
                      message_timestamp_type=None, retention_ms=None, message_timestamp_after_max_ms=None,
@@ -1149,8 +1201,9 @@ class VastdbApi:
 
     def _create_table_internal(self, bucket, schema, name, arrow_schema=None,
                                txid=0, client_tags=[], expected_retvals=[], topic_partitions=0,
-                               create_imports_table=False, use_external_row_ids_allocation=False, table_props=None,
-                               sorting_key=[]):
+                               create_imports_table=False, use_external_row_ids_allocation=False,
+                               table_props=None, sorting_key=[],
+                               vector_index: Optional[VectorIndexSpec] = None):
         """
         Create a table, use the following request
         POST /bucket/schema/table?table HTTP/1.1
@@ -1176,6 +1229,10 @@ class VastdbApi:
         if use_external_row_ids_allocation:
             headers['use-external-row-ids-alloc'] = str(use_external_row_ids_allocation)
 
+        if vector_index is not None:
+            headers['tabular-vector-index-column'] = vector_index.column
+            headers['tabular-vector-index-distance-metric'] = vector_index.distance_metric
+
         url_params = {'topic_partitions': str(topic_partitions)} if topic_partitions else {}
         if create_imports_table:
             url_params['sub-table'] = IMPORTED_OBJECTS_TABLE_NAME
@@ -1188,10 +1245,10 @@ class VastdbApi:
             url=self._url(bucket=bucket, schema=schema, table=name, command="table", url_params=url_params),
             data=serialized_schema, headers=headers)
 
-    def get_topic_stats(self, bucket, name, expected_retvals=[]):
+    def get_topic_stats(self, bucket, name, expected_retvals=[]) -> TableStats:
         return self.get_table_stats(bucket=bucket, schema=KAFKA_TOPICS_SCHEMA_NAME, name=name, expected_retvals=expected_retvals)
 
-    def get_table_stats(self, bucket, schema, name, txid=0, client_tags=[], expected_retvals=[], imports_table_stats=False):
+    def get_table_stats(self, bucket, schema, name, txid=0, client_tags=[], expected_retvals=[], imports_table_stats=False) -> TableStats:
         """
         GET /mybucket/myschema/mytable?stats HTTP/1.1
         tabular-txid: TransactionId
@@ -1218,8 +1275,40 @@ class VastdbApi:
         sorting_score = sorting_score_raw & ((1 << 63) - 1)
         sorting_done = bool(sorting_score_raw >> 63)
 
+        vector_index_metadata: Optional[VectorIndexMetadata] = stats.VectorIndexMetadata()
+
+        if vector_index_metadata is not None:
+            column_name = vector_index_metadata.ColumnName()
+            distance_metric = vector_index_metadata.DistanceMetric()
+            sql_distance_function = vector_index_metadata.SqlFunctionName()
+
+            if (column_name is None or
+                distance_metric is None or
+                sql_distance_function is None):
+                raise errors.ApiResponseError(
+                    "VectorIndexMetadata properties (column_name, distance_metric, sql_function_name) must all be set."
+                )
+
+            vector_index = VectorIndex(
+                column=column_name.decode('utf-8'),
+                distance_metric=distance_metric.decode('utf-8'),
+                sql_distance_function=sql_distance_function.decode('utf-8'))
+        else:
+            vector_index = None
+
         endpoints = [self.url]  # we cannot replace the host by a VIP address in HTTPS-based URLs
-        return TableStatsResult(num_rows, size_in_bytes, is_external_rowid_alloc, sorting_key_enabled, sorting_score, write_amplification, acummulative_row_inserition_count, sorting_done, tuple(endpoints))
+
+        return TableStats(
+          num_rows=num_rows,
+          size_in_bytes=size_in_bytes,
+          sorting_score=sorting_score,
+          write_amplification=write_amplification,
+          acummulative_row_inserition_count=acummulative_row_inserition_count,
+          is_external_rowid_alloc=is_external_rowid_alloc,
+          sorting_key_enabled=sorting_key_enabled,
+          sorting_done=sorting_done,
+          endpoints=tuple(endpoints),
+          vector_index=vector_index)
 
     def alter_topic(self, bucket, name,
                     new_name="", expected_retvals=[],
@@ -2339,7 +2428,7 @@ def build_field(builder: flatbuffers.Builder, f: pa.Field, include_name=True):
 class QueryDataRequest:
     def __init__(self, serialized, response_schema, response_parser):
         self.serialized = serialized
-        self.response_schema = response_schema
+        self.response_schema: pa.Schema = response_schema
         self.response_parser = response_parser
 
 

{vastdb-2.0.2 → vastdb-2.0.3}/vastdb/_table_interface.py

@@ -1,15 +1,18 @@
 from abc import ABC, abstractmethod
-from typing import TYPE_CHECKING, Iterable, Optional, Union
+from typing import TYPE_CHECKING, Iterable, Optional, TypeAlias, Union
 
 import ibis
 import pyarrow as pa
 
+from ._internal import VectorIndex
 from .config import ImportConfig, QueryConfig
 from .table_metadata import TableRef
 
 if TYPE_CHECKING:
     from .table import Projection
 
+IbisPredicate: TypeAlias = Union[ibis.expr.types.BooleanColumn, ibis.common.deferred.Deferred]
+
 
 class ITable(ABC):
     """Interface for VAST Table operations."""
@@ -71,8 +74,7 @@ class ITable(ABC):
     @abstractmethod
     def select(self,
                columns: Optional[list[str]] = None,
-               predicate: Union[ibis.expr.types.BooleanColumn,
-                                ibis.common.deferred.Deferred] = None,
+               predicate: Optional[IbisPredicate] = None,
                config: Optional[QueryConfig] = None,
                *,
                internal_row_id: bool = False,
@@ -134,3 +136,18 @@ class ITable(ABC):
         It is useful for constructing expressions for predicate pushdown in `ITable.select()` method.
         """
         pass
+
+    @property
+    @abstractmethod
+    def vector_index(self) -> Optional[VectorIndex]:
+        """Table's Vector Index if exists."""
+        pass
+
+    @abstractmethod
+    def vector_search(self,
+                      vec: list[float],
+                      columns: list[str],
+                      limit: int,
+                      predicate: Optional[IbisPredicate] = None) -> pa.RecordBatchReader:
+        """Top-n on vector-column."""
+        pass