vastdb 1.4.0__tar.gz → 2.0.0__tar.gz

{vastdb-1.4.0 → vastdb-2.0.0}/CHANGELOG.md

@@ -4,6 +4,21 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+## [2.0.0] (2025-09-17)
+
+### BREAKING CHANGE
+- Removed properties `Table.bucket` and `Table.schema`
+- Removed properties `Projection.bucket` and `Projection.schema`
+- Removed official support for Python 3.9
+- Added hard requirement for pyarrow~=18.0
+
+### Added
+- `ITable` - partial VAST tables interface (no DDL)
+- `TableMetadata` - table metadata allowing for reuse / or refraining from loading bucket/schema/table metadata on every transaction
+- `Transaction.table_from_metadata` allows creating `ITable` from `TableMetadata` inside a transaction
+  - This table is only for DQL and DML (query and manipulation)
+- Official support for Python 3.13
+
 ## [1.4.0] (2025-07-28)
 [1.4.0]: https://github.com/vast-data/vastdb_sdk/compare/v1.3.11...v1.4.0
 

{vastdb-1.4.0 → vastdb-2.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: vastdb
-Version: 1.4.0
+Version: 2.0.0
 Summary: VAST Data SDK
 Home-page: https://github.com/vast-data/vastdb_sdk
 Author: VAST DATA
@@ -9,22 +9,37 @@ License: Copyright (C) VAST Data Ltd.
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: License :: OSI Approved :: Apache Software License
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Database
 Classifier: Topic :: Database :: Front-Ends
-Requires-Python: >=3.9.0
+Requires-Python: >=3.10.0
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: aws-requests-auth
+Requires-Dist: ibis-framework~=10.1
+Requires-Dist: pyarrow~=18.0
+Requires-Dist: pyarrow-hotfix==0.7
 Requires-Dist: flatbuffers
-Requires-Dist: ibis-framework==9.0.0
-Requires-Dist: pyarrow
+Requires-Dist: packaging
+Requires-Dist: pandas
+Requires-Dist: numpy
 Requires-Dist: requests
 Requires-Dist: xmltodict
 Requires-Dist: backoff==2.2.1
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
 
 
 `vastdb` is a Python-based SDK designed for interacting

{vastdb-1.4.0 → vastdb-2.0.0}/README.md

@@ -18,7 +18,7 @@ For technical details about VAST Database architecture, see the [whitepaper](htt
 
 ### Requirements
 
-- Linux client with Python 3.9 - 3.12, and network access to the VAST Cluster
+- Linux client with Python 3.10 - 3.13, and network access to the VAST Cluster
 - [Virtual IP pool configured with DNS service](https://support.vastdata.com/s/topic/0TOV40000000FThOAM/configuring-network-access-v50)
 - [S3 access & secret keys on the VAST cluster](https://support.vastdata.com/s/article/UUID-4d2e7e23-b2fb-7900-d98f-96c31a499626)
 - [Tabular identity policy with the proper permissions](https://support.vastdata.com/s/article/UUID-14322b60-d6a2-89ac-3df0-3dfbb6974182)
@@ -156,6 +156,39 @@ snaps = bucket.list_snapshots()
 batches = snaps[0].schema('schema-name').table('table-name').select()
 ```
 
+## Interactive and Non-Interactive Workflows
+
+Interactive `Table` created via the `Schema` object (`tx.bucket('..').schema('..').table('..')`) loads metadata and stats eagrly allowing for interactive development.
+
+For more efficient use-cases where the extra round-trips to the server hurt use the Non-Interactive workflow.
+
+```python
+# load the table schema & stats into an object we can use across transactions
+table_md = TableMetadata(TableRef("bucket-name", "schema-name", "table-name"))
+with session.transaction() as tx:
+    table_md.load(tx)
+
+# you may init the TableMetadata with the schema in advanced (to save that round-trip)
+table_md = TableMetadata(TableRef("bucket-name", "schema-name", "table-name"),
+                         arrow_schema=<some-arrow-schema>)
+
+# now we can reuse without the overhead of reloading the schema and stats,
+# such as for inserts:
+with session.transaction() as tx:
+    table = tx.table_from_metadata(table_md)
+    rows = [{"id": 1, "name": "row1"}]
+    py_records = pa.RecordBatch.from_pylist(rows, schema=table_md.arrow_schema)
+    table.insert(py_records)
+
+# and selects:
+with session.transaction() as tx:
+    table = tx.table_from_metadata(table_md)
+    reader = table.select()
+    results = reader.read_all()
+    print(results)
+```
+
+
 ## Post-processing
 
 ### Export
@@ -214,6 +247,10 @@ with session.transaction() as tx:
     print("Distinct element types on the system:")
     print(distinct_elements)
 ```
+
+
+
+
 ## More Information
 
 See these blog posts for more examples:
@@ -222,3 +259,5 @@ See these blog posts for more examples:
 - https://vastdata.com/blog/the-vast-catalog-in-action-part-2
 
 See also the [full Vast DB Python SDK documentation](https://vastdb-sdk.readthedocs.io/en/latest/)
+
+

vastdb-2.0.0/requirements.txt

@@ -0,0 +1,11 @@
+aws-requests-auth
+ibis-framework~=10.1
+pyarrow~=18.0
+pyarrow-hotfix==0.7
+flatbuffers
+packaging
+pandas
+numpy
+requests
+xmltodict
+backoff==2.2.1

{vastdb-1.4.0 → vastdb-2.0.0}/setup.py

@@ -27,9 +27,9 @@ if os.environ.get('VASTDB_APPEND_VERSION_SUFFIX'):
 
 setup(
     name='vastdb',
-    python_requires='>=3.9.0',
+    python_requires='>=3.10.0',
     description='VAST Data SDK',
-    version='1.4.0' + suffix,
+    version='2.0.0' + suffix,
     url='https://github.com/vast-data/vastdb_sdk',
     author='VAST DATA',
     author_email='hello@vastdata.com',
@@ -42,10 +42,10 @@ setup(
         'Development Status :: 5 - Production/Stable',
         'License :: OSI Approved :: Apache Software License',
         'Programming Language :: Python :: 3',
-        'Programming Language :: Python :: 3.9',
         'Programming Language :: Python :: 3.10',
         'Programming Language :: Python :: 3.11',
         'Programming Language :: Python :: 3.12',
+        'Programming Language :: Python :: 3.13',
         'Topic :: Database',
         'Topic :: Database :: Front-Ends',
     ],

vastdb-2.0.0/vastdb/_ibis_support.py

@@ -0,0 +1,28 @@
+import ibis
+import pyarrow as pa
+from ibis.expr.types.structs import IbisError
+
+from vastdb import errors
+
+
+def validate_ibis_support_schema(arrow_schema: pa.Schema):
+    """Validate that the provided Arrow schema is compatible with Ibis.
+
+    Raises NotSupportedSchema if the schema contains unsupported fields.
+    """
+    unsupported_fields = []
+    first_exception = None
+    for f in arrow_schema:
+        try:
+            ibis.Schema.from_pyarrow(pa.schema([f]))
+        except (IbisError, ValueError, KeyError) as e:
+            if first_exception is None:
+                first_exception = e
+            unsupported_fields.append(f)
+
+    if unsupported_fields:
+        raise errors.NotSupportedSchema(
+            message=f"Ibis does not support the schema {unsupported_fields=}",
+            schema=arrow_schema,
+            cause=first_exception
+        )