crate 0.35.2__tar.gz → 1.0.0.dev0__tar.gz

{crate-0.35.2 → crate-1.0.0.dev0}/DEVELOP.rst

@@ -30,9 +30,7 @@ Run all tests::
 
 Run specific tests::
 
-    ./bin/test -vvvv -t SqlAlchemyCompilerTest
     ./bin/test -vvvv -t test_score
-    ./bin/test -vvvv -t sqlalchemy
 
 Ignore specific test directories::
 

{crate-0.35.2 → crate-1.0.0.dev0}/PKG-INFO

@@ -1,12 +1,12 @@
 Metadata-Version: 2.1
 Name: crate
-Version: 0.35.2
+Version: 1.0.0.dev0
 Summary: CrateDB Python Client
 Home-page: https://github.com/crate/crate-python
 Author: Crate.io
 Author-email: office@crate.io
 License: Apache License 2.0
-Keywords: crate db api sqlalchemy
+Keywords: cratedb db api dbapi database sql http rdbms olap
 Platform: any
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
@@ -26,11 +26,27 @@ Classifier: Programming Language :: Python :: Implementation :: PyPy
 Classifier: Topic :: Database
 Requires-Python: >=3.6
 Description-Content-Type: text/x-rst
-Provides-Extra: sqlalchemy
-Provides-Extra: test
-Provides-Extra: doc
 License-File: LICENSE
 License-File: NOTICE
+Requires-Dist: urllib3<2.3
+Requires-Dist: verlib2==0.2.0
+Provides-Extra: test
+Requires-Dist: tox<5,>=3; extra == "test"
+Requires-Dist: zope.testing<6,>=4; extra == "test"
+Requires-Dist: zope.testrunner<7,>=5; extra == "test"
+Requires-Dist: zc.customdoctests<2,>=1.0.1; extra == "test"
+Requires-Dist: backports.zoneinfo<1; python_version < "3.9" and extra == "test"
+Requires-Dist: certifi; extra == "test"
+Requires-Dist: createcoverage<2,>=1; extra == "test"
+Requires-Dist: dask[dataframe]; extra == "test"
+Requires-Dist: stopit<2,>=1.1.2; extra == "test"
+Requires-Dist: flake8<8,>=4; extra == "test"
+Requires-Dist: pandas<2.3; extra == "test"
+Requires-Dist: pueblo>=0.0.7; extra == "test"
+Requires-Dist: pytz; extra == "test"
+Provides-Extra: doc
+Requires-Dist: sphinx<8,>=3.5; extra == "doc"
+Requires-Dist: crate-docs-theme>=0.26.5; extra == "doc"
 
 =====================
 CrateDB Python Client
@@ -75,12 +91,11 @@ CrateDB Python Client
 
 |
 
-A Python client library for CrateDB_.
-
-This library:
+A Python client library for `CrateDB`_, implementing the Python `DB API 2.0`_
+specification.
 
-- Implements the Python `DB API 2.0`_ specification.
-- Includes support for SQLAlchemy_ in form of an `SQLAlchemy dialect`_.
+The CrateDB dialect for `SQLAlchemy`_ is provided by the `sqlalchemy-cratedb`_
+package.
 
 
 Installation
@@ -88,10 +103,9 @@ Installation
 
 The CrateDB Python client is available as package `crate`_ on `PyPI`_.
 
-To install the most recent driver version, including the SQLAlchemy dialect
-extension, run::
+To install the most recent driver version, run::
 
-    $ pip install "crate[sqlalchemy]" --upgrade
+    $ pip install --upgrade crate
 
 
 Documentation and help
@@ -121,7 +135,7 @@ GitHub`_. We appreciate contributions of any kind.
 .. _Developer documentation: DEVELOP.rst
 .. _managed on GitHub: https://github.com/crate/crate-python
 .. _PyPI: https://pypi.org/
-.. _SQLAlchemy: https://www.sqlalchemy.org
-.. _SQLAlchemy dialect: https://docs.sqlalchemy.org/dialects/
+.. _SQLAlchemy: https://www.sqlalchemy.org/
+.. _sqlalchemy-cratedb: https://github.com/crate-workbench/sqlalchemy-cratedb
 .. _StackOverflow: https://stackoverflow.com/tags/cratedb
 .. _support channels: https://crate.io/support/

{crate-0.35.2 → crate-1.0.0.dev0}/README.rst

@@ -41,12 +41,11 @@ CrateDB Python Client
 
 |
 
-A Python client library for CrateDB_.
+A Python client library for `CrateDB`_, implementing the Python `DB API 2.0`_
+specification.
 
-This library:
-
-- Implements the Python `DB API 2.0`_ specification.
-- Includes support for SQLAlchemy_ in form of an `SQLAlchemy dialect`_.
+The CrateDB dialect for `SQLAlchemy`_ is provided by the `sqlalchemy-cratedb`_
+package.
 
 
 Installation
@@ -54,10 +53,9 @@ Installation
 
 The CrateDB Python client is available as package `crate`_ on `PyPI`_.
 
-To install the most recent driver version, including the SQLAlchemy dialect
-extension, run::
+To install the most recent driver version, run::
 
-    $ pip install "crate[sqlalchemy]" --upgrade
+    $ pip install --upgrade crate
 
 
 Documentation and help
@@ -87,7 +85,7 @@ GitHub`_. We appreciate contributions of any kind.
 .. _Developer documentation: DEVELOP.rst
 .. _managed on GitHub: https://github.com/crate/crate-python
 .. _PyPI: https://pypi.org/
-.. _SQLAlchemy: https://www.sqlalchemy.org
-.. _SQLAlchemy dialect: https://docs.sqlalchemy.org/dialects/
+.. _SQLAlchemy: https://www.sqlalchemy.org/
+.. _sqlalchemy-cratedb: https://github.com/crate-workbench/sqlalchemy-cratedb
 .. _StackOverflow: https://stackoverflow.com/tags/cratedb
 .. _support channels: https://crate.io/support/

crate-1.0.0.dev0/docs/.crate-docs/.build/_sources/appendices/data-types.rst.txt

@@ -0,0 +1,158 @@
+.. _data-types:
+
+==========
+Data types
+==========
+
+The :ref:`Database API client <connect>` and the :ref:`SQLAlchemy dialect
+<using-sqlalchemy>` use different Python data types. Consult the corresponding
+section for further information.
+
+.. rubric:: Table of contents
+
+.. contents::
+   :local:
+
+.. _data-types-db-api:
+
+Database API client
+===================
+
+This section documents data types for the CrateDB :ref:`Database API client
+<connect>`.
+
+.. CAUTION::
+
+    The CrateDB Database API client implementation is incomplete. For the time
+    being, the client uses native Python types.
+
+In general, types are mapped as follows:
+
+============= ===========
+CrateDB       Python
+============= ===========
+`boolean`__   `boolean`__
+`string`__    `str`__
+`int`__       `int`__
+`long`__      `int`__
+`short`__     `int`__
+`double`__    `float`__
+
+`float`__     `float`__
+`byte`__      `int`__
+`geo_point`__ `list`__
+`geo_shape`__ `dict`__
+`object`__    `dict`__
+`array`__     `list`__
+============= ===========
+
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#boolean
+__ https://docs.python.org/3/library/stdtypes.html#boolean-values
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#character-data
+__ https://docs.python.org/3/library/stdtypes.html#str
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://docs.python.org/3/library/functions.html#int
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://docs.python.org/3/library/functions.html#int
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://docs.python.org/3/library/functions.html#int
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://docs.python.org/3/library/functions.html#float
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://docs.python.org/3/library/functions.html#float
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://docs.python.org/3/library/functions.html#int
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#geo-point
+__ https://docs.python.org/3/library/stdtypes.html#list
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#geo-shape
+__ https://docs.python.org/3/library/stdtypes.html#dict
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#object
+__ https://docs.python.org/3/library/stdtypes.html#dict
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#array
+__ https://docs.python.org/3/library/stdtypes.html#list
+
+When writing to CrateDB, the following conversions take place:
+
+============= ====================================
+Python        CrateDB
+============= ====================================
+`Decimal`__   `string`__
+`date`__      `integer`__, `long`__, or `string`__
+`datetime`__  `integer`__, `long`__, or `string`__
+============= ====================================
+
+__ https://docs.python.org/3/library/decimal.html
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#character-data
+__ https://docs.python.org/3/library/datetime.html#date-objects
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#character-data
+__ https://docs.python.org/3/library/datetime.html#datetime-objects
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#character-data
+
+.. NOTE::
+
+    The type that ``date`` and ``datetime`` objects are mapped depends on the
+    CrateDB column type.
+
+.. _data-types-sqlalchemy:
+
+SQLAlchemy
+==========
+
+This section documents data types for the CrateDB :ref:`SQLAlchemy dialect
+<using-sqlalchemy>`.
+
+.. _sqlalchemy-type-map:
+
+Type map
+--------
+
+The CrateDB dialect maps between data types like so:
+
+================= =========================================
+CrateDB           SQLAlchemy
+================= =========================================
+`boolean`__       `Boolean`__
+`byte`__          `SmallInteger`__
+`short`__         `SmallInteger`__
+`integer`__       `Integer`__
+`long`__          `NUMERIC`__
+`float`__         `Float`__
+`double`__        `DECIMAL`__
+`timestamp`__     `TIMESTAMP`__
+`string`__        `String`__
+`array`__         `ARRAY`__
+`object`__        :ref:`object` |nbsp| (extension type)
+`array(object)`__ :ref:`objectarray` |nbsp| (extension type)
+`geo_point`__     :ref:`geopoint` |nbsp| (extension type)
+`geo_shape`__     :ref:`geoshape` |nbsp| (extension type)
+================= =========================================
+
+
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#boolean
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.Boolean
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.SmallInteger
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.SmallInteger
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.Integer
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.NUMERIC
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.Float
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#numeric-data
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.DECIMAL
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#dates-and-times
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.TIMESTAMP
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#character-data
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.String
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#array
+__ http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.ARRAY
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#object
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#array
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#geo-point
+__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#geo-shape

crate-1.0.0.dev0/docs/.crate-docs/.build/_sources/appendices/index.rst.txt

@@ -0,0 +1,14 @@
+.. _appendices:
+
+==========
+Appendices
+==========
+
+Supplementary information for the CrateDB Python client library.
+
+.. rubric:: Table of contents
+
+.. toctree::
+   :maxdepth: 2
+
+   data-types

crate-1.0.0.dev0/docs/.crate-docs/.build/_sources/backlog.rst.txt

@@ -0,0 +1,6 @@
+:orphan:
+
+- Docs: Jupyter Notebook Example?
+- crate+pyodbc?
+- Add PyPy to CI
+- Check IronPython and JPython?

crate-1.0.0.dev0/docs/.crate-docs/.build/_sources/blobs.rst.txt

@@ -0,0 +1,162 @@
+.. _blobs:
+
+=====
+Blobs
+=====
+
+The CrateDB Python client library provides full access to the powerful
+:ref:`blob storage capabilities <crate-reference:blob_support>` of your
+CrateDB cluster.
+
+.. rubric:: Table of contents
+
+.. contents::
+   :local:
+
+Get a blob container
+====================
+
+The first thing you will need to do is connect to CrateDB. Follow the
+instructions in the :ref:`connection document <connect>` for more detailed
+information.
+
+For the sake of this example, we will do the following:
+
+    >>> from crate import client
+    >>> connection = client.connect("http://localhost:4200/")
+
+This is a simple connection that connects to a CrateDB node running on
+the local host with the :ref:`crate-reference:interface-http` listening
+on port 4200 (the default).
+
+To work with blobs in CrateDB, you must specifically create
+:ref:`blob tables <crate-reference:blob_support>`.
+
+The CrateDB Python client allows you to interact with these blob tables via a
+blob container, which you can create like this:
+
+    >>> blob_container = connection.get_blob_container('my_blobs')
+    >>> blob_container
+    <BlobContainer 'my_blobs'>
+
+Here, we have created a ``BlobContainer`` for the ``my_blobs`` table, using
+``connection`` object.
+
+Now we can start working with our blob container.
+
+Working with the blob container
+===============================
+
+Upload blobs
+------------
+
+The blob container can work with files or *file-like objects*, as long as
+produce bytes when read.
+
+What is a file-like object? Well, to put it simply, any object that provides a
+``read()`` method.
+
+The stream objects provided by the Python standard library :mod:`py:io` and
+:mod:`py:tempfile` modules are the most commonly used file-like objects.
+
+The :class:`py:io.StringIO` class is not suitable, as it produces Unicode strings when
+read. But you can easily encode a Unicode string and feed it to a :class:`py:io.BytesIO`
+object.
+
+Here's a trivial example:
+
+    >>> import io
+    >>> bytestream = "An example sentence.".encode("utf8")
+    >>> file = io.BytesIO(bytestream)
+
+This file can then be uploaded to the blob table using the ``put`` method:
+
+    >>> blob_container.put(file)
+    '6f10281ad07d4a35c6ec2f993e6376032b77181d'
+
+Notice that this method computes and returns an `SHA-1 digest`_. This is
+necessary for attempting to save the blob to CrateDB.
+
+If you already have the SHA-1 digest computed, or are able to compute it as part
+of an existing read, this may improve the performance of your application.
+
+If you pass in a SHA-1 digest, it will not be recomputed:
+
+    >>> file.seek(0) # seek to the beginning before attempting to re-upload
+
+    >>> digest = "6f10281ad07d4a35c6ec2f993e6376032b77181d"
+    >>> blob_container.put(file, digest=digest)
+    False
+
+Notice that the method returned ``False`` this time. If you specify a digest,
+the return value of the ``put`` method is a boolean indicating whether the
+object was written or not. In this instance, it was not written, because the
+digest is the same as an existing object.
+
+Let's make a new object:
+
+    >>> bytestream = "Another example sentence.".encode("utf8")
+    >>> digest = hashlib.sha1(bytestream).hexdigest()
+    >>> another_file = io.BytesIO(bytestream)
+
+And upload it:
+
+    >>> blob_container.put(another_file, digest)
+    True
+
+The ``put`` method returns ``True``, indicating that the object has been
+written to the blob container.
+
+Retrieve blobs
+--------------
+
+To retrieve a blob, you need to know its digest.
+
+Let's use the ``digest`` variable we created before to check whether that object
+exists with the ``exists`` method:
+
+    >>> blob_container.exists(digest)
+    True
+
+This method returns a boolean value. And in this instance, ``True`` indicates
+that the blob we're interested in is contained within the blob container.
+
+You can get the blob, with the ``get`` method, like so:
+
+    >>> blob_generator = blob_container.get(digest)
+
+Blobs are read in chunks. The default size of these chunks is 128 kilobytes,
+but this can be changed by supplying the desired chunk size to the ``get``
+method, like so:
+
+    >>> res = blob_container.get(digest, 1024 * 128)
+
+The ``blob`` object is a Python :term:`py:generator`, meaning that you can call
+``next(blob)`` for each new chunk you want to read, until you encounter a
+``StopIteration`` exception.
+
+Instead of calling ``next()`` manually, the idiomatic way to iterate over a
+generator is like so:
+
+    >>> blob_content = b''
+    >>> for chunk in blob_container.get(digest):
+    ...     blob_content += chunk
+
+
+Delete blobs
+------------
+
+You can delete a blob with the ``delete`` method and the blob digest, like so:
+
+    >>> blob_container.delete(digest)
+    True
+
+This method returns a boolean status value. In this instance, ``True``
+indicates that the blob was deleted.
+
+We can verify that, like so:
+
+    >>> blob_container.exists(digest)
+    False
+
+.. _SHA-1 digest: https://en.wikipedia.org/wiki/SHA-1

crate-1.0.0.dev0/docs/.crate-docs/.build/_sources/by-example/blob.rst.txt

@@ -0,0 +1,100 @@
+==================
+Blob container API
+==================
+
+The connection object provides a convenience API for easy access to
+:ref:`blob tables <crate-reference:blob_support>`.
+
+
+Get blob container handle
+=========================
+
+Create a connection:
+
+    >>> from crate.client import connect
+    >>> client = connect([crate_host])
+
+Get a blob container:
+
+    >>> container = client.get_blob_container('myfiles')
+
+
+Store blobs
+===========
+
+The container allows to store a blob without explicitly providing the hash
+for the blob. This feature is possible if the blob is provided as a seekable
+stream like object.
+
+Store a ``StringIO`` stream:
+
+    >>> from io import BytesIO
+    >>> f = BytesIO(b'StringIO data')
+    >>> stringio_bob = container.put(f)
+    >>> stringio_bob
+    '0cd4511d696823779692484029f234471cd21f28'
+
+Store from a file:
+
+    >>> from tempfile import TemporaryFile
+    >>> f = TemporaryFile()
+    >>> _ = f.write(b'File data')
+    >>> _ = f.seek(0)
+    >>> file_blob = container.put(f)
+    >>> file_blob
+    'ea6e03a4a4ee8a2366fe5a88af2bde61797973ea'
+    >>> f.close()
+
+If the blob data is not provided as a seekable stream the hash must be
+provided explicitly:
+
+    >>> import hashlib
+    >>> string_data = b'String data'
+    >>> string_blob = hashlib.sha1(string_data).hexdigest()
+    >>> container.put(string_data, string_blob)
+    True
+
+
+Check for existence
+===================
+
+    >>> container.exists(string_blob)
+    True
+    >>> container.exists('unknown')
+    False
+
+
+Retrieve blobs
+==============
+
+Blobs can be retrieved using its hash:
+
+    >>> blob_stream = container.get(string_blob)
+    >>> blob_stream
+    <generator ...>
+    >>> data = next(blob_stream)
+    >>> data == string_data
+    True
+
+
+Delete blobs
+============
+
+Blobs can be deleted using its hash:
+
+    >>> container.delete(string_blob)
+    True
+    >>> container.exists(string_blob)
+    False
+
+Trying to delete a not existing blob:
+
+    >>> container.delete(string_blob)
+    False
+
+Close connection
+================
+
+Close the connection to clear the connection pool:
+
+    >>> client.close()