questdb 2.0.0__tar.gz → 2.0.2__tar.gz

{questdb-2.0.0 → questdb-2.0.2}/CHANGELOG.rst

@@ -1,7 +1,48 @@
+.. _changelog:
 
 Changelog
 =========
 
+2.0.2 (2024-04-11)
+------------------
+
+Patch release with performance bug fix. No breaking changes.
+
+Bug fixes
+~~~~~~~~~
+* Fixed the defaulting logic for ``auto_flush_rows`` parameter for HTTPS.
+  It is now correctly set to 75000 rows by default. The old incorrect default
+  of 600 rows was causing the sender to flush too often, impacting performance.
+  Note that TCP, TCPS and HTTP were not affected.
+
+Features
+~~~~~~~~
+* The sender now exposes the ``auto_flush`` settings as read-only properties.
+  You can inspect the values in use with ``.auto_flush``, ``.auto_flush_rows``,
+  ``.auto_flush_interval`` and ``.auto_flush_bytes``.
+
+2.0.1 (2024-04-03)
+------------------
+
+Patch release with bug fixes, no API changes and some documentation tweaks.
+
+Bug fixes
+~~~~~~~~~
+* Fixed a bug where an internal "last flushed" timestamp used
+  by ``auto_flush_interval`` wasn't updated correctly causing the auto-flush
+  logic to trigger after each row.
+
+* Removed two unnecessary debugging ``print()`` statements that were
+  accidentally left in the code in ``Sender.from_conf()`` and
+  ``Sender.from_env()``.
+
+Documentation
+~~~~~~~~~~~~~
+* Introduced the ability to optionally install ``pandas`` and ``pyarrow`` via
+  ``python3 -m pip install -U questdb[dataframe]`` and updated the documentation
+  to reflect this.
+
+
 2.0.0 (2024-03-19)
 ------------------
 
@@ -158,7 +199,7 @@ The following example shows how to migrate to the new API.
         'token_y=token_y=Dt5tbS1dEDMSYfym3fgMv0B99szno-dFc1rYF9t0aac;' +
         'auto_flush_rows=off;' +
         'auto_flush_interval=off;' +
-        'auto_flush_bytes=64512')
+        'auto_flush_bytes=64512;')
     with Sender.from_conf(conf) as sender:
         sender.row(
             'test_table',

{questdb-2.0.0 → questdb-2.0.2}/DEV_NOTES.rst

@@ -125,41 +125,10 @@ The `-e` flag links to the current directory rather than copying.
     $ python3  sdist
 
 
-Uploading to PyPI
+Cutting a release
 -----------------
 
-Update the change log file that describes the latest changes in aggregate.
-
-.. code-block:: bash
-
-    vim CHANGELOG.rst
-
-
-Create a tag, update versions etc using ``bump2version``.
-
-For more command line options, see: https://pypi.org/project/bump2version/
-
-Note that here ``patch`` would bump from (for example) ``0.1.0`` to
-``0.1.1``. ``minor`` would bump from ``0.1.0`` to ``0.2.0`` and ``major``
-would bump from ``0.1.0`` to ``1.0.0``.
-
-.. code-block:: bash
-
-    bump2version --config-file .bumpversion.cfg patch
-
-
-Now make sure you push the tag or it will not show up in GitHub:
-
-.. code-block:: bash
-
-    git push --tags
-
-
-Once you've triple-checked everything is in ``dist/``, you can upload to PyPI.
-
-.. code-block:: bash
-
-    python3 -m twine upload dist/*
+This is a semi-automated process, covered in ``RELEASING.rst``.
 
 
 Debugging with on Linux

{questdb-2.0.0 → questdb-2.0.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: questdb
-Version: 2.0.0
+Version: 2.0.2
 Summary: QuestDB client library for Python
 Author-email: Adam Cimarosti <adam@questdb.io>
 License: Apache License 2.0
@@ -31,6 +31,10 @@ Requires-Dist: twine; extra == "publish"
 Requires-Dist: wheel; extra == "publish"
 Provides-Extra: ci
 Requires-Dist: cibuildwheel; extra == "ci"
+Provides-Extra: dataframe
+Requires-Dist: pandas; extra == "dataframe"
+Requires-Dist: pyarrow; extra == "dataframe"
+Requires-Dist: numpy; extra == "dataframe"
 
 =================================
 QuestDB Client Library for Python
@@ -52,28 +56,15 @@ and full-connection encryption with
 Quickstart
 ==========
 
-The latest version of the library is 2.0.0.
+The latest version of the library is 2.0.2 (`changelog <https://py-questdb-client.readthedocs.io/en/latest/changelog.html>`_).
 
 ::
 
-    python3 -m pip install -U questdb
+    python3 -m pip install -U questdb[dataframe]
 
 Please start by `setting up QuestDB <https://questdb.io/docs/quick-start/>`_ . Once set up, you can use this library to insert data.
 
-.. code-block:: python
-
-    from questdb.ingress import Sender, TimestampNanos
-
-    conf = f'http::addr=localhost:9000;'
-    with Sender.from_conf(conf) as sender:
-        sender.row(
-            'sensors',
-            symbols={'id': 'toronto1'},
-            columns={'temperature': 20.0, 'humidity': 0.5},
-            at=TimestampNanos.now())
-        sender.flush()
-
-You can also send Pandas dataframes:
+The most common way to insert data is from a Pandas dataframe.
 
 .. code-block:: python
 
@@ -90,8 +81,25 @@ You can also send Pandas dataframes:
     with Sender.from_conf(conf) as sender:
         sender.dataframe(df, table_name='sensors', at='timestamp')
 
+You can also send individual rows. This only requires a more minimal installation::
+
+    python3 -m pip install -U questdb
+
+.. code-block:: python
+
+    from questdb.ingress import Sender, TimestampNanos
+
+    conf = f'http::addr=localhost:9000;'
+    with Sender.from_conf(conf) as sender:
+        sender.row(
+            'sensors',
+            symbols={'id': 'toronto1'},
+            columns={'temperature': 20.0, 'humidity': 0.5},
+            at=TimestampNanos.now())
+        sender.flush()
+
 
-To connect via TCP, set the
+To connect via the `older TCP protocol <https://py-questdb-client.readthedocs.io/en/latest/sender.html#ilp-tcp-or-ilp-http>`_, set the
 `configuration string <https://py-questdb-client.readthedocs.io/en/latest/conf.html>`_ to:
 
 .. code-block:: python

{questdb-2.0.0 → questdb-2.0.2}/README.rst

@@ -18,28 +18,15 @@ and full-connection encryption with
 Quickstart
 ==========
 
-The latest version of the library is 2.0.0.
+The latest version of the library is 2.0.2 (`changelog <https://py-questdb-client.readthedocs.io/en/latest/changelog.html>`_).
 
 ::
 
-    python3 -m pip install -U questdb
+    python3 -m pip install -U questdb[dataframe]
 
 Please start by `setting up QuestDB <https://questdb.io/docs/quick-start/>`_ . Once set up, you can use this library to insert data.
 
-.. code-block:: python
-
-    from questdb.ingress import Sender, TimestampNanos
-
-    conf = f'http::addr=localhost:9000;'
-    with Sender.from_conf(conf) as sender:
-        sender.row(
-            'sensors',
-            symbols={'id': 'toronto1'},
-            columns={'temperature': 20.0, 'humidity': 0.5},
-            at=TimestampNanos.now())
-        sender.flush()
-
-You can also send Pandas dataframes:
+The most common way to insert data is from a Pandas dataframe.
 
 .. code-block:: python
 
@@ -56,8 +43,25 @@ You can also send Pandas dataframes:
     with Sender.from_conf(conf) as sender:
         sender.dataframe(df, table_name='sensors', at='timestamp')
 
+You can also send individual rows. This only requires a more minimal installation::
+
+    python3 -m pip install -U questdb
+
+.. code-block:: python
+
+    from questdb.ingress import Sender, TimestampNanos
+
+    conf = f'http::addr=localhost:9000;'
+    with Sender.from_conf(conf) as sender:
+        sender.row(
+            'sensors',
+            symbols={'id': 'toronto1'},
+            columns={'temperature': 20.0, 'humidity': 0.5},
+            at=TimestampNanos.now())
+        sender.flush()
+
 
-To connect via TCP, set the
+To connect via the `older TCP protocol <https://py-questdb-client.readthedocs.io/en/latest/sender.html#ilp-tcp-or-ilp-http>`_, set the
 `configuration string <https://py-questdb-client.readthedocs.io/en/latest/conf.html>`_ to:
 
 .. code-block:: python

questdb-2.0.2/RELEASING.rst

@@ -0,0 +1,209 @@
+Cutting a new release
+=====================
+
+Overview
+--------
+
+We will collate the generated binaries from CI, together with a final
+binary we need to build manually from for MacOS ARM within a MacOS VM.
+
+We cut from a VM to use as old of a MacOS version as possible to ensure
+backwards compatibility with older MacOS versions.
+
+From the VM we will also bundle a source distribution and perform the final
+upload to PyPI.
+
+Bumping Version and updating Changelog
+--------------------------------------
+
+Create a new PR with the new changes in ``CHANGELOG.rst``.
+
+Make a commit and push the changes to a new branch.
+
+You also want to bump the version.
+
+This process is automated by the following command:
+
+.. code-block:: bash
+
+    bump2version --config-file .bumpversion.cfg --no-tag patch
+
+Here use:
+
+* ``patch`` to bump the version to the next patch version, e.g. 1.0.0 -> 1.0.1
+
+* ``minor`` to bump the version to the next minor version, e.g. 1.0.0 -> 1.1.0
+
+* ``major`` to bump the version to the next major version, e.g. 1.0.0 -> 2.0.0
+
+Now merge the PR with the title "Bump version: V.V.V → W.W.W".
+
+Note that CI will run all the ``cibuildwheel`` jobs which will in turn 
+generate the binaries for all the platforms, except for MacOS ARM.
+
+Double-check the date in the CHANGELOG
+--------------------------------------
+
+Open ``CHANGELOG.rst`` and ensure that the date next to the version you are releasing matches today's date. 
+If the CHANGELOG was created earlier, it might have an older date. If so, update it.
+
+Preparing the MacOS VM
+----------------------
+
+Skip if you already have the MacOS VM set up in UTM.
+
+.. warn::
+    
+    Releasing from an up to date MacOS install will not work as the binaries
+    may be incompatible with older MacOS versions.
+
+From a MacOS ARM computer install UTM.
+
+* Download from https://mac.getutm.app/
+* Install MacOS X 12.4 (Monterey). See https://docs.getutm.app/guest-support/macos/
+* Install Xcode from the App Store
+* Install Rust from https://rustup.rs/
+* Install Firefox
+* Install *all* OFFICIAL Python Releases from Python 3.8 onwards. Use the latest patch version for each minor release. 
+    * https://www.python.org/downloads/macos/
+    * Do NOT use Homebrew to install Python.
+    * Python 3.8.10 requires Rosetta, install it when prompted to do so. 
+
+* Optionally install VS Code
+
+Now clone the repository. The rest of the steps will assume this is done as so::
+
+    cd ~
+    mkdir -p questdb
+    cd questdb
+    git clone https://github.com/questdb/py-questdb-client.git
+    cd py-questdb-client
+    git submodule update --init --recursive
+
+Updating the MacOS VM
+---------------------
+
+Do this before every release.
+
+Inside the VM, open a terminal (or use the terminal Window in VSCode) and run the following commands::
+
+    cd ~/questdb/py-questdb-client
+    git checkout main
+    git pull
+    git submodule update --init --recursive
+
+    rustup update stable
+
+    /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 \
+        -m pip install -U pip
+    /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 \
+        -m pip install -U setuptools wheel twine Cython cibuildwheel pandas numpy pyarrow
+
+Smoke-testing the build
+-----------------------
+
+From ``~/questdb/py-questdb-client`` run the following commands::
+
+    ./proj clean
+    ./proj build
+    ./proj test
+
+
+Building the MacOS ARM binaries
+-------------------------------
+
+Clean and build the final binaries for each Python version::
+
+    ./proj clean
+    ./proj cibuildwheel
+
+This should have created new binaries in the ``dist/`` directory.
+
+Prepare the source distribution
+-------------------------------
+
+The source code distribution is for any other platforms that we don't have
+binaries for. I don't think it's _actually_ used by anyone, but it might get
+used by IDEs.
+
+.. code-block:: bash
+
+    python3 setup.py sdist
+
+Download the other binaries from CI
+-----------------------------------
+
+From the MacOS VM, From a terminal, run::
+
+    cd ~/Downloads
+    rm drop.zip
+    rm -rf drop
+
+Launch Firefox and log into GitHub and open the last (closed and merged) PR.
+
+Click on the "Checks" tab and open up the last "questdb.py-questdb-client (1)"
+check. There will be a link to the Azure DevOps page.
+
+The following link might also work: https://dev.azure.com/questdb/questdb/_build?definitionId=21&_a=summary
+
+If you open up the last run, you'll find a link called "1 published".
+This will redirect you to the "Publushed artifacts" page.
+
+There will be a "drop" directory.
+* Don't open it.
+* Instead use click on the three vertical dots on the right-hand
+side and select download artifacts.
+
+This will download a file called "drop.zip".
+
+double-check it in Finder: It will extract to a directory called "drop".
+
+Now from the terminal, run::
+
+    cd ~/questdb/py-questdb-client
+    cp -vr ~/Downloads/drop/* dist/
+
+
+Tagging the release
+-------------------
+
+In GitHub with a web browser create a new release with the tag "vX.Y.Z"
+(where X.Y.Z is the new version number).
+
+The release notes should be copied from the ``CHANGELOG.rst`` file,
+but reformatted as Markdown.
+
+
+Uploading to PyPI
+-----------------
+
+Now the MacOS VM has all the binaries and the source distribution, ready to be
+uploaded to PyPI.
+
+This is a good time to double-check you can log into PyPI and have set up an
+API token. If you don't have one (or lost it), you can create a new one here:
+https://pypi.org/manage/account/ (scroll down to "API tokens").
+
+Once you've triple-checked everything is in ``dist/``, you can upload to PyPI.
+
+.. code-block:: bash
+
+    python3 -m twine upload dist/*
+
+This will prompt you for your PyPI username and token.
+
+Once the upload is complete, you can check the PyPI page to see if the new
+release is there: https://pypi.org/project/questdb/
+
+
+Updating the docs
+-----------------
+
+Log into ReadTheDocs and trigger a new build for the project.
+
+https://readthedocs.org/dashboard/py-questdb-client/users/
+
+Watch it to ensure there are no errors.
+
+Once the build is complete, COMMAND-SHIFT-R to refresh the page (without cache)
+and check the new version is there.

{questdb-2.0.0 → questdb-2.0.2}/pyproject.toml

@@ -2,7 +2,7 @@
 # See: https://packaging.python.org/en/latest/specifications/declaring-project-metadata/
 name = "questdb"
 requires-python = ">=3.8"
-version = "2.0.0"
+version = "2.0.2"
 description = "QuestDB client library for Python"
 readme = "README.rst"
 classifiers = [
@@ -29,6 +29,7 @@ email = "adam@questdb.io"
 [project.optional-dependencies]
 publish = ["twine", "wheel"]
 ci = ["cibuildwheel"]
+dataframe = ["pandas", "pyarrow", "numpy"]
 
 [project.urls]
 Homepage = "https://questdb.io/"

{questdb-2.0.0 → questdb-2.0.2}/setup.py

@@ -165,7 +165,7 @@ def readme():
 
 setup(
     name='questdb',
-    version='2.0.0',
+    version='2.0.2',
     platforms=['any'],
     python_requires='>=3.8',
     install_requires=[],

questdb-2.0.2/src/questdb/__init__.py

@@ -0,0 +1 @@
+__version__ = '2.0.2'