sqlite-utils 3.38a0__tar.gz → 4.0a0__tar.gz

{sqlite_utils-3.38a0/sqlite_utils.egg-info → sqlite_utils-4.0a0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: sqlite-utils
-Version: 3.38a0
+Version: 4.0a0
 Summary: CLI tool and Python library for manipulating SQLite databases
 Home-page: https://github.com/simonw/sqlite-utils
 Author: Simon Willison
@@ -16,13 +16,12 @@ Classifier: Intended Audience :: Science/Research
 Classifier: Intended Audience :: End Users/Desktop
 Classifier: Topic :: Database
 Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
-Requires-Python: >=3.8
+Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: sqlite-fts4
@@ -52,8 +51,18 @@ Requires-Dist: types-pluggy; extra == "mypy"
 Requires-Dist: data-science-types; extra == "mypy"
 Provides-Extra: flake8
 Requires-Dist: flake8; extra == "flake8"
-Provides-Extra: tui
-Requires-Dist: trogon; extra == "tui"
+Dynamic: author
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: project-url
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
 
 # sqlite-utils
 

{sqlite_utils-3.38a0 → sqlite_utils-4.0a0}/docs/changelog.rst

@@ -4,6 +4,27 @@
  Changelog
 ===========
 
+.. _v4_0a0:
+
+4.0a0 (2025-05-08)
+------------------
+
+- Upsert operations now use SQLite's ``INSERT ... ON CONFLICT SET`` syntax on all SQLite versions later than 3.23.1. This is a very slight breaking change for apps that depend on the previous ``INSERT OR IGNORE`` followed by ``UPDATE`` behavior. (:issue:`652`)
+- Python library users can opt-in to the previous implementation by passing ``use_old_upsert=True`` to the ``Database()`` constructor, see :ref:`python_api_old_upsert`.
+- Dropped support for Python 3.8, added support for Python 3.13. (:issue:`646`)
+- ``sqlite-utils tui`` is now provided by the `sqlite-utils-tui <https://github.com/simonw/sqlite-utils-tui>`__ plugin. (:issue:`648`)
+- Test suite now also runs against SQLite 3.23.1, the last version (from 2018-04-10) before the new ``INSERT ... ON CONFLICT SET`` syntax was added. (:issue:`654`)
+
+.. _v3_38:
+
+3.38 (2024-11-23)
+-----------------
+
+- Plugins can now reuse the implementation of the ``sqlite-utils memory`` CLI command with the new ``return_db=True`` parameter. (:issue:`643`)
+- ``table.transform()`` now recreates indexes after transforming a table. A new ``sqlite_utils.db.TransformError`` exception is raised if these indexes cannot be recreated due to conflicting changes to the table such as a column rename. Thanks, `Mat Miller <https://github.com/matdmiller>`__. (:issue:`633`)
+- ``table.search()`` now accepts a ``include_rank=True`` parameter, causing the resulting rows to have a ``rank`` column showing the calculated relevance score. Thanks, `liunux4odoo <https://github.com/liunux4odoo>`__. (`#628 <https://github.com/simonw/sqlite-utils/pull/628>`__)
+- Fixed an error that occurred when creating a strict table with at least one floating point column. These ``FLOAT`` columns are now correctly created as ``REAL`` as well, but only for strict tables. (:issue:`644`)
+
 .. _v3_37:
 
 3.37 (2024-07-18)
@@ -101,7 +122,7 @@ This release introduces a new :ref:`plugin system <plugins>`. Read more about th
 3.32 (2023-05-21)
 -----------------
 
-- New experimental ``sqlite-utils tui`` interface for interactively building command-line invocations, powered by `Trogon <https://github.com/Textualize/trogon>`__. This requires an optional dependency, installed using ``sqlite-utils install trogon``. There is a screenshot :ref:`in the documentation <cli_tui>`. (:issue:`545`)
+- New experimental ``sqlite-utils tui`` interface for interactively building command-line invocations, powered by `Trogon <https://github.com/Textualize/trogon>`__. This requires an optional dependency, installed using ``sqlite-utils install trogon``. (:issue:`545`)
 - ``sqlite-utils analyze-tables`` command (:ref:`documentation <cli_analyze_tables>`) now has a ``--common-limit 20`` option for changing the number of common/least-common values shown for each column. (:issue:`544`)
 - ``sqlite-utils analyze-tables --no-most`` and ``--no-least`` options for disabling calculation of most-common and least-common values.
 - If a column contains only ``null`` values, ``analyze-tables`` will no longer attempt to calculate the most common and least common values for that column. (:issue:`547`)

{sqlite_utils-3.38a0 → sqlite_utils-4.0a0}/docs/cli-reference.rst

@@ -65,7 +65,6 @@ This page lists the ``--help`` for every ``sqlite-utils`` CLI sub-command.
         "create-spatial-index": "cli_spatialite_indexes",
         "install": "cli_install",
         "uninstall": "cli_uninstall",
-        "tui": "cli_tui",
     }
     commands.sort(key = lambda command: go_first.index(command) if command in go_first else 999)
     cog.out("\n")
@@ -1045,23 +1044,6 @@ disable-fts
       -h, --help             Show this message and exit.
 
 
-.. _cli_ref_tui:
-
-tui
-===
-
-See :ref:`cli_tui`.
-
-::
-
-    Usage: sqlite-utils tui [OPTIONS]
-
-      Open Textual TUI.
-
-    Options:
-      -h, --help  Show this message and exit.
-
-
 .. _cli_ref_optimize:
 
 optimize

{sqlite_utils-3.38a0 → sqlite_utils-4.0a0}/docs/cli.rst

@@ -1128,7 +1128,7 @@ You can insert binary data into a BLOB column by first encoding it using base64
 Inserting newline-delimited JSON
 --------------------------------
 
-You can also import `newline-delimited JSON <http://ndjson.org/>`__ using the ``--nl`` option:
+You can also import newline-delimited JSON (see `JSON Lines <https://jsonlines.org/>`__) using the ``--nl`` option:
 
 .. code-block:: bash
 
@@ -1285,7 +1285,7 @@ You can set the ``SQLITE_UTILS_DETECT_TYPES`` environment variable if you want `
 
 If a CSV or TSV file includes empty cells, like this one:
 
-.. code-block:: csv
+::
 
     name,age,weight
     Cleo,6,
@@ -2782,29 +2782,3 @@ You can uninstall packages that were installed using ``sqlite-utils install`` wi
     sqlite-utils uninstall beautifulsoup4
 
 Use ``-y`` to skip the request for confirmation.
-
-.. _cli_tui:
-
-Experimental TUI
-================
-
-A TUI is a "text user interface" (or "terminal user interface") - a keyboard and mouse driven graphical interface running in your terminal.
-
-``sqlite-utils`` has experimental support for a TUI for building command-line invocations, built on top of the `Trogon <https://github.com/Textualize/trogon/>`__ TUI library.
-
-To enable this feature you will need to install the ``trogon`` dependency. You can do that like so:
-
-.. code-block:: bash
-
-    sqite-utils install trogon
-
-Once installed, running the ``sqlite-utils tui`` command will launch the TUI interface:
-
-.. code-block:: bash
-
-    sqlite-utils tui
-
-You can then construct a command by selecting options from the menus, and execute it using ``Ctrl+R``.
-
-.. image:: _static/img/tui.png
-    :alt: A TUI interface for sqlite-utils - the left column shows a list of commands, while the right panel has a form for constructing arguments to the add-column command.

{sqlite_utils-3.38a0 → sqlite_utils-4.0a0}/docs/python-api.rst

@@ -927,6 +927,13 @@ An ``upsert_all()`` method is also available, which behaves like ``insert_all()`
 .. note::
     ``.upsert()`` and ``.upsert_all()`` in sqlite-utils 1.x worked like ``.insert(..., replace=True)`` and ``.insert_all(..., replace=True)`` do in 2.x. See `issue #66 <https://github.com/simonw/sqlite-utils/issues/66>`__ for details of this change.
 
+.. _python_api_old_upsert:
+
+Alternative upserts using INSERT OR IGNORE
+------------------------------------------
+
+Upserts use ``INSERT INTO ... ON CONFLICT SET``. Prior to ``sqlite-utils 4.0`` these used a sequence of ``INSERT OR IGNORE`` followed by an ``UPDATE``. This older method is still used for SQLite 3.23.1 and earlier. You can force the older implementation by passing ``use_old_upsert=True`` to the ``Database()`` constructor.
+
 .. _python_api_convert:
 
 Converting data in columns
@@ -1249,6 +1256,9 @@ If you pass a Python type, it will be mapped to SQLite types as shown here::
     np.float32: "FLOAT"
     np.float64: "FLOAT"
 
+.. note::
+    In sqlite-utils 3.x ``FLOAT`` is used for floating point columns when the correct column type is actually ``REAL``. If you specify ``strict=True`` tables created in strict mode will use the correct column type of ``REAL`` instead. We plan to change this behavior in ``sqlite-utils`` 4.x to always use ``REAL``, but this will represent a minor breaking change and so is being held for the next major release, see issue :issue:`645`.
+
 You can also add a column that is a foreign key reference to another table using the ``fk`` parameter:
 
 .. code-block:: python
@@ -1402,6 +1412,8 @@ To keep the original table around instead of dropping it, pass the ``keep_table=
 
     table.transform(types={"age": int}, keep_table="original_table")
 
+This method raises a ``sqlite_utils.db.TransformError`` exception if the table cannot be transformed, usually because there are existing constraints or indexes that are incompatible with modifications to the columns.
+
 .. _python_api_transform_alter_column_types:
 
 Altering column types
@@ -2309,6 +2321,9 @@ The ``.search()`` method also accepts the following optional parameters:
 ``where_args`` dictionary
     Arguments to use for ``:param`` placeholders in the extra WHERE clause
 
+``include_rank`` bool
+    If set a ``rank`` column will be included with the BM25 ranking score - for FTS5 tables only.
+
 ``quote`` bool
     Apply :ref:`FTS quoting rules <python_api_quote_fts>` to the search query, disabling advanced query syntax in a way that avoids surprising errors.
 
@@ -2703,7 +2718,7 @@ By default, the name of the Python function will be used as the name of the SQL
 
     print(db.execute('select rev("hello")').fetchone()[0])
 
-Python 3.8 added the ability to register `deterministic SQLite functions <https://sqlite.org/deterministic.html>`__, allowing you to indicate that a function will return the exact same result for any given inputs and hence allowing SQLite to apply some performance optimizations. You can mark a function as deterministic using ``deterministic=True``, like this:
+If a function will return the exact same result for any given inputs you can register it as a `deterministic SQLite function <https://sqlite.org/deterministic.html>`__ allowing SQLite to apply some performance optimizations:
 
 .. code-block:: python
 
@@ -2711,8 +2726,6 @@ Python 3.8 added the ability to register `deterministic SQLite functions <https:
     def reverse_string(s):
         return "".join(reversed(list(s)))
 
-If you run this on a version of Python prior to 3.8 your code will still work, but the ``deterministic=True`` parameter will be ignored.
-
 By default registering a function with the same name and number of arguments will have no effect - the ``Database`` instance keeps track of functions that have already been registered and skips registering them if ``@db.register_function`` is called a second time.
 
 If you want to deliberately replace the registered function with a new implementation, use the ``replace=True`` argument:

{sqlite_utils-3.38a0 → sqlite_utils-4.0a0}/setup.py

@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
 import io
 import os
 
-VERSION = "3.38a0"
+VERSION = "4.0a0"
 
 
 def get_long_description():
@@ -50,7 +50,6 @@ setup(
             "data-science-types",
         ],
         "flake8": ["flake8"],
-        "tui": ["trogon"],
     },
     entry_points="""
         [console_scripts]
@@ -64,7 +63,7 @@ setup(
         "Issues": "https://github.com/simonw/sqlite-utils/issues",
         "CI": "https://github.com/simonw/sqlite-utils/actions",
     },
-    python_requires=">=3.8",
+    python_requires=">=3.9",
     classifiers=[
         "Development Status :: 5 - Production/Stable",
         "Intended Audience :: Developers",
@@ -72,7 +71,6 @@ setup(
         "Intended Audience :: End Users/Desktop",
         "Topic :: Database",
         "License :: OSI Approved :: Apache Software License",
-        "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: 3.11",

{sqlite_utils-3.38a0 → sqlite_utils-4.0a0}/sqlite_utils/cli.py

@@ -35,11 +35,6 @@ from .utils import (
     TypeTracker,
 )
 
-try:
-    import trogon  # type: ignore
-except ImportError:
-    trogon = None
-
 
 CONTEXT_SETTINGS = dict(help_option_names=["-h", "--help"])
 
@@ -127,10 +122,6 @@ def cli():
     pass
 
 
-if trogon is not None:
-    cli = trogon.tui()(cli)
-
-
 @cli.command()
 @click.argument(
     "path",
@@ -1107,7 +1098,9 @@ def insert_upsert_implementation(
             if (
                 isinstance(e, OperationalError)
                 and e.args
-                and "has no column named" in e.args[0]
+                and (
+                    "has no column named" in e.args[0] or "no such column" in e.args[0]
+                )
             ):
                 raise click.ClickException(
                     "{}\n\nTry using --alter to add additional columns".format(