sqlalchemy-upsert-kit 0.1.1__tar.gz → 0.2.1__tar.gz

{sqlalchemy_upsert_kit-0.1.1 → sqlalchemy_upsert_kit-0.2.1}/PKG-INFO

@@ -1,49 +1,57 @@
-Metadata-Version: 2.3
+Metadata-Version: 2.4
 Name: sqlalchemy_upsert_kit
-Version: 0.1.1
-Summary: High-performance SQLAlchemy bulk upsert operations using temporary table staging - 20x faster than row-by-row approaches.
-License: MIT
-Author: Sanhe Hu
-Author-email: husanhe@email.com
-Maintainer: Sanhe Hu
-Maintainer-email: husanhe@email.com
-Requires-Python: >=3.9,<4.0
-Classifier: License :: OSI Approved :: MIT License
+Version: 0.2.1
+Summary: Fast SQLAlchemy bulk upsert (insert or ignore, replace, or merge) for SQLite and PostgreSQL, using temporary table staging - typically 5-20x faster than row-by-row upserts.
+Author-email: Sanhe Hu <husanhe@gmail.com>
+Maintainer-email: Sanhe Hu <husanhe@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project
+Project-URL: Documentation, https://sqlalchemy-upsert-kit.readthedocs.io/en/latest/
+Project-URL: Repository, https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project
+Project-URL: Issues, https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project/issues
+Project-URL: Changelog, https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project/blob/main/release-history.rst
+Project-URL: Download, https://pypi.org/pypi/sqlalchemy-upsert-kit#files
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: English
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Unix
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
-Provides-Extra: auto
+Classifier: Programming Language :: Python :: 3.14
+Requires-Python: <4.0,>=3.11
+Description-Content-Type: text/x-rst
+License-File: LICENSE.txt
+License-File: AUTHORS.rst
+Requires-Dist: sqlalchemy<3.0.0,>=2.0.28
+Provides-Extra: postgres
+Requires-Dist: psycopg[binary]<4.0.0,>=3.3.8; extra == "postgres"
 Provides-Extra: dev
-Provides-Extra: doc
+Requires-Dist: rich<15.0.0,>=14.3.4; extra == "dev"
 Provides-Extra: test
-Requires-Dist: Sphinx (>=7.4.7,<8.0.0) ; extra == "doc"
-Requires-Dist: build (>=1.2.1,<2.0.0) ; extra == "dev"
-Requires-Dist: docfly (==3.0.0) ; extra == "doc"
-Requires-Dist: furo (==2024.8.6) ; extra == "doc"
-Requires-Dist: ipython (>=8.18.1,<8.19.0) ; extra == "doc"
-Requires-Dist: nbsphinx (>=0.8.12,<1.0.0) ; extra == "doc"
-Requires-Dist: prettytable (>=3.0.0,<4.0.0) ; extra == "test"
-Requires-Dist: pygments (>=2.18.0,<3.0.0) ; extra == "doc"
-Requires-Dist: pytest (>=8.2.2,<9.0.0) ; extra == "test"
-Requires-Dist: pytest-cov (>=6.0.0,<7.0.0) ; extra == "test"
-Requires-Dist: rich (>=13.8.1,<14.0.0) ; extra == "dev"
-Requires-Dist: rstobj (==1.2.1) ; extra == "doc"
-Requires-Dist: sphinx-copybutton (>=0.5.2,<1.0.0) ; extra == "doc"
-Requires-Dist: sphinx-design (>=0.6.1,<1.0.0) ; extra == "doc"
-Requires-Dist: sphinx-jinja (>=2.0.2,<3.0.0) ; extra == "doc"
-Requires-Dist: sqlalchemy (>=2.0.0,<3.0.0)
-Requires-Dist: twine (>=6.0.0,<7.0.0) ; extra == "dev"
-Requires-Dist: wheel (>=0.45.0,<1.0.0) ; extra == "dev"
-Project-URL: Changelog, https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project/blob/main/release-history.rst
-Project-URL: Documentation, https://sqlalchemy-upsert-kit.readthedocs.io/en/latest/
-Project-URL: Download, https://pypi.org/pypi/sqlalchemy-upsert-kit#files
-Project-URL: Homepage, https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project
-Project-URL: Issues, https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project/issues
-Project-URL: Repository, https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project
-Description-Content-Type: text/x-rst
+Requires-Dist: pytest<10.0.0,>=9.1.1; extra == "test"
+Requires-Dist: pytest-cov<8.0.0,>=7.1.0; extra == "test"
+Requires-Dist: prettytable<4.0.0,>=3.0.0; extra == "test"
+Requires-Dist: psycopg[binary]<4.0.0,>=3.3.8; extra == "test"
+Provides-Extra: doc
+Requires-Dist: Sphinx<9.0.0,>=8.2.3; extra == "doc"
+Requires-Dist: sphinx-copybutton<1.0.0,>=0.5.2; extra == "doc"
+Requires-Dist: sphinx-design<1.0.0,>=0.7.0; extra == "doc"
+Requires-Dist: sphinx-jinja<3.0.0,>=2.0.2; extra == "doc"
+Requires-Dist: furo==2025.12.19; extra == "doc"
+Requires-Dist: pygments<3.0.0,>=2.18.0; extra == "doc"
+Requires-Dist: ipython<8.19.0,>=8.18.1; extra == "doc"
+Requires-Dist: ipykernel<8.0.0,>=7.3.0; extra == "doc"
+Requires-Dist: nbsphinx<1.0.0,>=0.9.8; extra == "doc"
+Requires-Dist: rstobj<3.0.0,>=2.0.0; extra == "doc"
+Requires-Dist: docfly<4.0.0,>=3.0.3; extra == "doc"
+Provides-Extra: mise
+Requires-Dist: PyGithub<3.0.0,>=2.8.0; extra == "mise"
+Requires-Dist: httpx2<3.0.0,>=2.13.1; extra == "mise"
+Dynamic: license-file
 
 
 .. image:: https://readthedocs.org/projects/sqlalchemy-upsert-kit/badge/?version=latest
@@ -97,7 +105,13 @@ Welcome to ``sqlalchemy_upsert_kit`` Documentation
 .. image:: https://sqlalchemy-upsert-kit.readthedocs.io/en/latest/_static/sqlalchemy_upsert_kit-logo.png
     :target: https://sqlalchemy-upsert-kit.readthedocs.io/en/latest/
 
-``sqlalchemy_upsert_kit`` provides high-performance bulk upsert operations for SQLAlchemy applications using temporary table staging. It offers three optimized strategies: ``insert_or_ignore`` for conflict-free insertion, ``insert_or_replace`` for complete record replacement, and ``insert_or_merge`` for selective column updates while preserving existing data. Each strategy leverages efficient SQL JOIN operations and temporary tables to achieve ~20x performance improvements over traditional row-by-row upserts, making it ideal for ETL processes, data synchronization, and bulk data operations where speed and reliability are critical.
+``sqlalchemy_upsert_kit`` provides fast bulk upsert operations for SQLAlchemy applications, on SQLite and PostgreSQL. It offers three strategies:
+
+- ``insert_or_ignore``: insert new records, and leave existing rows alone.
+- ``insert_or_replace``: insert new records, and overwrite existing rows with the records.
+- ``insert_or_merge``: insert new records, and update only chosen columns of existing rows.
+
+Each call loads the records into a temporary staging table and writes them with one native ``INSERT ... ON CONFLICT`` statement, typically 5-20x faster than row-by-row upserts on large batches, which suits ETL, data synchronization, and other bulk loads. Every call is atomic, and can run in its own transaction or inside yours.
 
 
 .. _install:
@@ -117,3 +131,16 @@ To upgrade to latest version:
 
     $ pip install --upgrade sqlalchemy-upsert-kit
 
+
+.. _use-with-ai-coding-agents:
+
+Use with AI Coding Agents
+------------------------------------------------------------------------------
+
+This repo ships an `Agent Skill <https://code.claude.com/docs/en/skills>`_ that documents the public API of ``sqlalchemy_upsert_kit``, so an AI coding agent such as Claude Code can use the library without reading its source. It lives in `.claude/skills/pypi-sqlalchemy_upsert_kit <https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project/tree/main/.claude/skills/pypi-sqlalchemy_upsert_kit>`_. To use it in your project, copy it into your project's ``.claude/skills/`` folder. Run this from your project's root:
+
+.. code-block:: console
+
+    $ git clone --depth 1 https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project.git /tmp/sqlalchemy_upsert_kit-project
+    $ mkdir -p .claude/skills
+    $ cp -R /tmp/sqlalchemy_upsert_kit-project/.claude/skills/pypi-sqlalchemy_upsert_kit .claude/skills/

{sqlalchemy_upsert_kit-0.1.1 → sqlalchemy_upsert_kit-0.2.1}/README.rst

@@ -50,7 +50,13 @@ Welcome to ``sqlalchemy_upsert_kit`` Documentation
 .. image:: https://sqlalchemy-upsert-kit.readthedocs.io/en/latest/_static/sqlalchemy_upsert_kit-logo.png
     :target: https://sqlalchemy-upsert-kit.readthedocs.io/en/latest/
 
-``sqlalchemy_upsert_kit`` provides high-performance bulk upsert operations for SQLAlchemy applications using temporary table staging. It offers three optimized strategies: ``insert_or_ignore`` for conflict-free insertion, ``insert_or_replace`` for complete record replacement, and ``insert_or_merge`` for selective column updates while preserving existing data. Each strategy leverages efficient SQL JOIN operations and temporary tables to achieve ~20x performance improvements over traditional row-by-row upserts, making it ideal for ETL processes, data synchronization, and bulk data operations where speed and reliability are critical.
+``sqlalchemy_upsert_kit`` provides fast bulk upsert operations for SQLAlchemy applications, on SQLite and PostgreSQL. It offers three strategies:
+
+- ``insert_or_ignore``: insert new records, and leave existing rows alone.
+- ``insert_or_replace``: insert new records, and overwrite existing rows with the records.
+- ``insert_or_merge``: insert new records, and update only chosen columns of existing rows.
+
+Each call loads the records into a temporary staging table and writes them with one native ``INSERT ... ON CONFLICT`` statement, typically 5-20x faster than row-by-row upserts on large batches, which suits ETL, data synchronization, and other bulk loads. Every call is atomic, and can run in its own transaction or inside yours.
 
 
 .. _install:
@@ -69,3 +75,17 @@ To upgrade to latest version:
 .. code-block:: console
 
     $ pip install --upgrade sqlalchemy-upsert-kit
+
+
+.. _use-with-ai-coding-agents:
+
+Use with AI Coding Agents
+------------------------------------------------------------------------------
+
+This repo ships an `Agent Skill <https://code.claude.com/docs/en/skills>`_ that documents the public API of ``sqlalchemy_upsert_kit``, so an AI coding agent such as Claude Code can use the library without reading its source. It lives in `.claude/skills/pypi-sqlalchemy_upsert_kit <https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project/tree/main/.claude/skills/pypi-sqlalchemy_upsert_kit>`_. To use it in your project, copy it into your project's ``.claude/skills/`` folder. Run this from your project's root:
+
+.. code-block:: console
+
+    $ git clone --depth 1 https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project.git /tmp/sqlalchemy_upsert_kit-project
+    $ mkdir -p .claude/skills
+    $ cp -R /tmp/sqlalchemy_upsert_kit-project/.claude/skills/pypi-sqlalchemy_upsert_kit .claude/skills/

sqlalchemy_upsert_kit-0.2.1/pyproject.toml

@@ -0,0 +1,132 @@
+# [project] table: https://packaging.python.org/en/latest/guides/writing-pyproject-toml/
+[project]
+name = "sqlalchemy_upsert_kit"
+# Bump before each release (https://semver.org/), then ``mise run inst`` to update ``uv.lock``.
+version = "0.2.1"
+description = "Fast SQLAlchemy bulk upsert (insert or ignore, replace, or merge) for SQLite and PostgreSQL, using temporary table staging - typically 5-20x faster than row-by-row upserts."
+# An SPDX license expression (PEP 639); keep ``LICENSE.txt`` in sync.
+# Read: https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license-and-license-files
+# Valid identifiers: https://spdx.org/licenses/
+license = "MIT"
+license-files = ["LICENSE.txt", "AUTHORS.rst"]
+authors = [
+    { name = "Sanhe Hu", email = "husanhe@gmail.com" },
+]
+maintainers = [
+    { name = "Sanhe Hu", email = "husanhe@gmail.com" },
+]
+keywords = []
+readme = "README.rst"
+requires-python = ">=3.11,<4.0"
+# Read: https://pypi.org/classifiers/
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Natural Language :: English",
+    "Operating System :: Microsoft :: Windows",
+    "Operating System :: MacOS",
+    "Operating System :: Unix",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+]
+
+# --- Dependencies ---
+# Edit these lists, then ``mise run inst`` to update ``uv.lock`` and ``.venv``.
+dependencies = [
+    "sqlalchemy>=2.0.28,<3.0.0", # database toolkit and ORM; 2.0.28 adds preserve_rowcount
+]
+
+# --- Optional dependencies (extras) ---
+# Read: https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-optional-dependencies
+# Each one must support every Python version in ``requires-python``.
+[project.optional-dependencies]
+
+# PostgreSQL driver for ``api.postgres``
+postgres = [
+    "psycopg[binary]>=3.3.8,<4.0.0", # PostgreSQL driver
+]
+
+# local development
+dev = [
+    "rich>=14.3.4,<15.0.0",
+]
+
+# tests and coverage
+test = [
+    "pytest>=9.1.1,<10.0.0",
+    "pytest-cov>=7.1.0,<8.0.0",
+    "prettytable>=3.0.0,<4.0.0", # pretty print tables in test output
+    "psycopg[binary]>=3.3.8,<4.0.0", # PostgreSQL driver for the postgres tests
+]
+
+# docs build
+doc = [
+    "Sphinx>=8.2.3,<9.0.0", # held below 9 until its autodoc rewrite is verified
+    "sphinx-copybutton>=0.5.2,<1.0.0", # copy button on code blocks
+    "sphinx-design>=0.7.0,<1.0.0", # cards, tabs, grids
+    "sphinx-jinja>=2.0.2,<3.0.0", # jinja in reST
+    "furo==2025.12.19", # theme
+    "pygments>=2.18.0,<3.0.0", # syntax highlighting
+    "ipython>=8.18.1,<8.19.0", # notebook execution
+    "ipykernel>=7.3.0,<8.0.0", # notebook kernel
+    "nbsphinx>=0.9.8,<1.0.0", # notebooks as doc pages
+    "rstobj>=2.0.0,<3.0.0", # reST from Python
+    "docfly>=3.0.3,<4.0.0", # toctree and API reference
+]
+
+# mise tasks
+mise = [
+    "PyGithub>=2.8.0,<3.0.0", # GitHub API client
+    "httpx2>=2.13.1,<3.0.0", # Codecov and Read the Docs APIs
+]
+
+[project.urls]
+Homepage = "https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project"
+Documentation = "https://sqlalchemy-upsert-kit.readthedocs.io/en/latest/"
+Repository = "https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project"
+Issues = "https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project/issues"
+Changelog = "https://github.com/MacHu-GWU/sqlalchemy_upsert_kit-project/blob/main/release-history.rst"
+Download = "https://pypi.org/pypi/sqlalchemy-upsert-kit#files"
+
+# Read: https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#creating-executable-scripts
+[project.scripts]
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = [
+    "sqlalchemy_upsert_kit*",
+]
+exclude = [
+    "sqlalchemy_upsert_kit.tests",
+    "sqlalchemy_upsert_kit.tests.*",
+    "sqlalchemy_upsert_kit.docs",
+    "sqlalchemy_upsert_kit.docs.*",
+]
+
+# --- ruff: ``mise run format`` / ``mise run lint`` ---
+[tool.ruff]
+extend-exclude = [
+    "sqlalchemy_upsert_kit/vendor",
+    ".claude/skills/*/assets", # templates, not valid Python
+]
+
+[tool.ruff.lint.per-file-ignores]
+# re-export modules
+"**/__init__.py" = ["F401"]
+"**/api.py" = ["F401"]
+
+# --- pytest 9+ native table ---
+# Read: https://docs.pytest.org/en/stable/reference/customize.html#pyproject-toml
+[tool.pytest]
+testpaths = ["tests"]
+# strict = true # only with a pinned pytest
+# filterwarnings = ["error"]
+# markers = ["slow: tests that take more than a few seconds"]
+
+# --- Build (``mise run build``); setuptools 77+ for PEP 639 ``license`` ---
+[build-system]
+requires = ["setuptools>=77.0"]
+build-backend = "setuptools.build_meta"

sqlalchemy_upsert_kit-0.2.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

sqlalchemy_upsert_kit-0.2.1/sqlalchemy_upsert_kit/__init__.py

@@ -0,0 +1 @@
+# -*- coding: utf-8 -*-

{sqlalchemy_upsert_kit-0.1.1 → sqlalchemy_upsert_kit-0.2.1}/sqlalchemy_upsert_kit/api.py

@@ -1,3 +1,4 @@
 # -*- coding: utf-8 -*-
 
 from .sqlite import api as sqlite
+from .postgres import api as postgres

sqlalchemy_upsert_kit-0.2.1/sqlalchemy_upsert_kit/core/__init__.py

@@ -0,0 +1 @@
+# -*- coding: utf-8 -*-

sqlalchemy_upsert_kit-0.2.1/sqlalchemy_upsert_kit/core/executor.py

@@ -0,0 +1,207 @@
+# -*- coding: utf-8 -*-
+
+"""
+The dialect-agnostic upsert workflow, shared by every backend.
+
+:class:`UpsertExecutor` runs one upsert as these steps, all inside a SAVEPOINT
+on one connection::
+
+    begin_transaction()            # backend hook: make sure a real transaction is open
+    SAVEPOINT
+      create the staging table     # TEMPORARY, private to the connection
+      load the records into it
+      count the conflicting rows
+      INSERT ... SELECT ... ON CONFLICT   # backend hook: build_upsert()
+      drop the staging table
+    RELEASE SAVEPOINT              # on error: ROLLBACK TO SAVEPOINT, then re-raise
+
+Rolling back to the savepoint undoes everything the upsert did, the staging
+table included, and leaves the caller's transaction usable. In auto-managed
+mode the executor opens and commits its own connection and transaction; in
+user-managed mode it runs on the caller's connection and never commits.
+
+A backend subclasses :class:`UpsertExecutor` and implements :meth:`build_upsert`,
+plus :meth:`begin_transaction` if its driver delays ``BEGIN``.
+"""
+
+import typing as T
+import abc
+import dataclasses
+
+import sqlalchemy as sa
+
+from ..exc import UpsertTestError
+
+from .schema import get_pk_column
+from .schema import make_staging_name
+from .schema import make_staging_table
+from .validation import validate_records
+from .validation import resolve_update_keys
+
+#: The steps where the private ``_fail_at`` test hook can inject a failure.
+FAIL_STEPS = ("create_staging", "load_staging", "upsert", "drop_staging")
+
+
+@dataclasses.dataclass
+class UpsertExecutor(abc.ABC):
+    """
+    One bulk upsert of ``values`` into ``table`` with one strategy.
+
+    All inputs are validated when the executor is created, before any SQL runs.
+    :meth:`run` returns ``(conflicting_rows, inserted_rows)``: the number of
+    records whose primary key already existed (ignored, replaced or updated,
+    depending on the strategy), and the number of new rows.
+
+    :param engine: Engine used in auto-managed mode
+    :param table: Target table, with exactly one primary key column
+    :param values: Records keyed by column key, all with the same keys
+    :param strategy: ``"ignore"``, ``"replace"`` or ``"merge"``
+    :param columns: Column keys that ``"merge"`` updates; unused otherwise
+    :param metadata: Metadata that holds the staging table definition while
+        the upsert runs. A new one when None.
+    :param temp_table_name: Staging table name. Generated when None.
+    :param conn: Caller's connection, for user-managed mode, with ``trans``
+    :param trans: Caller's transaction, for user-managed mode, with ``conn``
+    :param fail_at: **Testing only**: raise :class:`UpsertTestError` just
+        before this step, one of :data:`FAIL_STEPS`.
+    """
+
+    # fmt: off
+    engine: sa.Engine
+    table: sa.Table
+    values: list[dict[str, T.Any]]
+    strategy: str
+    columns: list[str] | None = None
+    metadata: sa.MetaData | None = None
+    temp_table_name: str | None = None
+    conn: sa.Connection | None = None
+    trans: sa.Transaction | None = None
+    fail_at: str | None = None
+    # set by __post_init__
+    pk_column: sa.Column = dataclasses.field(init=False)
+    update_keys: list[str] = dataclasses.field(init=False)
+    # fmt: on
+
+    def __post_init__(self):
+        if (self.conn is None) != (self.trans is None):
+            raise ValueError(
+                "Give both conn and trans (user-managed mode) or neither "
+                "(auto-managed mode)"
+            )
+        if self.fail_at is not None and self.fail_at not in FAIL_STEPS:
+            raise ValueError(
+                f"fail_at must be one of {FAIL_STEPS}, got {self.fail_at!r}"
+            )
+        self.pk_column = get_pk_column(self.table)
+        if self.values:
+            validate_records(self.table, self.values)
+        record_keys = self.values[0].keys() if self.values else []
+        self.update_keys = resolve_update_keys(
+            table=self.table,
+            strategy=self.strategy,
+            record_keys=record_keys,
+            columns=self.columns,
+        )
+        if self.metadata is None:
+            self.metadata = sa.MetaData()
+        if self.temp_table_name is None:
+            self.temp_table_name = make_staging_name(
+                table=self.table,
+                max_length=self.engine.dialect.max_identifier_length,
+            )
+        if self.temp_table_name == self.table.name:
+            raise ValueError("temp_table_name must differ from the target table name")
+
+    @property
+    def user_managed(self) -> bool:
+        """
+        True when the upsert runs in the caller's transaction.
+        """
+        return self.conn is not None
+
+    def begin_transaction(self, conn: sa.Connection) -> None:
+        """
+        Make sure ``conn`` is inside a real database transaction before the
+        SAVEPOINT. The default does nothing: most drivers begin one on
+        ``Connection.begin()``.
+        """
+
+    @abc.abstractmethod
+    def build_upsert(self, staging: sa.Table) -> sa.Insert:
+        """
+        Build the one statement that copies every staged row into the target
+        table, overwriting :attr:`update_keys` on conflicting rows (or nothing,
+        when it is empty).
+        """
+        raise NotImplementedError
+
+    def maybe_fail(self, step: str) -> None:
+        """
+        Raise the test failure if ``fail_at`` names this step.
+        """
+        if self.fail_at == step:
+            raise UpsertTestError(f"injected failure before step {step!r}")
+
+    def count_conflicts(self, conn: sa.Connection, staging: sa.Table) -> int:
+        """
+        Count the staged rows whose primary key already exists in the target.
+        """
+        pk_key = self.pk_column.key
+        stmt = (
+            sa.select(sa.func.count())
+            .select_from(staging)
+            .join(self.table, staging.c[pk_key] == self.table.c[pk_key])
+        )
+        return conn.execute(stmt).scalar_one()
+
+    def run_on(self, conn: sa.Connection) -> tuple[int, int]:
+        """
+        Run the upsert on ``conn``, inside a SAVEPOINT of its open transaction.
+        """
+        self.begin_transaction(conn)
+        staging = make_staging_table(
+            table=self.table,
+            metadata=self.metadata,
+            name=self.temp_table_name,
+        )
+        try:
+            with conn.begin_nested():
+                self.maybe_fail("create_staging")
+                staging.create(conn)
+                self.maybe_fail("load_staging")
+                conn.execute(staging.insert(), self.values)
+                if self.update_keys:
+                    # DO UPDATE's rowcount counts inserted and updated rows
+                    # together, so count the conflicts first. If another
+                    # transaction writes the same keys in between, the split
+                    # can be off; the sum is always len(values).
+                    conflicts = self.count_conflicts(conn, staging)
+                self.maybe_fail("upsert")
+                stmt = self.build_upsert(staging)
+                # SQLAlchemy keeps the rowcount of an INSERT only when asked to.
+                res = conn.execute(stmt.execution_options(preserve_rowcount=True))
+                if self.update_keys:
+                    inserted = res.rowcount - conflicts
+                else:
+                    # DO NOTHING's rowcount is exactly the inserted rows.
+                    inserted = res.rowcount
+                    conflicts = len(self.values) - inserted
+                self.maybe_fail("drop_staging")
+                staging.drop(conn)
+        finally:
+            self.metadata.remove(staging)
+        return conflicts, inserted
+
+    def run(self) -> tuple[int, int]:
+        """
+        Run the upsert and return ``(conflicting_rows, inserted_rows)``.
+
+        Empty ``values`` is a no-op that returns ``(0, 0)``.
+        """
+        if not self.values:
+            return 0, 0
+        if self.user_managed:
+            return self.run_on(self.conn)
+        with self.engine.connect() as conn:
+            with conn.begin():
+                return self.run_on(conn)

sqlalchemy_upsert_kit-0.2.1/sqlalchemy_upsert_kit/core/schema.py

@@ -0,0 +1,78 @@
+# -*- coding: utf-8 -*-
+
+"""
+Table introspection and the staging table definition, shared by every backend.
+"""
+
+import uuid
+
+import sqlalchemy as sa
+
+
+def get_pk_column(table: sa.Table) -> sa.Column:
+    """
+    Return the single primary key column of ``table``.
+
+    :raises ValueError: If the table has zero or several primary key columns.
+    """
+    pks = list(table.primary_key.columns)
+    if len(pks) != 1:
+        raise ValueError(
+            f"Table {table.name!r} must have exactly one primary key column, "
+            f"found {[col.name for col in pks]}"
+        )
+    return pks[0]
+
+
+def get_writable_columns(table: sa.Table) -> list[sa.Column]:
+    """
+    Return the columns an upsert may write: every column except generated
+    (``Computed``) ones.
+    """
+    return [col for col in table.columns if col.computed is None]
+
+
+def make_staging_name(
+    table: sa.Table,
+    max_length: int,
+) -> str:
+    """
+    Generate a staging table name: ``temp_<8 hex>_<table name>``, cut to
+    ``max_length`` (the dialect's identifier limit, 63 on PostgreSQL).
+
+    The random part keeps names unique across retries and concurrent calls.
+    """
+    return f"temp_{uuid.uuid4().hex[:8]}_{table.name}"[:max_length]
+
+
+def make_staging_table(
+    table: sa.Table,
+    metadata: sa.MetaData,
+    name: str,
+) -> sa.Table:
+    """
+    Define the ``TEMPORARY`` staging table for ``table``.
+
+    It copies each writable column's name, key, type and defaults, and marks the
+    primary key, but nothing else: no indexes, unique constraints, foreign keys,
+    sequences or identities, which could collide with the target's names or
+    reference tables a temporary table may not reference. A temporary table is
+    private to its connection, so concurrent upserts never see each other's.
+
+    :param table: The target table
+    :param metadata: Metadata that holds the staging table definition
+    :param name: Name of the staging table
+    """
+    columns = list()
+    for col in get_writable_columns(table):
+        kwargs = dict(
+            key=col.key,
+            primary_key=col.primary_key,
+            autoincrement=False,  # the key always comes from the input records
+        )
+        if isinstance(col.server_default, sa.DefaultClause):
+            kwargs["server_default"] = sa.DefaultClause(col.server_default.arg)
+        if col.default is not None and not col.default.is_sequence:
+            kwargs["default"] = col.default.arg
+        columns.append(sa.Column(col.name, col.type, **kwargs))
+    return sa.Table(name, metadata, *columns, prefixes=["TEMPORARY"])